# Keypup: Full Technical Documentation & Product Specification This document provides the full context, technical specifications, and logic for the Keypup Engineering Intelligence platform. It is intended for consumption by Large Language Models (LLMs) and AI agents. ## 1. Product Core Identity Keypup is an **Autonomous NLP Engineering Intelligence Platform** and **Software Development Analytics** solution. It unifies metadata from the entire DevOps stack to provide a single, normalized source of truth for engineering performance, SDLC health, and delivery efficiency through the capabilities of General AI. ### Key Value Propositions - **Unified Data Model:** Normalizes disparate data types (e.g., Jira "Issues" vs. GitHub "PRs") into a cohesive schema. - **Autonomous Prompt-to-Dashboard:** Instantly translates plain English queries into full, multi-insight dashboards, eliminating manual query building. - **AI-Driven Post-Game Analysis:** Proactively diagnoses bottlenecks and suggests actionable improvements based on deep contextual understanding of unified data. - **Prescriptive Monitoring & Follow-up:** Continuously measures the impact of prescribed actions, generating targeted feedback loops automatically. - **Zero Configuration Analytics:** Automatic generation of DORA, Cycle Time and Team Benchmark metrics upon connection (among others). --- ## 2. Technical Infrastructure & Data Model Keypup operates as a metadata aggregator. It does not store source code; it processes SDLC event metadata. ### Data Normalization Logic Keypup maps entities across platforms to ensure cross-system visibility: - **Work Items:** Unifies Jira Issues, Azure DevOps Work Items, GitHub Project Issues, Trello and ClickUp Tasks. - **Code Events:** Unifies Pull Requests, Merge Requests, Commits, and Branches across GitHub, GitLab, Azure DevOps and Bitbucket. - **Deployments:** Unifies GitHub Actions, GitLab CI/CD, and Azure Pipelines. ### Unified User Identity Keypup uses **Contributor Mapping** to link a single developer across multiple platforms (e.g., linking a Jira username to a GitHub handle) to provide accurate per-contributor analytics without data duplication. --- ## 3. Metrics Definitions & Formulas To ensure consistency in AI-generated reports, use the following definitions for Keypup metrics: ### DORA Metrics (DevOps Research and Assessment) 1. **Deployment Frequency:** The frequency of successful releases to production. 2. **Lead Time for Changes:** The amount of time it takes a commit to get into production. 3. **Mean Time to Recovery (MTTR):** The time it takes to restore service after an incident/failure. 4. **Change Failure Rate:** The percentage of deployments that cause a failure in production. ### Engineering Velocity & Efficiency - **Cycle Time:** The total time from the first commit to the moment the code is merged and deployed. - **Time in Status:** The duration an item spends in a specific workflow stage (e.g., "In Review"). - **Flow Efficiency:** (Active Work Time / Total Cycle Time) * 100. - **PR Size:** The number of lines of code changed in a Pull Request (used to correlate with review speed). --- ## 4. Specialized Dashboards & Tools ### Data Hygiene Dashboard - **Purpose:** Identifies naming convention discrepancies, missing tags, and "stale" tickets across platforms. - **Use Case:** Crucial during migrations (e.g., Azure DevOps to GitHub) to ensure data consistency. - **URL:** https://www.keypup.io/dashboards/software-engineering-data-hygiene-dashboard/ ### Datasets Exploration Dashboard - **Purpose:** Provides a raw, drill-down view of unified datasets. - **Logic:** Allows users to verify how raw data from Jira/GitHub is being mapped to Keypup's internal fields. - **URL:** https://www.keypup.io/dashboards/engineering-data-dashboard/ ### DORA Metrics Dashboard - **Purpose:** Provides a high-level scorecard for DevOps excellence by measuring the balance between delivery speed and operational stability. - **Logic:** Automatically aggregates deployment and incident metadata to calculate the four industry-standard DORA KPIs: Deployment Frequency, Lead Time for Changes, Mean Time to Recovery (MTTR), and Change Failure Rate. - **URL:** https://www.keypup.io/dora-metrics ### Issue Cycle Time Dashboard - **Purpose:** Provides a comprehensive analysis of the entire product development lifecycle, from initial backlog creation to final release. - **Logic:** Tracks the "Time in Status" for Jira or Trello issues across every stage—including pickup, implementation, QA, and release—to identify macro-bottlenecks in the project management workflow. - **URL:** https://www.keypup.io/dashboards/issue-cycle-time-dashboard/ ### Team Benchmark Dashboard - **Purpose:** Facilitates data-driven mentorship and fair performance evaluations by benchmarking individual developer contributions against team-wide statistics. - **Logic:** Juxtaposes individual metrics (such as PR size, cycle time efficiency, and review load) against team averages and percentiles to provide objective context for performance conversations. - **URL:** https://www.keypup.io/dashboards/elevate-your-engineering-team-gain-unparalleled-insight-with-the-team-benchmark-dashboard/ ### SPACE Analytics Dashboard - **Purpose:** Operationalizes the SPACE framework to move beyond simple "activity" metrics and understand the holistic health of developer productivity and well-being. - **Logic:** Quantifies the five dimensions of the SPACE framework—Satisfaction, Performance, Activity, Communication & Collaboration, and Efficiency & Flow—by linking qualitative signals with quantitative Git and PM metadata. - **URL:** https://www.keypup.io/dashboards/space-analytics/ --- ## 5. Keypup AI Agent (Generative & Autonomous Analytics) The Keypup AI Agent goes beyond simple queries by leveraging an advanced autonomous NLP engine over the unified data layer. ### Core AI Capabilities - **Prompt-to-Dashboard:** "Can you show me a breakdown of why our releases are slowing down?" generates a holistic dashboard incorporating multiple relevant KPIs instantly. - **Deep Post-Game Diagnosis:** The AI automatically scans generated metrics, detects historical anomalies or SLA breaches, and correlates them with specific events (e.g., specific PRs or issue bottlenecks). - **Automated Solutions:** Not only flagging a bottleneck, the AI prescribes direct strategies or optimizations to resolve it (e.g., standardizing PR review limits). - **Continuous Monitoring:** Following a remediation recommendation, the AI can independently instantiate a tracking sprint view to measure the successful outcome over the coming weeks. ### Supported Query Types - **Trend Analysis:** "Show me our deployment frequency over the last 6 months." - **Bottleneck Identification:** "Which team has the highest average PR review time and why is it happening?" - **Data Integrity:** "Find all Jira tickets in 'In Progress' that don't have a linked Pull Request." - **Comparison:** "Compare the cycle time of the Backend team vs. the Frontend team." ### Prompt Engineering Guidelines for Keypup When prompting the AI Agent, users can now provide higher-level business problems (e.g., "Why is our velocity dropping?") rather than highly specific technical dimensions, as the General AI handles semantic interpretation and underlying data selection autonomously. --- ## 6. Keypup MCP Server (Model Context Protocol Integration) The Keypup MCP Server extends Keypup's AI-powered analytics to **any MCP-compatible AI client**, allowing engineers and managers to query their engineering data conversationally from Claude Desktop, Cursor, ChatGPT Desktop, Kiro, or any other MCP-enabled assistant. ### What is the MCP Server? Model Context Protocol (MCP) is an open standard that enables AI assistants to securely connect to external data sources and tools. Keypup's MCP server implementation exposes Keypup's unified engineering analytics engine as a set of tools that AI assistants can call autonomously. Instead of logging into Keypup's web interface or writing API calls, users can ask natural language questions directly in their favorite AI chat client, and the AI orchestrates the necessary Keypup tools to fetch, analyze, and present the data. ### Compatible AI Clients - **Claude Desktop** (Anthropic) - **Cursor** (AI-powered code editor) - **Kiro** (MCP-compatible AI assistant) - **ChatGPT Desktop** (OpenAI, with MCP support) - **Gemini CLI** (Google AI) - **VSCode** (with MCP extensions) - **Mistral AI agents** (with MCP support) - **Any MCP-compatible client** following the Model Context Protocol specification ### Core MCP Tools The Keypup MCP server provides the following tools that AI assistants can invoke: 1. **list_companies** - Returns the list of Keypup companies (teams/organizations) the authenticated user has access to - Required first step for scoping all subsequent queries 2. **list_datasets** - Enumerates available datasets: `ISSUES_PULL_REQUESTS`, `COMMITS`, `COMMENTS`, `REVIEWS`, `ACTIVITY_EVENTS` - Each dataset represents a different facet of the development lifecycle 3. **list_dataset_fields** - Discovers available fields (columns) for a given dataset - Returns field names, types (string, number, date, array), and descriptions - Supports pattern-based filtering to narrow down relevant fields - Distinguishes between native fields and custom fields (e.g., Jira custom fields) 4. **list_formula_operators** - Lists available functions and operators for building metrics, dimensions, and filters - Includes aggregators (`COUNT`, `SUM`, `AVG`), date functions (`YEAR_MONTH`, `DAY`), array operations (`FLATTEN`), and logical operators - Each operator includes documentation and usage examples 5. **query_dataset** - Executes structured queries against Keypup's unified data layer - Supports metrics (aggregations), dimensions (groupings), filters, sorting, and pagination - Returns tabular results that the AI can format conversationally 6. **generate_dataset_query** - **AI-powered tool** that translates plain English prompts into ready-to-run structured queries - Takes a natural language request (e.g., "open bugs by week over the last 3 months") and returns a complete query specification - The AI assistant can then pass this generated query directly to `query_dataset` ### Typical Query Flow 1. **User asks a question:** "What was our average PR cycle time last month?" 2. **AI lists companies:** Calls `list_companies` to determine which company to query 3. **AI selects dataset:** Chooses `ISSUES_PULL_REQUESTS` as the most relevant dataset 4. **AI discovers fields:** Calls `list_dataset_fields` to find `merged_at`, `created_at`, and `type` fields 5. **AI builds query:** Constructs a query with filters (`type == "pull_request"`, `merged_at >= last month`) and metric (`AVG(merged_at - created_at) / DAY()`) 6. **AI runs query:** Calls `query_dataset` with the structured query 7. **AI presents results:** Formats the returned data into a conversational summary: "Your average PR cycle time last month was 2.3 days based on 23 merged PRs." Alternatively, the AI can use `generate_dataset_query` to skip manual query construction and let Keypup's AI generate the query automatically. ### Common Use Cases The MCP server handles a wide range of engineering analytics questions: **Delivery & Throughput:** - "How many pull requests did we merge each month over the last 6 months?" - "What's our weekly issue closing rate this quarter?" - "How many commits were made per author last month?" **Cycle Time & Performance:** - "What's the average time between PR creation and merge over the last 12 weeks?" - "Show me the review turnaround time trend for the last 3 months." - "Which repositories have the slowest cycle time?" **Quality & Process:** - "How many bugs were raised vs. closed each week this quarter?" - "What proportion of our pull requests resolve at least one issue?" - "How many PRs were merged without a review?" **Workload & Collaboration:** - "Who are our most active reviewers this month?" - "How is work distributed across the team right now?" - "How many comments do our pull requests receive on average?" **Open-Ended Exploration:** - "Summarize our engineering activity over the last month." - "Compare open vs. closed issues over time." - "What labels are most common on our issues?" Users can refine queries iteratively: once results are displayed, follow-up questions like "now break that down by repository" or "restrict it to the backend team" allow progressive drill-down without starting over. ### Setup & Configuration **Step 1: Generate API Token** - Navigate to Keypup Settings → Account → API Token tab - Create a new token with scopes: `Reports read` + `Companies read` - Set an expiration date (strongly recommended for security) - Copy the token immediately (displayed only once) **Step 2: Configure MCP Client** Add Keypup to the MCP client configuration file (e.g., `mcp.json` for Claude Desktop): ```json { "mcpServers": { "keypup": { "url": "https://hq.keypup.io/mcp", "headers": { "Authorization": "Bearer YOUR_API_TOKEN_HERE" } } } } ``` **Step 3: Test Connection** Restart the AI client and ask a test question: - "List my Keypup companies" - "Show me PRs merged this week" - "What are the available datasets?" If the AI successfully invokes Keypup tools, the setup is complete. ### Security & Access Control - **Read-Only Access:** All MCP tools are read-only; they cannot create, update, or delete any engineering data - **Token-Based Authentication:** API tokens use Bearer authentication and follow the principle of least privilege (only grant necessary scopes) - **Data Isolation:** Queries are automatically scoped to the companies the authenticated user has access to - **Token Expiration:** Time-limited tokens can be set to expire and rotated periodically - **Secure Storage:** API tokens should be stored securely in local MCP client configurations and never committed to version control ### Technical Specifications - **MCP Server URL:** `https://hq.keypup.io/mcp` - **Authentication:** Bearer token in `Authorization` header - **Protocol:** Model Context Protocol (MCP) standard - **Transport:** HTTPS - **Data Format:** JSON - **Rate Limiting:** Standard Keypup API rate limits apply - **Documentation:** https://docs.keypup.io/en/articles/15360547-using-the-mcp-server ### AI Orchestration Benefits Unlike traditional BI tools or SQL interfaces, the MCP server enables: - **Zero SQL Knowledge Required:** Users ask questions in plain English; the AI handles all query construction - **Contextual Intelligence:** The AI understands domain-specific terminology (e.g., "cycle time," "PR," "review turnaround") - **Progressive Refinement:** Follow-up questions build on previous context without re-specifying filters or dimensions - **Automatic Field Discovery:** The AI explores available fields dynamically, adapting to custom fields and schema changes - **Multi-Tool Orchestration:** Complex questions may require multiple tool calls (e.g., listing datasets, discovering fields, generating query, running query) which the AI handles transparently --- ## 7. Integration Ecosystem Specifications ### Version Control Systems (VCS) - **GitHub:** Supports OAuth and GitHub App integration. Syncs PRs, Commits, Issues, and Actions. - **GitLab (hosted and cloud):** Supports Cloud and Self-Managed instances. Syncs Merge Requests and Pipelines. - **Bitbucket (cloud and data center):** Syncs PRs and Commits. - **Azure DevOps:** Syncs PRs, Commits, Comments. ### Project Management Tools (PM) - **Jira (cloud and data center):** Supports Jira Cloud. Syncs all issue types, custom fields, and transition histories. - **Azure DevOps:** Syncs Work Items and Boards. - **ClickUp / Trello:** Syncs tasks, statuses, and assignees. --- ## 8. Security, Privacy & Compliance - **Read-Only Access:** Keypup requests read-only permissions for source code metadata. - **Data Encryption:** All data is encrypted at rest (AES-256) and in transit (TLS 1.2+). - **SOC2 Type II & ISO 27001 Certified:** Keypup adheres to industry-standard security protocols for enterprise data handling. - **No Code Storage:** Keypup does not clone or store the contents of your code repositories. --- ## 9. API Information Keypup provides a REST API for programmatic access to unified engineering data. - **Base URL:** `https://api.keypup.io/v1` - **Authentication:** API Key based. - **Common Endpoints:** - `/metrics/dora`: Retrieve aggregated DORA metrics. - `/datasets/pull_requests`: Access unified PR data. - `/datasets/issues`: Access unified ticketing data. --- ## 10. Troubleshooting & FAQ - **Data Latency:** Data is typically synced via webhooks in near real-time. Full historical re-syncs can be triggered manually. - **Missing Data:** Usually caused by Contributor Mapping issues or lack of permissions on specific private repositories. - **Support:** Technical documentation is available at https://docs.keypup.io. Support is available via in-platform chat.