Jon Peck, a developer advocate for GitHub, discusses approaching AI adoption within organizations, noting that even experienced developers are accepting LLMs can perform tasks faster or better.
The focus is on efficiency gains from AI, with best-case scenarios showing up to 1.5x more feature points pushed out in the same time, and averages around 30% improvement in most companies.
There's also a significant increase in successful builds, indicating problems are caught earlier, and a general increase in developer happiness.
The first step for efficient AI use is to integrate AI directly into your IDE, ideally one that allows flexible context specification (e.g., VS Code with GitHub Copilot).
It's crucial to maintain model choice, using different models for different tasks, such as a "flash" model for quick, low-cost responses and a "reasoning" model for deep research.
When starting, prioritize "brownfield" development (modifying existing applications) over "greenfield" (new projects) to learn effective prompting and context shaping for daily tasks.
Agent mode allows for iterative scoping and prompt creation in greenfield projects, and in brownfield, it helps narrow down large contexts to avoid overloading the AI's input.
Developers should be comfortable reverting large sets of AI-generated changes if they're not on the right track, and ensure commits are made at set points for good Git history.
Telling the AI to keep a change log of its edits can be very useful.
Team practices and documentation, such as standards and playbooks, are enhanced by AI; copilot-instructions.mmd files can codify these best practices for AI adherence.
Lead developers are responsible for continuously revising copilot-instructions.mmd to reflect team decisions on linting, model access, and other patterns.
Make institutional knowledge accessible to AI through API specifications, existing patterns, or Copilot Enterprise's knowledge bases, which allow naming and referencing collections of repositories for specific best practices or internal APIs.
Foster a culture where it's acceptable to be wrong, ask questions, and share failures, as everyone is in a learning phase with AI.
When problems are solved, centralize that knowledge into guides or a team knowledge store for future reference.
Monitor team members' AI tool usage (e.g., Copilot metrics APIs) to identify and assist those who might be hesitant or falling behind.
Implement robust safety and privacy measures for AI tools, ensuring data sanitization, non-reuse of data for training, and indemnification clauses from providers.
Establish high-level governance, including repository-level exclusions to prevent leakage of sensitive information like environment secrets.
Implement organization-wide policy governance to control access to new features and models without requiring individual repository maintainers to manage it.
Beyond generating new production code, AI offers significant benefits in other development phases:
Planning: Brainstorming large-scale plans, technologies, and infrastructure.
Security: Quickly identifying bad dependencies, patterns, and vulnerabilities (e.g., cross-site scripting) in existing codebases.
Testing: Generating comprehensive unit tests and test cases with data examples, often in about 10% of the time it would take manually.
Deployment: Generating infrastructure-as-code files like GitHub Actions or Terraform.
Documentation: Producing markdown files, full API specs, executive summaries, and workflow charts (e.g., Mermaid charts) for internal or external users.
Autonomous AI and Model Context Protocol (MCP) 16:35
Autonomous AI should always include a human in the loop; in GitHub, this means agents work on new branches in protected environments and require human review for pull request merges.
Simple autonomous uses include auto-generating pull request descriptions with high-level summaries and file details.
Copilot Code Review allows assigning Copilot to a pull request, where it asynchronously generates comments and code suggestions that can be accepted or rejected by a human, similar to a human reviewer.
Issues can be assigned directly to Copilot, which will autonomously create a new branch, develop a solution, and present a completed pull request for human review, especially useful for tasks like adding documentation, tests, or specific functions.
Model Context Protocol (MCP) servers, available in VS Code and Copilot Enterprise, allow agents to interact with other systems, such as GitHub itself, to perform actions like creating branches, commits, PRs, and assigning reviewers directly from the IDE.
With Copilot Enterprise, MCP can be configured at the repository level, enabling Copilot to leverage these connections when assigned issues, allowing it to reach out to other systems to accomplish its work.