AI & ML Quick Path

Company case

GitHub: Code assistants

Developer toolsIntermediateRebuild effort 30–40 h (estimate)

What the published sources confirm

GitHub ran a controlled experiment with 95 professional developers writing an HTTP server in JavaScript, with and without GitHub Copilot, plus a survey of more than 2,000 developers.

Business problem and AI concept

Problem
Developers spend time on repetitive code; measuring productivity is difficult.
Concept
Context-aware code completion, and how to measure its effect with a controlled experiment.

Reported results

  • The Copilot group completed the task 55% faster on average (95% confidence interval 21%–89%); completion rate was 78% versus 70%.

    GitHub blog, Sep 2022. Caveat: One task in one language; does not measure long-term code quality.

Suggested workflow

Written for learning, not the company's system. This flow shows how you could build something similar. It is not a description of GitHub's private architecture.

  1. Step 1: Read permitted repository context near the cursor

  2. Step 2: Retrieve related symbols and files

  3. Step 3: Generate a suggestion with a model

  4. Step 4: Developer accepts, edits or rejects

  5. Step 5: Compile and run tests in a sandbox

  6. Step 6: Track acceptance and test results

Suggested technologies and alternatives

LayerSuggestedAlternatives
ContextRoslyn for C# symbolstree-sitter
RetrievalEmbeddings over code + docsBM25 over identifiers
SandboxDocker with no networkEphemeral CI runner

Small example

Input
Method signature `decimal CalculateVat(Order order)` and existing tests.
Output
Suggested implementation plus two new unit tests; tests run and pass in a sandbox.

Rebuild it yourself

Build a repository Q&A and test-suggestion assistant for a small sample .NET solution.

Definition of done

  • Answers cite file paths and lines
  • Generated tests compile and run in an isolated container
  • Measure on 10 tasks with and without the tool

Security and privacy risks

  • Leaking private code to third parties
  • Insecure suggested code
  • Licence issues in suggestions

Sources

Listed by title and publisher. This guide contains no external links; search for the title if you want to read the original.

  • Research: quantifying GitHub Copilot's impact on developer productivity and happiness. GitHub blog, 7 Sep 2022. Company research.
  • The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (Peng et al.). arXiv, 2023. Research paper.