Scaling AI-Powered Code Review: A Multi-Agent Architecture

Introduction

Code review is a cornerstone of software quality, but it often becomes a bottleneck in engineering workflows. A typical merge request enters a queue, waits for a reviewer to context-switch, then cycles through nitpicks and corrections. At Cloudflare, the median time for the first review across internal projects was measured in hours. To address this, we explored AI-powered code review—not just as a helper, but as a core part of our CI/CD pipeline.

Our journey began with off-the-shelf AI code review tools. Many worked well and offered customization, but none provided the flexibility needed for an organization our size. The next step was a naive approach: feeding a git diff into a large language model (LLM) with a simple prompt. The results were noisy—vague suggestions, hallucinated syntax errors, and advice to add error handling to functions that already had it. It became clear that a generic summarization would not scale for complex codebases.

Building a Multi-Agent Orchestration System

Instead of building a monolithic agent, we created a CI-native orchestration system around OpenCode, an open-source coding agent. When a Cloudflare engineer opens a merge request, the system launches up to seven specialized AI reviewers. Each focuses on a distinct domain: security, performance, code quality, documentation, release management, and compliance with our internal Engineering Codex. A coordinator agent manages these specialists—deduplicating findings, assessing severity, and posting a single structured review comment.

The Coordinator Agent

The coordinator is the brain of the operation. It aggregates outputs from all specialized agents, removes duplicates, and judges whether each issue is a genuine bug or a minor suggestion. Only critical and high-severity findings are flagged as blockers, while low-priority items are surfaced as recommendations. This prevents review fatigue and keeps the focus on real problems.

Architecture Deep Dive: Plugins and Flexibility

To support thousands of repositories with diverse tech stacks, our system had to be extensible. We built a plugin architecture that abstracts away version control systems, CI providers, and coding standards. Each specialized reviewer is a plugin, configurable per repository or team. The plugin system allows teams to add custom reviewers, adjust severity thresholds, or even disable certain checks for specific branches.

Plugin System

Plugins are lightweight containers that implement a standard interface: they receive a diff, context (e.g., repository metadata), and return a list of findings with severity, category, and suggested fix. The coordinator merges these lists. This design made it easy to experiment—our team added a performance reviewer using a fine-tuned model, and later swapped it for a smaller, faster model without changing the pipeline.

Results: Thousands of Merge Requests, Minimal Noise

We have been running this system across tens of thousands of merge requests internally. It approves clean code, flags real bugs with impressive accuracy, and actively blocks merges when it detects genuine security vulnerabilities or serious errors. The false positive rate is well below 5%, thanks to the specialization and deduplication. Engineers report that the AI review is now a trusted second pair of eyes, not an annoyance.

This initiative is part of our broader Code Orange: Fail Small program, which aims to improve engineering resiliency by catching issues early and reducing manual toil. By orchestrating multiple AI agents, we turned a bottleneck into a force multiplier.

Conclusion

Building a single, monolithic AI code reviewer is tempting but often fails in practice. Our approach—using a coordinator to manage specialized plugins—has scaled to thousands of repositories and tens of thousands of reviews. The key lessons: embrace specialization, invest in deduplication, and treat the AI as an integral part of your CI/CD pipeline, not an afterthought. The result is faster reviews, higher code quality, and happier engineers.