Why some findings are dropped before they reach your PR

ADO Pilot reviews each pull request in two passes. The first pass casts a wide net to catch every issue it can. The second pass re-reads each candidate finding in context and discards the ones that do not hold up. Roughly 30 to 50 percent of candidate findings are dropped before they ever reach your PR — and that rate is the design, not a defect.

The first pass — a broad sweep

The first pass is tuned for recall. Its job is to surface anything that might be a problem so the second pass has a complete candidate set to work with.

• The reviewer reads the entire diff plus enrichment context — static-analysis findings and code-structure summaries computed from your code.
• It produces an internal list of candidate findings, each with a location, a guessed severity, a category, and an evidence chain that explains the reasoning.
• The first pass deliberately tolerates false positives. A candidate finding only needs to clear the bar of "this could plausibly be wrong" — not "this is definitely wrong."
• None of these candidates are posted to your PR yet. They exist only inside the review while it runs.

While the first pass is running, the tracking comment on your PR shows the in-progress placeholder — Analyzing N files (+X / −Y lines) — with no finding bullets. That blank state is intentional. Posting first-pass findings directly would mean roughly one in three would later be retracted, which is exactly the noise the two-pass design is built to avoid.

The second pass — a critical re-check

The second pass takes the first-pass candidate set and re-evaluates each one with the full picture in view.

• It rescinds false positives and findings that are technically true but too trivial to mention. ("This isn't actually a bug because the caller already validates the input.")
• It confirms the findings that hold up and writes the user-facing comment text — the one-liner, the evidence prose, and the optional code suggestion.
• It adjusts severity when the first pass over-classified — for example, downgrading a "blocker" to a "suggestion" once the second pass sees that the surrounding code already handles the case.

Findings the second pass rescinds are the only ones dropped. Everything else — the findings it confirms and refines, plus any candidate it doesn't explicitly act on — posts to your PR.

Why the drop rate is a feature

A reviewer that is conservative on the first pass — "only flag what you are sure about" — misses real bugs and feels timid. A reviewer that is aggressive on the first pass and posts everything floods you with false alarms and trains you to ignore it. Two passes get both:

• The first pass is aggressive, so genuine defects are never silently dropped.
• The second pass is critical, so the noise floor on your PR stays low.
• The second pass runs a full re-check of its own, not a quick touch-up — see How long does a review take? for how that splits across the review's total run time.

The result is a design target of roughly 30 to 50 percent of candidate findings dropped before they reach the PR — the range the two-pass design is built around, not a per-review guarantee. If you compared the in-progress candidate set against the final review, you would see that gap. From the PR author's perspective, it shows up as a clean, scannable set of findings — never as a "thinking out loud" stream.

Timeline

push
  -> "review queued" comment posts
  -> first pass runs (broad sweep, internal candidates only)
  -> "review in progress" comment (still no findings shown)
  -> second pass runs (drop, refine, write user-facing text)
  -> inline comments post for each confirmed finding
  -> tracking comment finalizes to PASS, ADVISORY, or FAIL
  -> ai-pr-review status check updates

The whole sequence usually runs in 2 to 5 minutes. See How long does a review take? for the full timing breakdown.