Reviews never started: service-hook permission error

ADO Pilot connected to your Azure DevOps organization — the dashboard shows a successful connection — but no pull requests have ever been reviewed. Unlike Webhook 401s and reviews stopped firing, which covers reviews that used to work and then stopped, this page covers reviews that never started in the first place.

Symptom

What you observe

The dashboard's Integration settings page shows a connection, but a banner appears. Its color tells you the blast radius:

• Amber banner (some projects affected): "ADO Pilot can read your repositories but couldn't create the service-hook subscription for [N] project(s) ([names]) — pull requests are still being reviewed in your other projects, but new PRs in [these project(s)] won't be." Reviews are working normally everywhere else — only the named project(s) are affected.
• Red banner (no projects working): "ADO Pilot can read your Azure DevOps organization but couldn't create any service-hook subscriptions — pull requests aren't being reviewed anywhere in your organization right now." Nothing is being reviewed anywhere yet.

Either way, the Service Hooks table lists the affected project(s) with a specific reason next to them instead of a normal active status, and no pull request in those project(s) has ever received a review or a status check, even brand-new ones.

Both banners share the exact same root cause and fix below — the color only reflects how much of your organization is currently affected, not a different problem.

What's happening behind the scenes

Reading your code and posting review comments is a different Azure DevOps permission from creating the subscription that delivers pull-request events to us in the first place. The identity ADO Pilot uses — your personal access token's owner, or the ADO Pilot service principal — successfully authenticated and can read the repository, but Azure DevOps rejected the one-time request to create that subscription with a permission error. Nothing about your PR review setup is broken; the very first step, receiving the event, never completed.

Why it happens

Creating or editing a service-hook subscription requires the Edit Subscriptions and View Subscriptions permissions in Azure DevOps. By default, only Project Administrators (or, at the organization level, Project Collection Administrators) hold these — see Microsoft's Service Hooks FAQ. The Contributors group, which covers reading code and posting pull request comments, does not include it. If the identity ADO Pilot uses was only ever added to Contributors, subscription creation fails even though everything else about the connection looks healthy.

How to fix it

If you connected with the ADO Pilot service principal

The recommended fix, run once as a Project Collection Administrator, grants the service principal Edit Subscriptions and View Subscriptions, scoped to just the affected project:

az rest --method POST \
  --url "https://dev.azure.com/<YOUR_ORG>/_apis/accesscontrolentries/cb594ebe-87dd-4fc9-ac2c-6a10a4c92046?api-version=7.1-preview.1" \
  --resource 499b84ac-1321-427f-aa17-267ca6975798 \
  --headers "Content-Type=application/json" \
  --body '{"token":"PublisherSecurity/<PROJECT_ID>","merge":true,"accessControlEntries":[{"descriptor":"Microsoft.VisualStudio.Services.Claims.AadServicePrincipal;<YOUR_ENTRA_TENANT_ID>\\<SP_OBJECT_ID>","allow":3,"deny":0}]}'

Keep the \\ between the tenant ID and object ID exactly as shown (a literal backslash the JSON escapes), and leave 499b84ac-… as-is — it tells az rest to request an Azure DevOps token. See Setting up Service Principal auth, Step 2 for how to find <SP_OBJECT_ID> and <PROJECT_ID>, plus the multi-project (organization-wide grant) and no-CLI (Project Administrator) alternatives.

If you connected with a personal access token (PAT)

The Azure DevOps user whose PAT you connected needs the Project Administrator role on the affected projects. This is an organization permission on that user, not a PAT scope you can select when generating the token — no combination of PAT scopes grants it.

Confirm it worked

Once the permission is granted, open the dashboard Integration settings page:

PAT connections

Click Resync. This immediately re-creates the missing subscription using your existing token (it does not change your PAT) — the banner and per-project reason clear right away on success.

Service Principal connections

Click Re-check. This re-probes your organization directly and refreshes the status right away, but it's a read-only check — it doesn't itself re-create the subscription. If you grant the permission within about 48 hours of connecting the service principal, the banner clears automatically within a few minutes once ADO Pilot's background retry succeeds. Past that window the automatic retry has stopped, so open a support request below and we'll re-arm it for you.

Still stuck?

If Re-check or Resync keeps reporting the same failure after you've granted the permission above, double check it landed on the same project(s) the banner names — a permission granted at the wrong project, or only at a team level, won't take effect. If it still doesn't clear, open a support request with your Azure DevOps organization name, the affected project name(s), and your ADO Pilot dashboard email.

How to prevent it

Grant the service-hook permission up front during setup, alongside Contributors — the project-scoped Service Hooks grant is recommended (see Setting up Service Principal auth, Step 2) — or see Service hooks and webhooks for what ADO Pilot needs and why.