Skip to main content
2 video tutorials

Watching repositories

Watching a repository lets you see what change management would do with it before you switch change management on. Merge Pilot records the repository's merge requests and decides each one with the change policies, exactly as it would under change management, but writes nothing to the repository except the delivery of its events. Nothing is blocked and nobody's work changes.

For a short overview of what it is for, see the Monitor mode & bulk onboarding feature page.

The record built while watching is the evidence for the next step: how many merge requests would have waited or been blocked, how many change approvals a week change management would open, which controls would hold merges, and which changes were pushed without a merge request. When you are ready, move the repository to change management.

What is written to the repository​

VCSWritten when a repository is watchedRemoved when it stops being watched
GitLabOne project webhook, so the events arrive (see The webhook). Creating it needs the Maintainer role for the connection's bot userThe webhook
GitHubNothing. The GitHub App's webhook already delivers the events of every repository its installation covers; Merge Pilot only checks that the installation reaches the repositoryNothing

There is no branch protection, approval rule, ruleset, status check, comment or change approval. In the setup status this is the single step Event delivery (API key events).

What Merge Pilot does with a watched repository​

  • Merge requests into the target branches are recorded and decided by the policies. The decision (would pass, would wait, would be blocked, would need a change approval) is kept on the merge request. Nothing is written to the merge request on the VCS: no merge gate, no status comment, no change approval, no closing.
  • Merges are recorded. A merge that change management would have held is marked as such.
  • Open merge requests are read back from the VCS about every 6 hours, so an event the VCS did not deliver is caught. When this falls behind, the Background work box on the Projects page says so.
  • Changes pushed without a merge request to the target branches or the default branch are recorded as Would have been blocked (see Direct changes).
  • Scans run on watched merge requests only when the company turns them on (see Scans on watched repositories).
  • Notifications (to routes and personal emails) are sent for watched repositories only when the company's Watched repositories setting (Company → Notifications) is on. It is off by default. A merge request that would be blocked is then sent as Merge request blocked with the title MR would be blocked. Merge requests imported from the history, and those left watched after their repository moved to change management, never send any (see What holds events back).

The target branches of a watched repository are its own, else the company's branch patterns, less its excluded branches (see Project page).

Watching a repository​

From the Projects page​

  1. Open Projects and, with more than one VCS connection, choose the connection.
  2. Select Watch repositories.
  3. Choose the repositories. The dialog lists the connection's repositories 50 at a time, searched on the server however many there are:
    • Search repositories narrows the list to repositories whose path contains the text.
    • Each repository already in Merge Pilot shows its mode (Watching or Change management) and cannot be chosen.
    • Choose the N shown chooses every repository listed that is not in Merge Pilot yet; Show more loads the next 50.
    • What you choose stays chosen while you search for more. Clear empties the choice.
    • Take every repository matching "…" that is not in Merge Pilot yet (or, with no search text, Take every repository of the connection that is not in Merge Pilot yet) takes every match instead of a list; the repositories are found when the operation runs.
  4. Target branches (optional): the branches whose merge requests are followed. Leave it empty for the company's branch patterns, which the dialog names. The branches are set only on repositories added now; a watched repository's branches are changed on its project page.
  5. Select Watch N repositories (or Watch every matching repository).

At most 10,000 repositories are watched at a time. The dialog notes that nothing is written to the repositories and, when the company imports past merge requests, how many days are imported (Merge requests merged in the last 90 days are imported.).

The repositories are watched in the background as a bulk operation; its progress shows on the Registered projects tab.

Only repositories the connection can watch are listed: on GitLab those its token may add a webhook to (the Maintainer role); on GitHub those the App is installed on. People who are not company admins see only the repositories they hold the minimum role of the connection's rule for Set up change management on.

From a repository's page​

A repository that is not in Merge Pilot yet (opened from the search tab) shows Not registered for CM with two buttons: Watch and Set up CM. Watch watches that one repository, as a bulk operation of one, and the page then shows it as watched.

Who may watch​

Company admins and platform admins; and people the connection allows Set up change management who hold the rule's minimum role on each repository. See Who may do what.

A repository already under change management is not watched again (the repository is under change management already): switch it to watching instead. Watching a repository that is already watched sets its event delivery up again.

Automatic watching​

A company can have Merge Pilot add the repositories its VCS connections gain, without anyone choosing them. The settings are on Company → Repositories (Watch new repositories automatically, Check for new repositories every (minutes), New repositories are added); see Company repository settings.

  • When. Each active VCS connection of the company is checked at the company's interval (15 to 1,440 minutes; 60 by default). A GitHub connection is also checked as soon as GitHub reports that the App's installations changed (a repository was added to the installation, or the App was installed somewhere new).
  • What is added. Every repository the connection can reach that is not in Merge Pilot yet. Left out are:
    • repositories someone took out of Merge Pilot on purpose (stopped watching, or removed change management), until someone adds them again;
    • repositories that could not be added in the last 6 hours (they are tried again after that);
    • repositories outside the company's Watch only repositories matching patterns, inside its Never watch repositories matching patterns, and forks when Skip forks is on.
  • How many. At most Repositories one round adds (500 by default) are added per check; the next check goes on with the rest. When automatic watching would take more than Repositories one listing reads (20,000 by default) of a connection's repositories, the connection is not read: the check fails with automatic watching takes more than N of the connection's repositories, more than one listing reads (Company → Repositories → Repositories one listing reads): raise it, narrow automatic watching with include or exclude patterns, or watch them with bulk operations on the Projects page (see Bulk operations). Only repositories the include and exclude patterns and Skip forks take count towards that number, so narrowing with patterns helps a connection that reaches very many repositories.
  • Outcome. How each connection's last scheduled check went is kept on the connection and shown on Company → Repositories under Watch new repositories automatically: <connection>: last checked <when>, N repositories added., or <connection>: the check <when> failed: <error>. API: the VCS connection's discovery (at, added, error).
  • In which mode. As New repositories are added says:
    • Watched only: each repository is watched, as above.
    • Under change management: each repository is put under change management at once, with the setup steps the VCS connection has on by default (see Change management setup). The connection's defaults stand for the consent: the setup records it as connection defaults. A repository already under change management through another connection is watched instead (see One connection per repository under change management).
  • Audit. Each repository added is recorded as project_watched or cm_setup, made by Merge Pilot, with automatic in the details.

Only one check runs for a connection at a time, also across several server instances.

Importing past merge requests​

When a repository is first watched, Merge Pilot can import the merge requests it merged before, so the record covers a period from day one instead of starting empty.

By default nothing is imported: watching records only what happens from then on, and the repository's page says Past merge requests are not imported: the company imports no history (Company → Repositories → Import past merge requests). instead of offering an import. A company that wants the history sets the number of days; it is a background task that can take long for many repositories (each merge request is evaluated on its own, at a lower priority than live events, within the VCS's rate limit).

  • Which. Merge requests merged into the repository's target branches (less its excluded branches) in the last N days, N being the company's Import past merge requests (days) (0 by default: nothing is imported unless the company sets a number of days; at most 365). Open and closed merge requests are not imported: open ones are followed live.
  • How many. At most 1,000 merge requests per import. Merge requests Merge Pilot already knows (seen live, or imported before) are skipped.
  • How. Each one is evaluated in the background as it was, at the lowest priority (History import on the Jobs page), so live merge requests are never held up. The Background work box on the Projects page shows how many repositories are still being imported, with an estimate at the current pace.
  • Scans. Imported merge requests are scanned only when the company turns on both Scan watched merge requests for secrets, vulnerable dependencies and licenses and Also scan imported merge requests for secrets and vulnerable dependencies.
  • Marked. Imported merge requests are marked as imported (the monitoring report's CSV has an Imported column).
  • A repository moved to change management before its import ran gets no import.

Status​

The import's state shows in the CM status column of the Projects page and on the project page:

ShownAPI backfill.statusMeaning
Importing past merge requests…pending, runningQueued or running
N past merge requests found and imported in the backgrounddoneFinished: N were queued for import (those imported before included, judged again)
The import of past merge requests failed: …failedIt failed, with the reason. An import that no background job is working on any more (for example after a restart) fails after 2 minutes with interrupted: no job is working on it any more (a restart or a lost job); start it again

backfill also has since, imported, skipped, error, queued_at, started_at and finished_at.

Importing again​

On a watched repository's page, Import again (or Import past merge requests when none ran yet) imports the window again: merge requests imported before are judged again with the current policies (for example after policies were added), and those Merge Pilot recorded live are skipped. It needs the same rights as watching. It is refused for a repository under change management (only a watched repository's history is imported) and when the company imports no history (the company imports no history (Company → Repositories → Import past merge requests)); while an import is queued or running, nothing new starts. When the import cannot be queued, the answer says so and the import shows as failed; try again later.

API: POST /api/v1/projects/{id}/import-history (409 for the two refusals above, 503 when it cannot be queued). Audit: project_history_imported (failure when it cannot be queued).

The monitoring report​

Insights → Monitoring shows what change management would have done with the watched repositories' merge requests, from what Merge Pilot recorded while only watching them, decided by the current policies. Every member of the company can read it.

Period and scope​

  • Period: 30 days, 90 days (the default), 180 days or 365 days. A merge request counts when it was opened or merged in the period.
  • Scope: every watched repository of the company. From a repository's page, See what change management would have done opens the report for that one project (Project: …; the cross clears it). One project's report also works after the project moved to change management: it keeps its evidence.
  • At most 5,000 merge requests are counted, the newest of the watched repositories in scope. When there are more, the page says so and suggests a shorter period or one project.
  • Merge requests that matched no policy are left out of the report (its counts and the CSV), and the page says how many.

Cards​

Card (label in the UI)What it counts
Merge requestsThe merge requests of the period, with how many merged
Would be approved / wait / blockedThe merge gate change management would set
Would need a change approvalThose whose policy needs a change approval, with the projected load (About N change approvals a week)
Merges these policies would holdMerges that the current policies, evaluated now, would have held
Merges held when they mergedMerges recorded as held at the time they merged, under the policy then in force (what each merge request shows)
Direct changesChanges pushed to the watched branches without a merge request (change management would have stopped them)
Merges per weekThe delivery baseline
Median lead timeFrom opened to merged

Tables​

  • By policy: per policy, its change type (or Refuses for a policy that refuses merges), and its Merge requests, Approved, Would wait, Blocked and Need a change approval.
  • What would hold them: per control, how often it Failed or was Waiting.
  • By project: per project, its mode and Merge requests, Blocked, Need a change approval, Held merges and Direct changes. Projects are filterable by path; select one to open its page.
  • Findings: possible secrets, new vulnerabilities and dependencies with a denied license found by the scans.
  • Risk: the merge requests per risk level, when risk is assessed.

Download CSV exports the merge requests with the columns Project, Merge request, Title, Author, Opened, Merged, Policy, Change type, Would be, Needs change approval, Merge held under these policies, Merge held when it merged, Failing controls, Imported and Link.

API​

ActionRequest
ReportGET /api/v1/companies/{id}/monitoring/report with since and until (RFC 3339; by default the last 90 days; at most 366 days) and project_id
Report under draft policiesPOST /api/v1/companies/{id}/monitoring/report with the same parameters and {"policies": [...]}: the draft policies are tried instead of the saved ones; nothing is saved
CSVGET /api/v1/companies/{id}/monitoring/report.csv with the same parameters
What the chosen projects recorded while watchedPOST /api/v1/projects/evidence with project_ids (at most 10,000); answers recorded and held

The report has merge_requests, unmatched, merged, approved, pending, blocked, needs_change_approval, would_have_blocked, held_at_merge, direct_changes, change_approvals_per_week, merges_per_week, lead_time_hours, policies, controls, projects, findings, risk, truncated and drafts.

Scans on watched repositories​

With the company's Scan watched merge requests for secrets, vulnerable dependencies and licenses on (the default), watched merge requests are scanned as their policy says: secret scan, dependency scan, licenses, and the CI configuration check and commit signature check, which count as scans too. Only merge requests whose policy runs a scan are scanned.

With it off, watched merge requests are not scanned, and a re-scan of one is refused (scans are off for watched repositories (Company → Repositories)). The report then has no findings for them.

Stop watching​

On a watched repository's page, Stop watching asks Stop watching this repository? and, when confirmed:

  • removes the event delivery (GitLab: Merge Pilot's project webhook; GitHub: nothing to remove);
  • takes the project out of Merge Pilot. Its open merge requests are closed in Merge Pilot's record and no longer followed;
  • keeps what was recorded: merge requests stay, and its direct changes stay until the company's retention removes them. Its vulnerability inventory and analyses are deleted;
  • makes automatic watching leave the repository out until someone adds it again.

A repository that was once under change management and was switched to watching keeps on GitLab what change management created. For it, the dialog offers Also take back what change management created: the branch protections and bot membership Merge Pilot added are removed and the approval settings it changed put back. What GitLab refuses to take back is listed after the repository is removed, to remove by hand (see What change management created).

It needs the Remove change management permission on the connection (company admins always may) and, for everyone else, the rule's minimum role on the repository. A repository under change management cannot be unwatched (the project is under change management: remove change management instead).

API: POST /api/v1/projects/{id}/unwatch (with undo_setup to take back what change management created). Audit: project_unwatched.

Video tutorials

Short videos of this feature working on real GitLab and Jira projects. English audio, English and Turkish subtitles. All video tutorials →

Watch Repositories Before You Enforce AnythingYouTube ↗
The Monitoring Report of Watched RepositoriesYouTube ↗