Require passing checks before merging

Make the intended check a requirement for merging

The failure mode is always the same story. CI runs on every pull request and posts its little green checkmark. But nothing enforces it: someone merges while the run is still in progress, someone else clicks through a red X because “it’s probably flaky”, and main drifts into a state no commit ever tested. The pipeline exists; the gate doesn’t. Advisory checks measure quality culture — required checks enforce it.

The distinction lives in branch protection, not in your workflow file. GitHub Actions can only report results; whether those results block anything is decided by rules attached to the branch. That separation confuses people once and then costs them years of false confidence, so let’s make the enforcement layer explicit. If you haven’t built the underlying test job yet, start with our Playwright on GitHub Actions guide, then come back and make it binding.

Configure the required check in GitHub

In repository settings, under Branches (or Rules → Rulesets on newer plans), create a rule targeting main. Three switches matter most: require a pull request before merging, require status checks to pass, and select which checks are required from the list of checks that have recently run. Via CLI, the same configuration is one API call:

gh api repos/{owner}/{repo}/branches/main/protection -X PUT \
  --input - <<'JSON'
{
  "required_status_checks": {
    "strict": true,
    "contexts": ["e2e / verdict", "lint"]
  },
  "enforce_admins": true,
  "required_pull_request_reviews": {
    "required_approving_review_count": 1
  },
  "restrictions": null,
  "allow_force_pushes": false,
  "allow_deletions": false
}
JSON

Two settings deserve explanation. "strict": true requires the branch to be up to date with main before merging — it closes the “tested against stale base” hole where two compatible-looking PRs merge independently and only their combination is broken. "enforce_admins": true applies the rules to administrators too; leave it off and you’ve built a gate with a permanent, temptation-shaped hole in it. Note the contexts list: required checks must be registered by exact name, which is where most gates quietly fail — more on that below.

Advisory versus required, precisely

An advisory check runs and reports. Anyone with write access can merge past a failing advisory check; depending on settings they may even merge before it finishes. Nothing about this is visible in the UI as dangerous — the red X looks identical whether or not it has teeth. A required check, by contrast, changes what the merge button does: the PR cannot merge until every required context reaches success. Not “completed” — succeeded. Skipped jobs count as neither, which is why a required job guarded by if: conditions can leave a gate permanently stuck rather than safely passing.

Choose per check, not globally. Require what verifies correctness of the diff: build, lint, unit tests, critical-path E2E. Keep long-tail suites advisory or scheduled — a 40-minute exhaustive run as a hard requirement converts every typo fix into a coffee break and teaches the team to hate the gate. Tiering by cost is exactly the strategy behind continuous testing across the pipeline: fast blocking feedback on PRs, deep coverage on schedule.

The naming trap

Required contexts match check-run names byte-for-byte, and several common patterns change names invisibly. Matrix jobs append their matrix values: a job test sharded four ways produces e2e / test (1) through (4) — requiring plain e2e / test protects nothing. Renaming a workflow file changes names. Display-name edits change names. Each rename leaves the old required context pointing at a check that will never run again, which means the gate either wedges every PR or — if someone “fixes” it by removing the requirement — stops existing.

The robust pattern is an aggregator: a final job that runs after all shards and succeeds only if every shard succeeded. Require the aggregator’s single stable name, and future shard-count changes never touch branch protection again. External systems integrate the same way — Guard, for instance, posts its pass/fail/flaky verdict as a native check run precisely so it can occupy a slot in the required list like any first-class job (see the GitHub Actions integration).

Investigate checks that pass only after a retry

Before making a suite required, earn it: cap retries at one, track which tests consume them, and move repeat offenders to a quarantine lane that still runs and reports but doesn’t block, with a named owner and expiry date. Review flake rate weekly like an SLO; set a target your team can defend from observed history. Diagnosis comes before policy — timing races, shared state, third-party outages, and environment drift each need their own fix, laid out in our flaky-tests guide. A merge queue pairs well here: with strict mode plus a queue, each PR is validated against the true tip of main, serializing the small window where independent branches would otherwise conflict untested.

Define how to unblock a broken gate

Hard gates meet reality at the worst time — production is down, the fix is three lines, and CI takes twelve minutes you don’t have. Decide the protocol now, calmly, instead of improvising during an incident. The standard toolkit: an admin-override path that works but logs loudly (audit entries are reviewable later); a small, named bypass list for on-call rotations rather than “everyone”; and a written rule that any bypass triggers an immediate follow-up — revert forward, land the green fix behind it, and post the incident note. Some teams invert the pressure entirely: allow bypass only via a dedicated hotfix branch whose protection is slightly softer, keeping main’s gate absolute.

Review how often checks are bypassed and why. Repeated overrides may indicate slow tests, unreliable environments, or a policy that no longer fits the team. Retain the captured steps and failure details so an engineer can investigate the result. For rollout guidance, see regression testing strategies.