---
url: https://docs.nemo12.com/ops/qc-system.md
description: >-
  One CI job, parallel path-aware gates, two gate tiers (blocking vs report):
  the minimal QC system and the audit behind it (SRC-1431).
---

# Test and quality-control system

Owner-facing guide in Vietnamese: [harness.nemo12.com](https://harness.nemo12.com) (SRC-1434, VitePress on Cloudflare
Pages, source in `apps/harness/docs`). This page stays the technical source.

Owner, 18.10.2026: redesign testing and QC to be very fast, minimal and cheap; audit every place a
test runs. This page is the design and the audit it rests on. Numbers are measured, not estimated,
unless marked.

## Principles

1. **Each stage checks only what it can still catch cheaply.** Commit time catches mistakes in the
   files you touched, CI catches the whole-tree view, the weekly check catches slow things, bookkeeping and dependency upgrades,
   production checks catch what no test sees.
2. **A blocking gate must name what breaks for a user, a deploy or the data when it is skipped.**
   Everything else is a report: it tells you, it does not stop anyone.
3. **Pay for work, not for overhead.** GitHub rounds every job up to a minute and every job re-installs,
   so CI is one job. Gates run in parallel and only for the paths they cover.
4. **One gate list.** CI runs the same groups as the hook (`scripts/run-checks-groups.mjs`); there are no
   CI-only one-line gates (`check-ci-gates-local` keeps it that way).

## Stages and budgets

| Stage | What runs | Budget | Measured (18.10.2026) |
|---|---|---|---|
| pre-commit | autofix (last_reviewed, gen:reference, skills catalog); eslint + typecheck of staged workspaces; `run-checks code --staged` (parallel, path-aware); `run-checks docs --staged`, no-clobber, SRC canonical on docs commits | < 5 s | ~4 s for a learn or api commit (was 18-20 s) |
| pre-push | unit tests of touched workspaces and their consumers (`test-affected`); web mobile audit when `apps/web/src` changed | < 60 s | 14 s for learn + api |
| main CI (trunk-based: no PRs) | ONE job: scope, install from cache, typecheck + lint + unit tests + API suite in parallel, `check:code`, `check:docs`, ledgers, bindings; on main also migrations and deploy | < 5 min, 1 job | see below |
| weekly (`npm run weekly`, Sunday 22:00, checks `main`; owner 18.10.2026: "Mỗi tuần 1 lần thôi cho tiết kiệm") | `check:report` (non-blocking), minor/patch dependency upgrades, `npm audit fix` and removal of unused dependencies (knip) tried together and pushed when green (`scripts/deps-upgrade.mjs`, replaces Dependabot, SRC-1444), `npm audit --omit=dev`, E2E: the whole suite of every app (owner 18.10.2026: free on the Mac, so 10 times the old 10% set; `--quick-e2e` keeps the list in `scripts/lib/weekly-e2e.mjs`); macOS notification | < 30 min | learn full suite 5.4 min (903 cases); quick set 49 s |
| production | [synthetic checks](synthetic-checks.md) every 15 min (every public site, deployed Worker vs main), `check-production` after each main run and daily, Sentry | continuous | |

## Gate tiers

`npm run check:code` and `npm run check:docs` are **blocking**: types, builds, seeds matching their
source, the /v1 contract, migration safety and numbering, bindings, owner-mandated text rules (em dash,
date format, English identifiers, design stack), content gates that guard what learners see.

`npm run check:report` is **not blocking**: docs bookkeeping (REQ and SRC canonical, traceability
structure, coverage, freshness), one-way ratchets (shadcn adoption, learn UI debt, strict UI audit),
calendar-driven checks (test quarantine expiry) and harness links. The weekly check of main reports
the red ones. Before the split, a stale `last_reviewed` or an SRC row without a canonical doc held every app's
deploy.

## CI cost, before and after

Before: four jobs (`scope`, `gates`, `api-test`, `main`), 5.9 billed minutes per PR run on average, 4.4
per main push, 3 for a docs-only PR that checks for 5 seconds; about 9,000 billed minutes a month
(estimate from 24 sampled runs; the billing API is not readable with the session token). Seven gates ran
twice in the same job, one step never ran (it read `steps.scope` in a job without that step), and PR
runs saved their own 195 MB node_modules cache, pushing the repo to 10.6 of its 10 GB cache and evicting
main's copy.

After: one job. A PR run bills its own duration rounded up once (expected ~2-3 minutes), a docs-only
PR ~1 minute. Only `main` saves the node_modules cache; the npm download cache (`setup-node cache: npm`,
another ~208 MB per lockfile) is gone. 124 dead caches were deleted (10.6 GB to 1.4 GB).

A `package-lock.json` change no longer retests and redeploys all 21 apps: `scripts/lock-affected.mjs`
walks each workspace's dependency closure in the old and new lockfile and `scope.sh` takes only the
workspaces it reaches (root `package.json` still means all; an unreadable case prints `all`). On the
last six lockfile commits of main: one reached a single workspace, one five, and the design-system
dependency changes still reached every app.

## Local gate timings, before and after

| Command | Before | After |
|---|---|---|
| `check:code` (whole group) | 15.9 s, 51 gates in a row | 5.7 s in parallel |
| `check:code --staged`, learn commit | 15.9 s | 1.9 s (37 gates, 9 skipped by path) |
| `check:docs` | 12.2 s (`check-docs-fresh` alone 10.5 s) | 0.5 s (4 blocking gates) |
| `check-docs-fresh` | 10.5 s, one `git log` per doc | 0.4 s, one `git log` pass, identical result on 1,090 docs |

## Hooks and gates audit, 18.10.2026 (SRC-1444)

The owner asked whether the hooks, and the tests and gates they call, are still right. Measured on the shared
tree and a fresh worktree:

| Finding | Fix |
|---|---|
| Every Claude hook starts in ~55 ms; SessionStart hooks 1.4 s and 2.6 s | Fine, no change |
| The Claude hook tests (block-dangerous, project-scope) passed but no gate ran them | In `check:code` with `paths: .claude/hooks/` |
| Three gates red on the shared tree with misleading errors ("Could not resolve @nemo12/walk", an esbuild crash): node_modules lagged package-lock.json by 180 packages | `scripts/check-node-modules.mjs` (40 ms) in pre-commit and the SessionStart notice |
| pre-push ran the whole typecheck twice when it failed (stdout went to /dev/null, so it ran again to print) | One run, output captured; temp files removed by a trap |
| pre-push tested workspaces one after another | In parallel; a red one prints its log |
| The E2E list named two specs that moved to apps/grammar and apps/sat in the learn split; the night run would have gone red | List per app, program apps included; renamed `scripts/lib/weekly-e2e.mjs` |
| Dependabot kept opening PRs nobody merges in trunk-based work | Removed; `scripts/deps-upgrade.mjs` in the weekly run, holds in `scripts/deps-hold.json` |
| Nothing found unused dependencies: knip found 43 (Radix single packages replaced by `radix-ui`, unused fonts, `uqr`) | knip (`knip.jsonc`) removes them weekly; `check:report` lists what is left |
| The production-health notice expected a daily cron | Warns when a commit older than 2 h was never checked, or after 8 days |

Gates that still read the whole repo, merge candidates (four IELTS content gates into one, the two claim gates
into one test, the two i18n key gates into one), and wording left from the PR flow in skills are follow-ups.

## Audit findings kept for later

* **Speed to production was set by the nightly batch, not by the gates.** Resolved the same day: the owner
  moved to trunk-based work (commit, `git pull --rebase`, `git push origin HEAD:main`); `land`, `ready/*`
  and the nightly merge are gone, and the nightly job only checks main.
* **Low-value tests:** 6 source-regex tests (5 in `apps/web`, moving to nemo12-www), a byte-identical
  `useReferralClaim` hook and test in learn and marlins, content-length checks dressed as unit tests
  (`apps/learn/src/books/chapters.test.ts`). The suites are fast (api 3,144 cases in 10 s), so these
  cost clarity more than time.
* **E2E:** never on GitHub (gate `check-no-ci-e2e`, SRC-1444); the weekly check runs the whole suite on the Mac
  (owner 18.10.2026: "Nếu e2e test chạy trên local, ko tốn tiền thì có thể mở rộng gấp 10 lần"). The full learn
  suite measured 5.4 minutes, not the ~35 minutes once quoted.
* **Not adopted:** TypeScript 7 native, oxlint or Biome. Typecheck is 6-10 s and cached lint 2 s; the
  switch would add risk for seconds.

## Sources of the design

DORA continuous integration (build in a few minutes); Fowler's practical test pyramid (few end-to-end
tests, order stages by speed); Google's test sizes and flakiness data (*Software Engineering at Google*,
ch. 11); the CI workflows of Vite, Astro, Hono and Cloudflare workers-sdk (cancel stale runs, skip
docs-only work, one aggregate gate); Cloudflare rollbacks and gradual deployments for shift-right
safety.