03 — Testing and Verification
Commandment III: Verify, don’t declare.
Two different things, often conflated:
- Tests — an automated safety net you run repeatedly.
- Verification — proof that this specific change does what it was meant to, observed, not assumed.
Both are mandatory. A passing test doesn’t mean the feature works in the browser; manual verification without tests won’t protect you from tomorrow’s regression.
The pyramid (from the reference project)
| Level | Tool | What it covers | When |
|---|---|---|---|
| Unit | pytest / Jest | pure logic (normalizer, matcher, helpers, validators) | every commit |
| Integration | Supertest (HTTP) | routes, auth guards, JSON API — An agreed way for two programs to talk to each other — one asks, the other answers in a set format. Through an API your app connects to outside services (payments, maps, AI). Treat an API key like a password. | every commit |
| E2E / browser | Playwright | real browser paths, on a separate port | before deploy / when touching UI |
| Smoke | script (smoke.ps1) | “does the server come up and do N routes respond correctly” | after restart / before and after deploy |
Rules that have proven themselves:
- Run the tests before committing the critical path.
- Jest with
--runInBandwhen a dev/preview server is running (port/DB conflicts). - Smoke test — A quick “does it even work” test right after a deploy — checks the most important paths (e.g. login). Catches disasters in 30 seconds before a user sees them. A cheap way to sleep well after a deploy. have markers — they check that the right page rendered (e.g.
price-gatefor an anonymous user, not just HTTP 200). - Test auth from both sides — anonymous user bounced to login and logged-in user sees the content.
”Verify, don’t declare” — in practice
After a change observable in the app, show proof; don’t ask the user to check it themselves:
- HTTP status /
redirect_url(curl-o /dev/null -w), - a confirming HTML fragment (e.g.
og:imagepoints at the right file), - numbers from the database (how many accounts, how many prices,
integrity_check: ok), - a screenshot for visual changes.
After a deploy — a smoke test against the live server (via localhost, behind the maintenance
flag); and when you find a bug inside the window, fix it in the window if it’s trivial and safe (in the
reference project this is how we caught and fixed a pre-existing ERR_HTTP_HEADERS_SENT on /mapa).
The report must be honest
- Tests fail → say so with the output; don’t hide it.
- Something was skipped → say it was skipped.
- When 2 tests fail from data drift (e.g. cohort fixture
85% vs 85%) rather than a regression — flag it explicitly, don’t let it block the deploy, but propose a follow-up (regenerate the fixtures). - Say “done and verified” only when it’s actually verified — no hedging, but no bluffing either.
Verify numbers at the source
Don’t trust a number from memory or from the docs — query the database/test. Docs go stale;
SELECT COUNT(*) doesn’t lie. (Commandment X: “verify the numbers”.)
Web — a proven set of checks
For web sites/projects this set of tests has already proven itself in practice (a typical set runs to a few dozen methods and a few hundred subtests) — carry it over by default:
- SEO meta (title/description per page), canonical + hreflang — A tag that tells the search engine what language a page is in and where its versions in other languages are. Without it a multilingual site confuses Google — the language versions “fight” over the same phrase., JSON-LD — Structured data: a description of the page invisible to humans that the search engine reads (e.g. FAQ, author, breadcrumbs). Opens “rich snippets” in Google — richer results, higher CTR..
- Accessibility: color-token contrast, keyboard navigation, sensible
alt/aria. - EN↔PL parity (and every language pair): a missing key/page in one language = a test that fails.
- Dead internal links — no link in the build leads into the void.
- Theme consistency (theme cookie / dark-light) and build correctness (every page rendered).
Anti-patterns
- 🚫 “Should work” as a conclusion.
- 🚫 Running a single test and declaring “suite green”.
- 🚫 Skipping the smoke test after deploy because “the tests passed anyway”.
- 🚫 Confusing “the code compiles” with “the feature works for the user”.
TDD — the default working mode
Test first, not after the fact. The cycle:
- Red — write a test that describes the expected behavior and fails.
- Green — the smallest code change until the test passes.
- Refactor — clean up against a green suite (→ 02).
You fix a bug the same way: first a test that reproduces the bug, then the fix — the test stays as a regression guard so the bug doesn’t come back.
Every change in a commit carries a test. Hard rule: a commit that changes behavior but doesn’t add/change a test is incomplete. New Endpoint — A single “address” in an API you send a request to for a specific thing (e.g. the list of orders). Apps talk through endpoints — one endpoint = one function you expose. → route test; new threshold/fallback → threshold test; fixed bug → regression test. “I’ll add tests later” = a sin (later never comes). → 00, 08
- Test the contract, not the implementation — otherwise a refactor crumbles tests with no real regression.
- Keep the suite fast and deterministic — a slow or flaky suite stops getting run;
isolate I/O, set seeds,
--runInBandon a shared port/DB. - CI gates — red tests block merge and deploy; you run critical-path tests before committing.