The suite is green. The code it tested is gone.
If you are commissioning AI-built systems, the question you actually need answered is "how would I know if it stopped working?" — and the usual answer, "there are tests and they pass", is weaker than it sounds. Here are five ways a suite stays green while the thing it verifies is broken. All five are real; all five were found in one week, in one codebase, by the person who wrote them. Pick a case and undo the fix yourself.
Honest-AI note. No model runs on this page — and it isn't really a page about AI. It is about the discipline that makes AI-written code safe to depend on, because an agent that writes its own tests can write five green tests that prove nothing just as easily as a human can. Every result below is computed live: each case is a working pair of implementations and a working pair of locks, and the PASS/FAIL cells come from running them in your browser, not from a table someone typed.
Plain-language key (lock, fixture, gate, vacuous pass, bytecode cache)
- Lock
- A test written specifically to keep a past correction from being undone. Its whole value is that it turns red when the correction disappears.
- Fixture
- The sample data a test runs against. When the code changes shape and the fixture doesn't, the test can quietly stop touching the real path.
- Gate
- The check that must pass before work is released. A gate you only run when you remember is not a gate — wire it into the commit and prove it blocks.
- Vacuous pass
- "All of them are fine" is automatically true when there are none. Any check of that shape must first prove it found something.
- Bytecode cache
- A compiled copy the interpreter keeps to start faster. If it isn't invalidated, the suite can run code you have already edited away.