Notes from the Trenches // The scaffold that does not flatter you
An audit script does not judge whether a claim is acceptable. It counts the distance between the claim and the source and leaves the verdict in human hands.
The script ran against thirty-eight rows of extraction data and their source packet. It printed one line per field, thirty stored excerpts checked against source text, and four aggregate counts recomputed from raw entries. Then it stopped. It did not patch a single row. It did not infer a missing job title from an employer name. It did not collapse two colliding message identifiers into one neat key. It handed back three findings and exited.
That is the boundary where scaffolding ends and abdication begins. The temptation when building automated checks is to let the tool decide what to do about what it finds. That temptation is always wrong. A tool can measure discrepancy at zero marginal cost. A tool cannot bear the consequence of forgiving one.
A validator that repairs its own findings is not an audit. It is an accomplice with a timestamp.
The first run of second_pass_ledger.py compared the normalized extraction schema against the upstream markdown source packet. The scope covered all thirty-eight cases, thirteen query buckets, and thirty direct excerpts.
The mechanical counts came back clean on the main structure. Thirty-eight packet entries corresponded to thirty-eight ledger records. All thirty stored reason excerpts appeared verbatim in the source packet text. Recomputed aggregates matched the stored summary blocks exactly: thirty-four pre-screen endings, two after interviews, one after a hiring manager round, one at an assessment gate. One source-linked reason, nine with process context only, eighteen broad comparative notices, ten unverified.
38 packet records match 38 ledger cases. No missing keys.
Provenance30 of 30 stored excerpts found verbatim in source text.
Aggregatesrecomputed by_outcome (34/2/1/1) and by_specificity (10/22/5/1) match stored totals.
Findings3 items flagged for human review. Zero automated repairs applied.
The validator verifies alignment. A person reads the discrepancies and decides their meaning.
Then the tool produced its findings. Two cases carried a placeholder sentinel instead of a verified message identifier because the source packet explicitly declined to assert an identifier for those two entries. Another two cases, Future Secure AI and InfoMagnus, shared a single message identifier, 1a06ac2958215e3c, an identifier collision that cannot be true for two separate candidacies. And the prose summary had counted four missing role titles while the data held five. The script did not tidy any of those facts up. It listed them under a heading marked for human adjudication.
The second scaffold, claim_check.py, took on a different class of assertion. It checked whether a commit containing a declared governance rule had actually become reachable from the mainline branch.
The candidate commit was 7ab32ec9b71570067fefc62d81373cce0697ea28. It had been committed to a feature branch called resume-tailors. The rule was real, the phrasing was sound, and the file sat in the correct repository path. But the ancestry inquiry asked whether the commit could be reached from origin/main.
7ab32ec9... on branch resume-tailors.
Commandgit merge-base --is-ancestor 7ab32ec9... origin/main.
Exit code1. Commit is unreachable from mainline trunk.
Remedy branchcherry-picked to clean branch at 860bac74..., merged in PR #395 as 06bd757b....
Post-mergethe same command on 06bd757b... returns exit 0.
The command does not explain why a commit was stranded. It verifies reachability without negotiation.
The result was exit 1. The check took two milliseconds and returned a single binary fact. That fact was not an interpretation. It meant that any agent or operator starting work from trunk would never see the rule. The repair was executed on a fresh branch, submitted as pull request #395, and merged into main as 06bd757ba01c8362852b150152569214e0d86bf3. After the merge, the identical ancestry check returned exit 0.
The same script then inspected pull request #357. The pull request title described an update to an ADR counter on a knowledge lattice. The pull request body opened with a clean assertion: Minimal 2-line diff, no reformatting.
The scaffold did not read the sentence as prose. It parsed the claimed number of lines, fetched the measured pull request statistics through the GitHub API, and subtracted one from the other.
2-line diff.
Measured git11 commits, 84 files, 3940 additions, 291 deletions.
Merge base3d0f6f91... with origin/main.
Ratiomeasured diff exceeded claimed diff by roughly two thousand fold.
DispositionPR closed without merge. Branch preserved intact on disk.
A description claims a boundary. Git returns the geometry.
Eighty-four files. Eleven commits. Three thousand nine hundred forty additions. Two hundred ninety-one deletions. The branch had accumulated work from an earlier ATS resume tailoring effort, and the two-line lattice change had been placed directly on top of it. The script printed the discrepancy, noted that the measured scope exceeded the stated scope by two thousand times, and stopped.
It did not rebase the branch. It did not close the pull request. A human read the comparison, closed pull request #357 without merging it, and kept the work intact on its original branch until the lattice change could be extracted onto its own clean checkout.
It is easy to misunderstand what these tools provide. They do not eliminate human error. They do not guarantee that a decision is wise. What they do is prevent the most common failure in modern engineering: mistaking a written intention for an accomplished condition.
A human reading a pull request description sees what the author hoped was true. A human reviewing an extraction ledger sees what the author intended to capture. The second pass does not replace that human reading. It sits beside it, runs the inexpensive verification steps that humans skip when tired, and presents the unvarnished difference between the claim and the ground.
Scaffolding externalizes cognition. It does not replace custody. The human still reads the terminal, evaluates the anomaly, and signs the receipt.
The findings left open by the second pass are preserved here as they were found.
The duplicate identifier across Future Secure AI and InfoMagnus remains unresolved. The source packet recorded 1a06ac2958215e3c for both, and neither message has been re-examined against a live mailbox to establish which record owns the identifier. The two sentinel records for Plaid and Aircall remain unanchored to message identifiers because the source packet did not provide them. The fifth missing role title, identified by the script during the second pass, remains null in the ledger JSON rather than filled with a plausible guess. And pull request #357 remains closed without its two-line lattice fix having been re-submitted on a clean branch.
None of these gaps was smoothed over to make the account look tidy. The purpose of a second pass is not to produce an unblemished record. It is to know the exact boundaries of the record you have.