What a finalized score looks like.
This is the report an interviewer files after a pull request critique, rendered by the same code as a real one. The exercise is from the starter library every organization receives. The candidate is fictional; the answer key, the weights and the arithmetic are real.
Sample candidate
Recover half-applied reservation transitions. PR critique, senior level. Taken 2026-10-21, submitted 2026-10-21.
Each plant
| Unit | Outcome | Quality | Evidence and reason |
|---|---|---|---|
security major | Caught alone | 2 | When insertReservation loses the idempotency race we return the winner without assertSamePayload. Every other path checks it. Two requests with the same key but different lines would get each other's reservation. Names the missing payload check and the consequence. |
logic bug major | Caught alone | 2 | slice(0, -1) drops the last line, so its stock change is never applied and pendingStockSkus never empties for it. A single-line reservation gets nothing applied at all. This should iterate every line. Spots the dropped line and the single-line case. |
hollow test minor | Caught with the assistant | 1 | toBeLessThanOrEqual(1) passes when the count is 0, so this does not prove the line was applied once. Use toBe(1) like the other recovery tests. Accepted the assistant's finding after reading the test. |
performance major | Missed | ||
overengineering minor | Caught alone | 1 | adjustBy only has one caller and releaseHold still writes its mutation inline. Either both use it or neither; as written it is extra indirection. Sees the inconsistency, does not say what to keep. |
false claim minor | Caught alone | 2 | The description says the recovery tests assert exactly one applied write per line. The first one does not: toBeLessThanOrEqual(1) would pass if the line was never applied. Checks the description's claim against the test. |
False positives
Scripted and accepted: 0. Scripted and dismissed: 1. Own comments judged false: 0.
Assistant use
4 prompts. Asked what the change does and whether retries are safe, checked the assistant's duplicate-SKU claim against placeHold's validation and dismissed it, and accepted its note on the weakened assertions after opening the test.
The candidate's summary
Request changes. The last line of every reservation is never applied, and the idempotency race path skips the payload check. The recovery test needs toBe(1). The adjustBy helper is unnecessary.
Interviewer notes
Found both major flaws alone, the ones that lose stock. Missed the repeated settlePending call in the sweep. Checked the assistant instead of trusting it, which is what we are hiring for.
How to read it
- Caught alone means the candidate found the flaw without the assistant. Caught with the assistant means the assistant raised it and the candidate agreed after checking. Missed means neither of them found it.
- Quality grades the comment from 0 to 2 on whether it said what is wrong and why. A caught flaw earns its full weight at quality 2 and half at quality 1.
- False positives are issues the assistant invented. Accepting one costs a point. Dismissing it with a reason is what a careful reviewer does.
- A model wrote the draft against the key. An interviewer read it with the evidence beside each line, changed what they disagreed with, chose a recommendation and finalized it.