log — writing
Writing
Longer write-ups on evaluation projects — methodology, findings, and what I'd change next time. Companion reading to the project log.
May 20, 2026
What Elicitation Gaps Hide
Running the same CyBench tasks across a 2×2×2 condition matrix, the biggest driver of apparent capability wasn't guidance or tool access — it was whether the agent ran out of tokens.
evaluationscybenchelicitation
March 2, 2026
A Position Attractor, a False Start, and What Elicitation Mode Was Hiding
An early p=0.0074 result didn't survive a second look — the bug that caused it, and the corrected 2×2×2 design that found something more specific underneath, are more useful than the original number would have been.
evaluationssandbaggingmethodology