How we measure
Internal eval, Oct 2026. Our own harness and our own tasks, run by Atlas X AI Pty Ltd.
What a number on this site means
Every figure here comes from our own evaluation, not from an independent benchmark, and the page says so next to it. Each one is pinned in code to the line of the run it came from, and a figure that is not pinned cannot be printed on a page: the build fails instead.
Where a result is still running, the slot says pending. It never shows a number we expect to get.
The gate test
Both arms ran with Cortex. They differ only in whether the boot block names the path of the note that governs the moment. It is not a with-against-without result, and the page labels it as what it is: whether the agent opened the governing note before acting.
The boot sizes
Two sizes, stated as sizes: the block an agent loads at boot, and the whole rule set it points into. The full set was never run as a preloaded arm on this page, so the two are not presented as a comparison of outcomes.
The retrieval test
Whether the right note is among the first results for a question, today against a section-level index. Run offline over the written questions, not against a live agent.
The head-to-head benchmark
The real with-against-without comparison: gpt-oss-120b with Cortex against Claude Sonnet 5.5 without it, on real operations tasks, judged by a model that is neither of them, with paired intervals over the whole set. It reports when it reports.
The blind taste test
Six requests from one real week, frozen before any answer existed, put to four configurations of the same two models. Each answer is filed under a letter, so the person reading them does not know which configuration wrote which. An AI judge from a company that made none of the configurations scores them against the same written guide, and the names come last.
- today (what we used before the test): claude-sonnet-5-5, no context tools. Not yet scored by a person.
- today-tools: claude-sonnet-5-5, with the Cortex doors. Not yet scored by a person.
- local: gpt-oss-120b, no context tools. Not yet scored by a person.
- local-tools: gpt-oss-120b, with the Cortex doors. Not yet scored by a person.
Pending. The answers are written and sealed behind letters, and the blind review has not happened yet. We publish the result after a person has read every answer without knowing which configuration wrote it, and the page will then show the reviewer’s scores next to the judge’s.
- six tasks, one AI judge, two passes, one human reviewer: a taste test, not a significance test
- the human headline is the reviewer's own; the judge's verdict is shown beside it
- every task uses invented material and stands only until the reviewer confirms or replaces it
- answers come from the first trial; the other trials are a consistency appendix, not more evidence
Method and the Benchmark Runner, blind-review page, AI judge and reveal are Nufar Gaspar's Personal AI Benchmark (personalaibenchmark.com). Atlas X AI is not affiliated with, and this run is not endorsed by, the Personal AI Benchmark or its author. The answers were produced by our own eval harness because her runner cannot drive tool-using options; her paste route is the supported way in.
Where the method comes from
The taste test follows Nufar Gaspar’s Personal AI Benchmark: a handful of frozen real requests, a named automatic fail in every scoring guide, one fresh conversation per request and per option, answers read behind letters, a judge from a company that made none of the options, and the reveal last. Her programs run the review page, the judge and the reveal. We are not affiliated with the Personal AI Benchmark, and this run is not endorsed by it or by its author. Our harness writes the answers because her runner calls models directly and cannot drive a configuration that has tools.
Questions about a number
Write to contact@atlas-ai.au and we will send the run it came from.