[ J / C / K ] DECISION MODEL LAB

CONSOLIDATED EVALUATION / SEPTEMBER 2026

The action arrives.
Does the agent survive?

Five original 60-second T-Rex courses per model and condition. Compare survival, deaths, arrival times, and reliance on the safety shield.

JEV / HOSTED BASELINECLM / PUBLIC 8B HEADKEV / PUBLIC 4B CHECKPOINT
JEV BASELINE
TABLE VALUES
HIGHLIGHT KEY

BEST Best observed value. Ties included.

Jev: September 26. CLM: September 24. Kev: September 26.

VS JEV: accuracy and recall use percentage-point differences; F1 uses F1-point differences; latency uses a ratio to Jev; counts use an absolute difference.

[ 01 / MODEL ALONE ]

Apply the model action.

Unshielded outcomes

Loading frozen comparisons...

[ 02 / SHIELDED SYSTEM ]

Let the planner intervene.

Shielded outcomes

Loading frozen comparisons...

[ 03 / PER COURSE ]

A small sample, fully separated.

[ 04 / INTERPRETATION ]

Survival belongs to the whole system.

Without the shield, CLM and Kev each survived 0/5 courses. Jev survived 1/5 in its original run and 2/5 in its latest run. With the deterministic shield, all three survived 5/5. Shielded CLM needed 5,259 interventions and Kev needed 3,536; Jev needed 20 or 27 depending on the selected run.

The same original course, seeds 0-4, 60 seconds, labeled prompt, and six requests in flight were used. Actions and answer timing change future game states, so these live prompts are not identical across models. Planner agreement and response speed are diagnostics; neither guarantees survival.

This is a constrained direct-action simulation with safety labels already supplied by a physics planner, not an open-ended agent test. The shield can replace unsafe answers or act before an answer arrives. Five seeds are a small sample, and shielded survival evaluates a model-and-planner system.

The latest Jev unshielded run recorded three read timeouts. Its shielded run and the other selected game runs recorded no API errors. Shorter exploratory pilot courses are excluded from this consolidated comparison.