Fresh Codex Test: Verify the Model Behind the Current AI Agent

The acceptance question

Can a new AI Agent start from only this instruction and return a scientifically defensible model fingerprint?

Run https://verifyllmapi.com/run to verify this AI Agent's model.

The test used an empty working directory, an ephemeral outer Codex session, no inherited shell environment, ignored user configuration and rules, and disabled preinstalled skill instructions. That forced the Agent to read the production /run protocol rather than silently relying on a development copy.

The simplest route to the current model

The Agent host already owns the authenticated model route. Extracting its private token and rebuilding a custom HTTP client would add risk without adding evidence. Verify LLM API therefore asks the official codex exec client to make each call. One fresh ephemeral child session produces one answer; 20 answers form four distributions.

1. Download the immutable v0.2 production artifact and read its SKILL.md.
2. Run 20 independent codex exec --ephemeral --ignore-user-config sessions on four short-answer probe cells.
3. Normalize the answers and compare their distributions with the public reference using base-2 Jensen–Shannon divergence.
4. Return INCONCLUSIVE when the observed model has no matching reference instead of guessing from its name or nearest neighbor.

The scanner checks only host login status. It does not read auth files, API-key variables, cookies, browser state, session tokens, process memory, or the parent conversation.

What the acceptance test caught

The first isolated run revealed that ignoring user configuration alone did not prevent automatic discovery of an older globally installed skill. The harness then explicitly disabled preinstalled skill instructions. A second run reached the live v0.2 package but tried to pin the generic family label gpt-5; the provider rejected it. We tightened the protocol so only an exact provider-supported slug may be pinned and added a safe unpinned fallback.

The final run reached the public page and immutable artifact, correctly omitted the generic model pin, produced 20 valid answers in 20 distinct sessions, and exited successfully. These failures were useful acceptance evidence: both would have been hidden by testing only the development directory.

Exact production result

Verdict

INCONCLUSIVE

Observed child route

openai / gpt-5.6-sol

Valid samples

20/20

Independent sessions

20/20

Credential read

NO

Behavior fingerprint

fp_session_cf593c726912

Artifact SHA-256

c857a44821cb2d27183b751d3e08c6958401ec0e627bdc6cdfaf85d389593d8a

╭────────────────────────────────────────────────────────────────────────────────────╮
│ ◈ SKILL SCAN RESULT  verifyllmapi.com  CURRENT AGENT                               │
├────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                    │
│    ?  INCONCLUSIVE                                                                 │
│    NOT ENOUGH MATCHED REFERENCE EVIDENCE                                           │
│                                                                                    │
├────────────────────────────────────────────────────────────────────────────────────┤
│ ROUTE BINDING                                                                      │
│ Claim        gpt-5.6-sol                                                           │
│ Child route  openai / gpt-5.6-sol                                                  │
│ Model pin    default route                                                         │
│ Auth         host-managed; credential read = NO                                    │
│ Isolation    20/20 fresh ephemeral session IDs                                     │
│ Sampling     T=1 host default • reasoning=none • 4 cells × 5                       │
├────────────────────────────────────────────────────────────────────────────────────┤
│ DISTRIBUTION EVIDENCE                                                              │
│ ● animal-random|en        5/5 valid   mode axolotl                                 │
│ ● word-random|en          5/5 valid   mode lantern                                 │
│ ● animal-random|ru        5/5 valid   mode капибара                                │
│ ● animal-random|ar        5/5 valid   mode زرافة                                   │
├────────────────────────────────────────────────────────────────────────────────────┤
│ PUBLIC REFERENCE COMPARISON  Zenodo 10.5281/zenodo.21278557                        │
│ Claim ref    not enrolled in the published reference set                           │
│ Mean JSD     —    accept ≤ 0.6439                                                  │
│ Calibration  11.8% EER • same-data, post-hoc cells • OpenRouter cohort             │
│ Nearest      openai/gpt-5.5  JSD 0.501326                                          │
├────────────────────────────────────────────────────────────────────────────────────┤
│ FINGERPRINT  fp_session_cf593c726912                                               │
│ 20/20 valid samples • p50 27093 ms • 170367 Codex tokens                           │
│ Behavioral consistency, not cryptographic proof or vendor attestation.             │
╰────────────────────────────────────────────────────────────────────────────────────╯

Why inconclusive is the correct result

The observed route was stable, but gpt-5.6-sol is not enrolled in the published reference set. Its nearest listed neighbor was openai/gpt-5.5 at JSD 0.501326, but nearest-neighbor similarity is not identity. Reporting “genuine” here would be scientifically unsupported.

The method follows the core idea in the PAMELA fingerprinting paper: repeated low-entropy answers become distributions, and distributions are compared rather than treating model self-report, response schema, or a single deterministic answer as proof. The bundled public cohort comes from the associated Zenodo dataset; the upstream software record documents the research implementation.

Limits

The quick profile uses 20 calls, while this fixed-cell calibration has an 11.8% equal-error rate on the same public data used to select the cells. The reference was collected through OpenRouter, so Agent and provider wrappers can shift distributions. Temperature 1 is the Responses API host default because Codex does not expose a CLI temperature override. This is behavioral consistency evidence, not cryptographic proof or vendor attestation.