One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions

The core idea

Ask a model for one random animal once and you learn little. Ask the same model many times and count the answers. One model favors “elephant.” Another favors “axolotl.” Training data, tokenization, post-training, and decoding shape those stable biases.

The paper defines one task-language pair as a cell. It samples each cell at temperature 1, normalizes every short answer, and stores a categorical distribution. It then compares two distributions with base-2 Jensen–Shannon divergence (JSD), a symmetric distance for probability distributions first formalized by Jianhua Lin.

fingerprint(model) = { answer distribution for each task × language cell }
distance(A, B)     = mean JSD(distribution_A, distribution_B)

“One token” describes each visible answer, not the whole audit. The paper’s eight-cell operating point uses about 120 short-output queries. The collection code allows up to 16 completion tokens because some providers reject a lower cap and some one-word answers span several tokenizer tokens.

What the paper measured

The study sampled 165 models through OpenRouter with 10 fingerprint tasks in English, Russian, Chinese, and Arabic. It collected 326,047 responses for $34.44. The median cell entropy was 1.00 bit, and the median modal answer held 71% of samples. Same-model split halves had median cell JSD 0.075; different-model halves reached 0.489. Those gaps turn weak answers into useful aggregate evidence. The paper publishes the complete method and results.

Provider changes still move the signal. The paper reports median cross-provider distance 0.227 and cross-provider AUC 0.880, compared with 0.971 for the same provider mix. The result supports model verification as statistical evidence, not vendor attestation.

Why this method adds something new

Black-box model checks already existed. Model Equality Testing treats API monitoring as a two-sample test and found 11 of 31 commercial Llama endpoints differed from reference weights. LLMmap uses 3–8 crafted queries to identify 42 model versions with more than 95% accuracy in its setting.

Bruckner changes the measured object. He uses ordinary, low-entropy prompts and reads identity from repeated answer frequencies. This avoids long generations, logits, model-owner cooperation, and a fragile magic string.

Our first-principles design

We started with one job: measure the route used by the current AI Agent without asking the user for a key. Three facts remain after removing every optional part:

1. The probes must reach the route under test.
2. Each answer must come from a fresh generation.
3. A verdict needs a named reference distribution and a calibrated threshold.

Scanning environment variables fails the first test. A machine can hold many keys while the active Agent uses a host-managed account. Extracting a browser token adds risk and still says nothing about model identity. Starting a subagent adds another routing layer without improving the statistic.

The smallest useful adapter already exists: the official Codex CLI. OpenAI’s non-interactive mode documentation states that codex exec reuses saved CLI authentication. It also documents --ephemeral for runs without stored rollout files and --ignore-user-config for controlled automation.

one sample = codex exec --ephemeral --ignore-user-config → one short answer
20 samples = four cells × five independent Codex sessions → one quick scan

This path keeps authentication inside Codex. The scanner checks login status, launches clean child sessions, collects short answers, computes JSD locally, and deletes its temporary directory. It reads no auth file, cookie, browser state, API-key variable, or parent conversation.

Occam’s razor in the product

How we adapted the public data

The Zenodo dataset contains raw responses, normalized answers, distributions, split scores, divergence matrices, and verification results. The separate software archive contains the collection and analysis pipeline.

We streamed the published split scores and ranked each task-language cell by mean impostor JSD − mean genuine JSD. We selected eight high-gap cells, then built reference distributions from even repetitions. Our packaged library contains 161 models; four of the paper’s 165 verification models lacked five valid samples in both split halves for every selected cell.

We calibrated six product budgets on the same published split data. Quick uses 4×5 samples with threshold 0.643900 and EER 11.8%. Standard uses 8×5 with threshold 0.629197 and EER 8.1%. The 8×15 budget uses threshold 0.581200 and EER 6.7%.

The high-gap selection and thresholds use the same dataset. They provide a product operating point, not an independent accuracy claim. A future release should enroll new samples, hold out models, and test across dates and providers.

What “minimum cost” means in practice

The paper minimizes output-token cost. Our first release minimizes user work, credential risk, and host integration. Those budgets differ. A production English acceptance run used 20 fresh Codex sessions and returned 20 valid answers. Its median child latency was 27,093 ms and Codex reported 170,367 total tokens because every clean Agent process carries host instructions and tool context.

That result exposes the next optimization target: a host-native sampling API could preserve authentication while cutting Agent context. Until such an interface exists, the CLI path gives the broadest compatibility with no credential plumbing. We publish the measured token cost instead of calling the scan free.

Three verdicts keep the evidence honest

• CONSISTENT: the stable route matches an enrolled claimed reference below the calibrated JSD threshold.
• MISMATCH: the stable route has enough valid samples and exceeds that threshold.
• INCONCLUSIVE: the model lacks a reference, the route changes, samples fail, or no calibration fits the budget.

Our fresh production run observed openai/gpt-5.6-sol. The v0.2 fingerprint library does not contain that model, so the scanner returned INCONCLUSIVE. It also showed openai/gpt-5.5 as the nearest reference. It did not turn similarity into identity.

Limits that matter

• Model updates change behavior and require new enrollment.
• Provider and decoding differences shift distributions.
• Response-level caching can collapse a random distribution; low variance plus low latency should trigger an anomaly.
• Codex exposes no CLI temperature override. Its Responses transport returns temperature 1.0 when the field is omitted, so our report labels this a host default.
• Behavioral verification raises the cost of substitution. It does not provide cryptographic proof or vendor attestation.

Primary sources and further reading

1. Tomas Bruckner, One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions, arXiv:2607.10252v1, 2026.
2. Single-token output distributions as behavioral fingerprints of large language models — dataset, Zenodo, DOI 10.5281/zenodo.21278557.
3. Single-token output distributions as behavioral fingerprints of large language models — software, Zenodo, DOI 10.5281/zenodo.21278793.
4. Irena Gao, Percy Liang, and Carlos Guestrin, Model Equality Testing: Which Model is this API Serving?, ICLR 2025.
5. Dario Pasquini, Evgenios M. Kornaropoulos, and Giuseppe Ateniese, LLMmap: Fingerprinting for Large Language Models, USENIX Security 2025.
6. Jianhua Lin, Divergence Measures Based on the Shannon Entropy, IEEE Transactions on Information Theory 37(1), 1991.
7. OpenAI, Codex non-interactive mode and Responses API reference.