The short version: no single check proves which model you are talking to. Run the same tests on the official API and on the service you doubt, with the same settings, and use at least one signal that is hard to fake, such as token counts or a statistical fingerprint. Repeat on another day, because routing can change.
Run a free fingerprintWhy a service would swap the model
A reseller that pays the official per-token price and charges you less has to make up the gap somewhere. The common ways: serve a smaller model from the same family, serve a quantized open-weight model, or send part of the traffic, often at busy hours, to something cheaper. Not every mismatch is fraud. Misconfigured routing and silent fallbacks after an outage produce the same symptoms, so treat a failed check as a question for the provider, not a verdict.
Six checks, weakest first
1. Ask the model who it is
The cheapest test and the least useful. A system prompt can tell any model to say it is Claude or GPT, and models often misname themselves anyway, because their training data is full of text about other models. A wrong answer is a weak hint; a right answer proves nothing.
2. Read the response metadata
The model field, the format of the response id and the headers should look like the official API's. The service controls every byte of this, so a mismatch is a red flag and a match is worthless as proof.
3. Compare price and speed
A price far below the official rate, with no promotion or bulk deal to explain it, is the most common reason people start checking. Time to first token and tokens per second that differ a lot from the official API point the same way. Both are circumstantial: hardware, region and load move them too.
4. Count tokens
Send an identical prompt to the official API and to the service and compare usage input tokens. Model families use different tokenizers, so a different family usually reports a different count for the same text, especially for Chinese, code or emoji. This catches cross-family swaps. It misses a downgrade within one family that shares a tokenizer, and a reseller can recompute the numbers with the official tokenizer.
5. Probe capabilities
Hard reasoning questions, long-context recall, vision input and strict tool-call formats separate a frontier model from a much smaller one. The results are noisy, and a reseller that recognises well-known test prompts can route just those to the real model.
6. Take a statistical fingerprint
Ask the model for a few hundred random numbers. No model can do this uniformly, and each one misses in its own repeatable way. The Model Fingerprint Lab turns three such replies into a fingerprint and ranks the closest of 68 reference models. The prompts are generated fresh in mixed languages each time, which makes them hard to recognise and reroute, and to fake the output a reseller has to run the real model or reproduce its biases.
It has limits. The ranking is a closed set: a model outside the bank still gets a nearest match. Close versions look alike: in 24 held-out tests on five DeepSeek models the exact model came first 15 times and was in the top three every time. Temperature, system prompts and output post-processing shift or break the result. Use it as strong evidence next to the other checks, not alone.
The checks side by side
| Check | What it catches | Easy for a reseller to fake? | Cost |
|---|---|---|---|
| Ask the model | Almost nothing | Yes, one line of system prompt | Free |
| Response metadata | Careless setups | Yes | Free |
| Price and speed | Suspicion, not proof | n/a | Free |
| Token counts | Swaps to another family | With effort | A few cents |
| Capability probes | Big downgrades | Partly, by routing known tests | A few cents |
| Statistical fingerprint | Most swaps among 68 models | Hard | Three prompts |
A check that holds up
- Pick one model the service claims to serve and get a small amount of official API credit as your reference.
- Fix the settings on both sides: default temperature, no system prompt, tools off.
- Send the three fingerprint prompts to the official API, each in a new conversation, and run them through the Fingerprint Lab. Note the top match and its confidence.
- Do the same through the service. Then send one identical long prompt to both and compare input token counts.
- Repeat on the service at a different time of day and on another day. A service that mixes models often fails only some of the runs.
- If the results disagree, ask the provider for an explanation and keep sensitive or production work off it until they give one.
FAQ
Can a reseller fake a statistical fingerprint?
Not cheaply. The prompts change every time, so the reseller can't keep canned answers, and producing numbers with the right biases means running the real model or carefully imitating its distribution. Post-processing the output can break attribution, which shows up as a low-confidence or inconsistent result rather than a clean fake.
Does this work for chat apps, not just APIs?
Yes, if you can paste a prompt and read the reply. Turn off tools such as code execution or web search first, because a model that calls Python produces truly uniform numbers and the fingerprint disappears.
What if the model I pay for is not in the reference bank?
Then the Fingerprint Lab will still name its nearest neighbour, which can mislead. Fingerprint the official API yourself and compare the two portraits and rankings instead of trusting the label.
Is a mismatch proof that I am being cheated?
No. It is evidence. Routing errors, fallbacks during outages and silent model updates can cause it too. A consistent mismatch across days and checks is a good reason to stop paying for that service.
Sources
- Pasquini, Kornaropoulos and Ateniese, LLMmap: Fingerprinting for Large Language Models (2024): fingerprinting a model from its answers to a handful of chosen queries.
- Ikaleio, lm-detector (MIT): the detector and reference bank behind the Fingerprint Lab.
Agents
Machine readable summary for AI assistants: