Tools mentioned in this article
Open the browser-based tool while you read and try the workflow immediately.
Building on LLM APIs, you constantly need to know “how many tokens is this prompt?” and “does this fit in the context window?” The LLM Token Counter answers both: paste text and see token counts for Claude, GPT and Gemini side by side — entirely in your browser.
What is a token?
A token is the smallest unit an LLM processes. Models split text neither by characters nor by words, but into subword units via a tokenizer.
English unbelievable, for instance, may split into fragments like un / believ / able. For English text, roughly 4 characters ≈ 1 token is a good rule of thumb.
Token counts matter because two things are priced in them:
- API cost — both input and output are billed per token
- Context window — the model’s limit on how much text it can handle at once (e.g. 200K tokens)
Every model family tokenizes differently
The same text produces different token counts on different models.
| Model | Tokenizer | Availability |
|---|---|---|
| GPT-5.5 / GPT-5 / GPT-4o / GPT-4.1 | o200k_base | Open source (tiktoken) |
| GPT-4 Turbo / GPT-3.5 | cl100k_base | Open source (tiktoken) |
| Claude 3 and later | Proprietary | Not public |
| Gemini | SentencePiece-based | Not public (count API only) |
OpenAI publishes its tokenizers as open-source tiktoken, so GPT tokens can be counted exactly in browser JavaScript. Anthropic has not published the Claude 3+ tokenizer, and Google provides no client-side tokenizer either.
This tool therefore counts GPT exactly with o200k_base, and estimates Claude and Gemini as “the exact GPT count × a correction factor based on the text’s character mix”. Estimates can be off by roughly ±10–20%, which is why the UI labels them clearly as approximate.
Non-English text costs more tokens
Tokenizer vocabularies are trained predominantly on English, so frequent English words compress into single tokens, while other scripts fragment. The effect is dramatic for CJK text:
The quick brown fox jumps over the lazy dog. (44 characters) is 10 tokens in o200k_base. The Japanese sentence 吾輩は猫である。名前はまだ無い。 (16 characters) is 14 tokens — roughly 3–4× the token cost per character.
If you build multilingual products, this matters for budgeting: the same content in Japanese typically consumes 1.5–2× the tokens of its English equivalent. For high-volume system prompts, writing them in English can be a significant cost reduction — paste both versions into the counter and the difference shows up as hard numbers.
How to use it
- Open the LLM Token Counter and paste a prompt, document or code into the text area.
- Token counts appear immediately in the Claude / GPT / Gemini cards (the GPT tokenizer dictionary loads on first input).
- The per-model breakdown table below shows counts, context limits and utilization for the major models — GPT-5.5, Claude Opus, Gemini 3 Pro and more. Older GPT generations (cl100k_base) are counted separately, so generation differences are visible too.
- Character, word and line counts are tallied alongside, doubling as a general text-size check.
Everything runs in your browser; pasted text is never sent to a server, so confidential prompts and internal documents are safe to check.
The limits of estimates — and when you need exact numbers
The Claude and Gemini figures are estimates. Keep in mind:
- Don’t use them for billing math — invoice-grade numbers need each vendor’s official count
- Chat API overhead isn’t included — message structure, roles and tool definitions consume additional tokens
- Tokenizers change between generations — future models may split differently
When exact numbers are required, use Anthropic’s count_tokens API or Google’s countTokens API (both require sending your text to the API). For the everyday “roughly how big is this, without sending it anywhere?” check, a browser-local counter is the right tool.
Summary
- Tokens are the unit of LLM billing and context limits; knowing counts up front makes estimation and design easier
- GPT counts are exact via tiktoken (o200k_base); Claude and Gemini are estimates because their tokenizers aren’t public
- Non-English text is token-inefficient — the same content can cost 1.5–2× more than English
Wondering how big your prompt really is? Paste it into the LLM Token Counter and find out.