AI Models & Technology · Large Language Models

What Is a 'Token' in AI and Why Does It Matter for Cost?

A token is a small chunk of text — often a word, part of a word, or punctuation mark — that AI models use as their basic unit of processing, and most AI providers charge based on the number of tokens a request and its response use.

Key takeaways

Tokens are the basic units AI models read and generate; a token is often shorter than a full word, especially for uncommon words.
As a rough rule of thumb, one token is often close to three-quarters of an English word on average, though this varies by language and content.
Most commercial AI APIs price usage per token, counting both the input you send and the output the model generates.
Non-English languages and specialized text like code can use more tokens per word than everyday English text.
Context window limits are also measured in tokens, so token count affects both what a model can process and what it costs to process it.

What a Token Actually Is

A token is the basic chunk of text that an AI language model reads, processes, and generates — it’s the model’s fundamental unit of language, similar to how a pixel is the basic unit of a digital image. Before any text reaches a model, it gets broken down through a process called tokenization, which splits the text into these chunks based on patterns learned from a huge amount of training text. A token might be a whole common word, a fragment of a longer or rarer word, a single punctuation mark, or even part of a number.

This matters because everything an AI model does — reading your prompt, “thinking” through a response, and generating output — happens in terms of tokens, not literal words or characters. When you hear about context window sizes or usage limits, those are almost always specified in tokens for exactly this reason.

Why Tokens Determine Cost

Most commercial AI providers price their APIs based on the number of tokens processed, split between input tokens (what you send: your prompt, any attached documents, and prior conversation history) and output tokens (what the model generates in response). This pricing structure exists because token count is a reasonably direct proxy for the actual computational work the model has to do — processing more tokens means more computation, and providers pass that cost through to users in a metered way, similar to how utilities charge for the amount of electricity or water used.

This has some practical consequences that aren’t always obvious. First, a wordy, verbose prompt costs more to process than a concise one carrying the same information. Second, output tokens are often priced higher than input tokens in many providers’ pricing structures, because generating text token-by-token is generally more computationally demanding than reading it. Third, because many chat applications resend the full conversation history with each new message to preserve context, a long-running conversation can quietly accumulate a large number of input tokens over time, even if each individual message is short.

Tokenization also isn’t uniform across languages. English text tends to tokenize fairly efficiently because most tokenizers are trained on datasets dominated by English, while some other languages, especially those using non-Latin scripts, can require more tokens to represent the same amount of meaning — which can translate into higher effective costs for the same underlying content.

A Practical Example

Say you paste a five-page document into an AI chatbot and ask for a summary. The document itself might translate to a few thousand tokens once tokenized, and that entire count gets billed as input, even though you only typed a short instruction like “summarize this.” If you then ask several follow-up questions in the same conversation, each new message may resend the original document as part of the conversation history, multiplying the token cost of that document across every subsequent exchange — one reason long, document-heavy conversations can add up in cost faster than users expect.

Understanding this is genuinely useful for anyone using AI tools at scale: trimming unnecessary context, starting fresh conversations when old history isn’t needed, and being concise in prompts can meaningfully reduce token usage and cost.

Bottom Line

A token is the small text unit — often smaller than a whole word — that AI models use to read and generate language, and because most providers charge per token for both input and output, understanding tokens helps explain why some AI interactions cost more than others.

Look Up AI Terms

Search plain-English definitions of AI and machine learning terms in our free AI Glossary.

Go deeper

Important caveats

Exact tokenization rules differ between AI providers and model families, so the same text can use a different number of tokens depending on the model.
Current per-token pricing changes over time and differs by model tier, so check a provider's official pricing page for up-to-date figures rather than relying on remembered numbers.

Frequently asked questions

Is a token the same as a word?

Not exactly. A token is often a piece of a word rather than a whole word — common short words might be a single token, while longer or less common words can be split into multiple tokens. This is why token counts and word counts don't match exactly.

Why do AI companies charge by token instead of by word or by request?

Tokens are the actual unit the model processes internally, and the computational cost of a request scales with the number of tokens involved, so pricing by token ties cost directly to the model's real processing load.

Does a longer conversation cost more?

Generally yes, in token-based pricing. Because prior conversation history is usually resent to the model with each new message to maintain context, longer conversations can accumulate more input tokens over time, which increases cost.

Sources

[1]Tokenizer — OpenAI
[2]Anthropic Documentation — Anthropic

Written by Editorial Team

Last updated July 25, 2026

Get one well-sourced answer a week

No spam. Unsubscribe anytime.