What does 'tokens' mean when you're being billed for an AI API

What a Token Actually Is

A token is a chunk of text — often roughly a word or part of a longer word — that an AI model breaks text into as its basic unit of processing, rather than processing raw individual characters or whole sentences directly; a rough rule of thumb is that a token is somewhat shorter than an average word.

How Tokens Translate Into a Bill

API billing for most providers is based on the total number of tokens processed — both the tokens in the prompt sent to the model and the tokens the model generates in its response — rather than a simpler measure like total characters, number of requests, or a flat per-use fee.

Why Input and Output Are Often Priced Differently

Many providers charge different rates for input tokens versus output tokens, since generating new text is generally more computationally expensive for the model than processing text it’s given to read — this distinction matters for estimating real costs of an application that generates long responses versus one that mostly processes existing text.

Why This Matters for Building an Application

Because both longer prompts and longer generated responses directly increase token usage, an application processing large documents or generating long-form output will accumulate cost differently than one handling short, simple exchanges — worth estimating before assuming a rough cost based only on number of requests.

Bottom Line

Tokens are the basic text-processing unit AI models use, and API costs are calculated based on how many tokens are processed in both directions — understanding this is necessary for realistically estimating what a given application will actually cost to run.

Count Your Tokens

See an instant token, word, and character count for any text with our free Token Counter — no signup required.

Go deeper

• What is speculative decoding and how does it speed up ai responses?
• What Is Claude’s Context Window and Why Does It Matter?