Mastery practice and the context-window bridge

A teammate says, “Our form has 300 words, so it must be 300 tokens.” What should you correct?

Answer: Words and tokens are different units. A tokenizer may keep a common word whole, split an uncommon word into subwords, attach spaces to pieces, and represent punctuation, digits, emoji, or byte sequences separately. The exact count depends on the tokenizer and exact text. A word count can support rough human planning, but it cannot enforce a model limit.

Now consider reset, resetting, RESET!, and reset_password_v2. Do not guess their exact token IDs. Instead, predict why they may differ: leading space, inflection, capitalization, punctuation, underscore, and digits all change the character sequence. Then identify the only reliable next step for a hard limit: count each exact string with the target model’s tokenizer or provider counter.

This question tests the central mental model. Tokens are vocabulary entries selected by a segmentation process. They are not little boxes of fixed semantic size.

An illustrative API rate is $0.30 per million input tokens and $1.20 per million output tokens. A workflow handles 400,000 successful tasks. Each task uses 2,500 input and 500 output tokens across all its model calls.

Per-task input cost:

2,500 / 1,000,000 × $0.30 = $0.00075

Per-task output cost:

500 / 1,000,000 × $1.20 = $0.00060

Per-task total and workload total:

$0.00075 + $0.00060 = $0.00135
400,000 × $0.00135 = $540

If 8% of tasks retry with an additional 1,500 input and 300 output tokens, calculate that separately. The extra retry costs:

1,500 / 1,000,000 × $0.30
+ 300 / 1,000,000 × $1.20
= $0.00081

There are 400,000 × 8% = 32,000 retries, so the added cost is $25.92. Total illustrative workload cost is $565.92. In a real estimate, verify current rates, cache categories, failed calls, and every model step.

A team has a documented 24,000-token combined allowance. Its plan sends 2,000 tokens of instructions, 5,000 of history, 16,000 of retrieved passages, and allows a 3,000-token answer.

The arithmetic is:

2,000 + 5,000 + 16,000 + 3,000 = 26,000

It exceeds the allowance before accounting for the current user message, tool schemas, wrappers, or headroom. A good repair begins with a complete inventory. Preserve governing instructions and the current task. Rerank and deduplicate retrieval, summarize old history while retaining decisions and exact values, and reserve realistic output plus safety margin.

One possible plan is 1,800 instructions and wrappers, 2,500 recent or summarized history, 500 current request, 1,200 tools, 14,000 selected evidence, 2,000 output, and 2,000 headroom: exactly 24,000. This is a teaching allocation, not a universal recommendation. Measure actual components and follow the target model’s input/output rules.

If all 16,000 retrieval tokens are necessary, route the task differently: process documents in stages, retrieve per subquestion, or use a model with suitable capacity. Silent truncation is not a plan.

You must reduce a 6,000-token claims packet by half. Which changes are safest?

1. Delete the first 3,000 tokens.
2. Remove duplicate headers and quoted email copies.
3. Replace exact policy conditions with “claims are usually covered.”
4. Project a database result to fields used by the decision.
5. Summarize conflicting dates as “there was a delay.”
6. Build a sourced timeline that preserves both dates and their origins.

Answer: Start with 2 and 4 because they remove deterministic noise while preserving required facts. Use 6 when verified against the source. Reject 1 because position does not indicate relevance. Reject 3 because “usually” can erase exclusions. Reject 5 because the conflict may determine the result.

After compression, recount with the target tokenizer and run task-level tests. Compare answers from full and compact packets on ordinary and edge cases. The objective is not the smallest prompt; it is the smallest packet that still supports a correct, auditable answer.

Tokens are the units of the model’s immediate working space. The next concept, the context window, asks how many of those units can participate in one request and what happens as that space fills.

Carry forward four ideas:

• Capacity is shared. System instructions, history, tools, RAG evidence, the current message, and output all compete for room.
• Position and relevance matter. Fitting text does not guarantee the model will use every detail equally well.
• Context is not durable memory. Applications must deliberately retrieve or resend information needed later.
• More is not always better. Irrelevant context can raise cost and reduce clarity even when it fits.

When evaluating a context-window claim, ask whether the limit is input-only or combined, what output cap also applies, how the model behaves near the limit, and whether quality was tested on your task. A large advertised number is capacity, not proof that pasting an entire archive is the best design.