How Large Language Models Handle Context and Mixed Media Today
Large language models from Anthropic, OpenAI and Google now all offer a context window of about 1M tokens, roughly 2,500 pages of text. All of them also read images, so context window size no longer separates one model from another. What does is how you fill that context window, what each request costs and how you test the answers. Related reading: AI-Powered Customer Engagement for B2B SaaS.
A bigger window does not fix wrong answers: a million-token model can read a 600-page loan file and still miss a waived covenant on page 412, fluently and with confidence. This guide covers how to decide what goes into the context window, including long context versus RAG. It also covers cost, multimodal AI inputs, testing and governance, with every product number checked against vendor documentation between October 15 and 16, 2026. If you are scoping a build with large language models, our enterprise AI development services team works through these same trade-offs. For hands-on help with this, explore our AI consulting services.
Large Language Models in October 2026: What the Context Window Holds
A context window is the total number of tokens one of the large language models can consider in one request. That covers your prompt, documents, images, tool results and the model’s own reasoning and answer. Tokens are word fragments: Anthropic’s 2024 guidance put 200,000 tokens at about 500 pages, so 1M tokens is roughly 2,500 pages of text. Go deeper: Artificial Intelligence.
The headline context window has stopped being a differentiator. Here is what the official model pages list for today’s large language models.
| Model | Context window | Max output | Inputs | List price per 1M tokens (in / out) |
| Claude Fable 5.1 | 1M | 128K | Text, image, PDF | $10 / $50 |
| Claude Opus 5.5 | 1M | 128K | Text, image, PDF | $4 / $20 |
| Claude Sonnet 5.5 | 1M | 128K | Text, image, PDF | $2 / $10 |
| Claude Haiku 5.5 | 1M | 128K | Text, image, PDF | From $0.10 / $0.50 |
| GPT-6 Astra | 1.05M | 128K | Text, image | $10 / $50 |
| GPT-6.1 Sol | 1.05M | 128K | Text, image | $2 / $10 |
| GPT-6 Luna | 1.05M | 128K | Text, image | $0.10 / $0.50 |
| Gemini 4 Argon | 1,048,576 (shared) | 65,536 default | Text, image, video, audio, PDF | $2 / $10 through Nov 30, 2026, then $4 / $20 |
| Gemini 3.8 Flash | 1,048,576 | 65,536 | Text, image, video, audio, PDF | $0.75 / $3.75 through Dec 31, 2026, then $1.50 / $7.50 |
Three details about large language models hide behind that table, and each one has bitten a budget.
- Shared budgets: Gemini 4 Argon’s context window is shared across input, thinking and output tokens, and Anthropic counts thinking tokens against the context window too. A 1M-token input fills the whole context window and leaves no room for an answer.
- Price cliffs: OpenAI prices prompts above 272K input tokens at 2x input and 1.5x output for the full request. Anthropic bills 1M context window models at standard rates, except Claude Haiku 5.5, where prompts over 100,000 tokens move from $0.10 / $0.50 to $0.50 / $2.50.
- Model churn: large language models retire on a schedule. Anthropic lists Claude Sonnet 4.5 as deprecated, and Google limits Gemini 2.5 access to people who already used it. Pin versions and plan migrations.
Long Context vs RAG: Which Should You Use?
Use long context when the corpus fits comfortably in the context window, changes slowly and is queried often enough for caching to pay off. Use retrieval-augmented generation (RAG) when the corpus is huge, changes daily, or needs per-user permissions and citations. Most enterprise systems built on large language models end up as a hybrid of the two.
Retrieval-augmented generation is not obsolete just because one context window holds about 2,500 pages. For large language models it still wins on cost per question, citations and permissions. What changed is that it no longer has to be the default for small corpora.
Anthropic’s September 2024 advice on large language models was blunt: if your knowledge base is smaller than 200,000 tokens, put it all in the prompt and skip RAG. Context windows are five times bigger now, but Anthropic’s own documentation warns that accuracy and recall degrade as token count grows, a problem it calls context rot. A bigger context window raises the ceiling for large language models. It does not make filling it a good idea.
The SCPV Test for Choosing a Context Strategy for Large Language Models
This is Techverx practitioner judgment, not sourced data. Score four factors, then count which column wins.
| Factor | Favors long context | Favors RAG |
| Size of the corpus | Under about 200K tokens, well inside the context window | Over the context window, or growing without limit |
| Churn of the data | Stable for hours or days, so cached prefixes stay valid | Changes by the minute or by the user |
| Proof required | A summary-level answer is acceptable | Citations, audit trails or per-user access control |
| Volume of questions | Steady traffic keeps the cache warm | Thousands of distinct questions a day over a large corpus |
If three or four factors agree, pick that side. If it splits two to two, build the hybrid of long context and RAG: let RAG choose 20 to 50 relevant documents, then let a long context model reason across them. That pattern keeps cost down and gives large language models whole documents instead of orphaned snippets.
Retrieval-augmented generation also keeps improving. In Anthropic’s 2024 tests, adding contextual embeddings and contextual BM25 cut the top-20-chunk retrieval failure rate by 49%, and adding reranking brought the cut to 67%. Retrieval quality is mostly a data problem, which is why an AI-ready data foundation usually matters more than which of the large language models you pick.
When Not to Use Long Context With Large Language Models
- Multi-tenant data: if users have different permissions, anything you paste into a shared context window for large language models can leak across them.
- Tight latency targets: longer inputs raise time to first token on large language models, per Google’s long-context documentation.
- Moving data: a cache that expires every few minutes saves nothing for long context on a corpus that changes every few minutes.
What a Long Context Prompt Actually Costs
A long context prompt on any of today’s large language models costs its token count times the input price, on every request, unless caching applies. At 500,000 tokens that gets expensive fast, so run the arithmetic before you design the product.
The table below is our own calculation from the official list prices, input tokens only, for one question against a 500,000-token corpus. It assumes steady traffic keeps the cache warm and excludes output, retrieval infrastructure and embedding costs.
| Approach | Cost per question | At 1,000 questions a day |
| Claude Sonnet 5.5, no caching ($2 per 1M) | $1.00 | $1,000 |
| Claude Sonnet 5.5, cache read (0.05x, $0.10 per 1M) | $0.05 | $50 |
| GPT-6.1 Sol, no caching (2x rate above 272K, $4 per 1M) | $2.00 | $2,000 |
| GPT-6.1 Sol, cached (2x of $0.10, $0.20 per 1M) | $0.10 | $100 |
| Gemini 4 Argon, no caching ($2 per 1M until Nov 30; $4 from Dec 1) | $1.00, then $2.00 | $1,000, then $2,000 |
| RAG (retrieval-augmented generation) on Sonnet 5.5, about 8,000 retrieved tokens | $0.016 | $16 |
Caching closes most of the gap for long context on large language models, which is the useful lesson. Anthropic prices a five-minute cache write at 1.25x the base input rate and a cache read at 0.05x on Opus 5.5 and Sonnet 5.5. The minimum cacheable prompt on those models is 512 tokens. Gemini 4 Argon supports implicit caching only, so you cannot control cache hits explicitly.
Four levers cut the bill for large language models beyond caching:
- Route by difficulty: send classification and extraction to Claude Haiku 5.5 or GPT-6 Luna, and keep the flagship large language models for hard reasoning.
- Use batch for anything non-interactive: OpenAI prices Batch and Flex at 50% of standard, and Google’s batch rates for Gemini 4 Argon are half of standard too.
- Put the question last: Google’s guidance is to place the query at the end of a long context prompt.
- Cap spend per request: OWASP lists unbounded consumption as a top-ten LLM risk for a reason.
Prices for large language models move, so check the date on any price you see, including ours. Gemini 4 Argon input doubles from $2 to $4 per million tokens on December 1, 2026, and a budget built in October will be wrong by January.
Multimodal AI Inputs: Text, Images, PDFs, Audio and Video
Multimodal AI means one of the large language models accepts more than text in a single request and reasons across the pieces. A multimodal LLM can read a chart, a scanned form and a paragraph together. In practice, support across large language models differs sharply by vendor. All three accept images, Anthropic and Google document PDF input directly, and only Google’s current models list audio and video input.
Vendors market multimodal large language models as one capability, but it is really four: reading documents, reading images, hearing audio and watching video. Score each separately, because a model that excels at charts may struggle with noisy call audio.
Gartner predicted in September 2024 that 40% of generative AI solutions would be multimodal by 2027, up from 1% in 2023. Multimodal AI inputs make large language models more useful and harder to budget, so the token math below matters.
| Input | What the docs say | Token math |
| PDFs and documents | Claude: up to 600 pages and 32 MB per request on 1M context window models, each page read as text plus an image. Gemini: up to 1,000 pages or 50 MB. | Claude: about 1,500 to 3,000 text tokens per page, plus image tokens. Gemini: 258 tokens per page. |
| Images | Claude: up to 600 images per request, 8000 x 8000 px maximum (a stricter per-image limit applies above 20 images). | Tokens equal ceil(width/28) x ceil(height/28). A 1920 x 1080 image is 1,560 tokens on the standard tier and 2,691 on the high-resolution tier (Claude 4.7 and later). |
| Audio | Gemini: up to 9.5 hours per prompt, with transcription and speaker labels. GPT-6 model pages list audio as unsupported; voice runs through separate OpenAI products. | 32 tokens per second, so one hour is 115,200 tokens. |
| Video | Gemini: up to 3 hours at default resolution, sampled at 1 frame per second. GPT-6 pages list video as unsupported, and Claude’s model table lists text and image input only. | About 100 tokens per second at default resolution, so one hour is roughly 360,000 tokens. |
The gap on PDFs is large enough to change a design. A 100-page report is about 25,800 tokens on Gemini and 150,000 to 300,000 text tokens on Claude before image costs. The two families of large language models read pages differently, so count tokens on your own documents before you choose.
Native Multimodal AI or a Text Pipeline?
For multimodal AI, send the raw file when layout carries meaning: charts, scanned forms, slide decks, handwritten notes. Convert to text first when only the words matter and volume is high, because a transcript or OCR output is far cheaper to store, search and cache. For call recordings, a speech-to-text step plus a text model is often the better first build, and Google’s audio documentation itself points to Cloud Speech-to-Text for real-time transcription.
How to Test Context and Multimodal Quality
Whether you run long context, retrieval-augmented generation or a hybrid, large language models fail quietly, so test the system on your own documents, in your own failure modes. A needle-in-a-haystack demo proves that large language models can find one planted sentence, not that it can reconcile clause 14 with an amendment 300 pages later. Go deeper: Why AI Pilots Fail to Reach Production in Retail.
A 2023 study by Liu and colleagues, “Lost in the Middle,” tested several large language models. It found they often did best when the relevant text sat at the start or end of the input, and worse in the middle, even for models built for long contexts.
McKinsey’s State of AI survey, published in August 2026, found that nearly nine in ten respondents use AI regularly in at least one function. Yet only 37% report that AI has contributed positively to EBIT. In our view, weak measurement is part of that gap.
A workable evaluation checklist:
- Build a golden set per modality: 50 to 100 real questions with verified answers for text, scanned PDFs, images and audio each. Our figure is a starting point from practice, not a statistical rule.
- Move the evidence: place the answer near the start, middle and end of the context, and rerun at 25%, 50% and 100% of your working context window.
- Score cost per correct answer: a cheap model that is right 70% of the time can cost more than a flagship once review time is counted.
- Track p95 latency, not the average: users feel the slow tail.
- Check citations: every claim from large language models should point to a page, timestamp or region you can open.
- Rerun on every model change: upgrades and deprecations of large language models are routine now.
Prompts and pipelines drift as large language models change, so tie these tests to an operating process. Our guides on LLMOps for production models and prompt engineering cover the monitoring and prompt-versioning side.
Governance for Long Context and Multimodal AI Inputs
Everything in the context window is visible to the model, so governance of large language models starts before the request. Decide what may enter the context window, who may see the answer, and what gets logged. NIST’s Generative AI Profile (AI 600-1, July 2024) is a useful voluntary checklist for these questions.
Multimodal AI inputs add attack surface to large language models. OWASP’s LLM01 prompt injection entry warns that attackers can hide instructions in images that accompany benign text, and the same logic applies to scanned PDFs and audio. Treat every file a user or vendor uploads as untrusted input.
- Filter before the model: enforce document permissions in your RAG retrieval layer, not in the prompt. OWASP’s 2025 list includes vector and embedding weaknesses for this reason.
- Redact first: strip personal and regulated data before upload to large language models where the use case allows.
- Know the retention clock: Google states that Gemini Files API uploads are stored for 48 hours. Confirm retention and residency terms for every large language model vendor in the contract.
- Log the context: keep a record of which documents entered each context window, so you can answer an audit or a complaint.
- Keep a human in the loop: require review of large language models’ output for decisions in lending, healthcare and legal work.
Five Mistakes Teams Make With Large Language Models
- Filling the context window because you can: with large language models, more tokens means more cost, latency and distraction.
- Testing with one needle: a single long context retrieval test hides multi-hop failures.
- Ignoring price cliffs: crossing 272K tokens on OpenAI reprices the whole request.
- Forcing multimodal AI where text exists: sending a scan to large language models when the source is a clean file multiplies cost for no gain.
- Hard-coding a model name: keep the model ID in configuration, with a tested fallback.
What to Do This Quarter With Large Language Models
- Inventory your use cases: list each one with corpus size in tokens, update frequency and permission needs, then score it with SCPV, including how much of the context window it would use.
- Measure real token counts: use each vendor’s token counting on 20 of your own documents, including the PDFs and images your multimodal AI features will send.
- Run a bake-off: test large language models from two vendors, at two tiers each, on the same golden set. Compare cost per correct answer, with long context and RAG designs side by side.
- Turn on caching: it is the single biggest cost lever for any stable long context design on large language models.
- Write the guardrails: upload policy, retention terms and logging requirements, approved before launch.
The Short Verdict on Large Language Models
Million-token context windows are now table stakes for large language models. Cost discipline, retrieval quality, careful evaluation and governance decide the outcome. Start with retrieval-augmented generation or a hybrid for large and changing data. Use long context for stable, bounded corpora with caching. Send raw files to multimodal large language models only when layout matters. Choose among large language models by cost per correct answer on your own documents, not by headline context window size.
