Prompts & Writing
How Do You Prompt Well Over Documents Too Long to Read at Once?
By Jim Vernon, Editor, AI Intelligence International · Published 24 August 2026 · Reviewed against our editorial standards · About the author
Large context windows made it possible to paste an entire contract, report or transcript into a single prompt. Possible is not the same as reliable: quality degrades with length in ways that are easy to miss because the output still looks complete.
This article covers what actually goes wrong with long inputs and the patterns — chunking, ordering, extraction-then-reasoning, and verification — that keep the work accurate.
Key takeaways
- Long-context failures are silent: the output looks whole while quietly omitting middle material.
- Extract first, reason second; mixing both in one pass over a long document is the main source of error.
- Put the instruction after the document, and the most important material at the start or end.
- Always verify with a targeted question whose answer you already know.
What actually degrades with long inputs?
Recall of material in the middle. Content at the beginning and end of a long input is used far more reliably than content in the middle, which produces summaries that are accurate about the opening and closing and thin in between.
Instruction adherence also weakens as input length grows. A formatting rule that holds perfectly on a two-page input starts slipping on a sixty-page one.
Neither failure announces itself. The output remains fluent and well-structured, which is why long-document work needs verification rather than inspection.
When should you chunk rather than paste everything?
Whenever the task requires completeness — finding every clause of a type, every action item, every figure. Completeness tasks are exactly where middle-material loss hurts, and chunking converts one unreliable pass into many reliable ones.
Paste whole documents when the task is genuinely holistic and approximate: overall tone, general theme, whether a document broadly covers a topic.
Chunk on structure — sections, clauses, speakers, dated entries — rather than on a fixed character count. Arbitrary splits cut sentences and separate a clause from its definition.
What is the extract-then-reason pattern?
Two passes. The first pass extracts atomic facts with their location: each obligation, figure, date, or commitment as a separate line with a section reference. The second pass reasons only over the extracted list, which is short.
This works because extraction is a local task that chunking makes reliable, while reasoning benefits from a small, dense input. Mixing them forces the model to do both badly.
The extracted list is also auditable. A human can spot-check twenty lines against the source in minutes, which is impossible with a single narrative summary.
Does the order of prompt and document matter?
Yes. Placing the instruction after the document generally produces better adherence than placing it before, because the instruction is then in the high-attention final position.
For very long inputs, state the instruction both before and after. The repetition costs little and measurably improves compliance on formatting and scope.
Put the most decision-relevant material — the section you actually care about — near one end rather than buried in the middle, when you have control over ordering.
How do you verify long-document output?
Plant a check. Ask a question whose answer you know appears mid-document, and confirm it is reflected. If the middle was dropped, this catches it immediately.
Require citations to section or page for every claim, then verify a random sample of three. Unverifiable claims should be treated as absent rather than as merely unsourced.
For repeated workflows, keep a small regression set of documents with known answers and rerun it whenever you change the prompt, the chunking or the model.
What about cost at long lengths?
Cost scales with input, so a habit of pasting whole documents on every turn is expensive, especially in a conversation where the document is resent each time.
Extract once, then work from the extraction. This is usually cheaper by an order of magnitude and more accurate, which is an unusual combination.
Cache extractions keyed on document hash. Documents get re-analysed far more often than they change.
Worked example: obligations in a 90-page contract
A procurement analyst needed every obligation with a deadline from a 90-page master services agreement, roughly 46,000 words.
The single-pass version pasted the whole document and asked for a table of obligations. It returned 23 rows, well-formatted and confident. Manual review of the contract found 41 obligations, and 16 of the 18 missed were between pages 30 and 70.
The revised approach split the document by numbered clause into 61 chunks. Each chunk was processed with the same short prompt: list any obligation, the obliged party, the deadline and the clause number, or return an empty list.
That produced 44 rows, including three duplicates from cross-referencing clauses, which a second pass over the 44-row list deduplicated and sorted by deadline. Manual review found 41 of 41 present, with no fabricated entries.
A planted check — a payment obligation on page 52 — was present in the chunked run and absent in the single-pass run, which became their standard smoke test. Total cost was slightly lower than the single-pass version, because 61 short calls cost less than one 46,000-word call plus the rework.
Frequently asked questions
If the context window fits the document, why chunk at all?
Fitting and using well are different. Recall of middle material degrades long before the window limit is reached, so chunking is about reliability rather than capacity.
How large should chunks be?
Large enough to be self-contained — usually a full section or clause — and small enough that the model handles them without strain. In practice a few hundred to a couple of thousand words works for most documents.
Do overlapping chunks help?
Yes for prose where meaning crosses boundaries. A modest overlap of a paragraph or two reduces boundary misses at the cost of some duplicates, which the consolidation pass removes.
Can I trust a summary of a document I have not read?
Only with verification. Use planted checks and required citations, and treat the summary as a map to the source rather than a replacement for it when the stakes are real.