Prompts & Writing
How Do You Debug a Prompt That Keeps Producing the Wrong Thing?
By Jim Vernon, Editor, AI Intelligence International · Published 24 August 2026 · Reviewed against our editorial standards · About the author
The instinct when a prompt fails is to rewrite it longer. That occasionally works and usually creates a prompt nobody understands, which fails differently a week later.
Prompt failures fall into a small number of categories, and each has a different fix. This article gives the diagnostic order that identifies which one you have in a few minutes.
Key takeaways
- Adding instructions to a failing prompt is the most common and least effective fix.
- Most failures are one of five types: missing context, competing instructions, wrong format anchor, ambiguous success criteria, or an unsuitable task.
- Change one variable at a time and keep a fixed set of test inputs.
- If three targeted fixes do not work, the task usually needs decomposition rather than a better prompt.
What should you do before changing anything?
Fix a test set. Five to ten representative inputs, including two that previously failed and two easy ones, saved somewhere you can rerun them. Without this, every change is evaluated against a different input and you learn nothing.
Then characterise the failure precisely. 'Bad output' is not a diagnosis. Is it wrong facts, wrong format, wrong length, wrong tone, ignoring an instruction, or refusing?
Those categories map almost one-to-one onto causes, which is why naming the failure accurately does most of the diagnostic work.
Is it missing context?
The most frequent cause. The prompt assumes knowledge the model does not have: your internal terminology, the audience, the document being referenced, the constraint that makes the obvious answer wrong.
Test by supplying the missing information explicitly for one input. If the output becomes correct, the fix is context, not instruction.
Supply context as data rather than as description. Pasting the actual policy paragraph works far better than describing what the policy says.
Are the instructions competing?
Prompts accumulate. 'Be concise' and 'explain your reasoning' and 'cover all edge cases' can each be reasonable and collectively impossible, and the model resolves the conflict arbitrarily.
List every instruction in the prompt on separate lines and read them as a set. Conflicts are usually obvious once written out and invisible in flowing prose.
Rank them explicitly when they genuinely compete: 'If brevity and completeness conflict, prefer completeness.' Stated precedence resolves far more failures than stronger wording does.
Is the format anchor wrong or missing?
Describing a format produces approximate compliance; showing one produces close compliance. If output shape is the problem, add a literal example of the exact output you want, including its punctuation and field names.
For structured output, specify the schema and state what to do with missing values. Unspecified missing-value handling is behind a large share of downstream parsing failures.
One example is worth several sentences. Two examples covering different cases are worth more, and beyond three the returns fall off quickly for most tasks.
Is success actually defined?
Many prompts ask for something no reader could grade: 'make this better', 'write a professional summary', 'improve the tone'. If you cannot tell whether an output passes, neither can the model.
Define the pass condition inside the prompt: what must be present, what must be absent, and for whom. 'A summary a busy operations manager could act on without opening the source, under 120 words, with every number carried through exactly' is gradeable.
Where the criteria are complex, ask for a self-check as a final step. Having the output verified against the stated criteria in the same call catches a useful fraction of failures.
When is the task itself the problem?
Some tasks fail regardless of prompting: arithmetic over many rows, anything requiring information the model cannot access, anything where the correct answer depends on private current state.
The tell is inconsistent failure on inputs that look equivalent. Instruction-level problems fail consistently; capability-level problems fail randomly.
The fix is decomposition or tooling: retrieve the data first, calculate outside the model, or split one prompt into a chain where each step is individually checkable.
Worked example: a summarisation prompt fixed in four passes
A support team used a prompt to summarise ticket threads for handover. Roughly a third of summaries omitted the customer's actual request, which is the only thing the reader needed.
Pass one: they built a test set of eight threads, including three known failures. Failure characterisation showed the omissions clustered in long threads where the request appeared early and was restated later in different words.
Pass two: they suspected missing context and pasted the team's definition of 'the request' into the prompt. No change. That ruled out context and cost five minutes.
Pass three: writing the instructions out as a list revealed a conflict. The prompt asked for a summary under 80 words and for the full escalation history, which in long threads consumed the entire budget. They added explicit precedence: the customer's request and current status always come first, history is included only if space remains.
Pass four: they added one literal example output with three labelled lines — Request, Status, History. Failures on the test set went from three of eight to zero, and a follow-up sample of 50 live threads showed two omissions, both in threads where the customer never stated a clear request.
Total prompt length increased by about 60 words. The version they had considered writing, with eleven added instructions, was never needed.
Frequently asked questions
Should I make the prompt longer or shorter when it fails?
Neither by default. Change what the diagnosis indicates. In practice, failing prompts more often need a precedence rule and one concrete example than they need additional instructions.
Does switching models fix prompt problems?
Sometimes, and it is worth a single test, but it hides the cause. A prompt with competing instructions will fail again on the new model under slightly different inputs.
How many test inputs are enough?
Five to ten for iteration, and a larger held-out set of thirty or more before you rely on the prompt in production. Small sets are for speed; larger ones are for confidence.
Is it worth asking the model why it failed?
It is worth trying, but treat the answer as a hypothesis rather than a diagnosis. Self-explanations are plausible reconstructions and are frequently wrong about the actual cause.