Prompting techniques that actually work (and the mistakes that quietly ruin them)

Prompting has a reputation problem. Half the internet treats it as magic words, the other half dismisses it as trivial. Both are wrong. A prompt is an interface specification: it defines what goes into a model and what you expect to come out. Like any interface, it can be designed well or badly, and badly designed prompts fail in ways that are quiet, intermittent, and expensive to debug.

This post covers the techniques that consistently move quality, and the mistakes we see over and over when teams ship LLM features without treating prompts as engineered artifacts.

Start with the anatomy

Almost every reliable prompt has the same skeleton: a role, the context the model needs, one clear task, an explicit output format, and (when precision matters) a few examples. Most weak prompts are missing two or three of these pieces entirely.

You do not need every block every time. But when a prompt misbehaves, run down this list first. Nine times out of ten the failure maps to a missing or vague block, not to the model.

Part 1: Techniques that earn their keep

Zero-shot vs few-shot

Zero-shot prompting (instruction only, no examples) is the right default. It is cheap, short, and modern models handle it well for common tasks like summarization or classification with obvious labels.

Few-shot prompting adds two or three worked input-to-output pairs before the real input. The examples do something instructions cannot: they show the boundary cases and the exact output shape instead of describing them. If you have ever written three paragraphs trying to explain a formatting rule, one good example usually replaces all of them.

Use few-shot when:

• The output format is unusual or strict (custom markup, specific tone, domain jargon)
• The task has ambiguous edge cases and you can demonstrate how to resolve them
• Zero-shot output is almost right but drifts between runs

One warning: examples are instructions. If your examples are inconsistent with each other or with the written instruction, the model will follow the examples, and you will spend a day discovering that.

Chain-of-thought

For anything with multiple reasoning steps (math, planning, multi-condition eligibility rules, root-cause analysis), asking the model to work step by step before answering measurably improves accuracy. The intuition is simple: a model generating one token at a time cannot "hold" a long derivation invisibly. Giving it room to write the intermediate steps is giving it working memory.

The underrated benefit is debuggability. When a step-by-step answer is wrong, you can see which step went wrong and patch the prompt there. When a direct answer is wrong, all you know is that it is wrong.

Two practical notes: reasoning-tuned models already do this internally, so explicit chain-of-thought helps most on standard models; and if you need clean output downstream, ask for reasoning first and the final answer in a clearly delimited section you can parse out.

Role and context framing

"You are a senior tax accountant reviewing a filing for errors" is not decoration. It collapses the space of plausible responses: vocabulary, level of caution, what counts as an error worth flagging. The role should be specific enough to change behavior; "you are a helpful assistant" does nothing.

Context framing matters just as much: state the audience, the constraints, and what the output will be used for. "This summary will be read by an on-call engineer during an incident" produces a very different, and more useful, summary than no framing at all.

Structured output

If a machine consumes the output, specify the contract like an API schema, not a wish:

Return only a JSON object with this exact shape:
{
  "category": one of "billing" | "bug" | "feature_request",
  "confidence": number between 0 and 1,
  "reply_draft": string, max 120 words
}
No prose before or after the JSON.

Use delimiters (triple backticks, XML-style tags) to separate instructions from data. This also blunts prompt-injection from untrusted input, because the model can tell which text is instruction and which is payload. If your provider supports enforced structured output or function calling, use it; a schema the runtime enforces beats a schema the model is politely asked to follow.

Prompt decomposition

The single highest-leverage refactor for a struggling LLM feature: stop asking one prompt to do five jobs.

A mega-prompt that extracts, validates, summarizes, translates, and formats in one pass forces every instruction to compete for the model's attention, and gives you no way to see where a bad output came from. Split it into a pipeline where each call does one job. Each step becomes independently testable, independently cacheable, and independently replaceable with a cheaper model where quality allows. The extra latency is usually a bargain against the reliability you get back.

Part 2: Mistakes that quietly ruin results

Vague instructions

"Make it better," "keep it professional," "be concise": these mean nothing without a reference point. The model fills the vagueness with its own defaults, and its defaults are not your defaults.

The fix is always the same: replace adjectives with observable criteria. Not "be concise" but "maximum three sentences." Not "professional tone" but "no exclamation marks, no first person, address the reader as 'you'." If you cannot state how you would check the instruction was followed, the model cannot follow it reliably either.

Context overload

More context is not more accuracy. Pasting eight documents "just in case" dilutes the signal, raises cost, and increases the odds the model latches onto something irrelevant. Retrieve and include what the task needs; cut the rest. If everything genuinely is needed, that is your cue to decompose the task instead.

Negative instructions

"Don't mention pricing. Don't use bullet points. Don't be too formal." Negations are the weakest instruction form: you are making the model attend to the very concept you want absent, and you are leaving the desired behavior unspecified. State what you want instead: "Write flowing paragraphs in a conversational tone. If the customer asks about pricing, direct them to the sales team." Keep negative constraints for the few genuine hard bans, and pair each one with the positive alternative.

Burying the lede

Models recall the beginning and end of a long context far better than the middle, the well-documented "lost in the middle" effect. Yet the most common long-prompt layout puts instructions at the top, then a wall of documents, with the actual question and critical constraints buried somewhere in between.

Structure long prompts deliberately: key instructions at the start, reference material in the middle, and a restatement of the task and hard constraints at the end. If one sentence absolutely must be honored, it belongs in the final lines of the prompt, not on page two.

Iterating by vibes

The most damaging mistake is not in any single prompt; it is in the process. Someone tweaks a prompt, eyeballs one or two outputs, ships it. Nobody knows whether the change helped overall or just on those two cases; nobody can tell whether the model upgrade last month made things better or worse.

The fix is boring and non-negotiable: a fixed evaluation set of 20 to 50 real inputs with expected outputs, run on every prompt change, with pass rates written down. It takes an afternoon to build and permanently converts prompt work from folklore into engineering. Version your prompts in git next to the eval results, and every regression becomes a diff instead of a mystery.

Treat prompts like code

The through-line in all of this: prompts are load-bearing components, not incantations. The same habits that make code reliable make prompts reliable:

1. Specify the interface. Role, context, task, format. Say what you want, not what you don't.
2. Show, don't only tell. A few consistent examples outperform paragraphs of description.
3. Decompose. One prompt, one job. Pipelines beat mega-prompts.
4. Mind the geometry. Critical instructions at the start and end, never buried in the middle.
5. Measure. No eval set, no opinion. Version prompts and track pass rates like you track tests.

None of these techniques are exotic. That is exactly the point: the teams getting the best results from LLMs are not the ones with secret prompts. They are the ones applying ordinary engineering discipline to an unusual interface.