An invoice classifier handles normal expense reports but repeatedly labels a hotel minibar as a business meal. Adding an instruction to ‘be accurate’ does not explain the missing boundary. A useful few-shot example pairs the disputed transaction with the business rule and a contrasting case where an ordinary meal is reimbursable. Examples should also show what to do when the receipt lacks enough evidence. The Anthropic CCA-F prompt-engineering judgment is whether those demonstrations improve the decision boundary on held-out cases without damaging common ones, not whether the examples look convincing.
Identify the failure before adding examples
Start by examining cases the classifier gets wrong. If the receipt shows a meal for a client and an employee, the application may need more complete policy rules rather than extra examples. If the rule is clear but borderline transactions are misclassified, few-shot prompting can expose the distinction. Separate missing policy, insufficient source evidence, ambiguous category boundaries and malformed output: they have different remedies. A successful prompt revision addresses the observed failure rather than blindly increasing prompt length.
The minibar case should be written as a specific classification decision. A charge for room snacks is personal consumption under one policy, whereas an itemized, approved client dinner may qualify as a business meal. Include exact evidence fields, category, and a short justification tied to the rule. Without that structure the model may learn an incidental keyword, such as “hotel,” rather than the policy distinction. The goal is not general agreement with the examples but transfer to unseen invoices.
Choose contrasting examples, not convenient ones
Three nearly identical positive examples can create apparent consistency while failing at the edge. A stronger set might contain an unambiguous client meal, a personal minibar, an itemized expense with missing attendees, and a receipt where the total combines eligible and ineligible items. These cases teach both the normal path and the decisions that require rejection or review. Each example should carry the same fields as the production input and a verified label backed by the current policy.
Diversity should be purposeful rather than random. Vary the features likely to confuse the model: merchant descriptions, currencies, abbreviations, missing fields, tax treatment or document layout. Avoid changing several critical assumptions at once, because then the example no longer explains which boundary matters. A reviewer should be able to say why every demonstration was selected and which future failure it is meant to prevent. A large set of easy cases is rarely as useful as a smaller set of carefully contrasted examples.
Keep instructions, evidence and examples distinct
A prompt often contains a system rule, examples of expected behavior and the actual document being classified. Mixing those layers can make it difficult for Claude to recognize which text is a demonstration and which is a live input. Clear delimiters or structured tags help separate the sections. State the classification objective and criteria in plain language, show complete examples, then place the new invoice under an unmistakable task boundary. Do not let content from the new invoice override the policy.
This separation is especially important because documents can contain arbitrary text. An invoice footer asking the assistant to mark everything eligible is document data, not a new policy. The application should preserve source provenance and enforce permission decisions outside the prompt. Examples can help the model learn what to ignore, but they are not a security boundary. The safer pattern combines explicit instructions, trusted policy references, constrained output fields and backend checks that validate consequential decisions.
Make the labels reflect business meaning
Examples are only as good as their labels. If an expense team disagrees over whether a meal qualifies, the model will not solve the policy ambiguity by seeing both labels in a prompt. Resolve contested examples with the policy owner, documenting the rule and effective date. For categories that cannot be decided from the receipt alone, use an explicit needs_review outcome instead of forcing a yes-or-no answer. A valid example can teach when abstention is the correct business action.
Review each demonstration for accidental signals. If all approved examples have long descriptions and every rejected example has a short one, Claude might learn length rather than eligibility. If examples all come from one country, international receipts may cause avoidable errors. The training effect of a prompt is informal but real: the assistant looks for patterns. The architect should help it find policy-relevant patterns by controlling the examples and challenging unintended correlations.
Evaluate precision and recall separately
A prompt can appear more cautious because it rejects more claims, yet create an unacceptable number of false negatives. For expense fraud detection, missing a true violation may be costly; for automated payment refusal, wrongly rejecting legitimate expenses harms employees. Define the business objective and measure precision, recall and human-review volume on a frozen set of labeled cases. Do not optimize an overall accuracy number while ignoring the error class that matters most to the process.
Keep the holdout data separate from examples repeatedly shown to the model during prompt tuning. Otherwise improvements may reflect memorization of familiar receipts rather than a robust decision boundary. Include ordinary cases and adversarial edge cases, then compare against the previous prompt using the same model and output validator. Track prompt version and document selection so results are reproducible. If the new examples improve one policy category but worsen another, investigate the tradeoff rather than declaring victory from one impressive demonstration.
Use examples alongside a structured output contract
A few-shot demonstration can teach the boundary between an approved meal and an ambiguous receipt, but semantic output validation must still check the actual document evidence. When a required fact cannot be established, calibrated human review is more reliable than inventing another example that nudges Claude toward a yes-or-no answer. Prompt calibration and downstream acceptance are complementary, not interchangeable.
Few-shot prompting can clarify the meaning of fields, but it does not guarantee that every response is schema-valid. A production extractor may require a category code, accepted amount, evidence reference and review reason. Use a suitable structured response mechanism and validate the returned fields separately. A valid category string could still be semantically incorrect, and a model explanation can still invent an absent supporting document. The examples guide judgment; the schema and application checks govern form and acceptance.
Consider an example where a receipt includes a charge in two currencies. A demonstration can show that the extractor must preserve the observed currency and abstain when settlement currency cannot be established. The downstream system then checks whether the currency code is permitted and whether the source has sufficient evidence for conversion. This complements structured-output design without treating good formatting as proof of a correct classification.
Handle evolving policies and example drift
Expense policies change. A demonstration labeled correctly last year may now teach the wrong outcome if meal caps, regional rules or approval requirements have changed. Store examples with policy version and jurisdiction where relevant, and review them when the governing rule changes. Avoid silently mixing examples from incompatible effective periods. A prompt is part of the application release, so updating examples should trigger the same regression tests used for other behavior changes.
Watch for gradual expansion. Teams often append a new example after every failure until the prompt becomes enormous and contradictory. Group similar failures, remove redundant demonstrations and retain cases that meaningfully improve coverage. A concise set with an explicit business rule is easier to audit than fifty historical receipts with inconsistent explanations. When the use case becomes too complex for prompt-only categorization, consider upstream preprocessing, better evidence collection or a specialist review step rather than further stuffing the prompt.
Recognize when few-shot prompting is the wrong tool
If a payment service returns the wrong exchange rate, extra examples will not repair the underlying data. If a user is unauthorized, a prompt demonstration cannot grant permission. If a model returns malformed JSON, strict output validation and a targeted correction path may be more reliable than more examples. The architect should distinguish model decision uncertainty from missing inputs and deterministic system failures. Applying the wrong mechanism can produce beautifully worded output that remains operationally unusable.
A useful diagnostic starts with the question “what exactly failed?” If the failure is a judgment boundary, examples may be strong medicine. If it is a source-truth or state-transition problem, fix that layer directly. This is one of the recurring principles of the certification: choose the mechanism that addresses the failure where it occurs. The best prompt is not necessarily the longest prompt, and a high-confidence model response is not an alternative to authoritative evidence.
A minibar claim and the corrected decision
An employee submits a hotel receipt with a restaurant dinner, minibar charges and a resort fee. The old prompt labels every food-related line as an eligible meal. A revised prompt states the company’s meal policy and includes a clearly approved client dinner, a rejected minibar purchase and an ambiguous combined bill requiring review. Claude classifies each item separately, cites the receipt line and marks the resort fee outside the meal category. The application validates the fields before the expense system makes any reimbursement decision.
The team evaluates the new prompt on receipts not used as demonstrations. It measures false approvals, false refusals and cases needing a reviewer. If the revised examples improve the relevant edge cases without worsening normal claims, the change can be adopted with its policy-version record. What matters is not that Claude reproduces the expected answer to the example, but that it applies the correct rule to new evidence and makes uncertainty visible when the source cannot resolve it.