Question 210
A user reports that a chat reply stopped halfway through a paragraph. Which response metadata matters?
- For this workflow, examine the retrieval corpus version and ranking before changing the model.
- Inspect stop_reason and output token limits in the recorded API trace.
- Review request validation details and the supplied schema before retrying.
- Introduce quota-aware throttling and bounded backoff rather than immediate repeated retries.
Explanation
Correct Answer
B:A max_tokens stop distinguishes deliberate truncation from a natural end_turn and directs the next fix. Read stop_reason and actual output-token usage to tell a generation limit from network interruption or UI display truncation.
Incorrect Answers
A:A model can reason consistently over stale evidence; source freshness is a data pipeline defect; this implements outdated retrieval evidence rather than the demanded incomplete generation trace.
C:Bad-request errors indicate an invalid request contract; retrying unchanged payloads will not repair schema or parameter mistakes; for incomplete generation trace, a control centered on invalid-request recovery is not the appropriate substitute.
D:Rate limiting signals excessive request volume or tokens; paced retries and concurrency controls address that condition; the incomplete generation trace criterion differs from traffic throttling triage; the gap persists.
Question 211
An agent calls the same failed search tool repeatedly. What should an evaluator inspect to locate the issue?
- Associate request IDs with model, prompt, tool, latency, and failure-stage metadata.
- Repeat eligible requests with controlled backoff and an alternate path if available.
- Apply deterministic connector fixtures and compare model trajectories against a stable expected outcome.
- Review the sequence of tool requests, observations, and termination decisions.
Explanation
Correct Answer
D:A final answer alone hides recurrent loops; trajectory traces show whether the agent used error feedback appropriately.
Incorrect Answers
A:Cross-layer traces identify where a request diverged and which configuration produced the symptom, while respecting secret redaction; the consequence is improved correlated failure telemetry, not verified looping trajectory diagnostics.
B:An overloaded service differs from a malformed payload; resilience requires delay and limits, not changing the prompt blindly; where looping trajectory diagnostics matters, a control for overload response handling leaves that test unmet.
C:Fixed tool observations make prompt or model regressions reproducible; live-connector behavior should be covered by a separate integration test suite with its own assertions; even if reproducible agent test fixtures succeeds, looping trajectory diagnostics still needs separate verification.
Question 212
A tool-enabled workflow passes an acceptance test even though its final text is fluent but its database write is wrong. Which TWO pieces of evidence would improve testing? Choose TWO.
- Trace the agent’s tool requests and tool responses.
- Assert the authoritative database state after the run.
- Accept any successful HTTP status as proof of business success.
- Ignore tool-layer errors when model output looks confident.
- Compare only the stylistic quality of the final paragraph.
Explanation
Correct Answers
A:Execution traces reveal which operation, arguments, or observations caused the failure. Inspect arguments and tool errors to locate the first step at which the intended state diverged.
B:State assertions detect incorrect mutations that are invisible in the prose response. Compare final records against expected invariants, not only against a generated statement that an update succeeded.
Incorrect Answers
C:Transport delivery cannot verify the contents of external state. A successful API response only proves that an HTTP exchange occurred, not that a business invariant holds.
D:The execution outcome is independent of the model’s apparent certainty. A suppressed error may leave the system inconsistent even when the model confidently reports completion.
E:Fluency does not establish side-effect correctness. A polished explanation can coexist with an incorrect database primary key or an unintended write.
Question 213
An agent regression test is unstable because its inventory connector returns different stock counts on every run. What evaluation design isolates agent behavior?
- Use deterministic connector fixtures and compare model trajectories against a stable expected outcome.
- Check stop_reason and output token limits in the recorded API trace.
- Put in place quota-aware throttling and bounded backoff rather than immediate repeated retries.
- Examine the sequence of tool requests, observations, and termination decisions.
Explanation
Correct Answer
A:Fixed tool observations make prompt or model regressions reproducible; live-connector behavior should be covered by a separate integration test suite with its own assertions.
Incorrect Answers
B:A max_tokens stop distinguishes deliberate truncation from a natural end_turn and directs the next fix; this implements incomplete generation trace rather than the demanded reproducible agent test fixtures.
C:Rate limiting signals excessive request volume or tokens; paced retries and concurrency controls address that condition; for reproducible agent test fixtures, a control centered on traffic throttling triage is not the appropriate substitute.
D:A final answer alone hides recurrent loops; trajectory traces show whether the agent used error feedback appropriately; for reproducible agent test fixtures, this instead manages looping trajectory diagnostics; the requested decision remains unresolved.
Question 214
A support ticket names a failing API request but provides no trace ID or model version. What telemetry change most improves diagnosis?
- At the application layer, investigate the application's tool argument mapping and dispatch layer.
- Correlate request IDs with model, prompt, tool, latency, and failure-stage metadata.
- Classify it as model-output or task-semantic error and add domain validation.
- Check request validation details and the supplied schema before retrying.
- Evaluate stop_reason and output token limits in the recorded API trace.
Explanation
Correct Answer
B:Cross-layer traces identify where a request diverged and which configuration produced the symptom, while respecting secret redaction.
Incorrect Answers
A:The tool input chosen by Claude and the executed downstream call differ; that points to integration code, not model reasoning; the effect concerns mismapped tool arguments; the success condition is correlated failure telemetry.
C:The transport and schema succeeded; the incorrect business value must be caught by evaluation or authoritative checks; where correlated failure telemetry matters, a control for semantic failure isolation leaves that test unmet.
D:Bad-request errors indicate an invalid request contract; retrying unchanged payloads will not repair schema or parameter mistakes; changing invalid-request recovery does not establish correlated failure telemetry in this application.
E:A max_tokens stop distinguishes deliberate truncation from a natural end_turn and directs the next fix; this targets incomplete generation trace, while the system must demonstrate correlated failure telemetry.
Question 215
A developer estimates API cost by counting characters in a user prompt. What measure should replace this proxy?
- Generation must be paired with retrieval, verification, or controlled validation for factual guarantees.
- Configure an appropriate output budget and explicit stopping or response-length guidance.
- Use token counts for input and output on the chosen model.
- Differentiate the output generation limit from the model's total context capacity.
Explanation
Correct Answer
C:Billing and context limits are token based; equal-length strings can tokenize differently due to language and structure.
Incorrect Answers
A:A model optimizes plausible continuations, not an intrinsic proof system for external claims; changing multi-criterion integration does not establish token based resource accounting in this application.
B:Generation continues until a stop condition; output limits and clear completion criteria reduce unnecessary latency and spend; where token based resource accounting matters, a control for output length control leaves that test unmet.
D:Max_tokens budgets produced tokens; it does not change the model's supported context window or remove input constraints; even if generation budget versus context window succeeds, token based resource accounting still needs separate verification.
Question 216
A model gives two slightly different wordings for the same prompt with identical settings. What property explains this?
- Categorize the request as zero-shot prompting.
- Employ token counts for input and output on the chosen model.
- Reduce irrelevant context while retaining decisive examples and constraints.
- Treat generative decoding as potentially nondeterministic and evaluate stable outcomes.
Explanation
Correct Answer
D:Sampling and underlying execution can vary output sequences; functional consistency matters more than identical prose.
Incorrect Answers
A:Zero-shot tasks supply instructions but no demonstration examples; successful use does not require a worked exemplar; changing zero shot identification does not establish nondeterministic text generation in this application.
B:Billing and context limits are token based; equal-length strings can tokenize differently due to language and structure; because token based resource accounting is its concern, nondeterministic text generation remains outside its effect here.
C:Extra tokens increase cost and can dilute the actual task; prompt size should follow required information, not assumed model intelligence; it changes prompt economy; it does not prove nondeterministic text generation in this case.
Question 217
An engineer assumes a very long context window makes retrieval quality irrelevant. What is the flaw?
- A larger context window automatically ranks all retrieved passages by factual reliability.
- Including more irrelevant passages guarantees higher citation precision.
- Relevant evidence can still be diluted by excessive or poorly ordered context.
- Retrieval quality matters only when a prompt exceeds the context size limit.
Explanation
Correct Answer
C:Capacity is not guaranteed attention; curation, source selection, and context hygiene remain important at large window sizes.
Incorrect Answers
A:Capacity permits more content but does not authenticate, rank, or validate the evidence inside that content. A trusted source-selection process is still necessary for evidence quality.
B:Noise can compete with relevant material and make unsupported or outdated statements easier to select. The system should rank and filter before inserting the retrieved passages.
D:Source relevance and authority remain important even when the entire prompt fits within the allowed window. Correctness is also affected by content quality, freshness, and ordering.
Question 218
A prompt asks Claude to translate a sentence without showing examples. What prompting category is this?
- Classify the request as zero-shot prompting.
- Use few-shot prompting through multiple demonstrations.
- Tune the model's supported thinking or effort configuration by task class.
- Determine it as single-shot prompting.
Explanation
Correct Answer
A:Zero-shot tasks supply instructions but no demonstration examples; successful use does not require a worked exemplar.
Incorrect Answers
B:Diverse examples convey boundaries and output format beyond abstract instructions, particularly for tricky exceptions; for zero shot identification, a control centered on multiple demonstration prompting is not the appropriate substitute.
C:Thinking controls can trade latency and tokens for reasoning effort on compatible models; not every model supports identical controls; even if task-adaptive thinking effort succeeds, zero shot identification still needs separate verification.
D:A single demonstration anchors the desired format or mapping before the actual case is presented; it improves single example prompting only, leaving zero shot identification unproven.
Question 219
A classifier includes one labeled example before the target input. Which prompting setup is being used?
- Monitor input and generated output tokens separately at their applicable rates.
- Identify it as single-shot prompting.
- Adopt token counts for input and output on the chosen model.
- Regard generative decoding as potentially nondeterministic and evaluate stable outcomes.
Explanation
Correct Answer
B:A single demonstration anchors the desired format or mapping before the actual case is presented. One-shot examples can help establish a mapping but are not the same as a zero-shot request with only instructions.
Incorrect Answers
A:Providers can charge different amounts by token direction and caching state; a single blended count obscures the drivers; for single example prompting, a control centered on separate inference token directions is not the appropriate substitute.
C:Billing and context limits are token based; equal-length strings can tokenize differently due to language and structure; testing single example prompting would still fail after improving token based resource accounting.
D:Sampling and underlying execution can vary output sequences; functional consistency matters more than identical prose; because nondeterministic text generation is its concern, single example prompting remains outside its effect here.
Question 220
A data-cleaning prompt shows three varied input-output pairs and then a new case. What pattern does it use?
- Employ deterministic validation and tolerance-based tests instead of assuming guaranteed repeatability.
- Apply few-shot prompting through multiple demonstrations.
- Find it as single-shot prompting.
- Limit irrelevant context while retaining decisive examples and constraints.
Explanation
Correct Answer
B:Diverse examples convey boundaries and output format beyond abstract instructions, particularly for tricky exceptions.
Incorrect Answers
A:Decoding controls can reduce variation but do not provide an absolute guarantee of identical outcomes across environments and releases; for this request, multiple demonstration prompting is the missing piece, not sampling reproducibility limitation.
C:A single demonstration anchors the desired format or mapping before the actual case is presented; this implements single example prompting rather than the demanded multiple demonstration prompting.
D:Extra tokens increase cost and can dilute the actual task; prompt size should follow required information, not assumed model intelligence; it improves prompt economy only, leaving multiple demonstration prompting unproven.