Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 16

Question 150

A production agent times out but logs only the final error, hiding which MCP call stalled. What should the solution architect prioritize?

  1. Capture tool-call counts, token usage, costs and task completion per trace.
  2. Record spans for model calls, tool dispatch, connector execution and result ingestion.
  3. Break down failure metrics by connector version, tenant, scope and response code.
  4. Log selected tool names, arguments and task categories for audited error analysis.

Explanation

Correct Answer

B:

Distributed traces reveal the slow dependency and its contribution to the overall deadline. Correlate tool-level distributed tracing with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

By itself, this improves per-task agent cost attribution, but the case is not asking the architect to optimize that dimension. The requirement is tool-level distributed tracing, supported by an approach to record spans for model calls, tool dispatch, connector execution and result ingestion.

C:

The option might improve segmented authorization telemetry; that remains insufficient for tool-level distributed tracing. The case is resolved by tool-level distributed tracing, which depends on a design that will record spans for model calls, tool dispatch, connector execution and result ingestion.

D:

The proposed action prioritizes tool-selection diagnostic evidence. That can coexist with the right solution, but it cannot replace tool-level distributed tracing: the application must still record spans for model calls, tool dispatch, connector execution and result ingestion.

Question 151

Operations need to detect invoice rules rejected behind HTTP 200 responses. They also cannot separate asynchronous queue delay from tool execution time and cannot tell whether cited evidence came from a stale cached source revision. Which THREE trace capabilities should be added? Choose THREE.

  1. Track semantic success and validation outcomes in addition to HTTP status codes.
  2. Track worker participation, completion state and escalation reasons per orchestration run.
  3. Record queue-wait time and tool-execution spans under a shared asynchronous correlation identifier.
  4. Log selected tool names, arguments and task categories for audited error analysis.
  5. Record the revision and observation age of evidence returned by cached retrieval or tool responses.

Explanation

Correct Answers

A:

An HTTP 200 can carry an invoice rejection that the caller must treat as a business failure. Logging the validation result, committed state and application outcome prevents success dashboards from confusing a completed network exchange with an accepted invoice.

C:

Separate queue and execution spans show where the critical path is delayed. Preserving a causal ID across worker boundaries prevents a queued job from appearing as an unexplained interval in the user-facing trace.

E:

A successful tool response can still contain outdated data. Source revisions and ages reveal freshness failure separately from transport errors without requiring unrestricted storage of raw sensitive documents.

Incorrect Answers

B:

It emphasizes agent-workflow completeness metrics without closing the stated gap in application-level outcome telemetry and queue-versus-execution latency attribution and traceable age and revision of cited tool evidence. By contrast, the remedy to record the revision and observation age of evidence returned by cached retrieval or tool responses acts on the condition that makes this scenario fail.

D:

Improving tool-selection diagnostic evidence does not imply application-level outcome telemetry and queue-versus-execution latency attribution and traceable age and revision of cited tool evidence. The selected design must record queue-wait time and tool-execution spans under a shared asynchronous correlation identifier, whereas this choice omits that direct protection or verification step.

Question 152

A tool returns correct data but the model repeatedly chooses the wrong tool for similar questions. Which architecture adjustment is most appropriate?

  1. Log document identifiers, versions and evidence references with privacy controls.
  2. Record spans for model calls, tool dispatch, connector execution and result ingestion.
  3. Capture tool-call counts, token usage, costs and task completion per trace.
  4. Log selected tool names, arguments and task categories for audited error analysis.

Explanation

Correct Answer

D:

Selection errors are reasoning or description issues, distinct from API transport failures. Correlate tool-selection diagnostic evidence with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

The proposed measure is aimed at auditable evidence lineage, not tool-selection diagnostic evidence. The architecture must instead log selected tool names, arguments and task categories for audited error analysis; a separate change is still needed for tool-selection diagnostic evidence.

B:

The relevant trade-off is not primarily tool-level distributed tracing. A successful fix needs tool-selection diagnostic evidence; otherwise tool-selection diagnostic evidence remains unverified even after this change.

C:

The control would be justified if per-task agent cost attribution were the limiting factor. In this scenario the decisive target is tool-selection diagnostic evidence; success requires an architecture that will log selected tool names, arguments and task categories for audited error analysis.

Question 153

A new connector version increases authorization denials only for one tenant. Which next step is most defensible?

  1. Record spans for model calls, tool dispatch, connector execution and result ingestion.
  2. Break down failure metrics by connector version, tenant, scope and response code.
  3. Validate and monitor tool-result schemas with field-level drift alerts.
  4. Track worker participation, completion state and escalation reasons per orchestration run.

Explanation

Correct Answer

B:

Dimensions isolate a regression that global error averages would hide. Correlate segmented authorization telemetry with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

The proposed action prioritizes tool-level distributed tracing. That can coexist with the right solution, but it cannot replace segmented authorization telemetry: the application must still break down failure metrics by connector version, tenant, scope and response code.

C:

This addresses schema-drift detection at integration boundaries; however, the problem asks for segmented authorization telemetry. Without the safeguard to break down failure metrics by connector version, tenant, scope and response code, the original failure mode is still possible.

D:

That choice would be responsive to agent-workflow completeness metrics, but the reported defect is different. It leaves segmented authorization telemetry unproved because there is no commitment to break down failure metrics by connector version, tenant, scope and response code.

Question 154

A research agent makes dozens of unnecessary searches and spends ten times its estimated budget. Which approach should be recommended?

  1. Record spans for model calls, tool dispatch, connector execution and result ingestion.
  2. Use redacted structured telemetry, access controls and selective retention for diagnostic artifacts.
  3. Capture tool-call counts, token usage, costs and task completion per trace.
  4. Log selected tool names, arguments and task categories for audited error analysis.
  5. Track worker participation, completion state and escalation reasons per orchestration run.

Explanation

Correct Answer

C:

Operational observability must show inefficient loops as well as correctness failures. Correlate per-task agent cost attribution with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

Improving tool-level distributed tracing does not imply per-task agent cost attribution. The selected design must capture tool-call counts, token usage, costs and task completion per trace, whereas this choice omits that direct protection or verification step.

B:

This recommendation targets privacy-safe operational evidence at a different point in the workflow. It leaves per-task agent cost attribution outside the design contract and does not require the team to capture tool-call counts, token usage, costs and task completion per trace.

D:

This would make sense as an additional project activity for tool-selection diagnostic evidence. The priority required here is per-task agent cost attribution, so the workflow still needs a control that will capture tool-call counts, token usage, costs and task completion per trace.

E:

By itself, this improves agent-workflow completeness metrics, but the case is not asking the architect to optimize that dimension. The requirement is per-task agent cost attribution, supported by an approach to capture tool-call counts, token usage, costs and task completion per trace.

Question 155

The audit team cannot reconstruct which retrieved record justified a model's specific conclusion. Which change most directly satisfies this requirement?

  1. Log selected tool names, arguments and task categories for audited error analysis.
  2. Record spans for model calls, tool dispatch, connector execution and result ingestion.
  3. Use redacted structured telemetry, access controls and selective retention for diagnostic artifacts.
  4. Log document identifiers, versions and evidence references with privacy controls.

Explanation

Correct Answer

D:

Source provenance allows a disputed answer to be traced without retaining unnecessary sensitive raw data. Correlate auditable evidence lineage with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

The option belongs to the broader work on tool-selection diagnostic evidence, but it cannot by itself deliver auditable evidence lineage. A decision on this case needs the team to log document identifiers, versions and evidence references with privacy controls and verify that outcome.

B:

This shifts attention toward tool-level distributed tracing when the relevant decision is about auditable evidence lineage. Unless the architecture can log document identifiers, versions and evidence references with privacy controls, the failure described in the stem remains unresolved.

C:

For a workflow whose primary weakness was privacy-safe operational evidence, that proposal could be reasonable. Here the unmet obligation is auditable evidence lineage, and that requires the system to log document identifiers, versions and evidence references with privacy controls.

Question 156

A case worker sees the same incident retried repeatedly but cannot tell whether a write already succeeded. Which solution best resolves this operational gap?

  1. Track semantic success and validation outcomes in addition to HTTP status codes.
  2. Log document identifiers, versions and evidence references with privacy controls.
  3. Capture tool-call counts, token usage, costs and task completion per trace.
  4. Associate retries, idempotency keys and committed-side-effect records in one transaction trace.

Explanation

Correct Answer

D:

Joined identifiers distinguish harmless retries from potential duplicate effects. Correlate replay and side-effect observability with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

That choice would be responsive to application-level outcome telemetry, but the reported defect is different. It leaves replay and side-effect observability unproved because there is no commitment to associate retries, idempotency keys and committed-side-effect records in one transaction trace.

B:

There is a legitimate use case for auditable evidence lineage, but it does not supply the test or enforcement needed for replay and side-effect observability. The design needs to associate retries, idempotency keys and committed-side-effect records in one transaction trace.

C:

Consider its actual operating scope: per-task agent cost attribution. The requested assurance is replay and side-effect observability, which calls for an implementation that can associate retries, idempotency keys and committed-side-effect records in one transaction trace, not merely for a related improvement.

Question 157

An API provider changes response schema and answer quality drops without rising 5xx rates. What design decision would best fit these facts?

  1. Validate and monitor tool-result schemas with field-level drift alerts.
  2. Log selected tool names, arguments and task categories for audited error analysis.
  3. Track semantic success and validation outcomes in addition to HTTP status codes.
  4. Record spans for model calls, tool dispatch, connector execution and result ingestion.

Explanation

Correct Answer

A:

Silent schema changes can poison context while transport metrics remain green. Correlate schema-drift detection at integration boundaries with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

B:

By itself, this improves tool-selection diagnostic evidence, but the case is not asking the architect to optimize that dimension. The requirement is schema-drift detection at integration boundaries, supported by an approach to validate and monitor tool-result schemas with field-level drift alerts.

C:

Its main effect concerns application-level outcome telemetry. The case explicitly hinges on schema-drift detection at integration boundaries; to resolve it reliably, the implementation has to validate and monitor tool-result schemas with field-level drift alerts, which this choice does not establish.

D:

The option might improve tool-level distributed tracing; that remains insufficient for schema-drift detection at integration boundaries. The case is resolved by schema-drift detection at integration boundaries, which depends on a design that will validate and monitor tool-result schemas with field-level drift alerts.

Question 158

A multi-agent system reports aggregate success despite consistently skipping a required compliance specialist. Which intervention most directly addresses the failure?

  1. Track worker participation, completion state and escalation reasons per orchestration run.
  2. Capture tool-call counts, token usage, costs and task completion per trace.
  3. Track semantic success and validation outcomes in addition to HTTP status codes.
  4. Record spans for model calls, tool dispatch, connector execution and result ingestion.

Explanation

Correct Answer

A:

Final-output metrics alone conceal missing mandatory branches. Correlate agent-workflow completeness metrics with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

B:

It is aimed at a different outcome, per-task agent cost attribution. No evidence is given that it provides agent-workflow completeness metrics; the needed mechanism is to track worker participation, completion state and escalation reasons per orchestration run at the appropriate boundary.

C:

It emphasizes application-level outcome telemetry without closing the stated gap in agent-workflow completeness metrics. By contrast, the remedy to track worker participation, completion state and escalation reasons per orchestration run acts on the condition that makes this scenario fail.

D:

That is primarily a technique for tool-level distributed tracing. It lacks the explicit measure for agent-workflow completeness metrics that is necessary here, namely a system that will track worker participation, completion state and escalation reasons per orchestration run.

Question 159

Operators need to debug intermittent PII leakage without publishing raw customer prompts in monitoring dashboards. What should the solution architect prioritize?

  1. Log selected tool names, arguments and task categories for audited error analysis.
  2. Log document identifiers, versions and evidence references with privacy controls.
  3. Use redacted structured telemetry, access controls and selective retention for diagnostic artifacts.
  4. Capture tool-call counts, token usage, costs and task completion per trace.

Explanation

Correct Answer

C:

Observability must balance debuggability against sensitive-content exposure. Correlate privacy-safe operational evidence with trace IDs, component versions and tool outcomes so responders can distinguish model errors from connector failures.

Incorrect Answers

A:

The alternative could reduce risk associated with tool-selection diagnostic evidence, yet the described behavior requires privacy-safe operational evidence. That outcome depends on the capability to use redacted structured telemetry, access controls and selective retention for diagnostic artifacts.

B:

The proposed measure is aimed at auditable evidence lineage, not privacy-safe operational evidence. The architecture must instead use redacted structured telemetry, access controls and selective retention for diagnostic artifacts; a separate change is still needed for privacy-safe operational evidence.

D:

This addresses per-task agent cost attribution; however, the problem asks for privacy-safe operational evidence. Without the safeguard to use redacted structured telemetry, access controls and selective retention for diagnostic artifacts, the original failure mode is still possible.

Question 160

A policy repository stores long manuals; fixed-size chunks repeatedly separate headings from their requirements. Which design choice best matches the stated constraint?

  1. Use modality-appropriate extraction with normalized metadata and validation per source type.
  2. Use structure-aware chunking that preserves headings, section boundaries and nearby definitions.
  3. Apply tenant and document ACL filtering before evidence enters model context.
  4. Use table-aware parsing and preserve row, column and unit metadata.

Explanation

Correct Answer

B:

Semantic boundaries make retrieved evidence interpretable and reduce context loss around policy clauses. Test section-preserving document chunks before generation using source-version and permission fixtures; retrieval cannot recover missing evidence essential to section-preserving document chunks.

Incorrect Answers

A:

The option might improve format-aware ingestion design; that remains insufficient for section-preserving document chunks. The case is resolved by section-preserving document chunks, which depends on a design that will use structure-aware chunking that preserves headings, section boundaries and nearby definitions.

C:

The design concern is real but different: pre-context permission filtering. It does not impose the specific requirement for section-preserving document chunks; a suitable solution must use structure-aware chunking that preserves headings, section boundaries and nearby definitions and demonstrate the result.

D:

This addresses table-structure preservation during ingestion; however, the problem asks for section-preserving document chunks. Without the safeguard to use structure-aware chunking that preserves headings, section boundaries and nearby definitions, the original failure mode is still possible.

Claude Certified Architect - Professional (CCAR-P) Practice Tests