Anthropic CCAR-P (Claude Certified Architect - Professional) Practice Test 26

Question 250

A caching proposal uses confidential prompts across tenants in a shared response cache keyed only by question text. Which approach should be recommended?

  1. Compare cost per accepted work item, including corrections, retries and manual effort.
  2. Assess supported batch processing and throughput limits for the workload.
  3. Partition cache keys by permissions and tenant context or avoid shared response caching.
  4. Scale or optimize the bottlenecked connector rather than changing model tier.

Explanation

Correct Answer

C:

Performance techniques must not bypass data isolation; matching wording does not imply matching entitlements. Compare authorization-aware cost optimization per accepted work item under expected load; token cuts can increase rework unless authorization-aware cost optimization is measured.

Incorrect Answers

A:

Improving fully burdened cost optimization does not imply authorization-aware cost optimization. The selected design must partition cache keys by permissions and tenant context or avoid shared response caching, whereas this choice omits that direct protection or verification step.

B:

This recommendation targets asynchronous high-volume optimization at a different point in the workflow. It leaves authorization-aware cost optimization outside the design contract and does not require the team to partition cache keys by permissions and tenant context or avoid shared response caching.

D:

This would make sense as an additional project activity for bottleneck-specific performance remediation. The priority required here is authorization-aware cost optimization, so the workflow still needs a control that will partition cache keys by permissions and tenant context or avoid shared response caching.

Question 251

Production quality falls on a new document format. Monitoring also hides failures for very long prompts inside aggregate averages, and a telemetry pipeline sometimes silently drops sampled records. Which THREE monitoring changes are needed? Choose THREE.

  1. Monitor source-to-index lag and stale-citation rates.
  2. Monitor accuracy by ingestion version and source format using labeled production samples.
  3. Segment outcome metrics by effective context-length buckets and the versions of context-building components.
  4. Track expected versus observed telemetry sample counts, drop rates, and ingestion lag as monitoring-health signals.
  5. Monitor false refusal rates alongside policy-violation rates.

Explanation

Correct Answers

B:

A new ingestion format can degrade accuracy in only one workload slice while overall quality appears stable. Format- and parser-version segments with labeled production samples show when the change began; context-length breakdowns and telemetry completeness address other hidden regressions.

C:

Failures confined to long contexts can disappear in global rates. Length-aware slices distinguish truncation or evidence displacement from unrelated quality changes and connect the regression to a particular release.

D:

An absent sample is not a successful request. Instrumenting the data collection path makes blind spots visible before engineers trust apparently stable quality dashboards.

Incorrect Answers

A:

The option belongs to the broader work on continuous source freshness assurance, but it cannot by itself deliver format-specific quality drift detection and detection of context-length-dependent regressions and monitoring pipeline completeness and loss detection. A decision on this case needs the team to track expected versus observed telemetry sample counts, drop rates, and ingestion lag as monitoring-health signals and verify that outcome.

E:

It is aimed at a different outcome, balanced guardrail outcome monitoring. No evidence is given that it provides format-specific quality drift detection and detection of context-length-dependent regressions and monitoring pipeline completeness and loss detection; the needed mechanism is to monitor accuracy by ingestion version and source format using labeled production samples at the appropriate boundary.

Question 252

A multi-tenant assistant receives short weekday bursts of HTTP 429 responses, yet its daily average request volume remains within the provider quota. Its dashboards aggregate these as generic failures. Which production monitoring improvement best diagnoses the problem?

  1. Measure only total daily request count and compare it with the published monthly quota.
  2. Track 429 rates, retry-after behavior, queue age and concurrent requests by tenant and short time interval.
  3. Report the average length and fluency of successful model responses instead of transport failures.
  4. Retry every 429 immediately without recording the retry delay or affected tenant.
  5. Alert only on HTTP 500-level errors because 429 responses are client-side events.

Explanation

Correct Answer

B:

Quota pressure is driven by burst patterns, not just daily request totals. Tenant- and time-sliced telemetry exposes contention and whether callers respect throttling instructions while preserving a path to capacity planning.

Incorrect Answers

A:

Daily totals hide concentrated bursts and do not show whether a per-minute or concurrent request ceiling was exceeded. The relevant evidence must resolve rate-limit events by interval and workload.

C:

Fluency describes completed answers, not whether requests were throttled before generation. Ignoring 429 responses biases the denominator toward calls that escaped capacity pressure.

D:

Immediate synchronized retries can amplify bursts and extend queueing. Without rate-limit and retry-after telemetry, operations cannot distinguish recovery from a self-induced overload cycle.

E:

HTTP 429 is an explicit signal of quota or throttling pressure. Its omission hides user-visible failures even when the server is behaving as configured, so a comprehensive SLI must include it.

Question 253

A retrieval index is updated nightly but knowledge freshness failures remain undetected until complaints. What design decision would best fit these facts?

  1. Monitor false refusal rates alongside policy-violation rates.
  2. Monitor accuracy by ingestion version and source format using labeled production samples.
  3. Alert on deduplicated retries and post-commit replay attempts.
  4. Monitor source-to-index lag and stale-citation rates.

Explanation

Correct Answer

D:

Freshness is a measurable production property independent of general model uptime. Segment the signal for continuous source freshness assurance by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

This would make sense as an additional project activity for balanced guardrail outcome monitoring. The priority required here is continuous source freshness assurance, so the workflow still needs a control that will monitor source-to-index lag and stale-citation rates.

B:

By itself, this improves format-specific quality drift detection, but the case is not asking the architect to optimize that dimension. The requirement is continuous source freshness assurance, supported by an approach to monitor source-to-index lag and stale-citation rates.

C:

Its main effect concerns production side-effect integrity monitoring. The case explicitly hinges on continuous source freshness assurance; to resolve it reliably, the implementation has to monitor source-to-index lag and stale-citation rates, which this choice does not establish.

Question 254

An autonomous tool agent's token usage triples after a seemingly minor tool-description update. Which intervention most directly addresses the failure?

  1. Alert on per-task tool count, token spend and unsuccessful loops after releases.
  2. Monitor accuracy by ingestion version and source format using labeled production samples.
  3. Run schema validation and monitor business-field completeness on tool results.
  4. Alert on deduplicated retries and post-commit replay attempts.

Explanation

Correct Answer

A:

Usage anomalies can expose reasoning-path regressions even while requests technically complete. Segment the signal for runaway behavior cost detection by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

B:

It is aimed at a different outcome, format-specific quality drift detection. No evidence is given that it provides runaway behavior cost detection; the needed mechanism is to alert on per-task tool count, token spend and unsuccessful loops after releases at the appropriate boundary.

C:

It emphasizes semantic integration drift monitoring without closing the stated gap in runaway behavior cost detection. By contrast, the remedy to alert on per-task tool count, token spend and unsuccessful loops after releases acts on the condition that makes this scenario fail.

D:

That is primarily a technique for production side-effect integrity monitoring. It lacks the explicit measure for runaway behavior cost detection that is necessary here, namely a system that will alert on per-task tool count, token spend and unsuccessful loops after releases.

Question 255

A content filter's aggregate block rate is stable, but a new region reports harmful outputs. What should the solution architect prioritize?

  1. Monitor false refusal rates alongside policy-violation rates.
  2. Break out policy violations by locale, tenant and severity.
  3. Monitor accuracy by ingestion version and source format using labeled production samples.
  4. Alert on deduplicated retries and post-commit replay attempts.

Explanation

Correct Answer

B:

A global average conceals localized safety deterioration. Segment the signal for segmented safety signal monitoring by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

The alternative could reduce risk associated with balanced guardrail outcome monitoring, yet the described behavior requires segmented safety signal monitoring. That outcome depends on the capability to break out policy violations by locale, tenant and severity.

C:

The relevant trade-off is not primarily format-specific quality drift detection. A successful fix needs segmented safety signal monitoring; otherwise segmented safety signal monitoring remains unverified even after this change.

D:

The control would be justified if production side-effect integrity monitoring were the limiting factor. In this scenario the decisive target is segmented safety signal monitoring; success requires an architecture that will break out policy violations by locale, tenant and severity.

Question 256

An assistant uses an external CRM whose success codes remain normal while field names change silently. Which design choice best matches the stated constraint?

  1. Monitor false refusal rates alongside policy-violation rates.
  2. Monitor accuracy by ingestion version and source format using labeled production samples.
  3. Run schema validation and monitor business-field completeness on tool results.
  4. Monitor source-to-index lag and stale-citation rates.

Explanation

Correct Answer

C:

Transport monitoring misses semantic contract drift unless required fields are checked. Segment the signal for semantic integration drift monitoring by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

The option might improve balanced guardrail outcome monitoring; that remains insufficient for semantic integration drift monitoring. The case is resolved by semantic integration drift monitoring, which depends on a design that will run schema validation and monitor business-field completeness on tool results.

B:

The proposed action prioritizes format-specific quality drift detection. That can coexist with the right solution, but it cannot replace semantic integration drift monitoring: the application must still run schema validation and monitor business-field completeness on tool results.

D:

This addresses continuous source freshness assurance; however, the problem asks for semantic integration drift monitoring. Without the safeguard to run schema validation and monitor business-field completeness on tool results, the original failure mode is still possible.

Question 257

A production team looks only at offline evals and never audits live user queries for emergent failure types. Which architecture adjustment is most appropriate?

  1. Track p95 and p99 end-to-end latency along with timeout rates.
  2. Alert on deduplicated retries and post-commit replay attempts.
  3. Monitor source-to-index lag and stale-citation rates.
  4. Sample privacy-safe live cases for periodic rubric-based review and test-set updates.

Explanation

Correct Answer

D:

Real workloads evolve; governed production sampling closes blind spots left by a frozen benchmark. Segment the signal for continuous real-world evaluation by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

That is primarily a technique for tail performance monitoring. It lacks the explicit measure for continuous real-world evaluation that is necessary here, namely a system that will sample privacy-safe live cases for periodic rubric-based review and test-set updates.

B:

Improving production side-effect integrity monitoring does not imply continuous real-world evaluation. The selected design must sample privacy-safe live cases for periodic rubric-based review and test-set updates, whereas this choice omits that direct protection or verification step.

C:

This recommendation targets continuous source freshness assurance at a different point in the workflow. It leaves continuous real-world evaluation outside the design contract and does not require the team to sample privacy-safe live cases for periodic rubric-based review and test-set updates.

Question 258

A company discovers that the latest prompt deployment harmed answer quality but cannot relate errors to the release. Which next step is most defensible?

  1. Monitor source-to-index lag and stale-citation rates.
  2. Alert on deduplicated retries and post-commit replay attempts.
  3. Tag traces with prompt, model, retrieval and tool versions for rollback comparison.
  4. Monitor accuracy by ingestion version and source format using labeled production samples.

Explanation

Correct Answer

C:

Versioned telemetry turns an unexplained trend into a reproducible change attribution. Segment the signal for version-to-quality correlation by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

The control would be justified if continuous source freshness assurance were the limiting factor. In this scenario the decisive target is version-to-quality correlation; success requires an architecture that will tag traces with prompt, model, retrieval and tool versions for rollback comparison.

B:

The option belongs to the broader work on production side-effect integrity monitoring, but it cannot by itself deliver version-to-quality correlation. A decision on this case needs the team to tag traces with prompt, model, retrieval and tool versions for rollback comparison and verify that outcome.

D:

For a workflow whose primary weakness was format-specific quality drift detection, that proposal could be reasonable. Here the unmet obligation is version-to-quality correlation, and that requires the system to tag traces with prompt, model, retrieval and tool versions for rollback comparison.

Question 259

A model refuses legitimate requests more often after a new guardrail rule. Which approach should be recommended?

  1. Track p95 and p99 end-to-end latency along with timeout rates.
  2. Monitor accuracy by ingestion version and source format using labeled production samples.
  3. Monitor source-to-index lag and stale-citation rates.
  4. Break out policy violations by locale, tenant and severity.
  5. Monitor false refusal rates alongside policy-violation rates.

Explanation

Correct Answer

E:

Safety tuning requires visibility into both overblocking and missed harms. Segment the signal for balanced guardrail outcome monitoring by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

This addresses tail performance monitoring; however, the problem asks for balanced guardrail outcome monitoring. Without the safeguard to monitor false refusal rates alongside policy-violation rates, the original failure mode is still possible.

B:

That choice would be responsive to format-specific quality drift detection, but the reported defect is different. It leaves balanced guardrail outcome monitoring unproved because there is no commitment to monitor false refusal rates alongside policy-violation rates.

C:

There is a legitimate use case for continuous source freshness assurance, but it does not supply the test or enforcement needed for balanced guardrail outcome monitoring. The design needs to monitor false refusal rates alongside policy-violation rates.

D:

Consider its actual operating scope: segmented safety signal monitoring. The requested assurance is balanced guardrail outcome monitoring, which calls for an implementation that can monitor false refusal rates alongside policy-violation rates, not merely for a related improvement.

Question 260

An agent occasionally triggers duplicate writes after timeout but no metric tracks repeated transaction keys. Which change most directly satisfies this requirement?

  1. Monitor source-to-index lag and stale-citation rates.
  2. Tag traces with prompt, model, retrieval and tool versions for rollback comparison.
  3. Track p95 and p99 end-to-end latency along with timeout rates.
  4. Alert on deduplicated retries and post-commit replay attempts.

Explanation

Correct Answer

D:

Side-effect telemetry catches operational failures that response text metrics cannot see. Segment the signal for production side-effect integrity monitoring by release, workload and affected population, with alerts tied to an assigned responder.

Incorrect Answers

A:

This is a valid architectural concern when optimizing continuous source freshness assurance. Here it leaves production side-effect integrity monitoring unaddressed, whereas the intended change must alert on deduplicated retries and post-commit replay attempts before the case can be considered resolved.

B:

This would make sense as an additional project activity for version-to-quality correlation. The priority required here is production side-effect integrity monitoring, so the workflow still needs a control that will alert on deduplicated retries and post-commit replay attempts.

C:

By itself, this improves tail performance monitoring, but the case is not asking the architect to optimize that dimension. The requirement is production side-effect integrity monitoring, supported by an approach to alert on deduplicated retries and post-commit replay attempts.

Claude Certified Architect - Professional (CCAR-P) Practice Tests