The hardest incident in an AI application is often one in which every component seems to be running. Containers are healthy, the database responds, and the model endpoint answers test requests. Yet customers report that some responses take thirty seconds, while others appear almost instantly. Microsoft AI-200 treats monitoring and troubleshooting as backend skills because a distributed application can be healthy component by component and still fail as a system. The tools differ from those used in a general introduction to Microsoft certifications, but the underlying responsibility is familiar: collect evidence, isolate the bottleneck and verify a fix.
Suppose a support assistant begins timing out every Monday morning. The API layer reports HTTP 504 errors, workers are restarting, and a large share of the database queries have become slow. Guessing that the model is overloaded might lead to buying more inference capacity, but that would accomplish little if the real bottleneck is a database connection pool exhausted by retrying workers. Logs, metrics and traces each reveal a different part of this problem. Kusto Query Language, or KQL, helps turn collected telemetry into answers that operators can test.
Choose the signal that answers the operational question
Metrics summarize measured quantities over time, such as CPU usage, request rate, queue depth or a dependency’s latency. They are efficient for spotting trends, setting thresholds and comparing current behavior with a baseline. Logs represent records of events or observations with associated fields, and can support detailed filtering and correlation. Distributed traces show related operations across services, helping explain where a particular request spent its time. None is a substitute for the others in every troubleshooting situation.
A high average CPU value does not prove a service’s users are unhappy, and a low average CPU value does not prove requests are fast. A small number of very slow operations can disappear in a mean value while dominating customers’ experience. Look at latency distributions such as p95 or p99 where relevant, request volume and error classifications, not only one headline average. Similarly, an increasing queue depth can indicate an intake burst, insufficient worker capacity or a poisoned retry loop. The metric is a starting point for investigation.
Application Insights and Azure Monitor can provide application-oriented telemetry depending on instrumentation and configuration. Log Analytics gives a way to query collected Azure Monitor Logs with KQL. OpenTelemetry SDKs help instrument distributed systems and propagate trace context across service boundaries. The service names matter, but the diagnostic objective matters more: a team needs to know which request failed, where its work traveled and what evidence distinguishes a code defect from a dependency outage.
Instrumentation must be intentional. If the HTTP API emits a trace identifier but its background queue consumer starts an unrelated trace without linking the job, the operator cannot reconstruct the full workflow easily. Correlation IDs should pass through request headers, messages and durable job records as appropriate. A tracing library cannot invent a business correlation that the application never supplies. When an operation becomes asynchronous, its relationship to the initiating request should remain visible in the diagnostic model.
Telemetry must be safe. A prompt or retrieved document can include private customer information; a raw request header can carry credentials. Avoid logging unredacted tokens, full documents or other sensitive material merely because debugging is easier with more data. Design useful fields such as tenant-safe identifiers, operation names, resource classes, outcome and timing. Protect the monitoring workspace and define retention rather than treating logs as a second uncontrolled copy of the production database.
Use KQL to test a hypothesis, not decorate a dashboard
KQL queries operate on tables of structured records. A practical investigation usually begins with a time filter, then narrows by service, operation or correlation value, and finally summarizes the behavior of interest. For example, the shape SomeTable | where TimeGenerated > ago(1h) | summarize count() by ResultType illustrates the pipe-oriented flow. The exact table and field names depend on the data source and workspace schema, so a copied query must be adapted rather than assumed to work unchanged everywhere.
Filtering early matters for both relevance and efficiency. A failed-request investigation might start with the past thirty minutes and the deployment environment where customers observed the problem. Mixing staging, production and unrelated historical data can distort the picture. A service that has failed a hundred requests may be improving if the requests occurred during an earlier outage; a chart without a timeline cannot tell the story.
Aggregation allows comparison. A query that groups by application version, response classification or operation can reveal that one recently deployed endpoint is responsible for most errors. Operators can compare the same request path before and after a release or examine which tenant cohorts were affected. Avoid interpreting a small sample as a stable trend: one slow request among two requests is not the same operational signal as thousands of sustained slow responses.
Joins and correlations become useful when data from different stages must be compared, but they also require care. A message-processing log and an HTTP request log should be joined on a reliable identifier, not merely on a similar timestamp. Two independent requests can occur within the same second. If a correlation field changes format across services or is reused incorrectly, the joined timeline can become convincing but false. Validate the correlation contract before building advanced troubleshooting queries around it.
KQL is an investigative language as well as an alerting tool. A saved query may help operators investigate common incidents, but a query that worked once is not automatically a meaningful alert. Alerts require a suitable evaluation interval, threshold, noise policy and response owner. Before promoting a query into automation, ask what action the team will take when it fires and what a false positive would cost.
Follow one request through a failing backend
Imagine a user asks an assistant to summarize their recent orders. The API authenticates the request, queries a document store, retrieves policy information and calls an inference service. Total latency rises from two seconds to fifteen. Start with end-to-end request duration and the proportion of failed requests; then inspect trace spans for the slowest stage. If database calls suddenly dominate, buying a faster model endpoint would not solve the bottleneck.
The database may be slow because queries changed shape, an index is missing, a partition receives disproportionate traffic or a connection pool has too many concurrent users. Telemetry from the application should identify query classes and durations without recording confidential payloads. Database metrics can show resource pressure and rejected operations. A broad ‘database unhealthy’ alert is less useful than evidence showing whether the problem is query design, capacity, connectivity or throttling.
If inference spans are slow, inspect request sizes, provider errors, queueing and configured concurrency. A change that attaches ten times more retrieved context to each prompt may increase latency and costs even if model capacity has not changed. A burst of retries can amplify the effect. Tie model request metrics to the application operation that caused them; a token-cost chart by itself rarely reveals which product feature is responsible.
If message consumers are slow, compare oldest-message age, processing duration and dead-letter activity. Retries caused by a permanently invalid input can consume worker capacity that should serve valid jobs. A queue’s reported depth may decrease after deleting failed messages, but deleting them may violate the business workflow. Investigate why they could not be processed and determine the correct recovery action before clearing the symptom.
Look for phase changes near deployment events. A new container image may have changed dependency versions, request serialization or timeout settings. Correlate errors with version identifiers and rollout timestamps. If a rollback improves latency, that is useful evidence, but still identify the defect before repeating the deployment. Successful rollback is an incident mitigation, not a root-cause explanation.
Debug container, network and dependency layers separately
A container may pass readiness checks while its application is unable to reach a downstream dependency. Readiness probes should reflect the intended operational contract without creating excessive load on those dependencies. Logs from Azure Container Apps or AKS, container startup events and application traces can reveal different issues: image-pull failures, crash loops, DNS failures, permission problems or ordinary API errors. The label ‘container failure’ is too vague to guide the response.
Network symptoms require evidence. A connection timeout can arise from routing, firewall rules, DNS resolution, exhausted outbound connections or a dependency that accepts connections slowly. Credential failures usually surface differently from a missing route, though some client libraries obscure error details. Preserve the underlying error classification and dependency endpoint in safe logs. Increasing the API timeout should be a measured decision, not the default repair for every connection failure.
Identity and configuration changes can resemble application regressions. A rotated secret that never reached a running replica may cause only part of the deployment to fail. A missing role assignment can prevent one worker identity from accessing Key Vault while another still succeeds. Traces should identify the failing operation without exposing the secret or access token. If operators resolve each issue by granting broad permissions, a monitoring incident can create a lasting security incident.
Azure Monitor may also collect infrastructure signals relevant to container capacity, memory pressure and restarts. Use those alongside the application request path. If high memory triggers container eviction, an application-level retry storm may be a cause rather than an independent problem. Troubleshooting should connect the layers instead of assigning blame to whichever dashboard first turns red.
The operational practices here extend beyond AI-200. Microsoft AZ-104 includes Azure administration and monitoring responsibilities, while AI-200 places them in the context of SDK-based backends, containers, messaging and AI data services. Both domains reward systematic narrowing of hypotheses and sensible responses to real evidence.
Design alerts around consequences and recovery
An alert becomes useful when it identifies a condition that needs an action. High request latency combined with a sustained error rate might require investigation. An occasional retryable 429 response may be routine and handled by the client within a supported budget. A dead-letter queue that grows for two hours is often more urgent than a single transient function failure. Thresholds should reflect service-level objectives, traffic patterns and a baseline of normal behavior.
For a customer-facing AI assistant, reasonable objectives could include successful authorized requests, upper-percentile response time, retrieval correctness and completion time for asynchronous jobs. Define what counts as success: returning an HTTP 200 with an unrelated answer is not a good outcome. Infrastructure telemetry helps explain why the service is underperforming, but business outcome measures tell operators whether the problem matters.
Avoid creating alert storms in which ten dependent services notify separate teams about one underlying incident. Correlation, suppression and runbooks can reduce unnecessary pages while preserving visibility. An operator needs a concise starting question: is the incident isolated to one service version, one region, one tenant, one data dependency or the entire inference integration? Dashboards and queries should be organized around those decisions.
After an incident, add regression evidence. Reproduce the failure in a safe environment if possible, test the corrected timeout, index or configuration, and measure the effect under representative load. Record why the original alerts did or did not detect the problem. Improving observability is not only adding another chart; it is reducing the time between a user-visible fault and an accurate operational decision.
Practice AI-200 observability as an investigation
Take a hypothetical pipeline—API, Cosmos DB or PostgreSQL, event queue, container worker and inference endpoint—and invent three symptoms: rising API latency, stalled background jobs and intermittent secret-access errors. For each symptom, name the first metric or log query you would examine, the correlation field you need, and the evidence that would reject your initial hypothesis. If your plan is simply to look for errors everywhere, the investigation is not yet designed.
Then decide what must be instrumented before deployment. Record version and operation information, establish trace propagation, define safe logging boundaries and connect important alerts to people who can repair the relevant service. KQL can provide powerful analysis only when the telemetry has a meaningful schema. AI-200 readiness means understanding that observability is part of architecture, not a dashboard installed after customers begin reporting failures.