When a human employee is performing poorly, you usually notice. Their outputs are late, their quality is off, they ask more questions than they used to. An AI agent in production does not give you those visible signals. It either runs quietly and correctly, or it runs quietly and incorrectly. The monitoring layer is what makes silent failure visible before it becomes a problem.
This post covers the three metrics that matter most in the first 30 to 60 days of an agent deployment, how to interpret them, and what each one tells you about whether your configuration needs adjustment. The context is operational agents: finance, ops, sales support, and similar workflows at growing companies. Not research agents or code generation pipelines, which have different monitoring requirements.
Metric 1: Action count and action count rate
The most basic signal is how many actions your agent is taking per day, and whether that number is consistent. Action count matters because it tells you whether the agent is processing the volume of work it should be handling. If your finance agent is configured to process incoming invoices and your company receives 20 invoices per week, you should see a corresponding number of agent actions against those invoices each week. If action count is consistently lower than expected, the agent may be missing inputs, silently failing on some tasks, or escalating to a human queue that isn't being reviewed promptly.
Action count rate is the second dimension: is the rate stable over time, or is it drifting? A decreasing action rate on a stable workflow can indicate an integration issue (the agent is no longer receiving inputs), a scope misconfiguration (conditions that used to trigger agent actions have changed), or a growing backlog in a review queue that is blocking autonomous actions. An increasing action rate with no corresponding increase in actual workflow volume can indicate the agent is processing items it should be skipping, which suggests a scope expansion you didn't intend.
The baseline for normal action count should be established in the first two weeks, during the period when you are reviewing agent outputs closely. Once you have a baseline, deviations from it are your primary signal that something has changed.
Metric 2: Escalation rate and escalation resolution time
Escalation rate is the percentage of processed items that the agent routes to a human rather than handling autonomously. This metric has two failure directions, and each tells you something different.
An escalation rate that is too high means the agent is not confident enough in its scope to handle the cases you expected it to handle. This can indicate that the agent's confidence thresholds are set too conservatively, that the input data quality is lower than expected, or that the task variety in your workflow is greater than the scope definition anticipated. A finance agent with a 40 percent escalation rate is not providing much value over a human doing the work manually, because someone still has to review nearly half the items.
An escalation rate that is too low is harder to interpret but can be a warning sign. If your agent is handling 98 percent of items autonomously in week one, either your workflows are unusually clean and consistent (possible) or the agent's scope thresholds are set too permissively and it is handling cases it should be escalating (more common). Reviewing a random sample of the autonomously handled items is the fastest way to diagnose which situation you are in.
The right escalation rate depends on your workflow and your risk tolerance. As a rough internal benchmark from our early-access teams, a well-configured finance or ops agent in week one typically escalates somewhere between 10 and 25 percent of items. That range decreases over the first month as scope is calibrated and as the agent benefits from cleaner data.
Escalation resolution time is how long it takes a human to respond to an escalated item. This metric is about the human side of the system, not the agent side. If escalations are sitting in a queue for 12 hours before anyone reviews them, your effective agent workflow velocity is limited by human review speed, not by the agent. Monitoring this metric tells you whether the escalation path you designed is actually fast enough for your workflow's time requirements.
Metric 3: Action latency
Action latency is how long it takes the agent to process a single item from input received to action taken or escalation triggered. This metric is less important than the other two for most operational workflows, because a finance agent that takes 90 seconds to process an invoice is still far faster than a human doing the same task. But latency becomes important in two situations.
First, if latency is increasing over time on a stable workflow, it can indicate a performance problem in the underlying integration or in the agent's reasoning process. An agent that took 15 seconds per item in week one and is taking 45 seconds per item in week four without any change in task complexity has a performance regression somewhere in the stack.
Second, for time-sensitive workflows (customer support, real-time request routing), latency is a first-class concern. An agent that takes two minutes to classify and route an inbound support ticket is not fast enough if your team's SLA requires response within 10 minutes. In these cases, latency should be monitored against the workflow's time constraint and should trigger an alert if it exceeds a threshold.
What these three metrics do not tell you
These metrics tell you whether the agent is running, whether it is confident enough in its scope, and whether it is fast enough. They do not tell you whether the outputs it is producing are correct. That determination requires human review of a sample of outputs, especially during the first month of operation.
Output quality review and quantitative metrics work together as a monitoring system. The quantitative metrics flag that something has changed (action count drop, escalation rate spike). The output review tells you what specifically has changed and whether it constitutes a problem. Neither is sufficient alone.
We are also not claiming these are the only metrics worth tracking. Depending on your workflow, you might also care about the distribution of action types the agent is taking, the breakdown of escalation reasons, or the time of day when escalation rate is highest (which can reveal patterns in input data quality that vary by time). The three metrics above are the starting point: the minimum viable monitoring setup for a new deployment. More detailed metrics become useful once you have a baseline from the first few weeks and have a specific question you are trying to answer.
