Before Your Team Ships an AI Agent, Ask These Questions
The demo worked. The stakeholders were impressed. The sprint was a success. Six months later the system is sitting in a staging environment, consuming cloud budget, and the CTO is asking the engineering team why it never made it to production.
This is not an unusual story. According to IDC research, 88% of AI agent proof-of-concepts never graduate to production deployment. Deloitte's technology trends report confirms an 89% pilot-to-production failure rate across enterprise environments. Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027.
The failure is not rooted in technical incompetence. It is rooted in organization's approaching agentic AI as if it is just another software deployment. It is not.
The agents that survive production the ones that actually deliver the operational value that justified building them share a common characteristic. Before they were shipped, someone on the engineering team asked hard questions and got honest answers. Not the questions the vendor demo answers. The questions that reveal whether the system will hold up when real users stress-test it, when edge cases appear that the prototype never encountered, and when the business requirement changes three weeks after launch.
This blog is that set of questions. It is written for the CTO and VP of Engineering who are evaluating an agentic AI system before it ships or inheriting one that was built without this evaluation and need a clear framework for deciding what is actually ready for production and what is not.
Why Evaluating Agentic AI Is Fundamentally Different
Before getting into the questions themselves, it is worth being precise about why evaluating an AI agent requires a different mindset than evaluating traditional enterprise software.
Evaluating AI agents in production is fundamentally different from anything done in traditional software development. Agents are not deterministic systems. They do not execute predefined logic. A traditional software system given the same input produces the same output every time. An AI agent given the same input may reason differently depending on the context window state, the tools available, the external APIs it queries, and factors that are genuinely difficult to predict or reproduce.
This non-determinism has a direct implication for how evaluation must work. Unit tests that pass in a staging environment do not guarantee the agent behaves correctly in production. A demo that runs flawlessly is not evidence of production readiness. A stronger model on wrong context produces outputs that are coherent, well-reasoned, and convincing which is precisely what makes them dangerous when they are wrong.
The evaluation questions below are structured around this reality. They are not asking whether the agent works in controlled conditions. They are asking whether it will hold up when conditions are not controlled which is the only environment that matters.
Section 1: Goal and Scope Clarity
Question 1: Can you state the agent's goal in one sentence without using the word "AI"?
This is the first filter and the most revealing one. Agentic AI systems that are built around a clear, bounded outcome "process incoming support tickets, categories them by urgency, and route them to the correct team without human intervention" have a defined success criterion that can actually be measured. Agentic AI systems built around a general capability "help the operations team be more efficient" have no such criterion.
The primary driver of agentic AI failure is not technical incompetence but a lack of structural governance. When organization's rush to implement AI agents without a mature framework, they expose themselves to operational and existential risks.
A goal that cannot be stated clearly cannot be evaluated clearly, and a system that cannot be clearly evaluated cannot be trusted in production. If your engineering team cannot produce the one-sentence goal statement, the scoping work has not been completed and the build should not start.
Question 2: What is the agent authorized to do, and what is it explicitly not authorized to do?
The permission boundary of an AI agent is an engineering decision, not a product decision, and it must be made explicitly before deployment. The failure mode is agents running with overly broad credentials because it was the path of least resistance during deployment. An agent with write access to every system it touches because it might need it is a significant risk surface.
Every action the agent can take every API it can call, every record it can write, every message it can send represents a permission that was granted, and permissions that are granted cannot be assumed to be safe simply because the agent was not intended to misuse them. Evaluating the permission boundary means listing every action the agent is technically capable of taking and verifying that each one is intentional and appropriate for the task scope.
Question 3: What does the agent do when it encounters a situation outside its defined scope?
Agents that do not have explicit error handling and fallback logic will improvise and improvised agent behavior in edge cases is unpredictable. Graceful degradation design means the agent knows when to stop, ask for help, or hand off to a human rather than attempting to continue when it is operating outside its confidence envelope.
The answer to this question should be a specific, designed behavior not "it should handle it correctly" or "the model will figure it out." If the engineering team cannot describe exactly what happens when the agent hits an unexpected input, a failing API, a missing data field, or an ambiguous instruction, the error handling has not been designed. It will be discovered in production through failures.
Section 2: Data Readiness
Question 4: What data does the agent depend on, and is that data clean, current, and consistently structured?
Nearly two-thirds of enterprises worldwide have experimented with agents, but fewer than 10% have scaled them to deliver tangible value. Shaky data is often to blame eight in ten companies cite data limitations as a roadblock to scaling agentic AI.
Data readiness is the most consistently underestimated prerequisite in enterprise agentic AI deployment. An agent that retrieves information from a database to make decisions is only as reliable as the quality and consistency of that database. Inconsistent field formats, stale records, missing values, and siloed systems that do not reflect the same state are not problems the agent will compensate for they are problems the agent will act on, producing confident outputs grounded in incorrect information.
The data readiness evaluation for an agentic system needs to cover four things: data quality in the systems the agent reads from, update frequency relative to how time-sensitive the agent's decisions are, structural consistency across the data sources the agent integrates, and access governance to ensure the agent's read and write permissions align with data sensitivity.
Question 5: What happens when the agent's data source returns an unexpected value or becomes unavailable?
This question tests whether the data dependency has been engineered for or assumed. An agent that makes a routing decision based on a field that is sometimes null needs explicit logic for the null case. An agent that queries an external API that has an SLA of 99.5% uptime will encounter that 0.5% in production, and what happens in that moment needs to have been designed not discovered.
The evaluation here is straightforward: take the agent's data dependencies one by one and ask what the designed behavior is when each one fails or returns unexpected values. If any of those answers is "the model should handle it", the dependency has not been engineered.
Section 3: Reasoning Quality and Reliability
Question 6: How is the agent's reasoning quality being measured, not just its output?
Evaluation should extend to reasoning quality, not just outputs. That is where enterprise trust in agents is built.
Output evaluation checking whether the agent produced the correct final answer is necessary but not sufficient. An agent that produces the correct output through incorrect reasoning is brittle: change the input slightly and the reasoning breaks in ways that the output-level evaluation would not have caught.
Reasoning quality evaluation means inspecting the agent's decision trace the sequence of reasoning steps, tool calls, and intermediate conclusions that led to the output and verifying that the reasoning path is sound, not just that the endpoint was correct. This requires observability tooling that captures the full reasoning trace, not just the final output.
Question 7: What is the agent's performance on cases it has not seen before?
Demo environments are built on cases the system was tuned to handle. Production environments contain cases it has never encountered. The gap between these two sets is where most agentic AI systems fail first.
Evaluating generalization requires a deliberate adversarial testing process: constructing inputs that are plausible in production but were not part of the development or tuning dataset, and measuring the agent's performance on them. The failure modes to look for are not just wrong answers but confident wrong answers cases where the agent produces an output with apparent certainty that is factually or logically incorrect.
Confident wrong answers passing surface-level review without being caught mean the agent is producing outputs grounded in statistical approximations rather than governed business definitions. That is a production risk that no amount of output-level testing will catch.
Section 4: Security and Permission Architecture
Question 8: Has the agent been evaluated specifically for prompt injection vulnerabilities?
Prompt injection is the most prevalent security risk in deployed agentic systems and the one most frequently absent from pre-deployment security reviews. It occurs when an agent that processes external content reading documents, browsing web pages, handling user messages encounters content that has been crafted to manipulate the agent's instructions. The agent, unable to distinguish between its actual instructions and instructions embedded in the content it is processing, follows the injected instructions instead.
For enterprise systems, prompt injection is not a theoretical risk. An agent that processes incoming emails as part of a workflow can be manipulated by a carefully constructed email. An agent that summarizes documents can be manipulated by content embedded in a document it retrieves. The evaluation question is not whether prompt injection is possible in principle it almost always is but whether the system has been designed with defences: instruction isolation, output validation, and human review triggers for actions that cross defined risk thresholds.
Question 9: What is the audit trail for every action the agent takes?
Governed workflows where every step is logged and verifiable are the pattern that distinguishes successful enterprise agentic deployments from the ones that fail.
In regulated industries, the audit requirement is a compliance mandate. In every industry, it is an operational necessity. When an agent takes an action that produces an unexpected outcome, the engineering team needs to be able to reconstruct exactly what the agent perceived, what it reasoned, what tools it called, what those tools returned, and what decision it made at each step. A system without this trail cannot be debugged, cannot be improved, and cannot be trusted to operate at scale.
The evaluation here is binary: either a complete, inspectable audit trail exists for every agent action, or it does not. Partial logging capturing some steps but not others is operationally equivalent to no logging when the step that produced the failure is the one that was not captured.
Section 5: Human Oversight and Escalation Design
Question 10: At what point does the agent stop and escalate to a human, and has that threshold been tested?
The escalation threshold is one of the most critical design decisions in an agentic system and one of the most frequently underspecified. Every production agentic system needs a defined set of conditions under which it stops acting autonomously and routes to a human whether because the confidence of its reasoning falls below a threshold, because the action it is about to take exceeds a defined risk level, or because it has encountered a case type that has been flagged for human review.
The threshold itself is a design decision that requires input from the business, the engineering team, and whoever owns the operational risk of the system's actions. The evaluation question is whether that threshold has been defined explicitly, implemented in code, and tested with cases designed to trigger it.
An escalation design that exists only in documentation but has never been verified to trigger correctly in the actual system is not a functioning escalation design.
Question 11: How does a human reviewer understand what the agent did and why, well enough to make a meaningful correction?
Human-in-the-loop is only meaningful if the human has enough context to intervene effectively. An escalation that presents a human reviewer with the agent's final output but not its reasoning chain, the tools it called, or the data it used to reach its conclusion does not enable meaningful oversight it enables rubber-stamping.
Effective human oversight in an agentic system requires that the escalation interface surfaces the full decision context: what the agent was trying to do, what information it gathered, what reasoning it applied, and what action it was about to take. The reviewer needs to be able to agree, disagree, or correct the agent's reasoning not just its output.
Section 6: Operational Sustainability
Question 12: How will you know when the agent's performance has degraded, and what triggers a review?
AI agents are not static they evolve with new data, tasks, and contexts. Enterprises that treat them as one-off deployments risk drift, errors, and erosion of trust. A mature approach includes feedback loops between business users and engineering teams, ongoing monitoring for edge cases and unexpected behavior, and regular updates to prompts, tools, and governance policies.
Performance degradation in agentic systems is rarely sudden and visible. It is gradual and subtle: the agent's accuracy on a class of inputs drifts as the underlying data changes, the tools it depends on evolve in ways that break assumptions embedded in the agent's instructions, or the distribution of inputs it receives in production shifts away from what it was optimized for during development.
Detecting degradation requires monitoring at the reasoning level, not just the output level. Metrics to track include task completion rate, escalation frequency over time, human correction rate on escalated cases, and tool call failure rate. A change in any of these metrics over time is a signal that something in the agent's environment or performance has shifted and needs investigation.
Question 13: What is the cost model for running this agent in production, and has it been validated under realistic load?
Production means highly concurrent user requests and edge cases the team never anticipated. A pilot that runs flawlessly masks fundamental issues: latency under load and broken API handoffs.
Token consumption in agentic systems scales with task complexity in ways that are difficult to predict from a prototype. An agent that handles a simple task with a short reasoning chain in development may handle a complex task in production with a reasoning chain ten times longer and token costs that scale accordingly. Load testing at realistic concurrency levels, with realistic task complexity distributions, is not an optimization exercise. It is a prerequisite for understanding whether the system is economically viable at the scale at which it will actually operate.
How Levrez Technologies Approaches Agentic AI Deployment
At Levrez technologies, every agentic AI system we build goes through this evaluation framework before it ships not as a final review, but as a design input. The checklist questions inform architecture decisions: how permissions are scoped, how the error handling is designed, how observability is built in from the start rather than added after the first production incident.
For enterprise teams that have built agentic systems without this framework and are trying to understand whether they are production ready, we offer an independent evaluation engagement: a structured review against this checklist that identifies the specific gaps and the work required to address them.
The goal is not to find reasons not to ship. The goal is to find the gaps before production does because the cost of finding them in production is always higher than the cost of finding them in evaluation.
If your team is building or evaluating an agentic AI system and you want an engineering perspective on where it stands, that conversation starts with the checklist.
Frequently Asked Questions
1. What is the biggest reason enterprise agentic AI projects fail? The most consistent root cause is not a technical failure it is a governance and scoping failure. Systems are deployed without a clear goal, without defined permission boundaries, and without the monitoring infrastructure to detect when something goes wrong. The technology works; the organizational framework around it does not.
2. How is evaluating an AI agent different from evaluating traditional software? Traditional software is deterministic the same input produces the same output every time. AI agents are non-deterministic, meaning their behavior depends on reasoning that can vary with context. This means unit tests that pass do not guarantee production behavior, and evaluation must include adversarial testing, reasoning trace inspection, and performance measurement under realistic load not just output correctness in controlled conditions.
3. What is prompt injection and why does it matter for enterprise agents? Prompt injection is an attack where malicious instructions are embedded in content that an agent processes a document, an email, a web page causing the agent to follow those instructions instead of its designed behavior. It is the most prevalent security risk in deployed agentic systems and is frequently absent from pre-deployment security reviews. Any agent that processes external content must be evaluated specifically for this vulnerability.
4. What does "human-in-the-loop" actually require to work correctly? Effective human oversight requires that the escalation interface surfaces the full decision context what the agent perceived, what it reasoned, what tools it called, and what action it was about to take. An escalation that presents only the final output does not enable meaningful correction. It enables uninformed approval, which is not oversight.
5. How often should a production agentic AI system be reviewed after deployment? There is no universal frequency the right review cadence depends on how rapidly the agent's data environment and input distribution change. The minimum requirement is reasoning-level monitoring with defined alert thresholds that trigger a review when metrics shift. Treating a deployed agent as a static system that does not need ongoing attention is the most reliable path to gradual performance degradation that only becomes visible after it has caused operational damage.
6. What services does Levrez offer for enterprise agentic AI projects? Levrez provides end-to-end services for enterprise agentic AI programmes including architecture design, agent development, observability and monitoring infrastructure, security evaluation, and pre-deployment review against the framework outlined in this blog. For teams with existing systems, we offer independent evaluation engagements that identify production readiness gaps and scope the work required to address them.
Is your team preparing to ship an agentic AI system or evaluating one that was built without this framework? Connect with Levrez technologies for an engineering-led pre-deployment review we identify the gaps before production does.