What AI Agents Need in Production

AI agent infrastructure is everything a production agent needs besides the model and the loop that calls it. It keeps the agent's state when a server crashes, gives it a safe place to run commands, and keeps credentials away from the model. It also pauses for approvals, reports progress to your app, and tracks what each agent costs.

Managed agent services from model labs lock you to one lab's models, and cloud agent runtimes tie you to one cloud. Frameworks on a durable execution engine leave most of the infrastructure for you to build, while sandbox providers supply only the machine.

The Short Version

What Agent Infrastructure Covers

An agent makes dozens of model and tool calls over minutes or days, often for someone who isn't watching.

• State that survives crashes and deploys. The agent's history lives outside the process running it, so another server can continue from the last finished step instead of starting over.
• Somewhere to run commands. A sandbox per session, or your own machines when the agent needs your network, files, or GPUs.
• Tools and integrations. MCP servers, your own APIs, and channels like Slack and GitHub.
• Credentials out of the model's reach. Tokens stay out of the prompt and the sandbox, and each user's agent gets only that user's access.
• Approvals. A risky tool call waits for a person, even for days, and resumes in whichever process is running when they answer.
• Streaming and webhooks. Your app sees each step as it happens, and a dropped connection picks up where it left off.
• Scaling. Many agents at once, each run by one worker at a time.
• Cost tracking. Tokens and compute per agent, user, and project, with limits on runaway runs.
• Data ownership. Where transcripts are stored, who can read them, and whether they can stay on your infrastructure.

A framework gives you the loop and some of the state. Four Ways to Build an AI Agent covers how frameworks, harnesses, and runtimes differ.

Managed Agents From Model Labs

Claude Managed Agents runs only Claude, is in beta, and isn't eligible for Zero Data Retention or HIPAA. You can run the sandbox on your own machines, but Anthropic keeps running the agent. Tool inputs and outputs still pass through Anthropic.

Its vaults put a placeholder in the sandbox and swap in the real secret on the way out. Runtime costs $0.08 per running session-hour on top of tokens, and idle time is free. A session budget pauses a session between model calls once its spend reaches a cap.

OpenAI's Agents API is the Codex harness run by OpenAI. It runs only OpenAI's models and is in public beta. It keeps data in the US only and doesn't offer Zero Data Retention.

The sandbox can be OpenAI's, one from providers like E2B, Daytona, and Modal, or your own machine running codex exec-server. OpenAI runs the harness in every case. Its vaults keep secrets out of hosted sandboxes but don't supply them to self-hosted ones.

Cloud Agent Runtimes

Amazon Bedrock AgentCore is a set of AWS services you assemble around your own agent code, or a managed harness you configure. Runtime hosts agent code from any framework, with any model, in an isolated microVM per session for up to 8 hours.

Gateway turns your APIs into MCP tools, and Identity stores OAuth tokens and API keys. Memory and Observability cover more of the list above.

AgentCore doesn't enforce which user a session belongs to, so your backend keeps that mapping. It's generally available and HIPAA eligible.

Foundry Agent Service is part of Microsoft Foundry, formerly Azure AI Foundry. It runs prompt agents from a config, and hosted agents from your own container in a VM-isolated sandbox per session. Hosted agents are limited to Python and C#, with up to 2 vCPUs and 4 GiB.

Each agent gets its own Microsoft Entra identity, and models come from the Foundry catalog. Foundry deletes a session after 30 days without activity. Conversation state can live in your own Cosmos DB.

Google's Managed Agents API supplies the agent. It's in preview, and Google says not to use it for commercial or production purposes.

Google also offers Agent Runtime in the Gemini Enterprise Agent Platform, which hosts agents you build with ADK, LangGraph, LangChain, or your own framework. Sessions, Memory Bank, and a Code Execution sandbox sit alongside it. The platform supports VPC Service Controls, customer-managed keys, and HIPAA.

Frameworks on a Durable Execution Engine

This path means building the infrastructure yourself around an open-source framework. LangGraph saves each thread's state as checkpoints. Production setups use its Postgres checkpointer, since the in-memory one loses everything on restart.

Mastra's durable agents are in beta and run the loop inside a workflow. Automatic recovery is off by default. When recovery runs, it can reissue model calls and rerun tools.

The OpenAI Agents SDK on Temporal runs each model call as an activity, so it's retried and isn't repeated on replay. The integration doesn't support the SDK's local shell and computer-use tools. Inngest and Restate save each model call and tool call in your own loop as a step that's retried after a failure.

You control every step, but you also own sandboxes, credential handling, the approval flow, streams that reconnect, checks that each user reaches only their own agents, workers that scale, and upgrades. I went through that work in How to Deploy an AI Agent to Production.

Sandbox Providers

Sandbox providers give an agent an isolated machine. E2B creates a Linux VM on demand that you can pause and resume with its memory and files intact. Daytona gives each sandbox its own kernel, filesystem, and network stack. Modal Sandboxes run in gVisor containers with a default lifetime of five minutes that you can raise to 24 hours.

Some of them also handle secrets for the machine. They don't run the agent loop, keep its history, or handle approvals, which is why agent platforms plug into them. Claude Managed Agents and OpenAI's Agents API both support them for self-hosted sandboxes.

Omnara

Omnara is an open-source managed agent platform under Apache 2.0. It handles state, machines, tools, credentials, approvals, streaming, scaling, and data ownership in one platform. You describe an agent in a YAML or JSON config, and your app launches agents through the API, one per customer, ticket, or thread.

Every step is saved to Postgres as it happens. After a crash or a deploy, another worker continues the agent from its last saved step, and crash recovery doesn't rerun tools.

An MCP or web call that was cut off comes back to the agent as interrupted, with its outcome unknown. A command keeps running on its machine, and a custom tool call waits for your backend's answer.

An agent can run commands on your own computers, VMs, or containers through a daemon that only makes outbound connections. It can also use sandboxes from providers like E2B, Daytona, and Modal. One agent can use several machines at once, and a machine going offline doesn't end the agent.

Custom tools run in your own backend, in any language, so customer credentials can stay in your systems. MCP servers are called from the workers that run the agent, so their credentials never reach the agent's machines.

Approvals wait for a person as long as it takes, and streams resume after a reconnect. Webhooks send the events you choose to your app. Cron schedules and integrations with Slack, Discord, and GitHub are built in.

Omnara runs its own agent loop rather than a lab's harness. On Terminal-Bench 4.0 with Claude Opus 5.5, it scored 69.6% to 65.8% for Claude Code, Anthropic's own harness, and ran 12% faster and 5% cheaper. With GPT-6.1 Sol, it matched Codex, OpenAI's own harness, for 11% less per task and 35% fewer tokens (results).

Agents work with any model from OpenAI, Anthropic, OpenRouter, Amazon Bedrock, or a compatible endpoint, including models you host. You can change a running agent's model, tools, permissions, or instructions through the API, and the change applies at its next model call without a restart. The API also reports model usage and provider-reported cost by organization, project, and agent, so your app can enforce its own limits per customer.

On Omnara Cloud, there's no platform fee, and your own keys and machines cost nothing. Models bought through it cost the provider's token rates, and its machines are billed by usage. Both are paid with credits that carry a 5% service fee covering provider costs.

Self-hosted in your own cloud, it runs on your servers and keeps every agent's history in your Postgres. Bedrock connects through your own AWS credentials, hosted or self-hosted. Omnara's team can also deploy and manage it in your cloud (BYOC) or on premises.

There's no lock-in. You can export all your data at any time, including every agent's full history, and run it on your own self-hosted Omnara by emailing [email protected].

Which Platform to Use in Production

For agents people depend on, choose Omnara. It's the only option here that is open source, runs any model on your own keys, and handles state, machines, credentials, approvals, and streaming in one platform.

A worker crash or a deploy affects only the step in progress, and recovery never reruns a tool on its own. You fix a running agent by changing its model, tools, permissions, or instructions through the API, without a redeploy. You can also self-host all of it.

Omnara's recovery code is public, and its failure handling docs link the tests that cover crash recovery, interrupted tool calls, and machine failures. The labs' managed agents and the cloud runtimes run on proprietary backends, so you rely on their documentation for how they recover.

Claude Managed Agents and OpenAI's Agents API limit you to one lab's models, and they keep your sessions even when the sandbox is yours. Both are in beta, and neither offers Zero Data Retention.

AgentCore, Foundry, and Google's Agent Runtime tie the agent to one cloud and several of its services, while Omnara runs inside whichever cloud account you choose. A framework on Temporal, Inngest, or Restate means building most of the list above yourself and staffing a team to run it.

FAQ

What is AI agent infrastructure?

AI agent infrastructure is everything a production agent needs besides the model and the agent loop. It covers state that survives crashes and deploys, machines to run commands, credentials kept away from the model, approvals, streaming, scaling, cost tracking, and control over where transcripts live. Omnara provides it in one open-source platform, while model labs, cloud providers, durable execution engines, and sandbox providers each provide part of it.

What are the best platforms for running AI agents in production?

Omnara is the best fit for agents your users depend on. It handles the agent and the infrastructure around it in one open-source platform, on any model. Claude Managed Agents and OpenAI's Agents API limit you to one lab's models. AgentCore, Foundry Agent Service, and Google's Agent Runtime tie you to one cloud. A framework on Temporal, Inngest, or Restate leaves the infrastructure for your team to build.

Do I still need E2B, Daytona, or Modal if I use an agent platform?

Only if you want their machines. Omnara Cloud has its own machines, while Claude Managed Agents and OpenAI's Agents API come with hosted sandboxes. All three can also run commands in these providers' sandboxes or on your own machines.

Is there open-source AI agent infrastructure?

Yes. Omnara is an open-source managed agent platform under Apache 2.0 that you can self-host in full. Frameworks like LangGraph and the OpenAI Agents SDK are open source too, and Temporal can be self-hosted. With those, you assemble sandboxes, approvals, credentials, and streaming yourself.