Last month, our team hit a wall. We had three different AI agents running in production—one for infrastructure diagnostics, one for cost anomaly detection, and another for security alert triage. Each was built with a different framework by different teams at different times. Debugging them was a nightmare. Deploying updates required custom CI/CD pipelines for each one. And when an agent decided to call an internal API it shouldn't have, we had no centralized way to audit or control it.
I spent an entire weekend stitching together custom CloudFormation templates, Lambda authorizers, and CloudWatch dashboards just to get basic visibility. That's when I stumbled onto AWS Bedrock AgentCore. It promised to handle the exact mess I was dealing with: deployment, tool access control, observability, and scaling—all from one platform, regardless of the framework. After spending a few weeks putting it through its paces for our DevOps workflows, here's what I learned.
The Core Problem AgentCore Solves for DevOps
Building an agent that works on your laptop is easy. Getting it to production is where things fall apart. In a DevOps context, you need:
- Consistent deployment regardless of whether the agent uses LangChain, OpenAI Agents SDK, or something custom
- Strict access control for every tool and API the agent calls
- Full observability into what the agent did, why, and where it went wrong
- Iteration without rearchitecting when requirements change
AgentCore tackles these by providing a runtime, a gateway for tool connections, and built-in tracing—all tied together with a CLI that scaffolds and deploys your code.
Getting Started: The CLI Flow
First, install the CLI. It's distributed as an npm package, which surprised me—I expected a Python installer given the target audience, but it works fine.
npm install -g @aws/bedrock-agentcore-cli
You'll need Node.js 20+, Python 3.10+, and your AWS credentials configured (I use a named profile via ~/.aws/credentials).
Once installed, creating a project is straightforward:
agentcore init my-devops-agent
The CLI walks you through a few choices. The critical one is choosing between a managed harness or a code-based agent. The managed harness lets you declare everything in a config file—model, prompt, tools, memory—and AgentCore runs the loop for you. No framework, no orchestration code. It's the fastest path to a running agent.
I went with the code-based agent because our infrastructure diagnostic agent uses LangGraph, and I needed full control over the orchestration logic. The CLI scaffolded a project with the structure I needed, and I dropped in our existing agent code with minimal changes.
Testing Locally Before You Ship
This is where I made my first mistake. I skipped local testing and went straight to deploying. Don't do that.
agentcore dev
This runs your agent locally with the same runtime environment it'll use in production. I caught an import error and a misconfigured tool definition that would have cost me a deploy cycle. The local dev server also surfaces trace output in your terminal, so you can see the agent's reasoning steps as they happen.
For our DevOps agent, I tested it with a sample prompt: "Check the health of the production ECS cluster and report any tasks that are failing." Watching the trace output in real-time showed me exactly which API calls it was making and in what order. I spotted that it was calling DescribeTasks before ListTasks, which wasted API calls. I reordered the logic and retested in seconds.
Deploying to Production
When you're ready to deploy:
agentcore deploy
Behind the scenes, this uses AWS CDK to provision the infrastructure—ECS, Lambda, API Gateway, and all the IAM roles. The first deploy took about 8 minutes for me. Subsequent deploys were faster, around 2-3 minutes, since CDK only updates what changed.
One thing that caught me off guard: you need CDK bootstrapped in your target account. If you haven't done that yet:
cdk bootstrap aws://YOUR_ACCOUNT_ID/us-east-1
The deploy command handles the rest. Your agent gets an HTTPS endpoint you can invoke directly.
Invoking Your Deployed Agent
agentcore invoke --input "Check the health of the production ECS cluster"
Or via the HTTP endpoint:
curl -X POST https://your-agent-endpoint.agentcore.aws.amazon.com/invocations \
-H "Content-Type: application/json" \
-d '{"input": "Check the health of the production ECS cluster"}'
The response includes both the agent's output and a trace ID you can use to look up the full execution details later.
The Gateway: Controlling Tool Access
This is the feature that sold me. In DevOps, agents need access to sensitive systems—AWS APIs, internal dashboards, deployment pipelines. AgentCore Gateway lets you define tools and control exactly who can call what, with authentication and access control built in.
Instead of hardcoding API keys or IAM roles into your agent code, you define tools in the AgentCore configuration. The gateway handles authorization at the platform layer. When your agent needs to call an MCP server, a knowledge base, an internal API, and a Lambda function in the same turn, every connection goes through the gateway with proper auth.
For our security triage agent, I set up tools for querying CloudTrail logs, looking up Security Hub findings, and posting to our incident Slack channel. Each tool has its own permission boundary. The agent can read CloudTrail logs freely but needs explicit approval (human-in-the-loop) before posting to Slack. This kind of granular control was exactly what we were missing.
Observability: Debugging Non-Deterministic Failures
Agents are non-deterministic. The same input can produce different outputs on different runs. Traditional monitoring doesn't cut it.
AgentCore gives you built-in tracing. When something goes wrong, you can see every step the agent took, every tool it called, and where it went off track. You can view these traces in the AgentCore console or via the CLI:
agentcore traces --trace-id abc123
I used this extensively when our cost anomaly agent started hallucinating non-existent EC2 instances. The trace showed it was misinterpreting a Cost Explorer API response and inventing instance IDs. Without the trace, I would have spent hours guessing. With it, I had the exact reasoning chain and fixed the prompt in 20 minutes.
The AgentOps Discipline: Four Pillars
AWS talks about "AgentOps" as the operational discipline for agents, built on four pillars. From a DevOps perspective, these map directly to what you should be implementing:
Governance & Security: Use a multi-account strategy. Put agents in their own AWS accounts with deterministic controls. Implement human-in-the-loop for destructive actions. Every action must be traceable.
Build & Operations: Treat every agent, tool, and memory configuration as a versioned, deployable artifact with its own CI/CD pipeline. AgentCore's CLI integrates with standard CI/CD tools—we run agentcore deploy from GitHub Actions with environment-specific configs.
Evaluation: Don't just test that the agent runs. Test that it produces correct results. We built a small eval suite with known inputs and expected outputs, and run it against every deployment. AgentCore lets you test variations against real traffic and use data to decide what to ship next.
Observability: This isn't optional with agents. You need traces, metrics, and logs. AgentCore provides the traces. We pipe the rest into CloudWatch and Grafana.
Practical Tips
Start with the managed harness if you're prototyping. You can always migrate to code-based later without tearing everything down.
Version your agent configurations in git alongside your code. The
agentcore.yamlconfig file should be treated like infrastructure as code.Set up evals early. It's tempting to skip this, but agents drift. What works today might break tomorrow if the underlying model gets updated. Evals catch this before users do.
Use the gateway for everything. Even internal tools. It's tempting to bypass it for "quick" integrations, but you lose auditability and access control. The overhead is minimal.
Budget for the learning curve on traces. Reading agent traces is a different skill than reading application logs. Spend time understanding the format before you're debugging a production incident.
Honest Limitations
AgentCore is new, and it shows. The documentation has gaps—I had to dig into the AWS blog posts and samples to figure out several configurations that weren't covered in the official docs. The CLI error messages can be opaque; I hit a cryptic CDK error on deploy that turned out to be a missing IAM permission, but the message didn't point me there.
The managed harness is limited in customization. If your agent needs complex orchestration logic, you'll outgrow it quickly and need to switch to code-based. That transition isn't seamless—you'll rewrite some code.
Cold starts on the runtime can be slow. Our first invocation after a deploy sometimes takes 10-15 seconds. For a DevOps tool that runs asynchronously, this is fine. For a user-facing chatbot, it would be a problem.
Finally, this is an AWS-native solution. If your infrastructure spans multiple clouds, the tight AWS integration becomes a constraint rather than a benefit. The gateway, the runtime, the observability—all assume you're all-in on AWS.
Final Assessment
AgentCore fills a real gap. Before it, every team was building their own deployment, access control, and observability plumbing for agents. Now there's a standardized platform that handles the undifferentiated heavy lifting. For DevOps teams already running on AWS, it's worth serious consideration. Just go in with eyes open about the documentation gaps and cold start latency, and invest in evals and trace literacy from day one.