AWS Secrets Manager isn't built for AI agents

If you ask "how do I store API keys for my AI agent?", the first answer everyone gives is "use AWS Secrets Manager". The reasoning is reasonable: you're already in AWS, the tooling is mature, IAM integration is solid, rotation is built in.

The reasoning misses three things about agents specifically. This post walks through what AWS Secrets Manager is genuinely good at, where it stops being the right tool for an agent workload, and what to do at that point.

This isn't a knock on Secrets Manager. It's a great product for the case it was built for. The case it was built for is not "an AI agent that mostly calls third-party APIs on behalf of an interactive user".

What Secrets Manager is good at

AWS Secrets Manager was designed around four assumptions:

1. The consumer is an AWS workload (ECS task, Lambda, EC2 instance, EKS pod). IAM gives it permission to fetch.
2. The secret is a long-lived service credential (database password, third-party API key, internal service token). Rotation is months apart, manually triggered or on a schedule.
3. The secret is fetched at startup and held in process memory for the lifetime of the workload.
4. One workload, one secret, one identity at a time. Multi-tenancy is handled at the IAM-policy level (different roles for different workloads).

Under those assumptions, Secrets Manager is excellent. The credential never appears in your environment, your code, or your image. Rotation is automatic for supported services. Audit logs go to CloudTrail. Cross-region replication, encryption with KMS, deletion recovery - all the operational concerns are paid for.

For a backend service that talks to its own database and one or two internal APIs, this is the right tool.

Where the model breaks for agents

An AI agent has different shape on every one of those four assumptions.

1. The consumer is rarely just an AWS workload

Your laptop runs Claude Code. Your dev box runs a Python agent over SSH. Your CI runner runs a code-review agent. A teammate's machine runs a different agent on the same codebase. Maybe one is in EKS. The rest aren't.

AWS Secrets Manager is reachable from anywhere with valid AWS credentials, but giving every developer's laptop programmatic access to your prod Secrets Manager is a separate, larger problem. The default pattern (SSO-based STS tokens, scoped per developer) works but adds another auth layer in front of the secret-fetch path. Two OAuth flows to use one API.

2. Agents don't hold one long-lived credential

A typical agent's outbound calls in a single session:

• Anthropic for completions
• OpenAI for embeddings
• GitHub for repo reads and PR writes
• Linear or Jira for ticket reads
• Slack for posting status
• Sometimes Stripe, Notion, Vercel, AWS itself, a dozen others

Each of those is a separate credential. Some are OAuth (needs refresh). Some are API keys (don't need refresh, but expire on rotation). Some are user-scoped (the Linear API key for this user). Some are service-scoped (the Stripe restricted key for this environment).

Secrets Manager can store all of them. It just doesn't model the relationships. Each one is a separate secret with its own ARN. The agent has to know how to fetch each one, when to refresh OAuth ones (Secrets Manager doesn't do refresh; you have to run the OAuth flow yourself and update the stored value), and how to pick the right user-scoped credential per request.

Most agent frameworks just don't. They expect OPENAI_API_KEY in the env. You're stuck either fetching everything at startup (defeats the rotation point) or wiring a custom fetch layer (defeats the "use the tool that exists" pragmatism).

3. Agents are read primitives, and the secret in memory is a leak waiting to happen

This is the asymmetry that I think people underweight. Once the agent fetches the secret from Secrets Manager, it's a string in the agent's process memory. The agent can read its own memory. The agent can be tricked into reading its own memory.

We covered this in detail in How prompt injection becomes credential exfiltration, but the short form: any string in your agent's address space is one prompt-injection away from being POSTed somewhere. Secrets Manager doesn't help here. It got the secret to you safely. What you do with it after is your problem.

The broker pattern (proxy-based injection) keeps the secret out of the agent's address space entirely. Secrets Manager + a secure-by-default fetch into env vars doesn't.

4. Multi-identity per provider isn't modeled

You have a personal GitHub identity and a work one. Maybe a third for an open-source org. Secrets Manager can store all three secrets, but there's no built-in concept of "use the work identity when the agent is in the work repo, the personal one otherwise". You'd build that switching layer yourself.

This is exactly what credential brokers were designed for. The broker knows which identity to use based on the calling context (directory, repo, user, request annotation).

The cost question

For most laptop and small-team use cases, AWS Secrets Manager is also overkill on price. Each secret is $0.40 per month, plus $0.05 per 10,000 API calls. If your agent makes a handful of calls and you have a handful of secrets, that's small money. If you have dozens of secrets and a chatty agent, the per-call cost adds up surprisingly fast.

For a single developer with a dozen provider credentials and an agent making thousands of calls per day, you're looking at $5-15 per month for what a local SQLite vault does for free.

For a team deploying agents at scale, the dollars are inconsequential. The IAM-policy management overhead is what costs you, and that scales with the number of (developer, environment, agent) tuples.

When Secrets Manager IS the right tool

I want to be precise about this. AWS Secrets Manager is the right tool when:

• The agent runs only on AWS infrastructure (ECS, Lambda, EKS). IAM-based fetch is clean and audited.
• The agent uses a small number of long-lived credentials, not user OAuth. Database password, third-party API key, internal service token.
• Rotation is automated by AWS for supported services (RDS, Redshift, DocumentDB), and you want that handled for you.
• You don't have a prompt-injection threat model because the agent isn't reading attacker-influenced text.

For an internal service-to-service agent that doesn't process untrusted input, Secrets Manager + IAM is the right operational story. It's well-understood, audited, and integrates with the rest of your AWS footprint.

It just isn't right for the "agent that talks to 12 providers on behalf of a human user, sometimes reads PRs and emails, runs partially on laptops and partially in CI". That workload needs a broker.

What to use instead

The credential-broker pattern is what was missing in the Secrets Manager era. The shape of the answer:

• A local proxy on the same host as the agent, owning the credential vault and the substitution.
• OAuth refresh handled by the broker, so you never see "expired token" errors at runtime.
• Multi-identity per provider with per-context selection.
• Bundled provider definitions so you don't write OAuth client code yourself.

Authsome is one implementation. Agent Vault, Clawvisor, and OneCLI are others. The comparison post covers when to pick which.

The combination pattern is most realistic:

• AWS Secrets Manager for the service-credential layer that the broker itself needs (the OAuth client_secrets, the master encryption key, the AWS credentials the broker uses to fetch from Secrets Manager).
• A credential broker for the runtime agent-to-provider layer. The broker stores user-OAuth tokens and refresh state. The broker is what the agent talks to.

This way Secrets Manager does what it's good at (managing a small number of long-lived service credentials with rotation), and the broker does what it's good at (acting as the proxy boundary for the dozens of user-scoped credentials your agent needs at runtime).

The shorter version

AWS Secrets Manager solves the "where do I store this secret" problem for a backend service consumer with a stable identity and a stable set of credentials. AI agents don't fit any of those three. They have variable consumers (laptop, container, CI), variable credentials (per provider, per identity), and a hostile-input threat model that makes "fetch and hold in memory" a liability rather than a feature.

Use Secrets Manager where it shines. Use a credential broker for the agent layer. They compose well; pick the right tool for each tier.

Priyansh Khodiyar

Maintainer

Works on authsome and the agentr.dev tooling.