aws-mcp-server-lab
Hands-on tests of the AWS MCP Server: what an AI agent can do in your account through it, and which controls actually stop it.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/chestlyace--aws-mcp-server-lab)HTML
<a href="/listings/chestlyace--aws-mcp-server-lab"><img src="/api/listings/chestlyace--aws-mcp-server-lab/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
export ADMIN_PROFILE=<your-sandbox-admin-profile> REGION=us-east-1 ./scripts/deploy.sh
Then add the two agent profiles to `~/.aws/config` (replace `ACCOUNT_ID`, and set `source_profile` to your admin profile), and check them:
cat configs/aws-config-profiles.ini # copy into ~/.aws/config and edit aws sts get-caller-identity --profile agent aws sts get-caller-identity --profile agent-dev
On its first start, the MCP proxy downloads about 30 MB, which takes longer than Claude Code's 30-second connection check. Warm it once:
uvx mcp-proxy-for-aws-cli@latest --help
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 116 lines
- # AWS MCP Server lab
- Hands-on tests of the [AWS MCP Server](https://docs.aws.amazon.com/agent-toolkit/latest/userguide/mcp-server.html):
- what an AI agent can do in your account through it, and which controls actually stop it.
- This is the companion repo for the blog post **The AWS MCP Server, tested:
- what it guards and what it
- doesn't ([read it here](https://blog.chestlyace.me/aws-mcp-server)).
- **The short version** (details in [FINDINGS.md](FINDINGS.md)):
- - Almost everything the agent does in your account goes through one tool, `run_script`,
- a Python sandbox.
- - Calls through the server are labelled: `aws:CalledViaAWSMCP` in IAM, and
- `aws-mcp.amazonaws.com` in CloudTrail.
- - **A deny that uses that label only stops calls through the server.** With one attached, the
- same IAM role deleted a "protected" bucket through the plain AWS CLI.
- - A deny on the agent's own role, with no condition, is what blocked every route.
- > **Use a sandbox AWS account with nothing else in it.** The agent role can read everything in
- > the account, and `scripts/teardown.sh` deletes every bucket named `mcp-test-*`.
- ## What's here
- | Path | What it is |
- |---|---|
- | `template.yaml` | CloudFormation: two S3 buckets, a Lambda function that fails on purpose (SQS trigger, timeout too low, no dead-letter queue, X-Ray on), and the agent roles `mcp-agent` and `mcp-agent-dev` |
- | `policies/` | The deny policies attached to `mcp-agent` during the guardrail tests |
- | `configs/` | Claude Code setups: OAuth (`a-*`), SigV4 through the MCP Proxy for AWS (`b-*`), AWS profiles, permission rules |
- | `scripts/` | `deploy`, `make-deleteme`, `fail-lambda`, `policy`, `trail` (CloudTrail lookup), `teardown`. All run as your admin profile |
- | `headless/` | Runs every test automatically with `claude -p` and saves each transcript |
- | `MANUAL.md` | The same tests, step by step, to run by hand |
- | `FINDINGS.md` | What I found, test by test, on 18–19 November 2026 |
- | `results.md` | An empty results sheet for your own run |
- ## Prerequisites
- - An empty sandbox AWS account, and an admin profile for it in your AWS CLI config.
- - AWS CLI 2.32 or later, [`uv`](https://docs.astral.sh/uv/), Claude Code, Python 3.
- - A budget alarm on the account. A full run costs a few cents of AWS, plus the model usage of
- about 20 Claude Code sessions (about $2 with Claude Opus).
- ## Set up
- ```bash
- export ADMIN_PROFILE=<your-sandbox-admin-profile> REGION=us-east-1
- ./scripts/deploy.sh
- ```
- Then add the two agent profiles to `~/.aws/config` (replace `ACCOUNT_ID`, and set `source_profile`
- to your admin profile), and check them:
- ```bash
- cat configs/aws-config-profiles.ini # copy into ~/.aws/config and edit
- aws sts get-caller-identity --profile agent
- aws sts get-caller-identity --profile agent-dev
- ```
- On its first start, the MCP proxy downloads about 30 MB, which takes longer than Claude Code's
- 30-second connection check. Warm it once:
- ```bash
- uvx mcp-proxy-for-aws-cli@latest --help
- ```
- ## Run the tests
- **Automatically.** Each batch starts fresh Claude Code sessions in print mode with only the
- AWS MCP Server loaded, your plugins and hooks off, and an explicit list of allowed tools:
- ```bash
- bash headless/lab-run.sh batch1 # T0 (read-only), T1, T2, T3: reads only
- bash headless/lab-run.sh batch2 # T6, T4, T5, T7: creates and deletes a bucket, attaches policies
- bash headless/lab-run.sh batch3 # T8, T11, T9: includes about 13 minutes of waiting
- bash headless/lab-run.sh cap # T9 follow-up: the serverless capability by name
- bash headless/lab-run.sh batch4 # T10: CloudTrail and CloudWatch
- ```
- Transcripts go to `~/mcp-lab-run/evidence` (change it with `RUN=...`). In the T5 runs the agent
- may run any shell command on your machine; its AWS access is limited to the `mcp-agent` role.
- **By hand.** Follow [MANUAL.md](MANUAL.md). You need it for the OAuth checks, which sign in
- through a browser.
- | Test | Question |
- |---|---|
- | T0 | Which tools does the server expose, and what does `--read-only` hide? |
- | T1 | Does the model answer worse without the server? |
- | T2 | What works without signing in? |
- | T3 | One `run_script` call vs the AWS CLI |
- | T4 | Does a deny on `aws:CalledViaAWSMCP` block the agent? |
- | T5 | Does the same role get through the CLI, and which fixes hold? |
- | T6 | Does `--read-only` stop a write? |
- | T7 | Do preview-era `aws-mcp:*` denies still work? |
- | T8 | Several AWS profiles in one session |
- | T9 | Troubleshooting a failing Lambda, with and without the serverless capability |
- | T10 | Can CloudTrail tell agent calls apart? |
- | T11 | Uploading a local file |
- ## Clean up
- ```bash
- ./scripts/teardown.sh # asks you to type "delete" first
- cd ~/mcp-lab-run/with-mcp && claude mcp remove aws-mcp
- ```
- Remove the `agent` and `agent-dev` profiles from `~/.aws/config`, sign out of the AWS console
- (this ends the OAuth session), and close the sandbox account when you no longer need it.
- ## Before you share your results
- Transcripts contain your account ID, sometimes your IP address, and pre-signed S3 URLs with
- temporary session tokens. Redact them before you publish anything.
- ## License
- [MIT](LICENSE)
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.