Skip to content
Back to search

aws-mcp-server-lab

Hands-on tests of the AWS MCP Server: what an AI agent can do in your account through it, and which controls actually stop it.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/chestlyace--aws-mcp-server-lab/badge.svg)](/listings/chestlyace--aws-mcp-server-lab)

HTML

<a href="/listings/chestlyace--aws-mcp-server-lab"><img src="/api/listings/chestlyace--aws-mcp-server-lab/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

export ADMIN_PROFILE=<your-sandbox-admin-profile> REGION=us-east-1
./scripts/deploy.sh

Then add the two agent profiles to `~/.aws/config` (replace `ACCOUNT_ID`, and set `source_profile` to your admin profile), and check them:

cat configs/aws-config-profiles.ini      # copy into ~/.aws/config and edit
aws sts get-caller-identity --profile agent
aws sts get-caller-identity --profile agent-dev

On its first start, the MCP proxy downloads about 30 MB, which takes longer than Claude Code's 30-second connection check. Warm it once:

uvx mcp-proxy-for-aws-cli@latest --help

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 116 lines
  1. # AWS MCP Server lab
  2. Hands-on tests of the [AWS MCP Server](https://docs.aws.amazon.com/agent-toolkit/latest/userguide/mcp-server.html):
  3. what an AI agent can do in your account through it, and which controls actually stop it.
  4. This is the companion repo for the blog post **The AWS MCP Server, tested:
  5. what it guards and what it
  6. doesn't ([read it here](https://blog.chestlyace.me/aws-mcp-server)).
  7. **The short version** (details in [FINDINGS.md](FINDINGS.md)):
  8. - Almost everything the agent does in your account goes through one tool, `run_script`,
  9. a Python sandbox.
  10. - Calls through the server are labelled: `aws:CalledViaAWSMCP` in IAM, and
  11. `aws-mcp.amazonaws.com` in CloudTrail.
  12. - **A deny that uses that label only stops calls through the server.** With one attached, the
  13. same IAM role deleted a "protected" bucket through the plain AWS CLI.
  14. - A deny on the agent's own role, with no condition, is what blocked every route.
  15. > **Use a sandbox AWS account with nothing else in it.** The agent role can read everything in
  16. > the account, and `scripts/teardown.sh` deletes every bucket named `mcp-test-*`.
  17. ## What's here
  18. | Path | What it is |
  19. |---|---|
  20. | `template.yaml` | CloudFormation: two S3 buckets, a Lambda function that fails on purpose (SQS trigger, timeout too low, no dead-letter queue, X-Ray on), and the agent roles `mcp-agent` and `mcp-agent-dev` |
  21. | `policies/` | The deny policies attached to `mcp-agent` during the guardrail tests |
  22. | `configs/` | Claude Code setups: OAuth (`a-*`), SigV4 through the MCP Proxy for AWS (`b-*`), AWS profiles, permission rules |
  23. | `scripts/` | `deploy`, `make-deleteme`, `fail-lambda`, `policy`, `trail` (CloudTrail lookup), `teardown`. All run as your admin profile |
  24. | `headless/` | Runs every test automatically with `claude -p` and saves each transcript |
  25. | `MANUAL.md` | The same tests, step by step, to run by hand |
  26. | `FINDINGS.md` | What I found, test by test, on 18–19 November 2026 |
  27. | `results.md` | An empty results sheet for your own run |
  28. ## Prerequisites
  29. - An empty sandbox AWS account, and an admin profile for it in your AWS CLI config.
  30. - AWS CLI 2.32 or later, [`uv`](https://docs.astral.sh/uv/), Claude Code, Python 3.
  31. - A budget alarm on the account. A full run costs a few cents of AWS, plus the model usage of
  32. about 20 Claude Code sessions (about $2 with Claude Opus).
  33. ## Set up
  34. ```bash
  35. export ADMIN_PROFILE=<your-sandbox-admin-profile> REGION=us-east-1
  36. ./scripts/deploy.sh
  37. ```
  38. Then add the two agent profiles to `~/.aws/config` (replace `ACCOUNT_ID`, and set `source_profile`
  39. to your admin profile), and check them:
  40. ```bash
  41. cat configs/aws-config-profiles.ini # copy into ~/.aws/config and edit
  42. aws sts get-caller-identity --profile agent
  43. aws sts get-caller-identity --profile agent-dev
  44. ```
  45. On its first start, the MCP proxy downloads about 30 MB, which takes longer than Claude Code's
  46. 30-second connection check. Warm it once:
  47. ```bash
  48. uvx mcp-proxy-for-aws-cli@latest --help
  49. ```
  50. ## Run the tests
  51. **Automatically.** Each batch starts fresh Claude Code sessions in print mode with only the
  52. AWS MCP Server loaded, your plugins and hooks off, and an explicit list of allowed tools:
  53. ```bash
  54. bash headless/lab-run.sh batch1 # T0 (read-only), T1, T2, T3: reads only
  55. bash headless/lab-run.sh batch2 # T6, T4, T5, T7: creates and deletes a bucket, attaches policies
  56. bash headless/lab-run.sh batch3 # T8, T11, T9: includes about 13 minutes of waiting
  57. bash headless/lab-run.sh cap # T9 follow-up: the serverless capability by name
  58. bash headless/lab-run.sh batch4 # T10: CloudTrail and CloudWatch
  59. ```
  60. Transcripts go to `~/mcp-lab-run/evidence` (change it with `RUN=...`). In the T5 runs the agent
  61. may run any shell command on your machine; its AWS access is limited to the `mcp-agent` role.
  62. **By hand.** Follow [MANUAL.md](MANUAL.md). You need it for the OAuth checks, which sign in
  63. through a browser.
  64. | Test | Question |
  65. |---|---|
  66. | T0 | Which tools does the server expose, and what does `--read-only` hide? |
  67. | T1 | Does the model answer worse without the server? |
  68. | T2 | What works without signing in? |
  69. | T3 | One `run_script` call vs the AWS CLI |
  70. | T4 | Does a deny on `aws:CalledViaAWSMCP` block the agent? |
  71. | T5 | Does the same role get through the CLI, and which fixes hold? |
  72. | T6 | Does `--read-only` stop a write? |
  73. | T7 | Do preview-era `aws-mcp:*` denies still work? |
  74. | T8 | Several AWS profiles in one session |
  75. | T9 | Troubleshooting a failing Lambda, with and without the serverless capability |
  76. | T10 | Can CloudTrail tell agent calls apart? |
  77. | T11 | Uploading a local file |
  78. ## Clean up
  79. ```bash
  80. ./scripts/teardown.sh # asks you to type "delete" first
  81. cd ~/mcp-lab-run/with-mcp && claude mcp remove aws-mcp
  82. ```
  83. Remove the `agent` and `agent-dev` profiles from `~/.aws/config`, sign out of the AWS console
  84. (this ends the OAuth session), and close the sandbox account when you no longer need it.
  85. ## Before you share your results
  86. Transcripts contain your account ID, sometimes your IP address, and pre-signed S3 URLs with
  87. temporary session tokens. Redact them before you publish anything.
  88. ## License
  89. [MIT](LICENSE)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.