Skip to content
Back to search

AutomatedEmulation

An automated Adversary Emulation lab with terraform and MCP server. Build Caldera techniques and operations assisted with LLMs. Built for IaC stability, consistency, and speed.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/iknowjason--automatedemulation/badge.svg)](/listings/iknowjason--automatedemulation)

HTML

<a href="/listings/iknowjason--automatedemulation"><img src="/api/listings/iknowjason--automatedemulation/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 273 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Automated Emulation with Caldera MCP Server!
  2. ![Caldera MCP](images/mcp-overview.png "Caldera MCP Server")
  3. ## Update for 11/16/25: Caldera MCP Server plugin!
  4. Added the Caldera MCP server plugin, Bountyhunter plugin. Added automated download of emu plugin payloads. Caldera now has the capability to use the MCP server as a wrapper to the Caldera API, helping give context to LLMs for building new abilities, operation planners, and a CTI with RAG integration. See this [MITRE blog](https://medium.com/@mitrecaldera/connecting-llms-to-caldera-with-the-mcp-plugin-da1450254827) for more information.
  5. ## Overview
  6. Automated Emulation is a simple terraform template creating a customizable and automated Breach and Attack Simulation lab. It automically builds the following resources hosted in AWS:
  7. * One Linux server deploying Caldera and VECTR (vectr.io).
  8. * Linux Server: The latest Caldera 5.3 with Magma VueJS web application.
  9. * One Windows Client (Windows Server 2022) auto-configured for Caldera agent deployment, and other Red & Blue tools.
  10. * Windows Client: Atomic Red Team (ART) automated deployment.
  11. * Windows Client: Sysmon automated deployment.
  12. * Caldera MCP server plugin installed.
  13. * Exposes an MLflow server for observability, tracing, and debug of Caldera LLM inference calls.
  14. * Caldera Bountyhunter plugin installed.
  15. * Caldera EMU plugin: Auto-downloads payloads. This CALDERA Plugin converts Adversary Emulation Plans from the Center for Threat Informed Defense.
  16. See the **Features and Capabilities** section for more details.
  17. ## Requirements and Setup
  18. **Tested with:**
  19. * Mac OS 13.4
  20. * terraform 1.5.7
  21. **Clone this repository:**
  22. ```
  23. git clone https://github.com/iknowjason/AutomatedEmulation
  24. ```
  25. **Credentials Setup:**
  26. Generate an IAM programmatic access key that has permissions to build resources in your AWS account. Setup your .env to load these environment variables. You can also use the direnv tool to hook into your shell and populate the .envrc. Should look something like this in your .env or .envrc:
  27. ```
  28. export AWS_ACCESS_KEY_ID="VALUE"
  29. export AWS_SECRET_ACCESS_KEY="VALUE"
  30. ```
  31. ## Build and Destroy Resources
  32. ### Run terraform init
  33. Change into the AutomatedEmulation working directory and type:
  34. ```
  35. terraform init
  36. ```
  37. ### Run terraform plan or apply
  38. ```
  39. terraform apply -auto-approve
  40. ```
  41. or
  42. ```
  43. terraform plan -out=run.plan
  44. terraform apply run.plan
  45. ```
  46. ### Destroy resources
  47. ```
  48. terraform destroy -auto-approve
  49. ```
  50. ### View terraform created resources
  51. The lab has been created with important terraform outputs showing services, endpoints, IP addresses, and credentials. To view them:
  52. ```
  53. terraform output
  54. ```
  55. ## Features and Capabilities
  56. ### Important Firewall and White Listing
  57. By default when you run terraform apply, your public IPv4 address is determined via a query to ifconfig.so and the ```terraform.tfstate``` is updated automatically. If your location changes, simply run ```terraform apply``` to update the security groups with your new public IPv4 address. If ifconfig.me returns a public IPv6 address, your terraform will break. In that case you'll have to customize the white list. To change the white list for custom rules, update this variable in ```sg.tf```:
  58. ```
  59. locals {
  60. src_ip = "${chomp(data.http.firewall_allowed.response_body)}/32"
  61. #src_ip = "0.0.0.0/0"
  62. }
  63. ```
  64. ### Caldera
  65. **Caldera Linux Server**
  66. Caldera is built on an Ubuntu Linux 22.04 AMI automatically with SSL support for admin console. The following local project files are important for customization:
  67. * bas.tf: The terraform file that builds the Linux server and all terraform variables for Caldera.
  68. * files/bas/bootstrap.sh.tpl: The boostrap script for Caldera and other services.
  69. * files/bas/local.yml.tpl: The Caldera configuration file that is automatically deployed
  70. * files/bas/caldera.service: The caldera service file that is automatically installed
  71. * files/bas/abilities/: A local directory with custom abilities that will automatically deploy to the caldera server in ```/opt/caldera/data/abilities```
  72. * files/bas/payloads/: A local directory with custom payloads that will automatically deploy to the caldera server in ```/opt/caldera/data/payloads```
  73. **Troubleshooting Caldera:**
  74. SSH into the Caldera server by looking in ```terraform output``` for this line:
  75. ```
  76. ssh -i ssh_key.pem ubuntu@3.15.204.148
  77. ```
  78. Once in the system, tail the user-data logfile. You will see the steps from the ```bootstrap.sh.tpl``` script running:
  79. ```
  80. tail -f /var/log/user-data.log
  81. ```
  82. After the system shows that the bootstrap is complete, monitor the Caldera service:
  83. ```
  84. sudo systemctl status caldera
  85. ```
  86. After it is running, verify the listening ports for Caldera web admin TLS + API:
  87. ```
  88. sudo netstat -tulpn | grep 8443
  89. ```
  90. Verify the mlflow server is listening on port 5000
  91. ```
  92. sudo netstat -tulpn | grep 5000
  93. ```
  94. **MLflow server publicly exposed:**
  95. The MCP server plugin automatically exposes an MLflow server. MLflow provides log tracing, debugging, and monitoring of the LLM inference calls used by the Caldera MCP plugin, which is a wrapper to the Caldera API. One important note. By default this server only listens on localhost, but I've adapted it to allow remote connections. Be careful and make sure you have proper EC2 security groups to only allow authorized sources to connect to the MLflow server.
  96. ![Caldera MLflow](images/mlflow-overview.png "Caldera MLflow Server")
  97. **Customize Caldera Linux:**
  98. To customize Caldera, you can modify the default admin credentials for red, blue and api keys in ```bas.tf```. For other customizations, you can modify the ```local.yml.tpl``` Caldera configuration file.
  99. You can create custom abilities in your project that get automatically loaded into Caldera. This image shows a custom ability for lateral-movement that is built locally from ```files/bas/abilities/lateral-movement/```.
  100. ![Caldera](images/caldera2.png "Abilities")
  101. **Teraform Output:**
  102. View the terraform outputs for important Caldera access information:
  103. ```
  104. -------
  105. Caldera Console
  106. -------
  107. https://ec2-18-224-151-55.us-east-2.compute.amazonaws.com:8443
  108. Caldera MCP MLflow server
  109. -------
  110. http://ec2-18-224-151-55.us-east-2.compute.amazonaws.com:5000
  111. Caldera Console Credentials
  112. -------------------
  113. blue:Caldera2024
  114. red:Caldera2024
  115. admin:Caldera2024
  116. API Keys
  117. --------
  118. api_key_blue: blueadmin2024
  119. api_key_red: redadmin2024
  120. Caldera API Cheat Sheet
  121. -----------------------
  122. <CUSTOM>
  123. --------------
  124. ```
  125. **Caldera on Windows Client:**
  126. The Caldera sandcat agent is automatically installed and launches on the Windows client system. The bootstrap script waits until Caldera is up and available, then installs Sandcat caldera agent. It should look like this.
  127. ![Caldera](images/caldera1.png "Agent View")
  128. To troubleshoot this, look in the following logfile on the Windows system:
  129. ```
  130. C:\Terraform\caldera_log.log
  131. ```
  132. To modify this file locally, it is located in ```files\windows\caldera.ps1.tpl```
  133. ### VECTR
  134. VECTR by Security Risk Advisors is installed automatically. From their Github repo:
  135. _VECTR is a tool that facilitates tracking of your red and blue team testing activities to measure detection and prevention capabilities across different attack scenarios_
  136. ![VECTR](images/vectr.png "VECTR")
  137. Take a look at the terraform output to see the public URL and credentials for accessing VECTR:
  138. ```
  139. VECTR Console
  140. -------------
  141. https://ec2-3-15-204-148.us-east-2.compute.amazonaws.com:8081
  142. VECTR Credentials
  143. -----------------
  144. admin:11_ThisIsTheFirstPassword_11
  145. ```
  146. ### Red Tools
  147. On the Windows Client system, the following tools are automatically deployed into ```C:\Tools\```:
  148. * Atomic Red Team (ART)
  149. * PurpleSharp
  150. The local bootstrap script for customization is ```files\windows\red.ps1.tpl```
  151. To track monitoring of the deployment on the Windows Client, see the logfile at ```C:\Terraform\red_log.log```
  152. ### Blue Tools
  153. Sysmon service and customized configuration (SwiftOnSecurity) is deployed onto the Windows Client system. To update the sysmon version and configuration, make changes inside the ```files\sysmon``` directory.
  154. The local bootstrap script for customization is ```files\windows\sysmon.ps1.tpl```
  155. To track monitoring of the deployment on the Windows Client, see the logfile at ```C:\Terraform\blue_log.log```
  156. ### Windows Client
  157. The Windows Client system is built from ```win1.tf```. Windows Server 2022 Datacenter edition is currently used. You can upload your own AMI image and change the data reference in win1.tf. The local bootstrap script is located in ```files/windows/bootstrap-win.ps1.tpl```. RDP into the Windows system and follow this logfile to see how the system is bootstrapping:
  158. ```
  159. C:\Terraform\bootstrap_log.log
  160. ```
  161. **Customizing Build Scripts**
  162. For adding new scripts for a customized deployment, reference the arrays in ```scripts.tf``` and ```s3.tf```. For more complex deployments, the Windows system is built to have flexibility for adding customized scripts for post-deployment configuration management. This gets around the size limit of user-data not exceeding 16KB in size. The s3 bucket is used for staging to upload and download scripts, files, and any artifacts needed. How this is done: A small master script is always deployed via user-data. This script has instructions to download additional scripts. This is under your control and is configured in ```scripts.tf``` and ```s3.tf```. In ```scripts.tf```, take a look at the array called ```templatefiles```. Add any custom terraform templatefiles here and then add them locally to ```files/windows```. See the ```red.ps1.tpl``` and ```sysmon.ps1.tpl``` files as an example. The file should end in ```tpl```. This template file is generated as output into the directory called ```output```. The terraform code strips off the ```.tpl``` in the filename when it generates into the ```output``` directory. Make sure the filename is correct because the master script downloads based on this name. In ```s3.tf```, each little script referenced in ```templatefiles``` is uploaded. The master bootstrap script has a reference to this array. It will automatically download all generated scripts from the ```templatefiles``` array and execute each script.
  163. **Terraform Outputs**
  164. See the output from ```terraform output``` to get the IP address and credentials for RDP:
  165. ```
  166. -------------------------
  167. Virtual Machine win1
  168. -------------------------
  169. Instance ID: i-0eecf5439b1d8080b
  170. Computer Name: win1
  171. Private IP: 10.100.20.10
  172. Public IP: 18.119.101.237
  173. local Admin: RTCAdmin
  174. local password: wOFVYKYlk2
  175. ```
  176. ### Linux
  177. The linux system is built from ```bas.tf```. The bootstrap script is located in ```files/bas/bootstrap.sh.tpl```.
  178. To access the Linux system and troubleshoot any bootstrap issues, SSH into the system by looking at the terraform output:
  179. ```
  180. SSH
  181. ---
  182. ssh -i ssh_key.pem ubuntu@3.15.204.148
  183. ```
  184. Then tail the user-data logfile to monitor any potential issues with the bootstrap script:
  185. ```
  186. tail -f /var/log/user-data.log
  187. ```
  188. ## Key Differences
  189. This lab differs from other popular ```Cyber Ranges``` in its design and philosophy. No secondary tools like Ansible are necessary. Feel free to use them if you like. But they aren't required for configuration management. Instead of using 3rd party configuration management tools, this lab uses terraform providers (AWS SDK) and builtin AWS features (```user data```). You don't have to rely on a secondary agent or deal with outdated libraries or networking issues with agentless push or updating a secondary tool that causes issues over time. This increases ```stability, consistency, and speed``` for building and configuring cloud resources. Use terraform, bash, and powershell to build and configure. A small user-data script is pushed into the system and runs. Individual configuration management scripts are uploaded to an S3 bucket. The master script instructs the system which smaller scripts to run which builds the system. With good documentation, the location of these scripts should make it easy to add and customize. See the **Features and Capabilities** section for more details.
  190. ### Future
  191. This terraform was automatically generated by the Operator Lab tool. To get future releases of the tool, follow twitter.com/securitypuck.
  192. For an Azure version of this tool, check out PurpleCloud (https://www.purplecloud.network)
  193. <a href="https://twitter.com/intent/user?screen_name=securitypuck">![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/securitypuck)</a>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.