Skip to content
Back to search

web-agent-protocol

🌐Web Agent Protocol (WAP) - Record and replay user interactions in the browser with MCP support

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/ota-tech-ai--web-agent-protocol/badge.svg)](/listings/ota-tech-ai--web-agent-protocol)

HTML

<a href="/listings/ota-tech-ai--web-agent-protocol"><img src="/api/listings/ota-tech-ai--web-agent-protocol/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install the dependencies with the following command:

Create a conda env

conda create -n WAP python=3.11

Activate the conda env

conda activate WAP

Install the dependencies

pip install -r requirements.txt

Setup your repo source path:

set PYTHONPATH=C:/path/to/webagentprotocol # for Windows
export PYTHONPATH=/path/to/webagentprotocol # for Linux

Create **.env** file under the repo root directory with your own API keys:

OPENAI_API_KEY=sk-proj-...
DEEPSEEK_API_KEY=sk-...

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 190 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <!-- markdownlint-disable first-line-h1 -->
  2. <!-- markdownlint-disable html -->
  3. <!-- markdownlint-disable no-duplicate-header -->
  4. <div align="center">
  5. <img src="chrome-extension/assets/beholder-tool-kit-long.png" width="100%" alt="OTA-tool-kits" style="border-radius: 10px;" />
  6. </div>
  7. <br>
  8. <div align="center" style="line-height: 1;">
  9. <a href="https://www.otatech.ai/"><img alt="Homepage"
  10. src="https://img.shields.io/badge/Visit-otatech.ai-blue"/></a>
  11. <a href="https://huggingface.co/OTA-AI/OTA-v1"><img alt="Hugging Face"
  12. src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-OTA%20AI-ffc107?color=ffc107&logoColor=white"/></a>
  13. <a href="https://github.com/OTA-Tech-AI/webagentprotocol/blob/main/LICENSE"><img alt="Code License"
  14. src="https://img.shields.io/badge/Code_License-MIT-f5de53?&color=f5deff"/></a>
  15. <br><br><br>
  16. </div>
  17. # Web Agent Protocol
  18. ## Overview
  19. The Web Agent Protocol (WAP) is a standardized framework designed to enable seamless interaction between users, web agents, and browsers by recording and replaying browser actions. It separates the concerns of action recording and execution, allowing for efficient automation and reusability. The Python SDK for WAP implements the full specification, making it easy to:
  20. 1. **Collect** user‑interaction data with the [OTA‑WAP Chrome extension](https://github.com/OTA-Tech-AI/webagentprotocol/tree/main/chrome-extension).
  21. 2. **Convert** the raw event stream into either **_exact‑replay_** or **_smart‑replay_** action lists.
  22. 3. **Convert** recorded actions into **_MCP_** servers for reuse by any agent or user
  23. 4. **Replay** those lists using the **_WAP-Replay_** protocol to ensure accurate browser operations.
  24. ### WAP FULL DEMO
  25. [![Watch the video](https://img.youtube.com/vi/joh9FXJfnwk/0.jpg)](https://www.youtube.com/watch?v=joh9FXJfnwk)
  26. ### Without WAP
  27. ![image](https://github.com/user-attachments/assets/843ea9da-45c0-48e9-8a25-44f5bfb31786)
  28. ### WAP Record
  29. ![image](https://github.com/user-attachments/assets/3d041f56-9e76-4b61-9b56-0686070723a3)
  30. ### WAP Replay
  31. ![image](https://github.com/user-attachments/assets/e13ca7c7-3cc1-4952-8a79-3bd1e9e98580)
  32. ## Example using WAP
  33. ![image](https://github.com/user-attachments/assets/ccb7387b-0677-498c-b4ad-a10590e37e27)
  34. ## Setup
  35. Install the dependencies with the following command:
  36. Create a conda env
  37. ```bash
  38. conda create -n WAP python=3.11
  39. ```
  40. Activate the conda env
  41. ```bash
  42. conda activate WAP
  43. ```
  44. Install the dependencies
  45. ```bash
  46. pip install -r requirements.txt
  47. ```
  48. Setup your repo source path:
  49. ```
  50. set PYTHONPATH=C:/path/to/webagentprotocol # for Windows
  51. export PYTHONPATH=/path/to/webagentprotocol # for Linux
  52. ```
  53. Create **.env** file under the repo root directory with your own API keys:
  54. ```
  55. OPENAI_API_KEY=sk-proj-...
  56. DEEPSEEK_API_KEY=sk-...
  57. ```
  58. ## Record
  59. ### WAP record extension
  60. Please refer to [OTA‑WAP Chrome Extension](https://github.com/OTA-Tech-AI/webagentprotocol/tree/main/chrome-extension) to setup action capturer in your Chrome browser.
  61. ### Start data‑collection server
  62. Run the following command to start the server to collect data from the extension:
  63. ```bash
  64. python action_collect_server.py
  65. ```
  66. **Once the server is up, you can start to record from the page using WAP Chrome extension.**
  67. The server listens on http://localhost:4934/action-data by default, please make sure the Host and Port in the extension settings match this server config.
  68. Each session will be saved to:
  69. ```bash
  70. data/YYYYMMDD/taskid/summary_event_<timestamp>.json
  71. ```
  72. An example of the formatted data which you will received in the WAP backend server is like:
  73. ```json
  74. {
  75. "taskId": "MkCAhQsHgXn7YgaK",
  76. "type": "click",
  77. "actionTimestamp": 1746325231479,
  78. "eventTarget": {
  79. "type": "click",
  80. "target": "<a ota-use-interactive-target=\"1\" data-ordinal=\"3\" href=\"https://www.allrecipes.com/recipe/68925/cheesy-baked-salmon/\" data-tax-levels=\"\" data-doc-id=\"6592066\" class=\"comp mntl-card-list-card--extendable mntl-universal-card mntl-document-card mntl-card card card--no-image\" id=\"mntl-card-list-card--extendable_3-0\">\n<div class=\"loc card__top\"><div class=\"card__media mntl-image card__media universal-image__container\">...",
  81. "targetId": "mntl-card-list-card--extendable_3-0",
  82. "targetClass": "comp mntl-card-list-card--extendable mntl-universal-card mntl-document-card mntl-card card card--no-image"
  83. },
  84. "allEvents": {},
  85. "pageHTMLContent": "<header data-tracking-container=\"true\" data-collapsible=\"true\" class=\"comp header mntl-header mntl-header--magazine mntl-header--open-search-bar mntl-header--myr\" id=\"header_1-0\"><a data-tracking-container=\"true\" id=\"mntl-skip-to-content_1-0\" class=\"mntl-skip-to-content mntl-text-link\" rel=\"nocaes\" href=\"#main\"></a><div class=\"mntl-header__menu-top\">..."
  86. }
  87. ```
  88. ## Generate replay lists
  89. | Mode | Command |
  90. | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
  91. | **Exact replay** – exactly reproduce every action | `python wap_replay/generate_exact_replay_list.py --data_dir_path data/<date>/<task_id> --output_dir_path data_processed/exact_replay` |
  92. | **Smart replay** – condensed goal‑oriented steps | `python wap_replay/generate_smart_replay_list.py --data_dir_path data/<date>/<task_id> --output_dir_path data_processed/smart_replay` |
  93. Replace **<task_id>** with the folder produced by the extension
  94. (e.g. em3h6UBDZykz0gnH).
  95. Output structure:
  96. ```bash
  97. data_processed/smart_replay/
  98. ├─ subgoals_<task_id>/ # intermediate prompts & replies
  99. └─ wap_smart_replay_list_<task_id>.json # final smart replay list for the agent
  100. data_processed/exact_replay/
  101. └─ wap_smart_replay_list_<task_id>.json # final exact replay list for the agent
  102. ```
  103. ## Replay
  104. ```bash
  105. python run_replay.py --model-provider openai --wap_replay_list data_processed/exact_replay/wap_exact_replay_list_<task_id>.json --max-concurrent 1
  106. ```
  107. For **smart-replay**, replace the path with a smart‑replay JSON to test this mode.
  108. ## Convert to MCP Server
  109. ```bash
  110. python wap_replay\generate_mcp_server.py --task_id <task_id>
  111. ```
  112. converted MCP servers will be located under ``` mcp_servers ``` folder
  113. ## Replay with MCP
  114. You would need 2 terminals to replay with MCP. In the first termnial
  115. ```bash
  116. python wap_service.py
  117. ```
  118. In the second termnial
  119. ```bash
  120. python mcp_client.py
  121. ```
  122. Then enter your prompt in the second terminal
  123. ```bash
  124. example: find a top rated keyboard on amazon.ca using smart replay
  125. ```
  126. ## Replay with our Desktop App
  127. We provide out-of-box desktop app for running replay lists. It is easy to install and you don't need any extra steps for setup and deployments. Visit [WAP Replay Tool releases](https://github.com/OTA-Tech-AI/web-agent-protocol/releases) for more details.
  128. <img src="assets/wap_replay_tool_demo.gif" alt="WAP Replay Tool Demo GIF" width="500"/>
  129. ## Troubleshooting
  130. **ModuleNotFoundError** – run commands from the project root or export PYTHONPATH=. (set PYTHONPATH=. for Windows).
  131. “no task‑start file” – ensure the extension recorded a full session;
  132. the generators require exactly one task-start and one task-finish record.
  133. ## Acknowledgement
  134. Browser-Use: https://github.com/browser-use/browser-use
  135. MCP: https://github.com/modelcontextprotocol/python-sdk
  136. DOM Extension: https://github.com/kdzwinel/DOMListenerExtension

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.