Skip to content
Back to search

ai-dev-kit

Databricks Toolkit for Coding Agents provided by Field Engineering

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/databricks-solutions/ai-dev-kit with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/databricks-solutions/ai-dev-kit'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/databricks-solutions--ai-dev-kit/badge.svg)](/listings/databricks-solutions--ai-dev-kit)

HTML

<a href="/listings/databricks-solutions--ai-dev-kit"><img src="/api/listings/databricks-solutions--ai-dev-kit/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

| Options | Best For | Start Here | |----------------------------------|----------|------------| | :star: [**Install Skills**](#install-in-existing-project) | **Start here!** Follow quick install instructions to add Databricks skills for your user or existing project folder | [Quick Start (install)](#install-in-existing-project) | [**Genie Code Skills**](databricks-skills/) | Upload selected skills into your workspace for Genie Code. Note: much of the content in the public Databricks skills is already included in Genie Code. | [Genie Code skills (install)](#genie-code-skills) | | [**Visual Builder App**](#visual-builder-app) | Web-based UI for Databricks development | `databricks-builder-app/` |

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 11 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "databricks-ai-dev-kit",
  3. "owner": {
  4. "name": "Databricks Solutions"
  5. },
  6. "metadata": {
  7. "description": "DEPRECATED — the databricks-ai-dev-kit plugin is no longer published. Install skills with install.sh or `databricks aitools install` (which installs the official `databricks` plugin for Claude Code/Codex/Copilot, or skill files for other agents). See .claude-plugin/DEPRECATED.md.",
  8. "version": "1.1.9"
  9. },
  10. "plugins": []
  11. }
README.md · README · 447 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # Databricks AI Dev Kit
  2. <p align="center">
  3. <img src="https://img.shields.io/badge/Databricks-Certified%20Gold%20Project-FFD700?style=for-the-badge&logo=databricks&logoColor=black" alt="Databricks Certified Gold Project">
  4. </p>
  5. ---
  6. > 📣 **A big step forward: AI Dev Kit skills are now official Databricks AI Tools**
  7. >
  8. > The skills from this AI Dev Kit repository are now delivered as part of **Databricks AI Tools**,
  9. > a Databricks **engineering-owned repository**, built and maintained in close collaboration
  10. > with field engineering.
  11. >
  12. > **Databricks AI Tools**
  13. > are installed and kept up to date directly through the **Databricks CLI**
  14. > (`databricks aitools install`), or through the AI Dev Kit installer, which delegates to the CLI and
  15. > still guides you through the process.
  16. >
  17. > **AI Tools work with any agent** and include a
  18. > first-class **Genie One** integration.
  19. ---
  20. **Already installed?** Re-run the [Quick Start (install)](#install-in-existing-project) in this repo to uninstall older skills and the MCP
  21. server before installing the official AI Tools — see [Where Skills Come From](#where-skills-come-from).
  22. **What stays:** The MCP server and Builder App remain in this repository. The Builder App will keep
  23. being developed and improved, and the MCP server will be maintained and updated on a best-effort
  24. basis as GitHub issues are filed.
  25. **What's next:** AI Dev Kit will continue to guide you through setting up your AI coding environment
  26. and be a place to find experimental tools developed by Field Engineering. We're working on several
  27. tutorials to help you get started using coding agents for building on Databricks — including getting
  28. started with Genie Code and Omnigent — which will land in a future update. The bundled skill files
  29. in this repo are deprecated and kept for reference only.
  30. **A few skills were renamed or merged** in the official install. Most names are unchanged; the
  31. exceptions are:
  32. | Previously (AI Dev Kit) | Official Databricks skills |
  33. |---|---|
  34. | `databricks-bundles` | `databricks-dabs` |
  35. | `databricks-genie` | `databricks-genie-agents` |
  36. | `databricks-spark-declarative-pipelines` | `databricks-pipelines` |
  37. | `databricks-lakebase-autoscale`, `databricks-lakebase-provisioned` | `databricks-lakebase` (merged) |
  38. | `databricks-config` | `databricks-core` (merged) |
  39. > 🔒 Proactive Dependency Security
  40. > As part of our commitment to supply chain integrity, we continually monitor our dependency tree against known vulnerabilities and industry advisories. In response to a recently disclosed supply chain incident affecting litellm versions 1.82.7–1.82.8, we have audited our packages and removed the litellm dependency for most usage. It is solely used in the test directory for skills evaluation and optimization, and has been pinned to a safe version.
  41. > For full third-party attribution, see NOTICE.txt.
  42. ---
  43. ## Install or Upgrade
  44. | Options | Best For | Start Here |
  45. |----------------------------------|----------|------------|
  46. | :star: [**Install Skills**](#install-in-existing-project) | **Start here!** Follow quick install instructions to add Databricks skills for your user or existing project folder | [Quick Start (install)](#install-in-existing-project)
  47. | [**Genie Code Skills**](databricks-skills/) | Upload selected skills into your workspace for Genie Code. Note: much of the content in the public Databricks skills is already included in Genie Code. | [Genie Code skills (install)](#genie-code-skills) |
  48. | [**Visual Builder App**](#visual-builder-app) | Web-based UI for Databricks development | `databricks-builder-app/` |
  49. | [**MCP Tools**](databricks-mcp-server/) | Standalone MCP server exposing Databricks actions to AI clients. Note: We recommend skills instead, but maintain this as a good foundation if a custom MCP server is required. | [Register MCP server](databricks-mcp-server/) |
  50. ---
  51. ## AI-Assisted Development on Databricks
  52. Databricks offers two paths for AI-assisted coding. Choose the one that matches your environment.
  53. <table>
  54. <tr>
  55. <td width="50%" align="center" valign="top">
  56. <br>
  57. <img src="https://img.shields.io/badge/Genie_Code-1B3139?style=for-the-badge&logo=databricks&logoColor=FF3621" alt="Genie Code" height="42">
  58. <br><br>
  59. **Free, first-party AI coding inside Databricks**
  60. Built into every Databricks workspace with deep native product context — your notebooks, jobs, and Unity Catalog data are already in scope. Ideal for users who have not started using AI-driven development tools or that are comfortable in Databricks workspace.
  61. </td>
  62. <td width="50%" align="center" valign="top">
  63. <br>
  64. <img src="https://img.shields.io/badge/AI_Dev_Kit-FF3621?style=for-the-badge&logo=databricks&logoColor=white" alt="AI Dev Kit" height="42">
  65. <br><br>
  66. **Databricks expertise, in the editor you already use**
  67. Curated by Databricks field experts. Brings tutorials, patterns, and the official Databricks agent skills to your AI coding tools to build on Databricks — wherever you're already coding.
  68. <br>
  69. <img src="https://img.shields.io/badge/Claude_Code-D97757?style=flat-square&logo=anthropic&logoColor=white" alt="Claude Code">
  70. <img src="https://img.shields.io/badge/Cursor-000000?style=flat-square&logo=cursor&logoColor=white" alt="Cursor">
  71. <img src="https://img.shields.io/badge/Codex-412991?style=flat-square&logo=openai&logoColor=white" alt="Codex">
  72. <img src="https://img.shields.io/badge/Gemini-1B72E8?style=flat-square&logo=googlegemini&logoColor=white" alt="Gemini CLI">
  73. <img src="https://img.shields.io/badge/Copilot-000000?style=flat-square&logo=github&logoColor=white" alt="GitHub Copilot">
  74. <br>
  75. <sub>+ Antigravity · Windsurf · OpenCode · and more!</sub>
  76. </td>
  77. </tr>
  78. <tr>
  79. <td align="center">
  80. <a href="https://docs.databricks.com/aws/en/genie-code/"><img src="https://img.shields.io/badge/Learn_more-→-1B3139?style=for-the-badge" alt="Learn more" height="36"></a>
  81. <br>
  82. </td>
  83. <td align="center">
  84. <a href="#install-in-existing-project"><img src="https://img.shields.io/badge/Get_started-→-FF3621?style=for-the-badge" alt="Get started" height="36"></a>
  85. <br>
  86. </td>
  87. </tr>
  88. </table>
  89. ---
  90. ## What Can I Build?
  91. - **Spark Declarative Pipelines** (streaming tables, CDC, SCD Type 2, Auto Loader)
  92. - **Databricks Jobs** (scheduled workflows, multi-task DAGs)
  93. - **AI/BI Dashboards** (visualizations, KPIs, analytics)
  94. - **Unity Catalog** (tables, volumes, governance)
  95. - **Genie Spaces** (natural language data exploration)
  96. - **Knowledge Assistants** (RAG-based document Q&A)
  97. - **MLflow Experiments** (evaluation, scoring, traces)
  98. - **Model Serving** (deploy ML models and AI agents to endpoints)
  99. - **Databricks Apps** (full-stack web applications with foundation model integration)
  100. - ...and more
  101. > **Building a Databricks App?** The `databricks-apps-python` skill defaults to AppKit (TypeScript + React) — the recommended path for most Apps use cases — and falls back to the Python frameworks (Dash, Streamlit, Flask, FastAPI, Gradio, Reflex) when you need a specific one. If you specifically need the APX (FastAPI + React) framework, that skill now lives in the [databricks-solutions/apx](https://github.com/databricks-solutions/apx) repo.
  102. ---
  103. ## Quick Start
  104. ### Prerequisites
  105. - [Databricks CLI](https://docs.databricks.com/aws/en/dev-tools/cli/) **v1.0.0+** - Command line interface for Databricks (v1.0.0+ ships `databricks aitools`, which installs most skills)
  106. - AI coding environment (one or more):
  107. - [Claude Code](https://claude.ai/code)
  108. - [Cursor](https://cursor.com)
  109. - [Gemini CLI](https://github.com/google-gemini/gemini-cli)
  110. - [Antigravity](https://antigravity.google)
  111. - [Codex](https://openai.com/codex/)
  112. - [Copilot](https://github.com/features/copilot/cli)
  113. - [Windsurf](https://windsurf.com)
  114. - [OpenCode](https://opencode.ai)
  115. - [Kiro](https://kiro.dev)
  116. ### Install in existing project
  117. By default this will install at a project level rather than a user level. This is often a good fit, but requires you to run your client from the exact directory that was used for the install. You can install for your user across all projects on the machine by passing `--global` or setting the scope to global in the interactive install.
  118. _Note: Project configuration files can be found under these folder: .claude, .cursor, .gemini, .codex, .github, .agents, .windsurf, .codeium, .opencode, .kiro, or opencode.json_
  119. #### Mac / Linux
  120. **Basic installation** (uses DEFAULT profile, project scope)
  121. ```bash
  122. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh)
  123. ```
  124. <details>
  125. <summary><strong>Advanced Options</strong> (click to expand)</summary>
  126. **Global installation with force reinstall**
  127. ```bash
  128. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --global --force
  129. ```
  130. **Install specific skills only**
  131. ```bash
  132. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --skills databricks-core,databricks-bundles,databricks-pipelines
  133. ```
  134. </details>
  135. **Next steps:** Respond to interactive prompts and follow the on-screen instructions.
  136. - Note: Cursor and Copilot require updating settings manually after install.
  137. #### Windows (PowerShell)
  138. **Basic installation** (uses DEFAULT profile, project scope)
  139. ```powershell
  140. irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 | iex
  141. ```
  142. <details>
  143. <summary><strong>Advanced Options</strong> (click to expand)</summary>
  144. **Download script first**
  145. ```powershell
  146. irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 -OutFile install.ps1
  147. ```
  148. **Global installation with force reinstall**
  149. ```powershell
  150. .\install.ps1 -Global -Force
  151. ```
  152. **Install for specific tools only**
  153. ```powershell
  154. .\install.ps1 -Tools cursor,gemini,antigravity
  155. ```
  156. </details>
  157. **Next steps:** Respond to interactive prompts and follow the on-screen instructions.
  158. - Note: Cursor and Copilot require updating settings manually after install.
  159. ### Where Skills Come From
  160. Most skills are delivered as **Databricks AI Tools** through the Databricks CLI. The installer assembles them
  161. from two upstream sources — nothing is bundled in this repo anymore:
  162. | Source | Skills | Mechanism |
  163. |--------|--------|-----------|
  164. | [databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) | Most Databricks skills (jobs, pipelines, DABs, SQL, Unity Catalog, apps, Genie, …) | Delegated to `databricks aitools install` — requires **Databricks CLI v1.0.0+** |
  165. | [mlflow/skills](https://github.com/mlflow/skills) | MLflow skills | Fetched from `main` (override with `MLFLOW_REF`) |
  166. Skills installed via `databricks aitools` are managed by the CLI afterwards — update them with `databricks aitools update` and remove them with `databricks aitools uninstall`. Gemini CLI and Kiro are installed through the CLI too when it is new enough (v1.12.0+ for Gemini CLI, v1.20.0+ for Kiro). For tools the CLI can't target (Windsurf, or Gemini CLI/Kiro on an older CLI), the installer installs the same skills into each tool's skills directory.
  167. Use `--list-skills` to see every skill and profile, and `--dry-run` to preview exactly what an install would do (resolved refs and the `aitools` command) without changing anything. Some skills were renamed or consolidated in the move — see [Breaking change: skill sources and names](#breaking-change-skill-sources-and-names) below.
  168. <details>
  169. <summary><strong>Installer environment variables</strong> (click to expand)</summary>
  170. | Variable | Default | Purpose |
  171. |----------|---------|---------|
  172. | `MLFLOW_REF` | `main` | Ref for the MLflow skills fetch (the repo is tagless) |
  173. | `INCLUDE_PRERELEASES` | `0` | Set to `1` to allow `-rc`/`-beta` tags when resolving `latest` |
  174. | `DRY_RUN` | `false` | Set to `1` to print the install plan and exit |
  175. The installer also records what it installed (resolved refs, commit SHAs, `aitools` release) in `skills.lock` inside the scope-local `.ai-dev-kit/` state directory.
  176. </details>
  177. ### Update
  178. Re-run the same install command; it detects the existing install and updates it in place.
  179. ```bash
  180. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh)
  181. ```
  182. <details>
  183. <summary><strong>Windows (PowerShell)</strong></summary>
  184. ```powershell
  185. irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 | iex
  186. ```
  187. </details>
  188. ### Uninstall
  189. Run the same script with `--uninstall`. It removes the installed skill folders (including ones from older versions), the MCP server runtime (`~/.ai-dev-kit`), and the `databricks` MCP entry from each editor's config — leaving your other MCP servers, skills, and settings untouched. Editor config files are backed up to `<file>.bak` first.
  190. ```bash
  191. # Preview what would be removed (changes nothing)
  192. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --uninstall --dry-run
  193. # Uninstall the project-scope install in the current directory
  194. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --uninstall
  195. # Uninstall a global install (add -y to skip the confirmation prompt)
  196. bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --uninstall --global -y
  197. ```
  198. <details>
  199. <summary><strong>Windows (PowerShell)</strong></summary>
  200. ```powershell
  201. .\install.ps1 -Uninstall -DryRun # preview
  202. .\install.ps1 -Uninstall # project scope
  203. .\install.ps1 -Uninstall -Global -Yes # global, no prompt
  204. ```
  205. </details>
  206. Scope mirrors install: a project uninstall touches only the current directory's config; `--global` touches only the per-user (`$HOME`) config. Run it once per scope you installed into.
  207. ### Visual Builder App
  208. Full-stack web application with a chat UI for Databricks development. Its setup is **self-contained** — you do not need to run the kit-wide `install.sh` first; the scripts build their own environment, install the sibling packages, and generate config.
  209. ```bash
  210. cd ai-dev-kit/databricks-builder-app
  211. # Local development (provisions Lakebase, installs deps, generates .env.local, starts servers)
  212. ./scripts/start_local.sh --profile <your-profile>
  213. # Deploy to Databricks Apps (Lakebase + app + permissions)
  214. ./scripts/deploy.sh my-builder-app --profile <your-profile>
  215. # Deploy with the MCP gateway enabled (name must start with mcp- for Genie Code)
  216. ./scripts/deploy.sh mcp-builder-app --enable-mcp --profile <your-profile>
  217. ```
  218. With `--enable-mcp`, the app also serves as an **MCP server** at `/mcp`, exposing the 40+ Databricks tools to [Genie Code](https://docs.databricks.com/en/genie/genie-code.html), AI Playground, and other MCP clients. The builder UI and MCP server run in a single deployment.
  219. See [`databricks-builder-app/`](databricks-builder-app/) for full documentation.
  220. ### Core Library
  221. Use `databricks-tools-core` directly in your Python projects:
  222. ```python
  223. from databricks_tools_core.sql import execute_sql
  224. results = execute_sql("SELECT * FROM my_catalog.schema.table LIMIT 10")
  225. ```
  226. Works with LangChain, OpenAI Agents SDK, or any Python framework. See [databricks-tools-core/](databricks-tools-core/) for details.
  227. ---
  228. ## Skills
  229. Skills teach your AI assistant Databricks patterns and best practices. For your **editor**, they are
  230. installed and kept up to date by the Databricks CLI (v1.0.0+), which the AI Dev Kit installer already
  231. delegates to:
  232. ```bash
  233. databricks aitools install
  234. ```
  235. By default `databricks aitools install` installs the official **`databricks` plugin** through each
  236. agent's own plugin CLI for the agents that support one — **Claude Code, Codex, and GitHub Copilot** —
  237. and writes **raw skill files** for the agents that don't (**Cursor, OpenCode, Antigravity**). Pass
  238. `--skills-only` to force raw skill files for every agent. (This official `databricks` plugin is
  239. separate from — and replaces — the retired `databricks-ai-dev-kit` plugin this repo used to publish;
  240. see [`.claude-plugin/DEPRECATED.md`](.claude-plugin/DEPRECATED.md).)
  241. Skills come from [github.com/databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills).
  242. The skill copies that used to be bundled in this repo have been removed; if you need the exact
  243. historical files, they still exist on the older release `v0.1.14` (git tag `v0.1.14`). Some skills
  244. were renamed in the move — see the breaking-change note below.
  245. ### Genie Code Skills
  246. To use skills inside **Genie Code** in a Databricks workspace, upload them to
  247. `/Workspace/Users/<you>/.assistant/skills`. `databricks aitools install` does not cover this yet, so
  248. use the provided notebook to install:
  249. **From a Databricks notebook (recommended — no local clone):**
  250. Import [`install_genie_code_skills.py`](install_genie_code_skills.py)
  251. into your workspace as a notebook and run it. It downloads skills from GitHub and uploads them via the
  252. Databricks SDK. Works on any compute, including serverless.
  253. **Customizing skills:** after upload, skills live under
  254. `/Workspace/Users/<your_user_name>/.assistant/skills`. You can modify or remove skills there, or add
  255. your own skill folders (each with a `SKILL.md`) that Genie Code will use automatically in any session.
  256. ### Breaking change: skill sources and names
  257. Skills are no longer bundled in this repository — they come from
  258. [databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) via
  259. `databricks aitools install`, and a few were **renamed or consolidated** in the move (see the rename
  260. table in the notice at the top of this README). To see the current skill names, run
  261. `databricks aitools list` (CLI v1.0.0+) or browse the
  262. [databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) repo. To reproduce
  263. the old bundled layout and names exactly, use the older release `v0.1.14` (git tag `v0.1.14`), where
  264. the frozen copies still exist.
  265. ## Architecture
  266. The AI Dev Kit ships as four composable pieces — install the whole kit, or pick just the parts you need.
  267. <p align="center">
  268. <img src="databricks-tools-core/docs/architecture.svg" alt="Architecture" width="700">
  269. </p>
  270. ## What's Included
  271. | Component | Description |
  272. |-----------|-------------|
  273. | [`Skills`] | Skills that teach Databricks patterns (installed via `databricks aitools` |
  274. | [`databricks-builder-app/`](databricks-builder-app/) | Full-stack web app with Cl
  275. aude Code integration |
  276. | [`databricks-tools-core/`](databricks-tools-core/) | Python library with high-level Databricks functions |
  277. | [`databricks-mcp-server/`](databricks-mcp-server/) | Standalone MCP server exposing 40+ Databricks tools for AI assistants (installs independently of skills) |
  278. ---
  279. ## Star History
  280. <a href="https://star-history.com/#databricks-solutions/ai-dev-kit&Date">
  281. <picture>
  282. <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=databricks-solutions/ai-dev-kit&type=Date&theme=dark" />
  283. <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=databricks-solutions/ai-dev-kit&type=Date" />
  284. <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=databricks-solutions/ai-dev-kit&type=Date" />
  285. </picture>
  286. </a>
  287. ---
  288. ## License
  289. (c) 2026 Databricks, Inc. All rights reserved.
  290. The source in this project is provided subject to the [Databricks License](https://databricks.com/db-license-source). See [LICENSE.md](LICENSE.md) for details.
  291. <details>
  292. <summary><strong>Third-Party Licenses</strong></summary>
  293. | Package | Version | License | Project URL |
  294. |---------|---------|---------|-------------|
  295. | [fastmcp](https://github.com/jlowin/fastmcp) | ≥0.1.0 | MIT | https://github.com/jlowin/fastmcp |
  296. | [mcp](https://github.com/modelcontextprotocol/python-sdk) | ≥1.0.0 | MIT | https://github.com/modelcontextprotocol/python-sdk |
  297. | [sqlglot](https://github.com/tobymao/sqlglot) | ≥20.0.0 | MIT | https://github.com/tobymao/sqlglot |
  298. | [sqlfluff](https://github.com/sqlfluff/sqlfluff) | ≥3.0.0 | MIT | https://github.com/sqlfluff/sqlfluff |
  299. | [plutoprint](https://github.com/nicvagn/plutoprint) | ==0.19.0 | MIT | https://github.com/plutoprint/plutoprint |
  300. | [claude-agent-sdk](https://github.com/anthropics/claude-code) | ≥0.1.19 | MIT | https://github.com/anthropics/claude-code |
  301. | [fastapi](https://github.com/fastapi/fastapi) | ≥0.115.8 | MIT | https://github.com/fastapi/fastapi |
  302. | [uvicorn](https://github.com/encode/uvicorn) | ≥0.34.0 | BSD-3-Clause | https://github.com/encode/uvicorn |
  303. | [httpx](https://github.com/encode/httpx) | ≥0.28.0 | BSD-3-Clause | https://github.com/encode/httpx |
  304. | [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) | ≥2.0.41 | MIT | https://github.com/sqlalchemy/sqlalchemy |
  305. | [alembic](https://github.com/sqlalchemy/alembic) | ≥1.16.1 | MIT | https://github.com/sqlalchemy/alembic |
  306. | [asyncpg](https://github.com/MagicStack/asyncpg) | ≥0.30.0 | Apache-2.0 | https://github.com/MagicStack/asyncpg |
  307. | [greenlet](https://github.com/python-greenlet/greenlet) | ≥3.0.0 | MIT | https://github.com/python-greenlet/greenlet |
  308. | [psycopg2-binary](https://github.com/psycopg/psycopg2) | ≥2.9.11 | LGPL-3.0 | https://github.com/psycopg/psycopg2 |
  309. </details>
  310. ---
  311. <details>
  312. <summary><strong>Acknowledgments</strong></summary>
  313. MCP Databricks Command Execution API from [databricks-exec-code](https://github.com/databricks-solutions/databricks-exec-code-mcp) by Natyra Bajraktari and Henryk Borzymowski.
  314. </details>
.claude-plugin/setup.sh · Bundled script · 73 lines · 1 finding marked
  1. #!/bin/bash
  2. # ============================================================
  3. # DEPRECATED: The databricks-ai-dev-kit Claude Code plugin is no longer
  4. # published (see .claude-plugin/DEPRECATED.md). Install skills via
  5. # bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh)
    • Note: Runs a downloaded script through process substitution (bash <(curl …))
    • Line 5 on GitHub
  6. # or `databricks aitools install`. This script is kept for reference and still
  7. # runs if invoked directly.
  8. # ============================================================
  9. # Post-install setup for databricks-ai-dev-kit plugin
  10. # Creates virtual environment and installs MCP server dependencies
  11. # ============================================================
  12. # SECURITY NOTE: This script runs automatically on plugin load
  13. #
  14. # What it does:
  15. # 1. Creates .venv/ with Python 3.11 (if not present)
  16. # 2. Installs databricks-tools-core (local package)
  17. # 3. Installs databricks-mcp-server (local package)
  18. # 4. Verifies MCP server import works
  19. #
  20. # What it does NOT do:
  21. # - Collect or transmit credentials
  22. # - Modify files outside the plugin directory
  23. # - Run with elevated privileges
  24. # - Access your Databricks workspace (until you authenticate)
  25. #
  26. # Source: https://github.com/databricks-solutions/ai-dev-kit
  27. # ============================================================
  28. set -e
  29. # When run via hook, CLAUDE_PLUGIN_ROOT is set; otherwise derive from script location
  30. PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
  31. TOOLS_CORE_DIR="${PLUGIN_ROOT}/databricks-tools-core"
  32. MCP_SERVER_DIR="${PLUGIN_ROOT}/databricks-mcp-server"
  33. # Idempotency check: skip if already set up and working
  34. if [ -f "${PLUGIN_ROOT}/.venv/bin/python" ] && \
  35. "${PLUGIN_ROOT}/.venv/bin/python" -c "import databricks_mcp_server" 2>/dev/null; then
  36. echo "Databricks AI Dev Kit already set up."
  37. exit 0
  38. fi
  39. echo "Setting up Databricks AI Dev Kit..." >&2
  40. # Check for uv
  41. if ! command -v uv &> /dev/null; then
  42. echo "Error: 'uv' is required but not installed." >&2
  43. echo "Install it with: curl -LsSf https://astral.sh/uv/install.sh | sh" >&2
  44. exit 1
  45. fi
  46. # Check if tools-core directory exists
  47. if [ ! -d "$TOOLS_CORE_DIR" ]; then
  48. echo "Error: databricks-tools-core not found at $TOOLS_CORE_DIR" >&2
  49. exit 1
  50. fi
  51. # Create virtual environment at plugin root
  52. cd "$PLUGIN_ROOT"
  53. uv venv --python 3.11 >&2
  54. # Install dependencies
  55. uv pip install --python .venv/bin/python -e "$TOOLS_CORE_DIR" --quiet >&2
  56. uv pip install --python .venv/bin/python -e "$MCP_SERVER_DIR" --quiet >&2
  57. # Verify installation
  58. if .venv/bin/python -c "import databricks_mcp_server" 2>/dev/null; then
  59. touch "${PLUGIN_ROOT}/.venv/.setup-complete"
  60. echo "Databricks AI Dev Kit setup complete."
  61. else
  62. echo "Error: Could not verify MCP server import after install." >&2
  63. exit 1
  64. fi
databricks-builder-app/scripts/install_builder_skills.sh · Bundled script · 326 lines · 3 findings marked
  1. #!/bin/bash
  2. #
  3. # Builder App skills installer — mirrors ai-dev-kit PR #562 (install-from-databricks-agent-skills).
  4. #
  5. # Installs Databricks agent skills via `databricks aitools install` and MLflow skills via
  6. # raw fetch from mlflow/skills. Output lands in .claude/skills/ as real directories (copied
  7. # from the aitools canonical store) so deploy bundles are self-contained.
  8. #
  9. # Usage:
  10. # ./scripts/install_builder_skills.sh # from builder-app root
  11. # ./scripts/install_builder_skills.sh --silent # non-interactive (deploy)
  12. # ./scripts/install_builder_skills.sh --profile prod
  13. #
  14. # Requires: Databricks CLI v1.0.0+ (databricks aitools), curl, git (for ref resolution).
  15. #
  16. set -euo pipefail
  17. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
  18. PROJECT_DIR="${PROJECT_DIR:-$(dirname "$SCRIPT_DIR")}"
  19. MIN_AITOOLS_CLI_VERSION="1.0.0"
  20. # CLI ≥1.6 is plugin-first; Builder App needs raw skill files via --skills-only.
  21. SKILLS_ONLY_CLI_VERSION="1.6.0"
  22. # Pinned so two deploys of the same builder-app commit ship identical skill content.
  23. # mlflow/skills publishes no tags, so this is a commit SHA on main; bump deliberately
  24. # after reviewing upstream changes. Override with MLFLOW_REF=main to track latest.
  25. MLFLOW_REF="${MLFLOW_REF:-c8228eef0da8d18ac34aa632e1276ce7da985363}"
  26. PROFILE="${DATABRICKS_CONFIG_PROFILE:-DEFAULT}"
  27. SILENT=false
  28. INSTALL_EXPERIMENTAL=true
  29. # Set ALLOW_EMPTY_MLFLOW_SKILLS=1 to allow deploy/setup when MLflow fetch returns zero skills.
  30. # MLflow skills (mlflow/skills repo)
  31. MLFLOW_SKILLS="agent-evaluation analyze-mlflow-chat-session analyze-mlflow-trace instrumenting-with-mlflow-tracing mlflow-onboarding querying-mlflow-metrics retrieving-mlflow-traces searching-mlflow-docs"
  32. MLFLOW_BASE_URL="https://raw.githubusercontent.com/mlflow/skills"
  33. # Offline snapshot used only when `databricks aitools list` is unavailable. It is a
  34. # point-in-time copy and will drift from the live inventory — bump the stamp whenever
  35. # it is refreshed so warnings say which vintage was installed.
  36. AGENT_SKILLS_SNAPSHOT_VERSION="0.2.3"
  37. AGENT_B_STABLE_FALLBACK="databricks-apps databricks-core databricks-dabs databricks-jobs databricks-lakebase databricks-model-serving databricks-pipelines databricks-serverless-migration databricks-vector-search"
  38. AGENT_B_EXPERIMENTAL_FALLBACK="databricks-agent-bricks databricks-ai-functions databricks-aibi-dashboards databricks-apps-python databricks-dbsql databricks-docs databricks-execution-compute databricks-genie databricks-iceberg databricks-lakeflow-connect databricks-metric-views databricks-mlflow-evaluation databricks-python-sdk databricks-spark-structured-streaming databricks-synthetic-data-gen databricks-unity-catalog databricks-unstructured-pdf-generation databricks-zerobus-ingest spark-python-data-source"
  39. AGENT_B_EXCLUDED="databricks-execution-compute"
  40. AGENT_B_STABLE=""
  41. AGENT_B_EXPERIMENTAL=""
  42. SELECTED_AGENT_B_SKILLS=""
  43. G='\033[0;32m' Y='\033[1;33m' R='\033[0;31m' B='\033[1m' D='\033[2m' N='\033[0m'
  44. msg() { [ "$SILENT" = true ] || echo -e " $*"; }
  45. ok() { [ "$SILENT" = true ] || echo -e " ${G}✓${N} $*"; }
  46. # Warnings always print (even --silent/deploy) so soft failures are not hidden.
  47. warn() { echo -e " ${Y}!${N} $*" >&2; }
  48. die() { echo -e " ${R}✗${N} $*" >&2; exit 1; }
  49. step() { [ "$SILENT" = true ] || echo -e "\n${B}$*${N}"; }
  50. _in_list() { echo "$2" | tr ' ' '\n' | grep -Fxq "$1"; }
  51. _count() { echo $#; }
  52. version_gte() {
  53. printf '%s\n%s' "$2" "$1" | sort -V -C
  54. }
  55. while [ $# -gt 0 ]; do
  56. case $1 in
  57. -p|--profile) PROFILE="$2"; shift 2 ;;
  58. --silent) SILENT=true; shift ;;
  59. --experimental)
  60. case "${2:-}" in
  61. false|0) INSTALL_EXPERIMENTAL=false; shift 2 ;;
  62. true|1) INSTALL_EXPERIMENTAL=true; shift 2 ;;
  63. *) INSTALL_EXPERIMENTAL=true; shift ;;
  64. esac
  65. ;;
  66. -h|--help)
  67. echo "Builder App skills installer (databricks aitools + MLflow fetch)"
  68. echo ""
  69. echo "Usage: $0 [--silent] [--profile NAME] [--experimental true|false]"
  70. echo ""
  71. echo " --silent No progress output (errors still print)"
  72. echo " -p, --profile NAME Databricks CLI profile (default: DEFAULT)"
  73. echo " --experimental BOOL Include experimental agent skills (default: true)"
  74. exit 0
  75. ;;
  76. *) die "Unknown option: $1" ;;
  77. esac
  78. done
  79. # Parse `databricks aitools list -o json` into "<experimental> <name>" lines.
  80. # Exits non-zero when the payload is missing, malformed, or has no usable
  81. # entries, so the caller can distinguish "CLI absent" from "CLI changed shape".
  82. _parse_agent_b_inventory() {
  83. python3 -c '
  84. import sys, json
  85. try:
  86. data = json.loads(sys.stdin.read())
  87. except Exception:
  88. sys.exit(1)
  89. items = None
  90. if isinstance(data, list):
  91. items = data
  92. elif isinstance(data, dict):
  93. for key in ("skills", "items", "data", "results"):
  94. value = data.get(key)
  95. if isinstance(value, list):
  96. items = value
  97. break
  98. if not items:
  99. sys.exit(1)
  100. lines = []
  101. for item in items:
  102. if not isinstance(item, dict):
  103. continue
  104. name = item.get("name") or item.get("id") or ""
  105. if not name:
  106. continue
  107. experimental = item.get("experimental")
  108. if isinstance(experimental, str):
  109. experimental = experimental.strip().lower() in ("true", "1", "yes")
  110. lines.append(("true" if experimental else "false") + " " + str(name))
  111. if not lines:
  112. sys.exit(1)
  113. print("\n".join(lines))
  114. '
  115. }
  116. fetch_agent_b_inventory() {
  117. [ -n "$AGENT_B_STABLE" ] && return
  118. local json="" list_rc=0
  119. if command -v databricks >/dev/null 2>&1; then
  120. json=$(databricks aitools list -o json 2>/dev/null) || list_rc=$?
  121. else
  122. list_rc=127
  123. fi
  124. if [ "$list_rc" -eq 0 ] && [ -n "$json" ]; then
  125. local parsed=""
  126. parsed=$(printf '%s' "$json" | _parse_agent_b_inventory) || parsed=""
  127. if [ -n "$parsed" ]; then
  128. AGENT_B_STABLE=$(echo "$parsed" | awk '$1=="false"{print $2}' | tr '\n' ' ')
  129. AGENT_B_EXPERIMENTAL=$(echo "$parsed" | awk '$1=="true"{print $2}' | tr '\n' ' ')
  130. [ -n "$AGENT_B_STABLE" ] && return
  131. fi
  132. # The CLI answered but we could not read it. Falling back here would ship a
  133. # silently different skill set behind a green check, so treat it as fatal.
  134. if [ "${ALLOW_STALE_AGENT_SKILLS:-0}" = "1" ]; then
  135. warn "Unreadable 'databricks aitools list' output; using snapshot v${AGENT_SKILLS_SNAPSHOT_VERSION} (ALLOW_STALE_AGENT_SKILLS=1)"
  136. else
  137. die "Could not parse 'databricks aitools list -o json' — the CLI output format may have changed.
  138. Set ALLOW_STALE_AGENT_SKILLS=1 to install the offline snapshot (v${AGENT_SKILLS_SNAPSHOT_VERSION}) instead."
  139. fi
  140. fi
  141. AGENT_B_STABLE="$AGENT_B_STABLE_FALLBACK"
  142. AGENT_B_EXPERIMENTAL="$AGENT_B_EXPERIMENTAL_FALLBACK"
  143. warn "Databricks CLI inventory unavailable; using offline agent-skills snapshot v${AGENT_SKILLS_SNAPSHOT_VERSION} (may differ from the live aitools inventory)"
  144. }
  145. resolve_all_agent_skills() {
  146. fetch_agent_b_inventory
  147. SELECTED_AGENT_B_SKILLS=""
  148. local skill
  149. for skill in $AGENT_B_STABLE $AGENT_B_EXPERIMENTAL; do
  150. _in_list "$skill" "$AGENT_B_EXCLUDED" && continue
  151. [ "$INSTALL_EXPERIMENTAL" = false ] && _in_list "$skill" "$AGENT_B_EXPERIMENTAL" && continue
  152. SELECTED_AGENT_B_SKILLS="${SELECTED_AGENT_B_SKILLS:+$SELECTED_AGENT_B_SKILLS }$skill"
  153. done
  154. }
  155. ensure_aitools_cli() {
  156. local cli_version=""
  157. if command -v databricks >/dev/null 2>&1; then
  158. cli_version=$(databricks --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)
  159. fi
  160. if [ -n "$cli_version" ] && version_gte "$cli_version" "$MIN_AITOOLS_CLI_VERSION"; then
  161. return 0
  162. fi
  163. local found_msg="Databricks CLI not found."
  164. [ -n "$cli_version" ] && found_msg="Databricks CLI v${cli_version} is too old."
  165. die "$found_msg Agent skills require Databricks CLI v${MIN_AITOOLS_CLI_VERSION}+.
  166. Upgrade: curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 191 on GitHub
  167. Then re-run this installer."
  168. }
  169. agent_b_needs_experimental() {
  170. local skill
  171. for skill in $SELECTED_AGENT_B_SKILLS; do
  172. _in_list "$skill" "$AGENT_B_EXPERIMENTAL" && return 0
  173. done
  174. return 1
  175. }
  176. # Remove stale real copies of agent-managed skill dirs (aitools won't overwrite them).
  177. cleanup_stale_agent_skills() {
  178. local skills_dir="$PROJECT_DIR/.claude/skills"
  179. [ -d "$skills_dir" ] || return 0
  180. local skill
  181. for skill in $SELECTED_AGENT_B_SKILLS; do
  182. if [ -d "$skills_dir/$skill" ] && [ ! -L "$skills_dir/$skill" ]; then
  183. # :? guards against an empty expansion turning this into `rm -rf /`.
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 210 on GitHub
  184. rm -rf "${skills_dir:?}/${skill:?}"
  185. msg "${D}Removed stale bundled copy: $skill${N}"
  186. fi
  187. done
  188. }
  189. install_agent_skills() {
  190. [ -z "$SELECTED_AGENT_B_SKILLS" ] && return 0
  191. step "Installing agent skills (via databricks aitools)"
  192. ensure_aitools_cli
  193. cleanup_stale_agent_skills
  194. local skills_csv exp_flag="" skills_only_flag="" count cli_version=""
  195. skills_csv=$(echo "$SELECTED_AGENT_B_SKILLS" | tr -s ' ' ',' | sed 's/^,//;s/,$//')
  196. agent_b_needs_experimental && exp_flag="--experimental"
  197. count=$(_count $SELECTED_AGENT_B_SKILLS)
  198. # Plugin-first CLIs install the agent plugin by default; --skills-only forces
  199. # raw files into .databricks/aitools/skills so we can copy them for deploy.
  200. # Older CLIs reject the flag — only pass it when supported.
  201. cli_version=$(databricks --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)
  202. if [ -n "$cli_version" ] && version_gte "$cli_version" "$SKILLS_ONLY_CLI_VERSION"; then
  203. skills_only_flag="--skills-only"
  204. fi
  205. cd "$PROJECT_DIR"
  206. local aitools_out aitools_rc attempt
  207. for attempt in 1 2; do
  208. aitools_out=$(databricks aitools install --scope project --agents claude-code --skills "$skills_csv" $skills_only_flag $exp_flag -p "$PROFILE" 2>&1) && aitools_rc=0 || aitools_rc=$?
  209. if [ "$aitools_rc" -eq 0 ]; then
  210. break
  211. fi
  212. if [ "$attempt" -eq 1 ]; then
  213. warn "databricks aitools install failed (attempt 1); retrying once..."
  214. sleep 2
  215. fi
  216. done
  217. if [ "$SILENT" != true ] && [ -n "$aitools_out" ]; then
  218. echo "$aitools_out" | grep -v 'does not support project-scoped skills' || true
  219. fi
  220. if [ "$aitools_rc" -ne 0 ]; then
  221. # Always surface the CLI error — silent mode used to swallow it entirely.
  222. echo "$aitools_out" >&2
  223. die "databricks aitools install failed (exit $aitools_rc)"
  224. fi
  225. ok "Agent skills ($count) installed via aitools"
  226. # Copy real files into .claude/skills for deploy bundling (not symlinks).
  227. local store="$PROJECT_DIR/.databricks/aitools/skills"
  228. local dest="$PROJECT_DIR/.claude/skills"
  229. mkdir -p "$dest"
  230. for skill in $SELECTED_AGENT_B_SKILLS; do
  231. if [ ! -d "$store/$skill" ]; then
  232. warn "Agent skill '$skill' missing from aitools store — skipped"
  233. continue
  234. fi
  235. # :? guards against an empty expansion turning this into `rm -rf /`.
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 268 on GitHub
  236. rm -rf "${dest:?}/${skill:?}"
  237. cp -R "$store/$skill" "$dest/$skill"
  238. done
  239. ok "Agent skills copied to .claude/skills/"
  240. }
  241. install_mlflow_skills() {
  242. step "Installing MLflow skills"
  243. local dest="$PROJECT_DIR/.claude/skills"
  244. mkdir -p "$dest"
  245. local mlflow_raw_url="$MLFLOW_BASE_URL/${MLFLOW_REF}"
  246. local count=0 skill
  247. for skill in $MLFLOW_SKILLS; do
  248. local dest_dir="$dest/$skill"
  249. mkdir -p "$dest_dir"
  250. local url="$mlflow_raw_url/$skill/SKILL.md"
  251. if curl -fsSL "$url" -o "$dest_dir/SKILL.md" 2>/dev/null; then
  252. for ref in reference.md examples.md api.md; do
  253. curl -fsSL "$mlflow_raw_url/$skill/$ref" -o "$dest_dir/$ref" 2>/dev/null || true
  254. done
  255. count=$((count + 1))
  256. else
  257. rm -rf "$dest_dir"
  258. warn "Could not fetch MLflow skill: $skill"
  259. fi
  260. done
  261. local expected
  262. expected=$(_count $MLFLOW_SKILLS)
  263. # A partial fetch is as silent as an empty one: some skills simply go missing
  264. # while the installer still reports success. Treat any shortfall as an error.
  265. local allow_shortfall="${ALLOW_PARTIAL_MLFLOW_SKILLS:-${ALLOW_EMPTY_MLFLOW_SKILLS:-0}}"
  266. if [ "$count" -lt "$expected" ]; then
  267. if [ "$allow_shortfall" = "1" ]; then
  268. warn "MLflow skills incomplete: ${count}/${expected} installed (override set)"
  269. else
  270. die "MLflow skills incomplete: ${count}/${expected} installed from ${mlflow_raw_url}.
  271. Set ALLOW_PARTIAL_MLFLOW_SKILLS=1 to continue anyway, or point MLFLOW_REF at a reachable ref."
  272. fi
  273. else
  274. ok "MLflow skills ($count/$expected) → .claude/skills/"
  275. fi
  276. }
  277. # ─── Main ─────────────────────────────────────────────────────
  278. # Guarded so tests can source this file for its helpers without installing.
  279. if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
  280. resolve_all_agent_skills
  281. install_agent_skills
  282. install_mlflow_skills
  283. if [ "$SILENT" != true ]; then
  284. total=$(_count $SELECTED_AGENT_B_SKILLS $MLFLOW_SKILLS)
  285. echo ""
  286. echo -e " ${G}Done.${N} ${total} skills available under .claude/skills/"
  287. fi
  288. fi

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.