ai-dev-kit
Databricks Toolkit for Coding Agents provided by Field Engineering
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
Its scripts were only partly scanned.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/databricks-solutions/ai-dev-kit with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/databricks-solutions/ai-dev-kit'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/databricks-solutions--ai-dev-kit)HTML
<a href="/listings/databricks-solutions--ai-dev-kit"><img src="/api/listings/databricks-solutions--ai-dev-kit/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
| Options | Best For | Start Here | |----------------------------------|----------|------------| | :star: [**Install Skills**](#install-in-existing-project) | **Start here!** Follow quick install instructions to add Databricks skills for your user or existing project folder | [Quick Start (install)](#install-in-existing-project) | [**Genie Code Skills**](databricks-skills/) | Upload selected skills into your workspace for Genie Code. Note: much of the content in the public Databricks skills is already included in Genie Code. | [Genie Code skills (install)](#genie-code-skills) | | [**Visual Builder App**](#visual-builder-app) | Web-based UI for Databricks development | `databricks-builder-app/` |
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json · Manifest · 11 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- {
- "name": "databricks-ai-dev-kit",
- "owner": {
- "name": "Databricks Solutions"
- },
- "metadata": {
- "description": "DEPRECATED — the databricks-ai-dev-kit plugin is no longer published. Install skills with install.sh or `databricks aitools install` (which installs the official `databricks` plugin for Claude Code/Codex/Copilot, or skill files for other agents). See .claude-plugin/DEPRECATED.md.",
- "version": "1.1.9"
- },
- "plugins": []
- }
README.md · README · 447 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- # Databricks AI Dev Kit
- <p align="center">
- <img src="https://img.shields.io/badge/Databricks-Certified%20Gold%20Project-FFD700?style=for-the-badge&logo=databricks&logoColor=black" alt="Databricks Certified Gold Project">
- </p>
- ---
- > 📣 **A big step forward: AI Dev Kit skills are now official Databricks AI Tools**
- >
- > The skills from this AI Dev Kit repository are now delivered as part of **Databricks AI Tools**,
- > a Databricks **engineering-owned repository**, built and maintained in close collaboration
- > with field engineering.
- >
- > **Databricks AI Tools**
- > are installed and kept up to date directly through the **Databricks CLI**
- > (`databricks aitools install`), or through the AI Dev Kit installer, which delegates to the CLI and
- > still guides you through the process.
- >
- > **AI Tools work with any agent** and include a
- > first-class **Genie One** integration.
- ---
- **Already installed?** Re-run the [Quick Start (install)](#install-in-existing-project) in this repo to uninstall older skills and the MCP
- server before installing the official AI Tools — see [Where Skills Come From](#where-skills-come-from).
- **What stays:** The MCP server and Builder App remain in this repository. The Builder App will keep
- being developed and improved, and the MCP server will be maintained and updated on a best-effort
- basis as GitHub issues are filed.
- **What's next:** AI Dev Kit will continue to guide you through setting up your AI coding environment
- and be a place to find experimental tools developed by Field Engineering. We're working on several
- tutorials to help you get started using coding agents for building on Databricks — including getting
- started with Genie Code and Omnigent — which will land in a future update. The bundled skill files
- in this repo are deprecated and kept for reference only.
- **A few skills were renamed or merged** in the official install. Most names are unchanged; the
- exceptions are:
- | Previously (AI Dev Kit) | Official Databricks skills |
- |---|---|
- | `databricks-bundles` | `databricks-dabs` |
- | `databricks-genie` | `databricks-genie-agents` |
- | `databricks-spark-declarative-pipelines` | `databricks-pipelines` |
- | `databricks-lakebase-autoscale`, `databricks-lakebase-provisioned` | `databricks-lakebase` (merged) |
- | `databricks-config` | `databricks-core` (merged) |
- > 🔒 Proactive Dependency Security
- > As part of our commitment to supply chain integrity, we continually monitor our dependency tree against known vulnerabilities and industry advisories. In response to a recently disclosed supply chain incident affecting litellm versions 1.82.7–1.82.8, we have audited our packages and removed the litellm dependency for most usage. It is solely used in the test directory for skills evaluation and optimization, and has been pinned to a safe version.
- > For full third-party attribution, see NOTICE.txt.
- ---
- ## Install or Upgrade
- | Options | Best For | Start Here |
- |----------------------------------|----------|------------|
- | :star: [**Install Skills**](#install-in-existing-project) | **Start here!** Follow quick install instructions to add Databricks skills for your user or existing project folder | [Quick Start (install)](#install-in-existing-project)
- | [**Genie Code Skills**](databricks-skills/) | Upload selected skills into your workspace for Genie Code. Note: much of the content in the public Databricks skills is already included in Genie Code. | [Genie Code skills (install)](#genie-code-skills) |
- | [**Visual Builder App**](#visual-builder-app) | Web-based UI for Databricks development | `databricks-builder-app/` |
- | [**MCP Tools**](databricks-mcp-server/) | Standalone MCP server exposing Databricks actions to AI clients. Note: We recommend skills instead, but maintain this as a good foundation if a custom MCP server is required. | [Register MCP server](databricks-mcp-server/) |
- ---
- ## AI-Assisted Development on Databricks
- Databricks offers two paths for AI-assisted coding. Choose the one that matches your environment.
- <table>
- <tr>
- <td width="50%" align="center" valign="top">
- <br>
- <img src="https://img.shields.io/badge/Genie_Code-1B3139?style=for-the-badge&logo=databricks&logoColor=FF3621" alt="Genie Code" height="42">
- <br><br>
- **Free, first-party AI coding inside Databricks**
- Built into every Databricks workspace with deep native product context — your notebooks, jobs, and Unity Catalog data are already in scope. Ideal for users who have not started using AI-driven development tools or that are comfortable in Databricks workspace.
- </td>
- <td width="50%" align="center" valign="top">
- <br>
- <img src="https://img.shields.io/badge/AI_Dev_Kit-FF3621?style=for-the-badge&logo=databricks&logoColor=white" alt="AI Dev Kit" height="42">
- <br><br>
- **Databricks expertise, in the editor you already use**
- Curated by Databricks field experts. Brings tutorials, patterns, and the official Databricks agent skills to your AI coding tools to build on Databricks — wherever you're already coding.
- <br>
- <img src="https://img.shields.io/badge/Claude_Code-D97757?style=flat-square&logo=anthropic&logoColor=white" alt="Claude Code">
- <img src="https://img.shields.io/badge/Cursor-000000?style=flat-square&logo=cursor&logoColor=white" alt="Cursor">
- <img src="https://img.shields.io/badge/Codex-412991?style=flat-square&logo=openai&logoColor=white" alt="Codex">
- <img src="https://img.shields.io/badge/Gemini-1B72E8?style=flat-square&logo=googlegemini&logoColor=white" alt="Gemini CLI">
- <img src="https://img.shields.io/badge/Copilot-000000?style=flat-square&logo=github&logoColor=white" alt="GitHub Copilot">
- <br>
- <sub>+ Antigravity · Windsurf · OpenCode · and more!</sub>
- </td>
- </tr>
- <tr>
- <td align="center">
- <a href="https://docs.databricks.com/aws/en/genie-code/"><img src="https://img.shields.io/badge/Learn_more-→-1B3139?style=for-the-badge" alt="Learn more" height="36"></a>
- <br>
- </td>
- <td align="center">
- <a href="#install-in-existing-project"><img src="https://img.shields.io/badge/Get_started-→-FF3621?style=for-the-badge" alt="Get started" height="36"></a>
- <br>
- </td>
- </tr>
- </table>
- ---
- ## What Can I Build?
- - **Spark Declarative Pipelines** (streaming tables, CDC, SCD Type 2, Auto Loader)
- - **Databricks Jobs** (scheduled workflows, multi-task DAGs)
- - **AI/BI Dashboards** (visualizations, KPIs, analytics)
- - **Unity Catalog** (tables, volumes, governance)
- - **Genie Spaces** (natural language data exploration)
- - **Knowledge Assistants** (RAG-based document Q&A)
- - **MLflow Experiments** (evaluation, scoring, traces)
- - **Model Serving** (deploy ML models and AI agents to endpoints)
- - **Databricks Apps** (full-stack web applications with foundation model integration)
- - ...and more
- > **Building a Databricks App?** The `databricks-apps-python` skill defaults to AppKit (TypeScript + React) — the recommended path for most Apps use cases — and falls back to the Python frameworks (Dash, Streamlit, Flask, FastAPI, Gradio, Reflex) when you need a specific one. If you specifically need the APX (FastAPI + React) framework, that skill now lives in the [databricks-solutions/apx](https://github.com/databricks-solutions/apx) repo.
- ---
- ## Quick Start
- ### Prerequisites
- - [Databricks CLI](https://docs.databricks.com/aws/en/dev-tools/cli/) **v1.0.0+** - Command line interface for Databricks (v1.0.0+ ships `databricks aitools`, which installs most skills)
- - AI coding environment (one or more):
- - [Claude Code](https://claude.ai/code)
- - [Cursor](https://cursor.com)
- - [Gemini CLI](https://github.com/google-gemini/gemini-cli)
- - [Antigravity](https://antigravity.google)
- - [Codex](https://openai.com/codex/)
- - [Copilot](https://github.com/features/copilot/cli)
- - [Windsurf](https://windsurf.com)
- - [OpenCode](https://opencode.ai)
- - [Kiro](https://kiro.dev)
- ### Install in existing project
- By default this will install at a project level rather than a user level. This is often a good fit, but requires you to run your client from the exact directory that was used for the install. You can install for your user across all projects on the machine by passing `--global` or setting the scope to global in the interactive install.
- _Note: Project configuration files can be found under these folder: .claude, .cursor, .gemini, .codex, .github, .agents, .windsurf, .codeium, .opencode, .kiro, or opencode.json_
- #### Mac / Linux
- **Basic installation** (uses DEFAULT profile, project scope)
- ```bash
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh)
- ```
- <details>
- <summary><strong>Advanced Options</strong> (click to expand)</summary>
- **Global installation with force reinstall**
- ```bash
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --global --force
- ```
- **Install specific skills only**
- ```bash
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --skills databricks-core,databricks-bundles,databricks-pipelines
- ```
- </details>
- **Next steps:** Respond to interactive prompts and follow the on-screen instructions.
- - Note: Cursor and Copilot require updating settings manually after install.
- #### Windows (PowerShell)
- **Basic installation** (uses DEFAULT profile, project scope)
- ```powershell
- irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 | iex
- ```
- <details>
- <summary><strong>Advanced Options</strong> (click to expand)</summary>
- **Download script first**
- ```powershell
- irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 -OutFile install.ps1
- ```
- **Global installation with force reinstall**
- ```powershell
- .\install.ps1 -Global -Force
- ```
- **Install for specific tools only**
- ```powershell
- .\install.ps1 -Tools cursor,gemini,antigravity
- ```
- </details>
- **Next steps:** Respond to interactive prompts and follow the on-screen instructions.
- - Note: Cursor and Copilot require updating settings manually after install.
- ### Where Skills Come From
- Most skills are delivered as **Databricks AI Tools** through the Databricks CLI. The installer assembles them
- from two upstream sources — nothing is bundled in this repo anymore:
- | Source | Skills | Mechanism |
- |--------|--------|-----------|
- | [databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) | Most Databricks skills (jobs, pipelines, DABs, SQL, Unity Catalog, apps, Genie, …) | Delegated to `databricks aitools install` — requires **Databricks CLI v1.0.0+** |
- | [mlflow/skills](https://github.com/mlflow/skills) | MLflow skills | Fetched from `main` (override with `MLFLOW_REF`) |
- Skills installed via `databricks aitools` are managed by the CLI afterwards — update them with `databricks aitools update` and remove them with `databricks aitools uninstall`. Gemini CLI and Kiro are installed through the CLI too when it is new enough (v1.12.0+ for Gemini CLI, v1.20.0+ for Kiro). For tools the CLI can't target (Windsurf, or Gemini CLI/Kiro on an older CLI), the installer installs the same skills into each tool's skills directory.
- Use `--list-skills` to see every skill and profile, and `--dry-run` to preview exactly what an install would do (resolved refs and the `aitools` command) without changing anything. Some skills were renamed or consolidated in the move — see [Breaking change: skill sources and names](#breaking-change-skill-sources-and-names) below.
- <details>
- <summary><strong>Installer environment variables</strong> (click to expand)</summary>
- | Variable | Default | Purpose |
- |----------|---------|---------|
- | `MLFLOW_REF` | `main` | Ref for the MLflow skills fetch (the repo is tagless) |
- | `INCLUDE_PRERELEASES` | `0` | Set to `1` to allow `-rc`/`-beta` tags when resolving `latest` |
- | `DRY_RUN` | `false` | Set to `1` to print the install plan and exit |
- The installer also records what it installed (resolved refs, commit SHAs, `aitools` release) in `skills.lock` inside the scope-local `.ai-dev-kit/` state directory.
- </details>
- ### Update
- Re-run the same install command; it detects the existing install and updates it in place.
- ```bash
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh)
- ```
- <details>
- <summary><strong>Windows (PowerShell)</strong></summary>
- ```powershell
- irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 | iex
- ```
- </details>
- ### Uninstall
- Run the same script with `--uninstall`. It removes the installed skill folders (including ones from older versions), the MCP server runtime (`~/.ai-dev-kit`), and the `databricks` MCP entry from each editor's config — leaving your other MCP servers, skills, and settings untouched. Editor config files are backed up to `<file>.bak` first.
- ```bash
- # Preview what would be removed (changes nothing)
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --uninstall --dry-run
- # Uninstall the project-scope install in the current directory
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --uninstall
- # Uninstall a global install (add -y to skip the confirmation prompt)
- bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh) --uninstall --global -y
- ```
- <details>
- <summary><strong>Windows (PowerShell)</strong></summary>
- ```powershell
- .\install.ps1 -Uninstall -DryRun # preview
- .\install.ps1 -Uninstall # project scope
- .\install.ps1 -Uninstall -Global -Yes # global, no prompt
- ```
- </details>
- Scope mirrors install: a project uninstall touches only the current directory's config; `--global` touches only the per-user (`$HOME`) config. Run it once per scope you installed into.
- ### Visual Builder App
- Full-stack web application with a chat UI for Databricks development. Its setup is **self-contained** — you do not need to run the kit-wide `install.sh` first; the scripts build their own environment, install the sibling packages, and generate config.
- ```bash
- cd ai-dev-kit/databricks-builder-app
- # Local development (provisions Lakebase, installs deps, generates .env.local, starts servers)
- ./scripts/start_local.sh --profile <your-profile>
- # Deploy to Databricks Apps (Lakebase + app + permissions)
- ./scripts/deploy.sh my-builder-app --profile <your-profile>
- # Deploy with the MCP gateway enabled (name must start with mcp- for Genie Code)
- ./scripts/deploy.sh mcp-builder-app --enable-mcp --profile <your-profile>
- ```
- With `--enable-mcp`, the app also serves as an **MCP server** at `/mcp`, exposing the 40+ Databricks tools to [Genie Code](https://docs.databricks.com/en/genie/genie-code.html), AI Playground, and other MCP clients. The builder UI and MCP server run in a single deployment.
- See [`databricks-builder-app/`](databricks-builder-app/) for full documentation.
- ### Core Library
- Use `databricks-tools-core` directly in your Python projects:
- ```python
- from databricks_tools_core.sql import execute_sql
- results = execute_sql("SELECT * FROM my_catalog.schema.table LIMIT 10")
- ```
- Works with LangChain, OpenAI Agents SDK, or any Python framework. See [databricks-tools-core/](databricks-tools-core/) for details.
- ---
- ## Skills
- Skills teach your AI assistant Databricks patterns and best practices. For your **editor**, they are
- installed and kept up to date by the Databricks CLI (v1.0.0+), which the AI Dev Kit installer already
- delegates to:
- ```bash
- databricks aitools install
- ```
- By default `databricks aitools install` installs the official **`databricks` plugin** through each
- agent's own plugin CLI for the agents that support one — **Claude Code, Codex, and GitHub Copilot** —
- and writes **raw skill files** for the agents that don't (**Cursor, OpenCode, Antigravity**). Pass
- `--skills-only` to force raw skill files for every agent. (This official `databricks` plugin is
- separate from — and replaces — the retired `databricks-ai-dev-kit` plugin this repo used to publish;
- see [`.claude-plugin/DEPRECATED.md`](.claude-plugin/DEPRECATED.md).)
- Skills come from [github.com/databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills).
- The skill copies that used to be bundled in this repo have been removed; if you need the exact
- historical files, they still exist on the older release `v0.1.14` (git tag `v0.1.14`). Some skills
- were renamed in the move — see the breaking-change note below.
- ### Genie Code Skills
- To use skills inside **Genie Code** in a Databricks workspace, upload them to
- `/Workspace/Users/<you>/.assistant/skills`. `databricks aitools install` does not cover this yet, so
- use the provided notebook to install:
- **From a Databricks notebook (recommended — no local clone):**
- Import [`install_genie_code_skills.py`](install_genie_code_skills.py)
- into your workspace as a notebook and run it. It downloads skills from GitHub and uploads them via the
- Databricks SDK. Works on any compute, including serverless.
- **Customizing skills:** after upload, skills live under
- `/Workspace/Users/<your_user_name>/.assistant/skills`. You can modify or remove skills there, or add
- your own skill folders (each with a `SKILL.md`) that Genie Code will use automatically in any session.
- ### Breaking change: skill sources and names
- Skills are no longer bundled in this repository — they come from
- [databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) via
- `databricks aitools install`, and a few were **renamed or consolidated** in the move (see the rename
- table in the notice at the top of this README). To see the current skill names, run
- `databricks aitools list` (CLI v1.0.0+) or browse the
- [databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) repo. To reproduce
- the old bundled layout and names exactly, use the older release `v0.1.14` (git tag `v0.1.14`), where
- the frozen copies still exist.
- ## Architecture
- The AI Dev Kit ships as four composable pieces — install the whole kit, or pick just the parts you need.
- <p align="center">
- <img src="databricks-tools-core/docs/architecture.svg" alt="Architecture" width="700">
- </p>
- ## What's Included
- | Component | Description |
- |-----------|-------------|
- | [`Skills`] | Skills that teach Databricks patterns (installed via `databricks aitools` |
- | [`databricks-builder-app/`](databricks-builder-app/) | Full-stack web app with Cl
- aude Code integration |
- | [`databricks-tools-core/`](databricks-tools-core/) | Python library with high-level Databricks functions |
- | [`databricks-mcp-server/`](databricks-mcp-server/) | Standalone MCP server exposing 40+ Databricks tools for AI assistants (installs independently of skills) |
- ---
- ## Star History
- <a href="https://star-history.com/#databricks-solutions/ai-dev-kit&Date">
- <picture>
- <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=databricks-solutions/ai-dev-kit&type=Date&theme=dark" />
- <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=databricks-solutions/ai-dev-kit&type=Date" />
- <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=databricks-solutions/ai-dev-kit&type=Date" />
- </picture>
- </a>
- ---
- ## License
- (c) 2026 Databricks, Inc. All rights reserved.
- The source in this project is provided subject to the [Databricks License](https://databricks.com/db-license-source). See [LICENSE.md](LICENSE.md) for details.
- <details>
- <summary><strong>Third-Party Licenses</strong></summary>
- | Package | Version | License | Project URL |
- |---------|---------|---------|-------------|
- | [fastmcp](https://github.com/jlowin/fastmcp) | ≥0.1.0 | MIT | https://github.com/jlowin/fastmcp |
- | [mcp](https://github.com/modelcontextprotocol/python-sdk) | ≥1.0.0 | MIT | https://github.com/modelcontextprotocol/python-sdk |
- | [sqlglot](https://github.com/tobymao/sqlglot) | ≥20.0.0 | MIT | https://github.com/tobymao/sqlglot |
- | [sqlfluff](https://github.com/sqlfluff/sqlfluff) | ≥3.0.0 | MIT | https://github.com/sqlfluff/sqlfluff |
- | [plutoprint](https://github.com/nicvagn/plutoprint) | ==0.19.0 | MIT | https://github.com/plutoprint/plutoprint |
- | [claude-agent-sdk](https://github.com/anthropics/claude-code) | ≥0.1.19 | MIT | https://github.com/anthropics/claude-code |
- | [fastapi](https://github.com/fastapi/fastapi) | ≥0.115.8 | MIT | https://github.com/fastapi/fastapi |
- | [uvicorn](https://github.com/encode/uvicorn) | ≥0.34.0 | BSD-3-Clause | https://github.com/encode/uvicorn |
- | [httpx](https://github.com/encode/httpx) | ≥0.28.0 | BSD-3-Clause | https://github.com/encode/httpx |
- | [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) | ≥2.0.41 | MIT | https://github.com/sqlalchemy/sqlalchemy |
- | [alembic](https://github.com/sqlalchemy/alembic) | ≥1.16.1 | MIT | https://github.com/sqlalchemy/alembic |
- | [asyncpg](https://github.com/MagicStack/asyncpg) | ≥0.30.0 | Apache-2.0 | https://github.com/MagicStack/asyncpg |
- | [greenlet](https://github.com/python-greenlet/greenlet) | ≥3.0.0 | MIT | https://github.com/python-greenlet/greenlet |
- | [psycopg2-binary](https://github.com/psycopg/psycopg2) | ≥2.9.11 | LGPL-3.0 | https://github.com/psycopg/psycopg2 |
- </details>
- ---
- <details>
- <summary><strong>Acknowledgments</strong></summary>
- MCP Databricks Command Execution API from [databricks-exec-code](https://github.com/databricks-solutions/databricks-exec-code-mcp) by Natyra Bajraktari and Henryk Borzymowski.
- </details>
.claude-plugin/setup.sh · Bundled script · 73 lines · 1 finding marked
- #!/bin/bash
- # ============================================================
- # DEPRECATED: The databricks-ai-dev-kit Claude Code plugin is no longer
- # published (see .claude-plugin/DEPRECATED.md). Install skills via
- # bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh)
- Note: Runs a downloaded script through process substitution (bash <(curl …))
- Line 5 on GitHub
- # or `databricks aitools install`. This script is kept for reference and still
- # runs if invoked directly.
- # ============================================================
- # Post-install setup for databricks-ai-dev-kit plugin
- # Creates virtual environment and installs MCP server dependencies
- # ============================================================
- # SECURITY NOTE: This script runs automatically on plugin load
- #
- # What it does:
- # 1. Creates .venv/ with Python 3.11 (if not present)
- # 2. Installs databricks-tools-core (local package)
- # 3. Installs databricks-mcp-server (local package)
- # 4. Verifies MCP server import works
- #
- # What it does NOT do:
- # - Collect or transmit credentials
- # - Modify files outside the plugin directory
- # - Run with elevated privileges
- # - Access your Databricks workspace (until you authenticate)
- #
- # Source: https://github.com/databricks-solutions/ai-dev-kit
- # ============================================================
- set -e
- # When run via hook, CLAUDE_PLUGIN_ROOT is set; otherwise derive from script location
- PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
- TOOLS_CORE_DIR="${PLUGIN_ROOT}/databricks-tools-core"
- MCP_SERVER_DIR="${PLUGIN_ROOT}/databricks-mcp-server"
- # Idempotency check: skip if already set up and working
- if [ -f "${PLUGIN_ROOT}/.venv/bin/python" ] && \
- "${PLUGIN_ROOT}/.venv/bin/python" -c "import databricks_mcp_server" 2>/dev/null; then
- echo "Databricks AI Dev Kit already set up."
- exit 0
- fi
- echo "Setting up Databricks AI Dev Kit..." >&2
- # Check for uv
- if ! command -v uv &> /dev/null; then
- echo "Error: 'uv' is required but not installed." >&2
- echo "Install it with: curl -LsSf https://astral.sh/uv/install.sh | sh" >&2
- exit 1
- fi
- # Check if tools-core directory exists
- if [ ! -d "$TOOLS_CORE_DIR" ]; then
- echo "Error: databricks-tools-core not found at $TOOLS_CORE_DIR" >&2
- exit 1
- fi
- # Create virtual environment at plugin root
- cd "$PLUGIN_ROOT"
- uv venv --python 3.11 >&2
- # Install dependencies
- uv pip install --python .venv/bin/python -e "$TOOLS_CORE_DIR" --quiet >&2
- uv pip install --python .venv/bin/python -e "$MCP_SERVER_DIR" --quiet >&2
- # Verify installation
- if .venv/bin/python -c "import databricks_mcp_server" 2>/dev/null; then
- touch "${PLUGIN_ROOT}/.venv/.setup-complete"
- echo "Databricks AI Dev Kit setup complete."
- else
- echo "Error: Could not verify MCP server import after install." >&2
- exit 1
- fi
databricks-builder-app/scripts/install_builder_skills.sh · Bundled script · 326 lines · 3 findings marked
- #!/bin/bash
- #
- # Builder App skills installer — mirrors ai-dev-kit PR #562 (install-from-databricks-agent-skills).
- #
- # Installs Databricks agent skills via `databricks aitools install` and MLflow skills via
- # raw fetch from mlflow/skills. Output lands in .claude/skills/ as real directories (copied
- # from the aitools canonical store) so deploy bundles are self-contained.
- #
- # Usage:
- # ./scripts/install_builder_skills.sh # from builder-app root
- # ./scripts/install_builder_skills.sh --silent # non-interactive (deploy)
- # ./scripts/install_builder_skills.sh --profile prod
- #
- # Requires: Databricks CLI v1.0.0+ (databricks aitools), curl, git (for ref resolution).
- #
- set -euo pipefail
- SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
- PROJECT_DIR="${PROJECT_DIR:-$(dirname "$SCRIPT_DIR")}"
- MIN_AITOOLS_CLI_VERSION="1.0.0"
- # CLI ≥1.6 is plugin-first; Builder App needs raw skill files via --skills-only.
- SKILLS_ONLY_CLI_VERSION="1.6.0"
- # Pinned so two deploys of the same builder-app commit ship identical skill content.
- # mlflow/skills publishes no tags, so this is a commit SHA on main; bump deliberately
- # after reviewing upstream changes. Override with MLFLOW_REF=main to track latest.
- MLFLOW_REF="${MLFLOW_REF:-c8228eef0da8d18ac34aa632e1276ce7da985363}"
- PROFILE="${DATABRICKS_CONFIG_PROFILE:-DEFAULT}"
- SILENT=false
- INSTALL_EXPERIMENTAL=true
- # Set ALLOW_EMPTY_MLFLOW_SKILLS=1 to allow deploy/setup when MLflow fetch returns zero skills.
- # MLflow skills (mlflow/skills repo)
- MLFLOW_SKILLS="agent-evaluation analyze-mlflow-chat-session analyze-mlflow-trace instrumenting-with-mlflow-tracing mlflow-onboarding querying-mlflow-metrics retrieving-mlflow-traces searching-mlflow-docs"
- MLFLOW_BASE_URL="https://raw.githubusercontent.com/mlflow/skills"
- # Offline snapshot used only when `databricks aitools list` is unavailable. It is a
- # point-in-time copy and will drift from the live inventory — bump the stamp whenever
- # it is refreshed so warnings say which vintage was installed.
- AGENT_SKILLS_SNAPSHOT_VERSION="0.2.3"
- AGENT_B_STABLE_FALLBACK="databricks-apps databricks-core databricks-dabs databricks-jobs databricks-lakebase databricks-model-serving databricks-pipelines databricks-serverless-migration databricks-vector-search"
- AGENT_B_EXPERIMENTAL_FALLBACK="databricks-agent-bricks databricks-ai-functions databricks-aibi-dashboards databricks-apps-python databricks-dbsql databricks-docs databricks-execution-compute databricks-genie databricks-iceberg databricks-lakeflow-connect databricks-metric-views databricks-mlflow-evaluation databricks-python-sdk databricks-spark-structured-streaming databricks-synthetic-data-gen databricks-unity-catalog databricks-unstructured-pdf-generation databricks-zerobus-ingest spark-python-data-source"
- AGENT_B_EXCLUDED="databricks-execution-compute"
- AGENT_B_STABLE=""
- AGENT_B_EXPERIMENTAL=""
- SELECTED_AGENT_B_SKILLS=""
- G='\033[0;32m' Y='\033[1;33m' R='\033[0;31m' B='\033[1m' D='\033[2m' N='\033[0m'
- msg() { [ "$SILENT" = true ] || echo -e " $*"; }
- ok() { [ "$SILENT" = true ] || echo -e " ${G}✓${N} $*"; }
- # Warnings always print (even --silent/deploy) so soft failures are not hidden.
- warn() { echo -e " ${Y}!${N} $*" >&2; }
- die() { echo -e " ${R}✗${N} $*" >&2; exit 1; }
- step() { [ "$SILENT" = true ] || echo -e "\n${B}$*${N}"; }
- _in_list() { echo "$2" | tr ' ' '\n' | grep -Fxq "$1"; }
- _count() { echo $#; }
- version_gte() {
- printf '%s\n%s' "$2" "$1" | sort -V -C
- }
- while [ $# -gt 0 ]; do
- case $1 in
- -p|--profile) PROFILE="$2"; shift 2 ;;
- --silent) SILENT=true; shift ;;
- --experimental)
- case "${2:-}" in
- false|0) INSTALL_EXPERIMENTAL=false; shift 2 ;;
- true|1) INSTALL_EXPERIMENTAL=true; shift 2 ;;
- *) INSTALL_EXPERIMENTAL=true; shift ;;
- esac
- ;;
- -h|--help)
- echo "Builder App skills installer (databricks aitools + MLflow fetch)"
- echo ""
- echo "Usage: $0 [--silent] [--profile NAME] [--experimental true|false]"
- echo ""
- echo " --silent No progress output (errors still print)"
- echo " -p, --profile NAME Databricks CLI profile (default: DEFAULT)"
- echo " --experimental BOOL Include experimental agent skills (default: true)"
- exit 0
- ;;
- *) die "Unknown option: $1" ;;
- esac
- done
- # Parse `databricks aitools list -o json` into "<experimental> <name>" lines.
- # Exits non-zero when the payload is missing, malformed, or has no usable
- # entries, so the caller can distinguish "CLI absent" from "CLI changed shape".
- _parse_agent_b_inventory() {
- python3 -c '
- import sys, json
- try:
- data = json.loads(sys.stdin.read())
- except Exception:
- sys.exit(1)
- items = None
- if isinstance(data, list):
- items = data
- elif isinstance(data, dict):
- for key in ("skills", "items", "data", "results"):
- value = data.get(key)
- if isinstance(value, list):
- items = value
- break
- if not items:
- sys.exit(1)
- lines = []
- for item in items:
- if not isinstance(item, dict):
- continue
- name = item.get("name") or item.get("id") or ""
- if not name:
- continue
- experimental = item.get("experimental")
- if isinstance(experimental, str):
- experimental = experimental.strip().lower() in ("true", "1", "yes")
- lines.append(("true" if experimental else "false") + " " + str(name))
- if not lines:
- sys.exit(1)
- print("\n".join(lines))
- '
- }
- fetch_agent_b_inventory() {
- [ -n "$AGENT_B_STABLE" ] && return
- local json="" list_rc=0
- if command -v databricks >/dev/null 2>&1; then
- json=$(databricks aitools list -o json 2>/dev/null) || list_rc=$?
- else
- list_rc=127
- fi
- if [ "$list_rc" -eq 0 ] && [ -n "$json" ]; then
- local parsed=""
- parsed=$(printf '%s' "$json" | _parse_agent_b_inventory) || parsed=""
- if [ -n "$parsed" ]; then
- AGENT_B_STABLE=$(echo "$parsed" | awk '$1=="false"{print $2}' | tr '\n' ' ')
- AGENT_B_EXPERIMENTAL=$(echo "$parsed" | awk '$1=="true"{print $2}' | tr '\n' ' ')
- [ -n "$AGENT_B_STABLE" ] && return
- fi
- # The CLI answered but we could not read it. Falling back here would ship a
- # silently different skill set behind a green check, so treat it as fatal.
- if [ "${ALLOW_STALE_AGENT_SKILLS:-0}" = "1" ]; then
- warn "Unreadable 'databricks aitools list' output; using snapshot v${AGENT_SKILLS_SNAPSHOT_VERSION} (ALLOW_STALE_AGENT_SKILLS=1)"
- else
- die "Could not parse 'databricks aitools list -o json' — the CLI output format may have changed.
- Set ALLOW_STALE_AGENT_SKILLS=1 to install the offline snapshot (v${AGENT_SKILLS_SNAPSHOT_VERSION}) instead."
- fi
- fi
- AGENT_B_STABLE="$AGENT_B_STABLE_FALLBACK"
- AGENT_B_EXPERIMENTAL="$AGENT_B_EXPERIMENTAL_FALLBACK"
- warn "Databricks CLI inventory unavailable; using offline agent-skills snapshot v${AGENT_SKILLS_SNAPSHOT_VERSION} (may differ from the live aitools inventory)"
- }
- resolve_all_agent_skills() {
- fetch_agent_b_inventory
- SELECTED_AGENT_B_SKILLS=""
- local skill
- for skill in $AGENT_B_STABLE $AGENT_B_EXPERIMENTAL; do
- _in_list "$skill" "$AGENT_B_EXCLUDED" && continue
- [ "$INSTALL_EXPERIMENTAL" = false ] && _in_list "$skill" "$AGENT_B_EXPERIMENTAL" && continue
- SELECTED_AGENT_B_SKILLS="${SELECTED_AGENT_B_SKILLS:+$SELECTED_AGENT_B_SKILLS }$skill"
- done
- }
- ensure_aitools_cli() {
- local cli_version=""
- if command -v databricks >/dev/null 2>&1; then
- cli_version=$(databricks --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)
- fi
- if [ -n "$cli_version" ] && version_gte "$cli_version" "$MIN_AITOOLS_CLI_VERSION"; then
- return 0
- fi
- local found_msg="Databricks CLI not found."
- [ -n "$cli_version" ] && found_msg="Databricks CLI v${cli_version} is too old."
- die "$found_msg Agent skills require Databricks CLI v${MIN_AITOOLS_CLI_VERSION}+.
- Upgrade: curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
- Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
- Line 191 on GitHub
- Then re-run this installer."
- }
- agent_b_needs_experimental() {
- local skill
- for skill in $SELECTED_AGENT_B_SKILLS; do
- _in_list "$skill" "$AGENT_B_EXPERIMENTAL" && return 0
- done
- return 1
- }
- # Remove stale real copies of agent-managed skill dirs (aitools won't overwrite them).
- cleanup_stale_agent_skills() {
- local skills_dir="$PROJECT_DIR/.claude/skills"
- [ -d "$skills_dir" ] || return 0
- local skill
- for skill in $SELECTED_AGENT_B_SKILLS; do
- if [ -d "$skills_dir/$skill" ] && [ ! -L "$skills_dir/$skill" ]; then
- # :? guards against an empty expansion turning this into `rm -rf /`.
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 210 on GitHub
- rm -rf "${skills_dir:?}/${skill:?}"
- msg "${D}Removed stale bundled copy: $skill${N}"
- fi
- done
- }
- install_agent_skills() {
- [ -z "$SELECTED_AGENT_B_SKILLS" ] && return 0
- step "Installing agent skills (via databricks aitools)"
- ensure_aitools_cli
- cleanup_stale_agent_skills
- local skills_csv exp_flag="" skills_only_flag="" count cli_version=""
- skills_csv=$(echo "$SELECTED_AGENT_B_SKILLS" | tr -s ' ' ',' | sed 's/^,//;s/,$//')
- agent_b_needs_experimental && exp_flag="--experimental"
- count=$(_count $SELECTED_AGENT_B_SKILLS)
- # Plugin-first CLIs install the agent plugin by default; --skills-only forces
- # raw files into .databricks/aitools/skills so we can copy them for deploy.
- # Older CLIs reject the flag — only pass it when supported.
- cli_version=$(databricks --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)
- if [ -n "$cli_version" ] && version_gte "$cli_version" "$SKILLS_ONLY_CLI_VERSION"; then
- skills_only_flag="--skills-only"
- fi
- cd "$PROJECT_DIR"
- local aitools_out aitools_rc attempt
- for attempt in 1 2; do
- aitools_out=$(databricks aitools install --scope project --agents claude-code --skills "$skills_csv" $skills_only_flag $exp_flag -p "$PROFILE" 2>&1) && aitools_rc=0 || aitools_rc=$?
- if [ "$aitools_rc" -eq 0 ]; then
- break
- fi
- if [ "$attempt" -eq 1 ]; then
- warn "databricks aitools install failed (attempt 1); retrying once..."
- sleep 2
- fi
- done
- if [ "$SILENT" != true ] && [ -n "$aitools_out" ]; then
- echo "$aitools_out" | grep -v 'does not support project-scoped skills' || true
- fi
- if [ "$aitools_rc" -ne 0 ]; then
- # Always surface the CLI error — silent mode used to swallow it entirely.
- echo "$aitools_out" >&2
- die "databricks aitools install failed (exit $aitools_rc)"
- fi
- ok "Agent skills ($count) installed via aitools"
- # Copy real files into .claude/skills for deploy bundling (not symlinks).
- local store="$PROJECT_DIR/.databricks/aitools/skills"
- local dest="$PROJECT_DIR/.claude/skills"
- mkdir -p "$dest"
- for skill in $SELECTED_AGENT_B_SKILLS; do
- if [ ! -d "$store/$skill" ]; then
- warn "Agent skill '$skill' missing from aitools store — skipped"
- continue
- fi
- # :? guards against an empty expansion turning this into `rm -rf /`.
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 268 on GitHub
- rm -rf "${dest:?}/${skill:?}"
- cp -R "$store/$skill" "$dest/$skill"
- done
- ok "Agent skills copied to .claude/skills/"
- }
- install_mlflow_skills() {
- step "Installing MLflow skills"
- local dest="$PROJECT_DIR/.claude/skills"
- mkdir -p "$dest"
- local mlflow_raw_url="$MLFLOW_BASE_URL/${MLFLOW_REF}"
- local count=0 skill
- for skill in $MLFLOW_SKILLS; do
- local dest_dir="$dest/$skill"
- mkdir -p "$dest_dir"
- local url="$mlflow_raw_url/$skill/SKILL.md"
- if curl -fsSL "$url" -o "$dest_dir/SKILL.md" 2>/dev/null; then
- for ref in reference.md examples.md api.md; do
- curl -fsSL "$mlflow_raw_url/$skill/$ref" -o "$dest_dir/$ref" 2>/dev/null || true
- done
- count=$((count + 1))
- else
- rm -rf "$dest_dir"
- warn "Could not fetch MLflow skill: $skill"
- fi
- done
- local expected
- expected=$(_count $MLFLOW_SKILLS)
- # A partial fetch is as silent as an empty one: some skills simply go missing
- # while the installer still reports success. Treat any shortfall as an error.
- local allow_shortfall="${ALLOW_PARTIAL_MLFLOW_SKILLS:-${ALLOW_EMPTY_MLFLOW_SKILLS:-0}}"
- if [ "$count" -lt "$expected" ]; then
- if [ "$allow_shortfall" = "1" ]; then
- warn "MLflow skills incomplete: ${count}/${expected} installed (override set)"
- else
- die "MLflow skills incomplete: ${count}/${expected} installed from ${mlflow_raw_url}.
- Set ALLOW_PARTIAL_MLFLOW_SKILLS=1 to continue anyway, or point MLFLOW_REF at a reachable ref."
- fi
- else
- ok "MLflow skills ($count/$expected) → .claude/skills/"
- fi
- }
- # ─── Main ─────────────────────────────────────────────────────
- # Guarded so tests can source this file for its helpers without installing.
- if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
- resolve_all_agent_skills
- install_agent_skills
- install_mlflow_skills
- if [ "$SILENT" != true ]; then
- total=$(_count $SELECTED_AGENT_B_SKILLS $MLFLOW_SKILLS)
- echo ""
- echo -e " ${G}Done.${N} ${total} skills available under .claude/skills/"
- fi
- fi
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.