Back to search
alchemylake-databricks
SkillGitHub
Governed, provenance-sealed creative rendering for Databricks — a Streamlit App + Unity Catalog ai_render() function that turns any governed source (Sample Lakehouse, UC, Genie, or BYO data) into infographics, reports, decks, video briefings, music, and podcasts.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Checked today
Badge for your README
Markdown
[](/listings/zorost--alchemylake-databricks)HTML
<a href="/listings/zorost--alchemylake-databricks"><img src="/api/listings/zorost--alchemylake-databricks/badge.svg" alt="findsafeskills safety rating" /></a>Install
i
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 384 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- <p align="center">
- <img src=".github/sol-invictus-lapis.png" alt="AlchemyLake — Sol Invictus engraving on the lapis plate" width="640">
- </p>
- # AlchemyLake for Databricks
- [](./LICENSE)
- [](https://docs.databricks.com/dev-tools/cli/)
- [](https://app.alchemylake.com/docs#developers)
- Install a **governed creative** surface directly inside your Databricks
- workspace. AlchemyLake turns the data your lakehouse already trusts into
- finished deliverables **Genie doesn't make**: board-ready **PowerPoint decks
- with a read-aloud script and Q&A prep under every slide**, enterprise **PDF
- dossiers with a statistical appendix and an Excel evidence workbook**,
- **deep-research dossiers** (a planned multi-step investigation — Genie sources
- get governed SQL follow-ups in one continuing conversation), designed
- **infographics**, animated **video briefings with narration** (six formats),
- two-host **data podcasts** (five formats), and **sonified scores** (six
- genres) whose tempo follows your growth.
- ## Get started free — no card required
- 1. **[Sign up](https://app.alchemylake.com/sign-up)** — 50 free credits land in
- your account immediately, nothing to enter but an email.
- 2. **Grab a developer key** — sign in at [app.alchemylake.com](https://app.alchemylake.com) →
- **Studio → Developer · MCP & keys → Forge a new key** (starts with `alk_…`).
- 3. **Clone this repo and deploy it** into your Databricks workspace:
- ```bash
- git clone https://github.com/zorost/alchemylake-databricks.git
- cd alchemylake-databricks
- # edit the workspace host in databricks.yml (see Path 1 below), then:
- databricks bundle deploy -t prod
- databricks bundle run alchemylake_app -t prod
- ```
- Tracking `main` gets you the latest build; `git checkout v0.4.0` first pins
- the deploy to a tagged release instead, if you'd rather not move with `main`.
- 4. Open the App URL the CLI prints, paste your key, then **Load governed
- sources → pick one → Transmute.**
- That's a governed PDF, deck, infographic, or video briefing — sealed to your
- own data, running inside your own workspace — for free, in about five minutes.
- No Databricks Apps on your workspace? Skip straight to
- [Path 2](#path-2--register-alchemylake-as-an-mcp-server-genie--agent-bricks) —
- register one URL and Genie, Claude, or Cursor get the same 50 free credits to
- work with, no bundle required.
- **Every lane binds to a governed source**, and the numbers discipline holds
- throughout: the platform computes an analyst-grade statistical dossier from the
- bound rows by code — trend with fit quality, outliers, correlations, segment
- shares, concentration, pivots — the model never authors a figure, a **verifier
- checks every claim after generation**, and the result is sealed (source · row
- count · data sha256 · verification score) with the seal embedded in the file
- itself. Genie answers bind as sources too, and one-click **recipes** ship a
- board deck or a whole campaign pack from one table. Every credit accounted.
- **Where Genie stops, AlchemyLake starts.** Genie answers questions about your
- data inside the workspace — tables and charts. AlchemyLake takes the same
- governed rows the last mile: *"Ask Genie for Q2 revenue by region, then render
- an 8-slide deck titled Q2 Momentum"* leaves the workspace as a .pptx the CFO
- can present cold — every figure verified against the rows, provenance sealed
- into the file.
- This repository **is** a self-contained **Databricks Asset Bundle**. One command
- deploys a Databricks App (SSO-authenticated, running on your workspace) that
- calls the AlchemyLake platform over MCP. Nothing bypasses governance: the same
- credit ledger, provenance seals, and role checks that protect the web app
- protect every call made from inside Databricks. Your data never leaves the
- lakehouse except as the exact rows you choose to bind.
- ```
- .
- ├── databricks.yml # the Asset Bundle (App resource + targets)
- ├── app/
- │ ├── app.py # the Streamlit App (thin MCP client)
- │ ├── app.yaml # App runtime config (command + env)
- │ └── requirements.txt
- └── sql/
- └── ai_render.sql # optional UC function: call AlchemyLake from SQL/Genie
- ```
- ---
- ## Three ways to use AlchemyLake with Databricks
- | Path | What you get | Setup |
- |---|---|---|
- | **1. The App (this bundle)** | A governed render UI inside your workspace, SSO’d, next to your data | `databricks bundle deploy` |
- | **2. MCP for Genie / Agent Bricks** | Every agent gains **13 governed tools** (`list_governed_sources`, `upload_source`, `get_wallet`, `render_governed_chat`, `render_deep_research`, `render_infographic`, `render_report`, `render_presentation`, `render_video_briefing`, `render_music`, `render_podcast`, `list_recipes`, `run_recipe`) — all render tools accept `source_id` for data-bound, verified output, and chat returns a `thread_id` so a Genie conversation continues across agent turns | Register one URL |
- | **3. `ai_render()` in SQL** | Sealed narrative from a query or Genie space | Run `sql/ai_render.sql` |
- **What agents can ship from a table** (things a Genie answer alone cannot):
- - `render_deep_research` — a planned multi-step investigation: the brief is
- decomposed into sub-questions, each answered with real evidence (Genie
- sources get governed SQL follow-ups in one continuing conversation), then
- synthesized into a sealed PDF dossier + Excel evidence workbook.
- - `render_presentation` — a .pptx board deck, 5–20 slides, speaker script + Q&A
- in every notes pane, AI cover art, real charts from the rows.
- - `render_report` — an enterprise PDF dossier (KPI band, chart sections,
- statistical appendix, citations, methodology) **plus** an Excel evidence
- workbook: raw rows, facts, statistics, pivot, correlations.
- - `render_infographic` — a designed KPI poster with the exact figures rendered
- in-image and a branded provenance strip.
- - `render_video_briefing` — an animated video briefing: platform-drawn charts,
- spoken narration, motion and crossfades. Six formats (`style` param):
- consultant walkthrough, newsroom segment, executive stand-up, documentary
- deep-dive, field report, social recap.
- - `render_podcast` — a two-host audio briefing with a sealed transcript. Five
- formats: two-host interview, skeptic's debate, executive stand-up, narrative
- deep-dive, plain-language walkthrough.
- - `render_music` — a sonified score (tempo ↔ momentum, mode ↔ trend) plus the
- literal data-motif WAV of the rows. Six genres: cinematic score, corporate
- uplift, ambient data fields, electronic pulse, orchestral arc, lo-fi data
- study.
- Prefer plain REST? The same keys work on the public API:
- `https://app.alchemylake.com/api/public/v1` (OpenAPI at `/api/public/v1/openapi.json`).
- Prefer a terminal? `npx alchemylake` is a zero-dependency CLI over that API:
- ```bash
- export ALCHEMYLAKE_API_KEY=alk_YOUR_KEY
- npx alchemylake upload ./q3-actuals.xlsx # CSV/Excel/PDF/Word/text → governed source
- npx alchemylake render report --prompt "Board brief" --source up.1a2b3c
- npx alchemylake render deep_research --prompt "Why did Q3 dip?" --source genie:dbx1
- ```
- ---
- ## How it fits together
- AlchemyLake never runs *inside* this bundle — the App is a thin, SSO'd client that
- calls the governed platform over HTTPS/MCP. Your tables stay in your lakehouse; only
- the exact rows you bind for a given render ever leave the workspace, and only to
- produce that one sealed deliverable.
- ```mermaid
- flowchart LR
- subgraph WS["Your Databricks workspace"]
- UC[("Unity Catalog<br/>tables & volumes")]
- Genie["Genie space"]
- CSV["Uploaded CSV / Excel"]
- App["AlchemyLake App<br/>(this bundle · Streamlit · SSO'd)"]
- Agents["Genie / Agent Bricks<br/>Claude · Cursor"]
- end
- AL(["AlchemyLake<br/>governed render platform"])
- UC -- "bind rows" --> App
- Genie -- "bind an answer<br/>(conversation continues)" --> App
- CSV -- "bind (BYO-data)" --> App
- App <-- "MCP: render, verify, seal" --> AL
- Agents <-- "13 MCP tools, same contract" --> AL
- AL -. "no-egress text tier" .-> FM["Your Databricks<br/>Foundation Model endpoint"]
- AL --> Out(["Sealed deliverable<br/>PDF · PPTX · image · video · audio"])
- ```
- Every render — from the App, from an agent, or from `ai_render()` in SQL — goes
- through the same discipline before it comes back:
- ```mermaid
- flowchart LR
- A["Bind<br/><sub>pick a governed source</sub>"] --> B["Analyze<br/><sub>compute the facts</sub>"]
- B --> C["Design<br/><sub>plan the deliverable</sub>"]
- C --> D["Transmute<br/><sub>generate</sub>"]
- D --> E["Verify<br/><sub>check every claim</sub>"]
- E --> F["Seal<br/><sub>embed provenance</sub>"]
- ```
- The model is never the source of a number — every figure is computed from your rows
- first, and whatever the render says is checked against those figures afterward. The
- seal (source · row count · data hash · verification score) rides embedded in the file
- itself, in every format, so it survives being forwarded, downloaded, or printed.
- ### Beyond a Genie answer
- | | Genie alone | Genie / Agent Bricks + AlchemyLake |
- |---|:---:|:---:|
- | Answers a question inside the workspace | ✅ | ✅ |
- | Ships a board-ready **.pptx** with a read-aloud script + Q&A prep per slide | | ✅ |
- | Ships an enterprise **PDF dossier + Excel evidence workbook** | | ✅ |
- | Runs a **deep-research investigation** (planned Genie SQL follow-ups → sealed dossier) | | ✅ |
- | Ships a designed **infographic** with figures rendered in-image | | ✅ |
- | Ships a narrated **video briefing** or two-host **podcast** | | ✅ |
- | Every number checked against the source *after* generation and scored | | ✅ |
- | Provenance (source, rows, hash, score — plus Genie reasoning, SQL, and trusted-asset status) embedded in the output | | ✅ |
- | Scores your Genie space's curation health and certifies it with live probes | | ✅ |
- | Metered by an auditable, per-render credit ledger | | ✅ |
- ### What people actually ask it for
- Every lane binds to a governed source, so the figures in the output are the figures
- in your table — not a paraphrase. A sample of real workflows across the eight lanes:
- | Lane | Ships | Example asks |
- |---|---|---|
- | **Analyst** (chat) | A sealed, sourced answer — follow-ups stay in context | *"Draft the board narrative from the certified loan book."* · *"Now break that down by region."* |
- | **Deep Research** | A sealed investigation dossier (PDF + Excel) | *"Why did Q3 dip, and what should we do about it?"* · *"Investigate concentration risk across our segments."* |
- | **Infographic** | A KPI poster, figures rendered in-image | *"This week's sales milestone, ready for social."* · *"'45 days without a recordable incident' for the plant floor."* |
- | **Report** | PDF dossier + Excel evidence workbook | *"Quarterly board-packet narrative, sealed."* · *"Compliance rollup with a hash of the source data baked in."* |
- | **Presentation** | .pptx, speaker script + Q&A per slide | *"QBR deck with the CRM's exact numbers."* · *"Steering-committee deck anyone on the team can present cold."* |
- | **Video briefing** | A narrated, animated data video | *"All-hands opener with the real ARR on screen."* · *"30-second earnings-day teaser, safe to post at the bell."* |
- | **Music** | A score that follows the actual trend | *"A launch-video bed whose tempo rises with the real growth curve."* |
- | **Podcast** | A two-host audio briefing + transcript | *"A 3-minute briefing leadership can play on the commute."* · *"An audio version of the same sealed report, for accessibility."* |
- Free to try: [sign up](https://app.alchemylake.com/sign-up) for 50 credits, no card
- required, and point this bundle at your own key.
- ---
- ## Prerequisites
- 1. **Databricks CLI ≥ 0.230** and authentication to your workspace:
- ```bash
- databricks auth login --host https://<your-workspace>.cloud.databricks.com
- ```
- 2. **Databricks Apps enabled** on the workspace (Premium/Enterprise).
- 3. An **AlchemyLake developer key** (`alk_…`): [sign up free](https://app.alchemylake.com/sign-up)
- (50 credits, no card required) or sign in at <https://app.alchemylake.com> →
- **Studio → Developer · MCP & keys → Forge a new key**.
- ---
- ## Path 1 — Deploy the App (one command)
- First set your workspace host in `databricks.yml` — replace the
- `your-workspace…` placeholder under **both** `targets.dev` and `targets.prod`
- (run `databricks auth profiles` if you need to find it). Then:
- ```bash
- databricks bundle deploy -t prod # upload source + create the App
- databricks bundle run alchemylake_app -t prod # start it; prints the App URL
- ```
- > **CLI note (Terraform pin).** Some Databricks CLI builds download a Terraform
- > whose signing key has expired, which breaks `bundle deploy`. If you hit that,
- > point the CLI at a local Terraform binary and pin the version to match it:
- > ```bash
- > export DATABRICKS_TF_EXEC_PATH="$(command -v terraform)"
- > export DATABRICKS_TF_VERSION="1.9.8" # set to your local `terraform version`
- > ```
- > This is the exact, verified path AlchemyLake's own reference App is deployed
- > with — it runs live in-workspace as a Streamlit App under SSO.
- Open the App URL that the CLI prints. Paste your `alk_…` key in the sidebar
- (or bind it from a secret — see below), then **Load governed sources → pick one
- → Transmute**. Every result carries its provenance seal.
- The pasted key is remembered in that browser (client-side only, until cleared
- or replaced) so it survives a refresh — a workspace-bound secret still always
- takes priority. The App also carries a self-contained **Docs** tab and an
- expanded **About / Install** tab (how to use it, why the verification model
- matters, where the render compute actually runs, and the fastest path to a
- key) so it's useful even where the public docs site isn't reachable.
- ### Bind the key from a secret (recommended for teams)
- So analysts never paste a key:
- ```bash
- databricks secrets create-scope alchemylake
- databricks secrets put-secret alchemylake api_key # paste alk_...
- ```
- Then uncomment the `ALCHEMYLAKE_API_KEY` / `valueFrom` block in
- `app/app.yaml` and `databricks.yml`, and redeploy.
- ---
- ## Path 2 — Register AlchemyLake as an MCP server (Genie / Agent Bricks)
- No bundle required. In your agent/MCP client configuration add:
- ```json
- {
- "mcpServers": {
- "alchemylake": {
- "url": "https://app.alchemylake.com/api/mcp",
- "headers": { "Authorization": "Bearer alk_YOUR_KEY" }
- }
- }
- }
- ```
- Genie / Agent Bricks, Claude, and Cursor will discover the thirteen tools
- automatically. Ask, for example: *“Using AlchemyLake, write a sealed 3-bullet
- executive summary of `gold_ridership_national_monthly`”* — or go bigger:
- *“Render an 8-slide presentation of that table titled Ridership Momentum”*
- (a .pptx with a presenter script under every slide), *“Run deep research on
- why ridership dipped this quarter”* (`render_deep_research` returns a sealed
- dossier + evidence workbook), or *“Run the campaign-pack recipe”*
- (`run_recipe` renders the poster, the report, and the copy in one call, all
- verified and sealed).
- ---
- ## Path 3 — `ai_render()` in SQL / Genie
- Edit `sql/ai_render.sql` (set your catalog/schema) and run it in a SQL editor
- on **serverless compute with external network access enabled**. Then:
- ```sql
- SELECT ai_render(
- 'One-sentence executive read of this quarter''s ridership',
- 'ntd_demo.ntd.gold_ridership_national_monthly'
- ) AS narrative;
- ```
- See the header of `sql/ai_render.sql` for the network-egress and secret
- prerequisites, and the MCP fallback if egress cannot be enabled.
- ---
- ## Validate before deploying
- ```bash
- databricks bundle validate -t prod
- ```
- ---
- ## FAQ
- **Does my data leave the lakehouse?** Only the exact rows you bind for a given
- render are sent to produce that render — nothing else is read, and bound data is
- never used to train models. A no-egress **text tier** can route even that narrative
- step to your own Databricks Foundation Model endpoint instead; see
- [the residency docs](https://app.alchemylake.com/docs#residency).
- **Do I need Databricks Apps enabled for this?** Only for Path 1 (the App). Path 2
- (MCP) and Path 3 (`ai_render()`) need nothing beyond CLI/SQL access and a developer
- key — no App, no extra compute resource.
- **Can agents use this without a human in the loop?** Yes — that's Path 2. Register
- `/api/mcp` once and Genie, Agent Bricks, Claude, and Cursor all gain the same thirteen
- governed tools, metered and sealed exactly like the UI.
- **What does a Genie question cost?** Genie compute bills inside *your* Databricks
- workspace (serverless DBUs — roughly 1.5 DBU ≈ $0.11 per question; Databricks
- gives every user 150 free Genie DBUs a month). AlchemyLake credits meter only the
- render itself, and the Studio shows the DBU estimate next to any Genie source.
- **I don't have Databricks — can I still use everything?** Yes. Upload CSV, Excel,
- PDF, Word, text or Markdown (in the web Studio, this App's Sources tab, the API,
- the CLI, or the `upload_source` MCP tool) and every lane — including Deep
- Research — runs with the same verification and seals.
- **Is AlchemyLake a Databricks product?** No. AlchemyLake is an independent platform
- by [Zorost Intelligence](mailto:info@zorost.com) that plugs into Databricks; this
- repository is Apache-2.0 licensed and is how it plugs in. The hosted platform it
- talks to is a separate, metered service (50 free credits to start, no card
- required).
- **What does this bundle actually contain?** Only a Streamlit client (`app/app.py`),
- its App runtime config, and an optional SQL function — everything under
- [`sql/`](./sql) and [`app/`](./app). It holds no model weights, no proprietary
- rendering code, and no secrets; it is a thin, auditable client over the same MCP/REST
- contract documented at [app.alchemylake.com/docs](https://app.alchemylake.com/docs).
- **Something's broken.** Open an issue in this repository, or email
- info@zorost.com — see [`SECURITY.md`](./SECURITY.md) for vulnerability reports
- specifically.
- ---
- ## Documentation
- - [`docs/DEVELOPER_GUIDE.md`](./docs/DEVELOPER_GUIDE.md) — **CLI, REST API & MCP**: get a key, every command, VS Code/Cursor/Claude setup, use cases, Python/Node examples, GitHub Actions.
- - [`docs/GENIE_GUIDE.md`](./docs/GENIE_GUIDE.md) — Genie integration: connect a space, health score, certification, all eight lanes, sharing model.
- - Platform docs: [app.alchemylake.com/docs](https://app.alchemylake.com/docs) (Studio walkthrough, credits, security).
- ---
- ## Uninstall
- ```bash
- databricks bundle destroy -t prod
- ```
- ---
- *AlchemyLake is a product of **Zorost Intelligence**. Platform docs:
- <https://app.alchemylake.com/docs>. Support: info@zorost.com.*
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.