Skip to content
Back to search

okf-generator

OKF v0.1 knowledge bundle generator — Claude skill + OpenCode integration

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/umairbaig8--okf-generator/badge.svg)](/listings/umairbaig8--okf-generator)

HTML

<a href="/listings/umairbaig8--okf-generator"><img src="/api/listings/umairbaig8--okf-generator/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

Install in 30 seconds

# Via pip
pip install okf-generator

# Or one-liner (macOS / Linux)
curl -fsSL https://raw.githubusercontent.com/UmairBaig8/okf-generator/main/scripts/install.sh | bash

Generate your first bundle

okf generate ./my_project ./okf_bundle

That's it — no API key, no vector DB, no config required.

Incremental updates (fast)

After the first generate, use `okf update` to re-scan only changed files:

okf update ./my_project ./okf_bundle          # incremental (default)
okf update ./my_project ./okf_bundle --force  # full re-scan
okf update ./my_project ./okf_bundle --watch  # continuous watcher mode

A SHA256 manifest tracks file state. Only changed files are re-parsed; edge-diff detects cascade changes; only dirty concepts are written.

Look up any concept

okf lookup WorldBankConnector

# Filter by type
okf lookup --type Function

# Filter by ecosystem
okf lookup --tag ecosystem:pip

# JSON output for programmatic use
okf lookup --json WorldBankConnector

# Fuzzy / camelCase search
okf lookup repo          # finds UserRepository

Visualize as interactive HTML

okf visualize ./okf_bundle viz.html
# Open viz.html in any browser

Launch the live dashboard

okf dashboard ./okf_bundle --open

Opens a 3-panel FastAPI web UI at `http://127.0.0.1:8700` with search, detail inspection, and force-directed graphs.

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 225 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. ---
  2. name: okf-generator
  3. description: >
  4. Generate OKF (Open Knowledge Format) v0.2 knowledge bundles from codebases,
  5. and look up exact concepts for AI agent context injection. Use this skill
  6. whenever the user wants to: index a codebase, generate OKF bundles, extract
  7. code knowledge into structured markdown, convert codebases into training data,
  8. look up functions/classes/modules by name, prime an AI agent with codebase
  9. context, or integrate codebase knowledge with OpenCode or other AI coding agents.
  10. Also trigger for phrases like "index my code", "generate knowledge bundle",
  11. "extract codebase concepts", "what does X class do", or "look up X in OKF".
  12. ---
  13. # OKF Generator & Lookup Skill
  14. Generates structured OKF v0.2 knowledge bundles from codebases (18 languages via
  15. tree-sitter AST + stdlib ast), and provides fast concept lookup for AI agents
  16. like OpenCode.
  17. ## Pipeline Overview
  18. ```
  19. codebase
  20. |
  21. v
  22. okf generate --> okf_bundle/ (domain/resource-path layout)
  23. |
  24. okf lookup (zero-LLM concept search)
  25. |
  26. okf pairs --> okf_pairs.jsonl (training data)
  27. ```
  28. ## CLI Reference
  29. All features via single `okf` CLI (installed from PyPI).
  30. | Command | Purpose |
  31. |---------|---------|
  32. | `okf generate` | Scan codebase and write OKF bundle |
  33. | `okf lookup` | Search bundle and return exact concept |
  34. | `okf pairs` | Convert bundle to JSONL training pairs |
  35. | `okf summarize` | Regenerate SUMMARY.md from existing bundle |
  36. ## Dependencies
  37. ```bash
  38. pip install okf-generator
  39. # With LLM enrichment:
  40. pip install "okf-generator[llm]"
  41. ```
  42. ---
  43. ## Task: Generate OKF Bundle
  44. **When**: user says "index my codebase", "generate OKF bundle", "extract knowledge from code"
  45. ### Static extraction (no LLM — always run this first)
  46. ```bash
  47. okf generate <source_dir> <output_dir>
  48. ```
  49. ### With LLM enrichment (fills missing docstrings and descriptions)
  50. Configure enrichment in `.okfconfig` (JSON), then run:
  51. ```bash
  52. okf generate <source_dir> <output_dir> --enrich
  53. # or run enrichment standalone on an existing bundle:
  54. okf enrich <output_dir>
  55. ```
  56. ```jsonc
  57. // .okfconfig
  58. {
  59. "llm": { "enabled": true, "base_url": "http://localhost:8080/v1", "api_key": "llamabarn" },
  60. "providers": {
  61. "default": { "model": "ggml-org/gemma-3-4b-it-qat-GGUF:Q4_0", "max_workers": 2 }
  62. }
  63. }
  64. ```
  65. Enrichment is **resumable** — rerun safely if interrupted. Already-enriched
  66. concepts are skipped automatically (checks disk on every run).
  67. ### Output layout (mirrors source tree)
  68. ```
  69. okf_bundle/
  70. ├── SUMMARY.md <- bird's-eye view for AI agents
  71. ├── index.md <- root index
  72. ├── log.md <- generation history
  73. └── <domain>/
  74. └── <module>/
  75. ├── index.md <- lists all concepts in folder
  76. ├── <module>.md <- Module concept
  77. └── <ClassName>.md <- Class / Function concepts
  78. ```
  79. ---
  80. ## Task: Look Up a Concept
  81. **When**: user asks "what does X do", "find class X", "look up X", or needs to
  82. prime OpenCode with exact concept context before editing code.
  83. ```bash
  84. # Full detail — signature, docstring, params, returns, related
  85. okf lookup WorldBankConnector
  86. # All concepts from one source file
  87. okf lookup --file StockAI/RnD/python/connectors/economic_data.py
  88. # Filter by type
  89. okf lookup --type Class connector
  90. # Filter by tag
  91. okf lookup --tag lang:python --tag type:Function fetch
  92. # Compact list (many results)
  93. okf lookup --compact connector
  94. # JSON output (programmatic / agent use)
  95. okf lookup --json WorldBankConnector
  96. # Custom bundle path
  97. okf lookup --bundle ./Knowlege/okf_bundle WorldBankConnector
  98. ```
  99. Default bundle path: `./okf_bundle`. Also auto-tries `./Knowlege/okf_bundle`
  100. and `./knowledge/okf_bundle`.
  101. ---
  102. ## Task: Generate Training Pairs
  103. **When**: user wants JSONL training data from the OKF bundle.
  104. ```bash
  105. # Static only (instant, no LLM)
  106. SKIP_SYNTH=1 okf pairs <bundle_dir> output.jsonl
  107. # With LLM (QA, doc, summarize pairs)
  108. SYNTH_BASE_URL="http://localhost:8080/v1" \
  109. SYNTH_API_KEY="llamabarn" \
  110. SYNTH_MODEL="ggml-org/gemma-3-4b-it-qat-GGUF:Q4_0" \
  111. MAX_WORKERS=2 \
  112. QA_PER_CONCEPT=3 \
  113. okf pairs <bundle_dir> output.jsonl
  114. # Specific pair types only
  115. PAIR_TYPES="codegen,qa" okf pairs <bundle_dir> output.jsonl
  116. ```
  117. ### Pair types
  118. | Type | Static | LLM | Covers |
  119. |------|--------|-----|--------|
  120. | `codegen` | yes | yes | Functions, Classes |
  121. | `qa` | no | yes | All (purpose/params/return/edge) |
  122. | `doc` | no | yes | Functions, Classes |
  123. | `summarize` | yes | yes | Modules, Classes |
  124. | `crosslink` | yes | no | All with related concepts |
  125. ---
  126. ## Task: Regenerate SUMMARY.md Only
  127. ```bash
  128. okf summarize <bundle_dir>
  129. ```
  130. Use after enrichment finishes to refresh the summary without re-scanning.
  131. ---
  132. ## OpenCode Integration
  133. See `references/opencode-integration.md` for full setup.
  134. Quick setup:
  135. ```bash
  136. # 1. Add to AGENTS.md (auto-loaded by OpenCode)
  137. echo "OKF bundle at ./okf_bundle — use: okf lookup <Name>" >> AGENTS.md
  138. # 2. Add lookup command
  139. mkdir -p .opencode/commands
  140. echo "RUN okf lookup --bundle ./okf_bundle \$NAME" \
  141. > .opencode/commands/lookup.md
  142. ```
  143. ---
  144. ## Supported Languages
  145. | Language | Parser | Extracts |
  146. |----------|--------|---------|
  147. | Python | stdlib ast | functions, classes, params, return types, docstrings |
  148. | JS / TS | tree-sitter | functions, arrow fns, classes, JSDoc |
  149. | Go | tree-sitter | funcs, methods, structs, interfaces, GoDoc |
  150. | Java | tree-sitter | classes, methods, constructors, Javadoc |
  151. | Rust | tree-sitter | fns, structs, enums, traits, impl blocks, doc comments |
  152. | Ruby | tree-sitter | defs, classes, modules, hash comments |
  153. | C / C++ / C# | tree-sitter | funcs, structs, classes, headers, XML-doc |
  154. | Swift / Kotlin | tree-sitter | funcs, classes, protocols, visibility |
  155. | PHP / Dart | tree-sitter | classes, functions, docblocks |
  156. | Scala / Julia | tree-sitter | defs, classes, traits, objects |
  157. | SQL | tree-sitter | tables, views, functions |
  158. | YAML | PyYAML | documents, keys, anchors |
  159. ---
  160. ## Troubleshooting
  161. **No concepts found**: Check that source dir is not inside a SKIP_DIRS name
  162. (node_modules, .venv, dist, etc). Leading path components like `/tmp` are
  163. no longer skipped (fixed in v0.1.3).
  164. **Enrichment slow**: Set `max_workers: 1` in the provider config. Local models process ~1 request at a time. At 32 tok/sec expect ~3-5s per concept.
  165. **Enrichment interrupted**: Rerun same command. Enriched files are skipped.
  166. **JS/TS concepts missing**: Ensure tree-sitter-typescript is installed.
  167. TypeScript uses a separate grammar from JavaScript.
  168. **Lookup wrong result**: Add --type or --file to narrow the search scope.
README.md · README · 424 lines · 1 finding marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <img src="https://cdn.jsdelivr.net/gh/UmairBaig8/okf-generator@main/docs/images/okf_banner.jpg" alt="okf-generator" width="700">
  3. </p>
  4. <p align="center">
  5. <a href="https://pypi.org/project/okf-generator/"><img src="https://img.shields.io/pypi/v/okf-generator?style=flat-square&label=PyPI" alt="PyPI"></a>
  6. <a href="https://pypi.org/project/okf-generator/"><img src="https://img.shields.io/pypi/pyversions/okf-generator?style=flat-square" alt="Python"></a>
  7. <a href="https://github.com/UmairBaig8/okf-generator/stargazers"><img src="https://img.shields.io/github/stars/UmairBaig8/okf-generator?style=flat-square" alt="GitHub Stars"></a>
  8. <a href="https://github.com/UmairBaig8/okf-generator/actions"><img src="https://img.shields.io/github/actions/workflow/status/UmairBaig8/okf-generator/ci.yml?style=flat-square&label=tests" alt="Tests"></a>
  9. <a href="https://github.com/UmairBaig8/okf-generator/commits/main"><img src="https://img.shields.io/github/last-commit/UmairBaig8/okf-generator?style=flat-square" alt="Last commit"></a>
  10. <a href="https://github.com/UmairBaig8/okf-generator/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue?style=flat-square" alt="MIT"></a>
  11. <a href="https://github.com/UmairBaig8/okf-generator/blob/main/CONTRIBUTING.md"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen?style=flat-square" alt="PRs Welcome"></a>
  12. <a href="https://umairbaig8.github.io/okf-generator/getting-started/"><img src="https://img.shields.io/badge/📖-Docs-7c3aed?style=flat-square" alt="Docs"></a>
  13. <a href="https://modelcontextprotocol.io"><img src="https://img.shields.io/badge/MCP-Server-5b21b6?style=flat-square" alt="MCP"></a>
  14. <a href="https://www.cursor.com"><img src="https://img.shields.io/badge/Cursor-Ready-7c3aed?style=flat-square" alt="Cursor"></a>
  15. <a href="https://github.com/anthropics/claude-code"><img src="https://img.shields.io/badge/Claude-Ready-7c3aed?style=flat-square&logo=anthropic" alt="Claude"></a>
  16. </p>
  17. <h1 align="center">OKF Generator</h1>
  18. <h3 align="center">The knowledge layer for AI coding agents</h3>
  19. <p align="center">
  20. <b>Scan any codebase into structured, agent-ready knowledge — 18 languages, <strong>~100x fewer tokens</strong> than reading whole files, <strong>zero LLM required</strong>.
  21. </p>
  22. <p align="center">
  23. <a href="#quick-start"><b>Quick Start</b></a> ·
  24. <a href="#the-why"><b>Why OKF?</b></a> ·
  25. <a href="#core-features"><b>Features</b></a> ·
  26. <a href="#agent-integration"><b>Agents</b></a> ·
  27. <a href="#comparison"><b>vs RAG</b></a> ·
  28. <a href="#faq"><b>FAQ</b></a>
  29. </p>
  30. <br>
  31. ## Visual Hook & Demos
  32. <p align="center">
  33. <img src="https://cdn.jsdelivr.net/gh/UmairBaig8/okf-generator@main/docs/demo.gif" alt="okf-generator demo" width="540">
  34. <br>
  35. <em><code>okf generate</code> → <code>okf lookup</code> → <code>okf diff</code> → <code>okf visualize</code> → <code>okf mcp</code> → <code>okf pairs</code></em>
  36. </p>
  37. <p align="center">
  38. <a href="https://umairbaig8.github.io/okf-generator/viz.html"><b>Live Viz Graph</b></a>
  39. <span>&nbsp;&middot;&nbsp;</span>
  40. <a href="https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=UmairBaig8/okf-generator"><b>Interactive Dev Container</b></a>
  41. <span>&nbsp;&middot;&nbsp;</span>
  42. <a href="https://okf-generator.onrender.com"><b>Render App Demo</b></a>
  43. </p>
  44. ---
  45. ## The Why
  46. ### AI agents waste tokens re-reading code
  47. Every context reset forces agents to re-read entire files to find one function signature. A 600-line file costs **14,000 tokens** — just to find one class. Cloud models with 200K windows mask this cost; local SLMs on a MacBook run out of memory immediately.
  48. Vector databases (RAG) don't solve this — they shred code into arbitrary chunks, losing syntax structure, import pathways, and call relationships.
  49. ### The OKF Edge: deterministic AST extraction
  50. Instead of chunking text, `okf-generator` uses **tree-sitter AST parsers** to create a mathematical map of your codebase. Every function, class, module, and dependency becomes a structured node with typed edges (calls, called-by, imports, depends-on).
  51. **The result:** an average lookup drops from **45,000 tokens → 1,200 tokens** — a **~97.3% reduction**.
  52. ```bash
  53. # Before touching any file, get the exact concept card
  54. okf lookup WorldBankConnector
  55. ```
  56. ```
  57. CLASS: WorldBankConnector
  58. Source : StockAI/RnD/python/connectors/economic_data.py line 51
  59. Description : Fetches World Bank development indicators via wbdata API.
  60. Methods : get_indicator, search
  61. Signature : class WorldBankConnector
  62. Calls : [wbdata.get_indicator, pandas.DataFrame]
  63. Called-by : [DataPipeline.fetch_economic]
  64. ```
  65. No re-reading the file. No guessing. No LLM call required.
  66. ---
  67. ## Core Features
  68. | Capability | What it means |
  69. |---|---|
  70. | **18 languages** | Native AST extraction for Python, JS/TS, Go, Rust, Java, C++, Swift, Kotlin, Ruby, C/C++/C#, SQL, PHP, Dart, Scala, Julia, YAML — one parser file per language, add a new one in minutes |
  71. | **20 manifest formats** | Automatic cross-indexing of `package.json`, `Cargo.toml`, `Dockerfile`, `go.mod`, `requirements.txt`, `Gemfile`, `pyproject.toml`, and 13 more |
  72. | **Deterministic & offline** | Core extraction is **100% offline** — zero LLM calls, zero API keys, zero vector infrastructure. Same output every run |
  73. | **Cross-reference linker** | Resolves imports → dependency edges and function calls → caller/callee across all languages. Multi-hop reasoning without grep |
  74. | **LSP call-graph enrichment** | Optional LSP pass (`okf enrich --lsp`) uses local language servers for compiler-accurate caller/callee resolution — 4 servers mapped (pyright ✅, gopls, rust-analyzer, typescript-language-server), zero token cost |
  75. | **Interactive visualization** | Self-contained D3.js HTML dashboard — force-directed graph, search, filter, dark/light theme, no server required |
  76. | **Token efficiency** | ~140 tokens per concept lookup vs 14,000+ reading whole files. Local SLMs become viable for enterprise-scale codebases |
  77. | **4 enrichment tiers** | Optional LLM layer: `base` (descriptions), `deep` (examples + side effects), `security` (risk audit), `full` (all + semantic links). Runs at generate time or standalone |
  78. | **Multi-provider routing** | Route cheap work to local llama.cpp, security audits to Claude — one config file |
  79. | **Incremental updates** | `okf update` re-scans only changed files — SHA256 manifest tracks mtime + content hashes, edge-diff detects cascade changes, writes only dirty concepts. 8 writes for a 1-file edit in a 68-concept bundle |
  80. | **Bundle diff & impact** | `okf diff --impact` shows exactly what changed between runs — which deps affect which modules |
  81. | **Training data export** | `okf pairs` converts any bundle into JSONL fine-tuning pairs (codegen, QA, doc, summarize, crosslink) |
  82. ### How it works
  83. ![okf-generator pipeline](https://cdn.jsdelivr.net/gh/UmairBaig8/okf-generator@main/docs/images/workflow.png)
  84. ```
  85. 1. Scan → tree-sitter AST parsers extract functions, classes, modules (17 langs)
  86. 2. Link → cross-reference linker resolves imports→deps, calls→callees
  87. 3. Write → OKF v0.2 bundle: structured markdown, mirrors your source tree
  88. 4. Update → incremental: SHA256 manifest detects changes, re-parses only dirty files, re-links, edge-diffs, writes only dirty concepts
  89. 5. Use → lookup, ask, diff, visualize, mcp, dashboard — 19 commands
  90. 6. Enrich → optional LSP pass (deterministic call-graph refinement) + optional LLM layer: 4 modes, multi-provider routing
  91. ```
  92. Core extraction is fully deterministic and offline. Enrichment is optional, resumable, works with any AI provider. LSP enrichment requires language server binaries on `$PATH`.
  93. > Full language table with per-parser details: **[docs/languages-and-manifests.md](docs/languages-and-manifests.md)**
  94. ### Supported languages (18)
  95. `Python` · `JavaScript` · `TypeScript` · `Go` · `Java` · `Rust` · `Swift` · `Kotlin` · `PHP` · `Dart` · `Scala` · `Julia` · `Ruby` · `C` · `C++` · `C#` · `SQL` · `YAML`
  96. Each language lives in its own file under `okf/parsers/`. Add a new one in minutes — one file + one registry entry.
  97. ### Domain Classification
  98. Use `--domains crossplane` to re-classify YAML concepts using data-driven rules. Built-in rules for Crossplane (XRD, Composition, Claim, ProviderConfig, ManagedResource). Custom domains via `--domain-rules ./my-rules.yaml`. See **[docs/domain-classification.md](docs/domain-classification.md)**.
  99. ### Manifest formats (20)
  100. `requirements.txt` · `pyproject.toml` · `package.json` · `Cargo.toml` · `Cargo.lock` · `yarn.lock` · `pnpm-lock.yaml` · `go.mod` · `go.sum` · `poetry.lock` · `composer.json` · `pom.xml` · `Gemfile` · `build.gradle` / `.kts` · `Package.swift` · `Dockerfile` / `Containerfile` · `docker-compose.yml` · `mix.exs` · `project.clj` · `package.swift`
  101. ![Before and after: 14,000 tokens vs 140 tokens](https://cdn.jsdelivr.net/gh/UmairBaig8/okf-generator@main/docs/images/before_after.svg)
  102. ### What a bundle looks like
  103. ```
  104. okf_bundle/
  105. ├── SUMMARY.md ← agent's bird's-eye view
  106. ├── index.md ← root navigation
  107. ├── _dependencies/ ← deps by ecosystem (pip, npm, cargo…)
  108. └── my_project/
  109. └── connectors/
  110. ├── index.md ← lists all concepts in folder
  111. ├── economic_data.md ← Module concept
  112. └── economic_data/
  113. ├── WorldBankConnector.md ← Class (signature, methods, calls)
  114. ├── get_indicator.md ← Function (params, returns)
  115. └── search.md ← Function
  116. ```
  117. Each file follows the OKF v0.2 dialect (extended from Google's OKF v0.1) with structured YAML frontmatter — agents parse it deterministically, no guessing.
  118. ---
  119. ## Quick Start
  120. ### Install in 30 seconds
  121. ```bash
  122. # Via pip
  123. pip install okf-generator
  124. # Or one-liner (macOS / Linux)
  125. curl -fsSL https://raw.githubusercontent.com/UmairBaig8/okf-generator/main/scripts/install.sh | bash
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 168 on GitHub
  126. ```
  127. ### Generate your first bundle
  128. ```bash
  129. okf generate ./my_project ./okf_bundle
  130. ```
  131. That's it — no API key, no vector DB, no config required.
  132. ### Incremental updates (fast)
  133. After the first generate, use `okf update` to re-scan only changed files:
  134. ```bash
  135. okf update ./my_project ./okf_bundle # incremental (default)
  136. okf update ./my_project ./okf_bundle --force # full re-scan
  137. okf update ./my_project ./okf_bundle --watch # continuous watcher mode
  138. ```
  139. A SHA256 manifest tracks file state. Only changed files are re-parsed; edge-diff detects cascade changes; only dirty concepts are written.
  140. ### Look up any concept
  141. ```bash
  142. okf lookup WorldBankConnector
  143. # Filter by type
  144. okf lookup --type Function
  145. # Filter by ecosystem
  146. okf lookup --tag ecosystem:pip
  147. # JSON output for programmatic use
  148. okf lookup --json WorldBankConnector
  149. # Fuzzy / camelCase search
  150. okf lookup repo # finds UserRepository
  151. ```
  152. ### Visualize as interactive HTML
  153. ```bash
  154. okf visualize ./okf_bundle viz.html
  155. # Open viz.html in any browser
  156. ```
  157. ### Launch the live dashboard
  158. ```bash
  159. okf dashboard ./okf_bundle --open
  160. ```
  161. Opens a 3-panel FastAPI web UI at `http://127.0.0.1:8700` with search, detail inspection, and force-directed graphs.
  162. ---
  163. ## Agent Integration
  164. ### One command per agent
  165. `okf install` writes the exact rules, instructions, and commands each agent needs. No manual configuration.
  166. ```bash
  167. # Install for all detected agents at once
  168. okf install all
  169. # Or pick specific agents
  170. okf install claude # Claude Code skill (auto-triggers on "index my codebase")
  171. okf install cursor # Cursor rules (.cursorrules)
  172. okf install copilot # GitHub Copilot instructions
  173. okf install windsurf # Windsurf rules
  174. okf install cline # Cline rules
  175. okf install opencode # OpenCode /lookup command
  176. okf install mcp # Register MCP server (OpenCode + Claude Desktop)
  177. ```
  178. ### MCP server (11 tools)
  179. Start an offline-first MCP server that exposes your knowledge bundle to any MCP client:
  180. ```bash
  181. okf mcp ./okf_bundle --port 4567
  182. ```
  183. Tools: `lookup`, `get_concept`, `find_callers`, `find_callees`, `list_by_file`, `list_dependencies`, `bundle_info`, `list_by_type`, `search_by_tag`, `get_related`, `get_manifest_source`.
  184. Connect from **Cursor**, **Cline**, **Claude Desktop**, or any MCP-compatible IDE.
  185. ### Quick agent context
  186. ```markdown
  187. Add to your agent instructions:
  188. This project has an OKF knowledge bundle at ./okf_bundle/.
  189. - Use `okf lookup <Name>` for full concept context.
  190. - Use `okf lookup --type <Type>` to filter by type.
  191. - Read `SUMMARY.md` for the full knowledge map.
  192. ```
  193. ---
  194. ## Comparison
  195. ### okf-generator vs RAG / vector search
  196. | Capability | **okf-generator** | RAG / Vector Search |
  197. |---|---|---|
  198. | Retrieval | ✓ Precise AST lookup (exact symbols) | ~ Approximate (chunk similarity) |
  199. | Token cost per lookup | ✓ **~140 tokens** | ~ Varies by chunk strategy |
  200. | Cross-reference edges | ✓ Calls / called-by / imports | ✗ Not supported |
  201. | Privacy | ✓ **100% offline**, no data leaves | ✗ Needs embeddings API (data may egress) |
  202. | Search speed | ✓ **~3-4ms** (indexed) | ~ 200-500ms (embed + search) |
  203. | Dependency manifest parsing | ✓ **20 formats** automatically | ✗ Not designed for this |
  204. | CI/CD integration | ✓ **Built-in GitHub Action** + impact diff | ✗ Custom pipeline required |
  205. | Training data export | ✓ **Built-in JSONL** pairs | ✗ Not a feature |
  206. | Context compression | ✓ **~97% reduction** (45K→1.2K tokens) | ~ Varies by chunk strategy |
  207. | Setup | ✓ **pip install + 1 command** | ⚠ Vector DB + embedding pipeline |
  208. > Full comparison with Sourcegraph, Graphify/CodeSee, and dep-graph tools: **[docs/comparison.md](docs/comparison.md)**
  209. ---
  210. ## Enterprise & CI/CD
  211. ### Built-in GitHub Action
  212. A pre-built workflow (`.github/workflows/okf-bundle.yml`) auto-generates the bundle on every push/PR to `main`, caches previous bundles per branch, diffs with `--impact`, and posts a PR comment showing which dependency changes affect which code:
  213. ```yaml
  214. # .github/workflows/okf-bundle.yml
  215. name: OKF Bundle
  216. on:
  217. push: { branches: [main] }
  218. pull_request: { branches: [main] }
  219. jobs:
  220. generate:
  221. runs-on: ubuntu-latest
  222. steps:
  223. - uses: actions/checkout@v4
  224. - run: pip install okf-generator
  225. - run: okf generate . okf_bundle
  226. - run: okf diff .okf_bundle_prev okf_bundle --impact
  227. - uses: actions/github-script@v7 # post/update PR comment
  228. ```
  229. ### Fine-tuning data export
  230. Convert your bundle into JSONL training pairs for fine-tuning private local SLMs:
  231. ```bash
  232. # 5 pair types: codegen, qa, doc, summarize, crosslink
  233. okf pairs ./okf_bundle ./train.jsonl
  234. ```
  235. ### Docker image
  236. ```bash
  237. docker pull ghcr.io/umairbaig8/okf-generator/okf-generator:latest
  238. docker run ghcr.io/umairbaig8/okf-generator/okf-generator --help
  239. ```
  240. ### Pre-commit hook
  241. Auto-regenerates the bundle when source files change — no stale knowledge graphs:
  242. ```yaml
  243. # .pre-commit-config.yaml
  244. repos:
  245. - repo: local
  246. hooks:
  247. - id: okf-generate
  248. name: okf generate
  249. entry: okf generate
  250. language: system
  251. pass_filenames: false
  252. ```
  253. > CI/CD guide + copy-paste workflow template: **[docs/user-guide/ci-cd.md](docs/user-guide/ci-cd.md)**
  254. ---
  255. ## CLI Reference
  256. ```bash
  257. okf --help Show available commands
  258. okf <command> --help Show options for a specific command
  259. okf --version Show version
  260. ```
  261. | Command | Usage | Description |
  262. |---|---|---|
  263. | `generate` | `okf generate [src] [out] [--enrich]` | Scan codebase (auto-detects project root) |
  264. | `update` | `okf update [src] [out] [--force] [--watch]` | Incremental re-scan of changed files |
  265. | `lookup` | `okf lookup <query>` | Instant exact-symbol concept retrieval |
  266. | `ask` | `okf ask <question>` | AI-powered Q&A about your codebase |
  267. | `enrich` | `okf enrich <bundle> [--mode]` | LLM-enhance an existing bundle |
  268. | `lsp` | `okf lsp [bundle]` | LSP call-graph enrichment pass |
  269. | `diff` | `okf diff <old> <new> [--impact]` | Compare two bundles, show dependency impact |
  270. | `visualize` | `okf visualize <bundle> [out.html]` | Generate interactive D3 graph |
  271. | `mcp` | `okf mcp <bundle> [--port]` | Start MCP server with 11 tools |
  272. | `dashboard` | `okf dashboard <bundle> [--port]` | Launch FastAPI live bundle browser |
  273. | `serve` | `okf serve [dir] [--port]` | Browse bundle via local HTTP server |
  274. | `pairs` | `okf pairs <bundle> [output.jsonl]` | Export fine-tuning training pairs |
  275. | `install` | `okf install [agent]` | Write agent integration rules/configs |
  276. | `init` | `okf init [dir]` | Interactive wizard for bundle setup |
  277. | `summarize` | `okf summarize <bundle>` | Regenerate SUMMARY.md for agents |
  278. | `domains` | `okf domains` | List available domain classification rule sets |
  279. | `config` | `okf config [key=value]` | Read/write .okfconfig settings |
  280. | `migrate` | `okf migrate <bundle>` | Migrate v0.1 bundles to v0.2 |
  281. | `agent` | `okf agent [--bundle]` | Interactive REPL over the bundle |
  282. | `plugin` | `okf plugin [list\|install\|uninstall]` | Manage external parser plugins |
  283. > Full CLI reference: **[docs/cli-reference.md](docs/cli-reference.md)**
  284. ---
  285. ## FAQ
  286. **Does this require an API key or internet connection?**
  287. No. Core extraction (`okf generate`) is **fully offline and deterministic** — no LLM call is made unless you explicitly enable `--enrich`. All 18 language parsers use tree-sitter (or stdlib equivalent) and work completely air-gapped.
  288. **How is this different from RAG / vector search?**
  289. RAG retrieves chunks by semantic similarity, which is approximate and can miss exact symbols. `okf lookup` is **exact**: it indexes real functions, classes, modules, and dependencies by name and resolves to the precise concept, with zero embedding/vector infrastructure required. Same result every run.
  290. **Does my proprietary code leave my local environment?**
  291. **Never.** Core extraction runs 100% offline on your local CPU. No code or metadata is sent to any third-party cloud. Optional LLM enrichment can be enabled manually and works with self-hosted models or private enterprise APIs.
  292. **Does this work on monorepos or very large codebases?**
  293. Yes — scanning is linear in file count. Scope `okf generate` to a subdirectory if you only need part of the codebase indexed. Multi-bundle support visualizes cross-project edges.
  294. **Can I use this without any LLM at all, ever?**
  295. Yes. `okf generate` + `okf lookup` form a complete **zero-LLM workflow**. LLM enrichment and training-pair synthesis are optional layers on top — you can get full value without ever providing an API key.
  296. **Is the bundle safe to commit to git?**
  297. Yes — that's the intended workflow. Bundles are plain Markdown with YAML frontmatter — they diff cleanly, version alongside the code they describe, and can be reviewed in any code review tool.
  298. ---
  299. ## Contributing
  300. ```bash
  301. git clone https://github.com/UmairBaig8/okf-generator
  302. cd okf-generator
  303. pip install -e ".[dev]"
  304. pytest tests/ -q
  305. ```
  306. **415+ tests passing.** See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.
  307. ---
  308. ## License
  309. [MIT](LICENSE) — Copyright © 2026 Umair Baig
  310. `okf-generator` produces OKF v0.2 bundles — an extended dialect of the [Open Knowledge Format (OKF) v0.1](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) by Google. Our v0.2 adds schema version, concept identity, language, status, and typed relationships to every concept file. Not affiliated with Google.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.