Back to search
dspy-skills-marketplace
MarketplaceWorks with Claude CodeGitHub
Collection of Claude Skills for DSPy framework - program language models, optimize prompts, and build RAG pipelines systematically
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Checked today
Badge for your README
Markdown
[](/listings/omidzamani--dspy-skills)HTML
<a href="/listings/omidzamani--dspy-skills"><img src="/api/listings/omidzamani--dspy-skills/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
/plugin marketplace add OmidZamani/dspy-skills /plugin install dspy-skills@dspy-skills-marketplace
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json · Manifest · 19 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- {
- "name": "dspy-skills-marketplace",
- "owner": {
- "name": "OmidZamani",
- "url": "https://github.com/OmidZamani"
- },
- "plugins": [
- {
- "name": "dspy-skills",
- "source": "./",
- "description": "Collection of 22 focused skills for building, optimizing, evaluating, and deploying DSPy applications.",
- "version": "2.0.0",
- "author": {
- "name": "OmidZamani",
- "url": "https://github.com/OmidZamani"
- }
- }
- ]
- }
README.md · README · 108 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- # DSPy Skills Collection
- A Claude Code plugin containing 22 focused skills for programming, optimizing, evaluating, and deploying LLM applications with [DSPy](https://dspy.ai/).
- **Stable DSPy baseline:** `3.2.1`, released May 5, 2026. Install the stable series with:
- ```bash
- pip install -U "dspy>=3.2.1,<3.3"
- ```
- DSPy `3.3.0b1` is a prerelease published May 28, 2026. See [prerelease notes](docs/dspy-prerelease-3.3.md) before opting in.
- ## Skills Index
- ### Start Here
- | Skill | Use it for |
- |-------|------------|
- | [dspy-signature-designer](skills/dspy-signature-designer/SKILL.md) | Typed signatures and structured outputs |
- | [dspy-custom-module-design](skills/dspy-custom-module-design/SKILL.md) | Reusable `dspy.Module` architecture |
- | [dspy-evaluation-suite](skills/dspy-evaluation-suite/SKILL.md) | Metrics, baselines, and comparisons |
- | [dspy-optimizer-selection](skills/dspy-optimizer-selection/SKILL.md) | Choosing among DSPy's optimizer families |
- ### Retrieval and Data
- | Skill | Use it for |
- |-------|------------|
- | [dspy-rag-pipeline](skills/dspy-rag-pipeline/SKILL.md) | RAG and multi-hop retrieval pipelines |
- | [dspy-embedding-retrieval](skills/dspy-embedding-retrieval/SKILL.md) | `Embedder`, `Embeddings`, FAISS, and local corpora |
- | [dspy-haystack-integration](skills/dspy-haystack-integration/SKILL.md) | Optimizing Haystack-backed pipelines |
- ### Agents and Reasoning
- | Skill | Use it for |
- |-------|------------|
- | [dspy-react-agent-builder](skills/dspy-react-agent-builder/SKILL.md) | ReAct tool agents |
- | [dspy-mcp-tool-integration](skills/dspy-mcp-tool-integration/SKILL.md) | MCP tools in DSPy agents |
- | [dspy-reasoning-modules](skills/dspy-reasoning-modules/SKILL.md) | `RLM`, `ProgramOfThought`, `CodeAct`, and `Parallel` |
- | [dspy-advanced-module-composition](skills/dspy-advanced-module-composition/SKILL.md) | Ensembles and composed modules |
- ### Optimizers
- | Skill | Use it for |
- |-------|------------|
- | [dspy-bootstrap-fewshot](skills/dspy-bootstrap-fewshot/SKILL.md) | Fast demo bootstrapping |
- | [dspy-miprov2-optimizer](skills/dspy-miprov2-optimizer/SKILL.md) | Instruction and demo search |
- | [dspy-gepa-reflective](skills/dspy-gepa-reflective/SKILL.md) | Reflective instruction optimization |
- | [dspy-simba-optimizer](skills/dspy-simba-optimizer/SKILL.md) | Mini-batch introspective optimization |
- | [dspy-finetune-bootstrap](skills/dspy-finetune-bootstrap/SKILL.md) | Weight optimization and distillation |
- | [dspy-better-together](skills/dspy-better-together/SKILL.md) | Sequencing prompt and weight optimizers |
- | [dspy-optimize-anything](skills/dspy-optimize-anything/SKILL.md) | GEPA optimization for text artifacts outside DSPy |
- ### Production and Interfaces
- | Skill | Use it for |
- |-------|------------|
- | [dspy-output-refinement-constraints](skills/dspy-output-refinement-constraints/SKILL.md) | `Refine` and `BestOfN` constraints |
- | [dspy-adapters-multimodal](skills/dspy-adapters-multimodal/SKILL.md) | Adapters, native tools, image, audio, and file inputs |
- | [dspy-production-deployment](skills/dspy-production-deployment/SKILL.md) | Cache hardening, save/load, usage tracking, async, and streaming |
- | [dspy-debugging-observability](skills/dspy-debugging-observability/SKILL.md) | Inspection, callbacks, and MLflow tracing |
- ## Typical Workflow
- 1. Define typed inputs and outputs with [dspy-signature-designer](skills/dspy-signature-designer/SKILL.md).
- 2. Build a module, RAG pipeline, or agent.
- 3. Establish a baseline with [dspy-evaluation-suite](skills/dspy-evaluation-suite/SKILL.md).
- 4. Choose an optimizer with [dspy-optimizer-selection](skills/dspy-optimizer-selection/SKILL.md).
- 5. Harden the runtime with [dspy-production-deployment](skills/dspy-production-deployment/SKILL.md).
- ## Optional Dependencies
- ```bash
- # Bayesian search for MIPROv2
- pip install -U "dspy[optuna]>=3.2.1,<3.3"
- # MCP tool integration
- pip install -U "dspy[mcp]>=3.2.1,<3.3"
- # Large local embedding corpora
- pip install faiss-cpu
- # optimize_anything
- pip install -U "gepa>=0.1.1,<0.2"
- ```
- ## Install as a Claude Code Plugin
- ```text
- /plugin marketplace add OmidZamani/dspy-skills
- /plugin install dspy-skills@dspy-skills-marketplace
- ```
- ## Documentation
- - [DSPy framework guide](docs/dspy-framework.md)
- - [DSPy official documentation](https://dspy.ai/)
- - [DSPy releases](https://github.com/stanfordnlp/dspy/releases)
- ## Validate Changes
- ```bash
- python3 scripts/validate_repo.py
- python3 -m compileall -q examples skills/dspy-haystack-integration/examples
- ```
- ## License
- MIT License. See [LICENSE](LICENSE).
examples/code-snippets.py · Bundled script · 322 lines · 1 finding marked
- """Production-ready DSPy code examples.
- This module contains comprehensive examples for all DSPy skills.
- """
- import dspy
- from typing import Literal, Optional
- import logging
- logger = logging.getLogger(__name__)
- # =============================================================================
- # CONFIGURATION
- # =============================================================================
- def configure_dspy(model: str = "openai/gpt-4o-mini", retriever_url: Optional[str] = None):
- """Configure DSPy with LM and optional retriever."""
- lm = dspy.LM(model)
- if retriever_url:
- rm = dspy.ColBERTv2(url=retriever_url)
- dspy.configure(lm=lm, rm=rm)
- else:
- dspy.configure(lm=lm)
- return lm
- # =============================================================================
- # SIGNATURES
- # =============================================================================
- class QASignature(dspy.Signature):
- """Answer questions concisely."""
- question: str = dspy.InputField()
- answer: str = dspy.OutputField(desc="Brief, factual answer")
- class ClassificationSignature(dspy.Signature):
- """Classify text sentiment."""
- text: str = dspy.InputField()
- sentiment: Literal['positive', 'negative', 'neutral'] = dspy.OutputField()
- confidence: float = dspy.OutputField(desc="0.0 to 1.0")
- class RAGSignature(dspy.Signature):
- """Answer using retrieved context."""
- context: list[str] = dspy.InputField(desc="Retrieved passages")
- question: str = dspy.InputField()
- answer: str = dspy.OutputField(desc="Answer grounded in context")
- # =============================================================================
- # MODULES
- # =============================================================================
- class SimpleQA(dspy.Module):
- """Basic question answering."""
- def __init__(self):
- self.predict = dspy.Predict(QASignature)
- def forward(self, question: str):
- return self.predict(question=question)
- class ChainOfThoughtQA(dspy.Module):
- """QA with reasoning."""
- def __init__(self):
- self.cot = dspy.ChainOfThought("question -> answer")
- def forward(self, question: str):
- return self.cot(question=question)
- class RAGModule(dspy.Module):
- """Retrieval-Augmented Generation."""
- def __init__(self, num_passages: int = 3):
- super().__init__()
- self.retrieve = dspy.Retrieve(k=num_passages)
- self.generate = dspy.ChainOfThought(RAGSignature)
- def forward(self, question: str):
- try:
- context = self.retrieve(question).passages
- pred = self.generate(context=context, question=question)
- return dspy.Prediction(context=context, answer=pred.answer)
- except Exception as e:
- logger.error(f"RAG failed: {e}")
- return dspy.Prediction(context=[], answer="Error processing question")
- class ReActAgent(dspy.Module):
- """Agent with tool use."""
- def __init__(self, tools: list):
- self.react = dspy.ReAct("question -> answer", tools=tools)
- def forward(self, question: str):
- return self.react(question=question)
- # =============================================================================
- # TOOLS FOR AGENTS
- # =============================================================================
- def search_wikipedia(query: str) -> str:
- """Search Wikipedia for information."""
- try:
- results = dspy.ColBERTv2(
- url='http://20.102.90.50:2017/wiki17_abstracts'
- Note: Talks to a raw IP address or throwaway host, where malware is often staged
- Line 113 on GitHub
- )(query, k=1)
- return results[0]['text'] if results else "No results found"
- except Exception as e:
- return f"Search error: {e}"
- def calculate(expression: str) -> float:
- """Evaluate a mathematical expression."""
- try:
- with dspy.PythonInterpreter() as interpreter:
- return interpreter.execute(expression)
- except Exception as e:
- return f"Calculation error: {e}"
- # =============================================================================
- # METRICS
- # =============================================================================
- def exact_match_metric(example, pred, trace=None) -> bool:
- """Check for exact answer match."""
- if not hasattr(pred, 'answer') or not pred.answer:
- return False
- return example.answer.lower().strip() == pred.answer.lower().strip()
- def contains_answer_metric(example, pred, trace=None) -> float:
- """Check if prediction contains the answer."""
- if not hasattr(pred, 'answer') or not pred.answer:
- return 0.0
- return float(example.answer.lower() in pred.answer.lower())
- def gepa_feedback_metric(example, pred, trace=None, pred_name=None, pred_trace=None):
- """Metric with feedback for GEPA optimizer."""
- if not hasattr(pred, 'answer') or not pred.answer:
- return dspy.Prediction(score=0.0, feedback="No answer generated")
- correct = example.answer.lower() in pred.answer.lower()
- if correct:
- return dspy.Prediction(score=1.0, feedback="Correct answer provided")
- else:
- return dspy.Prediction(
- score=0.0,
- feedback=f"Expected '{example.answer}', got '{pred.answer}'"
- )
- # =============================================================================
- # OPTIMIZATION EXAMPLES
- # =============================================================================
- def optimize_with_bootstrap(program, trainset, metric=None):
- """Optimize using BootstrapFewShot."""
- from dspy.teleprompt import BootstrapFewShot
- metric = metric or contains_answer_metric
- optimizer = BootstrapFewShot(
- metric=metric,
- max_bootstrapped_demos=4,
- max_labeled_demos=4
- )
- return optimizer.compile(program, trainset=trainset)
- def optimize_with_miprov2(program, trainset, auto="medium"):
- """Optimize using MIPROv2."""
- optimizer = dspy.MIPROv2(
- metric=dspy.evaluate.answer_exact_match,
- auto=auto,
- num_threads=24
- )
- return optimizer.compile(program, trainset=trainset)
- def optimize_with_gepa(program, trainset):
- """Optimize using GEPA with reflection."""
- optimizer = dspy.GEPA(
- metric=gepa_feedback_metric,
- reflection_lm=dspy.LM("openai/gpt-4o"),
- auto="medium"
- )
- return optimizer.compile(program, trainset=trainset)
- def optimize_anything_single_task(seed_artifact, evaluator_fn, background_info=None):
- """Optimize any text artifact using GEPA's optimize_anything (single-task)."""
- import gepa.optimize_anything as oa
- from gepa.optimize_anything import EngineConfig, GEPAConfig
- result = oa.optimize_anything(
- seed_candidate=seed_artifact,
- evaluator=evaluator_fn,
- background=background_info,
- config=GEPAConfig(engine=EngineConfig(max_metric_calls=100)),
- )
- return result.best_candidate
- def optimize_anything_generalization(seed_artifact, evaluator_fn, trainset, valset, background_info=None):
- """Optimize a text artifact that generalizes to unseen examples."""
- import gepa.optimize_anything as oa
- from gepa.optimize_anything import EngineConfig, GEPAConfig
- result = oa.optimize_anything(
- seed_candidate=seed_artifact,
- evaluator=evaluator_fn,
- dataset=trainset,
- valset=valset,
- background=background_info,
- config=GEPAConfig(engine=EngineConfig(max_metric_calls=100)),
- )
- return result.best_candidate
- def optimize_anything_seedless(evaluator_fn, objective, background_info=None):
- """Optimize from scratch — describe what you need, no seed required."""
- import gepa.optimize_anything as oa
- from gepa.optimize_anything import EngineConfig, GEPAConfig
- result = oa.optimize_anything(
- evaluator=evaluator_fn,
- objective=objective,
- background=background_info,
- config=GEPAConfig(engine=EngineConfig(max_metric_calls=100)),
- )
- return result.best_candidate
- def finetune_program(program, trainset, output_dir="./finetuned"):
- """Fine-tune model weights."""
- optimizer = dspy.BootstrapFinetune(
- metric=exact_match_metric,
- train_kwargs={
- 'learning_rate': 5e-5,
- 'num_train_epochs': 3,
- 'output_dir': output_dir
- }
- )
- program.set_lm(dspy.settings.lm)
- return optimizer.compile(program, trainset=trainset)
- # =============================================================================
- # EVALUATION
- # =============================================================================
- def evaluate_program(program, devset, metric=None, num_threads=8):
- """Evaluate program on development set."""
- from dspy.evaluate import Evaluate
- metric = metric or contains_answer_metric
- evaluator = Evaluate(
- devset=devset,
- metric=metric,
- num_threads=num_threads,
- display_progress=True
- )
- return evaluator(program)
- def compare_programs(programs: dict, devset, metric=None):
- """Compare multiple program variants."""
- results = {}
- for name, program in programs.items():
- score = evaluate_program(program, devset, metric)
- results[name] = score
- logger.info(f"{name}: {score:.2%}")
- best = max(results, key=results.get)
- logger.info(f"Best: {best} ({results[best]:.2%})")
- return results
- # =============================================================================
- # EXAMPLE USAGE
- # =============================================================================
- if __name__ == "__main__":
- # Configure
- configure_dspy()
- # Simple QA
- qa = SimpleQA()
- result = qa(question="What is the capital of France?")
- print(f"Answer: {result.answer}")
- # Chain of Thought
- cot_qa = ChainOfThoughtQA()
- result = cot_qa(question="What is 15% of 80?")
- print(f"Reasoning: {result.reasoning}")
- print(f"Answer: {result.answer}")
- # Agent with tools
- agent = ReActAgent(tools=[search_wikipedia, calculate])
- result = agent(question="What is the population of Paris divided by 1000?")
- print(f"Agent answer: {result.answer}")
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.