Skip to content
Back to search

OpenJudge

OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/agentscope-ai/OpenJudge with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/agentscope-ai/OpenJudge'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/agentscope-ai--openjudge/badge.svg)](/listings/agentscope-ai--openjudge)

HTML

<a href="/listings/agentscope-ai--openjudge"><img src="/api/listings/agentscope-ai--openjudge/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

> πŸ’‘ **Don't want to install anything?** [Try OpenJudge online](https://openjudge.me/app/) β€” use graders directly in your browser, no setup needed.

pip install py-openjudge

> πŸ’‘ More installation methods can be found in the [Quickstart Guide](https://agentscope-ai.github.io/OpenJudge/get_started/quickstart/#installation).

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md Β· README Β· 436 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. <img src="./docs/images/logo.svg" alt="Open-Judge Logo" width="500">
  3. <br/>
  4. <h3>
  5. <em>Holistic Evaluation, Quality Rewards: Driving Application Excellence</em>
  6. </h3>
  7. <p>
  8. 🌟 <em>If you find OpenJudge helpful, please give us a <b>Star</b>!</em> 🌟
  9. </p>
  10. [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue?logo=python)](https://pypi.org/project/py-openjudge/)
  11. [![PyPI](https://img.shields.io/badge/pypi-v0.2.0-blue?logo=pypi)](https://pypi.org/project/py-openjudge/)
  12. [![Documentation](https://img.shields.io/badge/docs-online-blue?logo=readthedocs&logoColor=white)](https://agentscope-ai.github.io/OpenJudge/)
  13. [![Website](https://img.shields.io/badge/website-openjudge.me-blue?logo=googlechrome&logoColor=white)](https://openjudge.me/)
  14. [![Try Online](https://img.shields.io/badge/try%20online-free-brightgreen?logo=rocket&logoColor=white)](https://openjudge.me/app/)
  15. [![PawBench](https://img.shields.io/badge/PawBench-Model%20Γ—%20Harness%20Benchmark-orange?logo=github)](https://github.com/agentscope-ai/PawBench)
  16. [🌐 Website](https://openjudge.me/) | [πŸš€ Try Online](https://openjudge.me/app/) | [πŸ“– Documentation](https://agentscope-ai.github.io/OpenJudge/) | [🀝 Contributing](https://agentscope-ai.github.io/OpenJudge/community/contributing/) | [🐾 PawBench](https://github.com/agentscope-ai/PawBench) | [δΈ­ζ–‡](./README_zh.md)
  17. </div>
  18. OpenJudge is an **open-source evaluation framework** for **AI applications** (e.g., AI agents or chatbots) designed to **evaluate quality** and drive **continuous application optimization**.
  19. > In practice, application excellence depends on a trustworthy evaluation workflow: Collect test data β†’ Define graders β†’ Run evaluation at scale β†’ Analyze weaknesses β†’ Iterate quickly.
  20. OpenJudge provides **ready-to-use graders** and supports generating **scenario-specific rubrics (as graders)**, making this workflow **simpler**, **more professional**, and **easy to integrate** into your workflow.
  21. It can also convert grading results into **reward signals** to help you **fine-tune** and optimize your application.
  22. > **πŸš€ Try it now!** Visit [openjudge.me/app](https://openjudge.me/app/) to use graders online β€” no installation required. Test built-in graders, build custom rubrics, and explore evaluation results directly in your browser.
  23. ---
  24. ## πŸ“‘ Table of Contents
  25. - [Key Features](#-key-features)
  26. - [News](#news)
  27. - [Online Playground](#-online-playground)
  28. - [Installation](#-installation)
  29. - [Quickstart](#-quickstart)
  30. - [Integrations](#-integrations)
  31. - [Ecosystem](#-ecosystem)
  32. - [Contributing](#-contributing)
  33. - [Community](#-community)
  34. - [Citation](#-citation)
  35. ---
  36. ## News
  37. - **2026-11-10** - πŸš€ **OpenJudge v0.2.5** is now available on PyPI, with fixes for grading correctness, multimodal evaluation prompts, model request compatibility, cancellation cleanup, and Auto Arena credential handling. πŸ‘‰ [Release Notes](https://github.com/agentscope-ai/OpenJudge/releases/tag/v0.2.5) | [PyPI](https://pypi.org/project/py-openjudge/0.2.5/)
  38. - **2026-06-17** - 🐾 **PawBench v1.0** - A Model Γ— Harness co-evaluation benchmark for agentic AI: 150 tasks Β· 9 models Β· 3 harnesses, with public prompts, graders, task labels, submissions, and leaderboard slices. πŸ‘‰ [GitHub](https://github.com/agentscope-ai/PawBench) | [Leaderboard](https://agentscope-ai.github.io/PawBench/)
  39. - **2026-04-07** - πŸ”’ **Skill Graders** - 5 new LLM-based graders for evaluating AI Agent Skill packages: threat analysis (AITech taxonomy), declaration alignment, completeness, relevance, and design quality. πŸ‘‰ [Documentation](./docs/built_in_graders/skills.md) | [Cookbook](./cookbooks/skills_evaluation/README.md)
  40. - **2026-03-10** - πŸ› οΈ **New Skills** - Claude authenticity verification, find skills combo, and more. πŸ‘‰ [Browse Skills](https://openjudge.me/skills)
  41. - **2026-02-12** - πŸ“š **Reference Hallucination Arena** - Benchmark for evaluating LLM academic reference hallucination. πŸ‘‰ [Documentation](./docs/validating_graders/ref_hallucination_arena.md) | πŸ“Š [Leaderboard](https://openjudge.me/leaderboard)
  42. - **2026-01-27** - πŸ†• **Paper Review** - Automatically review academic papers using LLM-powered evaluation. πŸ‘‰ [Documentation](https://agentscope-ai.github.io/OpenJudge/applications/paper_review/)
  43. - **2026-01-27** - πŸ–₯️ **OpenJudge UI** - A Streamlit-based visual interface for grader testing and Auto Arena. πŸ‘‰ [Try Online](https://openjudge.me/app/) | Run locally: `streamlit run ui/app.py`
  44. ---
  45. ## ✨ Key Features
  46. ### πŸ“¦ Systematic & Quality-Assured Grader Library
  47. Access **50+ production-ready graders** featuring a comprehensive taxonomy, rigorously validated for reliable performance.
  48. <table>
  49. <tr>
  50. <td width="33%" valign="top">
  51. #### 🎯 General
  52. **Focus:** Semantic quality, functional correctness, structural compliance
  53. **Key Graders:**
  54. - `Relevance` - Semantic relevance scoring
  55. - `Similarity` - Text similarity measurement
  56. - `Syntax Check` - Code syntax validation
  57. - `JSON Match` - Structure compliance
  58. </td>
  59. <td width="33%" valign="top">
  60. #### πŸ€– Agent
  61. **Focus:** Agent lifecycle, tool calling, memory, plan feasibility, trajectory quality
  62. **Key Graders:**
  63. - `Tool Selection` - Tool choice accuracy
  64. - `Memory` - Context preservation
  65. - `Plan` - Strategy feasibility
  66. - `Trajectory` - Path optimization
  67. </td>
  68. <td width="33%" valign="top">
  69. #### πŸ–ΌοΈ Multimodal
  70. **Focus:** Image-text coherence, visual generation quality, image helpfulness
  71. **Key Graders:**
  72. - `Image Coherence` - Visual-text alignment
  73. - `Text-to-Image` - Generation quality
  74. - `Image Helpfulness` - Image contribution
  75. </td>
  76. </tr>
  77. </table>
  78. - 🌐 **Multi-Scenario Coverage:** Extensive support for diverse domains including Agent, text, code, math, and multimodal tasks. πŸ‘‰ [Explore Supported Scenarios](https://agentscope-ai.github.io/OpenJudge/built_in_graders/overview/)
  79. - πŸ”„ **Holistic Agent Evaluation:** Beyond final outcomes, we assess the entire lifecycleβ€”including trajectories, Memory, Reflection, and Tool Use. πŸ‘‰ [Agent Lifecycle Evaluation](https://agentscope-ai.github.io/OpenJudge/built_in_graders/agent_graders/)
  80. - βœ… **Quality Assurance:** Every grader comes with benchmark datasets and pytest integration for validation. πŸ‘‰ [View Benchmark Datasets](https://huggingface.co/datasets/agentscope-ai/OpenJudge)
  81. ### πŸ› οΈ Flexible Grader Building Methods
  82. Choose the build method that fits your requirements:
  83. * **Customization:** Clear requirements, but no existing grader? If you have explicit rules or logic, use our Python interfaces or Prompt templates to quickly define your own grader. πŸ‘‰ [Custom Grader Development Guide](https://agentscope-ai.github.io/OpenJudge/building_graders/create_custom_graders/)
  84. * **Zero-shot Rubrics Generation:** Not sure what criteria to use, and no labeled data yet? Just provide a task description and optional sample queriesβ€”the LLM will automatically generate evaluation rubrics for you. Ideal for rapid prototyping when you want to get started immediately. πŸ‘‰ [Zero-shot Rubrics Generation Guide](https://agentscope-ai.github.io/OpenJudge/building_graders/generate_rubrics_as_graders/#simple-rubric-zero-shot-generation)
  85. * **Data-driven Rubrics Generation:** Ambiguous requirements, but have few examples? Use the GraderGenerator to automatically
  86. summarize evaluation Rubrics from your annotated data, and generate a llm-based grader. πŸ‘‰ [Data-driven Rubrics Generation Guide](https://agentscope-ai.github.io/OpenJudge/building_graders/generate_rubrics_as_graders/#iterative-rubric-data-driven-generation)
  87. * **Training Judge Models:** Massive data and need peak performance? Use our training pipeline to train a dedicated Judge Model. This is ideal for complex scenarios where prompt-based grading falls short.πŸ‘‰ [Train Judge Models](https://agentscope-ai.github.io/OpenJudge/building_graders/training_judge_models/)
  88. ### πŸ”Œ Easy Integration
  89. Using mainstream observability platforms like **LangSmith** or **Langfuse**? We offer seamless integration to enhance their evaluators and automated evaluation capabilities. We also provide integrations with training frameworks like **VERL** for RL training. πŸ‘‰ See [Integrations](#-integrations) for details
  90. ### 🌐 Online Playground
  91. Explore OpenJudge without writing a single line of code. Our online platform at [openjudge.me/app](https://openjudge.me/app/) lets you:
  92. - **Test graders interactively** β€” select a built-in grader, input your data, and see results instantly
  93. - **Build custom rubrics** β€” use the zero-shot generator to create graders from task descriptions
  94. - **View leaderboards** β€” compare model performance across evaluation benchmarks at [openjudge.me/leaderboard](https://openjudge.me/leaderboard)
  95. ---
  96. ## πŸ“₯ Installation
  97. > πŸ’‘ **Don't want to install anything?** [Try OpenJudge online](https://openjudge.me/app/) β€” use graders directly in your browser, no setup needed.
  98. ```bash
  99. pip install py-openjudge
  100. ```
  101. > πŸ’‘ More installation methods can be found in the [Quickstart Guide](https://agentscope-ai.github.io/OpenJudge/get_started/quickstart/#installation).
  102. ---
  103. ## πŸš€ Quickstart
  104. > πŸ“š Complete Quickstart can be found in the [Quickstart Guide](https://agentscope-ai.github.io/OpenJudge/get_started/quickstart/).
  105. ### Simple Example
  106. A simple example to evaluate a single response:
  107. ```python
  108. import asyncio
  109. from openjudge.models import OpenAIChatModel
  110. from openjudge.graders.common.relevance import RelevanceGrader
  111. async def main():
  112. # 1️⃣ Create model client
  113. model = OpenAIChatModel(model="qwen3-32b")
  114. # 2️⃣ Initialize grader
  115. grader = RelevanceGrader(model=model)
  116. # 3️⃣ Prepare data
  117. data = {
  118. "query": "What is machine learning?",
  119. "response": "Machine learning is a subset of AI that enables computers to learn from data.",
  120. }
  121. # 4️⃣ Evaluate
  122. result = await grader.aevaluate(**data)
  123. print(f"Score: {result.score}") # Score: 4
  124. print(f"Reason: {result.reason}")
  125. if __name__ == "__main__":
  126. asyncio.run(main())
  127. ```
  128. ### Evaluate LLM Applications with Built-in Graders
  129. Use multiple built-in graders to comprehensively evaluate your LLM application: πŸ‘‰ [Explore All built-in graders](https://agentscope-ai.github.io/OpenJudge/built_in_graders/overview/)
  130. > **Business Scenario:** Evaluating an e-commerce customer service agent that handles order inquiries. We assess the agent's performance across three dimensions: **relevance**, **hallucination**, and **tool selection**.
  131. ```python
  132. import asyncio
  133. from openjudge.models import OpenAIChatModel
  134. from openjudge.graders.common import RelevanceGrader, HallucinationGrader
  135. from openjudge.graders.agent.tool.tool_selection import ToolSelectionGrader
  136. from openjudge.runner import GradingRunner
  137. from openjudge.runner.aggregator import WeightedSumAggregator
  138. from openjudge.analyzer.statistical import DistributionAnalyzer
  139. TOOL_DEFINITIONS = [
  140. {"name": "query_order", "description": "Query order status and logistics information", "parameters": {"order_id": "str"}},
  141. {"name": "query_logistics", "description": "Query detailed logistics tracking", "parameters": {"order_id": "str"}},
  142. {"name": "estimate_delivery", "description": "Estimate delivery time", "parameters": {"order_id": "str"}},
  143. ]
  144. # Prepare your dataset
  145. dataset = [{
  146. "query": "Where is my order ORD123456?",
  147. "response": "Your order ORD123456 has arrived at the Beijing distribution center and is expected to arrive tomorrow.",
  148. "context": "Order ORD123456: Arrived at Beijing distribution center, expected to arrive tomorrow.",
  149. "tool_definitions": TOOL_DEFINITIONS,
  150. "tool_calls": [{"name": "query_order", "arguments": {"order_id": "ORD123456"}}],
  151. # ... more test cases
  152. }]
  153. async def main():
  154. # 1️⃣ Initialize judge model
  155. model = OpenAIChatModel(model="qwen3-max")
  156. # 2️⃣ Configure multiple graders
  157. grader_configs = {
  158. "relevance": {"grader": RelevanceGrader(model=model), "mapper": {"query": "query", "response": "response"}},
  159. "hallucination": {"grader": HallucinationGrader(model=model), "mapper": {"query": "query", "response": "response", "context": "context"}},
  160. "tool_selection": {"grader": ToolSelectionGrader(model=model), "mapper": {"query": "query", "tool_definitions": "tool_definitions", "tool_calls": "tool_calls"}},
  161. }
  162. # 3️⃣ Set up aggregator for overall score
  163. aggregator = WeightedSumAggregator(name="overall_score", weights={"relevance": 0.3, "hallucination": 0.4, "tool_selection": 0.3})
  164. # 4️⃣ Run evaluation
  165. results = await GradingRunner(grader_configs=grader_configs, aggregators=[aggregator], max_concurrency=5).arun(dataset)
  166. # 5️⃣ Generate evaluation report
  167. overall_stats = DistributionAnalyzer().analyze(dataset, results["overall_score"])
  168. print(f"{'Overall Score':<20} | {overall_stats.mean:>15.2f}")
  169. if __name__ == "__main__":
  170. asyncio.run(main())
  171. ```
  172. ### Build Custom Graders for Your Scenario
  173. #### Zero-shot Rubric Generation
  174. Generate a custom grader from task description without labeled data: πŸ‘‰ [Zero-shot Rubrics Generation Guide](https://agentscope-ai.github.io/OpenJudge/building_graders/generate_rubrics_as_graders/#simple-rubric-zero-shot-generation)
  175. **When to use:** Quick prototyping when you have no labeled data but can clearly describe your task.
  176. ```python
  177. import asyncio
  178. from openjudge.generator.simple_rubric import SimpleRubricsGenerator, SimpleRubricsGeneratorConfig
  179. from openjudge.models import OpenAIChatModel
  180. async def main():
  181. # 1️⃣ Configure generator
  182. config = SimpleRubricsGeneratorConfig(
  183. grader_name="customer_service_grader",
  184. model=OpenAIChatModel(model="qwen3-max"),
  185. task_description="E-commerce AI customer service primarily handles order inquiry tasks (such as logistics status and ETA) while focusing on managing customer emotions.",
  186. min_score=1,
  187. max_score=3,
  188. )
  189. # 2️⃣ Generate grader
  190. generator = SimpleRubricsGenerator(config)
  191. grader = await generator.generate(dataset=[], sample_queries=[])
  192. # 3️⃣ View generated rubrics
  193. print("Generated Rubrics:", grader.kwargs.get("rubrics"))
  194. # 4️⃣ Use the grader
  195. result = await grader.aevaluate(
  196. query="My order is delayed, what should I do?",
  197. response="I understand your concern. Let me check your order status..."
  198. )
  199. print(f"\nScore: {result.score}/3\nReason: {result.reason}")
  200. if __name__ == "__main__":
  201. asyncio.run(main())
  202. ```
  203. #### Data-driven Rubric Generation
  204. Learn evaluation criteria from labeled examples: πŸ‘‰ [Data-driven Rubrics Generation Guide](https://agentscope-ai.github.io/OpenJudge/building_graders/generate_rubrics_as_graders/#iterative-rubric-data-driven-generation)
  205. **When to use:** You have labeled data and need high-accuracy graders for production use, especially when evaluation criteria are implicit.
  206. ```python
  207. import asyncio
  208. from openjudge.generator.iterative_rubric.generator import IterativeRubricsGenerator, IterativePointwiseRubricsGeneratorConfig
  209. from openjudge.models import OpenAIChatModel
  210. from openjudge.models.schema.prompt_template import LanguageEnum
  211. # Prepare labeled dataset (simplified example, recommend 10+ samples in practice)
  212. labeled_dataset = [
  213. {"query": "My order hasn't arrived after 10 days, I want to complain!", "response": "I sincerely apologize for the delay. I completely understand your frustration! Your order was delayed due to weather conditions, but it has now resumed shipping and is expected to arrive tomorrow. I've marked it for priority delivery.", "label_score": 5},
  214. {"query": "Where is my package? I need it urgently!", "response": "I understand your urgency! Your package is currently out for delivery and is expected to arrive before 2 PM today. The delivery driver's contact number is 138xxxx.", "label_score": 5},
  215. {"query": "Why hasn't my order arrived yet? I've been waiting for days!", "response": "Your order is expected to arrive the day after tomorrow.", "label_score": 2},
  216. {"query": "The logistics hasn't updated in 3 days, is it lost?", "response": "Hello, your package is not lost. It's still in transit, please wait patiently.", "label_score": 3},
  217. # ... more labeled examples
  218. ]
  219. async def main():
  220. # 1️⃣ Configure generator
  221. config = IterativePointwiseRubricsGeneratorConfig(
  222. grader_name="customer_service_grader_v2", model=OpenAIChatModel(model="qwen3-max"),
  223. min_score=1, max_score=5,
  224. enable_categorization=True, categories_number=5, # Enable categorization, Aggregate into 5 themes
  225. )
  226. # 2️⃣ Generate grader from labeled data
  227. generator = IterativeRubricsGenerator(config)
  228. grader = await generator.generate(labeled_dataset)
  229. # 3️⃣ View learned rubrics
  230. print("\nLearned Rubrics from Labeled Data:\n",grader.kwargs.get("rubrics", "No rubrics generated"))
  231. # 4️⃣ Evaluate new samples
  232. test_cases = [
  233. {"query": "My order hasn't moved in 5 days, can you check? I'm a bit worried", "response": "I understand your concern! Let me check immediately: Your package is currently at XX distribution center. Due to recent high order volume, there's a slight delay, but it's expected to arrive the day after tomorrow. I'll proactively contact you if there are any issues."},
  234. {"query": "Why is this delivery so slow? I'm waiting to use it!", "response": "Checking, please wait."},
  235. ]
  236. print("\n" + "=" * 70, "\nEvaluation Results:\n", "=" * 70)
  237. for i, case in enumerate(test_cases):
  238. result = await grader.aevaluate(query=case["query"], response=case["response"])
  239. print(f"\n[Test {i+1}]\n Query: {case['query']}\n Response: {case['response']}\n Score: {result.score}/5\n Reason: {result.reason[:200]}...")
  240. if __name__ == "__main__":
  241. asyncio.run(main())
  242. ```
  243. ---
  244. ## πŸ”— Integrations
  245. Seamlessly connect OpenJudge with mainstream observability and training platforms:
  246. | Category | Platform | Status | Documentation |
  247. |:---------|:---------|:------:|:--------------|
  248. | **Observability** | [LangSmith](https://smith.langchain.com/) | βœ… Available | πŸ‘‰ [LangSmith Integration Guide](https://agentscope-ai.github.io/OpenJudge/integrations/langsmith/) |
  249. | | [Langfuse](https://langfuse.com/) | βœ… Available | πŸ‘‰ [Langfuse Integration Guide](https://agentscope-ai.github.io/OpenJudge/integrations/langfuse/) |
  250. | | Other frameworks | πŸ”΅ Planned | β€” |
  251. | **Training** | [verl](https://github.com/volcengine/verl) | βœ… Available | πŸ‘‰ [VERL Integration Guide](https://agentscope-ai.github.io/OpenJudge/integrations/verl/) |
  252. | | [Trinity-RFT](https://github.com/modelscope/Trinity-RFT) | πŸ”΅ Planned | β€” |
  253. > πŸ’¬ Have a framework you'd like us to prioritize? [Open an Issue](https://github.com/agentscope-ai/OpenJudge/issues)!
  254. ---
  255. ## 🌍 Ecosystem
  256. OpenJudge is the foundation of a growing evaluation ecosystem. These projects share OpenJudge's philosophy of **evaluation-driven optimization** while targeting specific verticals.
  257. ### 🐾 [PawBench](https://github.com/agentscope-ai/PawBench) β€” Model Γ— Harness Co-Evaluation Benchmark
  258. The same model can behave very differently depending on which agent runtime (harness) it runs inside. PawBench evaluates **the model and the harness together**, keeping enough metadata to analyze both dimensions independently:
  259. $$\text{Agent Performance} = f(\text{Model}, \text{Harness})$$
  260. | Dimension | Coverage |
  261. |:----------|:---------|
  262. | **Tasks** | 150 tasks from 6 sources (ClawEval, QwenClawBench, PinchBench, SkillsBench, WildClawBench, self-built) |
  263. | **Models** | 9 models (Qwen, Claude, GLM, etc.) |
  264. | **Harnesses** | 3 harnesses (QwenPaw, OpenClaw, Hermes) |
  265. | **Task labels** | 5 dimensions: scenario, capability, complexity, modality, environment |
  266. Key findings from v1.0: harness design alone can shift a model's score by **10+ points** β€” a gap comparable to many model upgrades. PawBench provides slice diagnostics to pinpoint whether regressions come from the model, the harness, or the grader.
  267. πŸ‘‰ [GitHub](https://github.com/agentscope-ai/PawBench) | [Leaderboard](https://agentscope-ai.github.io/PawBench/) | [Documentation](https://github.com/agentscope-ai/PawBench#readme)
  268. ---
  269. ## 🀝 Contributing
  270. We love your input! We want to make contributing to OpenJudge as easy and transparent as possible.
  271. > **🎨 Adding New Graders** β€” Have domain-specific evaluation logic? Share it with the community!
  272. > **πŸ› Reporting Bugs** β€” Found a glitch? Help us fix it by [opening an issue](https://github.com/agentscope-ai/OpenJudge/issues)
  273. > **πŸ“ Improving Docs** β€” Clearer explanations or better examples are always welcome
  274. > **πŸ’‘ Proposing Features** β€” Have ideas for new integrations? Let's discuss!
  275. πŸ“– See full [Contributing Guidelines](https://agentscope-ai.github.io/OpenJudge/community/contributing/) for coding standards and PR process.
  276. ---
  277. ## πŸ’¬ Community
  278. Join our DingTalk group to connect with the community:
  279. <div align="center">
  280. <img src="./docs/images/dingtalk_qr_code.png" alt="DingTalk QR Code" width="200">
  281. </div>
  282. ---
  283. ## Migration Guide (v0.1.x β†’ v0.2.0)
  284. > OpenJudge was previously distributed as the legacy package `rm-gallery` (v0.1.x). Starting from v0.2.0, it is published as `py-openjudge` and the Python import namespace is `openjudge`.
  285. **OpenJudge v0.2.0 is NOT backward compatible with v0.1.x.**
  286. If you are currently using v0.1.x, choose one of the following paths:
  287. - **Stay on v0.1.x (legacy)**: keep using the old package
  288. ```bash
  289. pip install rm-gallery
  290. ```
  291. We preserved the source code of **v0.1.7 (the latest v0.1.x release)** in the [`v0.1.7-legacy` branch](https://github.com/agentscope-ai/OpenJudge/tree/v0.1.7-legacy).
  292. - **Migrate to v0.2.0 (recommended)**: follow the **[Installation](#-installation)** section above, then walk through **[Quickstart](#-quickstart)** (or the full [Quickstart Guide](https://agentscope-ai.github.io/OpenJudge/get_started/quickstart/)) to update your imports / usage.
  293. If you run into migration issues, please [open an issue](https://github.com/agentscope-ai/OpenJudge/issues) with your minimal repro and current version.
  294. ---
  295. ## πŸ“„ Citation
  296. If you use OpenJudge in your research, please cite:
  297. ```bibtex
  298. @software{
  299. title = {OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards},
  300. author = {The OpenJudge Team},
  301. url = {https://github.com/agentscope-ai/OpenJudge},
  302. month = {07},
  303. year = {2025}
  304. }
  305. ```
  306. ---
  307. <div align="center">
  308. **Made with ❀️ by the OpenJudge Team**
  309. [🌐 Website](https://openjudge.me/) Β· [πŸš€ Try Online](https://openjudge.me/app/) Β· [⭐ Star Us](https://github.com/agentscope-ai/OpenJudge) Β· [πŸ› Report Bug](https://github.com/agentscope-ai/OpenJudge/issues) Β· [πŸ’‘ Request Feature](https://github.com/agentscope-ai/OpenJudge/issues)
  310. </div>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.