Skip to content
Back to search

minicpm

MiniCPM5: SOTA on-device LLMs, small yet powerful.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/openbmb--minicpm/badge.svg)](/listings/openbmb--minicpm)

HTML

<a href="/listings/openbmb--minicpm"><img src="/api/listings/openbmb--minicpm/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

We recommend using the following sets of sampling parameters for generation: `temperature=1.0, top_p=0.95, min_p=0.0`.

If you encounter repetitive outputs, try: `temperature=1.0, top_p=0.95, min_p=0.0, repetition_penalty=1.05`.

Please note that the support for sampling parameters varies according to inference frameworks.

vLLM

pip install "vllm>=0.21"
vllm serve openbmb/MiniCPM5-2B --port 8000
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openbmb/MiniCPM5-2B",
    "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
    "max_tokens": 128,
    "temperature": 1.0, "top_p": 0.95
  }'

SGLang

pip install "sglang[srt]>=0.5.16"
python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000
curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openbmb/MiniCPM5-2B",
    "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
    "max_tokens": 128,
    "temperature": 1.0, "top_p": 0.95
  }'

**Speculative decoding (DSpark)**: we also release [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:

python -m sglang.launch_server \
  --model-path openbmb/MiniCPM5-2B \
  --trust-remote-code \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \
  --speculative-dspark-block-size 7 \
  --port 30000

Llama.cpp

llama-server -m MiniCPM5-2B-F16.gguf -a MiniCPM5-2B --port 8080 -ngl 99 -c 8192 --jinja

`-c 8192` sets the context length. You can adjust this value as needed.

curl http://localhost:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "MiniCPM5-2B",
        "messages": [{"role": "user", "content": "1+1=?"}],
        "temperature": 1.0, "top_p": 0.95, "min_p": 0.0, "max_tokens": 256
    }'

In llama.cpp, the default `min_p=0.05` can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set `min_p=0.0`.

Transformers

pip install -U "transformers>=5.6" accelerate torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "openbmb/MiniCPM5-2B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)
messages = [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 1,373 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. <img src="./assets/minicpm_logo.png" width="500em" ></img>
  3. </div>
  4. <h4 align="center">
  5. <p>
  6. <a href="https://github.com/OpenBMB/MiniCPM/blob/main/README-cn.md">中文</a> | <b>English</b>
  7. <p>
  8. </h4>
  9. <p align="center">
  10. <a href="https://arxiv.org/pdf/2506.07900" target="_blank">MiniCPM Tech Report</a> |
  11. <a href="https://openbmb.cn/docs/" target="_blank">MiniCPM Wiki (in Chinese)</a> |
  12. <a href="https://github.com/OpenBMB/MiniCPM-V/" target="_blank">MiniCPM-V Repo</a> |
  13. <a href="https://ultradata.openbmb.cn/" target="_blank">UltraData</a> |
  14. <a href="https://huggingface.co/spaces/openbmb/MiniCPM5-2B-Demo" target="_blank">Online Demo</a>
  15. </p>
  16. <p align="center">
  17. Join our <a href="https://discord.gg/3cGQn9b3YM" target="_blank">discord</a> and <a href="https://applink.feishu.cn/client/chat/chatter/add_by_link?link_token=559o65ab-9086-442a-96c9-cf382e80b22d" target="_blank">Feishu/Lark</a> |
  18. <a href="https://mp.weixin.qq.com/s/KIhH2nCURBXuFXAtYRpuXg?poc_token=HBIsUWijxino8oJ5s6HcjcfXFRi0Xj2LJlxPYD9c">Join Us</a>
  19. </p>
  20. ## ✨ Highlights
  21. We are releasing **MiniCPM5-2B**, the second model in the **MiniCPM5** series, following [MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B). It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.
  22. 🏆 **2B-class open-source SOTA**: compared with strong open-source models of similar size, MiniCPM5-2B achieves SOTA performance within this comparison set. It remains competitive with 4B-class models overall, while showing its advantages over models of comparable size in coding, mathematics, long-context understanding, tool use, and agentic tasks.
  23. ![MiniCPM5-2B capability comparison by domain](./assets/minicpm5/minicpm5_2b_public_leaderboard_radar_en.png)
  24. 📂 **Open High-Quality Data**: Alongside the model, we are releasing the high-quality training datasets behind it as part of the [UltraData](https://ultradata.openbmb.cn/) family: [UltraX](https://huggingface.co/datasets/openbmb/UltraX-Preview), a high-quality web pre-training dataset; [UltraData-Code](https://huggingface.co/datasets/openbmb/UltraData-Code), featuring L0–L3 tiered code data management to drive a significant leap in coding capabilities; [UltraData-SFT-Agent-2609](https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609), comprising 500K agent training samples to enhance comprehensive on-device agent capabilities; and [UltraData-RL-2609](https://huggingface.co/datasets/openbmb/UltraData-RL-2609), with 80K+ high-quality RL training samples covering mathematics, code, general knowledge, and long-context reasoning.
  25. ## 🔥 Changelog
  26. - 📌 [2026.09.07] **[MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)** is released: a compact 2B-class dense model for on-device and resource-constrained use, paired with deployment / fine-tuning [Agent Skills](./skills/).
  27. - [2026.05.19] **[MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)** is released: a compact 1B-class dense model for on-device and resource-constrained use, paired with deployment / fine-tuning [Agent Skills](./skills/).
  28. - [2026.02.11] **[MiniCPM-SALA](https://huggingface.co/openbmb/MiniCPM-SALA)** is released: a sparse-and-linear hybrid attention model for million-token context modeling and efficient inference.
  29. - [2025.09.05] **[MiniCPM4.1 series](https://huggingface.co/collections/openbmb/minicpm-4-6841ab29d180257e940baa9b)** is released: a trainable sparse-attention model with hybrid reasoning.
  30. - [2025.06.06] **[MiniCPM4](https://huggingface.co/collections/openbmb/minicpm-4-6841ab29d180257e940baa9b)** is released: an end-side model with over 5x generation acceleration on typical edge chips.
  31. <details>
  32. <summary>Older entries (2024 + InfLLM-V2 paper)</summary>
  33. - [2025.09.29] **[InfLLM-V2 paper](https://arxiv.org/abs/2509.24663) is released!** We can train a sparse attention model with only 5B long-text tokens.
  34. - [2024.09.05] We release **[MiniCPM3-4B](https://huggingface.co/openbmb/MiniCPM3-4B)**! This model outperforms Phi-3.5-mini-instruct and GPT-3.5-Turbo-0125 and is comparable to several models with 7B-9B parameters like Llama3.1-8B-Instruct, Qwen2-7B-Instruct, and GLM-4-9B-Chat.
  35. - [2024.07.05] Released **[MiniCPM-S-1B](https://huggingface.co/openbmb/MiniCPM-S-1B-sft)**! This model achieves an average sparsity of 87.89% in the FFN layer, reducing FFN FLOPs by 84%, while maintaining downstream task performance.
  36. - [2024.04.11] Released **[MiniCPM-2B-128k](https://huggingface.co/openbmb/MiniCPM-2B-128k)**, **[MiniCPM-MoE-8x2B](https://huggingface.co/openbmb/MiniCPM-MoE-8x2B)** and **[MiniCPM-1B](https://huggingface.co/openbmb/MiniCPM-1B-sft-bf16)**! Click [here](https://openbmb.vercel.app/) to read our technical blog.
  37. - [2024.02.01] Released **[MiniCPM-2B](https://huggingface.co/openbmb/MiniCPM-2B-sft-bf16)**! This model performs similarly to Mistral-7B on public benchmarks (with better performance in Chinese, math, and code abilities) and overall outperforms models like Llama2-13B, MPT-30B, and Falcon-40B.
  38. </details>
  39. ## 🧭 Quick Links
  40. - [✨ Highlights](#-highlights)
  41. - [🔥 Changelog](#-changelog)
  42. - [📦 Model Downloads](#-model-downloads)
  43. - [🚀 MiniCPM5-2B](#-minicpm5-2b)
  44. - [Introduction](#introduction)
  45. - [Evaluation Results](#evaluation-results)
  46. - [Training Recipe](#training-recipe)
  47. - [What does RL + OPD bring?](#what-does-rl--opd-bring)
  48. - [Quickstart](#quickstart)
  49. - [Deployment and Fine-tuning Cookbooks and Agent Skills](#deployment-and-fine-tuning-cookbooks-and-agent-skills)
  50. - [Other Supported Frameworks](#other-supported-frameworks)
  51. - [🚀 MiniCPM5-1B](#-minicpm5-1b)
  52. - [🧪 MiniCPM-SALA](#-minicpm-sala)
  53. - [⚡ MiniCPM4 & MiniCPM4.1 Series](#-minicpm4-and-minicpm41-series)
  54. - [Legacy topics →](./docs/README-legacy.md): BitCPM4 quantization, MiniCPM4 applications
  55. - [📄 LICENSE](#-license) · [🏛 Institutions](#-institutions) · [📚 Citation](#-citation)
  56. ## 📦 Model Downloads
  57. **Current release: MiniCPM5-2B / MiniCPM5-1B**(BF16 / GGUF / MLX):
  58. **MiniCPM5-2B**
  59. | HuggingFace | ModelScope |
  60. |---|---|
  61. | [MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) | [MiniCPM5-2B](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B) |
  62. | [MiniCPM5-2B-SFT](https://huggingface.co/openbmb/MiniCPM5-2B-SFT) | [MiniCPM5-2B-SFT](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-SFT) |
  63. | [MiniCPM5-2B-Midtrain](https://huggingface.co/openbmb/MiniCPM5-2B-Midtrain) | [MiniCPM5-2B-Midtrain](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-Midtrain) |
  64. | [MiniCPM5-2B-Base](https://huggingface.co/openbmb/MiniCPM5-2B-Base) | [MiniCPM5-2B-Base](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-Base) |
  65. | [MiniCPM5-2B-GGUF](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) | [MiniCPM5-2B-GGUF](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-GGUF) |
  66. | [MiniCPM5-2B-MLX](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) | [MiniCPM5-2B-MLX](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-MLX) |
  67. | [MiniCPM5-2B-GPTQ](https://huggingface.co/openbmb/MiniCPM5-2B-GPTQ) | [MiniCPM5-2B-GPTQ](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-GPTQ) |
  68. | [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark) | [MiniCPM5-2B-DSpark](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-2B-DSpark) |
  69. **MiniCPM5-1B**
  70. | HuggingFace | ModelScope |
  71. |---|---|
  72. | [MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B) | [MiniCPM5-1B](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-1B) |
  73. | [MiniCPM5-1B-SFT](https://huggingface.co/openbmb/MiniCPM5-1B-SFT) | [MiniCPM5-1B-SFT](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-1B-SFT) |
  74. | [MiniCPM5-1B-Base](https://huggingface.co/openbmb/MiniCPM5-1B-Base) | [MiniCPM5-1B-Base](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-1B-Base) |
  75. | [MiniCPM5-1B-GGUF](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | [MiniCPM5-1B-GGUF](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-1B-GGUF) |
  76. | [MiniCPM5-1B-MLX](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | [MiniCPM5-1B-MLX](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-1B-MLX) |
  77. **Other key releases:**
  78. | HuggingFace | ModelScope |
  79. |---|---|
  80. | [MiniCPM-SALA](https://huggingface.co/openbmb/MiniCPM-SALA) | [MiniCPM-SALA](https://www.modelscope.cn/models/OpenBMB/MiniCPM-SALA) |
  81. | [MiniCPM4.1-8B](https://huggingface.co/openbmb/MiniCPM4.1-8B) | [MiniCPM4.1-8B](https://www.modelscope.cn/models/OpenBMB/MiniCPM4.1-8B) |
  82. | [MiniCPM4-0.5B](https://huggingface.co/openbmb/MiniCPM4-0.5B) | [MiniCPM4-0.5B](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-0.5B) |
  83. <details>
  84. <summary>📋 Click to view earlier MiniCPM releases: 4, BitCPM, applications, MiniCPM3 / 2B / 1B</summary>
  85. **Earlier flagships:**
  86. | HuggingFace | ModelScope |
  87. |---|---|
  88. | [MiniCPM4-8B](https://huggingface.co/openbmb/MiniCPM4-8B) | [MiniCPM4-8B](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-8B) |
  89. **MiniCPM4.1 quantized & speculative variants:**
  90. | HuggingFace | ModelScope |
  91. |---|---|
  92. | [MiniCPM4.1-8B-GPTQ](https://huggingface.co/openbmb/MiniCPM4.1-8B-GPTQ) | [MiniCPM4.1-8B-GPTQ](https://www.modelscope.cn/openbmb/MiniCPM4.1-8B-GPTQ) |
  93. | [MiniCPM4.1-8B-AutoAWQ](https://huggingface.co/openbmb/MiniCPM4.1-8B-AutoAWQ) | [MiniCPM4.1-8B-AutoAWQ](https://www.modelscope.cn/openbmb/MiniCPM4.1-8B-AutoAWQ) |
  94. | [MiniCPM-4.1-8B-Marlin](https://huggingface.co/openbmb/MiniCPM-4.1-8B-Marlin) | [MiniCPM-4.1-8B-Marlin](https://www.modelscope.cn/openbmb/MiniCPM-4.1-8B-Marlin) |
  95. | [MiniCPM4.1-8B-GGUF](https://huggingface.co/openbmb/MiniCPM4.1-8B-GGUF) | [MiniCPM4.1-8B-GGUF](https://www.modelscope.cn/openbmb/MiniCPM4.1-8B-GGUF) |
  96. | [MiniCPM4.1-8B-MLX](https://huggingface.co/openbmb/MiniCPM4.1-8B-MLX) | [MiniCPM4.1-8B-MLX](https://www.modelscope.cn/openbmb/MiniCPM4.1-8B-MLX) |
  97. | [MiniCPM4.1-8B-Eagle3](https://huggingface.co/openbmb/MiniCPM4.1-8B-Eagle3) | [MiniCPM4.1-8B-Eagle3](https://www.modelscope.cn/openbmb/MiniCPM4.1-8B-Eagle3) |
  98. **BitCPM4 ternary-quantized + MiniCPM4 Applications:**
  99. | HuggingFace | ModelScope |
  100. |---|---|
  101. | [BitCPM4-1B](https://huggingface.co/openbmb/BitCPM4-1B) | [BitCPM4-1B](https://www.modelscope.cn/models/OpenBMB/BitCPM4-1B) |
  102. | [BitCPM4-0.5B](https://huggingface.co/openbmb/BitCPM4-0.5B) | [BitCPM4-0.5B](https://www.modelscope.cn/models/OpenBMB/BitCPM4-0.5B) |
  103. | [MiniCPM4-Survey](https://huggingface.co/openbmb/MiniCPM4-Survey) | [MiniCPM4-Survey](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-Survey) |
  104. | [MiniCPM4-MCP](https://huggingface.co/openbmb/MiniCPM4-MCP) | [MiniCPM4-MCP](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-MCP) |
  105. **MiniCPM4 Eagle speculative decoding, QAT, and pre-2025 releases:**
  106. | HuggingFace | ModelScope |
  107. |---|---|
  108. | [MiniCPM4-8B-Eagle-FRSpec](https://huggingface.co/openbmb/MiniCPM4-8B-Eagle-FRSpec) | [MiniCPM4-8B-Eagle-FRSpec](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-8B-Eagle-FRSpec) |
  109. | [MiniCPM4-8B-Eagle-FRSpec-QAT](https://huggingface.co/openbmb/MiniCPM4-8B-Eagle-FRSpec-QAT) | [MiniCPM4-8B-Eagle-FRSpec-QAT](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-8B-Eagle-FRSpec-QAT) |
  110. | [MiniCPM4-8B-Eagle-vLLM](https://huggingface.co/openbmb/MiniCPM4-8B-Eagle-vLLM) | [MiniCPM4-8B-Eagle-vLLM](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-8B-Eagle-vLLM) |
  111. | [MiniCPM4-8B-marlin-Eagle-vLLM](https://huggingface.co/openbmb/MiniCPM4-8B-marlin-Eagle-vLLM) | [MiniCPM4-8B-marlin-Eagle-vLLM](https://www.modelscope.cn/models/OpenBMB/MiniCPM4-8B-marlin-Eagle-vLLM) |
  112. | [MiniCPM4-0.5B-QAT-Int4-unquantized](https://huggingface.co/openbmb/MiniCPM4-0.5B-QAT-Int4-unquantized) | [MiniCPM4-0.5B-QAT-Int4-unquantized](https://modelscope.cn/models/OpenBMB/MiniCPM4-0.5B-QAT-Int4-unquantized) |
  113. | [MiniCPM4-0.5B-QAT-Int4-GPTQ-format](https://huggingface.co/openbmb/MiniCPM4-0.5B-QAT-Int4-GPTQ-format) | [MiniCPM4-0.5B-QAT-Int4-GPTQ-format](https://modelscope.cn/models/OpenBMB/MiniCPM4-0.5B-QAT-Int4-GPTQ-format) |
  114. | [MiniCPM3-4B](https://huggingface.co/openbmb/MiniCPM3-4B) | [MiniCPM3-4B](https://www.modelscope.cn/models/OpenBMB/MiniCPM3-4B) |
  115. | [MiniCPM-2B-sft](https://huggingface.co/openbmb/MiniCPM-2B-sft-bf16) | [MiniCPM-2B-sft](https://modelscope.cn/models/OpenBMB/miniCPM-bf16) |
  116. | [MiniCPM-2B-dpo](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16) | [MiniCPM-2B-dpo](https://modelscope.cn/models/OpenBMB/MiniCPM-2B-dpo-bf16/summary) |
  117. | [MiniCPM-2B-128k](https://huggingface.co/openbmb/MiniCPM-2B-128k) | [MiniCPM-2B-128k](https://modelscope.cn/models/openbmb/MiniCPM-2B-128k/summary) |
  118. | [MiniCPM-MoE-8x2B](https://huggingface.co/openbmb/MiniCPM-MoE-8x2B) | [MiniCPM-MoE-8x2B](https://modelscope.cn/models/OpenBMB/MiniCPM-MoE-8x2B) |
  119. | [MiniCPM-1B](https://huggingface.co/openbmb/MiniCPM-1B-sft-bf16) | [MiniCPM-1B](https://modelscope.cn/models/OpenBMB/MiniCPM-1B-sft-bf16) |
  120. | [MiniCPM-S-1B](https://huggingface.co/openbmb/MiniCPM-S-1B-sft) | [MiniCPM-S-1B](https://modelscope.cn/models/OpenBMB/MiniCPM-S-1B-sft) |
  121. </details>
  122. ## 🚀 MiniCPM5-2B
  123. ### Introduction
  124. MiniCPM5-2B is the second checkpoint in the MiniCPM5 series, scaling the MiniCPM5-1B recipe up to 2B parameters for users who can afford a larger footprint in exchange for stronger capability. It is designed for local assistants, coding agents, tool-use workflows, and reasoning scenarios where a compact model is preferred. The model keeps a small deployment footprint while providing native long-context support.
  125. | | |
  126. |---|---|
  127. | **Architecture** | Standard `LlamaForCausalLM` |
  128. | **Parameters** | 2,516,756,480 (non-embedding: 1,981,982,720) |
  129. | **Layers** | 42 |
  130. | **Attention Heads (GQA)** | 16 Q / 2 KV |
  131. | **Context Length** | 131,072 |
  132. ### Evaluation Results
  133. We compare **MiniCPM5-2B** with strong open-source models in the same size class, including **LFM2.5-2.6B**, **Qwen3.5-2B**, and **Gemma-4-E2B-it**, while also listing larger models such as **Qwen3.5-4B**, **granite-4.2-3B**, **Nemotron-3-Nano-4B**, **Gemma-4-E4B-it**, and **LFM2.5-8B-A1B** for reference.
  134. Within this comparison set, MiniCPM5-2B reaches 2B-class open-source SOTA with an average score of **53.9**, and also exceeds all of the larger models included here (the highest is **51.1**). Its advantages are most visible in code reasoning, math reasoning, long-context understanding, tool use, and multiple agentic tasks.
  135. ![MiniCPM5-2B public leaderboard](./assets/minicpm5/minicpm5_2b_public_leaderboard_en.png)
  136. ### Training Recipe
  137. The training of MiniCPM5-2B is a full-stack practice of **[UltraData Tiered Data Management](https://arxiv.org/pdf/2602.09003)**, covering three stages: base training, mid-training, and post-training.
  138. During **base training**, the model goes through stable training and decay training to build core language capability and training stability. It then enters **mid-training** to further strengthen target capabilities and adapt to the target data distribution. The training corpus is released alongside the model as [Ultra-FineWeb](https://huggingface.co/datasets/openbmb/Ultra-FineWeb), [Ultra-FineWeb-L3](https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L3), [UltraX](https://huggingface.co/datasets/openbmb/UltraX-Preview), [UltraData-Code](https://huggingface.co/datasets/openbmb/UltraData-Code) and [UltraData-Math](https://huggingface.co/datasets/openbmb/UltraData-Math).
  139. During **post-training**, we proceed in three steps: **SFT**, **RL**, and **OPD**. We first use **400B tokens of deep-thinking SFT** to establish deep-thinking and general chat abilities; the SFT data is released as [UltraData-SFT-2605](https://huggingface.co/datasets/openbmb/UltraData-SFT-2605) and Agent SFT data as [UltraData-SFT-Agent-2609](https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609). We then train specialized **RL teachers** for math, code, agentic tasks, writing, and related domains(with the corresponding data also open-sourced as [UltraData-RL-2609](https://huggingface.co/datasets/openbmb/UltraData-RL-2609)), and use **On-Policy Distillation (OPD)** to distill these teachers back into one release model.
  140. ![MiniCPM5-2B Training Recipe](./assets/minicpm5/minicpm5_2b_training_recipe.jpg)
  141. #### What does RL + OPD bring?
  142. **RL + OPD** is a key part of MiniCPM5-2B post-training. During the **RL** stage, we adopted the critic-based algorithm described in [JustRL II](https://panhaoxuan.notion.site/justrl-ii-scaling-small-llms-to-128k-reasoning-with-a-critic), substantially improving training stability and achieving significant gains across multiple domains. On the benchmarks listed below, RL + OPD improves reasoning and general capabilities by an average of **↑10.96 points**, and agentic capabilities by **↑6.96 points**.
  143. **OPD** merges the capabilities of 16 expert models produced by RL training, including 5 agentic expert models. At each response position, we compute the full-vocabulary reverse KL divergence between student and teacher logits as the advantage estimate, replacing the original verification-based advantage. OPD directly reuses the prompts used to train each RL teacher as distillation data, so no additional corpus construction is required.
  144. ![MiniCPM5-2B RL + OPD Gains](./assets/minicpm5/minicpm5_2b_rl_opd_score_gains.png)
  145. ### Quickstart
  146. We recommend using the following sets of sampling parameters for generation: `temperature=1.0, top_p=0.95, min_p=0.0`.
  147. If you encounter repetitive outputs, try: `temperature=1.0, top_p=0.95, min_p=0.0, repetition_penalty=1.05`.
  148. Please note that the support for sampling parameters varies according to inference frameworks.
  149. #### vLLM
  150. ```bash
  151. pip install "vllm>=0.21"
  152. vllm serve openbmb/MiniCPM5-2B --port 8000
  153. ```
  154. ```bash
  155. curl http://localhost:8000/v1/chat/completions \
  156. -H "Content-Type: application/json" \
  157. -d '{
  158. "model": "openbmb/MiniCPM5-2B",
  159. "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
  160. "max_tokens": 128,
  161. "temperature": 1.0, "top_p": 0.95
  162. }'
  163. ```
  164. #### SGLang
  165. ```bash
  166. pip install "sglang[srt]>=0.5.16"
  167. python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000
  168. ```
  169. ```bash
  170. curl http://localhost:30000/v1/chat/completions \
  171. -H "Content-Type: application/json" \
  172. -d '{
  173. "model": "openbmb/MiniCPM5-2B",
  174. "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
  175. "max_tokens": 128,
  176. "temperature": 1.0, "top_p": 0.95
  177. }'
  178. ```
  179. **Speculative decoding (DSpark)**: we also release [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:
  180. ```bash
  181. python -m sglang.launch_server \
  182. --model-path openbmb/MiniCPM5-2B \
  183. --trust-remote-code \
  184. --speculative-algorithm DSPARK \
  185. --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \
  186. --speculative-dspark-block-size 7 \
  187. --port 30000
  188. ```
  189. #### Llama.cpp
  190. ```bash
  191. llama-server -m MiniCPM5-2B-F16.gguf -a MiniCPM5-2B --port 8080 -ngl 99 -c 8192 --jinja
  192. ```
  193. `-c 8192` sets the context length. You can adjust this value as needed.
  194. ```bash
  195. curl http://localhost:8080/v1/chat/completions \
  196. -H "Content-Type: application/json" \
  197. -d '{
  198. "model": "MiniCPM5-2B",
  199. "messages": [{"role": "user", "content": "1+1=?"}],
  200. "temperature": 1.0, "top_p": 0.95, "min_p": 0.0, "max_tokens": 256
  201. }'
  202. ```
  203. In llama.cpp, the default `min_p=0.05` can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set `min_p=0.0`.
  204. #### Transformers
  205. ```bash
  206. pip install -U "transformers>=5.6" accelerate torch
  207. ```
  208. ```python
  209. from transformers import AutoModelForCausalLM, AutoTokenizer
  210. model_id = "openbmb/MiniCPM5-2B"
  211. tokenizer = AutoTokenizer.from_pretrained(model_id)
  212. model = AutoModelForCausalLM.from_pretrained(
  213. model_id,
  214. torch_dtype="auto",
  215. device_map="auto",
  216. )
  217. messages = [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}]
  218. inputs = tokenizer.apply_chat_template(
  219. messages,
  220. tokenize=True,
  221. add_generation_prompt=True,
  222. enable_thinking=True,
  223. return_dict=True,
  224. return_tensors="pt",
  225. ).to(model.device)
  226. outputs = model.generate(**inputs, max_new_tokens=128)
  227. print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
  228. ```
  229. ### Tool Calling
  230. For tool / function calling, **SGLang is the recommended backend**. MiniCPM5-2B emits XML-style tool calls and SGLang's built-in `minicpm5` parser converts them to OpenAI-compatible `tool_calls` natively:
  231. ```bash
  232. python -m sglang.launch_server --model-path openbmb/MiniCPM5-2B --port 30000 \
  233. --tool-call-parser minicpm5 # or: --tool-call-parser auto
  234. ```
  235. ### GitHub Cookbooks and Agent Skills
  236. MiniCPM5-2B uses the **standard `LlamaForCausalLM` architecture**, so mainstream inference engines can load it directly: **no custom kernels, no model-code fork**. For step-by-step deployment and fine-tuning instructions, use the GitHub cookbooks below. Agent Skills are linked as GitHub resources for users working with Cursor / Claude Code style coding agents.
  237. #### Deployment
  238. | Backend | Model format / use case | Cookbook | Agent Skill |
  239. | --- | --- | --- | --- |
  240. | Transformers | BF16 / FP16 local Python inference, GPU + CPU | [transformers.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/transformers.md) | [minicpm5-deploy-transformers](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-transformers/SKILL.md) |
  241. | vLLM | BF16 / FP16 OpenAI server | [vllm.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm.md) | [minicpm5-deploy-vllm](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm/SKILL.md) |
  242. | SGLang | BF16 / FP16 OpenAI server, recommended for tool calling | [sglang.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/sglang.md) | [minicpm5-deploy-sglang](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-sglang/SKILL.md) |
  243. | llama.cpp | GGUF local inference, CPU/GPU | [llama_cpp.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/llama_cpp.md) | [minicpm5-deploy-llama-cpp](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-llama-cpp/SKILL.md) |
  244. | Ollama | GGUF local on-device runtime | [ollama.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/ollama.md) | [minicpm5-deploy-ollama](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-ollama/SKILL.md) |
  245. | LM Studio | GGUF Mac desktop app and OpenAI server | [lmstudio.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/lmstudio.md) | [minicpm5-deploy-lmstudio](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-lmstudio/SKILL.md) |
  246. | MLX | MLX / 4bit local inference on Apple Silicon | [mlx.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/mlx.md) | [minicpm5-deploy-mlx](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-mlx/SKILL.md) |
  247. | ArcLight | GGUF local on-device, CPU, Desktop & Server | [arclight.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/arclight.md) | [minicpm5-deploy-arclight](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-arclight/SKILL.md) |
  248. | LiteRT-LM | `.litertlm` on-device runtime: Android / iOS / desktop / IoT, CPU + GPU | [litert.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/litert.md) | [minicpm5-deploy-litert](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-litert/SKILL.md) |
  249. | vLLM Ascend | BF16 / FP16 OpenAI server | [vllm_ascend.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm_ascend.md) | [minicpm5-deploy-vllm-ascend](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-deploy-vllm-ascend/SKILL.md) |
  250. #### Fine-tuning
  251. | Framework | Use case | Cookbook | Agent Skill |
  252. | --- | --- | --- | --- |
  253. | TRL + PEFT | LoRA / SFT fine-tuning | [trl.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/trl.md) | [minicpm5-finetune-trl](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-trl/SKILL.md) |
  254. | LLaMA-Factory | Fine-tuning | [llamafactory.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/llamafactory.md) | [minicpm5-finetune-llamafactory](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-llamafactory/SKILL.md) |
  255. | ms-swift | Fine-tuning | [ms_swift.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/ms_swift.md) | [minicpm5-finetune-ms-swift](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-ms-swift/SKILL.md) |
  256. | unsloth | Fine-tuning | [unsloth.md](https://github.com/OpenBMB/MiniCPM/blob/main/docs/finetune/unsloth.md) | [minicpm5-finetune-unsloth](https://github.com/OpenBMB/MiniCPM/blob/main/skills/minicpm5-finetune-unsloth/SKILL.md) |
  257. #### Other Supported Frameworks
  258. In addition to the deployment and fine-tuning frameworks listed above, MiniCPM5-2B is also supported by FlagOS for multi-chip deployment.
  259. ##### FlagOS Overview
  260. To enable large-scale deployment across different AI chips, Beijing Zhiyuan Research Institute, together with numerous research institutions, chip manufacturers, system vendors, and algorithm and software organizations both domestically and internationally, jointly initiated and established the FlagOS Open Source Community.
  261. The FlagOS community is dedicated to building a unified, open-source system software stack for various AI chips, encompassing core open-source projects such as a large-scale operator library, a unified AI compiler, parallel training and inference frameworks, and a unified communication library. It aims to create an open technology ecosystem connecting the “model-system-chip” layers. By enabling “develop once, deploy across chips”, FlagOS unlocks the computational potential of hardware, breaks down the ecosystem silos between different chip software stacks, and effectively reduces migration costs for developers.The FlagOS community fosters an AI hardware and software ecosystem, overcomes single-vendor closed-source monopolies, promotes widespread deployment of AI hardware technologies, and is committed to rooted in China while embracing global collaboration.
  262. Official website express: [https://flagos.io](https://flagos.io/)
  263. <details>
  264. <summary>FlagOS multi-chip support and usage</summary>
  265. ##### FlagOS: Supporting Multiple AI Chips
  266. Thanks to FlagOS’s unified multi-chip AI system software stack, MiniCPM5-2B was adapted to 9 different AI chips in an extremely short time. Currently, the multi-chip version of MiniCPM5-2B has been released on FlagRelease, FlagOS’s platform for automatic migration, adaptation, and deployment of large models across multi-architecture AI chips. Details are as follows:
  267. |Vendor|ModelScope|Huggingface|
  268. |---|---|---|
  269. |Nvidia|[MiniCPM5-2B-nvidia-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS)|[MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS)|
  270. |Hygon|[MiniCPM5-2B-hygon-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-hygon-FlagOS)|[MiniCPM5-2B-hygon-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-hygon-FlagOS)|
  271. |Metax|[MiniCPM5-2B-metax-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-metax-FlagOS)|[MiniCPM5-2B-metax-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-metax-FlagOS)|
  272. |Iluvatar|[MiniCPM5-2B-iluvatar-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-iluvatar-FlagOS)|[MiniCPM5-2B-iluvatar-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-iluvatar-FlagOS)|
  273. |Zhenwu|[MiniCPM5-2B-zhenwu-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-zhenwu-FlagOS)|[MiniCPM5-2B-zhenwu-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-zhenwu-FlagOS)|
  274. |Mthreads|[MiniCPM5-2B-mthreads-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-mthreads-FlagOS)|[MiniCPM5-2B-mthreads-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-mthreads-FlagOS)|
  275. |Kunlunxin|[MiniCPM5-2B-kunlunxin-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-kunlunxin-FlagOS)|[MiniCPM5-2B-kunlunxin-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-kunlunxin-FlagOS)|
  276. |Ascend|[MiniCPM5-2B-ascend-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-ascend-FlagOS)|[MiniCPM5-2B-ascend-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-ascend-FlagOS)|
  277. |ARM-v9|[MiniCPM5-2B-Armv9-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-Armv9-FlagOS)|[MiniCPM5-2B-Armv9-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-Armv9-FlagOS)|
  278. ##### FlagOS Usage
  279. ###### FlagOS Performance Acceleration on Nvidia
  280. **From FlagRelease (Recommendation)**
  281. FlagRelease is a platform developed by the FlagOS team for automatic migration, adaptation, and deployment of large models across multi-architecture AI chips. The multi-chip version of MiniCPM5-2B has already been released on FlagRelease. All necessary software packages are pre-installed on the platform, so users do not need to install anything.
  282. **FlagRelease Quick Start**
  283. |Vendor|ModelScope|Huggingface|
  284. |---|---|---|
  285. |Nvidia|[MiniCPM5-2B-nvidia-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS)|[MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS)|
  286. |Hygon|[MiniCPM5-2B-hygon-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-hygon-FlagOS)|[MiniCPM5-2B-hygon-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-hygon-FlagOS)|
  287. |Metax|[MiniCPM5-2B-metax-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-metax-FlagOS)|[MiniCPM5-2B-metax-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-metax-FlagOS)|
  288. |Iluvatar|[MiniCPM5-2B-iluvatar-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-iluvatar-FlagOS)|[MiniCPM5-2B-iluvatar-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-iluvatar-FlagOS)|
  289. |Zhenwu|[MiniCPM5-2B-zhenwu-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-zhenwu-FlagOS)|[MiniCPM5-2B-zhenwu-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-zhenwu-FlagOS)|
  290. |Mthreads|[MiniCPM5-2B-mthreads-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-mthreads-FlagOS)|[MiniCPM5-2B-mthreads-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-mthreads-FlagOS)|
  291. |Kunlunxin|[MiniCPM5-2B-kunlunxin-FlagOS](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-kunlunxin-FlagOS)|[MiniCPM5-2B-kunlunxin-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-kunlunxin-FlagOS)|
  292. |Ascend|[MiniCPM5-2B-ascend-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-ascend-FlagOS)|[MiniCPM5-2B-ascend-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-ascend-FlagOS)|
  293. |ARM-v9|[MiniCPM5-2B-Armv9-FlagOS](https://modelscope.cn/models/FlagRelease/MiniCPM5-2B-Armv9-FlagOS)|[MiniCPM5-2B-Armv9-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-Armv9-FlagOS)|
  294. **From Scratch**
  295. - Dependencies: Python 3.12, GLIBC 2.39, GLIBCXX 3.4.33, CXXABI 1.3.15
  296. **Installing the FlagOS Operator Library**
  297. Official Repository: https://github.com/flagos-ai/FlagGems
  298. ```PowerShell
  299. pip install flag-gems==4.2.1rc0
  300. pip install triton==3.5.1
  301. ```
  302. **Activating Acceleration**
  303. You can enable flagGems acceleration by adding the import of flagGems in the source code of vllm where inference is performed.
  304. ```Bash
  305. import flag_gems
  306. flag_gems.enable(record=True, once=True, path="/root/gems.txt")
  307. ```
  308. ```PowerShell
  309. vllm serve ${model_path} \
  310. --trust-remote-code \
  311. --dtype bfloat16 \
  312. --enforce-eager \
  313. --port ${Port} \
  314. --served-model-name ${model_name} \
  315. --gpu-memory-utilization 0.85
  316. ```
  317. ###### Using FlagOS Unified Multi-Chip Backend Plugin
  318. [**vllm-plugin-FL**](https://github.com/flagos-ai/vllm-plugin-FL) is a plugin built for the vLLM inference/service framework. Developed on top of FlagOS’s unified multi-chip backend, it is designed to extend vLLM’s capabilities and performance across a variety of hardware environments.
  319. **Using vllm-plugin-FL**
  320. |Vendor|From Scratch|From FlagRelease||
  321. |---|---|---|---|
  322. |Nvidia|[vllm-plugin-FL/MiniCPM5-2B](https://github.com/flagos-ai/vllm-plugin-FL/blob/main/examples/minicpm/README.md)|[MiniCPM5-2B-ModelScope](https://www.modelscope.cn/models/FlagRelease/MiniCPM5-2B-nvidia-FlagOS)|[MiniCPM5-2B-nvidia-FlagOS](https://huggingface.co/FlagRelease/MiniCPM5-2B-nvidia-FlagOS)|
  323. </details>
  324. ## 🚀 MiniCPM5-1B
  325. > **MiniCPM5-1B** is the first model in the **MiniCPM5** series. It is a dense 1B Transformer built for on-device, local deployment, and resource-constrained scenarios, reaching 1B-class open-source SOTA.
  326. <details>
  327. <summary>Click to expand: highlights, introduction, evaluation, inference setup</summary>
  328. ### Highlights
  329. 🏆 **1B-class open-source SOTA**: compared with strong open-source models in the same size class, MiniCPM5-1B reaches SOTA within this comparison set. Its advantage is most visible in agentic tool use, code generation, and difficult reasoning.
  330. ![MiniCPM5-1B capability comparison by domain](./assets/minicpm5/public_leaderboard_radar_en.png)
  331. 🧠 **Hybrid Reasoning**: built-in `<think>` chat template, switch via `enable_thinking`. The same checkpoint serves as both a fast assistant and a deliberate reasoner.
  332. 🛠️ **Deployment / Fine-tuning Resources**: the MiniCPM GitHub repo provides single-page cookbooks and Agent Skills for major inference backends and fine-tuning frameworks.
  333. 🐱 **Desktop Pet**: a local-LLM desktop pet driven by MiniCPM5-1B.
  334. ### Introduction
  335. MiniCPM5-1B is the first checkpoint in the MiniCPM5 series. It is designed for local assistants, coding agents, tool-use workflows, and reasoning scenarios where a compact model is preferred. The model keeps a small deployment footprint while providing native long-context support and both Think / No Think chat modes through the same checkpoint.
  336. ### Evaluation Results
  337. We compare MiniCPM5-1B with strong open-source models in the same size class, including **LFM2.5-1.2B-Thinking**, **Qwen3-0.6B/think** and **Qwen3.5-0.8B/think**. These are capable baselines; within this comparison set, MiniCPM5-1B reaches 1B-class open-source SOTA, with its advantage most visible in tool use, code generation, and difficult reasoning. This makes it a practical choice for local coding agents, tool assistants, and reasoning assistants.
  338. ![MiniCPM-5 1B Public Leaderboard](./assets/minicpm5/public_leaderboard_en.png)
  339. ### Training Recipe
  340. The training of MiniCPM5-1B is a full-stack practice of **[UltraData Tiered Data Management](https://arxiv.org/pdf/2602.09003)**, covering three stages: base training, mid-training, and post-training.
  341. During **base training**, the model goes through stable training and decay training to build core language capability and training stability. It then enters **mid-training** to further strengthen target capabilities and adapt to the target data distribution. The training corpus is released alongside the model as [Ultra-FineWeb](https://huggingface.co/datasets/openbmb/Ultra-FineWeb), [Ultra-FineWeb-L3](https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L3), and [UltraData-Math](https://huggingface.co/datasets/openbmb/UltraData-Math).
  342. During **post-training**, we proceed in three steps: **SFT**, **RL**, and **OPD**. We first use **200B tokens of deep-thinking SFT** and **200B tokens of hybrid-thinking SFT** to establish deep-thinking, hybrid-thinking, and general chat abilities; the SFT data is released as [UltraData-SFT-2605](https://huggingface.co/datasets/openbmb/UltraData-SFT-2605). We then train specialized **RL teachers** for math, code, closed-book QA, writing, and related domains, and use **On-Policy Distillation (OPD)** to distill these teachers back into one release model.
  343. ![MiniCPM5-1B Training Recipe](./assets/minicpm5/training_recipe.png)
  344. #### What does RL + OPD bring?
  345. **RL + OPD** is a key part of MiniCPM5-1B post-training. On math, code and instruction-following tasks, RL + OPD raises the average score by **↑16 points** while cutting the share of responses that hit the max-tokens budget by **↓29 percentage points**. The figures below show the two-stage Reasoning RL pipeline, score gains, and the drop in overlong responses.
  346. **RL** combines complementary training signals for reasoning, closed-book QA, writing, instruction following, long-context understanding, and general dialogue. Reasoning RL is based on [DAPO-Math-17k](https://huggingface.co/datasets/BytedTsinghua-SIA/DAPO-Math-17k) (inspired by [JustRL](https://arxiv.org/pdf/2512.16649)'s minimalist recipe) and uses a two-stage length schedule to reduce overlong responses while improving reasoning accuracy. We also use [TriviaQA](https://huggingface.co/datasets/mandarjoshi/trivia_qa), [NQ-Open](https://huggingface.co/datasets/google-research-datasets/nq_open), [LongWriter-Zero-RLData](https://huggingface.co/datasets/THU-KEG/LongWriter-Zero-RLData), synthesized verifiable RLVR data, and pair-wise RLHF signals to improve reliability, instruction following, and user experience.
  347. ![MiniCPM5-1B RL Two-stage Pipeline](./assets/minicpm5/rl_two_stage_overview.png)
  348. **OPD** builds on Thinking Machines Lab's [On-Policy Distillation](https://thinkingmachines.ai/blog/on-policy-distillation/) and incorporates implementation improvements from [Rethinking On-Policy Distillation](https://arxiv.org/pdf/2604.13016). In the RL framework, we use reverse KL divergence as the advantage estimate, replacing the original verification-based advantage. At each response position, we take top-k logits from both the student and teacher models, compute reverse KL on the union of the two token sets, and balance the accuracy of the RKL signal with training efficiency. OPD reuses the in-domain prompts used to train each RL teacher as distillation data, so no additional data curation is required.
  349. ![MiniCPM5-1B RL + OPD Gains](./assets/minicpm5/rl_gains.png)
  350. ![MiniCPM5-1B RL + OPD Overlong Response Rate Drop](./assets/minicpm5/rl_overlong.png)
  351. ### Quickstart
  352. For the three most common inference paths, you can start a service or run local inference as follows:
  353. **vLLM** (OpenAI-compatible server, NVIDIA GPU):
  354. ```bash
  355. pip install "vllm>=0.21" && vllm serve openbmb/MiniCPM5-1B --port 8000
  356. ```
  357. Test request:
  358. ```bash
  359. curl http://localhost:8000/v1/chat/completions \
  360. -H "Content-Type: application/json" \
  361. -d '{
  362. "model": "openbmb/MiniCPM5-1B",
  363. "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
  364. "max_tokens": 128,
  365. "temperature": 0.7
  366. }'
  367. ```
  368. **SGLang** (OpenAI-compatible server, NVIDIA GPU):
  369. ```bash
  370. pip install "sglang[srt]>=0.5.12" && python -m sglang.launch_server --model-path openbmb/MiniCPM5-1B --port 30000
  371. ```
  372. Test request:
  373. ```bash
  374. curl http://localhost:30000/v1/chat/completions \
  375. -H "Content-Type: application/json" \
  376. -d '{
  377. "model": "openbmb/MiniCPM5-1B",
  378. "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
  379. "max_tokens": 128,
  380. "temperature": 0.7
  381. }'
  382. ```
  383. **Transformers** (local Python inference, CPU or GPU):
  384. ```bash
  385. pip install -U "transformers>=5.6" accelerate torch
  386. ```
  387. Local inference test:
  388. ```python
  389. from transformers import AutoModelForCausalLM, AutoTokenizer
  390. model_id = "openbmb/MiniCPM5-1B"
  391. tokenizer = AutoTokenizer.from_pretrained(model_id)
  392. model = AutoModelForCausalLM.from_pretrained(
  393. model_id,
  394. torch_dtype="auto",
  395. device_map="auto",
  396. )
  397. messages = [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}]
  398. inputs = tokenizer.apply_chat_template(
  399. messages,
  400. tokenize=True,
  401. add_generation_prompt=True,
  402. enable_thinking=False,
  403. return_dict=True,
  404. return_tensors="pt",
  405. ).to(model.device)
  406. outputs = model.generate(**inputs, max_new_tokens=128)
  407. print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
  408. ```
  409. Recommended chat template sampling:
  410. | Mode | Recommended params | Enable |
  411. | --- | --- | --- |
  412. | **Think** | `temperature=0.9, top_p=0.95` | `enable_thinking=True` |
  413. | **No Think** | `temperature=0.7, top_p=0.95` | `enable_thinking=False` |
  414. For other backends (llama.cpp / Ollama / LM Studio / MLX), see the cookbooks table below.
  415. #### With tool calling
  416. For tool / function calling, **SGLang is the recommended backend** — MiniCPM5-1B emits XML-style tool calls and SGLang's built-in `minicpm5` parser converts them to OpenAI-compatible `tool_calls` natively:
  417. ```bash
  418. python -m sglang.launch_server --model-path openbmb/MiniCPM5-1B --port 30000 \
  419. --tool-call-parser minicpm5 # or: --tool-call-parser auto
  420. ```
  421. ### Deployment and Fine-tuning Cookbooks and Agent Skills
  422. MiniCPM5-1B uses the **standard `LlamaForCausalLM` architecture**, so mainstream inference engines can load it directly: **no custom kernels, no model-code fork**. To help developers reproduce deployment and fine-tuning workflows, we provide single-page cookbooks paired with [Cursor Agent Skills](https://docs.cursor.com/agent/skills): the cookbooks are for manual execution, while Agent Skills let Cursor / Claude Code agents choose the route based on the target backend, hardware, and data path.
  423. The two top-level skills cover deployment and fine-tuning:
  424. | Top-level skill | What it does | Routes to |
  425. | --- | --- | --- |
  426. | **[`minicpm5-deploy`](./skills/minicpm5-deploy/SKILL.md)** | Inference router | `transformers` · `vllm` · `sglang` · `llama-cpp` · `ollama` · `lmstudio` · `mlx` · `litert` |
  427. | **[`minicpm5-finetune`](./skills/minicpm5-finetune/SKILL.md)** | Fine tuning router | `trl` · `llamafactory` · `ms-swift` · `unsloth` · `xtuner` |
  428. In Cursor / Claude Code, you can call them like this: the agent reads the top-level skill, selects the matching sub-skill and cookbook based on the target backend, hardware, and data path, then runs the command and reports back.
  429. ```
  430. @minicpm5-deploy serve openbmb/MiniCPM5-1B with vLLM on port 8000
  431. @minicpm5-finetune use unsloth + LoRA on /data/my_chat.jsonl, write to ./out
  432. ```
  433. The tables below list the cookbook and sub-skill for each inference backend and fine-tuning framework. Quantized models are not listed as standalone backends; they are described under the inference backend that can load each format.
  434. **Inference Deployment** (8 backends)
  435. | Backend | Model format / use case | Cookbook | Paired Agent Skill |
  436. | --- | --- | --- | --- |
  437. | [Transformers](https://github.com/huggingface/transformers) | BF16 / FP16 local Python inference, GPU + CPU | [`docs/deployment/transformers.md`](./docs/deployment/transformers.md) | [`minicpm5-deploy-transformers`](./skills/minicpm5-deploy-transformers/SKILL.md) |
  438. | [SGLang](https://github.com/sgl-project/sglang) | BF16 / FP16 OpenAI server, recommended for tool calling | [`docs/deployment/sglang.md`](./docs/deployment/sglang.md) | [`minicpm5-deploy-sglang`](./skills/minicpm5-deploy-sglang/SKILL.md) |
  439. | [llama.cpp](https://github.com/ggml-org/llama.cpp) | GGUF local inference, CPU/GPU | [`docs/deployment/llama_cpp.md`](./docs/deployment/llama_cpp.md) | [`minicpm5-deploy-llama-cpp`](./skills/minicpm5-deploy-llama-cpp/SKILL.md) |
  440. | [Ollama](https://github.com/ollama/ollama) | GGUF local on-device runtime | [`docs/deployment/ollama.md`](./docs/deployment/ollama.md) | [`minicpm5-deploy-ollama`](./skills/minicpm5-deploy-ollama/SKILL.md) |
  441. | [LM Studio](https://lmstudio.ai) | GGUF Mac desktop app and OpenAI server | [`docs/deployment/lmstudio.md`](./docs/deployment/lmstudio.md) | [`minicpm5-deploy-lmstudio`](./skills/minicpm5-deploy-lmstudio/SKILL.md) |
  442. | [MLX](https://github.com/ml-explore/mlx-lm) | MLX / 4bit local inference on Apple Silicon | [`docs/deployment/mlx.md`](./docs/deployment/mlx.md) | [`minicpm5-deploy-mlx`](./skills/minicpm5-deploy-mlx/SKILL.md) |
  443. | [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) | `.litertlm` on-device runtime: Android / iOS / desktop / IoT, CPU + GPU | [`docs/deployment/litert.md`](./docs/deployment/litert.md) | [`minicpm5-deploy-litert`](./skills/minicpm5-deploy-litert/SKILL.md) |
  444. | [ArcLight](https://github.com/OpenBMB/ArcLight) | GGUF local on-device, CPU, Desktop & Server | [`docs/deployment/arclight.md`](./docs/deployment/arclight.md) | [`minicpm5-deploy-arclight`](./skills/minicpm5-deploy-arclight/SKILL.md) |
  445. **Fine tuning** (5 frameworks)
  446. | Framework | Cookbook | Paired Agent Skill |
  447. | --- | --- | --- |
  448. | [TRL](https://github.com/huggingface/trl) + [PEFT](https://github.com/huggingface/peft) | [`docs/finetune/trl.md`](./docs/finetune/trl.md) | [`minicpm5-finetune-trl`](./skills/minicpm5-finetune-trl/SKILL.md) |
  449. | [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | [`docs/finetune/llamafactory.md`](./docs/finetune/llamafactory.md) | [`minicpm5-finetune-llamafactory`](./skills/minicpm5-finetune-llamafactory/SKILL.md) |
  450. | [ms-swift](https://github.com/modelscope/ms-swift) | [`docs/finetune/ms_swift.md`](./docs/finetune/ms_swift.md) | [`minicpm5-finetune-ms-swift`](./skills/minicpm5-finetune-ms-swift/SKILL.md) |
  451. | [unsloth](https://github.com/unslothai/unsloth) | [`docs/finetune/unsloth.md`](./docs/finetune/unsloth.md) | [`minicpm5-finetune-unsloth`](./skills/minicpm5-finetune-unsloth/SKILL.md) |
  452. | [xtuner](https://github.com/InternLM/xtuner) | [`docs/finetune/xtuner.md`](./docs/finetune/xtuner.md) | [`minicpm5-finetune-xtuner`](./skills/minicpm5-finetune-xtuner/SKILL.md) |
  453. ### Other Supported Frameworks
  454. In addition to the deployment and fine-tuning frameworks listed above, MiniCPM5-1B is also supported by FlagOS for multi-chip deployment.
  455. ### Desktop Pet
  456. We also ship **[OpenBMB/MiniCPM-Desk-Pet](https://github.com/OpenBMB/MiniCPM-Desk-Pet)**, a desktop pet driven locally by MiniCPM5-1B. It uses a thin `llama.cpp` `llama-server` sidecar to load the GGUF model and serves an OpenAI-compatible local endpoint to an Electron pet UI.
  457. <a href="https://youtu.be/Ee0slMW8SEk"><img src="https://img.youtube.com/vi/Ee0slMW8SEk/0.jpg" alt="MiniCPM Desk Pet video demo" width="720"></a>
  458. The pet supports Apple Silicon / NVIDIA GPU / CPU paths, can work with coding agents such as Cursor, Claude Code, and Codex, and supports LoRA persona switching.
  459. - **User install**: grab `Clawd-on-Desk-*-arm64.dmg` from [Releases](https://github.com/OpenBMB/MiniCPM-Desk-Pet/releases), then follow the onboarding flow for environment checks, model download, and sidecar startup.
  460. - **Developer run**: `git clone git@github.com:OpenBMB/MiniCPM-Desk-Pet.git && ./go.sh` — see [`MiniCPM-Desk-Pet/README.md`](https://github.com/OpenBMB/MiniCPM-Desk-Pet#给开发者) for the full setup.
  461. > The pet UI layer is forked from [@rullerzhou-afk/clawd-on-desk](https://github.com/rullerzhou-afk/clawd-on-desk) (AGPL-3.0). On top of the upstream pet runtime, animation packs, and multi-agent integrations, we add the local MiniCPM5-1B sidecar, onboarding flow, and LoRA persona switching. Full attribution in [`NOTICE.md`](https://github.com/OpenBMB/MiniCPM-Desk-Pet/blob/main/NOTICE.md).
  462. </details>
  463. ## 🧪 MiniCPM-SALA
  464. > First large-scale **Sparse + Linear Attention hybrid** for million-token context (2026-02). [HuggingFace](https://huggingface.co/openbmb/MiniCPM-SALA) · [ModelScope](https://www.modelscope.cn/models/OpenBMB/MiniCPM-SALA)
  465. <details>
  466. <summary>Click to expand: highlights, evaluation, inference setup</summary>
  467. #### Highlights
  468. MiniCPM-SALA (Sparse Attention and Linear Attention) is the first large-scale hybrid model effectively integrating sparse and linear attention for million-token context modeling
  469. ✅ Hybrid Architecture: combines 25% Sparse Attention (InfLLM-v2) for long-context modeling with 75% Linear Attention (Lightning Attention) for global efficiency.
  470. ✅ Inference Efficiency: achieves 3.5× inference speed and lower KV-cache overhead compared with dense baselines.
  471. ✅ Million-Token Context: uses HyPE (Hybrid Positional Embedding) to scale to 1M+ tokens while maintaining length generalization.
  472. ✅ HALO Adaptation: uses Hybrid Attention via Layer Optimization (HALO), a distillation recipe that transfers dense attention capabilities to the hybrid architecture and mitigates the degradation often seen in pure linear models.
  473. #### Introduction
  474. MiniCPM-SALA is an efficient hybrid model in which 25% of the layers adopt [InfLLM-V2](https://arxiv.org/abs/2509.24663) and the remaining 75% utilize Lightning Attention. This architecture enables inference of one million tokens on consumer GPUs such as the NVIDIA RTX 5090.
  475. - **SALA Hybrid Attention Mechanism**
  476. - Integrates 25% InfLLM-V2 and 75% Lightning Attention, effectively leveraging the granular focus of sparse attention for local details and the high efficiency of linear attention for broad context.
  477. - **Transformer-to-Hybrid Continue Training**
  478. - Circumvents the inefficiencies of cold-start training by performing an architectural transformation on the pre-trained weights, thereby reducing the total training budget to approximately 25% relative to training a comparable model from scratch.
  479. - **[HyPE](https://arxiv.org/abs/2601.22156) (Hybrid Positional Encoding)**
  480. - Balances short-context and long-context performance, maintaining general capabilities (e.g., knowledge, mathematics, and coding) close to full-attention models such as Qwen3-8B while scoring higher on multiple long-context benchmarks.
  481. - **Efficient Inference on Long Sequences**
  482. - Achieves up to 3.5x the inference speed of Qwen3-8B at a sequence length of 256K tokens on A6000D, supports inference at context lengths of up to 1M tokens on both NVIDIA A6000D and 5090 GPUs, whereas Qwen3-8B fails at this length due to out-of-memory (OOM) errors.
  483. ### Evaluation Results
  484. #### Efficiency Evaluation
  485. We benchmarked MiniCPM-SALA (9B) against Qwen3-8B on NVIDIA A6000D and RTX 5090 GPUs to evaluate inference speed and memory efficiency. MiniCPM-SALA achieves up to a 2.5x speedup in time-to-first-token (TTFT) and reduces the memory pressure of full-attention architectures at ultra-long lengths. In this setup, Qwen3-8B runs into OOM errors at extended lengths, while MiniCPM-SALA can process 1M-token contexts on a single consumer-grade RTX 5090.
  486. ![inference_speed_a6000d](./assets/minicpm_sala/inference_speed_a600d.png)
  487. ![inference_speed_5090](./assets/minicpm_sala/inference_speed_5090.png)
  488. #### Long-Context Evaluation
  489. MiniCPM-SALA scores higher than the tested open-source LLMs of similar scale on most long-context benchmarks. It achieves the highest scores in the RULER and NoLiMa tests at all context lengths up to 128K, with an overall average score of 38.97.
  490. ![long_text_evaluation](./assets/minicpm_sala/long_text_evaluation.png)
  491. #### Ultra-long Context Evaluation
  492. MiniCPM-SALA shows effective length extrapolation, maintaining a score of 81.6 at a 2048K context length despite being trained on up to 520K tokens. The model does this without auxiliary techniques like YaRN, likely due to its NoPE configuration in sparse attention layers.
  493. ![ultra_long_text_evaluation](./assets/minicpm_sala/ultra_long_text_evaluation.png)
  494. #### Standard Evaluation
  495. MiniCPM-SALA achieves an average score of 76.53 across standard benchmarks, outperforming comparable models such as Qwen3-8B and Falcon-H1R-7B. The architecture maintains robust performance in Knowledge, Code, and Math.
  496. ![benchmark](./assets/minicpm_sala/benchmark.png)
  497. ### Inference
  498. Recommended inference setting: `Temperature=0.9`.
  499. #### HuggingFace
  500. Our model is readily compatible with 🤗 Hugging Face transformers. You can perform inference with our model as follows:
  501. ```python
  502. import torch
  503. from transformers import AutoModelForCausalLM, AutoTokenizer
  504. model_path = "openbmb/MiniCPM-SALA"
  505. tokenizer = AutoTokenizer.from_pretrained(model_path)
  506. model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, device_map="auto")
  507. model.eval()
  508. prompts = ["My name is", "The capital of China is"]
  509. with torch.no_grad():
  510. inputs = tokenizer(prompts, return_tensors="pt").to(model.device)
  511. outputs = model.generate(**inputs)
  512. output_texts = tokenizer.batch_decode(outputs)
  513. print(output_texts)
  514. ```
  515. #### SGLang
  516. ##### Requirements
  517. - CUDA 12.x or higher
  518. - `gcc` / `g++` compiler
  519. - `uv` package manager (script will check)
  520. ##### Installation
  521. ```bash
  522. # Clone repository
  523. git clone -b minicpm_sala https://github.com/OpenBMB/sglang.git
  524. cd sglang
  525. # One-click installation (creates venv and compiles all dependencies)
  526. bash install_minicpm_sala.sh
  527. # Or specify PyPI mirror
  528. bash install_minicpm_sala.sh https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
  529. ```
  530. The installation script performs the following steps:
  531. 1. Creates `sglang_minicpm_sala_env` virtual environment (Python 3.12)
  532. 2. Clones dependencies to `3rdparty/` (infllmv2) and initializes submodules (sparse_kernel)
  533. 3. Installs MiniCPM-SALA (current repo)
  534. 4. Compiles and installs `infllmv2_cuda_impl`
  535. 5. Compiles and installs `sparse_kernel`
  536. 6. Installs `tilelang` & `flash-linear-attention`
  537. ##### Usage
  538. ```bash
  539. # Activate environment
  540. source sglang_minicpm_sala_env/bin/activate
  541. # Launch Inference Server (Replace MODEL_PATH with actual path)
  542. MODEL_PATH=/path/to/your/MiniCPM-SALA
  543. python3 -m sglang.launch_server \
  544. --model ${MODEL_PATH} \
  545. --trust-remote-code \
  546. --disable-radix-cache \
  547. --attention-backend minicpm_flashinfer \
  548. --chunked-prefill-size 8192 \
  549. --max-running-requests 32 \
  550. --skip-server-warmup \
  551. --port 31111 \
  552. --dense-as-sparse
  553. ```
  554. | Parameter | Description |
  555. |-----------|-------------|
  556. | `--trust-remote-code` | Allow custom code in model |
  557. | `--disable-radix-cache` | Disable RadixAttention prefix cache |
  558. | `--attention-backend minicpm_flashinfer` | Use MiniCPM FlashInfer backend |
  559. | `--chunked-prefill-size 8192` | Chunked prefill size |
  560. | `--max-running-requests 32` | Max concurrent requests |
  561. | `--skip-server-warmup` | Skip server warmup |
  562. | `--port 31111` | Server port |
  563. | `--dense-as-sparse` | Use dense-as-sparse mode |
  564. ##### Manual Installation
  565. If the script doesn't work for you, follow these steps:
  566. ```bash
  567. # 0. Ensure uv is installed
  568. pip install uv
  569. # 1. Create venv
  570. uv venv --python 3.12 sglang_minicpm_sala_env
  571. source sglang_minicpm_sala_env/bin/activate
  572. # 2. Install SGLang
  573. uv pip install --upgrade pip setuptools wheel
  574. uv pip install -e ./python[all]
  575. # 3. Compile CUDA Extensions
  576. # (Ensure dependencies are cloned to 3rdparty/)
  577. cd 3rdparty/infllmv2_cuda_impl && python setup.py install && cd ../..
  578. cd 3rdparty/sparse_kernel && python setup.py install && cd ../..
  579. # 4. Install extra deps
  580. uv pip install tilelang flash-linear-attention
  581. ```
  582. ##### Q&A
  583. **Q: CUDA extension compilation failed?**
  584. - Ensure CUDA 12+ is installed (`nvcc --version`).
  585. - Ensure `gcc` / `g++` are available.
  586. - If `CXX` is set to `clang++ -pthread`, manually `export CXX=g++`.
  587. </details>
  588. ## ⚡ MiniCPM4 and MiniCPM4.1 Series
  589. > 8B-scale **trainable sparse attention** with hybrid reasoning (2025-09 / 2025-06). [MiniCPM4.1-8B](https://huggingface.co/openbmb/MiniCPM4.1-8B) · [MiniCPM4-8B](https://huggingface.co/openbmb/MiniCPM4-8B) · [ModelScope](https://www.modelscope.cn/models/OpenBMB/MiniCPM4.1-8B)
  590. <details>
  591. <summary>Click to expand: highlights, evaluation, inference (HF / vLLM / SGLang / CPM.cu / llama.cpp / Ollama)</summary>
  592. <div align="center">
  593. <a href="https://www.youtube.com/watch?v=VouXjLHKDUY"><img src="https://img.youtube.com/vi/VouXjLHKDUY/0.jpg", width=70%></a>
  594. </div>
  595. #### Highlights
  596. MiniCPM 4.1-8B is the first open-source reasoning LLM with trainable sparse attention:
  597. ✅ Strong Reasoning Capability: Surpasses similar-sized models on 15 tasks!
  598. ✅ Fast Generation: 3x decoding speedup for reasoning
  599. ✅ Efficient Architecture: Trainable sparse attention, frequency-ranked speculative decoding
  600. #### Introduction
  601. MiniCPM4 and MiniCPM4.1 series are large language models designed for end-side devices, with efficiency optimizations across model architecture, training data, training algorithms, and inference systems.
  602. - 🏗️ **Efficient Model Architecture:**
  603. - InfLLM-V2 -- Trainable Sparse Attention Mechanism: Adopts a trainable sparse attention mechanism architecture where each token only needs to compute relevance with less than 5% of tokens in 128K long text processing, significantly reducing computational overhead for long texts ([InfLLM-V2 Training Kernels](https://github.com/OpenBMB/infllmv2_cuda_impl))
  604. - 🧠 **Efficient Learning Algorithms:**
  605. - Model Wind Tunnel 2.0 -- Efficient Predictable Scaling: Introduces scaling prediction methods for performance of downstream tasks, enabling more precise model training configuration search
  606. - BitCPM -- Ternary Quantization: Compresses model parameter bit-width to 3 values, achieving 90% model bit-width reduction
  607. - Efficient Training Engineering Optimization: Adopts FP8 low-precision computing technology combined with Multi-token Prediction training strategy
  608. - 📚 **High-Quality Training Data:**
  609. - UltraClean -- High-quality Pre-training Data Filtering and Generation: Builds iterative data cleaning strategies based on efficient data verification, open-sourcing high-quality Chinese and English pre-training dataset [UltraFinweb](https://huggingface.co/datasets/openbmb/Ultra-FineWeb)
  610. - UltraChat v2 -- High-quality Supervised Fine-tuning Data Generation: Constructs large-scale high-quality supervised fine-tuning datasets covering multiple dimensions including knowledge-intensive data, reasoning-intensive data, instruction-following data, long text understanding data, and tool calling data
  611. - ⚡ **Efficient Inference and Deployment System:**
  612. - CPM.cu -- Lightweight and Efficient CUDA Inference Framework: Integrates sparse attention, model quantization, and speculative sampling to achieve efficient prefilling and decoding ([Inference Kernels and Framework](https://github.com/openbmb/cpm.cu))
  613. - ArkInfer -- Cross-platform Deployment System: Supports efficient deployment across multiple backend environments, providing flexible cross-platform adaptation capabilities
  614. ### Evaluation Results
  615. #### Efficiency Evaluation
  616. On two typical end-side chips, Jetson AGX Orin and RTX 4090, MiniCPM4 and MiniCPM4.1 show faster processing speed than similar-size models in long-text processing tasks. As text length increases, the speed gains become more pronounced. On Jetson AGX Orin, compared with Qwen3-8B, MiniCPM4 and MiniCPM4.1 achieve approximately 7x decoding speed improvement.
  617. ![benchmark](./assets/minicpm4/efficiency.png)
  618. MiniCPM4.1 achieves 3x decoding speed improvement in reasoning.
  619. ![benchmark](./assets/minicpm4/minicpm4.1_speed.png)
  620. #### Comprehensive Evaluation
  621. MiniCPM4 launches end-side versions with 8B and 0.5B parameter scales, both showing competitive performance in their respective categories.
  622. ![benchmark](./assets/minicpm4/benchmark.png)
  623. MiniCPM4.1 launches an 8B end-side version with competitive performance in deep reasoning mode.
  624. ![benchmark](./assets/minicpm4/benchmark4.1.png)
  625. #### Long Text Evaluation
  626. MiniCPM4 is pre-trained on 32K long texts and achieves length extension through YaRN. In the 128K needle-in-a-haystack task, MiniCPM4 maintains stable performance. MiniCPM4.1 is pre-trained on 64K long texts and also uses YaRN for length extension, with stable performance on the 128K needle-in-a-haystack task.
  627. ![long-niah](./assets/minicpm4/128k-niah.png)
  628. ### Inference
  629. MiniCPM 4.1 can be used with the following frameworks: Huggingface Transformers, SGLang, vLLM, and CPM.cu. For inference efficiency, CPM.cu is a good first option.
  630. MiniCPM4/MiniCPM4.1 supports both dense attention inference and sparse attention inference modes, where vLLM and SGLang currently only support dense inference mode. If you want to use sparse inference mode, please use Huggingface Transformers and CPM.cu.
  631. - Dense attention inference: vLLM, SGLang, Huggingface Transformers
  632. - Sparse attention inference: Huggingface Transformers, CPM.cu
  633. #### Hybrid Reasoning Mode
  634. MiniCPM4.1 supports hybrid reasoning mode, which can be used in both deep reasoning mode and non-reasoning mode. To enable hybrid reasoning mode. User can set `enable_thinking=True` in `tokenizer.apply_chat_template` to enable hybrid reasoning mode, and set `enable_thinking=False` to enable non-reasoning mode. Similarly, user can directly add `/no_think` at the end of the query to enable non-reasoning mode. If not add any special token or add `/think` at the end of the query, the model will enable reasoning mode.
  635. ```python
  636. # Enable reasoning mode
  637. prompt_text = tokenizer.apply_chat_template(
  638. messages,
  639. tokenize=False,
  640. add_generation_prompt=True,
  641. enable_thinking=True
  642. )
  643. # Enable non-reasoning mode
  644. prompt_text = tokenizer.apply_chat_template(
  645. messages,
  646. tokenize=False,
  647. add_generation_prompt=True,
  648. enable_thinking=False
  649. )
  650. ```
  651. #### HuggingFace
  652. - **Inference with Dense Attention**
  653. ```python
  654. from transformers import AutoModelForCausalLM, AutoTokenizer
  655. import torch
  656. torch.manual_seed(0)
  657. path = 'openbmb/MiniCPM4.1-8B'
  658. device = "cuda"
  659. tokenizer = AutoTokenizer.from_pretrained(path)
  660. model = AutoModelForCausalLM.from_pretrained(path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)
  661. # User can directly use the chat interface
  662. # responds, history = model.chat(tokenizer, "Write an article about Artificial Intelligence.", temperature=0.7, top_p=0.7)
  663. # print(responds)
  664. # User can also use the generate interface
  665. messages = [
  666. {"role": "user", "content": "Write an article about Artificial Intelligence."},
  667. ]
  668. prompt_text = tokenizer.apply_chat_template(
  669. messages,
  670. tokenize=False,
  671. add_generation_prompt=True,
  672. )
  673. model_inputs = tokenizer([prompt_text], return_tensors="pt").to(device)
  674. model_outputs = model.generate(
  675. **model_inputs,
  676. max_new_tokens=32768,
  677. top_p=0.95,
  678. temperature=0.6
  679. )
  680. output_token_ids = [
  681. model_outputs[i][len(model_inputs[i]):] for i in range(len(model_inputs['input_ids']))
  682. ]
  683. responses = tokenizer.batch_decode(output_token_ids, skip_special_tokens=True)[0]
  684. print(responses)
  685. ```
  686. - **Inference with Sparse Attention**
  687. This model supports InfLLM v2, a sparse attention mechanism designed for efficient long-sequence inference. It requires the [infllmv2_cuda_impl](https://github.com/OpenBMB/infllmv2_cuda_impl) library.
  688. You can install it by running the following command:
  689. ```bash
  690. git clone -b feature_infer https://github.com/OpenBMB/infllmv2_cuda_impl.git
  691. cd infllmv2_cuda_impl
  692. git submodule update --init --recursive
  693. pip install -e . # or python setup.py install
  694. ```
  695. To enable InfLLM v2, you need to add the `sparse_config` field in `config.json`:
  696. ```json
  697. {
  698. ...,
  699. "sparse_config": {
  700. "kernel_size": 32,
  701. "kernel_stride": 16,
  702. "init_blocks": 1,
  703. "block_size": 64,
  704. "window_size": 2048,
  705. "topk": 64,
  706. "use_nope": false,
  707. "dense_len": 8192
  708. }
  709. }
  710. ```
  711. These parameters control the behavior of InfLLM v2:
  712. * `kernel_size` (default: 32): The size of semantic kernels.
  713. * `kernel_stride` (default: 16): The stride between adjacent kernels.
  714. * `init_blocks` (default: 1): The number of initial blocks that every query token attends to. This ensures attention to the beginning of the sequence.
  715. * `block_size` (default: 64): The block size for key-value blocks.
  716. * `window_size` (default: 2048): The size of the local sliding window.
  717. * `topk` (default: 64): The specifies that each token computes attention with only the top-k most relevant key-value blocks.
  718. * `use_nope` (default: false): Whether to use the NOPE technique in block selection for improved performance.
  719. * `dense_len` (default: 8192): Since Sparse Attention offers limited benefits for short sequences, the model can use standard (dense) attention for shorter texts. The model will use dense attention for sequences with a token length below `dense_len` and switch to sparse attention for sequences exceeding this length. Set this to `-1` to always use sparse attention regardless of sequence length.
  720. - **Long Context Extension**
  721. MiniCPM4.1 natively supports context lengths of up to 65,536(64k) tokens. For conversations where the total length (including both input and output) significantly exceeds this limit, we recommend using RoPE scaling techniques for effective handling of long texts. By modifying the LongRoPE factor, the model can stably handle context lengths of up to 131,072 tokens.
  722. You can apply the LongRoPE factor modification by modifying the model files. Specifically, in the `config.json` file, adjust the `rope_scaling` fields.
  723. ```json
  724. {
  725. ...,
  726. "rope_scaling": {
  727. "rope_type": "longrope",
  728. "long_factor": [0.9982316082870437, 1.033048153422584, 1.0749920956484724, 1.1255096879436193, 1.1863348602111476, 1.259543828902579, 1.3476188888731149, 1.4535223827776373, 1.5807816745852985, 1.7335856049489526, 1.9168922912975785, 2.1365471404135326, 2.3994084200118646, 2.713475511863602, 3.0880118452194134, 3.533650295140154, 4.062463396503134, 4.687974098908333, 5.425075306704039, 6.289818967956352, 7.29902962722721, 8.6357018163639, 10.210822723989212, 12.053807765671676, 14.193944598909404, 16.65780676784363, 19.463620727694074, 22.628311203524586, 26.150106147261315, 30.02526691405111, 34.23183327975347, 38.73811934094828, 43.502489489729555, 48.47627117965394, 53.61139491762471, 58.857366522037935, 64.16798299215064, 69.51359464319125, 74.86555458220285, 80.21497790341579, 85.55322183307433, 90.89611806932027, 96.26245306514224, 101.68269304046481, 107.18619510219668, 112.82253283014026, 118.63764063163615, 119.88866203644656, 120.9462882391725, 121.837565139014, 122.58663780572562, 123.2147719894291, 123.74049454862576, 124.17980424685767, 124.54641761955492, 124.85202548028222, 125.10654406389756, 125.31835105170659, 125.49450117164764, 125.64091910903052, 125.76256945356558, 125.86360463815589, 125.94749252260765, 126.01712561287873],
  729. "short_factor": [0.9982316082870437, 1.033048153422584, 1.0749920956484724, 1.1255096879436193, 1.1863348602111476, 1.259543828902579, 1.3476188888731149, 1.4535223827776373, 1.5807816745852985, 1.7335856049489526, 1.9168922912975785, 2.1365471404135326, 2.3994084200118646, 2.713475511863602, 3.0880118452194134, 3.533650295140154, 4.062463396503134, 4.687974098908333, 5.425075306704039, 6.289818967956352, 7.29902962722721, 8.6357018163639, 10.210822723989212, 12.053807765671676, 14.193944598909404, 16.65780676784363, 19.463620727694074, 22.628311203524586, 26.150106147261315, 30.02526691405111, 34.23183327975347, 38.73811934094828, 43.502489489729555, 48.47627117965394, 53.61139491762471, 58.857366522037935, 64.16798299215064, 69.51359464319125, 74.86555458220285, 80.21497790341579, 85.55322183307433, 90.89611806932027, 96.26245306514224, 101.68269304046481, 107.18619510219668, 112.82253283014026, 118.63764063163615, 119.88866203644656, 120.9462882391725, 121.837565139014, 122.58663780572562, 123.2147719894291, 123.74049454862576, 124.17980424685767, 124.54641761955492, 124.85202548028222, 125.10654406389756, 125.31835105170659, 125.49450117164764, 125.64091910903052, 125.76256945356558, 125.86360463815589, 125.94749252260765, 126.01712561287873],
  730. "original_max_position_embeddings": 65536
  731. }
  732. }
  733. ```
  734. #### vLLM
  735. ##### Speculative Decoding
  736. For accelerated inference with speculative decoding using vLLM, follow these steps:
  737. ###### 1. Download MiniCPM4.1 Draft Model
  738. First, download the MiniCPM4.1 draft model:
  739. ```bash
  740. cd /your_path
  741. git clone https://huggingface.co/openbmb/MiniCPM4.1-8B-Eagle3
  742. ```
  743. ###### 2. Install EAGLE3-Compatible vLLM
  744. The EAGLE3 vLLM PR has been submitted. For now, use our repository for installation:
  745. ```bash
  746. git clone https://github.com/LDLINGLINGLING/vllm.git
  747. cd vllm
  748. pip install -e .
  749. ```
  750. ###### 3. Launch vLLM Server with Speculative Decoding
  751. Start the vLLM inference server with speculative decoding enabled. Make sure to update the model path in the speculative-config to point to your downloaded MiniCPM4_1-8B-Eagle3-bf16 folder:
  752. ```bash
  753. VLLM_USE_V1=1 \
  754. vllm serve openbmb/MiniCPM4.1-8B \
  755. --seed 42 \
  756. --trust-remote-code \
  757. --speculative-config '{
  758. "model": "your/path/MiniCPM4_1-8B-Eagle3-bf16",
  759. "num_speculative_tokens": 3,
  760. "method": "eagle3",
  761. "draft_tensor_parallel_size": 1
  762. }'
  763. ```
  764. ###### 4. Client Usage Example
  765. The client usage remains the same for both standard and speculative decoding:
  766. ```python
  767. import openai
  768. client = openai.Client(base_url="http://localhost:8000/v1", api_key="EMPTY")
  769. response = client.chat.completions.create(
  770. model="openbmb/MiniCPM4.1-8B",
  771. messages=[
  772. {"role": "user", "content": "Write an article about Artificial Intelligence."},
  773. ],
  774. temperature=0.6,
  775. max_tokens=32768,
  776. extra_body=dict(add_special_tokens=True), # Ensures special tokens are added for chat template
  777. )
  778. print(response.choices[0].message.content)
  779. ```
  780. ###### vLLM Configuration Parameters
  781. - `VLLM_USE_V1=1`: Enables vLLM v1 API
  782. - `--speculative-config`: JSON configuration for speculative decoding
  783. - `model`: Path to the draft model for speculation
  784. - `num_speculative_tokens`: Number of speculative tokens (default: 3)
  785. - `method`: Speculative decoding method (eagle3)
  786. - `draft_tensor_parallel_size`: Tensor parallel size for draft model (default: 1)
  787. - `--seed`: Random seed for reproducibility
  788. - `--trust-remote-code`: Allow execution of remote code for custom models
  789. ##### Standard Inference (Without Speculative Decoding)
  790. For now, you need to install the latest version of vLLM.
  791. ```bash
  792. pip install -U vllm \
  793. --pre \
  794. --extra-index-url https://wheels.vllm.ai/nightly
  795. ```
  796. Then you can inference MiniCPM4.1-8B with vLLM:
  797. ```python
  798. from transformers import AutoTokenizer
  799. from vllm import LLM, SamplingParams
  800. model_name = "openbmb/MiniCPM4.1-8B"
  801. prompt = [{"role": "user", "content": "Write an article about Artificial Intelligence."}]
  802. tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
  803. input_text = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)
  804. llm = LLM(
  805. model=model_name,
  806. trust_remote_code=True,
  807. max_num_batched_tokens=65536,
  808. dtype="bfloat16",
  809. gpu_memory_utilization=0.8,
  810. )
  811. sampling_params = SamplingParams(top_p=0.95, temperature=0.6, max_tokens=32768)
  812. outputs = llm.generate(prompts=input_text, sampling_params=sampling_params)
  813. print(outputs[0].outputs[0].text)
  814. ```
  815. Also, you can start the inference server by running the following command:
  816. > **Note**: In vLLM's chat API, `add_special_tokens` is `False` by default. This means important special tokens—such as the beginning-of-sequence (BOS) token—will not be added automatically. To ensure the input prompt is correctly formatted for the model, you should explicitly set `extra_body={"add_special_tokens": True}`.
  817. ```bash
  818. vllm serve openbmb/MiniCPM4.1-8B --trust-remote-code
  819. ```
  820. Then you can use the chat interface by running the following code:
  821. ```python
  822. import openai
  823. client = openai.Client(base_url="http://localhost:8000/v1", api_key="EMPTY")
  824. response = client.chat.completions.create(
  825. model="openbmb/MiniCPM4.1-8B",
  826. messages=[
  827. {"role": "user", "content": "Write an article about Artificial Intelligence."},
  828. ],
  829. temperature=0.6,
  830. max_tokens=32768,
  831. extra_body=dict(add_special_tokens=True), # Ensures special tokens are added for chat template
  832. )
  833. print(response.choices[0].message.content)
  834. ```
  835. #### SGLang
  836. ##### Speculative Decoding
  837. For accelerated inference with speculative decoding, follow these steps:
  838. ###### 1. Download MiniCPM4.1 Draft Model
  839. First, download the MiniCPM4.1 draft model:
  840. ```bash
  841. cd /your_path
  842. git clone https://huggingface.co/openbmb/MiniCPM4.1-8B-Eagle3
  843. ```
  844. ###### 2. Install EAGLE3-Compatible SGLang
  845. The EAGLE3 adaptation PR has been submitted. For now, use our repository for installation:
  846. ```bash
  847. git clone https://github.com/LDLINGLINGLING/sglang.git
  848. cd sglang
  849. pip install -e .
  850. ```
  851. ###### 3. Launch SGLang Server with Speculative Decoding
  852. Start the SGLang server with speculative decoding enabled:
  853. ```bash
  854. python -m sglang.launch_server \
  855. --model-path "openbmb/MiniCPM4.1-8B" \
  856. --host "127.0.0.1" \
  857. --port 30002 \
  858. --mem-fraction-static 0.9 \
  859. --speculative-algorithm EAGLE3 \
  860. --speculative-draft-model-path "your/path/MiniCPM4_1-8B-Eagle3-bf16" \
  861. --speculative-num-steps 3 \
  862. --speculative-eagle-topk 1 \
  863. --speculative-num-draft-tokens 32 \
  864. --temperature 0.7
  865. ```
  866. ###### 4. Client Usage
  867. The client usage remains the same for both standard and speculative decoding:
  868. ```python
  869. import openai
  870. client = openai.Client(base_url=f"http://localhost:30002/v1", api_key="None")
  871. response = client.chat.completions.create(
  872. model="openbmb/MiniCPM4.1-8B",
  873. messages=[
  874. {"role": "user", "content": "Write an article about Artificial Intelligence."},
  875. ],
  876. temperature=0.6,
  877. max_tokens=32768,
  878. )
  879. print(response.choices[0].message.content)
  880. ```
  881. Note: Make sure to update the port number in the client code to match the server port (30002 in the speculative decoding example).
  882. ###### Configuration Parameters
  883. - `--speculative-algorithm EAGLE3`: Enables EAGLE3 speculative decoding
  884. - `--speculative-draft-model-path`: Path to the draft model for speculation
  885. - `--speculative-num-steps`: Number of speculative steps (default: 3)
  886. - `--speculative-eagle-topk`: Top-k parameter for EAGLE (default: 1)
  887. - `--speculative-num-draft-tokens`: Number of draft tokens (default: 32)
  888. - `--mem-fraction-static`: Memory fraction for static allocation (default: 0.9)
  889. ##### Standard Inference (Without Speculative Decoding)
  890. For now, you need to install our forked version of SGLang.
  891. ```bash
  892. git clone -b openbmb https://github.com/OpenBMB/sglang.git
  893. cd sglang
  894. pip install --upgrade pip
  895. pip install -e "python[all]"
  896. ```
  897. You can start the inference server by running the following command:
  898. ```bash
  899. python -m sglang.launch_server --model openbmb/MiniCPM4.1-8B --trust-remote-code --port 30000 --chat-template chatml
  900. ```
  901. Then you can use the chat interface by running the following command:
  902. ```python
  903. import openai
  904. client = openai.Client(base_url=f"http://localhost:30000/v1", api_key="None")
  905. response = client.chat.completions.create(
  906. model="openbmb/MiniCPM4.1-8B",
  907. messages=[
  908. {"role": "user", "content": "Write an article about Artificial Intelligence."},
  909. ],
  910. temperature=0.6,
  911. max_tokens=32768,
  912. )
  913. print(response.choices[0].message.content)
  914. ```
  915. #### CPM.cu
  916. We **recommend** using [CPM.cu](https://github.com/OpenBMB/CPM.cu) for the inference of MiniCPM4 and MiniCPM4.1. CPM.cu is a CUDA inference framework developed by OpenBMB, which integrates efficient sparse, speculative sampling, and quantization techniques, fully leveraging the efficiency advantages of MiniCPM4 and MiniCPM4.1.
  917. You can install CPM.cu by running the following command:
  918. ```bash
  919. git clone https://github.com/OpenBMB/CPM.cu.git --recursive
  920. cd CPM.cu
  921. python3 setup.py install
  922. ```
  923. You can run the following command to test the speed of the model.
  924. ```bash
  925. python3 tests/long_prompt_gen.py # generate prompt.txt
  926. python3 tests/test_generate.py --prompt-file prompt.txt
  927. ```
  928. You can run the following command to infer with EAGLE3 speculative decoding algorithm.
  929. ```bash
  930. python3 -m cpmcu.cli \
  931. --model-path $BASE_MODEL_PATH \
  932. --draft-model-path $EAGLE3_DRAFT_MODEL_PATH \
  933. --prompt-text "Tell me about Tsinghua University" \
  934. --use-eagle3 true
  935. ```
  936. For more details about CPM.cu, please refer to the repo of [CPM.cu](https://github.com/OpenBMB/CPM.cu).
  937. #### llama.cpp and Ollama
  938. We also support inference with [llama.cpp](https://github.com/ggml-org/llama.cpp) and [Ollama](https://ollama.com/).
  939. ##### llama.cpp
  940. You can download the GGUF format of MiniCPM4.1-8B model from [huggingface](https://huggingface.co/openbmb/MiniCPM4.1-8B-GGUF) and run it with llama.cpp for efficient CPU or GPU inference.
  941. ```
  942. # case 1: main-cli
  943. ./build/bin/llama-cli -m MiniCPM4.1-8B-Q4_K_M.gguf -p "Write an article about Artificial Intelligence." -n 1500
  944. # case 2: server
  945. ## launch server
  946. ./build/bin/llama-server -m MiniCPM4.1-8B-Q4_K_M.gguf --host 127.0.0.1 --port 8080 -c 4096 -fa on &
  947. ## send request
  948. curl -X POST http://127.0.0.1:8080/v1/chat/completions \
  949. -H "Content-Type: application/json" \
  950. -d '{
  951. "model": "gpt-3.5-turbo",
  952. "messages": [{"role": "user", "content": "Write an article about Artificial Intelligence."}],
  953. "max_tokens": 1500
  954. }'
  955. ```
  956. ##### Ollama
  957. Please refer to [model hub](https://ollama.com/openbmb/minicpm4.1) for model download. After installing ollama package, you can use MiniCPM4.1 with following commands:
  958. ```
  959. ollama run openbmb/minicpm4.1
  960. ```
  961. </details>
  962. > Quantization (**BitCPM4**) and MiniCPM4 applications (**Survey** / **MCP** / **Intel AIPC Client**): see [`docs/README-legacy.md`](./docs/README-legacy.md).
  963. ## 📄 LICENSE
  964. #### Model LICENSE
  965. * This repository and MiniCPM models are released under the [Apache-2.0](https://github.com/OpenBMB/MiniCPM/blob/main/LICENSE) License.
  966. #### Statement
  967. * As a language model, MiniCPM generates content by learning from a vast amount of text.
  968. * However, it does not possess the ability to comprehend or express personal opinions or value judgments.
  969. * Any content generated by MiniCPM does not represent the viewpoints or positions of the model developers.
  970. * Therefore, when using content generated by MiniCPM, users should take full responsibility for evaluating and verifying it on their own.
  971. ## 🏛 Institutions
  972. This project is developed by the following institutions:
  973. - <img src="assets/modelbest.png" width="28px"> [Modelbest Inc.](https://modelbest.cn/)
  974. - <img src="assets/thunlp.png" width="28px"> [THUNLP](https://nlp.csai.tsinghua.edu.cn/)
  975. - <img src="assets/RUC.png" width="28px"> [Gaoling School of Artificial Intelligence of RUC](https://linyankai.github.io/)
  976. ## 📚 Citation
  977. * Please cite our paper: [MiniCPM4](https://arxiv.org/abs/2506.07900) if you find our work valuable.
  978. ```
  979. @article{minicpm4,
  980. title={Minicpm4: Ultra-efficient llms on end devices},
  981. author={MiniCPM, Team},
  982. journal={arXiv preprint arXiv:2506.07900},
  983. year={2025}
  984. }
  985. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.