Skip to content
Back to search

GLM-V

GLM-4.6V/4.5V/4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/zai-org--glm-v/badge.svg)](/listings/zai-org--glm-v)

HTML

<a href="/listings/zai-org--glm-v"><img src="/api/listings/zai-org--glm-v/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 277 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # GLM-V
  2. [中文阅读.](./README_zh.md)
  3. <div align="center">
  4. <img src=resources/logo.svg width="40%"/>
  5. </div>
  6. <p align="center">
  7. 👋 Join our <a href="resources/WECHAT.md" target="_blank">WeChat</a> or <a href="https://discord.gg/Hc5z9bx5Xw" target="_blank">Discord</a> communities.
  8. <br>
  9. 📖 Check out the GLM-5.3-Flash <a href="https://z.ai/blog/glm-5.3-flash" target="_blank">blog</a> and the GLM-4.5V & GLM-4.1V <a href="https://arxiv.org/abs/2507.01006" target="_blank">technical report</a>.
  10. <br>
  11. 📍 Use the GLM-5.3-Flash API service on the <a href="https://docs.bigmodel.cn/cn/guide/models/text/glm-5.3" target="_blank">API platform</a>.
  12. <br>
  13. 🔜 Try GLM-5.3-Flash on <a href="https://z.ai" target="_blank">z.ai</a> (coming soon).
  14. </p>
  15. ## Introduction
  16. Vision-language models (VLMs) have become a key cornerstone of intelligent systems. As real-world AI tasks grow
  17. increasingly complex, VLMs urgently need to enhance reasoning capabilities beyond basic multimodal perception —
  18. improving accuracy, comprehensiveness, and intelligence — to enable complex problem solving, long-context understanding,
  19. and multimodal agents.
  20. Through our open-source work, we aim to explore the technological frontier together with the community while empowering
  21. more developers to create exciting and innovative applications.
  22. **This open-source repository contains our `GLM-4.6V`, `GLM-4.5V` and `GLM-4.1V` series models.** For performance and
  23. details, see [Model Overview](#model-overview). For known issues,
  24. see [Fixed and Remaining Issues](#fixed-and-remaining-issues).
  25. ## Project Updates
  26. - **News**: `2026/09/02`: This repository will no longer be maintained. For questions about GLM-5.3-Flash, please visit the [GLM-5](https://github.com/zai-org/GLM-5) repository for discussion.
  27. - **News**: `2026/08/26`: We released [GLM-5.3-Flash](https://z.ai/blog/glm-5.3-flash), our first open-source natively multimodal model.
  28. - **News**: `2026/04/02`: We released [GLM-5V-Turbo](https://docs.z.ai/guides/vlm/glm-5v-turbo)
  29. and [GLM-skills](https://github.com/zai-org/GLM-skills).
  30. - **News**: `2026/03/28`: We have released multiple GLM-V related Skills, covering several specialized areas
  31. such as GLM-V-Grounding and GLM-V-Prompt-Gen. You are welcome to try them [here](skills).
  32. - **News**: `2025/11/10`: We released **UI2Code^N**, a RL-enhanced UI coding model with UI-to-code, UI-polish, and
  33. UI-edit capabilities. The model is trained based on `GLM-4.1V-Base`. Check it
  34. out [here](https://huggingface.co/zai-org/UI2Code_N).
  35. - **News**: `2025/10/27`: We’ve released **Glyph**, a framework for scaling the context length through visual-text
  36. compression, the glyph model trained based on `GLM-4.1V-Base`. Check it
  37. out [here](https://huggingface.co/zai-org/Glyph).
  38. - **News**: `2025/08/11`: We released **GLM-4.5V** with significant improvements across multiple benchmarks. We also
  39. open-sourced our handcrafted **desktop assistant app** for debugging. Once connected to GLM-4.5V, it can capture
  40. visual information from your PC screen via screenshots or screen recordings. Feel free to try it out or customize it
  41. into your own multimodal assistant. Click [here](https://huggingface.co/spaces/zai-org/GLM-4.5V-Demo-App) to download
  42. the installer or [build from source](examples/vllm-chat-helper/README.md)!
  43. - **News**: `2025/07/16`: We have open-sourced the **VLM Reward System** used to train GLM-4.1V-Thinking.View
  44. the [code repository](glmv_reward) and run locally: `python examples/reward_system_demo.py`.
  45. - **News**: `2025/07/01`: We released **GLM-4.1V-9B-Thinking** and
  46. its [technical report](https://arxiv.org/abs/2507.01006).
  47. ## Model Implementation Code
  48. - GLM-4.5V and GLM-4.6V model algorithm: see the full implementation
  49. in [transformers](https://github.com/huggingface/transformers/tree/main/src/transformers/models/glm4v_moe).
  50. - GLM-4.1V-9B-Thinking model algorithm: see the full implementation
  51. in [transformers](https://github.com/huggingface/transformers/tree/main/src/transformers/models/glm4v).
  52. - Both models share identical multimodal preprocessing, but use different conversation templates — please distinguish
  53. carefully.
  54. ## Model Downloads
  55. | Model | Download Links | Type |
  56. |----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|------------------|
  57. | GLM-4.6V | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.6V)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.6V) | Hybrid Reasoning |
  58. | GLM-4.6V-FP8 | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.6V-FP8)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.6V-FP8) | Hybrid Reasoning |
  59. | GLM-4.6V-Flash | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.6V-Flash)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.6V-Flash) | Hybrid Reasoning |
  60. | GLM-4.5V | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.5V)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.5V) | Hybrid Reasoning |
  61. | GLM-4.5V-FP8 | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.5V-FP8)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.5V-FP8) | Hybrid Reasoning |
  62. | GLM-4.1V-9B-Thinking | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.1V-9B-Thinking)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.1V-9B-Thinking) | Reasoning |
  63. | GLM-4.1V-9B-Base | [🤗 Hugging Face](https://huggingface.co/zai-org/GLM-4.1V-9B-Base)<br>[🤖 ModelScope](https://modelscope.cn/models/ZhipuAI/GLM-4.1V-9B-Base) | Base |
  64. + Hugging Face provides GGUF format model weights. You can download the GGUF format model of GLM-V from [here](https://huggingface.co/collections/ggml-org/glm-v).
  65. ## Using Case
  66. ### Grounding
  67. GLM-4.5V / GLM-4.6V / GLM-4.1V equips precise grounding capabilities. Given a prompt that requests the location of a specific object, the model
  68. is able to reasoning step-by-step and identify the bounding boxes of the target object. The query prompt supports
  69. complex descriptions of the target object as well as specified output formats, for example:
  70. >
  71. > - Help me to locate <expr> in the image and give me its bounding boxes.
  72. > - Please pinpoint the bounding box [[x1,y1,x2,y2], …] in the image as per the given description. <expr>
  73. Here, `<expr>` is the description of the target object. The output bounding box is a quadruple $$[x_1,y_1,x_2,y_2]$$
  74. composed of the coordinates of the top-left and bottom-right corners, where each value is normalized by the image
  75. width (for x) or height (for y) and scaled by 1000.
  76. In the response, the special tokens `<|begin_of_box|>` and `<|end_of_box|>` are used to mark the image bounding box in
  77. the answer. The bracket style may vary ([], [[]], (), <>, etc.), but the meaning is the same: to enclose the coordinates
  78. of the box.
  79. ### GUI Agent
  80. - `examples/gui-agent`: Demonstrates prompt construction and output handling for GUI Agents, including strategies for
  81. mobile, PC, and web. Prompt templates differ between GLM-4.1V and GLM-4.5V.
  82. ### Quick Demo
  83. - `examples/vlm-helper`: A desktop assistant for GLM multimodal models (mainly GLM-4.5V, compatible with GLM-4.1V),
  84. supporting text, images, videos, PDFs, PPTs, and more. Connects to the GLM multimodal API for intelligent services
  85. across scenarios. Download the [installer](https://huggingface.co/spaces/zai-org/GLM-4.5V-Demo-App)
  86. or [build from source](examples/vlm-helper/README.md).
  87. ## Serve GLM-V Series Models Locally
  88. The GLM-V series supports deployment with the following frameworks. Feel free to try them out:
  89. - [SGLang](https://github.com/sgl-project/sglang) (v0.5.15.post1+) — see [cookbook](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-4.6V)
  90. - [vLLM](https://github.com/vllm-project/vllm) (v0.25.0+) — see [recipes](https://recipes.vllm.ai/zai-org/GLM-4.6V)
  91. - [Transformers](https://github.com/huggingface/transformers) (v0.5.13+) — see [transformers docs](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/glm4v_moe.md)
  92. - For deployment on the `Ascend NPU` platform, inference frameworks such as xLLM, see [here](examples/Ascend_NPU/README_zh.md).
  93. ## Integration with Other Automation Tools
  94. ### Midscene.js
  95. [Midscene.js](https://midscenejs.com/en/index.html) is an open-source UI automation SDK driven by vision models, supporting multi-platform automation through JavaScript or Yaml-format process syntax.
  96. Midscene.js has completed integration with GLM-V models. You can quickly experience GLM-V through the [Midscene.js Integration Guide](https://midscenejs.com/model-common-config.html#glm-v).
  97. Here are two examples to help you get started quickly:
  98. - [Call Midscene.js via TypeScript scripts](./examples/midscene-ts-demo)
  99. - [Experience Midscene.js via Yaml scripts](./examples/midscene-yaml-demo)
  100. ## Model Fine-tuning
  101. [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) already supports fine-tuning for GLM-4.5V &
  102. GLM-4.1V-9B-Thinking models. Below is an example of dataset construction using two images. You should organize your
  103. dataset into `finetune.json` in the following format, This is an example for fine-tuning GLM-4.1V-9B.
  104. ```json
  105. [
  106. {
  107. "messages": [
  108. {
  109. "content": "<image>Who are they?",
  110. "role": "user"
  111. },
  112. {
  113. "content": "<think>\nUser asked me to observe the image and find the answer. I know they are Kane and Goretzka from Bayern Munich.</think>\n<answer>They're Kane and Goretzka from Bayern Munich.</answer>",
  114. "role": "assistant"
  115. },
  116. {
  117. "content": "<image>What are they doing?",
  118. "role": "user"
  119. },
  120. {
  121. "content": "<think>\nI need to observe what these people are doing. Oh, they are celebrating on the soccer field.</think>\n<answer>They are celebrating on the soccer field.</answer>",
  122. "role": "assistant"
  123. }
  124. ],
  125. "images": [
  126. "mllm_demo_data/1.jpg",
  127. "mllm_demo_data/2.jpg"
  128. ]
  129. }
  130. ]
  131. ```
  132. 1. The content inside `<think> ... </think>` will **not** be stored as conversation history or in fine-tuning data.
  133. 2. The `<image>` tag will be replaced with the corresponding image information.
  134. 3. For the GLM-4.5V model, the <answer> and </answer> tags should be removed.
  135. Then, you can fine-tune following the standard LLaMA-Factory procedure.
  136. ## Model Overview
  137. ### GLM-4.6V
  138. GLM-4.6V series model includes two versions: GLM-4.6V (106B), a foundation model designed for cloud and high-performance
  139. cluster scenarios,
  140. and GLM-4.6V-Flash (9B), a lightweight model optimized for local deployment and low-latency applications.
  141. GLM-4.6V scales its context window to 128k tokens in training,
  142. and achieves SoTA performance in visual understanding among models of similar parameter scales.
  143. Crucially, we integrate native Function Calling capabilities for the first time.
  144. This effectively bridges the gap between "visual perception" and "executable action"
  145. providing a unified technical foundation for multimodal agents in real-world business scenarios.
  146. ![GLM-4.6V Benchmarks](resources/bench_46v.jpeg)
  147. Beyond achieves SoTA performance across major multimodal benchmarks at comparable model scales. GLM-4.6V introduces
  148. several key features:
  149. - **Native Multimodal Function Calling**
  150. Enables native vision-driven tool use. Images, screenshots, and document pages can be passed directly as tool inputs without text conversion, while visual outputs (charts, search images, rendered pages) are interpreted and integrated into the reasoning chain. This closes the loop from perception to understanding to execution.
  151. - **Interleaved Image-Text Content Generation**
  152. Supports high-quality mixed media creation from complex multimodal inputs. GLM-4.6V takes a multimodal context—spanning documents, user inputs, and tool-retrieved images—and synthesizes coherent, interleaved image-text content tailored to the task. During generation it can actively call search and retrieval tools to gather and curate additional text and visuals, producing rich, visually grounded content.
  153. - **Multimodal Document Understanding**
  154. GLM-4.6V can process up to 128K tokens of multi-document or long-document input, directly interpreting richly formatted pages as images. It understands text, layout, charts, tables, and figures jointly, enabling accurate comprehension of complex, image-heavy documents without requiring prior conversion to plain text.
  155. - **Frontend Replication & Visual Editing**
  156. Reconstructs pixel-accurate HTML/CSS from UI screenshots and supports natural-language-driven edits. It detects layout, components, and styles visually, generates clean code, and applies iterative visual modifications through simple user instructions.
  157. ### GLM-4.5V
  158. GLM-4.5V is based on ZhipuAI’s GLM-4.5-Air.
  159. It continues the technical approach of GLM-4.1V-Thinking, achieving SOTA performance among models of the same scale on
  160. 42 public vision-language benchmarks.
  161. It covers common tasks such as image, video, and document understanding, as well as GUI agent operations.
  162. Beyond benchmark performance, GLM-4.5V focuses on real-world usability. Through efficient hybrid training, it can handle
  163. diverse types of visual content, enabling full-spectrum vision reasoning, including:
  164. - **Image reasoning** (scene understanding, complex multi-image analysis, spatial recognition)
  165. - **Video understanding** (long video segmentation and event recognition)
  166. - **GUI tasks** (screen reading, icon recognition, desktop operation assistance)
  167. - **Complex chart & long document parsing** (research report analysis, information extraction)
  168. - **Grounding** (precise visual element localization)
  169. The model also introduces a **Thinking Mode** switch, allowing users to balance between quick responses and deep
  170. reasoning. This switch works the same as in the `GLM-4.5` language model.
  171. ### GLM-4.1V-9B
  172. Built on the [GLM-4-9B-0414](https://github.com/zai-org/GLM-4) foundation model, the **GLM-4.1V-9B-Thinking** model
  173. introduces a reasoning paradigm and uses RLCS (Reinforcement Learning with Curriculum Sampling) to comprehensively
  174. enhance model capabilities.
  175. It achieves the strongest performance among 10B-level VLMs and matches or surpasses the much larger Qwen-2.5-VL-72B in
  176. 18 benchmark tasks.
  177. We also open-sourced the base model **GLM-4.1V-9B-Base** to support researchers in exploring the limits of
  178. vision-language model capabilities.
  179. ![rl](resources/rl.jpeg)
  180. Compared with the previous generation CogVLM2 and GLM-4V series, **GLM-4.1V-Thinking** brings:
  181. 1. The series’ first reasoning-focused model, excelling in multiple domains beyond mathematics.
  182. 2. **64k** context length support.
  183. 3. Support for **any aspect ratio** and up to **4k** image resolution.
  184. 4. A bilingual (Chinese/English) open-source version.
  185. GLM-4.1V-9B-Thinking integrates the **Chain-of-Thought** reasoning mechanism, improving accuracy, richness, and
  186. interpretability.
  187. It leads on 23 out of 28 benchmark tasks at the 10B parameter scale, and outperforms Qwen-2.5-VL-72B on 18 tasks despite
  188. its smaller size.
  189. ## Remaining Issues
  190. Since the open-sourcing of GLM-4.1V, we have received extensive feedback from the community and are well aware that the model still has many shortcomings. In subsequent iterations, we attempted to address several common issues — such as repetitive thinking outputs and formatting errors — which have been mitigated to some extent in this new version.
  191. However, the model still has several limitations and issues that we will fix as soon as possible:
  192. 1. Pure text QA capabilities still have significant room for improvement. In this development cycle, our primary focus was on visual multimodal scenarios, and we will enhance pure text abilities in upcoming updates.
  193. 2. The model may still overthink or even repeat itself in certain cases, especially when dealing with complex prompts.
  194. 3. In some situations, the model may restate the answer again at the end.
  195. 4. There remain certain perception limitations, such as counting accuracy and identifying specific individuals, which still require improvement.
  196. Thank you for your patience and understanding. We also welcome feedback and suggestions in the issue section — we will respond and improve as much as we can!
  197. ## Citation
  198. If you use this model, please cite the following paper:
  199. ```bibtex
  200. @misc{vteam2025glm45vglm41vthinkingversatilemultimodal,
  201. title={GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning},
  202. author={V Team and Wenyi Hong and Wenmeng Yu and Xiaotao Gu and Guo Wang and Guobing Gan and Haomiao Tang and Jiale Cheng and Ji Qi and Junhui Ji and Lihang Pan and Shuaiqi Duan and Weihan Wang and Yan Wang and Yean Cheng and Zehai He and Zhe Su and Zhen Yang and Ziyang Pan and Aohan Zeng and Baoxu Wang and Bin Chen and Boyan Shi and Changyu Pang and Chenhui Zhang and Da Yin and Fan Yang and Guoqing Chen and Jiazheng Xu and Jiale Zhu and Jiali Chen and Jing Chen and Jinhao Chen and Jinghao Lin and Jinjiang Wang and Junjie Chen and Leqi Lei and Letian Gong and Leyi Pan and Mingdao Liu and Mingde Xu and Mingzhi Zhang and Qinkai Zheng and Sheng Yang and Shi Zhong and Shiyu Huang and Shuyuan Zhao and Siyan Xue and Shangqin Tu and Shengbiao Meng and Tianshu Zhang and Tianwei Luo and Tianxiang Hao and Tianyu Tong and Wenkai Li and Wei Jia and Xiao Liu and Xiaohan Zhang and Xin Lyu and Xinyue Fan and Xuancheng Huang and Yanling Wang and Yadong Xue and Yanfeng Wang and Yanzi Wang and Yifan An and Yifan Du and Yiming Shi and Yiheng Huang and Yilin Niu and Yuan Wang and Yuanchang Yue and Yuchen Li and Yutao Zhang and Yuting Wang and Yu Wang and Yuxuan Zhang and Zhao Xue and Zhenyu Hou and Zhengxiao Du and Zihan Wang and Peng Zhang and Debing Liu and Bin Xu and Juanzi Li and Minlie Huang and Yuxiao Dong and Jie Tang},
  203. year={2025},
  204. eprint={2507.01006},
  205. archivePrefix={arXiv},
  206. primaryClass={cs.CV},
  207. url={https://arxiv.org/abs/2507.01006},
  208. }
  209. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.