Skip to content
Back to search

Spark-H3

Adaptive Block Sparse Attention for MiniMax-H3

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/zechengtang--spark-h3/badge.svg)](/listings/zechengtang--spark-h3)

HTML

<a href="/listings/zechengtang--spark-h3"><img src="/api/listings/zechengtang--spark-h3/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Set up your H3 pipeline using the [upstream MiniMax-H3 instructions](https://github.com/MiniMax-AI/MiniMax-H3), then install Spark in the same environment with a compatible PyTorch/CUDA stack:

python -m pip install -e '.[cuda]'

Wrap an already loaded pipeline with the Spark installer:

from h3_sparse_attention import install_h3_spark_attn

# MiniMaxH3Scheduler includes the terminal sigma=0 in num_inference_steps,
# but that terminal point does not run the transformer.
num_denoise_steps = 19
num_inference_steps = num_denoise_steps + 1

with install_h3_spark_attn(
    pipe.transformer,
    num_denoise_steps=num_denoise_steps,
    warmup_mode="warmup_steps",
    warmup_steps=4,
    dense_layers=0,
    topk_ratio=0.1,
):
    result = pipe(**inputs, num_inference_steps=num_inference_steps)

<a id="comfyui-installation"></a>

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 218 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. <h1><img src="assets/spark-h3-wordmark.png" alt="Spark-H3" width="270"></h1>
  3. <h3>Adaptive Block Sparse Attention for MiniMax-H3</h3>
  4. <p><strong>Reblock similar tokens · Reweight tail tokens</strong></p>
  5. <p>
  6. <a href="https://zechengtang.github.io/Spark-H3/"><img src="https://img.shields.io/badge/Blog-Spark--H3-f97316?style=flat-square" alt="Spark-H3 technical blog"></a>
  7. <img src="https://img.shields.io/badge/arXiv-Spark--H3-b31b1b?style=flat-square" alt="Spark-H3 on arXiv">
  8. <a href="https://huggingface.co/Aazeus/Spark-H3"><img src="https://img.shields.io/badge/%F0%9F%A4%97_Hugging_Face-Spark--H3-ffc107?style=flat-square" alt="Spark-H3 on Hugging Face"></a>
  9. <a href="comfyui/README.md"><img src="https://img.shields.io/badge/ComfyUI-Spark_Node-f97316?style=flat-square" alt="Spark-H3 ComfyUI node"></a>
  10. <a href="#special-thanks"><img src="https://img.shields.io/badge/Special_Thanks-Community_Contributors-e11d48?style=flat-square" alt="Special thanks to community contributors"></a>
  11. <a href="h3_sparse_attention/README.md"><img src="https://img.shields.io/badge/Docs-Usage_%26_Configuration-2563eb?style=flat-square" alt="Usage and configuration guide"></a>
  12. <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-15803d?style=flat-square" alt="License: Apache 2.0"></a>
  13. <a href="https://github.com/zechengtang/Spark-H3/releases"><img src="https://img.shields.io/github/downloads/zechengtang/Spark-H3/total?style=flat-square&amp;label=Downloads" alt="Total GitHub release downloads"></a>
  14. </p>
  15. <p>
  16. <strong>English</strong> · <a href="README.zh-CN.md">简体中文</a>
  17. </p>
  18. <p>
  19. <a href="#demo">🎬 Demo</a> &nbsp;·&nbsp;
  20. <a href="#quick-start">🚀 Quick Start</a> &nbsp;·&nbsp;
  21. <a href="#comfyui-installation">🧩 ComfyUI</a> &nbsp;·&nbsp;
  22. <a href="https://zechengtang.github.io/Spark-H3/">📖 Technical Blog</a> &nbsp;·&nbsp;
  23. <a href="#project-status">🧭 Project Status</a>
  24. </p>
  25. </div>
  26. > [!IMPORTANT]
  27. > **ComfyUI v0.2.0 release:** Spark-H3 now uses the isolated
  28. > `spark-h3-kitchen` distribution and `spark_h3_kitchen` import namespace.
  29. > It no longer replaces or pins ComfyUI's official `comfy-kitchen` package.
  30. > The release supports ComfyUI 0.38.x–0.40.x and 0.42.x–0.44.x.
  31. > **For production use, we recommend the default `spark-h3-kitchen` 0.2.37,
  32. > which delivered better speed and stability in release testing.**
  33. > **[Read the v0.2.0 release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.2.0)**
  34. > **VPack v0.1.0:** A separate ComfyUI plugin that compresses reference
  35. > videos in Ref2VA workflows.
  36. > **[VPack GitHub repository](https://github.com/zechengtang/VPack)**
  37. > 👉 [Ref2VA 任务加速解决方案](docs/sparse_scope.md#ref2va-任务加速解决方案)
  38. ---
  39. <a id="demo"></a>
  40. ## 🎬 Demo
  41. Native MiniMax-H3 · VDN Selected Prompt 06 · 1344×768 · 14.4 seconds.
  42. The three panels compare Dense, Spark-H3 Fidelity (2.19×), and Spark-H3 Speed (3.12×).
  43. The animated preview and linked clip cover the final 5.4 seconds, starting at 9 seconds.
  44. [![Animated Native MiniMax-H3 attic comparison: Dense, Spark-H3 Fidelity, and Spark-H3 Speed](assets/h3-native-attic-comparison-9s-to-end.webp)](assets/h3-native-attic-comparison-9s-to-end.mp4)
  45. <p align="center"><a href="https://huggingface.co/Aazeus/Spark-H3/resolve/main/comfyui/demos/H3_native_vdn_selected_0006_dense_fidelity_speed_native_resolution.mp4">Watch the full 14.4-second comparison</a></p>
  46. - [Veda / Sol / Spark: subjective visual quality and speed comparison (Bilibili)](https://www.bilibili.com/video/BV1zynA6aEZG/)
  47. - [Spark-H3 + VPack on Ref2VA (Bilibili)](https://www.bilibili.com/video/BV1i6J163EGG/)
  48. ## ✨ Spark-Attn
  49. Spark-Attn is a block sparse attention method for MiniMax-H3 video generation.
  50. It combines two operations:
  51. - **🧩 Reblocking** groups similar tokens into blocks so sparse attention can
  52. select more relevant interactions.
  53. - **⚖️ Reweighting** assigns different weights to tokens within each tail block
  54. when building key/value summaries, with a correction for attention mass.
  55. Spark-H3 provides the Spark attention kernels and MiniMax-H3 inference integration.
  56. <a id="quick-start"></a>
  57. ## 🚀 Quick Start
  58. Set up your H3 pipeline using the [upstream MiniMax-H3 instructions](https://github.com/MiniMax-AI/MiniMax-H3),
  59. then install Spark in the same environment with a compatible PyTorch/CUDA stack:
  60. ```bash
  61. python -m pip install -e '.[cuda]'
  62. ```
  63. Wrap an already loaded pipeline with the Spark installer:
  64. ```python
  65. from h3_sparse_attention import install_h3_spark_attn
  66. # MiniMaxH3Scheduler includes the terminal sigma=0 in num_inference_steps,
  67. # but that terminal point does not run the transformer.
  68. num_denoise_steps = 19
  69. num_inference_steps = num_denoise_steps + 1
  70. with install_h3_spark_attn(
  71. pipe.transformer,
  72. num_denoise_steps=num_denoise_steps,
  73. warmup_mode="warmup_steps",
  74. warmup_steps=4,
  75. dense_layers=0,
  76. topk_ratio=0.1,
  77. ):
  78. result = pipe(**inputs, num_inference_steps=num_inference_steps)
  79. ```
  80. <a id="comfyui-installation"></a>
  81. ## 🧩 ComfyUI Installation
  82. Spark-H3 v0.2.0 provides standalone nodes for ComfyUI 0.38.x–0.40.x and
  83. 0.42.x–0.44.x. Its isolated `spark-h3-kitchen` backend is installed alongside
  84. ComfyUI's official `comfy-kitchen` package instead of replacing it. Linux and
  85. Windows x86_64 packages are produced for SM86, SM89, and SM120.
  86. Choose the CUDA 13.0 package matching your operating system and GPU
  87. architecture. ComfyUI and model weights are not included.
  88. For installation and example workflows, see the **[ComfyUI guide](comfyui/README.md)**
  89. or **[中文安装指南](comfyui/INSTALL.zh-CN.md)**.
  90. For reference-video compression in Ref2VA workflows, install the separate
  91. **[VPack v0.1.0](https://github.com/zechengtang/VPack)** plugin.
  92. ## Step-count convention
  93. `MiniMaxH3Scheduler` includes the final `σ=0` endpoint in
  94. `num_inference_steps`, so the default `num_inference_steps=50` runs 49 denoising steps. See the
  95. [Hugging Face scheduler documentation](https://huggingface.co/docs/diffusers/main/api/schedulers/minimax_h3).
  96. <!-- ### Benchmark comparability
  97. Match the **actual** `torch.compile` state across all Diffusers runs before
  98. comparing quality or speed. In a controlled 25-prompt H3 check, switching
  99. denoising compilation changed Spark PSNR by **3.15 dB**, despite otherwise
  100. matched settings. This is a protocol difference, not evidence of a Spark
  101. kernel regression. Do not mix compiled and uncompiled benchmark results.
  102. The sibling MiniMax-H3-Benchmark's standard Diffusers denoising runner and
  103. radial/attention harnesses require compilation and verify that all transformer
  104. blocks were wrapped. Do not mix their results
  105. with historical uncompiled experiments.
  106. `inputs` contains your pipeline's generation arguments. The context manager
  107. restores the original attention processors on exit.
  108. → See the [usage and configuration guide](h3_sparse_attention/README.md) for
  109. requirements, defaults, and options. -->
  110. <a id="todo"></a>
  111. <a id="project-status"></a>
  112. ## 🧭 Project Status & Roadmap
  113. The Diffusers and ComfyUI integrations are separate implementation tracks.
  114. Their status is listed independently below so that backend work is not confused
  115. with ComfyUI packaging or compatibility work.
  116. ### Diffusers / Python
  117. | Status | Item | Resources |
  118. |---|---|---|
  119. | ✅ Available | Spark attention integration for MiniMax-H3, with fused kernels for SM80, SM86, SM89, and SM120 | [Usage and kernel documentation](h3_sparse_attention/README.md) · [Performance matrix](docs/kernel_performance_matrix.md) |
  120. | ✅ Available | Ref2VA pipeline support and released inference examples | [Spark-Ref2VA Preview](https://zechengtang.github.io/Spark-H3/#spark-ref2va-preview) |
  121. | 🗓️ Planned | Fused Spark kernels for SM90 and SM100 | Pending hardware resources |
  122. | 🗓️ Planned | Technical report release | — |
  123. ### ComfyUI
  124. | Version | Date | Status | Main improvements / expected updates | Release / tracking |
  125. |---|---|---|---|---|
  126. | v0.1.1 | 2026-10-09 | ✅ Released | First stable Linux packages for SM89 and SM120; fixed the v0.1.0 CUDA Graph allocation-lifetime issue; bundled five representative workflows | [Release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.1.1) |
  127. | v0.1.2 | 2026-10-11 | ✅ Released | Added validated Windows SM89 and SM120 packages; corrected packaged workflows and optimized the LBH two-pass example | [Release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.1.2) |
  128. | v0.1.3 | 2026-10-13 | ✅ Released | Added validated SM86 packages; integrated with KJNodes low-VRAM patches; added `fidelity`, `speed`, `custom`, and `dense` presets plus selectable dense backends | [Release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.1.3) |
  129. | v0.1.4 | 2026-10-15 | ✅ Released | Added ComfyUI 0.40.x support, Ref2VA-aware `sparse_scope`, configurable `producer_chunk`, and in-node Ref2VA guidance | [Release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.1.4) |
  130. | v0.1.5 | 2026-10-19 | ✅ Released | Added ComfyUI 0.42.x support through comfy-kitchen 0.2.39; released Linux and Windows SM86/SM89/SM120 packages, each also bundling wheels for ComfyUI 0.38.x–0.40.x | [Release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.1.5) |
  131. | v0.2.0 | 2026-11-06 | ✅ Released | Decouples Spark-H3 from ComfyUI's official `comfy-kitchen` dependency by using an isolated backend; supports ComfyUI 0.38.x–0.40.x and 0.42.x–0.44.x; recommends the default 0.2.37 backend for better tested speed and stability | [Release notes](https://github.com/zechengtang/Spark-H3/releases/tag/comfyui-v0.2.0) |
  132. The v0.2.0 release supports ComfyUI 0.38.x–0.40.x and 0.42.x–0.44.x with
  133. Linux and Windows x86_64 packages for SM86, SM89, and SM120.
  134. See the [ComfyUI guide](comfyui/README.md)
  135. for workflows and the [installation guide](comfyui/standalone/README.md) for
  136. package details. Reference-video compression is provided separately by
  137. [VPack](https://github.com/zechengtang/VPack).
  138. ## 🤝 Acknowledgments
  139. Thanks to [MiniMax-H3](https://github.com/MiniMax-AI/MiniMax-H3) and
  140. [Sol-Attn / Sol-Engine](https://github.com/NVlabs/Sana/tree/sol-engine) for the
  141. model and attention infrastructure. See the [third-party notices](sol_attn/THIRD_PARTY_NOTICES.md)
  142. for kernel attributions and included licenses.
  143. <a id="special-thanks"></a>
  144. ## ❤️ Special Thanks
  145. - Thanks to **Smthem** ([GitHub](https://github.com/smthemex) · [Bilibili](https://space.bilibili.com/439655233)) for support and help during the development of Spark-H3's ComfyUI plugin.
  146. - Thanks to **AI绘画KK** ([GitHub](https://github.com/kkget) · [Bilibili](https://space.bilibili.com/313528845)) for valuable guidance and support in exploring and applying MiniMax-H3 workflows, and for helping communicate the community's need for Windows installation packages.
  147. - Thanks to **AIwood爱屋研究室** ([GitHub](https://github.com/slmonker) · [Bilibili](https://space.bilibili.com/503934057)) for identifying a bug when Spark-H3 was used in a two-pass sampling workflow and suggesting a switch to Kitchen as the dense backend, which further improved the plugin's speedup ratio.
  148. ## ⭐ Star History
  149. <p align="center">
  150. <a href="https://www.star-history.com/?repos=zechengtang%2FSpark-H3&amp;type=date&amp;legend=top-left">
  151. <picture>
  152. <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=zechengtang/Spark-H3&amp;type=date&amp;theme=dark&amp;legend=top-left">
  153. <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=zechengtang/Spark-H3&amp;type=date&amp;legend=top-left">
  154. <img alt="Spark-H3 GitHub star history chart" src="https://api.star-history.com/chart?repos=zechengtang/Spark-H3&amp;type=date&amp;legend=top-left" width="720">
  155. </picture>
  156. </a>
  157. </p>
  158. ## 📄 License
  159. Spark-Attn’s original code and documentation are licensed under the
  160. [Apache License 2.0](LICENSE). Third-party components retain their original
  161. licenses and notices; see [third-party notices](sol_attn/THIRD_PARTY_NOTICES.md).
  162. MiniMax-H3 model weights remain subject to their upstream license.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.