Skip to content
Back to search

automodel

πŸš€ Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/nvidia-nemo/automodel with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/nvidia-nemo/automodel'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/nvidia-nemo--automodel/badge.svg)](/listings/nvidia-nemo--automodel)

HTML

<a href="/listings/nvidia-nemo--automodel"><img src="/api/listings/nvidia-nemo--automodel/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

We recommend using **uv** for reproducible Python environments.

# Setup environment before running any recipes
uv venv

# Choose ONE:
uv sync --frozen  # LLM recipes (default)
# uv sync --frozen --extra vlm --extra vlm-media  # VLM recipes (Qwen/Mistral/Omni need vlm-media for video/vision; fixes: ImportError: qwen_vl_utils is not installed)
# uv sync --frozen --extra cuda  # Optional CUDA deps (e.g., Transformer Engine, Mamba SSM)
# uv sync --frozen --extra cuda_source  # Optional bitsandbytes dependency
# uv sync --frozen --extra all  # Most optional deps (includes `vlm` and `cuda`; NOTE: excludes media β€” add --extra media for video/image decode)
# uv sync --frozen --all-extras  # Everything (includes `fa`, `moe`, `media`, etc.)

# One-off runs (examples):
# uv run --extra vlm <command>
# uv run --extra cuda <command>

uv run python -c "import nemo_automodel; print('NeMo AutoModel ready')"

Run a Recipe

All recipes are launched via the `automodel` CLI (or its short alias `am`). Each YAML config specifies the recipe class and all training parameters:

# LLM example: multi-GPU fine-tuning with FSDP2
automodel examples/llm_finetune/llama3_2/llama3_2_1b_hellaswag.yaml --nproc-per-node 8

# VLM example: single-GPU fine-tuning (Gemma-3-VL) with LoRA
automodel examples/vlm_finetune/gemma3/gemma3_vl_4b_cord_v2_peft.yaml

# Both commands also work with uv run:
uv run automodel examples/llm_finetune/llama3_2/llama3_2_1b_hellaswag.yaml --nproc-per-node 8

> [!TIP] > **NeMo-Run submission:** The `cli` extra adds NeMo Run to the base package: `uv pip install "nemo-automodel[cli]"`. It is additive; the base package still installs its core training dependencies, including PyTorch.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md Β· README Β· 522 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # πŸš€ NeMo AutoModel
  3. </div>
  4. <div align="center">
  5. <!-- [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -->
  6. [![codecov](https://codecov.io/github/NVIDIA-NeMo/Automodel/graph/badge.svg?token=4NMKZVOW2Z)](https://codecov.io/github/NVIDIA-NeMo/Automodel)
  7. [![CICD NeMo](https://github.com/NVIDIA-NeMo/Automodel/actions/workflows/cicd-main.yml/badge.svg)](https://github.com/NVIDIA-NeMo/Automodel/actions/workflows/cicd-main.yml)
  8. [![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/release/python-3110/)
  9. [![Contributions Welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg)](CONTRIBUTING.md)
  10. [![GitHub Stars](https://img.shields.io/github/stars/NVIDIA-NeMo/Automodel.svg?style=social&label=Star)](https://github.com/NVIDIA-NeMo/Automodel/stargazers/)
  11. <!-- **Day-0 integration with Hugging Face models automating fine-tuning and pretraining with pytorch-native parallelism, custom-kernels and optimized recipes**
  12. **Pytorch DTensor‑native SPMD library for large‑scale training**-->
  13. [πŸ“– Documentation](https://docs.nvidia.com/nemo/automodel/latest/index.html) β€’ [πŸ”₯ Ready-to-Use Recipes](https://github.com/NVIDIA-NeMo/Automodel/#supported-models) β€’ [πŸ’‘ Examples](https://github.com/NVIDIA-NeMo/Automodel/tree/main/examples) β€’ [Model Coverage](https://docs.nvidia.com/nemo/automodel/latest/model-coverage/overview.html) β€’ [Performance](https://docs.nvidia.com/nemo/automodel/latest/performance-summary.html) β€’ [🀝 Contributing](https://github.com/NVIDIA-NeMo/Automodel/blob/main/CONTRIBUTING.md)
  14. </div>
  15. ## πŸ“£ News and Discussions
  16. - [11/12/2026][**Inkling-2-Small**](https://huggingface.co/thinkingmachines/Inkling-2-Small) We added fine-tuning recipes for Thinking Machines' 594B-parameter, 17B-active vision-language MoE model on GB200. Check out the [128K packed Tulu 3 SFT EP32/PP2/CP32](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/inkling2/inkling2_tulu3_packed128k_ep32_pp2_cp32.yaml), [MedPix LoRA EP32](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/inkling2/inkling2_medpix_lora_ep32.yaml), and [MedPix SFT EP64/PP4](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/inkling2/inkling2_medpix_sft_ep64_pp4.yaml) recipes and the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/thinkingmachines/inkling2.mdx).
  17. - [11/04/2026][**REINFORCE-Zero**](https://github.com/hijkzzz/REINFORCE-Zero) We added a [train-inference consistency skill](https://github.com/NVIDIA-NeMo/Automodel/blob/main/.agents/contributor-skills/train-inference-consistency/SKILL.md): the trainer calls the rollout engine's own kernels, so its logprobs match vLLM's bitwise and the train-inference mismatch is exactly 0 (`mismatch` 0, `vllm_kl` 0). [labs-molt](https://github.com/NVIDIA-NeMo/labs-molt) has the end-to-end implementation and an example. It follows REINFORCE-Zero by [@hijkzzz](https://github.com/hijkzzz): with aligned kernels, asynchronous RL stays stable with plain importance sampling, no clipping.
  18. - [11/07/2026][**Kimi K3.1 Pro**](https://huggingface.co/moonshotai/Kimi-K3.1-Pro) We now support full-parameter fine-tuning for Moonshot AI's 2.8T-parameter, 104B-active MoE vision-language model with hybrid KDA and gated DSA attention on NVIDIA GB200, many thanks to [@Khazic](https://github.com/khazic). Check out the [MedPix EP32/PP8 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/kimi/k31_medpix_nonpacked.yaml) on 256 GB200 and the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/moonshotai/kimi-k31-pro.mdx).
  19. - [11/01/2026][**Muown optimizer**](https://arxiv.org/abs/2605.10797) We now support the Muown optimizer for dense and MoE models with FSDP2 and expert parallelism. Check out the [Qwen3-30B-A3B recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_moe_30b_tulu3_muown.yaml).
  20. - [10/29/2026][**Mistral Large 4**](https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B) Fine-tune Mistral AI's 1T-A52B multimodal MoE model with expert, pipeline, and context parallelism. See the [model card](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/mistralai/mistral-large-4.mdx). Explore the [MedPix SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral/mistral_large4_medpix_ep64_pp2.yaml) and [128K long-context packed SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral/mistral_large4_tulu3_packed128k_ep64_pp2_cp32_tilelang.yaml) recipes.
  21. - [10/25/2026][**Nanbeige4.3-33B**](https://huggingface.co/Nanbeige/Nanbeige4.3-33B) Fine-tune Nanbeige's dense 33B agentic model on 128K packed sequences with context parallelism. See the [model card](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/nanbeige/nanbeige4-3-33b.mdx). Explore the [128K SFT CP16](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nanbeige/nanbeige4_3_33b_tulu3_packed128k_cp16.yaml) and [128K LoRA CP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nanbeige/nanbeige4_3_33b_tulu3_packed128k_cp8_lora.yaml) recipes.
  22. - [10/23/2026][**NVIDIA Nemotron 3.5 Bolt**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-BF16) Fine-tune [NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-BF16) with the [MedPix full-parameter SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_bolt/nemotron_3_5_bolt_e4b.yaml) and [MedPix LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_bolt/nemotron_3_5_bolt_e4b_peft.yaml) recipes, and [NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16) with the [CORD-v2 LoRA recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_bolt/nemotron_3_5_bolt_cord_v2_lora.yaml). See the [E4B](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/nvidia/nvidia-nemotron-3-5-bolt-e4b-4gb-bf16.mdx) and [74B-A9B](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/nvidia/nvidia-nemotron-3-5-bolt-74b-a9b-bf16.mdx) model coverage pages.
  23. - [10/23/2026][**MiniMax-M3.1-Flash**](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash) We now support fine-tuning MiniMax AI's MoE vision-language model, loading its NVFP4 and MXFP8 checkpoint directly. Check out the [MedPix SFT EP128 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/minimax_m3/minimax_m3_1_vl_sft_medpix_ep128_100steps.yaml) and the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/MiniMaxAI/minimax-m3-1-flash.mdx).
  24. - [10/21/2026][**Flint**](https://huggingface.co/RecursiveSI/flint) Fine-tune Recursive's 39.8B-A2.5B MoE model with expert and context parallelism. See the [model card](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/recursivesi/flint.mdx). Explore the [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/flint/flint_tulu3_sft.yaml), [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/flint/flint_tulu3_lora.yaml), [128K packed TE](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/flint/flint_tulu3_packed128k_te.yaml), and [128K packed MagiAttention (H100)](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/flint/flint_tulu3_packed128k_magi_h100.yaml) recipes.
  25. - [10/21/2026][**EAGLE-4**](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/speculative/eagle.mdx) We now support training EAGLE-4 drafters: the EAGLE-3.1 draft with a sliding-window layer and a low-rank candidate head over the full vocabulary, trained in two stages and exported to the `LlamaForCausalLMEagle4` serving layout, many thanks to [@Khazic](https://github.com/khazic). Check out the Qwen3-8B [draft](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/speculative/eagle4/qwen3_8b_eagle4.yaml) and [candidate-head](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/speculative/eagle4/qwen3_8b_eagle4_candidate_head.yaml) recipes and the [speculative decoding overview](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/speculative/README.md).
  26. - [10/16/2026][**Beam-1.0-501B-A23B**](https://huggingface.co/reflection/Beam-1.0-501B-A23B) We now support LoRA and full-parameter fine-tuning for Reflection AI's 501B-A23B sparse MoE with mixed sliding-window and full attention, using FSDP2, expert parallelism, and packed sequences. Check out the [Tulu3 LoRA EP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/beam/beam_1_0_501b_a23b_tulu3_lora_ep8.yaml) and [Tulu3 SFT EP64 8-node](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/beam/beam_1_0_501b_a23b_tulu3_ep64_8nodes.yaml) recipes and the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/reflection/beam-1-0-501b-a23b.mdx).
  27. - [10/15/2026][**Muse Spark Open**](https://huggingface.co/meta-models/Muse-Spark-open-816B-A42B) Fine-tune Meta's MoE model with expert, pipeline, and context parallelism. See the [model card](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/muse/muse_spark_open.mdx). Explore the [image-text SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/9c9c4cfdb27461da359263bb342a086307ecaba7/examples/vlm_finetune/muse_spark/muse_spark_open_medpix_nonpacked4k_ep64_pp4_100steps.yaml), [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/9c9c4cfdb27461da359263bb342a086307ecaba7/examples/vlm_finetune/muse_spark/muse_spark_open_medpix_nonpacked4k_lora_ep32.yaml), and [128K long-context packed SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/9c9c4cfdb27461da359263bb342a086307ecaba7/examples/llm_finetune/muse_spark/muse_spark_open_tulu3_packed128k_ep32_pp2_cp32_magi.yaml) recipes.
  28. - [10/15/2026][**NVIDIA Nemotron 3.5 Super VL**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) Fine-tune the 120B-A12B vision-language model with [CORD-v2 SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_cord_v2.yaml), [CORD-v2 PEFT (LoRA)](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_cord_v2_peft.yaml), [256K packed SFT (GB200)](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_tulu3_packed_256k_cp8_ep16_gb200.yaml). See the [CORD-v2 tutorial](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/vlm/nemotron-3-5-super-vl.mdx) for data preparation, training, and evaluation.
  29. - [10/14/2026][**Step-5-Preview-BF16**](https://huggingface.co/stepfun-ai/Step-5-Preview-BF16) We now support StepFun's 600B-A27B sliding-window and sparse-attention MoE with FSDP2, expert, pipeline and context parallelism. Check out the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/stepfun-ai/step-5-preview-bf16.mdx) and the [Tulu3 SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/stepfun/step_5_preview_tulu3_sft.yaml), [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/stepfun/step_5_preview_tulu3_lora.yaml), and [128K long-context CP32](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/stepfun/step_5_preview_tulu3_packed128k_cp32.yaml) recipes.
  30. - [10/12/2026][**FLUX 3 Image Klein Base**](https://docs.nvidia.com/nemo/automodel/nightly/model-coverage/diffusion/black-forest-labs/flux-3-image-klein-base) We added text-to-image and image-editing LoRA fine-tuning for Black Forest Labs' base image model. See the [T2I LoRA recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/flux3_t2i_flow_lora.yaml) and [model guide](https://docs.nvidia.com/nemo/automodel/nightly/model-coverage/diffusion/black-forest-labs/flux-3-image-klein-base) for checkpoint setup and the 100-step validation scope.
  31. - [10/04/2026][**HunyuanImage-3.0**](https://huggingface.co/tencent/HunyuanImage-3.0) We now support text-to-image full fine-tuning and LoRA for Tencent's 80B-A13B MoE multimodal decoder with FSDP2 and expert parallelism, many thanks to [@Khazic](https://github.com/khazic). Check out the [full fine-tuning](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/hunyuan_image3_t2i_flow.yaml) and [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/hunyuan_image3_t2i_flow_lora.yaml) recipes and the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/diffusion/tencent/hunyuan-image-3.mdx).
  32. - [09/28/2026][**Qwen-Image-2.1**](https://huggingface.co/Qwen/Qwen-Image-2.1) We now support text-to-image full fine-tuning and LoRA for Qwen's 7B single-stream DiT conditioned on Qwen3-VL. Check out the [full fine-tuning](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/qwen_image_21_t2i_flow.yaml) and [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/qwen_image_21_t2i_flow_lora.yaml) recipes and the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/diffusion/qwen/qwen-image-2-1.mdx).
  33. - [09/26/2026][**MiMo-V2.5-Pro**](https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro) We now support full-parameter fine-tuning for Xiaomi's hybrid full and sliding-window attention Mixture-of-Experts model from its FP8 checkpoint. Check out the [HellaSwag PP18/EP8 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v25/mimo_v25_pro_hellaswag.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/xiaomimimo/mimo-v2-5-pro.mdx).
  34. - [09/22/2026][**MiMo-V2.6-Flash-RL**](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) We now support full-parameter fine-tuning for Xiaomi's multimodal Mixture-of-Experts model with expert, pipeline, and context parallelism. Check out the [packed Tulu3 EP64/CP2 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v2_flash/mimo_v2_6_flash_rl_tulu3_packed4k_ep64_cp2_100steps.yaml), [non-packed MedPix EP64 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mimo_v2_flash/mimo_v2_6_flash_rl_medpix_nonpacked4k_ep64_100steps.yaml), and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/xiaomimimo/mimo-v2-flash.mdx).
  35. - [09/11/2026][**DeepSeek-V4.1-Flash**](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash) We now support SFT and LoRA for DeepSeek's causal encoder/decoder backbone with CSA2, single-pass mHC, and Engram, many thanks to [@Khazic](https://github.com/khazic). Check out the [HellaSwag EP64 16-node recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v41/deepseek_v41_flash_hellaswag_ep64_16nodes.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/deepseek-ai/deepseek-v41-flash.mdx).
  36. - [08/29/2026][**GLM-5.3**](https://huggingface.co/zai-org/GLM-5.3) We now support full-parameter fine-tuning of Z.ai's GLM-5.3 with cuDNN DSA and HybridEP. Check out the [EP64/PP4 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_5.3_tulu3_4k_cudnn_100step.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/thudm/glm5-moe-dsa.mdx).
  37. - [08/28/2026][**Ox Alpha / GLM-5.3-Flash**](https://huggingface.co/zai-org/GLM-5.3-Flash) We now support fine-tuning Z.ai's 320B-A18B hybrid-attention MoE VLM with packed CP and EP. Check out the [MedPix EP72/CP2 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/glm5_next/glm5_3_flash_medpix_packed2k_ep72_cp2_100steps.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/thudm/glm-5-3-flash.mdx).
  38. - [08/26/2026][**Qwen3.8-Flash-Next**](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) We now support full-parameter fine-tuning for the 180B hybrid-attention Engram MoE with FlexAttention sparse QSA, packed (THD) sequences, and model-owned context parallelism. Check out the [HellaSwag EP64 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_8_flash_next_180b_hellaswag_ep64.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/qwen/qwen3-8-flash-next.mdx).
  39. - [08/21/2026][**DFlash 2**](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/speculative/dflash.mdx) We now support training DFlash 2 drafters: the DFlash block-diffusion backbone plus a two-tap dynamic convolution and a pairwise path selector for higher acceptance lengths. Check out the [Qwen3-8B recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/speculative/dflash/qwen3_dflash2.yaml) and the [speculative decoding overview](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/speculative/README.md).
  40. - [08/14/2026][**Qwen3.8-27B**](https://huggingface.co/Qwen/Qwen3.8-27B) We now support full-parameter SFT and LoRA fine-tuning for Qwen's dense 27B vision-language model. Check out the [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_8/qwen3_8_27b.yaml) and [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_8/qwen3_8_27b_lora.yaml) recipes, [fine-tuning guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/vlm/qwen3-8.mdx), and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/qwen/qwen3-8-vl.mdx).
  41. - [08/12/2026][**Qwen3.8-2.4T-A95B**](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B) We now support full-parameter fine-tuning for `Qwen/Qwen3.8-2.4T-A95B` checkpoints. Check out the [HellaSwag EP32/PP8 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_8_2_4t_a95b_hellaswag_ep32_pp8.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/qwen/qwen3-8-2-4t-a95b.mdx).
  42. - [08/12/2026][**North Micro Vision**](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) We now support LoRA fine-tuning for Cohere Labs' 2.4B-parameter native-resolution vision-language model. Check out the [RDR recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/cohere_micro_vision/north_micro_vision_rdr.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/coherelabs/north-micro-vision.mdx).
  43. - [08/10/2026][**MuseGlimmer**](https://huggingface.co/meta-models/Muse-Glimmer-30B) We now support SFT/LoRA the dense 30B MuseGlimmer vision-language model, including TP/CP packed sequence recipes. Check out the [recipes](https://github.com/NVIDIA-NeMo/Automodel/tree/main/examples/vlm_finetune/muse_glimmer) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/muse/muse_glimmer.mdx).
  44. - [08/08/2026][**Nemotron 3.5 Lightning**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16) We now support LoRA fine-tuning for NVIDIA's 30B-A3B hybrid MoE model with multi-token prediction. Check out the [HellaSwag recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nemotron/nemotron_nano_v3_5_lightning_hellaswag_peft.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/nvidia/nemotron-h.mdx).
  45. - [07/30/2026][**Inkling-Small**](https://huggingface.co/thinkingmachines/Inkling-Small) We now support full-parameter fine-tuning for the 276B-parameter, 12B-active Inkling-Small model on 64 H100 GPUs. Check out the [MedPix EP64 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/inkling/Inkling_small_medpix_ep64.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/thinkingmachines/inkling.mdx).
  46. - [07/29/2026][**Kimi K3**](https://huggingface.co/moonshotai/Kimi-K3) We now support full-parameter fine-tuning for Moonshot AI's 2.8T-parameter MoE model on NVIDIA GB200. Check out the [HellaSwag EP32/PP8 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/kimi/k3_hellaswag.yaml) on 256 GB200.
  47. - [07/21/2026][**Laguna S 2.1**](https://huggingface.co/poolside/Laguna-S-2.1) We now support finetuning Poolside's 118B-A8B Laguna S 2.1 MoE model. Check out our [HellaSwag EP16 recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/laguna/laguna_s_2p1_hellaswag_ep16.yaml).
  48. - [07/18/2026][**Inkling VLM MoE**](https://huggingface.co/thinkingmachines/Inkling) We now support finetuning Inkling. Check out our [MedPix recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/inkling/inkling_medpix.yaml).
  49. - [06/27/2026] <img src="https://raw.githubusercontent.com/NVIDIA-NeMo/Automodel/refs/heads/main/docs/assets/speculative-decoding-spark.svg" width="136" height="24" alt="animated speculative decoding marker" /> [**Speculative decoding in NeMo AutoModel**](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/speculative/eagle.mdx) Train target-aligned drafters end to end: EAGLE-1/2/3, P-EAGLE, DFlash, and DeepSeek's newly released [DSpark](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/speculative/dspark/qwen3_0.6b_dspark.yaml). Huge thanks to [@Khazic](https://github.com/khazic) for the speculative decoding stack and [@kashif](https://github.com/kashif) for DSpark support.
  50. - [06/21/2026][**GLM-5.2**](https://huggingface.co/zai-org/GLM-5.2) We now support finetuning `zai-org/GLM-5.2` with IndexShare DSA, optional TileLang sparse kernels, and long-context CP recipes. Check out our [32K long-context recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_5.2_tulu3_32k_tilelang_cp8.yaml) and [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/llm/thudm/glm5-moe-dsa.mdx).
  51. - [06/15/2026][**DiffusionGemma**](https://huggingface.co/google/diffusiongemma-26B-A4B-it). We now support fine-tuning the `google/diffusiongemma-26B-A4B-it` model. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/dllm_sft/diffusion_gemma_sft.yaml) and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/dllm/diffusiongemma.mdx).
  52. - [06/12/2026][**MiniMax M3**](https://huggingface.co/MiniMaxAI/MiniMax-M3) We now support finetuning MiniMax's MiniMax-M3. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/minimax_m3/minimax_m3_vl_sft_ep32pp4.yaml) and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/vlm/minimax-m3.mdx).
  53. - [06/04/2026][**Nemotron-3 Ultra**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16) We now support finetuning NVIDIA's Nemotron 3 Ultra 550B A55B. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nemotron/nemotron_ultra_v3_hellaswag_peft.yaml) and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/llm/nemotron-3-ultra.md).
  54. - [06/03/2026][**Gemma 4 12B**](https://huggingface.co/google/gemma-4-12B) We now support finetuning the dense `google/gemma-4-12B` model. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_4_12b_hellaswag.yaml).
  55. - [05/27/2026][**Step-3.7-Flash**](https://huggingface.co/stepfun-ai/Step-3.7-Flash) We added model coverage for Stepfun AI's 198B-A13B MoE vision-language model, targeting image/video agentic developer workflows with a 256k context language backbone and 1.8B ViT vision tower. See the [model coverage page](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/stepfun-ai/step-3-7.md).
  56. - [05/19/2026][**Ling 2.0**](https://huggingface.co/collections/inclusionAI/ling-20) We now support finetuning the inclusionAI Ling 2.0 MoE family (`inclusionAI/Ling-mini-2.0`, `inclusionAI/Ling-flash-2.0`, and `inclusionAI/Ling-1T`), thanks to [@Hayden727](https://github.com/Hayden727). Check out our [recipes](https://github.com/NVIDIA-NeMo/Automodel/tree/main/examples/llm_finetune/ling).
  57. - [05/17/2026][**ERNIE 4.5**](https://huggingface.co/baidu) and [**MiMo-V2-Flash**](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) We now support finetuning `baidu/ERNIE-4.5-0.3B-PT`, `baidu/ERNIE-4.5-21B-A3B-PT`, and `XiaomiMiMo/MiMo-V2-Flash`. Check out our ERNIE [dense recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ernie4_5/ernie4_5_0p3b_hellaswag.yaml), ERNIE [MoE recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ernie4_5/ernie4_5_21b_a3b_hellaswag.yaml), and MiMo [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v2_flash/mimo_v2_flash_hellaswag.yaml).
  58. - [04/29/2026][**Mistral Medium 3.5**](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B) We now support finetuning Mistral AI's 128B FP8-native VLM Mistral Medium 3.5. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral3p5/mistral3p5_128b_medpix.yaml) and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/vlm/mistral-medium-3-5.md).
  59. - [04/28/2026][**Nemotron-3-Nano-Omni**](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16) We now support finetuning `nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16`, NVIDIA's 30B-A3B omnimodal MoE (text Β· image Β· audio) with NemotronH hybrid Mamba+Attention backbone. Check out our [SFT recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_omni/nemotron_omni_cord_v2.yaml), [LoRA recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_omni/nemotron_omni_cord_v2_peft.yaml), and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/vlm/nemotron-omni.md).
  60. - [04/28/2026][**Hy3-preview**](https://huggingface.co/tencent/Hy3-preview) We now support finetuning `tencent/Hy3-preview`, thanks to [@Khazic](https://github.com/khazic). Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/hy_v3/hy3_preview_deepep.yaml).
  61. - [04/25/2026][**DeepSeek V4 Flash**](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) We now support finetuning `deepseek-ai/DeepSeek-V4-Flash`, thanks to [@Khazic](https://github.com/khazic). Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v4/deepseek_v4_flash_packed_sequence_hellaswag.yaml) and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/llm/dsv4-flash.md).
  62. - [04/22/2026][**Qwen3.6-27B**](https://huggingface.co/Qwen/Qwen3.6-27B) We now support finetuning `Qwen/Qwen3.6-27B`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_6_27b.yaml).
  63. - [04/20/2026][**Qwen-Image**](https://huggingface.co/Qwen/Qwen-Image) We now support finetuning `Qwen/Qwen-Image`, thanks to [@harshareddy832](https://github.com/harshareddy832). Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/qwen_image_t2i_flow.yaml).
  64. - [04/16/2026][**Qwen3.6 MoE**](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) We now support finetuning `Qwen/Qwen3.6-35B-A3B`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_6_35b.yaml).
  65. - [04/16/2026][**LLaVA-OneVision-1.5**](https://huggingface.co/lmms-lab/LLaVA-OneVision-1.5-4B-Instruct) We now support finetuning `lmms-lab/LLaVA-OneVision-1.5-4B-Instruct`, thanks to [@vgauraha62](https://github.com/vgauraha62). Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/llava_onevision/llava_ov_1_5_4b_finetune.yaml).
  66. - [04/12/2026][**MiniMax-M2.7**](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) We now support finetuning `MiniMaxAI/MiniMax-M2.7`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/minimax_m2/minimax_m2.7_hellaswag_pp.yaml).
  67. - [04/07/2026][**GLM-5.1**](https://huggingface.co/zai-org/GLM-5.1) We now support finetuning `zai-org/GLM-5.1`. GLM-5.1 is Zhipu AI's latest open-source MoE model featuring MLA + DeepSeek Sparse Attention. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_5.1_hellaswag_pp.yaml) and [discussion](https://github.com/NVIDIA-NeMo/Automodel/discussions/1719).
  68. - [04/02/2026][**Gemma 4**](https://huggingface.co/collections/google/gemma-4) We support fine-tuning for Gemma4 (2B, 4B, 31B, 26BA4B)! Check out our [recipes](https://github.com/NVIDIA-NeMo/Automodel/tree/main/examples/vlm_finetune/gemma4).
  69. - [03/30/2026]**NeMo AutoModel** ships with **agent-friendly skills** in [skills/](https://github.com/NVIDIA-NeMo/Automodel/tree/main/skills) to help you with common development tasks (e.g., running a recipe, model onboarding, development) across the repo. We welcome PRs that improve existing skills or add new ones.
  70. - [03/16/2026][**Mistral Small 4**](https://huggingface.co/mistralai/Mistral-Small-4-119B-2603) We support fine-tuning for Mistral4 119B! Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/mistral4/mistral4_medpix.yaml).
  71. - [03/11/2026][**Nemotron Super v3**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) We support fine-tuning for `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nemotron/nemotron_super_v3_hellaswag.yaml).
  72. - [03/11/2026][**GLM-5**](https://huggingface.co/zai-org/GLM-5) We now support finetuning `zai-org/GLM-5`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_5_hellaswag_pp.yaml).
  73. - [03/02/2026][**Qwen3.5 small models**](https://huggingface.co/collections/Qwen/qwen35) We support finetuning for Qwen3.5 small models 0.8B, 2B, 4B ([recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_4b.yaml)) and 9B ([recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_5_9b.yaml))
  74. - [02/16/2026][**Qwen3.5 MoE**](https://huggingface.co/collections/Qwen/qwen35) We support finetuning for `Qwen/Qwen3.5-397B-A17B` ([recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_moe_medpix.yaml)) and `Qwen/Qwen3.5-35B-A3B` ([recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_35b.yaml))
  75. <details>
  76. <summary>Previous News</summary>
  77. - [02/13/2026] [**MiniMax-M2.5**](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) We support finetuning for `MiniMaxAI/MiniMax-M2.5`. Checkout our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/minimax_m2/minimax_m2.5_hellaswag_pp.yaml)
  78. - [02/11/2026] [**GLM-4.7-Flash**](https://huggingface.co/zai-org/GLM-4.7-Flash) We now support finetuning GLM-4.7-Flash. Checkout our [packed sequence recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_4.7_flash_te_packed_sequence.yaml)
  79. - [02/09/2026] [**MiniMax-M2**](https://huggingface.co/MiniMaxAI/MiniMax-M2) We support finetuning for `MiniMaxAI/MiniMax-M2`. Checkout our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/5f63eb428bacf4146e9a5ae9949d58c5751df7b9/examples/llm_finetune/minimax_m2/minimax_m2.1_hellaswag_pp.yaml)
  80. - [02/06/2026] [**Qwen3 VL 235B**](https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct) We support finetuning for `Qwen/Qwen3-VL-235B-A22B-Instruct`. Checkout our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3/qwen3_vl_moe_235b.yaml)
  81. - [02/06/2026] [**GLM4.7**](https://huggingface.co/zai-org/GLM-4.7) We now support finetuning GLM4.7. Checkout our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_4.7_te_deepep.yaml)
  82. - [02/06/2026] [**Step3.5-flash**](https://huggingface.co/stepfun-ai/Step-3.5-Flash) is out! Finetune it with our [finetune recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/stepfun/step_3.5_flash_hellaswag_pp.yaml)
  83. - [02/05/2026] [**DeepSeek-V3.2**](https://huggingface.co/deepseek-ai/DeepSeek-V3.2) is out! Checkout out [the finetune recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v32/deepseek_v32_hellaswag_pp.yaml)!
  84. - [02/04/2026] [**Kimi K2.5 VL**](https://huggingface.co/moonshotai/Kimi-K2.5) is out! Finetune it with [NeMo AutoModel](https://github.com/NVIDIA-NeMo/Automodel/discussions/1161)
  85. - [01/30/2026] [**Kimi VL**](https://huggingface.co/moonshotai/Kimi-VL-A3B-Instruct) We support fine-tuning for `moonshotai/Kimi-VL-A3B-Instruct`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/kimi/kimi2vl_cordv2.yaml).
  86. - [01/12/2026] [**Nemotron Flash**](https://huggingface.co/nvidia/Nemotron-Flash-1B) We support fine-tuning for `nvidia/Nemotron-Flash-1B`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nemotron_flash/nemotron_flash_1b_squad.yaml).
  87. - [01/12/2026] [**Nemotron Parse**](https://huggingface.co/nvidia/NVIDIA-Nemotron-Parse-v1.1) We support fine-tuning `nvidia/NVIDIA-Nemotron-Parse-v1.1` ([recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron/nemotron_parse_v1_1.yaml), [tutorial](https://github.com/NVIDIA-NeMo/Automodel/blob/main/tutorials/nemotron-parse/finetune.ipynb) and [try on Brev](https://brev.nvidia.com/launchable/deploy/now?launchableID=env-3C6LDKU2DfOvpVTFhjw3YQ4djPM)).
  88. - [01/07/2026] [**Devstral-Small**](https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512) We support fine-tuning for `mistralai/Devstral-Small-2-24B-Instruct-2512`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/devstral/devstral2_small_2512_squad.yaml).
  89. - [12/18/2025] [**FunctionGemma**](https://huggingface.co/google/functiongemma-270m-it) is out! Finetune it with [NeMo AutoModel](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/llm/toolcalling.md)!
  90. - [12/15/2025] [**NVIDIA-Nemotron-3-Nano-30B-A3B**](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8) is out! Finetune it with [NeMo AutoModel](https://github.com/NVIDIA-NeMo/Automodel/discussions/976)!
  91. - [11/6/2025] [Accelerating Large-Scale Mixture-of-Experts Training in PyTorch](https://developer.nvidia.com/blog/accelerating-large-scale-mixture-of-experts-training-in-pytorch/)
  92. - [10/6/2025] [Enabling PyTorch Native Pipeline Parallelism for πŸ€— Hugging Face Transformer Models](https://github.com/NVIDIA-NeMo/Automodel/discussions/589)
  93. - [9/22/2025] [Fine-tune Hugging Face Models Instantly with Day-0 Support with NVIDIA NeMo AutoModel](https://github.com/NVIDIA-NeMo/Automodel/discussions/477)
  94. - [9/18/2025] [πŸš€ NeMo Framework Now Supports Google Gemma 3n: Efficient Multimodal Fine-tuning Made Simple](https://github.com/NVIDIA-NeMo/Automodel/discussions/494)
  95. </details>
  96. ## Overview
  97. Nemo AutoModel is a Pytorch DTensor‑native SPMD open-source training library under [NVIDIA NeMo Framework](https://github.com/NVIDIA-NeMo), designed to streamline and scale training and finetuning for LLMs, VLMs, diffusion models, and retrieval models. Designed for flexibility, reproducibility, and scale, NeMo AutoModel enables both small-scale experiments and massive multi-GPU, multi-node deployments for fast experimentation in research and production environments.
  98. <p align="center">
  99. <a href="https://github.com/NVIDIA-NeMo/Automodel"><picture>
  100. <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/NVIDIA-NeMo/Automodel/refs/heads/main/docs/automodel_diagram.png">
  101. <img alt="AutoModel Logo" src="https://raw.githubusercontent.com/NVIDIA-NeMo/Automodel/refs/heads/main/docs/automodel_diagram.png">
  102. </picture></a>
  103. </p>
  104. What you can expect:
  105. - **Hackable** with a modular design that allows easy integration, customization, and quick research prototypes.
  106. - **Minimal ceremony**: YAML-driven recipes; override any field using CLI.
  107. - **High performance and flexibility** with custom kernels and DTensor support.
  108. - **Seamless integration** with Hugging Face for day-0 model support, ease of use, and wide range of supported models.
  109. - **Efficient resource management** using Kubernetes and Slurm, enabling scalable and flexible deployment across configurations.
  110. - **Documentation** with step-by-step guides and runnable examples.
  111. <!-- Please refer to our design documents for more details on the architecture and design philosophy. -->
  112. <!-- NeMo Framework is NVIDIA's GPU accelerated, end-to-end training framework for large language models (LLMs), multi-modal models and speech models. It enables seamless scaling of training (both pretraining and post-training) workloads from single GPU to thousand-node clusters for both πŸ€—Hugging Face/PyTorch and Megatron models. It includes a suite of libraries and recipe collections to help users train models from end to end. The **AutoModel library ("NeMo AutoModel")** provides GPU-accelerated PyTorch training for πŸ€—Hugging Face models on **Day-0**. Users can start training and fine-tuning models instantly without conversion delays, scale effortlessly with PyTorch-native parallelisms, optimized custom kernels, and memory-efficient recipes-all while preserving the original checkpoint format for seamless use across the Hugging Face ecosystem. -->
  113. ### Why PyTorch Distributed and SPMD
  114. - **One program, any scale**: The same training script runs on 1 GPU or 1000+ by changing the mesh.
  115. - **PyTorch Distributed native**: Partition model/optimizer states with `DeviceMesh` + placements (`Shard`, `Replicate`).
  116. - **SPMD first**: Parallelism is configuration. No model rewrites when scaling up or changing strategy.
  117. - **Decoupled concerns**: Model code stays pure PyTorch; parallel strategy lives in config.
  118. - **Composability**: Mix **tensor**, **sequence**, and **data** parallel by editing placements.
  119. - **Portability**: Fewer bespoke abstractions; easier to reason about failure modes and restarts.
  120. <!-- - **Interoperability**: HF models/tokenizers/optimizers plug in directly; no format round‑trips. -->
  121. <!-- ### Key Features -->
  122. <!-- - **Mesh‑defined parallelism**: Compose tensor/sequence/pipeline/data parallel by changing placements and sizes. -->
  123. <!-- - **FSDP2 on DTensor**: Memory‑efficient sharding (HSDP included) for large scale training. -->
  124. <!-- - **Pretraining, SFT & PEFT**: Day‑0 support for LLMs both regimes with shared configs/utilities.
  125. - **Mixed precision**: BF16/FP16/FP8; sequence packing; optimized CUDA kernels. -->
  126. <!-- - **Mesh‑aware DCP**: Sharded SafeTensors with merge/reshard utilities; interoperable with HF. -->
  127. <!-- - **Large-Scale Distributed Training**: Built-in FSDP2 and Megatron-FSDP for seamless multi-node scaling. -->
  128. <!-- - **Vision-Language Model Ready**: Native support for VLMs (Qwen2-VL, Gemma-3-VL, etc). -->
  129. <!-- - **Day-0 Hugging Face Support**: Instantly fine-tune any model from the Hugging Face Hub. -->
  130. ## Table of Contents
  131. - [Feature Roadmap](#feature-roadmap)
  132. - [Getting Started](#getting-started)
  133. - [LLM](#llm-pre-training)
  134. - [Pre-training](#llm-pre-training)
  135. - [Supervised Fine-Tuning (SFT)](#llm-supervised-fine-tuning-sft)
  136. - [Parameter-Efficient Fine-Tuning (PEFT)](#llm-parameter-efficient-fine-tuning-peft)
  137. - [VLM](#vlm-supervised-fine-tuning-sft)
  138. - [Supervised Fine-Tuning (SFT)](#vlm-supervised-fine-tuning-sft)
  139. - [Parameter-Efficient Fine-Tuning (PEFT)](#vlm-parameter-efficient-fine-tuning-peft)
  140. - [Supported Models](#supported-models)
  141. - [Performance](#performance)
  142. - [Interoperability](#-interoperability)
  143. - [Contributing](#-contributing)
  144. - [License](#-license)
  145. > TL;DR: SPMD turns β€œhow to parallelize” into a *runtime layout choice*, not a code fork.
  146. ## Feature Roadmap
  147. βœ… _Implemented on `main`_ | πŸ”œ _Planned_
  148. For released features, see the [release notes](https://github.com/NVIDIA-NeMo/Automodel/releases). Docker examples target the [26.10 container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo-automodel/tags?version=26.10.00).
  149. High-throughput scalable training
  150. - βœ… **PyTorch DTensor-native SPMD training** Same training script can scale from 1 GPU to large multi-node jobs by changing the device mesh/config.
  151. - βœ… **Composable Parallelism** - PyTorch native FSDP2, HSDP, TP, CP, SP and PP for distributed training.
  152. - βœ… **Optimized kernels** - Uses NVIDIA-oriented kernel paths such as Transformer Engine, DeepEP, FlexAttn, TorchSDPA, fused attention, rotary embeddings, Triton, and optional kernel patches.
  153. - βœ… **MoE acceleration** - Includes MoE routing and DeepEP integration, plus expert-parallel configurations used in DeepSeek, Qwen MoE, GPT-OSS, and Nemotron MoE benchmarks.
  154. - βœ… **FP8 and mixed precision** - FP8 support with torchao and Transformer Engine.
  155. - βœ… **MXFP8 MoE training** - Transformer Engine and torchao MXFP8 grouped-expert training on GB200.
  156. - βœ… **Activation checkpointing** - Trades recomputation for lower activation memory, especially useful with FSDP and memory-efficient losses.
  157. - βœ… **Memory-efficient loss** - Linear-Cut / fused linear cross entropy avoids materializing full logits for the loss, reducing output-layer memory pressure.
  158. - βœ… **Sequence packing** - Packs variable-length examples together to reduce padding compute and improve GPU utilization.
  159. - βœ… **FlashAttention packed-sequence support** - Packed masks can feed variable-length FlashAttention paths using per-document cu_seqlens.
  160. - βœ… **DCP** - Supports PyTorch DCP and SafeTensors, sharded and consolidated layouts, merge/reshard utilities, and Hugging Face-compatible outputs.
  161. - βœ… **Async checkpointing** - Can write checkpoints in the background to reduce training stalls caused by I/O.
  162. - βœ… **Dion, Muon, and Muown optimizers** - Distributed optimizer integrations with typed recipe configuration.
  163. - βœ… **Environment Support** - SLURM, interactive, SkyPilot, and Kubernetes (via SkyPilot) launchers.
  164. SOTA algorithms
  165. - βœ… **Pre-training** - Support for model pre-training, including DeepSeekV3.
  166. - βœ… **Learning Algorithms** - SFT (Supervised Fine-Tuning), PEFT (LoRA, QLoRA), and QAT (Quantization-Aware Training).
  167. - βœ… **Knowledge distillation** - Support for knowledge distillation with LLMs.
  168. - βœ… **VLM knowledge distillation** - Chunked KD loss and a Qwen3.5-VL teacher-student recipe.
  169. - βœ… **Speculative decoding training** - Train target-aligned EAGLE-1/2/3/4, P-EAGLE, DFlash, DFlash 2, and DSpark drafters for faster verified generation.
  170. Model Coverage and πŸ€— Ecosystem compatibility
  171. - βœ… **Transformers v5 πŸ€—** - Built on latest transformers with device-mesh driven parallelism.
  172. - βœ… **πŸ€— HuggingFace Integration** - Works with dense models (e.g., Qwen, Llama3, etc) and large MoEs (e.g., DSv3, DSv4).
  173. - βœ… **VLM** - Finetuning for VLMs (Qwen2.5/3/3.5/3.6 VL, Gemma-3/3n/4 VL, Mistral 3.5/4, LLaVA-OneVision-1.5, Kimi-VL, etc.).
  174. - βœ… **Omnimodal** - Finetuning for omnimodal MoE models (Nemotron-3-Nano-Omni, Qwen3-Omni).
  175. - βœ… **Diffusion** - Pretraining and LoRA finetuning for image/video diffusion models (Qwen-Image, Qwen-Image-2.1, FLUX, [FLUX 3 Image Klein Base](https://docs.nvidia.com/nemo/automodel/nightly/model-coverage/diffusion/black-forest-labs/flux-3-image-klein-base), Wan2.1, Wan2.2-T2V-A14B, Hunyuan).
  176. - βœ… **dLLM** - Discrete diffusion LM finetuning (LLaDA, LLaDA2, Nemotron-Labs-Diffusion, DiffusionGemma).
  177. - βœ… **Retrieval** - Bi-encoder and cross-encoder training with in-batch negative sampling.
  178. - βœ… **Extended MoE support** - GPT-OSS, Kimi K3, Qwen3 / Qwen3.5 / Qwen3.6 MoE, Qwen-next, MiniMax-M2.x, GLM-4.7 / GLM-5 / GLM-5.1 / GLM-5.2 / GLM-5.3 / GLM-5.3-Flash, DeepSeek V3.2 / V4 / V4-Flash, ERNIE 4.5, MiMo-V2-Flash / MiMo-V2.5-Pro, Ling 2.0, Hy3-preview.
  179. Agentic Development and UX
  180. - βœ… **Agent-friendly skills** - Curated [`skills/`](https://github.com/NVIDIA-NeMo/Automodel/tree/main/skills) for common dev tasks (recipe runs, model onboarding, CI).
  181. Planned
  182. - πŸ”œ **Unified Engine API and recipes** - Introduce a common engine and consolidate the LLM and VLM recipe paths.
  183. - πŸ”œ **Composable component configuration** - Complete the typed config and `.build()` refactor across data and remaining components.
  184. - πŸ”œ **Packed long-context training with CP** - Combine THD sequence packing with context parallelism, including DeepSeek V4 coverage.
  185. - πŸ”œ **Kernel and runtime upgrades** - Add partial CUDA graphs, evaluate FlashAttention 3/4, and upgrade to DeepEP v2.
  186. - πŸ”œ **Multimodal retrieval** - Expand vision-language retrieval datasets and models and improve retriever performance.
  187. - πŸ”œ **Convergence and regression testing** - Add automated loss, evaluation, and performance gates to NeMo CI.
  188. - πŸ”œ **Transformers 5.12 alignment** - Upgrade the Transformers integration and restore affected model parity coverage.
  189. - πŸ”œ **Recipe consistency** - Align diffusion recipes with the shared AutoModel recipe and configuration patterns.
  190. ## Getting Started
  191. Run a 10-step Qwen3-0.6B fine-tuning example on two GPUs with the container:
  192. ```bash
  193. mkdir -p checkpoints
  194. docker run --gpus all -it --rm --shm-size=8g \
  195. -v "$PWD/checkpoints:/opt/Automodel/checkpoints" \
  196. nvcr.io/nvidia/nemo-automodel:26.10.00 \
  197. automodel /opt/Automodel/examples/llm_finetune/qwen/qwen3_0p6b_hellaswag.yaml \
  198. --nproc-per-node 2 \
  199. --step_scheduler.max_steps 10
  200. ```
  201. The model and HellaSwag dataset are public; no Hugging Face token is required. Checkpoints are saved to the mounted `checkpoints` directory.
  202. For source installation, use **uv** for reproducible Python environments.
  203. ```bash
  204. # Setup environment before running any recipes
  205. uv venv
  206. # Choose ONE:
  207. uv sync --frozen # LLM recipes (default)
  208. # uv sync --frozen --extra vlm --extra vlm-media # VLM recipes (Qwen/Mistral/Omni need vlm-media for video/vision; fixes: ImportError: qwen_vl_utils is not installed)
  209. # uv sync --frozen --extra cuda # Optional CUDA deps (e.g., Transformer Engine, Mamba SSM)
  210. # uv sync --frozen --extra cuda_source # Optional bitsandbytes dependency
  211. # uv sync --frozen --extra all # Most optional deps (includes `vlm` and `cuda`; NOTE: excludes media β€” add --extra media for video/image decode)
  212. # uv sync --frozen --all-extras # Everything (includes `fa`, `moe`, `media`, etc.)
  213. # One-off runs (examples):
  214. # uv run --extra vlm <command>
  215. # uv run --extra cuda <command>
  216. uv run python -c "import nemo_automodel; print('NeMo AutoModel ready')"
  217. ```
  218. ### Run a Recipe
  219. All recipes are launched via the `automodel` CLI (or its short alias `am`). Each YAML config specifies the recipe class and all training parameters:
  220. ```bash
  221. # LLM example: multi-GPU fine-tuning of Qwen3-0.6B with FSDP2 (no HF token required)
  222. automodel examples/llm_finetune/qwen/qwen3_0p6b_hellaswag.yaml --nproc-per-node 8
  223. # VLM example: single-GPU fine-tuning (Gemma-3-VL) with LoRA
  224. automodel examples/vlm_finetune/gemma3/gemma3_vl_4b_cord_v2_peft.yaml
  225. # Both commands also work with uv run:
  226. uv run automodel examples/llm_finetune/qwen/qwen3_0p6b_hellaswag.yaml --nproc-per-node 8
  227. ```
  228. > [!TIP]
  229. > **NeMo-Run submission:** The `cli` extra adds NeMo Run to the base package: `uv pip install "nemo-automodel[cli]"`. It is additive; the base package still installs its core training dependencies, including PyTorch.
  230. ## LLM Pre-training
  231. ### LLM Pre-training Single Node
  232. We provide an example SFT experiment using the [FineWeb dataset](https://arxiv.org/abs/2406.17557/) with a nano-GPT model, ideal for quick experimentation on a single node.
  233. ```sh
  234. automodel examples/llm_pretrain/nanogpt_pretrain.yaml --nproc-per-node 8
  235. ```
  236. <!-- ### LLM Pre-training Multi Node -->
  237. ## LLM Supervised Fine-Tuning (SFT)
  238. We provide an example SFT experiment using the [HellaSwag dataset](https://rowanzellers.com/hellaswag/).
  239. <!-- Refer to `examples/llm_finetune/annotated.yaml` for a full list of parameters that can be overridden. -->
  240. ### LLM SFT Single Node
  241. The default SFT configuration is set to run on a single GPU. To start the experiment:
  242. ```sh
  243. automodel examples/llm_finetune/qwen/qwen3_0p6b_hellaswag.yaml
  244. ```
  245. This fine-tunes [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) on HellaSwag using a single GPU. The model and dataset are public; no Hugging Face token is required.
  246. To use multiple GPUs on a single node, add the `--nproc-per-node` argument:
  247. ```sh
  248. automodel examples/llm_finetune/qwen/qwen3_0p6b_hellaswag.yaml --nproc-per-node 8
  249. ```
  250. ### LLM SFT Multi Node
  251. To launch on a SLURM cluster, copy the reference sbatch script and adapt it to your cluster:
  252. ```sh
  253. cp slurm.sub my_cluster.sub
  254. # Edit my_cluster.sub β€” change CONFIG, #SBATCH directives, container, mounts, etc.
  255. sbatch my_cluster.sub
  256. ```
  257. All cluster-specific settings (nodes, GPUs, partition, container, mounts) live in your sbatch script.
  258. NeMo-Run (`nemo_run:`) sections are also supported -- see our
  259. [cluster guide](https://docs.nvidia.com/nemo/automodel/latest/launcher/cluster.html) for details.
  260. ## LLM Parameter-Efficient Fine-Tuning (PEFT)
  261. We provide a PEFT example using the [HellaSwag dataset](https://rowanzellers.com/hellaswag/).
  262. ### LLM PEFT Single Node
  263. ```bash
  264. # Memory-efficient SFT with LoRA
  265. automodel examples/llm_finetune/qwen/qwen3_0p6b_hellaswag_peft.yaml
  266. # Override any YAML parameter via the command line:
  267. automodel examples/llm_finetune/qwen/qwen3_0p6b_hellaswag_peft.yaml \
  268. --step_scheduler.local_batch_size 16
  269. ```
  270. > [!NOTE]
  271. > Launching a multi-node PEFT example uses the same `sbatch slurm.sub` workflow as the SFT case above.
  272. ## VLM Supervised Fine-Tuning (SFT)
  273. We provide a VLM SFT example using Qwen2.5-VL for end-to-end fine-tuning on image-text data.
  274. ### VLM SFT Single Node
  275. ```bash
  276. # Qwen2.5-VL on 8 GPUs
  277. automodel examples/vlm_finetune/qwen2_5/qwen2_5_vl_3b_rdr.yaml --nproc-per-node 8
  278. ```
  279. ## VLM Parameter-Efficient Fine-Tuning (PEFT)
  280. We provide a VLM PEFT (LoRA) example for memory-efficient adaptation with Gemma3 VLM.
  281. ### VLM PEFT Single Node
  282. ```bash
  283. # Gemma-3-VL PEFT on 8 GPUs
  284. automodel examples/vlm_finetune/gemma3/gemma3_vl_4b_medpix_peft.yaml --nproc-per-node 8
  285. ```
  286. ## Supported Models
  287. NeMo AutoModel provides native support for a wide range of models available on the Hugging Face Hub, enabling efficient fine-tuning for various domains. Below is a small sample of ready-to-use families (train as-is or swap any compatible πŸ€— causal LM), you can specify nearly any LLM/VLM model available on πŸ€— hub:
  288. | Domain | Model Family | Model ID | Recipes |
  289. |--------|--------------|----------|---------|
  290. | **LLM** | **GPT-OSS** | [`GPT-OSS-20B`](https://huggingface.co/openai/gpt-oss-20b) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gpt_oss/gpt_oss_20b_te_packed_sequence.yaml) |
  291. | | | [`GPT-OSS-120B`](https://huggingface.co/openai/gpt-oss-120b) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gpt_oss/gpt_oss_120b.yaml) |
  292. | **LLM** | **DeepSeek** | [`DeepSeek-V3`](https://huggingface.co/deepseek-ai/DeepSeek-V3) | [Pretrain](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_pretrain/deepseekv3_pretrain.yaml) |
  293. | **LLM** | **Kimi K3** | [`moonshotai/Kimi-K3`](https://huggingface.co/moonshotai/Kimi-K3) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/kimi/k3_hellaswag.yaml) |
  294. | **LLM** | **GLM-5.3** | [`zai-org/GLM-5.3`](https://huggingface.co/zai-org/GLM-5.3) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/glm/glm_5.3_tulu3_4k_cudnn_100step.yaml) |
  295. | **LLM** | **Moonlight** | [`Moonlight-16B-TE`](https://huggingface.co/moonshotai/Moonlight-16B-A3B) | [Pretrain](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_pretrain/megatron_pretrain_moonlight_16b_te_slurm.yaml), [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/moonlight/moonlight_16b_te.yaml) |
  296. | **LLM** | **Nemotron-H** | [`nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16) | [LoRA](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/nemotron/nemotron_nano_v3_5_lightning_hellaswag_peft.yaml) |
  297. | **LLM** | **Ling 2.0** | [`inclusionAI/Ling-mini-2.0`](https://huggingface.co/inclusionAI/Ling-mini-2.0) | [LoRA SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ling/ling_mini_2_0_squad.yaml), [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ling/ling_mini_2_0_sft.yaml) |
  298. | | | [`inclusionAI/Ling-flash-2.0`](https://huggingface.co/inclusionAI/Ling-flash-2.0) | [LoRA SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ling/ling_flash_2_0_lora.yaml), [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ling/ling_flash_2_0_sft.yaml) |
  299. | | | [`inclusionAI/Ling-1T`](https://huggingface.co/inclusionAI/Ling-1T) | [LoRA SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ling/ling_1t_lora_pp.yaml), [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ling/ling_1t_sft.yaml) |
  300. | **LLM** | **ERNIE 4.5** | [`baidu/ERNIE-4.5-0.3B-PT`](https://huggingface.co/baidu/ERNIE-4.5-0.3B-PT) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ernie4_5/ernie4_5_0p3b_hellaswag.yaml) |
  301. | | | [`baidu/ERNIE-4.5-21B-A3B-PT`](https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-PT) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/ernie4_5/ernie4_5_21b_a3b_hellaswag.yaml) |
  302. | **LLM** | **MiMo V2 Flash** | [`XiaomiMiMo/MiMo-V2-Flash`](https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v2_flash/mimo_v2_flash_hellaswag.yaml) |
  303. | **LLM** | **MiMo-V2.5-Pro** | [`XiaomiMiMo/MiMo-V2.5-Pro`](https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mimo_v25/mimo_v25_pro_hellaswag.yaml) |
  304. | **LLM** | **LLaMA** | [`meta-llama/Llama-3.2-1B`](https://huggingface.co/meta-llama/Llama-3.2-1B) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_2/llama3_2_1b_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_2/llama3_2_1b_hellaswag_peft.yaml) |
  305. | | | [`meta-llama/Llama-3.2-3B-Instruct`](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_2/llama_3_2_3b_instruct_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_2/llama_3_2_3b_instruct_squad_peft.yaml) |
  306. | | | [`meta-llama/Llama-3.1-8B`](https://huggingface.co/meta-llama/Llama-3.1-8B) | [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_1/llama3_1_8b_hellaswag_fp8.yaml) |
  307. | | | [`meta-llama/Llama-3.3-70B-Instruct`](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_3/llama_3_3_70b_instruct_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/llama3_3/llama_3_3_70b_instruct_squad_peft.yaml) |
  308. | **LLM** | **Mistral** | [`mistralai/Mistral-7B-v0.1`](https://huggingface.co/mistralai/Mistral-7B-v0.1) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mistral_7b_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mistral_7b_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mistral_7b_hellaswag_fp8.yaml) |
  309. | | | [`mistralai/Mistral-Nemo-Base-2407`](https://huggingface.co/mistralai/Mistral-Nemo-Base-2407) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mistral_nemo_2407_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mistral_nemo_2407_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mistral_nemo_2407_hellaswag_fp8.yaml) |
  310. | | | [`mistralai/Mixtral-8x7B-Instruct-v0.1`](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mixtral-8x7b-v0-1_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/mistral/mixtral-8x7b-v0-1_squad_peft.yaml) |
  311. | **LLM** | **Qwen** | [`Qwen/Qwen2.5-7B`](https://huggingface.co/Qwen/Qwen2.5-7B) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen2_5_7b_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen2_5_7b_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen2_5_7b_hellaswag_fp8.yaml) |
  312. | | | [`Qwen/Qwen3-0.6B`](https://huggingface.co/Qwen/Qwen3-0.6B) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_0p6b_hellaswag.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_0p6b_hellaswag_peft.yaml) |
  313. | | | [`Qwen/Qwen3.8-2.4T-A95B`](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_8_2_4t_a95b_hellaswag_ep32_pp8.yaml) |
  314. | | | [`Qwen/Qwen3.8-Flash-Next`](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwen3_8_flash_next_180b_hellaswag_ep64.yaml) |
  315. | | | [`Qwen/QwQ-32B`](https://huggingface.co/Qwen/QwQ-32B) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwq_32b_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/qwen/qwq_32b_squad_peft.yaml) |
  316. | **LLM** | **Gemma** | [`google/gemma-3-270m`](https://huggingface.co/google/gemma-3-270m) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_3_270m_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_3_270m_squad_peft.yaml) |
  317. | | | [`google/gemma-2-9b-it`](https://huggingface.co/google/gemma-2-9b-it) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_2_9b_it_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_2_9b_it_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_2_9b_it_hellaswag_fp8.yaml) |
  318. | | | [`google/gemma-7b`](https://huggingface.co/google/gemma-7b) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_7b_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/gemma/gemma_7b_squad_peft.yaml) |
  319. | **LLM** | **Phi** | [`microsoft/phi-2`](https://huggingface.co/microsoft/phi-2) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_2_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_2_squad_peft.yaml) |
  320. | | | [`microsoft/Phi-3-mini-4k-instruct`](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_3_mini_it_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_3_mini_it_squad_peft.yaml) |
  321. | | | [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_4_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_4_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/phi/phi_4_hellaswag_fp8.yaml) |
  322. | **LLM** | **Seed** | [`ByteDance-Seed/Seed-Coder-8B-Instruct`](https://huggingface.co/ByteDance-Seed/Seed-Coder-8B-Instruct) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/seed/seed_coder_8b_instruct_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/seed/seed_coder_8b_instruct_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/seed/seed_coder_8b_instruct_hellaswag_fp8.yaml) |
  323. | | | [`ByteDance-Seed/Seed-OSS-36B-Instruct`](https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Instruct) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/seed/seed_oss_36B_hellaswag.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/seed/seed_oss_36B_hellaswag_peft.yaml) |
  324. | **LLM** | **Baichuan** | [`baichuan-inc/Baichuan2-7B-Chat`](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/baichuan/baichuan_2_7b_squad.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/baichuan/baichuan_2_7b_squad_peft.yaml), [FP8](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/baichuan/baichuan_2_7b_mock_fp8.yaml) |
  325. | **VLM** | **NVIDIA Nemotron 3.5 Super VL** | [`nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) | [CORD-v2 SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_cord_v2.yaml), [CORD-v2 PEFT (LoRA)](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_cord_v2_peft.yaml), [256K packed SFT (GB200)](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/nemotron_3_5_super_vl/nemotron_3_5_super_vl_tulu3_packed_256k_cp8_ep16_gb200.yaml) |
  326. | **VLM** | **Gemma** | [`google/gemma-3-4b-it`](https://huggingface.co/google/gemma-3-4b-it) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma3/gemma3_vl_4b_cord_v2.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma3/gemma3_vl_4b_cord_v2_peft.yaml) |
  327. | | | [`google/gemma-3n-e4b-it`](https://huggingface.co/google/gemma-3n-e4b-it) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma3n/gemma3n_vl_4b_medpix.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/gemma3n/gemma3n_vl_4b_medpix_peft.yaml) |
  328. | **VLM** | **Ox Alpha / GLM-5.3-Flash** | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) | [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/glm5_next/glm5_3_flash_medpix_packed2k_ep72_cp2_100steps.yaml) |
  329. | **VLM** | **Muse Spark Open** | [`meta-models/Muse-Spark-open-816B-A42B`](https://huggingface.co/meta-models/Muse-Spark-open-816B-A42B) | [Model card](docs/model-coverage/vlm/muse/muse_spark_open.mdx), [SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/9c9c4cfdb27461da359263bb342a086307ecaba7/examples/vlm_finetune/muse_spark/muse_spark_open_medpix_nonpacked4k_ep64_pp4_100steps.yaml), [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/9c9c4cfdb27461da359263bb342a086307ecaba7/examples/vlm_finetune/muse_spark/muse_spark_open_medpix_nonpacked4k_lora_ep32.yaml), [128K packed SFT](https://github.com/NVIDIA-NeMo/Automodel/blob/9c9c4cfdb27461da359263bb342a086307ecaba7/examples/llm_finetune/muse_spark/muse_spark_open_tulu3_packed128k_ep32_pp2_cp32_magi.yaml) |
  330. | **VLM** | **North Micro Vision** | [`CohereLabs/North-Micro-Vision-Instruct`](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) | [PEFT](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/cohere_micro_vision/north_micro_vision_rdr.yaml) |
  331. | **Embedding** | **Nemotron 3 Embed** | [`nvidia/Nemotron-3-Embed-1B-BF16`](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16) | [End-to-end text bi-encoder recipe](https://github.com/NVIDIA-NeMo/Nemotron/tree/main/docs/nemotron/embed) |
  332. > [!NOTE]
  333. > Check out more [LLM](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune) and [VLM](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune) examples. Any causal LM on Hugging Face Hub can be used with the base recipe template, just overwrite `--model.pretrained_model_name_or_path <model-id>` in the CLI or in the YAML config.
  334. ## Performance
  335. NeMo AutoModel achieves great training performance on NVIDIA GPUs. Below are highlights from our benchmark results:
  336. | Model | #GPUs | Seq Length | Model TFLOPs/sec/GPU | Tokens/sec/GPU | Kernel Optimizations |
  337. |-------|------:|-----------:|---------------------:|---------------:|----------------------|
  338. | DeepSeek V3 671B | 256 | 4096 | 250 | 1,002 | TE + DeepEP |
  339. | GPT-OSS 20B | 8 | 4096 | 279 | 13,058 | TE + DeepEP + FlexAttn |
  340. | Qwen3 MoE 30B | 8 | 4096 | 212 | 11,842 | TE + DeepEP |
  341. For complete benchmark results including configuration details, see the [Performance Summary](docs/performance-summary.mdx).
  342. <!--
  343. ## Mesh‑Aware Checkpointing
  344. AutoModel writes **Distributed Checkpoints (DCP)** with SafeTensors
  345. shards. Checkpoints carry partition metadata to:
  346. - **Merge** into a single HF‑compatible checkpoint for inference.
  347. - **Reshard** when loading onto a different mesh/topology.
  348. YAML sketch:
  349. ```yaml
  350. checkpoint:
  351. enabled: true
  352. checkpoint_dir: ./checkpoints
  353. save_consolidated: final
  354. model_save_format: safetensors
  355. ``` -->
  356. ## πŸ”Œ Interoperability
  357. - **[NeMo RL](https://github.com/NVIDIA-NeMo/RL)**: Use AutoModel checkpoints directly as starting points for DPO/RM/GRPO pipelines.
  358. - **[Hugging Face](https://github.com/huggingface/transformers)**: Train any LLM/VLM from πŸ€— without format conversion.
  359. - **[Megatron Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge)**: Optional conversions to/from Megatron formats for specific workflows.
  360. ## πŸ—‚οΈ Project Structure
  361. ```
  362. NeMo-Automodel/
  363. β”œβ”€β”€ cli/ # `automodel` / `am` CLI entry-point
  364. β”‚ └── app.py
  365. β”œβ”€β”€ docker/ # Container build files
  366. β”œβ”€β”€ docs/ # Documentation and guides
  367. β”œβ”€β”€ examples/
  368. β”‚ β”œβ”€β”€ convergence/ # Convergence test configs
  369. β”‚ β”œβ”€β”€ diffusion/ # Diffusion pretrain/finetune configs
  370. β”‚ β”œβ”€β”€ dllm_sft/ # Discrete diffusion LM SFT configs
  371. β”‚ β”œβ”€β”€ dllm_generate/ # Discrete diffusion LM generation
  372. β”‚ β”œβ”€β”€ llm_benchmark/ # LLM benchmarking configs
  373. β”‚ β”œβ”€β”€ llm_finetune/ # LLM finetune YAML configs
  374. β”‚ β”œβ”€β”€ llm_kd/ # LLM knowledge-distillation configs
  375. β”‚ β”œβ”€β”€ llm_pretrain/ # LLM pretrain configs
  376. β”‚ β”œβ”€β”€ llm_seq_cls/ # LLM sequence classification configs
  377. β”‚ β”œβ”€β”€ retrieval/ # Bi-encoder / cross-encoder configs
  378. β”‚ β”œβ”€β”€ vlm_benchmark/ # VLM benchmarking configs
  379. β”‚ β”œβ”€β”€ vlm_finetune/ # VLM finetune configs
  380. β”‚ └── vlm_generate/ # VLM generation configs
  381. β”œβ”€β”€ nemo_automodel/
  382. β”‚ β”œβ”€β”€ _diffusers/ # HF Diffusers integration (NeMoAutoDiffusionPipeline)
  383. β”‚ β”œβ”€β”€ _transformers/ # HF Transformers integration
  384. β”‚ β”œβ”€β”€ components/ # Core library
  385. β”‚ β”‚ β”œβ”€β”€ _peft/ # PEFT implementations (LoRA, QLoRA)
  386. β”‚ β”‚ β”œβ”€β”€ attention/ # Attention implementations
  387. β”‚ β”‚ β”œβ”€β”€ checkpoint/ # Distributed checkpointing
  388. β”‚ β”‚ β”œβ”€β”€ config/
  389. β”‚ β”‚ β”œβ”€β”€ datasets/ # LLM, VLM, diffusion, retrieval datasets
  390. β”‚ β”‚ β”œβ”€β”€ distributed/ # FSDP2, Megatron FSDP, pipelining, CP, etc.
  391. β”‚ β”‚ β”œβ”€β”€ launcher/ # Launcher backends (SLURM, NeMo-Run, SkyPilot)
  392. β”‚ β”‚ β”œβ”€β”€ loggers/ # Loggers
  393. β”‚ β”‚ β”œβ”€β”€ loss/ # Optimized loss functions
  394. β”‚ β”‚ β”œβ”€β”€ models/ # User-defined model examples
  395. β”‚ β”‚ β”œβ”€β”€ moe/ # Optimized kernels for MoE models
  396. β”‚ β”‚ β”œβ”€β”€ optim/ # Optimizer/LR scheduler components (incl. Dion)
  397. β”‚ β”‚ β”œβ”€β”€ quantization/ # FP8, QAT, QLoRA
  398. β”‚ β”‚ β”œβ”€β”€ training/ # Train utils
  399. β”‚ β”‚ └── utils/ # Misc utils
  400. β”‚ β”œβ”€β”€ recipes/
  401. β”‚ β”‚ β”œβ”€β”€ llm/ # Main LLM train loop
  402. β”‚ β”‚ β”œβ”€β”€ vlm/ # Main VLM train loop
  403. β”‚ β”‚ β”œβ”€β”€ diffusion/ # Diffusion training loop
  404. β”‚ β”‚ β”œβ”€β”€ dllm/ # Discrete diffusion LM training loop
  405. β”‚ β”‚ └── retrieval/ # Retrieval / biencoder training loop
  406. β”‚ └── shared/
  407. β”œβ”€β”€ tools/ # Developer tooling
  408. └── tests/ # Comprehensive test suite
  409. ```
  410. ## Citation
  411. If you use NeMo AutoModel in your research, please cite it using the following BibTeX entry:
  412. ```
  413. @misc{nemo-automodel,
  414. title = {NeMo AutoModel: DTensor-native SPMD library for scalable and efficient training},
  415. howpublished = {\url{https://github.com/NVIDIA-NeMo/Automodel}},
  416. year = {2025--2026},
  417. note = {GitHub repository},
  418. }
  419. ```
  420. ## 🀝 Contributing
  421. We welcome contributions! Please see our [Contributing Guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/CONTRIBUTING.md) for details.
  422. ## πŸ“„ License
  423. NVIDIA NeMo AutoModel is licensed under the [Apache License 2.0](https://github.com/NVIDIA-NeMo/Automodel/blob/main/LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.