Skip to content
Back to search

bionemo-recipes

BioNeMo Recipes: For building and adapting AI models in drug discovery at scale

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/NVIDIA-BioNeMo/bionemo-recipes with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/NVIDIA-BioNeMo/bionemo-recipes'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/nvidia-bionemo--bionemo-recipes/badge.svg)](/listings/nvidia-bionemo--bionemo-recipes)

HTML

<a href="/listings/nvidia-bionemo--bionemo-recipes"><img src="/api/listings/nvidia-bionemo--bionemo-recipes/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

This section describe how you can get started with BioNeMo Recipes.

Loading Models

Run the following to load the BioNeMo model.

from transformers import AutoModel, AutoTokenizer

# Load a BioNeMo model directly from Hugging Face
model = AutoModel.from_pretrained("nvidia/AMPLIFY_120M")
tokenizer = AutoTokenizer.from_pretrained("nvidia/AMPLIFY_120M")

Running Recipes

Build and run recipes with the following.

# Navigate to a recipe
cd recipes/esm2_native_te

# Build and run
docker build -t esm2_recipe .
docker run --rm -it --gpus all esm2_recipe python train.py

For a realized multi-stage computational biology example, see the [agent-free PhiX174 whole-genome SFT and GDPO workflow](recipes/evo2_phage_gen/examples/README.md). It provides one top-level command for an 8×H100 run, with public-input preparation, current safety screens, checkpoint selection, monitoring, generation, selected-SFT likelihood ranking with a residual length-bias check, and final reporting.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 257 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # BioNeMo Recipes
  2. BioNeMo Recipes provides an easy path for the biological foundation model training community to scale up transformer-based models efficiently. Rather than offering a batteries-included training framework, BioNeMo Recipes provide **model checkpoints** with TransformerEngine (TE) layers and **training recipes** that demonstrate how to achieve maximum throughput with popular open-source frameworks and fully sharded data parallel (FSDP) scale-out.
  3. ## Overview
  4. The biological AI community actively prototypes model architectures and needs tooling that prioritizes extensibility, interoperability, and ease-of-use, alongside performance. BioNeMo Recipes addresses this by offering:
  5. - **Flexible scaling**: Scales from single-GPU prototyping to multi-node training without complex parallelism configurations
  6. - **Framework compatibility**: Works with popular frameworks like HuggingFace Accelerate, PyTorch Lightning, and vanilla PyTorch
  7. - **Performance optimization**: Leverages TransformerEngine and megatron-FSDP for state-of-the-art training efficiency
  8. - **Research-friendly**: Contains hackable and readable code that researchers can easily adapt for their experiments
  9. ### Performance Benchmarks
  10. <p align="center">
  11. <img src="docs/docs/assets/images/esm2/esm2_native_te_benchmarks.svg" alt="ESM2 native TE benchmarks chart showing performance results" width="600">
  12. <br>
  13. <em> Training benchmarks for ESM-2 using the <code>esm2_native_te</code> recipe.</em>
  14. </p>
  15. ### Use Cases
  16. The use cases of BioNeMo Recipes include:
  17. - **Foundation Model Developers**: AI researchers and ML engineers developing novel biological foundation models who need to scale up prototypes efficiently
  18. - **Foundation Model Customizers**: Domain scientists looking to fine-tune existing models with proprietary data for drug discovery and biological research
  19. ## Supported Recipes and Models
  20. | Directory | Description | FSDP | BF16 | FP8<sup>[1]</sup> | THD | FP8 + THD | MXFP8<sup>[2]</sup> | NVFP4<sup>[3]</sup> | CP |
  21. | ------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- | ------------ | ---- | ----------------- | --- | --------- | ------------------- | ------------------- | --- |
  22. | `models/amplify`,<br> [available on Hugging Face](https://huggingface.co/nvidia/AMPLIFY_350M) | TE accelerated protein BERT, [Amgen](https://www.biorxiv.org/content/10.1101/2024.09.23.614603v1) | ✅ | ✅ | ✅ | 🚧 | 🚧 | ❌ | ❌ | ❌ |
  23. | `models/esm2`,<br> [available on Hugging Face](https://huggingface.co/nvidia/esm2_t48_15B_UR50D) | TE accelerated protein BERT, [Meta](https://www.biorxiv.org/content/10.1101/2022.07.20.500902v1) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
  24. | `models/llama3` | TE accelerated Llama 3, [Meta](https://www.llama.com/models/llama-3/) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
  25. | `models/mixtral` | TE accelerated Mixtral-style MoE model | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🚧 |
  26. | `models/qwen` | TE accelerated Qwen2.5/Qwen3 model | 🚧 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
  27. | `recipes/`<br>`esm2_native_te` | Recipe for `esm2/amplify` + native PyTorch | mFSDP, FSDP2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
  28. | `recipes/`<br>`llama3_native_te` | Recipe for `llama3` + native PyTorch | FSDP2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
  29. | `recipes/`<br>`opengenome2_llama_native_te` | OpenGenome2 recipe for `llama3` + native PyTorch | FSDP2 | ✅ | ✅ | ✅ | ✅ | ✅ | 🚧 | ✅ |
  30. | `recipes/`<br>`codonfm_native_te` | Native PyTorch recipe for [CodonFM](https://research.nvidia.com/labs/dbr/assets/data/manuscripts/nv-codonfm-preprint.pdf) | FSDP2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
  31. | `recipes/`<br>`esm2_accelerate_te` | Recipe for `esm2/amplify` TE + HF Accelerate | FSDP, FSDP2 | ✅ | ✅ | 🚧 | 🚧 | 🚧 | 🚧 | ❌ |
  32. | `recipes/`<br>`codonfm_ptl_te` | PyTorch Lightning recipe for [CodonFM](https://research.nvidia.com/labs/dbr/assets/data/manuscripts/nv-codonfm-preprint.pdf) | FSDP | ✅ | 🚧 | ✅ | 🚧 | 🚧 | 🚧 | ❌ |
  33. | `recipes/`<br>`geneformer_native_te_mfsdp_fp8` | Recipe for geneformer HF model | mFSDP | ✅ | ✅ | 🚧 | 🚧 | 🚧 | 🚧 | ❌ |
  34. | `recipes/`<br>`vit` | Recipe for vision transformer | mFSDP | ✅ | 🚧 | ❌ | ❌ | ❌ | ❌ | ❌ |
  35. ✅: Supported <br/>
  36. 🚧: Under development, will be supported soon <br/>
  37. ❌: Not supported <br/>
  38. Abbreviations:
  39. - FSDP: Fully sharded data parallel. In this repository, we focus on pytorch native [FSDP2](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) and [megatron-FSDP](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core/distributed/fsdp/src) (mFSDP) support.
  40. - BF16: [brain-float 16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format), a common 16 bit float format for deep learning.
  41. - FP8<sup>[1]</sup>: [8-bit floating point](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html), a compact format for weights allowing for faster training and inference.
  42. - MXFP8<sup>[2]</sup>: [Multi Scale 8-bit floating point](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html), as compact as FP8 but with better numerical precision.
  43. - NVFP4<sup>[3]</sup>: [NVIDIA 4-bit floating point](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html#Beyond-FP8---training-with-NVFP4), faster than FP8, retaining accuracy using multi-scale.
  44. - THD: **T**otal **H**eads **D**imension, also known as ["sequence packing"](https://docs.nvidia.com/nemo-framework/user-guide/24.07/nemotoolkit/features/optimizations/sequence_packing.html#sequence-packing-for-sft-peft). A way to construct a batch with sequences of different lengths so there are no pads, which results in no compute wasted on computing attention for padding tokens. This is in contrast to **B**atch **S**equence **H**ead **D**imension (BSHD) format, which uses pads to create a rectangular batch.
  45. - CP: Context parallel, also known as sequence parallel. A way to distribute the memory required to process long sequences across multiple GPUs. For more information, refer to [context parallel](./recipes/context_parallel.md)
  46. \[1\]: Requires [compute capability](https://developer.nvidia.com/cuda-gpus) 9.0 and above (Hopper+) <br/>
  47. \[2\]: Requires [compute capability](https://developer.nvidia.com/cuda-gpus) 10.0 and 10.3 (Blackwell), 12.0 support pending <br/>
  48. \[3\]: Requires [compute capability](https://developer.nvidia.com/cuda-gpus) 10.0 and above (Blackwell+) <br/>
  49. ## Repository Structure
  50. This repository contains three types of components:
  51. ### Models (`models/`)
  52. Huggingface-compatible `PreTrainedModel` classes that use TransformerEngine layers internally. These are designed to be:
  53. - **Distributed through Hugging Face Hub**: Pre-converted checkpoints available at [huggingface.co/nvidia](https://huggingface.co/nvidia)
  54. - **Drop-in replacements**: Compatible with `AutoModel.from_pretrained()` without additional dependencies
  55. - **Performance optimized**: Leverage TransformerEngine features like FP8 training and context parallelism
  56. Example models include ESM-2, Geneformer, and AMPLIFY.
  57. ### Recipes (`recipes/`)
  58. Self-contained training examples demonstrating best practices for scaling biological foundation models. Each recipe is a complete Docker container with:
  59. - **Framework examples**: Vanilla PyTorch, HuggingFace Accelerate, PyTorch Lightning
  60. - **Feature demonstrations**: FP8 training, megatron-FSDP, context parallelism, sequence packing
  61. - **Scaling strategies**: Single-GPU to multi-node training patterns
  62. - **Benchmarked performance**: Validated throughput and convergence metrics
  63. Recipes are source-distributed reference implementations that users can adapt for their own research. Most are run directly;
  64. recipes that include package metadata, such as `nemotron-stitch`, document how to install from a checkout or immutable Git revision.
  65. ### Interpretability (`interpretability/`)
  66. Research tools and workflows for inspecting biological foundation models, including sparse autoencoder training,
  67. feature analysis, and model behavior exploration.
  68. ## Quick Start
  69. This section describe how you can get started with BioNeMo Recipes.
  70. ### Loading Models
  71. Run the following to load the BioNeMo model.
  72. ```python
  73. from transformers import AutoModel, AutoTokenizer
  74. # Load a BioNeMo model directly from Hugging Face
  75. model = AutoModel.from_pretrained("nvidia/AMPLIFY_120M")
  76. tokenizer = AutoTokenizer.from_pretrained("nvidia/AMPLIFY_120M")
  77. ```
  78. ### Running Recipes
  79. Build and run recipes with the following.
  80. ```bash
  81. # Navigate to a recipe
  82. cd recipes/esm2_native_te
  83. # Build and run
  84. docker build -t esm2_recipe .
  85. docker run --rm -it --gpus all esm2_recipe python train.py
  86. ```
  87. For a realized multi-stage computational biology example, see the
  88. [agent-free PhiX174 whole-genome SFT and GDPO workflow](recipes/evo2_phage_gen/examples/README.md).
  89. It provides one top-level command for an 8×H100 run, with public-input preparation, current safety
  90. screens, checkpoint selection, monitoring, generation, selected-SFT likelihood ranking with a
  91. residual length-bias check, and final reporting.
  92. ## Setting Up the Development Environment
  93. 1. Install pre-commit hooks:
  94. ```bash
  95. pre-commit install
  96. ```
  97. Run hooks manually:
  98. ```bash
  99. pre-commit run --all-files
  100. ```
  101. 2. **Test your changes:**
  102. Each model and recipe has its own build and test setup following this pattern:
  103. ```bash
  104. cd models/my_model # or recipes/my_recipe
  105. docker build . -t my_tag
  106. docker run --rm -it --gpus all my_tag pytest -v .
  107. ```
  108. ## Coding Guidelines
  109. BioNeMo Recipes prioritize **readability and simplicity** over comprehensive feature coverage:
  110. - **KISS (Keep It Simple) over DRY (Don't Repeat Yourself)**: It's better to have clear, duplicated code than complex
  111. abstractions
  112. - **One thing well**: Each recipe should demonstrate specific features clearly rather than trying to cover everything
  113. - **Self-contained**: Recipes cannot depend on cutting-edge code from other parts of the repository
  114. ### Testing Strategy
  115. BioNeMo Recipes use a three-tier testing approach:
  116. #### L0 Tests (Pre-merge)
  117. - **Purpose**: Fast validation that code works
  118. - **Runtime**: \<10 minutes, single GPU
  119. - **Frequency**: Run automatically on PRs
  120. - **Scope**: Basic functionality, checkpoint creation/loading
  121. #### L1 Tests (Performance Monitoring)
  122. - **Purpose**: Performance benchmarking and partial convergence validation
  123. - **Runtime**: Up to 4 hours, up to 16 GPUs
  124. - **Frequency**: Nightly/weekly
  125. - **Scope**: Throughput metrics, scaling validation
  126. #### L2 Tests (Release Validation)
  127. - **Purpose**: Full convergence and large-scale validation
  128. - **Runtime**: Multiple days, hundreds of GPUs
  129. - **Frequency**: Monthly or before releases
  130. - **Scope**: Complete model convergence, cross-platform validation
  131. ### Adding New Components
  132. With BioNeMo Recipes, you can add new components including models and recipes.
  133. #### Adding a New Model
  134. Models should be pip-installable packages that can export checkpoints to Hugging Face. Refer to the
  135. [models README](models/README.md) for detailed guidelines on:
  136. - Package structure and conventions
  137. - Checkpoint export procedures
  138. - Testing requirements
  139. - CI/CD integration
  140. #### Adding a New Recipe
  141. Recipes should be self-contained Docker environments demonstrating specific training patterns. Refer to
  142. the [recipes README](recipes/README.md) for guidance on:
  143. - Directory structure and naming
  144. - Hydra configuration management
  145. - Docker best practices
  146. - SLURM integration examples
  147. ### CI/CD Contract
  148. All components must pass this basic validation:
  149. ```bash
  150. docker build -t {component_tag} .
  151. docker run --rm -it --gpus all {component_tag} pytest -v .
  152. ```
  153. #### Running CI/CD
  154. To run the CI/CD pipeline locally, run the following command:
  155. ```bash
  156. ./ci/build_and_test.py
  157. ```
  158. ### Performance Expectations
  159. We aim to provide the fastest available training implementations for biological foundation models, with documented benchmarks across NVIDIA hardware (A100, H100, H200, B100, B200, etc.).
  160. ## Contributing
  161. We welcome contributions that advance the state of biological foundation model training. Ensure your contributions:
  162. - Follow our coding guidelines emphasizing clarity
  163. - Include appropriate tests (L0 minimum, L1/L2 as applicable)
  164. - Provide clear documentation and examples
  165. - Maintain compatibility with our supported frameworks
  166. For detailed contribution guidelines, refer to our individual component READMEs:
  167. - [Models Development Guide](models/README.md)
  168. - [Recipes Development Guide](recipes/README.md)
  169. ## License
  170. This project is licensed under the terms described in [LICENSE/license.txt](/LICENSE/license.txt).
  171. ## Support
  172. For technical support and questions:
  173. - Check existing issues before opening a new one
  174. - Review our training recipes for implementation examples
  175. - Consult the TransformerEngine and megatron-FSDP documentation for underlying technologies

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.