Skip to content
Back to search

nanochat

The best ChatGPT that $100 can buy.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/karpathy--nanochat/badge.svg)](/listings/karpathy--nanochat)

HTML

<a href="/listings/karpathy--nanochat"><img src="/api/listings/karpathy--nanochat/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Setup

nanochat uses [uv](https://docs.astral.sh/uv/) for dependency management. To install:

uv sync --extra gpu    # Use for CUDA (A100/H100/etc.)
uv sync --extra cpu    # (or) Use for CPU-only / MPS
source .venv/bin/activate

For development (adds pytest, matplotlib, ipykernel, transformers, etc.):

uv sync --extra gpu --group dev

Reproduce and talk to GPT-2

The most fun you can have is to train your own GPT-2 and talk to it. The entire pipeline to do so is contained in the single file [runs/speedrun.sh](runs/speedrun.sh), which is designed to be run on an 8XH100 GPU node. Boot up a new 8XH100 GPU box from your favorite provider (e.g. I use and like [Lambda](https://lambda.ai/service/gpu-cloud)), and kick off the training script:

bash runs/speedrun.sh

You may wish to do so in a screen session as this will take ~1.5 hours to run. Once it's done, you can talk to your model over the CLI. Make sure again that your local uv virtual environment is active (run `source .venv/bin/activate`), and chat:

python -m scripts.chat_cli

Get it to write stories or poems. Ask it to tell you who you are to see a hallucination. Ask it why the sky is blue. Or why it's green. The speedrun is a 4e19 FLOPs capability model so it's a bit like talking to a kindergartener :). An example conversation with a speedrun model:

> **User:** Hello whats up? > > **Assistant:** Hello! How can I help you today? > > **User:** Why is the sky blue? > > **Assistant:** The sky is blue due to an optical illusion called the Rayleigh Scattering of Light, which occurs when sunlight passes through tiny particles in the air. These particles are made up of tiny blue and violet particles that cause the light to bend in a particular way. > > **User:** That's cool. Write a poem about it. > > **Assistant:** In the twilight's glow, the sky is blue,<br> > A canvas of shimmering blue, where skies unfold,<br> > A world of blue, where blue and more blue,<br> > Are woven together in a tapestry so fine,<br> > Where every hue seems to sing a story.<br> > ...

A few more notes:

- The code will run just fine on the Ampere 8XA100 GPU node as well, but a bit slower. - All code will run just fine on even a single GPU by omitting `torchrun`, and will produce ~identical results (code will automatically switch to gradient accumulation), but you'll have to wait 8 times longer. - If your GPU(s) have less than 80GB, you'll have to tune some of the hyperparameters or you will OOM / run out of VRAM. Look for `--device-batch-size` in the scripts and reduce it until things fit. E.g. from 32 (default) to 16, 8, 4, 2, or even 1. Less than that you'll have to know a bit more what you're doing and get more creative. - Most of the code is fairly vanilla PyTorch so it should run on anything that supports that - xpu, mps, or etc, but I haven't personally exercised all of these code paths so there might be sharp edges.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 230 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # nanochat
  2. ![nanochat logo](dev/nanochat.png)
  3. ![scaling laws](dev/scaling_laws_jan26.png)
  4. nanochat is the simplest experimental harness for training LLMs. It is designed to run on a single GPU node, the code is minimal/hackable, and it covers all major LLM stages including tokenization, pretraining, finetuning, evaluation, and inference. For example, you can train your own GPT-2 capability LLM (which cost ~$43,000 to train in 2019) for only $48 (~2 hours of 8XH100 GPU node) and then talk to it over a simple CLI. On a spot instance, the total cost can be closer to ~$15. More generally, nanochat is configured out of the box to train an entire miniseries of compute-optimal models by setting one single complexity dial: `--depth`, the number of layers in the GPT transformer model (GPT-2 capability happens to be approximately depth 26). All other hyperparameters (the width of the transformer, number of heads, learning rate adjustments, training horizons, weight decays, ...) are calculated automatically in an optimal way.
  5. For questions about the repo, I recommend either using [DeepWiki](https://deepwiki.com/karpathy/nanochat) from Devin/Cognition to ask questions about the repo, or use the [Discussions tab](https://github.com/karpathy/nanochat/discussions), or come by the [#nanochat](https://discord.com/channels/1020383067459821711/1427295580895314031) channel on Discord.
  6. ## Time-to-GPT-2 Leaderboard
  7. Presently, the main focus of development is on tuning the pretraining stage, which takes the most amount of compute. Inspired by the modded-nanogpt repo and to incentivise progress and community collaboration, nanochat maintains a leaderboard for a "GPT-2 speedrun", which is the wall-clock time required to train a nanochat model to GPT-2 grade capability, as measured by the DCLM CORE score. The [runs/speedrun.sh](runs/speedrun.sh) script always reflects the reference way to train GPT-2 grade model and talk to it. The current leaderboard looks as follows:
  8. | # | time | val_bpb | CORE | Description | Date | Commit | Contributors |
  9. |---|-------------|---------|------|-------------|------|--------|--------------|
  10. | 0 | 168 hours | - | 0.2565 | Original OpenAI GPT-2 checkpoint | 2019 | - | OpenAI |
  11. | 1 | 3.04 | 0.74833 | 0.2585 | d24 baseline, slightly overtrained | Jan 29 2026 | 348fbb3 | @karpathy |
  12. | 2 | 2.91 | 0.74504 | 0.2578 | d26 slightly undertrained **+fp8** | Feb 2 2026 | a67eba3 | @karpathy |
  13. | 3 | 2.76 | 0.74645 | 0.2602 | bump total batch size to 1M tokens | Feb 5 2026 | 2c062aa | @karpathy |
  14. | 4 | 2.02 | 0.71854 | 0.2571 | change dataset to NVIDIA ClimbMix | Mar 4 2026 | 324e69c | @ddudek @karpathy |
  15. | 5 | 1.80 | 0.71808 | 0.2690 | autoresearch [round 1](https://x.com/karpathy/status/2031135152349524125) | Mar 9 2026 | 6ed7d1d | @karpathy |
  16. | 6 | 1.65 | 0.71800 | 0.2626 | autoresearch round 2 | Mar 14 2026 | a825e63 | @karpathy |
  17. The primary metric we care about is "time to GPT-2" - the wall clock time needed to outperform the GPT-2 (1.6B) CORE metric on an 8XH100 GPU node. The GPT-2 CORE score is 0.256525. In 2019, the training of GPT-2 cost approximately $43,000 so it is incredible that due to many advances over 7 years across the stack, we can now do so much faster and for well below $100 (e.g. at the current ~$3/GPU/hr, an 8XH100 node is ~$24/hr, so 2 hours is ~$48).
  18. See [dev/LEADERBOARD.md](dev/LEADERBOARD.md) for more docs on how to interpret and contribute to the leaderboard.
  19. ## Getting started
  20. ### Setup
  21. nanochat uses [uv](https://docs.astral.sh/uv/) for dependency management. To install:
  22. ```bash
  23. uv sync --extra gpu # Use for CUDA (A100/H100/etc.)
  24. uv sync --extra cpu # (or) Use for CPU-only / MPS
  25. source .venv/bin/activate
  26. ```
  27. For development (adds pytest, matplotlib, ipykernel, transformers, etc.):
  28. ```bash
  29. uv sync --extra gpu --group dev
  30. ```
  31. ### Reproduce and talk to GPT-2
  32. The most fun you can have is to train your own GPT-2 and talk to it. The entire pipeline to do so is contained in the single file [runs/speedrun.sh](runs/speedrun.sh), which is designed to be run on an 8XH100 GPU node. Boot up a new 8XH100 GPU box from your favorite provider (e.g. I use and like [Lambda](https://lambda.ai/service/gpu-cloud)), and kick off the training script:
  33. ```bash
  34. bash runs/speedrun.sh
  35. ```
  36. You may wish to do so in a screen session as this will take ~1.5 hours to run. Once it's done, you can talk to your model over the CLI. Make sure again that your local uv virtual environment is active (run `source .venv/bin/activate`), and chat:
  37. ```bash
  38. python -m scripts.chat_cli
  39. ```
  40. Get it to write stories or poems. Ask it to tell you who you are to see a hallucination. Ask it why the sky is blue. Or why it's green. The speedrun is a 4e19 FLOPs capability model so it's a bit like talking to a kindergartener :). An example conversation with a speedrun model:
  41. > **User:** Hello whats up?
  42. >
  43. > **Assistant:** Hello! How can I help you today?
  44. >
  45. > **User:** Why is the sky blue?
  46. >
  47. > **Assistant:** The sky is blue due to an optical illusion called the Rayleigh Scattering of Light, which occurs when sunlight passes through tiny particles in the air. These particles are made up of tiny blue and violet particles that cause the light to bend in a particular way.
  48. >
  49. > **User:** That's cool. Write a poem about it.
  50. >
  51. > **Assistant:** In the twilight's glow, the sky is blue,<br>
  52. > A canvas of shimmering blue, where skies unfold,<br>
  53. > A world of blue, where blue and more blue,<br>
  54. > Are woven together in a tapestry so fine,<br>
  55. > Where every hue seems to sing a story.<br>
  56. > ...
  57. A few more notes:
  58. - The code will run just fine on the Ampere 8XA100 GPU node as well, but a bit slower.
  59. - All code will run just fine on even a single GPU by omitting `torchrun`, and will produce ~identical results (code will automatically switch to gradient accumulation), but you'll have to wait 8 times longer.
  60. - If your GPU(s) have less than 80GB, you'll have to tune some of the hyperparameters or you will OOM / run out of VRAM. Look for `--device-batch-size` in the scripts and reduce it until things fit. E.g. from 32 (default) to 16, 8, 4, 2, or even 1. Less than that you'll have to know a bit more what you're doing and get more creative.
  61. - Most of the code is fairly vanilla PyTorch so it should run on anything that supports that - xpu, mps, or etc, but I haven't personally exercised all of these code paths so there might be sharp edges.
  62. ## Research
  63. If you are a researcher and wish to help improve nanochat, two scripts of interest are [runs/scaling_laws.sh](runs/scaling_laws.sh) and [runs/miniseries.sh](runs/miniseries.sh). See [Jan 7 miniseries v1](https://github.com/karpathy/nanochat/discussions/420) for related documentation. For quick experimentation (~5 min pretraining runs) my favorite scale is to train a 12-layer model (GPT-1 sized), e.g. like this:
  64. ```
  65. OMP_NUM_THREADS=1 torchrun --standalone --nproc_per_node=8 -m scripts.base_train -- \
  66. --depth=12 \
  67. --run="d12" \
  68. --model-tag="d12" \
  69. --core-metric-every=999999 \
  70. --sample-every=-1 \
  71. --save-every=-1 \
  72. ```
  73. This uses wandb (run name "d12"), only runs the CORE metric on last step, and it doesn't sample and save intermediate checkpoints. I like to change something in the code, re-run a d12 (or a d16 etc) and see if it helped, in an iteration loop. To see if a run helps, I like to monitor the wandb plots for:
  74. 1. `val_bpb` (validation loss in vocab-size-invariant units of bits per byte) as a function of `step`, `total_training_time` and `total_training_flops`.
  75. 2. `core_metric` (the DCLM CORE score)
  76. 3. VRAM utilization, `train/mfu` (Model FLOPS utilization), `train/tok_per_sec` (training throughput)
  77. See an example [here](https://github.com/karpathy/nanochat/pull/498#issuecomment-3850720044).
  78. The important thing to note is that nanochat is written and configured around one single dial of complexity - the depth of the transformer. This single integer automatically determines all other hyperparameters (the width of the transformer, number of heads, learning rate adjustments, training horizons, weight decays, ...) so that the trained model comes out compute optimal. The idea is that the user doesn't have to think about or set any of this, they are simply asking for a smaller or bigger model using `--depth`, and everything "just works". By sweeping out the depth, you achieve the nanochat miniseries of compute optimal models at various sizes. GPT-2 capability model (which is of most interest at the moment) happens to be somewhere around d24-d26 range with the current code. But any candidate changes to the repo have to be principled enough that they work for all settings of depth.
  79. ## Running on CPU / MPS
  80. The script [runs/runcpu.sh](runs/runcpu.sh) shows a very simple example of running on CPU or Apple Silicon. It dramatically shrinks the LLM that is being trained to make things fit into a reasonable time interval of a few ten minutes of training. You will not get strong results in this way.
  81. ## Precision / dtype
  82. nanochat does not use `torch.amp.autocast`. Instead, precision is managed explicitly through a single global `COMPUTE_DTYPE` (defined in `nanochat/common.py`). By default this is auto-detected based on your hardware:
  83. | Hardware | Default dtype | Why |
  84. |----------|--------------|-----|
  85. | CUDA SM 80+ (A100, H100, ...) | `bfloat16` | Native bf16 tensor cores |
  86. | CUDA SM < 80 (V100, T4, ...) | `float32` | No bf16; fp16 available via `NANOCHAT_DTYPE=float16` (uses GradScaler) |
  87. | CPU / MPS | `float32` | Safe default. On recent macOS, MPS also runs `NANOCHAT_DTYPE=bfloat16` fine (~25% less memory, similar speed) |
  88. You can override the default with the `NANOCHAT_DTYPE` environment variable:
  89. ```bash
  90. NANOCHAT_DTYPE=float32 python -m scripts.chat_cli -p "hello" # force fp32
  91. NANOCHAT_DTYPE=bfloat16 torchrun --nproc_per_node=8 -m scripts.base_train # force bf16
  92. ```
  93. How it works: model weights are stored in fp32 (for optimizer precision), but our custom `Linear` layer casts them to `COMPUTE_DTYPE` during the forward pass. Embeddings are stored directly in `COMPUTE_DTYPE` to save memory. This gives us the same mixed-precision benefit as autocast but with full explicit control over what runs in which precision.
  94. Note: `float16` training automatically enables a `GradScaler` in `base_train.py` to prevent gradient underflow. SFT supports this too but RL currently does not. Inference in fp16 works fine everywhere.
  95. ## Guides
  96. I've published a number of guides that might contain helpful information, most recent to least recent:
  97. - [Feb 1 2026: Beating GPT-2 for <<$100: the nanochat journey](https://github.com/karpathy/nanochat/discussions/481)
  98. - [Jan 7 miniseries v1](https://github.com/karpathy/nanochat/discussions/420) documents the first nanochat miniseries of models.
  99. - To add new abilities to nanochat, see [Guide: counting r in strawberry (and how to add abilities generally)](https://github.com/karpathy/nanochat/discussions/164).
  100. - [Oct 13 2025: original nanochat post](https://github.com/karpathy/nanochat/discussions/1) introducing nanochat, though now it contains some deprecated information and the model is a lot older (with worse results) than current master.
  101. ## File structure
  102. ```
  103. .
  104. ├── LICENSE
  105. ├── README.md
  106. ├── dev
  107. │ ├── nanochat.png
  108. │ └── repackage_data_reference.py # Pretraining data shard generation
  109. ├── nanochat
  110. │ ├── __init__.py # empty
  111. │ ├── checkpoint_manager.py # Save/Load model checkpoints
  112. │ ├── common.py # Misc small utilities, quality of life
  113. │ ├── core_eval.py # Evaluates base model CORE score (DCLM paper)
  114. │ ├── dataloader.py # Tokenizing Distributed Data Loader
  115. │ ├── dataset.py # Download/read utils for pretraining data
  116. │ ├── engine.py # Efficient model inference with KV Cache
  117. │ ├── execution.py # Allows the LLM to execute Python code as tool
  118. │ ├── gpt.py # The GPT nn.Module Transformer
  119. │ ├── loss_eval.py # Evaluate bits per byte (instead of loss)
  120. │ ├── optim.py # AdamW + Muon optimizer, 1GPU and distributed
  121. │ └── tokenizer.py # BPE Tokenizer wrapper in style of GPT-4
  122. ├── pyproject.toml
  123. ├── runs
  124. │ ├── miniseries.sh # Miniseries training script
  125. │ ├── runcpu.sh # Small example of how to run on CPU/MPS
  126. │ ├── scaling_laws.sh # Scaling laws experiments
  127. │ └── speedrun.sh # Train the ~$100 nanochat d20
  128. ├── scripts
  129. │ ├── base_eval.py # Base model: CORE score, bits per byte, samples
  130. │ ├── base_train.py # Base model: train
  131. │ ├── chat_cli.py # Chat model: talk to over CLI
  132. │ ├── chat_eval.py # Chat model: eval tasks
  133. │ ├── chat_rl.py # Chat model: reinforcement learning
  134. │ ├── chat_sft.py # Chat model: train SFT
  135. │ ├── infer_bench.py # Inference: latency/throughput/VRAM bench
  136. │ ├── tok_eval.py # Tokenizer: evaluate compression rate
  137. │ └── tok_train.py # Tokenizer: train it
  138. ├── tasks
  139. │ ├── arc.py # Multiple choice science questions
  140. │ ├── common.py # TaskMixture | TaskSequence
  141. │ ├── gsm8k.py # 8K Grade School Math questions
  142. │ ├── humaneval.py # Misnomer; Simple Python coding task
  143. │ ├── mmlu.py # Multiple choice questions, broad topics
  144. │ └── smoltalk.py # Conglomerate dataset of SmolTalk from HF
  145. ├── tests
  146. │ ├── test_attention_fallback.py # FA3/SDPA attention fallback
  147. │ ├── test_engine.py # Inference engine, KV cache
  148. │ ├── test_execution.py # Sandboxed code execution
  149. │ ├── test_optim.py # MuonAdamW optimizer (needs GPU)
  150. │ ├── test_tasks.py # Task slicing, mixtures, HubDataset
  151. │ └── test_tokenizer.py # BPE round-trips, chat rendering
  152. └── uv.lock
  153. ```
  154. ## Contributing
  155. The goal of nanochat is to improve the state of the art in micro models that are accessible to work with end to end on budgets of < $1000 dollars. Accessibility is about overall cost but also about cognitive complexity - nanochat is not an exhaustively configurable LLM "framework"; there are no giant configuration objects, model factories, or if-then-else monsters in the code base. It is a single, cohesive, minimal, readable, hackable, maximally-forkable "strong baseline" codebase designed to run start to end and produce a ChatGPT model you can talk to. Currently, the most interesting part personally is speeding up the latency to GPT-2 (i.e. getting a CORE score above 0.256525). Currently this takes ~1.5 hours (down from 3h), but by improving the pretraining stage we can improve this further.
  156. Current AI policy: disclosure. When submitting a PR, please declare any parts that had substantial LLM contribution and that you have not written or that you do not fully understand.
  157. ## Acknowledgements
  158. - The name (nanochat) derives from my earlier project [nanoGPT](https://github.com/karpathy/nanoGPT), which only covered pretraining.
  159. - nanochat is also inspired by [modded-nanoGPT](https://github.com/KellerJordan/modded-nanogpt), which gamified the nanoGPT repo with clear metrics and a leaderboard, and borrows a lot of its ideas and some implementation for pretraining.
  160. - Thank you to [HuggingFace](https://huggingface.co/) for fineweb and smoltalk.
  161. - Thank you [Lambda](https://lambda.ai/service/gpu-cloud) for the compute used in developing this project.
  162. - Thank you to chief LLM whisperer 🧙‍♂️ Alec Radford for advice/guidance.
  163. - Thank you to the repo czar Sofie [@svlandeg](https://github.com/svlandeg) for help with managing issues, pull requests and discussions of nanochat.
  164. ## Cite
  165. If you find nanochat helpful in your research cite simply as:
  166. ```bibtex
  167. @misc{nanochat,
  168. author = {Andrej Karpathy},
  169. title = {nanochat: The best ChatGPT that \$100 can buy},
  170. year = {2025},
  171. publisher = {GitHub},
  172. url = {https://github.com/karpathy/nanochat}
  173. }
  174. ```
  175. ## License
  176. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.