Skip to content
Back to search

breeze-tts2-dialogue

Write multi-speaker dialogue scripts as paste-ready JSON for the ComfyUI "Breeze TTS 2 Multi-Speaker" node. Use when the user provides character names, a bit about each character, and a target duration, and wants a conversation/argument/interview formatted for TTS generation.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/saganaki22--comfyui-breeze-tts-2/badge.svg)](/listings/saganaki22--comfyui-breeze-tts-2)

HTML

<a href="/listings/saganaki22--comfyui-breeze-tts-2"><img src="/api/listings/saganaki22--comfyui-breeze-tts-2/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 78 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. ---
  2. name: breeze-tts2-dialogue
  3. description: Write multi-speaker dialogue scripts as paste-ready JSON for the ComfyUI "Breeze TTS 2 Multi-Speaker" node. Use when the user provides character names, a bit about each character, and a target duration, and wants a conversation/argument/interview formatted for TTS generation.
  4. ---
  5. # Breeze TTS 2 dialogue writer
  6. Produce a dialogue script the user can paste straight into the `text` widget of the
  7. **Breeze TTS 2 Multi-Speaker** ComfyUI node.
  8. ## Gather from the user
  9. 1. **Characters** — a name and a short description for each (personality, speech style,
  10. running gags, how they talk to each other).
  11. 2. **Duration** — target length of the spoken audio.
  12. 3. Optional: language (default English), tone (comedy, drama, interview...), and any plot
  13. or topic the conversation must cover.
  14. If the user already gave all of this, do not ask follow-up questions — just write.
  15. ## Output contract
  16. - Output **only** a raw JSON array. **No markdown code fences, no commentary, no title,
  17. no stage directions.** The first character of your reply must be `[`.
  18. - Exact shape: `[{"speaker": "Name", "text": "what they say"}, ...]`
  19. - The `speaker` value must be a character name **exactly as the user gave it** (the node
  20. matches names forgivingly, but exact is safest). Every character must speak at least
  21. 2–3 times.
  22. - `text` is one or two short spoken sentences. No quotation marks inside it — if a
  23. character quotes someone, paraphrase. No asterisks, no emojis, no parenthetical stage
  24. directions like `(angrily)` — only the vocal event tags listed below.
  25. ## Length budget
  26. Speech runs at roughly **150 words per minute**. Compute the total word budget as
  27. `duration_minutes × 150` and split it across 10–18 turns per minute of audio
  28. (short turns feel like real conversation; long monologues sound robotic).
  29. | Target | Total words | Turns |
  30. | --- | --- | --- |
  31. | 1 min | ~150 | 10–14 |
  32. | 1.3 min | ~200 | 14–18 |
  33. | 2 min | ~300 | 18–26 |
  34. | 5 min | ~750 | 40–70 |
  35. ## Vocal event tags (the ONLY markup allowed)
  36. Confirmed tags — use only these:
  37. - English: `(laugh)` `(cough)` `(clears throat)` `(sigh)`
  38. - Chinese: `[笑]` `[咳嗽]` `[清嗓子]` `[叹气]`
  39. Place them exactly where the sound should happen, e.g. `"(laugh) You paid WHAT?"`.
  40. They count toward the words of that turn. Use them sparingly (a handful across the whole
  41. script) and only where the character's personality calls for it. Close variants like
  42. `(laughing)` or `(sighs)` usually also work, but prefer the confirmed forms above.
  43. If a reaction isn't on this list, write it as normal words ("Oh come on") instead of
  44. inventing a tag.
  45. ## Writing rules
  46. - Make it a real exchange: characters react to, interrupt, and build on what was just
  47. said. Never four disconnected monologues.
  48. - Give each character a distinct voice driven by their description — vocabulary, rhythm,
  49. catchphrases, running gags. Let the running gags recur and escalate.
  50. - Dialogue is spoken aloud: prefer natural speech over written-style prose. Numbers and
  51. abbreviations should be written the way they're said ("fifty grand", not "50k").
  52. - Match the requested language. For Chinese scripts use the Chinese tag set.
  53. - End on a beat: a punchline, a callback, or someone having the last word — not mid-thought.
  54. ## Example
  55. User asks: "Ada (a pirate captain who treats every problem like a naval battle) argues
  56. with Bob (a nervous first mate who over-explains everything), 60 seconds."
  57. Correct output:
  58. [{"speaker": "Ada", "text": "(sigh) Who steered my ship into the harbor wall this time?"}, {"speaker": "Bob", "text": "Technically, captain, the harbor steered into us. The tide charts were very clear about that."}, {"speaker": "Ada", "text": "The tide charts. (laugh) Twenty years at sea and I am outmaneuvered by paperwork."}, {"speaker": "Bob", "text": "If I may, the paperwork did warn us. Twice. In bold letters."}, ...]
README.md · README · 187 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # ComfyUI-Breeze-TTS-2
  2. **English** | [简体中文](README_ZH.md)
  3. <img width="1483" height="1170" alt="Screenshot 2026-08-26 160830" src="https://github.com/user-attachments/assets/95f2f346-472e-45a5-bd3e-625007c6cc39" />
  4. https://github.com/user-attachments/assets/d4226032-3337-450b-83c6-ab9c732172d6
  5. [Breeze TTS 2](https://huggingface.co/BreezeBlue/Breeze-TTS-2) nodes for ComfyUI: **voice clone**,
  6. **voice design**, and **voice direction** with bilingual (EN/ZH) speech, inline vocal events,
  7. INT8 ConvRot quantized builds, and full ComfyUI / AIMDO dynamic-VRAM integration.
  8. ## Nodes (category: `Breeze TTS 2`)
  9. | Node | Purpose |
  10. | --- | --- |
  11. | **Breeze TTS 2 Load Model** | Loads the checkpoint with `dtype` (auto/bf16/fp32), `device` (auto/cuda/cpu), `attention` (auto/eager/sdpa/flash_attention/sageattention), `decode_mode` (eager/cuda_graphs), and `download_if_missing`. |
  12. | **Breeze TTS 2 Voice Clone** | Clones a speaker from clean reference audio + its exact transcript (CFG 1.0). |
  13. | **Breeze TTS 2 Voice Design** | Creates a voice from a natural-language description, no reference audio (CFG 4). |
  14. | **Breeze TTS 2 Voice Direction** | Clones a reference voice and steers tone, emotion, pace, and delivery with an instruction (CFG 4). `stitch_reference` can play the original reference clip before or after the generated speech in the output. |
  15. | **Breeze TTS 2 Whisper Transcribe** | Whisper helper that produces the exact transcript cloning needs. |
  16. | **Breeze TTS 2 Speaker** | One named cast member for Multi-Speaker. Pick a reference clip from the input folder (browse/upload dropdown, or wire audio in) to clone it, or select `none` and write an instruction to design the voice. An empty transcript is auto-transcribed with Whisper and shown on the node. |
  17. | **Breeze TTS 2 Multi-Speaker** | Generates a whole dialogue script — plain `Name:` lines or pasted JSON — with up to 8 wired speakers, freely mixing cloned and designed voices. |
  18. Vocal events work inline in the text: `(laugh)`, `(cough)`, `(clears throat)`, `(sigh)` in English;
  19. `[笑]`, `[咳嗽]`, `[清嗓子]`, `[叹气]` in Chinese. Match the instruction language to the text language
  20. for design/direction. Output is 24 kHz mono.
  21. ## Multi-speaker dialogue
  22. https://github.com/user-attachments/assets/bf124113-f9ef-4a5d-a501-5935d8fd7b60
  23. Wire up to 8 **Speaker** nodes into the **Multi-Speaker** node's `speaker_1…8` inputs, then write
  24. (or paste) the script. Speaker matching is forgiving about case, spaces, and punctuation
  25. (`Ali G` matches a speaker named `alig`), and every speaker name is validated before any
  26. generation starts, so a bad script fails instantly instead of after burning GPU time.
  27. **Option 1 — plain text.** One `Name: line` per line. A line without a name continues the
  28. previous speaker; `[Name]:` and markdown `**Name:**` also work:
  29. ```
  30. Ada: (sigh) Who steered my ship into the harbor wall this time?
  31. Bob: Technically, captain, the harbor steered into us.
  32. Ada: That is not how harbors work, Bob.
  33. Bob: If I may, the tide charts were very clear about it.
  34. ```
  35. **Option 2 — pasted JSON.** Ideal for LLM-written scripts. If the text starts with `[` or `{`
  36. it is parsed as JSON:
  37. ```json
  38. [
  39. {"speaker": "Ada", "text": "(sigh) Who steered my ship into the harbor wall this time?"},
  40. {"speaker": "Bob", "text": "Technically, captain, the harbor steered into us."},
  41. {"speaker": "Ada", "text": "That is not how harbors work, Bob."}
  42. ]
  43. ```
  44. Also accepted: a single `{"speaker": ..., "text": ...}` object, a wrapper like
  45. `{"script": [...]}`, and the key aliases `speaker`/`name`/`character`/`role` and
  46. `text`/`line`/`content`/`message`. Inline vocal events work identically in both formats.
  47. The `text` widget can be converted to an input (right-click) to wire a string from any
  48. text-generating node.
  49. Before inference the console prints a summary —
  50. ```
  51. [BreezeTTS2] Multi-speaker script: 4 speaker(s), 16 turn(s), ~75s of speech total | cast: Ada (clone), Bob (clone), Carol (design), Dave (design)
  52. ```
  53. — then one progress bar per turn (`[3/16] Bob (~6s)`). Reference clips are encoded once per
  54. speaker, each speaker keeps a stable seed offset across turns (so designed voices stay
  55. consistent), and `pause_between_speakers` controls the gap. Reference clips: keep them under
  56. ~20 s (hard max 60 s). To get an LLM to write scripts in this format, hand it
  57. [SKILL.md](SKILL.md).
  58. <img width="776" height="1153" alt="Screenshot 2026-08-28 035631" src="https://github.com/user-attachments/assets/be2ad864-bb9d-42b2-a474-ad22315a4dad" />
  59. ## Checkpoints
  60. The loader downloads into `ComfyUI/models/breezetts2/` when `download_if_missing` is enabled:
  61. | Choice | File | Notes |
  62. | --- | --- | --- |
  63. | int8 hybrid **(default)** | `Breeze-TTS-2-int8-hybrid.safetensors` | Backbone + text encoder INT8 ConvRot, depth decoder bf16. bf16-level speed at −27 % VRAM. |
  64. | bf16 combined | `Breeze-TTS-2-bf16.safetensors` | Same weights merged into one file (bit-exact). Fastest. |
  65. | INT8 all linears | `Breeze-TTS-2-int8-convrot.safetensors` | Smallest build; slower in the decode loop (see below). |
  66. | INT8 text encoder only | `Breeze-TTS-2-int8-text-encoder.safetensors` | Decode path fully bf16; smallest quality-irrelevant VRAM cut. |
  67. All mirrors live in [drbaph/Breeze-TTS-2-comfyui](https://huggingface.co/drbaph/Breeze-TTS-2-comfyui).
  68. ## Model Files
  69. Weights are stored per source repo under `ComfyUI/models/breezetts2/` (primary folder first; symlinks are searched too). Every models tree mapped through `extra_model_paths.yaml` is also searched, including sections that only list standard folders like `checkpoints:` — a `breezetts2/` folder inside any mapped tree is picked up, so one fixed location can serve every ComfyUI install. Mapped folders are read-only to the node: new downloads always land in the running install's own models folder. An explicit `breezetts2:` entry in the yaml works as well. Missing files download automatically when `download_if_missing` is on, otherwise the error names the expected folder.
  70. ```
  71. 📂 ComfyUI/
  72. └── 📂 models/
  73. └── 📂 breezetts2/
  74. └── 📂 drbaph_Breeze-TTS-2-comfyui/ (all builds, default source)
  75. ├── Breeze-TTS-2-bf16.safetensors (6.49 GiB, official shards merged bit-exact)
  76. ├── Breeze-TTS-2-int8-hybrid.safetensors (4.53 GiB, recommended int8 build)
  77. ├── Breeze-TTS-2-int8-convrot.safetensors (4.22 GiB, all 462 linears quantized)
  78. ├── Breeze-TTS-2-int8-text-encoder.safetensors (5.84 GiB, decode path bf16)
  79. ├── config.json
  80. ├── generation_config.json
  81. ├── tokenizer.json
  82. ├── tokenizer_config.json
  83. ├── special_tokens_map.json
  84. └── 📂 audio_tokenizer/ (Qwen3-TTS 12 Hz codec, shared)
  85. ├── model.safetensors (0.64 GiB)
  86. ├── config.json
  87. ├── configuration.json
  88. └── preprocessor_config.json
  89. ```
  90. Only the selected weights file is downloaded; every build shares the same `config.json`, tokenizer files, and `audio_tokenizer/` codec.
  91. | Component | bf16 build | int8 builds |
  92. | --- | --- | --- |
  93. | Model weights | 6.49 GiB | 4.22–5.84 GiB |
  94. | `audio_tokenizer` codec | 0.64 GiB | unchanged |
  95. | Tokenizer + configs | ~35 MB | unchanged |
  96. Expect roughly **5.3–7.5 GiB VRAM** depending on build and dtype; AIMDO DynamicVRAM pages castable weights to keep live VRAM pressure low alongside other models.
  97. ## Benchmarks (RTX 5090, ComfyUI loader, sdpa)
  98. | Build | avg RTF | Peak VRAM | Whisper WER vs prompt |
  99. | --- | --- | --- | --- |
  100. | bf16 | 5.68 | 7.51 GiB | baseline |
  101. | int8-convrot (all linears) | 9.08 | 5.21 GiB | matches bf16 |
  102. | **int8-hybrid (depth decoder bf16)** | **5.64** | **5.53 GiB** | matches bf16 |
  103. | int8-text-encoder only | 5.34 | 6.82 GiB | matches bf16 |
  104. Why full INT8 regresses: the depth decoder runs 15 skinny single-token GEMMs per 12.5 Hz frame
  105. (≈1260 linear calls/frame). At M=1–2 the int8 per-call overhead (~0.1 ms quantize + dispatch +
  106. allocations) dwarfs the GEMM savings, so those layers lose to plain bf16 `F.linear` (~0.03 ms).
  107. The backbone contributes only ~196 calls/frame and the text encoder runs once at prefill, so
  108. quantizing them is nearly free — that is exactly what the hybrid build does. Full measurement
  109. data is in `benchmark_report.json` on the mirror repo.
  110. ## Requirements
  111. - ComfyUI with **Transformers 4.57 or 5.3+**. On 5.x the text encoder uses the native
  112. `T5Gemma2TextEncoder` wired into flash attention through a custom-registered FA2 path
  113. (upstream blocks it over softcapping concerns this checkpoint does not have); on 4.x it
  114. uses the vendored upstream compat implementation, which also runs flash attention.
  115. - torch/torchaudio from your ComfyUI install. `flash_attn` and `sageattention` are optional.
  116. - `comfy-kitchen` ships with current ComfyUI and is only imported when an INT8 checkpoint is loaded.
  117. ## Model architecture (what actually runs)
  118. - **Text encoder**: T5Gemma2 (26 layers, 1152 hidden) + linear projection into the backbone space.
  119. - **Backbone**: Qwen3-style decoder (28 layers, 2048 hidden) predicting the first codebook token per
  120. 12.5 Hz frame, with an audio-token embedding summed over the 16 codebooks.
  121. - **Depth decoder**: 12-layer decoder generating codebooks 1–15 per frame with its own CFG.
  122. - **Audio tokenizer**: vendored Qwen3-TTS 12 Hz codec (Mimi-style encoder + streaming decoder),
  123. 24 kHz, 16 codebooks. Reference audio is encoded with the same codec.
  124. - The in-checkpoint `codec_model` (a Mimi copy used only by the upstream training forward) is not
  125. built at inference.
  126. The eager generation loop reimplements the official `FastBreezeStreamingRuntime` with all fast
  127. stages disabled (DynamicCache instead of CUDA graphs) so it stays compatible with AIMDO dynamic
  128. VRAM paging and INT8 weight casting. With `decode_mode=cuda_graphs` the depth decode steps are
  129. captured into CUDA graphs instead — the captured decode runs **7–9x faster** than the eager
  130. AIMDO-paged path (~3–4x faster end-to-end generation); that mode keeps the model weights
  131. VRAM-resident rather than AIMDO-paged. The official CUDA-graph fast path is intentionally not ported.
  132. ## License
  133. Node-pack code: Apache-2.0 (see `LICENSE`). Vendored model code keeps its upstream licenses
  134. (Apache-2.0 from breezeblue-ai/breeze-tts and Qwen/Qwen3-TTS).
  135. **Model weights, derivative checkpoints, and self-hosted outputs are governed by the BreezeBlue
  136. Research and Non-Commercial License** — see [MODEL_LICENSE](https://huggingface.co/BreezeBlue/Breeze-TTS-2/blob/main/LICENSE).
  137. Commercial use requires written authorization from RESONIA, INC.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.