Skip to content
Back to search

yue2-rocm-v620

Sub-30s AMD ROCm (RDNA2/gfx1030) HIP-accelerated runtime, Studio Web UI, and MCP server for m-a-p/YuE2-3B

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/franktheglock--yue2-rocm-v620/badge.svg)](/listings/franktheglock--yue2-rocm-v620)

HTML

<a href="/listings/franktheglock--yue2-rocm-v620"><img src="/api/listings/franktheglock--yue2-rocm-v620/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

1. Prerequisites

- AMD ROCm GPU (tested on Radeon PRO V620 / `gfx1030`, 24 GB+ VRAM recommended) - PyTorch with ROCm (`torch` + `hipcc`) - `yue2_infer` wheel from [`m-a-p/YuE2-3B`](https://huggingface.co/m-a-p/YuE2-3B)

# Install dependencies (using your ROCm PyTorch Python)
pip install -e . --no-deps
pip install transformers==4.57.6 huggingface-hub==0.36.2 safetensors==0.7.0 \
            tiktoken==0.12.0 soundfile accelerate fastapi uvicorn httpx mcp

# Install yue2_infer wheel (without overwriting ROCm torch)
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
pip install --no-deps ./yue2_infer-0.1.5-py3-none-any.whl

# Compile custom HIP kernels (auto-detects GPU arch, or set GPU_ARCH=gfx1030 / gfx1100)
./build.sh

2. CLI Generation

python run_yue2.py --ar-quant w4 --cot off --ode-steps 8 --ode-method midpoint --max-seconds 120
# Full symbolic ABC score planning:
python run_yue2.py --ar-quant w4 --cot full --ode-steps 5 --ode-method midpoint --max-seconds 120

3. Studio Web UI & REST API

python web_server.py
# Open http://localhost:8789/  (Swagger API docs at /docs)

Optional AI song auto-naming & album cover art (runs in parallel with audio generation; supports **OpenAI-compatible**, **OpenRouter**, **Anthropic**, and **Ollama** LLMs + **SDAPI/A1111/Forge**, **OpenAI**, **OpenRouter**, and **fal.ai** image gen):

# Example 1: Local llama.cpp + SDAPI / Qwen-Image
export YUE2_COVER_LLM_PROVIDER="openai"          # openai | openrouter | anthropic | ollama
export YUE2_COVER_LLM_URL="http://192.168.1.142:18081"
export YUE2_COVER_LLM_MODEL="maple-cpu"
export YUE2_COVER_IMG_PROVIDER="sdapi"           # sdapi | openai | openrouter | fal
export YUE2_COVER_IMG_URL="http://192.168.1.142:8080"
export YUE2_COVER_IMG_MODEL="qwen-image-2.1-turbo"

# Example 2: OpenRouter (LLM + Image Gen)
# export YUE2_COVER_LLM_PROVIDER="openrouter"
# export YUE2_COVER_LLM_MODEL="google/gemini-2.5-flash"
# export YUE2_COVER_LLM_API_KEY="sk-or-..."
# export YUE2_COVER_IMG_PROVIDER="openrouter"
# export YUE2_COVER_IMG_MODEL="google/gemini-2.5-flash-image-preview"
# export YUE2_COVER_IMG_API_KEY="sk-or-..."

# Example 3: fal.ai Image Gen (Flux)
# export YUE2_COVER_IMG_PROVIDER="fal"
# export YUE2_COVER_IMG_MODEL="fal-ai/flux/schnell"
# export YUE2_COVER_IMG_API_KEY="fal_..."

python web_server.py

4. MCP Server (stdio or LAN Streamable HTTP)

# Stdio (for Claude Code / Cursor / Antigravity):
python mcp_server.py

# Streamable HTTP on LAN:
YUE2_API_URL=http://127.0.0.1:8789 python mcp_server.py --http --host 0.0.0.0 --port 8790

Songs triggered via MCP stream live into the open Web UI over SSE (`/api/events`).

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 142 lines
  1. # YuE2-3B ROCm Accelerator (AMD Radeon PRO V620 / RDNA2 `gfx1030`)
  2. Custom Wave32 HIP C++ kernels, FP16 NAR flow-matching runtime, Suno-inspired Studio Web UI, REST API, and MCP server for [`m-a-p/YuE2-3B`](https://huggingface.co/m-a-p/YuE2-3B) + [`m-a-p/YuE2-Vae`](https://huggingface.co/m-a-p/YuE2-Vae) on AMD ROCm GPUs.
  3. Generates a full **120-second 48 kHz stereo song in 25.4 seconds** (**25.1× faster** than stock PyTorch BF16, **4.72× faster than realtime**) on a single AMD Radeon PRO V620 (32 GB).
  4. ---
  5. ## Audio Samples (Generated on V620)
  6. > **[▶ Open the Interactive Web Audio Player](https://franktheglock.github.io/yue2-rocm-v620/)** — stream all 6 full-length 48 kHz stereo tracks directly in your browser (no download required).
  7. All tracks below were generated end-to-end on a single AMD Radeon PRO V620 (`ar_quant="w4"`, FP16 NAR, 8-step Midpoint ODE, 48 kHz stereo):
  8. | Cover | Track & Genre | Mode | Length | Render Time | Play Online |
  9. |:---:|---|---|---:|---:|:---:|
  10. | — | **Electric in the Rain**<br>Upbeat synth-pop, energetic female vocal, analog synths, 120 BPM | `cot="off"` · 8-mid | `2:00` | **25.4s** (4.72× RT) | [▶ Play](https://franktheglock.github.io/yue2-rocm-v620/#electric-in-the-rain) |
  11. | <img src="docs/samples/velvet_coffee.jpg" width="64" alt="Velvet Coffee at 2 AM" /> | **Velvet Coffee at 2 AM**<br>Late-night lo-fi neo-soul jazz, dusty Rhodes, upright bass, brushed snare, 86 BPM | `cot="full"` · 8-mid | `2:00` | **39.1s** (3.07× RT) | [▶ Play](https://franktheglock.github.io/yue2-rocm-v620/#velvet-coffee) |
  12. | <img src="docs/samples/cedar_lantern.jpg" width="64" alt="Cedar & Lantern Light" /> | **Cedar & Lantern Light**<br>Warm acoustic indie folk ballad, male vocal, fingerpicked guitar, cello, 94 BPM | `cot="full"` · 8-mid | `2:00` | **39.5s** (3.04× RT) | [▶ Play](https://franktheglock.github.io/yue2-rocm-v620/#cedar-lantern) |
  13. | <img src="docs/samples/overclocked_skyline.jpg" width="64" alt="Overclocked Skyline" /> | **Overclocked Skyline**<br>Cyber metal / hard rock, 7-string distorted guitars, double-kick drums, 145 BPM | `cot="full"` · 8-mid | `2:00` | **54.7s** (2.19× RT) | [▶ Play](https://franktheglock.github.io/yue2-rocm-v620/#overclocked-skyline) |
  14. | <img src="docs/samples/training_agi_kpop.jpg" width="64" alt="Training AGI (K-pop Ver.)" /> | **Training AGI (K-pop Ver.)**<br>Upbeat K-pop dance-pop, bright female vocal, sparkling synths, 124 BPM | `cot="full"` · 8-mid | `2:47` | **54.3s** (3.07× RT) | [▶ Play](https://franktheglock.github.io/yue2-rocm-v620/#training-agi-kpop) |
  15. | <img src="docs/samples/another_me.jpg" width="64" alt="Another Me" /> | **Another Me**<br>Dreamy indie electronic pop, warm analog synths, piano, airy female vocal, 98 BPM | `cot="full"` · 8-mid | `3:57` | **80.6s** (2.94× RT) | [▶ Play](https://franktheglock.github.io/yue2-rocm-v620/#another-me) |
  16. ---
  17. ## Benchmarks (AMD Radeon PRO V620 · `gfx1030` · 120s 48 kHz Stereo Song)
  18. | Configuration | AR (3,000 tok) | NAR (ODE) | VAE (FP32) | **Total E2E** | **Speedup** |
  19. |---|---:|---:|---:|---:|---:|
  20. | **Stock YuE2-3B BF16** (PyTorch Eager + Math SDPA, 32-mid) | 454.0s (6.6 tok/s) | 180.6s (64 NFE) | 1.98s* | **636.6s** | 1.0× |
  21. | **V620 Optimized** (`cot="off"`, 32-step Midpoint, 64 NFE) | 13.82s (219.1 tok/s) | 40.21s (64 NFE) | 1.42s | **55.44s** | **11.5×** |
  22. | **V620 Optimized** (`cot="full"`, 1,587 ABC + 3,000 Sem, 5-mid) | 22.73s (205.8 tok/s) | 8.04s (10 NFE) | 1.41s | **32.19s** | **19.8×** |
  23. | **V620 Optimized** (`cot="off"`, 12-step Midpoint, 24 NFE) | 13.82s (219.1 tok/s) | 15.02s (24 NFE) | 1.42s | **30.25s** | **21.0×** |
  24. | **V620 Optimized** (`cot="off"`, **8-step Midpoint, 16 NFE**) | **13.82s (219.1 tok/s)** | **10.18s (16 NFE)** | **1.41s** | **25.41s** | **25.1×** |
  25. *\*Stock VAE on `gfx1030` hits an MIOpen 1D conv workspace bug (>360s hang) unless `torch.backends.cudnn.enabled = False`.*
  26. ### Numerical Accuracy vs Stock BF16 / 32-Step Midpoint Reference
  27. | Stage & Comparison | Cosine Sim | SNR (dB) | Notes |
  28. |---|---:|---:|---|
  29. | AR Logits: **W8A16** vs BF16 | `0.99993` | `38.06 dB` | Top-1 Match |
  30. | AR Logits: **W4A16 (g=128)** vs BF16 | `0.99679` | `21.87 dB` | Top-1 Match, 100% Top-5 overlap |
  31. | NAR Precision: **FP16 (32-mid)** vs BF16 (32-mid) | `0.99993` | `38.33 dB` | Virtually identical latents |
  32. | NAR ODE: **12-step Midpoint** vs 32-step Midpoint | `0.99908` | `27.34 dB` | Audio Cosine `0.9970` |
  33. | NAR ODE: **8-step Midpoint (25.4s)** vs 32-step Midpoint | `0.99550` | `20.47 dB` | Audio Cosine `0.9857` |
  34. | VAE Audio: **Fused HIP `SnakeBeta`** vs PyTorch FP32 | `0.999999999999` | `102.68 dB` | `max_abs = 1.79e-5` |
  35. ---
  36. ## Why Stock PyTorch Is Slow on RDNA2 (`gfx1030`) & What We Built
  37. 1. **Emulated BF16 → Native FP16 + Custom W4A16/W8A16 HIP GEMV (`v620_kernels.hip`):** On `gfx1030`, BF16 GEMV runs at 29.9 GB/s vs FP16 at 262 GB/s. Our 8-warp LDS-fused RMSNorm + 128-bit `uint4` W4A16 GEMV & fused SwiGLU kernels + sliced `lm_head` (32,769 allowed semantic tokens instead of 184,704) push AR decode from **6.6 → 219.1 tok/s** (**33.1×**).
  38. 2. **Missing FlashAttention on `gfx1030` → Infinity-Cache rocBLAS + Wave32 HIP Online-Softmax:** PyTorch SDPA falls back to Math SDPA (`35.84 ms / 4.80 TFLOPS`). Our tiled split-KV attention (`BN=512` tuned for RDNA2's 128 MB Infinity Cache, zero `torch.cat` copies) hits **10.38 ms / 16.59 TFLOPS** (**3.45×** faster).
  39. 3. **GQA-Paired Split-KV FlashDecode:** Processes both Q heads per KV head in a single block with 64-bit coalesced loads, hitting **429.1 GB/s** (84% of V620 HBM peak, `3e-5` error).
  40. 4. **Oobleck VAE Acceleration:** Keeps both YuE2-3B and YuE2-Vae resident on 32 GB VRAM (~9.9 GB total), fuses `weight_norm`, replaces 6-kernel `SnakeBeta` with a single fused HIP kernel (**5.88×** faster), and bypasses MIOpen 1D conv workspace bugs (`1.41s` for 120s FP32 audio).
  41. ---
  42. ## Quick Start
  43. ### 1. Prerequisites
  44. - AMD ROCm GPU (tested on Radeon PRO V620 / `gfx1030`, 24 GB+ VRAM recommended)
  45. - PyTorch with ROCm (`torch` + `hipcc`)
  46. - `yue2_infer` wheel from [`m-a-p/YuE2-3B`](https://huggingface.co/m-a-p/YuE2-3B)
  47. ```bash
  48. # Install dependencies (using your ROCm PyTorch Python)
  49. pip install -e . --no-deps
  50. pip install transformers==4.57.6 huggingface-hub==0.36.2 safetensors==0.7.0 \
  51. tiktoken==0.12.0 soundfile accelerate fastapi uvicorn httpx mcp
  52. # Install yue2_infer wheel (without overwriting ROCm torch)
  53. hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
  54. pip install --no-deps ./yue2_infer-0.1.5-py3-none-any.whl
  55. # Compile custom HIP kernels (auto-detects GPU arch, or set GPU_ARCH=gfx1030 / gfx1100)
  56. ./build.sh
  57. ```
  58. ### 2. CLI Generation
  59. ```bash
  60. python run_yue2.py --ar-quant w4 --cot off --ode-steps 8 --ode-method midpoint --max-seconds 120
  61. # Full symbolic ABC score planning:
  62. python run_yue2.py --ar-quant w4 --cot full --ode-steps 5 --ode-method midpoint --max-seconds 120
  63. ```
  64. ### 3. Studio Web UI & REST API
  65. ```bash
  66. python web_server.py
  67. # Open http://localhost:8789/ (Swagger API docs at /docs)
  68. ```
  69. Optional AI song auto-naming & album cover art (runs in parallel with audio generation; supports **OpenAI-compatible**, **OpenRouter**, **Anthropic**, and **Ollama** LLMs + **SDAPI/A1111/Forge**, **OpenAI**, **OpenRouter**, and **fal.ai** image gen):
  70. ```bash
  71. # Example 1: Local llama.cpp + SDAPI / Qwen-Image
  72. export YUE2_COVER_LLM_PROVIDER="openai" # openai | openrouter | anthropic | ollama
  73. export YUE2_COVER_LLM_URL="http://192.168.1.142:18081"
  74. export YUE2_COVER_LLM_MODEL="maple-cpu"
  75. export YUE2_COVER_IMG_PROVIDER="sdapi" # sdapi | openai | openrouter | fal
  76. export YUE2_COVER_IMG_URL="http://192.168.1.142:8080"
  77. export YUE2_COVER_IMG_MODEL="qwen-image-2.1-turbo"
  78. # Example 2: OpenRouter (LLM + Image Gen)
  79. # export YUE2_COVER_LLM_PROVIDER="openrouter"
  80. # export YUE2_COVER_LLM_MODEL="google/gemini-2.5-flash"
  81. # export YUE2_COVER_LLM_API_KEY="sk-or-..."
  82. # export YUE2_COVER_IMG_PROVIDER="openrouter"
  83. # export YUE2_COVER_IMG_MODEL="google/gemini-2.5-flash-image-preview"
  84. # export YUE2_COVER_IMG_API_KEY="sk-or-..."
  85. # Example 3: fal.ai Image Gen (Flux)
  86. # export YUE2_COVER_IMG_PROVIDER="fal"
  87. # export YUE2_COVER_IMG_MODEL="fal-ai/flux/schnell"
  88. # export YUE2_COVER_IMG_API_KEY="fal_..."
  89. python web_server.py
  90. ```
  91. ### 4. MCP Server (stdio or LAN Streamable HTTP)
  92. ```bash
  93. # Stdio (for Claude Code / Cursor / Antigravity):
  94. python mcp_server.py
  95. # Streamable HTTP on LAN:
  96. YUE2_API_URL=http://127.0.0.1:8789 python mcp_server.py --http --host 0.0.0.0 --port 8790
  97. ```
  98. Songs triggered via MCP stream live into the open Web UI over SSE (`/api/events`).
  99. ---
  100. ## License
  101. - Code in this repository: MIT.
  102. - Model weights (`m-a-p/YuE2-3B` and `m-a-p/YuE2-Vae`): [CC BY-NC 4.0](https://huggingface.co/m-a-p/YuE2-3B/blob/main/LICENSE) (downloaded separately from Hugging Face).
  103. ---
  104. ## Sharing the GPU with a model router (llama.cpp router / Hermes Console)
  105. If another service on the same machine loads LLMs onto the same GPU, YuE2 Studio can free that VRAM automatically. Before every generation it queries the router (default `http://127.0.0.1:8084`) and unloads every loaded model. The router reloads models on demand afterwards.
  106. - Configure it in **Settings → Shared GPU · Model Router**, or via `YUE2_UNLOAD_URL` / `YUE2_UNLOAD_ENABLED=0`.
  107. - Manual trigger: `POST /api/settings/unload-now`.
  108. - **Reverse direction:** while the studio is idle and the router has a model loading/loaded, YuE2 releases its own ~9.9 GB of VRAM (checked every second). It reloads in ~3 s when the next song is requested. Toggle with **Settings** or `YUE2_YIELD_ENABLED=0`.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.