Skip to content
Back to search

audio

High-level audio manipulations: loading, playback, analysis and editing. 140+ plugins: denoise, dynamics, EQ, filters, effects, reverb, time and pitch, spatial, synthesis, music analysis.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/audiojs/audio with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/audiojs/audio'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/audiojs--audio/badge.svg)](/listings/audiojs--audio)

HTML

<a href="/listings/audiojs--audio"><img src="/api/listings/audiojs--audio/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 1,017 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # audio [![test](https://github.com/audiojs/audio/actions/workflows/test.yml/badge.svg)](https://github.com/audiojs/audio/actions/workflows/test.yml) [![npm](https://img.shields.io/npm/v/audio?color=white)](https://npmjs.org/package/audio)
  2. _High-level audio manipulations: loading, playback, analysis and editing. 140+ plugins: denoise, dynamics, EQ, filters, effects, reverb, time and pitch, spatial, synthesis, music analysis._
  3. <!-- <img src="docs/preview.svg?v=1" alt="Audiojs demo" width="540"> -->
  4. * **Any Format** — fast [wasm codecs](https://github.com/audiojs/decode), no ffmpeg.
  5. * **Non-destructive** — virtual edits, infinite undo, instant clone.
  6. * **Stream-first** — playback/encode during decode, realtime editing.
  7. * **Paged** — no 2Gb memory limit, open 10Gb+ files.
  8. * **Analysis** — loudness, spectrum, beats, pitch, chords, key.
  9. * **Modular** – pluggable ops, tree-shakable.
  10. * **CLI** — playback, batch processing, scripting, unix pipes, tab completion.
  11. * **Cross-platform** — browsers, node, deno, bun.
  12. <!--
  13. * [Architecture](docs/architecture.md) – stream-first design, pages & blocks, non-destructive editing, plan compilation
  14. * [Plugins](docs/plugins.md) – custom ops, stats, descriptors (process, plan, resolve, call), persistent ctx
  15. -->
  16. <div align=center>
  17. <br>
  18. <a href="https://www.npmjs.com/package/audio"><img src="https://raw.githubusercontent.com/audiojs/.github/main/profile/terminal.svg" alt="npm install audio"></a>
  19. #### [Start](#start)&nbsp;&nbsp;&nbsp;[Recipes](#recipes)&nbsp;&nbsp;&nbsp;[API](#api)&nbsp;&nbsp;&nbsp;[CLI](#cli)&nbsp;&nbsp;&nbsp;[FAQ](#faq)&nbsp;&nbsp;&nbsp;[Plugins](docs/plugins.md)&nbsp;&nbsp;&nbsp;[Architecture](docs/architecture.md)&nbsp;&nbsp;&nbsp;[Comparison](docs/comparison.md)&nbsp;&nbsp;&nbsp;[RX 12](docs/rx.md)
  20. </div>
  21. ## Start
  22. ### Node
  23. `npm i audio`
  24. ```js
  25. import audio from 'audio'
  26. let song = audio('song.mp3')
  27. song.play() // the speakers, as a page plays it: no ffmpeg, no player to spawn
  28. song.pause(); song.seek(60); song.resume()
  29. let voice = audio('voice.wav').trim().normalize('podcast')
  30. await voice.save('clean.mp3')
  31. let { pass, rules } = await voice.check('podcast') // each rule of Apple's spec, measured
  32. ```
  33. ### Browser
  34. ```html
  35. <script type="module">
  36. import audio from 'https://esm.sh/audio'
  37. audio('./song.mp3').trim().normalize().fade(0.5, 2).clip({ at: 60, duration: 30 }).play()
  38. </script>
  39. ```
  40. ### `Audio` anywhere
  41. ```js
  42. import 'audio/polyfill' // the page's HTMLAudioElement, in Node, Bun and Deno
  43. let song = new Audio('song.mp3')
  44. song.onended = () => console.log('done')
  45. await song.play()
  46. song.currentTime = 30; song.volume = 0.5; song.loop = true
  47. ```
  48. Events, their order, promises and errors as Chromium's own element has them, compared observation by observation ([test/polyfill.js](test/polyfill.js)). `import { Audio } from 'audio/polyfill'` leaves the global alone.
  49. ### CLI
  50. ```sh
  51. npm i -g audio # or: npx audio …
  52. audio voice.wav trim normalize podcast save clean.mp3
  53. audio clean.mp3 check podcast
  54. # Apple Podcasts https://podcasters.apple.com/support/893-audio-requirements
  55. # ✓ Loudness -16.44 LUFS -17 to -15
  56. # ✓ True peak -1.18 dBTP ≤ -1
  57. # pass
  58. ```
  59. A fail exits 1, so the same line guards a build: [Check in CI](#check-in-ci).
  60. ### Skill
  61. ```sh
  62. npx skills add audiojs/audio
  63. ```
  64. ### MCP
  65. ```sh
  66. npx add-mcp "npx -y audio --mcp" # asks which of your agents: Claude Code, Codex, Cursor, Gemini CLI, Kimi Code, Pi, OpenCode, Zed…
  67. ```
  68. `Prompt: make ~/Desktop/interview.m4a podcast-ready and tell me the loudness before and after`
  69. One agent at a time: `claude mcp add audio -- npx -y audio --mcp`, `qwen mcp add audio npx -y audio --mcp`, `droid mcp add audio "npx -y audio --mcp"`.
  70. ### The repl and agents
  71. ```sh
  72. npx audio --bridge
  73. # audio bridge on http://127.0.0.1:7777
  74. #
  75. # key 3e37b8c61acf5b50f010852168f4843d
  76. # agents Claude Code, Codex, Pi, Kimi Code
  77. #
  78. # In the repl opened with ?ui={"agent":true}, paste the key in the line under the code, then Connect. It stays the same next time.
  79. ```
  80. Open the [repl with its agent line](https://audio.js.org/audio/repl/?ui=%7B%22agent%22:true%7D), every edit of it a line of `audio` code, and connect it to the bridge with the key (once: the bridge keeps it), and the line under its code runs an agent of yours, which measures, looks at, edits and plays the sound open there, and finds which edit did what: it measures the sound before and after each, and what each took out; its conversations are kept, each with the agent, and its model, picked under the line. The bridge finds Claude Code, Codex, Pi, Gemini CLI, Qwen Code, Kimi Code, OpenCode, Kilo Code, Cline, Goose, Factory Droid, Cursor, Augment, Kiro and Mistral Vibe on PATH; any other that speaks [ACP](https://agentclientprotocol.com/get-started/registry) runs by its command line, `--agent "my-agent --acp"`.
  81. Any MCP agent gets the repl's tools (`state`, `measure`, `look`, `edit`, `select`, `play`, `check`, …), the running bridge found by itself: `npx add-mcp "npx -y audio --mcp --repl"`.
  82. An agent thinks with the model its own settings name: Pi takes local ones for good from `ollama launch pi --config`; Claude Code takes any Anthropic-compatible endpoint from the bridge's environment:
  83. ```sh
  84. ANTHROPIC_BASE_URL=http://localhost:11434 ANTHROPIC_AUTH_TOKEN=ollama ANTHROPIC_API_KEY= ANTHROPIC_MODEL=qwen3-coder npx audio --bridge
  85. ```
  86. | Models | `ANTHROPIC_BASE_URL`, with the provider's key as `ANTHROPIC_AUTH_TOKEN` |
  87. |---|---|
  88. | Ollama | `http://localhost:11434`, token `ollama` ([docs](https://docs.ollama.com/integrations/claude-code)) |
  89. | Z.ai GLM | `https://api.z.ai/api/anthropic` ([docs](https://docs.z.ai/devpack/tool/claude)) |
  90. | Kimi | `https://api.moonshot.ai/anthropic` ([docs](https://platform.kimi.ai/docs/guide/claude-code-kimi)) |
  91. | Qwen | `https://coding-intl.dashscope.aliyuncs.com/apps/anthropic`, Coding Plan ([docs](https://www.alibabacloud.com/help/en/model-studio/claude-code)) |
  92. | DeepSeek | `https://api.deepseek.com/anthropic` ([docs](https://api-docs.deepseek.com/quick_start/agent_integrations/claude_code)) |
  93. ### Embed
  94. The repl in any page, its script the element's own text: it runs where the reader scrolls to it, plays, takes edits, and opens whole in the repl. Nothing of the reader's is kept.
  95. ```html
  96. <script type="module" src="https://audio.js.org/embed.js"></script>
  97. <audio-repl panel="code" display="wave" height="320">
  98. <pre><code>audio('chime.wav')
  99. .trim()
  100. .normalize(-1)</code></pre>
  101. </audio-repl>
  102. ```
  103. What it shows, the frame takes as one JSON: the element's `ui` attribute, with these attributes over it, each a key of it. `panel`: `code` (default), `edits` (its steps, each with its settings) or `none`; `display`: `wave` or `spec`; `controls`: the parts shown, each in its own place: over the picture `sound` (its name, a menu of its ways in: a file, a take, a sample), `link` and `name` (the way to the repl), `ruler` (the time row; left out, none: the big time says it), `look` (a band's dropdown of how it draws); in the bar `play` `record` `loop` `clock` `history` `message` `heard`, a level (`level`, or what it reads: `peak` `rms` `loudness`), `rate` `channels` `depth` `before`, `view` (a switch: the waveform, the spectrogram), `open` and `save` (a file of the reader's in; the result out, in the format picked from its menu) — without it, all but `sound`, `view`, `open` and `save`; `ops`: edits a press away, one button after the history, a menu of them with their values, as `trim normalize(-1) stretch(1.25) reverse` (a call's own arguments, or the defaults the menus write); `background`: the ground it sits on and fades into, a CSS colour, or `page`, the page's own under the element (the element reads it; a frame can't); `height` in px. A sound named by its path from the site's root (`audio('/sounds/take.mp3')`) is that site's file. The `<pre>` shows until the frame comes, and where scripts don't run. Every key, `zoom` and `overshoot` too: [repl/README.md](repl/README.md).
  104. ## Recipes
  105. ### Clean up
  106. ```js
  107. // master a raw take
  108. let a = audio('raw-take.wav')
  109. a.trim(-30).normalize('podcast').fade(0.3, 0.5)
  110. await a.save('clean.wav')
  111. // full restoration chain via ecosystem plugins (see API › Plugins)
  112. a.gate(-45).dehum().deesser().compressor({ threshold: -18 }).limiter({ ceiling: -1 })
  113. // cut 2:00–2:15, smooth the splice
  114. a.remove({ at: 120, duration: 15 }).fade(0.1, { at: 120 })
  115. // find clipped blocks
  116. let clips = await a.stat('clipping')
  117. // does it pass? each rule of the spec, measured
  118. let { pass, rules } = await a.check('podcast') // acx, podcast, spotify, broadcast, netflix
  119. ```
  120. ### Master & deliver
  121. ```js
  122. // master a song to a reference track: its tone (mid and side), width and loudness, under -1 dBTP
  123. audio('mix.wav').master(await audio('reference.wav')).save('master.wav')
  124. // audiobook chapter for ACX: RMS -23..-18 dB, peaks under -3 dB, floor under -60 dB, room tone at each end, 44.1 kHz
  125. let ch = audio('chapter-01.wav')
  126. .highpass(80).omlsa({ gMin: -12 }).compressor({ threshold: -24, ratio: 2.5 })
  127. .normalize(-20, 'rms', { ceiling: -3.5 })
  128. .trim().pad(1.5, 2).roomtone() // room tone, not digital silence
  129. .resample(44100)
  130. console.log(await ch.check('acx')) // passed 10 of 10 real narrations (.work/pro.md)
  131. await ch.save('chapter-01.mp3', { bitrate: 192 })
  132. // tighten pauses in a talking-head video, then hand the cuts to the video editor
  133. let talk = audio('talk.mp4').shrink(0.3)
  134. await talk.save('talk.m4a') // the sound, cut
  135. await talk.save('talk.edl') // the same cuts for the picture (Premiere, Resolve)
  136. ```
  137. ### Compose
  138. ```js
  139. // podcast montage
  140. let ep = audio([intro, interview.trim().normalize('podcast'), outro], { crossfade: 0.5 })
  141. await ep.save('episode.mp3')
  142. // voiceover over music
  143. music.gain(-12).mix(voice, { at: 2 })
  144. // ringtone: the chorus + fades
  145. audio('song.mp3').crop({ at: 45, duration: 30 }).fade(0.5, 2).normalize().save('ringtone.mp3')
  146. // split an audiobook into chapters
  147. let [ch1, ch2, ch3] = audio('audiobook.mp3').split(1800, 3600)
  148. // glitch: stutter + reverse
  149. let v = a.clip({ at: 1, duration: 0.25 })
  150. audio([v, v, v, v]).reverse({ at: 0.25, duration: 0.25 })
  151. ```
  152. ### Analyze
  153. ```js
  154. // waveform bars — and progressively, as it decodes
  155. let [mins, peaks] = await a.stat(['min', 'max'], { bins: canvas.width })
  156. a.on('data', ({ delta }) => appendBars(delta.max[0], delta.min[0]))
  157. // features for ML
  158. let mfcc = await a.stat('cepstrum', { bins: 13 })
  159. let [loud, rms] = await a.stat(['loudness', 'rms'])
  160. // notes, chords, key
  161. let notes = await a.stat('notes') // [{time, duration, freq, midi, note, clarity}]
  162. let chords = await a.stat('chords') // [{time, duration, label, root, quality, confidence}]
  163. let key = await a.stat('key') // {tonic, mode, label, confidence}
  164. ```
  165. ### Record & generate
  166. ```js
  167. // mic take
  168. let a = audio()
  169. a.record()
  170. // …later
  171. a.stop()
  172. a.trim().normalize()
  173. // tone — any t => sample function
  174. let tone = audio.from(t => Math.sin(440 * Math.PI * 2 * t), { duration: 2 })
  175. // sonify data
  176. let s = audio.from(t => Math.sin((200 + data[t / 0.2 | 0]) * Math.PI * 2 * t) * 0.5, { duration: data.length * 0.2 })
  177. ```
  178. ### Automate
  179. ```js
  180. a.gain(t => -12 * (0.5 + 0.5 * Math.cos(t * Math.PI * 4))) // 2Hz tremolo in dB
  181. a.lowpass(t => 400 + 4000 * t) // filter sweep
  182. a.pan({ t: [0, 2, 4], v: [-1, 1, -1] }) // curve L→R→L over 4s, serializable
  183. music.ducker({ key: voice }) // sidechain (plugin)
  184. ```
  185. ### Stream & persist
  186. ```js
  187. // stream to network — encode/playback during decode
  188. for await (let chunk of audio('2hour-mix.flac').highpass(40)) socket.send(chunk[0].buffer)
  189. // serialize edits, restore later
  190. let json = JSON.stringify(a) // { source, edits, ... }
  191. let b = audio(JSON.parse(json)) // re-decode + replay edits
  192. ```
  193. ## API
  194. ### Create
  195. | Method                         | Description                                                                                                                         |
  196. |:--|:--|
  197. | `audio(source, opts?)` | decode from file, URL, bytes, or a byte stream. Returns instantly — decodes in background; streams decode as they arrive. |
  198. | `audio.from(source, opts?)` | wrap existing PCM, AudioBuffer, silence, or function. Sync, no I/O. |
  199. ```js
  200. let a = audio('voice.mp3') // file path
  201. let b = audio('https://cdn.ex/track.mp3') // URL
  202. let c = audio(inputEl.files[0]) // Blob, File, Response, ArrayBuffer
  203. let d = audio() // empty, ready for .push() or .record()
  204. let e = audio([intro, body, outro]) // concat (virtual, no copy)
  205. let f = audio([a, b, c], { crossfade: 2 }) // concat with 2s crossfade
  206. let g = audio(process.stdin) // byte stream: pipe, socket, fetch body – decodes as it arrives
  207. // opts: { sampleRate, channels, crossfade, curve, storage: 'memory' | 'persistent' | 'auto' }
  208. await a // await for decode — if you need .duration, full stats etc
  209. let a = audio.from([left, right]) // Float32Array[] channels
  210. let b = audio.from(3, { channels: 2 }) // 3s silence
  211. let c = audio.from(t => Math.sin(440*TAU*t), { duration: 2 }) // generator
  212. let d = audio.from(audioBuffer) // Web Audio AudioBuffer
  213. let e = audio.from(int16arr, { format: 'int16' }) // typed array + format
  214. let f = audio.from(store, { length, channels, sampleRate }) // pages in a store ({ read(i), has(i), write(i, page) }), read as needed
  215. ```
  216. ### Properties
  217. | Property                         | Description                                                                                                                         |
  218. |:--|:--|
  219. | `.duration` | total seconds, after edits. |
  220. | `.channels` | channel count. |
  221. | `.sampleRate` | sample rate. |
  222. | `.length` | samples per channel. |
  223. | `.currentTime` | what the speakers play now, in seconds: their latency compensated, smooth; at pause it holds where playback resumes. |
  224. | `.playing` | true during playback. |
  225. | `.paused` | true when paused. |
  226. | `.volume` | 0..1 linear. Settable. |
  227. | `.muted` | mute, independent of volume. Settable. |
  228. | `.loop` | settable, mid-playback too: the span repeats, each seam a 10 ms equal-power crossfade. |
  229. | `.playbackRate` | 0.0625..16, settable mid-playback, click-free. The pitch stays (WSOLA, as a browser's media element plays at a speed) unless `.preservesPitch` is false: then it glides over ~50 ms and the pitch follows, tape-style. `.speed()` bakes it. |
  230. | `.preservesPitch` | true: at a rate other than 1, the pitch kept. false: varispeed, the pitch with the speed. Settable mid-playback. |
  231. | `.ended` | true when playback reached the end, not after `stop()`. |
  232. | `.seeking` | true during a seek. |
  233. | `.played` | promise, resolves when playback sounds. |
  234. | `.recording` | true during mic recording. |
  235. | `.ready` | promise, resolves when fully decoded. |
  236. | `.source` | original source. |
  237. | `.bitDepth` | stored sample depth of the source: 16, 24, 32 (float); null for lossy or generated audio. Lossless `save` keeps it. |
  238. | `.pages` | `Float32Array` page store. |
  239. | `.stats` | per-block stats (peak, rms, etc.). |
  240. | `.edits` | edit list. |
  241. | `.version` | increments on each edit. |
  242. ### Structure
  243. | Method                         | Description                                                                                                                         |
  244. |:--|:--|
  245. | `.trim(threshold?)` | strip leading/trailing silence (dB, default auto). On a live stream a given threshold streams, holding a silent tail until sound resumes; the automatic one reads the whole input, so it waits for the end. |
  246. | `.shrink(gap?, threshold?, crossfade?)` | shorten silent pauses to `gap` seconds (default 0.3); `0` removes them. Each cut a hard join, or with `crossfade` (`'10ms'`) remove()'s equal-power crossfade over it, the length the same. Streams with a given threshold, like `trim`.<br><sub>≡ FFmpeg `silenceremove`, Audacity truncate-silence</sub> |
  247. | `.crop({at, duration})` | keep range, discard rest. |
  248. | `.remove(at, duration, crossfade?)` | delete range, close gap. `crossfade` (`'10ms'`) makes the splice an equal-power crossfade centered on the cut; the length stays the same. |
  249. | `.insert(source, at?, crossfade?)` | insert audio (default: at end), or a number of seconds of silence; `crossfade` fades both seams. |
  250. | `.copy({at?, duration?})` | copy range (default: all) to this instance's clipboard. |
  251. | `.cut({at?, duration?, crossfade?})` | copy, then remove. |
  252. | `.paste({at?, crossfade?})` | insert the clipboard (default: at end). |
  253. | `.move({at, duration, to, crossfade?})` | slide a range to `to`, over what is there; silence where it was, the length kept (past the end, extended). `crossfade` crossfades each edge, centered.<br><sub>≡ a DAW's clip moved in slip mode</sub> |
  254. | `.clip({at, duration})` | zero-copy excerpt as a new instance. |
  255. | `.split(...offsets)` | zero-copy excerpts between timestamps. |
  256. | `.pad(before, after?)` | silence at edges (seconds). |
  257. | `.repeat(n)` | repeat n times. |
  258. | `.reverse({at?, duration?})` | reverse audio or range. |
  259. | `.speed(rate, {type?})` | changes pitch and duration together; `type: 'sinc'` reads between samples by a Lanczos sinc (16 zero crossings) instead of linearly, the top octave kept at a rate near 1. |
  260. | `.stretch(factor, {voice?, method?})` | changes duration, keeps pitch (phase-locked vocoder). A `t => f` or `{t, v}` factor slides the tempo; duration becomes ∫factor dt. A range `{at, duration}` comes out `round(round(duration · sampleRate) · factor)` samples long, to the sample; the audio around it as it was. `{ voice: true }` keeps a voice's pulse shape and consonants, which the vocoder makes distant: shortened, its waveform copied a segment at a time (WSOLA, `@audio/stretch-wsola`); slowed, the vocoder's frames restarted from the waveform where it fits (PVSOLA, `@audio/stretch-pvsola`), so breath and reverberation are not repeated into a flanger. One voice, not chords. `{ method }` stretches another way, its module loaded only then: `'vocoder'`, `'pghi'`, `'wsola'`, `'psola'`, `'sms'`, `'transient'`, `'hybrid'`, `'paulstretch'` (`@audio/stretch-*`); `'phase-lock'`, the default, is the one that streams, slides and takes `voice`. The others render what they stretch at once, from a constant factor; a range, with a crossfade into the audio either side.<br><sub>≡ Logic Flex Time Monophonic, Ableton Tones</sub> |
  261. | `.warp(markers)` | move moments in time: `[[from, to], …]` in seconds. Between markers the audio stretches to fit, pitch kept; start and end stay.<br><sub>≡ Logic Flex Time, Ableton warp markers</sub> |
  262. | `.pitch(semitones, {voice?, method?})` | changes pitch, keeps duration. Semitones may be a curve `{t, v}` (seconds → semitones, straight between points, flat past the ends, as the gain line's) or `t => semitones`; where it is zero the audio is as it was. `{ voice: true }` re-spaces a voice's own glottal cycles, found on its pYIN pitch track (TD-PSOLA, the optional `@audio/tune-curve`): formants and consonants kept, one voice; a range's edges glide as fast as a voice moves its pitch, and its cycles land back on the input's. `{ method }` shifts another way, its module loaded only then (`@audio/shift-*`): `'vocoder'`, `'formant'` (the voice's tone kept), `'psola'`, `'wsola'`, `'lpc'`, `'transient'`, `'hybrid'`, `'hpss'`, `'sms'`, `'sample'`, `'delay'`, `'ola'`, `'granular'`, `'paulstretch'`, or `'auto'` (`@audio/shift` chooses by the sound, loading them all); `'phase-lock'`, the default, is the one that streams, follows a curve and takes `voice`. The others render what they shift at once, by a constant number of semitones; a range crossfades in and out of it over 10 ms.<br><sub>≡ Melodyne pitch drawing</sub> |
  263. | `.intonation(factor)` | a voice's rises and falls wider or flatter about its median pitch: `1` as it was, `0` a monotone, `2` twice as wide. Its own cycles re-spaced (as `pitch({ voice: true })`): timing, formants and consonants kept, one voice.<br><sub>≡ Praat's pitch range factor</sub> |
  264. | `.modulation(factor)` | each note's vibrato and wobble about its own slow line: `1` as sung, `0` a smooth line, `2` twice as wide, `-1` inverted. The notes are Tony's (as `stat('notes')`); a note's line, its pitch smoothed over its own vibrato period twice. Its cycles re-spaced as `intonation`'s.<br><sub>≡ Melodyne pitch modulation tool</sub> |
  265. | `.drift(factor)` | each note's slow line about its centre: `1` as sung, `0` held at its centre with its vibrato kept, `2` twice the wander.<br><sub>≡ Melodyne pitch drift tool</sub> |
  266. | `.center(factor, {scale?, root?, a4?})` | each note's centre (the median of its middle half) about the nearest note of `scale` from `root`: `1` as sung, `0` on the note, `0.5` half way, all that moves inside it kept; chromatic by default, `@audio/note`'s scale names or degrees. A note is moved from its first frame where it starts after silence; where two meet, the move crosses over 100 ms.<br><sub>≡ Melodyne Correct Pitch, pitch center</sub> |
  267. | `.formant(semitones)` | moves the formants (the spectral envelope: a voice's vowels, the size of its head), keeps the pitch. A number, a curve `{t, v}` or `t => semitones`. Any sound; with `pitch()`, a voice kept its own or made another's.<br><sub>≡ Melodyne formant tool, Praat's formant shift ratio</sub> |
  268. | `.remix(channels)` | channel count (down per ITU-R BS.775: 7.1 → 5.1, stereo, mono), or a map: `[1, 0]` swaps L/R, `null` a silent channel. |
  269. Every op takes a trailing `{at, duration, channel}` range, except channel-changing `remix` and `crossover`, and every op that keeps the timeline a `mix`: how much of its output is heard, the rest its input, 0 to 1 or a curve `{t, v}` over time (`{ mix: { t: [11.99, 12, 14, 14.01], v: [1, 0, 0, 1] } }` turns it off from 12 s to 14 s, as RX's Restore Selection; an effect's own `mix` is its own). Times are seconds or strings (`'1:30'`, `'2m'`); negative counts from the end. FFmpeg's short names work wherever the long ones do: `d` for `duration`, `xfade` for `crossfade` (`a.remove({ at: 1, d: 0.5, xfade: 0.01 })`).
  270. ```js
  271. a.trim(-30) // strip silence below -30dB
  272. a.remove({ at: '2m', duration: 15 }) // delete 2:00–2:15, close gap
  273. a.remove(12.3, 0.4, '10ms') // cut a breath, crossfaded: no click
  274. a.insert(intro, { at: 0 }) // prepend; .insert(3) appends 3s silence
  275. a.copy(60, 30).paste(120) // duplicate the chorus at 2:00
  276. a.cut(2, 1).paste(5) // move 2s–3s to 5s of the shortened timeline
  277. a.move({ at: 2, duration: 1, to: 5 }) // slide 2s–3s over 5s–6s, silence left at 2s–3s
  278. let [pt1, pt2] = a.split('30m') // zero-copy parts
  279. let hook = a.clip({ at: 60, duration: 30 }) // zero-copy excerpt
  280. a.stretch(1.1) // 10% longer, same pitch
  281. a.warp([[1, 1], [2, 2.4], [3, 3]]) // the hit at 2s lands at 2.4s; 1s–3s keeps its length
  282. a.pitch(-2) // 2 semitones down, same tempo
  283. a.pitch({ t: [1, 1.2, 2, 2.2], v: [0, 3, 3, 0] }, { voice: true }) // a note drawn 3 semitones up, formants kept
  284. a.intonation(1.5, { at: 2, d: 3 }) // 2s–5s: every rise and fall half as wide again
  285. a.center(0).drift(.5).modulation(.7) // each note on its semitone, half its drift and 70% of its vibrato left
  286. a.formant(-2) // a larger, darker voice on the same notes
  287. a.remix([0, 0]) // L→both; .remix(1) for mono
  288. ```
  289. ### Process
  290. | Method                         | Description                                                                                                                         |
  291. |:--|:--|
  292. | `.gain(dB, opts?)` | `{ unit: 'linear' }` takes a multiplier. |
  293. | `.fade(in, out?, curve?)` | curves `'linear'` `'exp'` `'log'` `'cos'`, as functions in `audio.op('fade').curves`. `{start, end}` levels 0..1 fade between any levels (a duck); `{mid}` skews the half-amplitude point.<br><sub>≡ Audacity adjustable-fade</sub> |
  294. | `.normalize(target?, mode?)` | remove DC, normalize. Loudness targets hold a true-peak ceiling, -1 dBTP by default: a lookahead limiter, then the loudness it took made back up. Presets per Apple Podcasts, Spotify, EBU R 128 (ITU-R BS.1770-4):<br>`'podcast'` -16 LUFS<br>`'spotify'` -14 LUFS (YouTube, Tidal, Amazon Music too)<br>`'broadcast'` -23 LUFS<br>`-18, 'lufs'` any loudness; `-3` peak dB; no arg: peak 0 dBFS; `'rms'` mode<br>an audio instance: its integrated loudness<br>`{ ceiling: -2 }` dBTP, `false` off<br>`{ dc: false }` keep DC<br>`{ adaptive: true }` on a live stream, start at once: the gain follows what it has heard, the ceiling (the target itself in peak mode) guards what it hasn't. Without it, one gain for the whole selection: a live stream waits for its end.<br><sub>≡ FFmpeg `loudnorm`</sub> |
  295. | `.roomtone(threshold?)` | fill digital silence (≥ 10 ms under -90 dBFS: edited-out pauses, `pad()`) with the recording's own room tone: its quiet stretches that hold still for 0.3 s, 20 dB under the program. `.trim().pad(1.5, 2).roomtone()` gives an audiobook chapter its room tone at each end (ACX rejects digital silence). Where every pause was gated or cut, there is no room to take, and nothing changes.<br><sub>≡ iZotope RX Ambience Match</sub> |
  296. | `.mix(source, at?, gain?)` | overlay at `at` seconds, source level `gain` dB.<br><sub>≡ FFmpeg `amix` weights</sub> |
  297. | `.crossfade(source, duration?, curve?)` | append with overlap, default 0.5s. `'cos'` (default) suits similar material; `'equal'` (equal-power) keeps loudness across unrelated tracks; each in `audio.op('crossfade').curves`. With no source, `.crossfade({ at, duration })` crossfades across the range, as an editor crossfades a selection: the audio before it fades into the audio after it, and the range goes.<br><sub>≡ FFmpeg `acrossfade`</sub> |
  298. | `.pan(value, opts?)` | −1 left, 0 center, 1 right. |
  299. | `.write(data, {at?, d?})` | overwrite from `at` with raw PCM or another sound, as a tape records over what is there; what runs past the end extends it. With `d`, all of `d` is overwritten, silence past the source's end. |
  300. | `.transform(fn)` | inline `(input, output, ctx) => void`. Not serialized. |
  301. ```js
  302. a.gain(-3) // reduce 3dB
  303. a.gain(6, { at: 10, duration: 5 }) // boost range
  304. a.gain(t => -12 * Math.cos(t * TAU)) // automate over time
  305. a.fade(0.5, -2, 'exp') // 0.5s in, 2s exp fade-out
  306. a.normalize('podcast') // -16 LUFS, -1 dBTP
  307. a.normalize(-27, 'lufs', { ceiling: -2 }) // Netflix
  308. a.mix(voice, { at: 2 }) // overlay at 2s
  309. a.mix(bed, 0, -18) // music bed, 18 dB under
  310. a.crossfade(next, 2) // 2s crossfade into next
  311. a.crossfade(song2, 4, 'equal') // equal-power, for unrelated tracks
  312. a.pan(-0.3, { at: 10, duration: 5 }) // pan left for range
  313. ```
  314. ### Filter
  315. | Method                         | Description                                                                                                                         |
  316. |:--|:--|
  317. | `.highpass(freq, order?)`, `.lowpass(freq, order?)` | Butterworth pass filter; even integer order ≥ 2: 2 (12 dB/oct, default), 4 (24), 6, 8, … Other orders are rejected. |
  318. | `.bandpass(freq, Q?)`, `.notch(freq, Q?)` | band-pass / notch. |
  319. | `.allpass(freq, Q?)` | phase shift, unity magnitude. |
  320. | `.phase(angle?)` | every frequency's phase turned by `angle` degrees (a Hilbert transform; a number, a curve `{t, v}` or `t => degrees`): magnitudes, RMS and loudness as they were, and `-angle` turns it back. `180` inverts polarity, sample for sample (`{ channel: 1 }`: one channel's). Unset, the angle over time that lowers the peaks most, the channels together, never raising one: a voice stands more evenly about zero: its peak 0.7 and 0.8 dB lower on average on VoiceBank's two test speakers (30 sentences each), up to 2.9; where two voices take turns, each its own angle. Streams 0.28 s behind.<br><sub>≡ iZotope RX Phase</sub> |
  321. | `.lowshelf(freq, dB)`, `.highshelf(freq, dB)` | shelf EQ. |
  322. | `.eq(freq, gain, Q?)` | parametric EQ. |
  323. | `.filter(type, ...params)` | by type name, or a custom filter function. |
  324. All biquads.
  325. ```js
  326. a.highpass(80).lowshelf(200, -3) // rumble + mud
  327. a.eq(3000, 2, 1.5).highshelf(8000, 3) // presence + air
  328. a.notch(50) // remove hum
  329. a.allpass(1000) // phase shift at 1kHz
  330. a.phase() // the peaks lowered, nothing heard
  331. a.phase(180, { channel: 1 }) // the right channel's polarity
  332. a.filter(customFn, { cutoff: 2000 }) // custom filter function
  333. ```
  334. ### Effect
  335. | Method                         | Description                                                                                                                         |
  336. |:--|:--|
  337. | `.vocals(mode?, {model?})` | mid/side: `'isolate'` (default) keeps center, `'remove'` keeps sides. `model` separates with a trained model instead, `'mel-roformer'` (Kim's Mel-Band RoFormer, MIT weights, the vocals alone, on WebGPU where there is one: MUSDB18 previews' vocals 12.08 dB SDR, the highest here, 0.44 real-time on an M4 Max), `'scnet-large'` (SCNet, MIT weights), `'umxhq'` (Open-Unmix, MIT weights) or `'htdemucs'` (Hybrid Transformer Demucs, weights for research only), through the optional `@audio/neural-separate`; `'remove'` then subtracts the model's vocals, so the two add back to the input sample for sample: a voice and the rest, each repaired apart and mixed (in Node, `audio.memo = new Map()` runs the model once for both; the repl's Split the voice to a new track). `'scnet-large'` and `'scnet'` are hosted; other weights are exported locally ([how](https://github.com/audiojs/neural/tree/main/packages/neural-separate#weights)) or served from `weights`.<br><sub>≡ SoX `oops`; SCNet, Demucs, Open-Unmix</sub> |
  338. | `.rebalance(vocals?, bass?, drums?, other?, {model?, residual?, step?})` | a song's vocals, bass, drums and the rest, each at its own level, dB (−Infinity mutes): a separation model splits the input into the four stems and adds each one's change to it, so 0 dB each leaves the input sample for sample and what the model gives no stem stays as it was, or at `residual` dB (−Infinity: the stems alone, so a stem alone and the input less it add back to it, sample for sample). `model` `'scnet-large'` (default: SCNet-large, MIT weights, 169 MB), `'scnet'` (43 MB, a third of the time), `'htdemucs'` (weights for research only) or `'umxhq'`, through the optional `@audio/neural-separate`, its weights exported locally ([how](https://github.com/audiojs/neural/tree/main/packages/neural-separate#weights)) or served from `weights`; the input is separated once, in 11 s segments every `step` of one (0.5; 0.25 hears each sample four times, twice the time for 0.07 to 0.17 dB more), gains moved read the stems back. MUSDB18 test previews, BSSEval v4 SDR of each stem soloed, vocals · bass · drums · other: 10.75 · 8.17 · 10.30 · 6.94, iZotope RX 12 Music Rebalance at Best 10.89 · 9.66 · 9.85 · 6.45, ahead on 33, 32, 38, 32 of the 50 songs; vocals 6 dB up 20.2 dB against 19.5, drums 6 dB down 21.9 against 20.9 (`bench/rx/separate.mjs`).<br><sub>≡ iZotope RX Music Rebalance</sub> |
  339. | `.scene(dialogue?, music?, effects?, {model?})` | a soundtrack's dialogue, music and effects, each at its own level, dB (−Infinity mutes): a separation model splits the input into the three stems and adds each one's change to it, so 0 dB each leaves the input sample for sample. `model` `'mrx'` (default: MRX, Petermann et al., ICASSP 2022, MERL's MIT weights, 122 MB, a fifth of real time) or `'tiger'` (TIGER, ICLR 2025, Apache-2.0 weights, 29 MB, about 50 times the time: on 30 of the test clips 12.7 · 10.2 · 8.1 dB against MRX's 10.9 · 5.2 · 5.7, the dialogue 6 dB up 21.0 against 18.7), through the optional `@audio/neural-separate`, its weights exported locally ([how](https://github.com/audiojs/neural/tree/main/packages/neural-separate#weights)) or served from `weights`; the input is separated once, gains moved read the stems back. Divide and Remaster v3's English test set, 150 clips of 60 s, SNR median, dialogue · music · effects: 11.8 · 5.3 · 6.0 dB (the dialogue as `deepfilter(0)` takes it: 10.5; its authors' Bandit v2, weights CC BY-SA: 15.6 · 10.4 · 9.9); the dialogue 6 dB up 18.9 dB from the true remix, against 7.2 left as it is (`bench/rx/scene.mjs`). RX 12 Scene Rebalance runs only inside Pro Tools (AAX): not measured.<br><sub>≡ iZotope RX Scene Rebalance</sub> |
  340. | `.dither(bits?, {shape?})` | TPDF, default 16-bit. `shape: true` adds 2nd-order noise shaping: quantization noise moves above ~Nyquist/2, audibly quieter. |
  341. | `.crossfeed(freq?, level?)` | headphone crossfeed, default 700 Hz, 0.3.<br><sub>≡ SoX `earwax`, bs2b</sub> |
  342. | `.convolve(ir, mix?, {normalize?, tail?})` | the sound through an impulse response: a room, a plate, a cabinet, a microphone as captured. `ir` a file or URL, an audio instance or channels of samples, resampled to the sound's rate; channel c through its channel c, wrapping. No latency (Gardner's partitioned convolution: a direct head, FFT partitions behind it), its decay rendered past the end (`tail: false`, none). `mix` 0 to 1 (1), `normalize: true` the IR at unit energy.<br><sub>≡ Pedalboard `Convolution`, FFmpeg `afir`</sub> |
  343. | `.resample(rate, {type?})` | upsampling defaults to linear, downsampling to anti-aliased windowed sinc, its taps widening with the ratio. `type: 'sinc'` or `'linear'` forces one. |
  344. | `.crossover(...freqs)` | N split frequencies → N+1 bands × channels, band-major. Linkwitz-Riley 4th order; bands sum back flat.<br><sub>≡ FFmpeg `acrossover`</sub> |
  345. | `.match(ref, amount?)` | match EQ: up to 8 parametric bands fit to the reference/source spectrum ratio. Tone only; loudness stays with `normalize`. Streams `{lookahead}` s behind (10), refitting as it hears more. `{ midside: true }` matches a stereo pair's mid and side apart, and the side level to the reference's width.<br><sub>≡ iZotope Ozone Match EQ</sub> |
  346. | `.align(ref, {within?, span?, drift?})` | moved in time onto a reference it plays with: a part remade elsewhere or recorded apart put back where the original stood, its head cut where it is late, silence before it where early. The lag to the sample where the two share a waveform (GCC-PHAT), else by their onsets (spectral flux); within ±`within` s (10), read from the first `span` s (60). Where both last 20 s and the lag grows along it (`drift: false` skips this), a rate first: a copy on a clock running apart read back at the reference's, to the sample, its pitch with it; a part played at another tempo (±10 %) stretched onto it, its pitch kept, its ends within 2 ms on most of `bench/align.mjs`. |
  347. | `.master(ref, opts?)` | master to a reference track: `match` in mid and side, then `normalize` to the reference's integrated loudness under -1 dBTP (`{ ceiling }`).<br><sub>≡ Matchering</sub> |
  348. | `.auto(type?, {intensity?, targetLufs?, ceiling?})` | the take repaired and finished by what it carries (`@audio/chain`): measured, then each repair on its own evidence (declip, declick, deplosive, dehum, denoise, dereverb, deesser), and only where measured need calls for them an EQ (a voice past the spread clean voices keep about the speech target), glue (a loudness range over 12 LU), and loudness at the `type`'s target (`'speech'`, default, −16 LUFS; `'music'` −14; `'voice-music'`), its true peak under `ceiling` dB (−1) by up to 6 dB × `intensity` of true-peak limiting where the peaks need it: a take that needs nothing comes back as it was but for its level. A bed under speech goes to DeepFilterNet3 where `@audio/neural-denoise` is installed (`deepfilter()`'s model, the noise 60 dB × `intensity` down, the recipe naming it), else to OM-LSA; under `'speech'` the voice is what is kept. `intensity` 0–2 (1) scales how hard; 0 stops the gain at the ceiling. On takes with several defects made on clean recordings (`bench/rx/assistant.mjs`), its repairs against iZotope RX 12 Repair Assistant tuned: speech PESQ 2.99 against 2.56, SI-SDR 18.4 dB against 13.0, DNSMOS OVRL 3.12 against 3.12, music ODG −1.69 against −2.23; clean speech 4.46 against 3.81; steady beds level (PESQ 2.97 against 2.94), RX ahead on DEMAND's (2.53 against 2.90: four of nine read as no bed). Without the neural package speech 2.79, beds 2.39 and 2.07. `auto()` whole: 89 % of speech and 98 % of music within 1 LU of the target, true peak at most −1.00 dBTP; the limiting costs a clean mix ODG 0.30 against the repairs (−0.22, input 0.21), a clean voice nothing (PESQ 4.45).<br><sub>≡ iZotope RX Repair Assistant, Dolby.io Media Enhance</sub> |
  349. | `.spectral(band?, gain?, {at, duration})` | gain on a time × frequency region, `band` = `[lo, hi]` Hz; default removes it.<br><sub>≡ Audacity spectral edit, FFmpeg `afftfilt`</sub> |
  350. | `.repair(band?, {at, duration, method?, window?})` | rebuild a damaged range (dropout, beep, click burst) from its surroundings. `method` `'auto'` (default) transplants the passage that joins seamlessly, searched in the `window` s (10) before the range; failing that, AR interpolation up to 70 ms, a sinusoidal bridge beyond. `'ar'`, `'sinusoidal'`, `'similarity'`, `'spectral'` force one.<br><sub>≡ iZotope RX Spectral Repair</sub> |
  351. | `.declick(threshold?, longest?, {at?, duration?})` | remove clicks: a vinyl tick, a bad splice, a digital glitch, a mouth click on a voice. Each stands `threshold` times (8) out of the AR prediction error around it; how far it reaches and whether it rings on are what explains the sound best, and it is rebuilt as the sound's most likely value under it from 46 ms either side; a pulse with its like 2.5–15 ms away (a voice, a plucked string), the sound's own excitation (a plosive, an attack) or a burst over `longest` ms (6) is sound, left alone. Given `{ at, duration }`, the clicks there: looked for only there, none passed over for its likes or its length. Nothing a click doesn't reach changes. Streams about 0.9 s behind. Clicks at 5× the sound around them: ticks 34 dB down, glitches 46, mouth clicks 28, more than iZotope RX 12 De-click and Mouth De-click take off in every kind and size measured (`@audio/denoise-declick`'s README).<br><sub>≡ iZotope RX De-click, Audacity Click Removal</sub> |
  352. | `.denoise(reduction?, threshold?, {noise})` | remove a noise that holds still (hiss, hum and buzz, a fan, room tone, tape), learned where it plays alone: `noise` is that `{ at, duration }` of the op's input, or several, or a print saved from `stat('print')`. It goes `reduction` dB down (12) everywhere, or in the op's own `{ at, duration }`, or only in a `band` `[low, high]` Hz, the rest as it was; what stays is the same noise, quieter, without musical tones. `threshold` (dB) raises the print: more of the quiet counts as noise. OM-LSA on the held noise (`@audio/denoise-omlsa`), each channel its own print; a live source renders once the range has arrived. VoiceBank+DEMAND PESQ, the noise learned from the half second before each speaker starts: noisy 1.97, `omlsa()` 2.40, `denoise()` 2.48. For noise that moves: `omlsa()`, `deepfilter()`.<br><sub>≡ iZotope RX Spectral De-noise (Learn), Adobe Audition Noise Reduction (noise print), Audacity Noise Reduction</sub> |
  353. | `.recover(cutoff?)` | a voice's band lost above `cutoff` Hz (a phone line, a narrowband codec, a 16 kHz file) rebuilt from its own harmonics: its pitch by pYIN, each harmonic above the edge synthesized at the level the ones under it set, falling 12 dB an octave, the breath between them as noise, falling 2; added only where the input lacks the band, so it meets the input's roll-off and leaves what lies under the edge as it was. The cutoff, unless given, where the long-term spectrum falls 25 dB within a third of an octave and stays under (no voice rolls off so steeply); none, nothing changes. VoiceBank's test set cut to 8 and 16 kHz: log-spectral distance over the band rebuilt 13.9 and 12.3 dB, `sbr()` 20.0 and 16.7, the input 41.3 and 40.6 (`bench/recover.mjs`); PESQ falls with any band added (`bench/recover.py`), so a listening test is still to rank it.<br><sub>≡ iZotope RX Spectral Recovery</sub> |
  354. | `.deepfilter(limit?, {floor?, music?, weights?, device?})`, `.rnnoise(limit?)` | neural speech denoising through the optional `@audio/neural-denoise`: it also removes noise that moves (keys, traffic, a busy room). `deepfilter` runs DeepFilterNet3, its 8 MB model downloaded once, over the whole input before rendering; `rnnoise` streams RNNoise, weights in the package, 30 ms behind. `limit` is how far the noise goes down, in dB: `deepfilter`'s 18 is the most before the voice itself sounds filtered (DNSMOS SIG holds to 18 and falls past it), so room tone stays, and noise closer to the voice than `floor` dB (40) goes down to that floor, so noise as loud as the voice is taken, not left 18 dB under it; `rnnoise`'s 16 is the most before it cuts into the voice: RNNoise turns down word ends even with no noise at all (10 dB at 16, 31 unlimited); `0` lifts it. `deepfilter` keeps held sung notes, which the model alone takes for noise (VocalSet: 1.9 dB down, not 39), and hears a 16 kHz file's or a codec's empty top band as the noise floor the model trained with (16 kHz VoiceBank+DEMAND PESQ 2.83, not 2.72). VoiceBank+DEMAND PESQ: noisy 1.97, `wiener()` 2.19, `rnnoise()` 2.49, `deepfilter()` 3.05, `deepfilter(0)` 3.15, iZotope RX 12 Dialogue Isolate 2.72; in noise, babble and music beds 5 to −5 dB under the voice and in rooms, `deepfilter()` 1.99 against RX's 1.86 (`bench/rx/isolate.mjs`). Music alone passes untouched (`music: 'enhance'` enhances it too).<br><sub>≡ DeepFilterNet, RNNoise, iZotope RX Dialogue Isolate</sub> |
  355. | `.dereverb(reduction?, {music?, method?, weights?, device?})` | the room off a voice, early reflections and tail alike, by a trained model through the optional `@audio/neural-separate`: anvuew's room de-reverb, a mono Band-Split RoFormer trained on voices in rooms simulated with pyroomacoustics (GPL-3.0 weights, 60 MB, downloaded once), over the whole input before rendering, each channel apart. The room goes `reduction` dB down (60: gone; 12 leaves a hint of it), mixed back from what the model took, so moving it runs no model again. To the model all that is not a voice is room, so music passes untouched (`music: 'pass'`, `@audio/neural-denoise`'s speech/music guard, as `deepfilter`'s, heard on the take and on the model's voice: a long room makes a voice ring like a held note); a voice over a music bed counts as speech, and the bed goes with the room. Voices in 19 MIT IR Survey rooms the model never heard (`bench/rx/dereverb.mjs`, test; T60 0.38–1.85 s): PESQ 1.39 → 2.65, STOI 0.77 → 0.94, the tail 15.6 dB down (long takes 1.37 → 2.58, 17.5 dB), against iZotope RX 12 De-reverb tuned 1.45 and Dialogue Isolate with its reverb off 1.79; with noise 10 dB under the voice 1.26 → 1.70 (RX Dialogue Isolate 1.33); dry voices 4.64 → 4.55, their 125 Hz octave 2.1 dB down. Music through the guard comes back within 0.9 dB in every octave (MUSDB18 previews, GuitarSet, four pieces: as they are, cut into phrases, in a room); sung tones, a voice to the model and to the guard, lose up to 6 dB in an octave in a room. `method: 'wpe'` needs no model: weighted prediction error and a late-reverb gain (`@audio/denoise-dereverb`; `strength`), about 6 dB of the late tail (PESQ 1.53), a dry take, a take without pauses or music untouched.<br><sub>≡ iZotope RX De-reverb, Dialogue Isolate's reverb</sub> |
  356. | `.vocos({crossover?, weights?, device?})` | a voice made again by Vocos, a neural vocoder (Siuzdak, ICLR 2024; charactr's MIT weights), through the optional `@audio/neural-vocode`: its mel spectrogram, each STFT frame's magnitude and phase predicted afresh, so what another vocoder left (a TTS's frames ringing on, faint copies 10, 20, 30 ms later) gives way to Vocos's own. Vocos makes nothing above 12 kHz: the input is kept above `crossover` Hz (11000), Linkwitz-Riley split; 0 takes Vocos's alone. Its 27 MB model downloads once. A speech vocoder: music comes back altered |
  357. | `.restore(strength?, {weights?, device?})` | a voice back to a studio take by one network through the optional `@audio/neural-restore` (trained on English, French and Russian speech and on singing, with rooms, noise, microphones, codecs and band limits made on the fly): each STFT bin's level changed, unbounded, so a band can come back from nothing, and its phase turned; the room, the noise, the codec and a missing top band gone, 48 kHz full band. `strength` (1) mixes the input back: `0.7` is 70 % the model's. 17.9M parameters, 35 MB, over the whole input before rendering, each channel apart. On 24 degraded VoiceBank takes: log-spectral distance 0.79 against `deepfilter(0)`'s 2.71, PESQ 2.40 against 2.07. The package is not on npm yet and its weights are not hosted: in Node the neural cache's copy, in a page `weights`. |
  358. | `.derustle(reduction?, {ambience?})` | a clip-on mic's clothing rustle taken off the voice, the room's steady tone kept: `deepfilter`'s model (the two share its run on the same input), what it took mixed back as `deepfilter` mixes it, rustle down to `reduction` dB under the voice (45), the room's tone 12 dB down (`ambience: false` takes it all). Settings chosen on clothing rustle under 10 to 37 s talkers (Freesound's CC0 and CC BY clothing foley and mic handling: no lavalier rustle is openly licensed). Test, 55 talkers, rustle as loud as the voice: PESQ 1.21 → 2.13, SI-SDR −0.3 → 13.5 dB; the room alone 4.37, a clean take 4.61 (`deepfilter()` 2.12, 4.31, 4.59; iZotope RX 12 Dialogue Isolate, RX's hostable dialogue separator, 2.06, 2.96, 3.96) (`bench/rx/derustle.mjs`). RX 12 De-rustle runs only inside Pro Tools (AAX): not measured.<br><sub>≡ iZotope RX De-rustle</sub> |
  359. | `.deconstruct(tonal?, noise?, transient?, {separation?})` | the tonal, noisy and transient parts, each at its own level, dB: median filters over time and over frequency (Fitzgerald 2010), the noise what is neither by `separation` (2; Driedger, Müller & Disch 2014), soft masks the channels share. The parts add back to the input; 0 dB each leaves it sample for sample. A chord lands 100 % in the tonal part, clicks in the transient, a hiss 68 % in the noise (1 % in each other).<br><sub>≡ iZotope RX Deconstruct</sub> |
  360. | `.azimuth(delay?)` | a stereo pair lined up in time and polarity, the second channel moved onto the first. Unset, the delay is measured every second (generalized cross-correlation with Knapp & Carter's 1976 weighting, refined between samples: an exact fractional shift comes back to 0.0001 samples) and followed as it wanders; a channel wired backwards turned back; a pair already in line, as a mix's stereo image is, left sample for sample. `delay` ms sets it by hand.<br><sub>≡ iZotope RX Azimuth</sub> |
  361. | `.dewow(mode?)` | wow and flutter: the speed of the disc or tape measured over time and the sound read back at its inverse; from the music's own notes (a disc's once-a-turn wow, `'partial'`), a pilot or calibration tone (`'reference'`, `refFreq`), or one voice's pitch (`'pitch'`). Nothing proven, nothing changed: a recording without wow comes back sample for sample (`@audio/denoise-dewow`).<br><sub>≡ iZotope RX Wow & Flutter</sub> |
  362. | `.desqueak(squeak?, {pick?, amp?})` | a guitar's string squeaks taken down, where a finger slides along a wound string: told from the notes by what a note is not (partials that hold their bins, read on 93 ms frames; a pluck's low partials or click), held 45 ms or more, each bin down to what it holds around the squeak, `squeak` dB at most (30); the notes ringing under it keep their partials, and the take outside the squeaks stays sample for sample. `pick` dB softens each pluck's click to the level its note rings at 15 ms on (-9 restores attacks made 6–12 dB harsher); `amp` dB takes a steady hiss and buzz down (OM-LSA on a print from the quietest frames, a note's peaks kept). GuitarSet test takes against iZotope RX 12 Guitar De-noise: squeaks added, 3.3 dB of their error gone against 2.7, SNR 23.2 dB against 20.5, 2.4 % of a clean take's samples moved against 12.9 %; harsh picks 8.9 dB against 3.3 tuned; squeaks as recorded 12.8 dB median against 13.5, half the plucks and clicks RX touches (`@audio/denoise-desqueak`, `bench/rx/guitar.mjs`).<br><sub>≡ iZotope RX Guitar De-noise</sub> |
  363. | `.codec(format?, bitrate?, {quality?})` | the sound as a lossy codec gives it back: `'mp3'` (default), `'aac'`, `'opus'` or `'vorbis'` at `bitrate` kbps (128), `'mp3'` VBR at `quality` (LAME's -V: 0 best, under 10), `'gsm'` (GSM 06.10 full rate, 13 kbps at 8 kHz: a 1990s phone line), encoded and decoded in place. Each codec's delay undone (its own gapless information, then checked by correlation), so it lines up with the input within 0.03 of a sample (at 44.1 and 48 kHz): A/B it, or hear what it takes out. A codec moves the peaks: masters normalized to -14 LUFS under -1 dBTP came back up to 0.5 dB over it through Opus at 160 kbps, 0.3 through Vorbis, under 0.1 through MP3 320 and AAC 256 (7 s excerpts of 8 MUSDB18 mixes); `.codec('opus', 160).stat('truepeak')` measures it.<br><sub>≡ iZotope RX Streaming Preview, Pedalboard `MP3Compressor`, `GSMFullRateCompressor`</sub> |
  364. ```js
  365. a.vocals() // isolate center-panned vocals
  366. a.vocals('remove') // remove vocals (karaoke)
  367. a.vocals({ model: 'umxhq' }) // vocals by a separation model
  368. a.rebalance({ vocals: 6, drums: -3 }) // the vocals up, the drums down
  369. a.rebalance(-Infinity) // a karaoke track
  370. a.scene(6) // a film's dialogue 6 dB up over its music and effects
  371. a.scene(-Infinity) // a music and effects track
  372. a.dither(16) // TPDF dither to 16-bit
  373. a.dither(16, { shape: true }) // noise-shaped
  374. a.crossfeed() // headphone crossfeed
  375. a.convolve('hall.wav', 0.3) // 30 % of a captured hall
  376. a.resample(48000) // resample to 48kHz (linear)
  377. a.resample(96000, { type: 'sinc' }) // high-quality windowed-sinc
  378. a.match(reference, 0.7) // 70% of the way to its tone
  379. a.align(original) // a part remade elsewhere, put back where the original stood
  380. a.spectral([1000, 4000], -30, { at: 2.1, duration: 0.3 }) // a cough
  381. a.repair({ at: 1.2, duration: 0.05 }) // a dropout
  382. a.repair({ at: 42, duration: 1 }) // a lost second of music: the passage that fits
  383. a.declick() // every click
  384. a.declick({ at: 12.31, duration: 0.02 }) // the clicks seen there
  385. a.denoise({ noise: { at: 1.2, duration: 0.5 } }) // hiss learned from a pause, 12 dB down everywhere
  386. a.deepfilter() // speech out of noise: room tone 18 dB down and kept, loud noise to 40 under the voice
  387. a.rnnoise() // the same, streaming
  388. a.dereverb() // the room off a voice
  389. a.dereverb(12) // the room 12 dB down
  390. a.derustle() // a clip-on mic's clothing rustle, the room kept
  391. a.vocos() // a TTS voice made again by a neural vocoder, its frame echo gone
  392. a.deconstruct(0, -12) // the hiss 12 dB down, tones and attacks as they were
  393. a.azimuth() // a tape's two tracks back in line
  394. a.dewow({ mode: 'reference', refFreq: 1000 }) // a transfer's wow, read from its calibration tone
  395. a.desqueak() // a guitar's string squeaks
  396. a.desqueak({ pick: -9, amp: -20 }) // and its harsh picks, and the amp's hiss and buzz
  397. a.codec('aac', 256) // what a 256 kbps AAC stream does to it
  398. a.codec('gsm') // a phone call
  399. ```
  400. ### I/O
  401. | Method                         | Description                                                                                                                         |
  402. |:--|:--|
  403. | `await .read(opts?)` | rendered PCM. `{ format, channel }` to convert. A source still arriving is waited for: a range until it has arrived, all of it until the end (an endless stream: read ranges, or `stream()`). |
  404. | `await .save(path, opts?)` | encode + write, format from extension. Lossless keeps the source depth; `{ bitDepth, bitrate, quality, codec }` set the encoder; m4a and mp3 write markers as chapters. Output streams as it encodes, headers patched with their totals at the end (a pipe keeps them "unknown"); m4a from a live source is fragmented. A video source saved to `.mp4`/`.mov` keeps its picture: only the audio track changes. `.edl`, `.otio`, `.fcpxml` write the cuts. |
  405. | `await .encode(format?, opts?)` | encode to `Uint8Array`; `'edl'`, `'otio'`, `'fcpxml'`: the cuts, as UTF-8. |
  406. | `await .cuts(format?, opts?)` | the edits as a cut list for a video editor: `'edl'` (CMX 3600: Premiere, Resolve, Avid), `'fcpxml'` (Final Cut Pro, Resolve), `'otio'` (OpenTimelineIO); none gives `{ fps, clips: [{ at, duration, from, rate, source }] }`. Cuts, moves, gaps and inserted files place the clips; speed, stretch and warp set their rate; markers go along. Processing is not in the list: lay the processed audio under the picture. Cuts land on the video track's frames (MP4/MOV), else `{ fps }` (30), each within half a frame of the sound; its timecode track starts the source times, drop-frame as it counts (`{ dropFrame }`). The CLI's `save cuts.edl` writes one. |
  407. | `.clone()` | independent edits, shared pages. |
  408. | `.push(data, format?)` | feed PCM into a pushable instance; `.stop()` finalizes. |
  409. ```js
  410. let pcm = await a.read() // Float32Array[]
  411. let raw = await a.read({ format: 'int16', channel: 0 })
  412. for await (let block of a) send(block) // async-iterable over blocks
  413. await a.save('out.mp3') // format from extension
  414. await a.save('book.mp3', { bitrate: 192 }) // ACX: 192 kbps CBR
  415. await a.save('master.wav', { bitDepth: 24 }) // 24-bit
  416. await a.save('talk.mp4') // video in, video out
  417. let bytes = await a.encode('flac') // Uint8Array
  418. let b = a.clone() // independent copy, shared pages
  419. let src = audio() // pushable source
  420. src.push(buf, 'int16') // feed PCM
  421. src.stop() // finalize
  422. ```
  423. ### Playback / Recording
  424. | Method                         | Description                                                                                                                         |
  425. |:--|:--|
  426. | `.play(opts?)` | `{ at, duration, loop, volume, rate, paused, device }`. `at` defaults to `currentTime` (the start once ended); playing already, it jumps there without a gap. `device`: an output by id or name (a page plays it on a context of its own, sunk there). |
  427. | `.play({ from: b })` | take over `b`'s playback where it is (its span, loop, volume, rate, pause), crossfaded, no gap; `b` stops. |
  428. | `.pause()`, `.resume()`, `.seek(t)`, `.stop()` | each ramps over 5 ms, none clicks; `seek` crossfades, and in a loop stays in its span. `stop()` also ends recording. |
  429. | `.record(opts?)` | mic. `{ device, sampleRate, channels, monitor }`. `device`: an input by id or name. `monitor`: the take through its edits as it comes in, to the default output (`true`) or one by id or name, about 50 ms behind (the input's read, one block, the output's ring; and the edits' own lookahead): input monitoring, as a DAW's through a track's inserts.<br><sub>≡ Pedalboard `AudioStream`</sub> |
  430. | `audio.devices()` | the inputs and outputs: `{ input: [{ id, name, default }], output: [...] }`. Node: the platform's own ids (CoreAudio's UID, WASAPI's endpoint id, ALSA's name); a page: `enumerateDevices()`, names once the microphone is allowed. |
  431. | `audio.context` | the page's one AudioContext, which playback uses: made on first use, resumed by the first gesture; set your own before playing. |
  432. Playback renders up to 2 s ahead into an AudioWorklet on `audio.context` (Node: @audio/speaker), so a busy main thread doesn't stop it, and sounds within milliseconds of `play()` (the device's own latency aside). An edit to the playing instance is heard ~50 ms later where it happens: the audio rendered ahead gives way, crossfaded. A source still arriving (decoding, pushed) plays what has come and goes on as more comes. Any channel count plays as it is; the device downmixes.
  433. ```js
  434. a.play({ at: 30, duration: 10 }) // play 30s–40s
  435. await a.played // wait for sound
  436. a.volume = 0.5; a.loop = true // live adjustments
  437. a.muted = true // mute without changing volume
  438. a.playbackRate = 1.5 // faster, the pitch kept
  439. a.preservesPitch = false // the pitch follows the speed, as a tape's
  440. a.pause(); a.seek(60); a.resume() // jump to 1:00
  441. a.highpass(80) // an edit while playing: heard where it happens
  442. b.play({ from: a }) // b takes over at the same place, crossfaded
  443. b.stop() // end playback or recording
  444. await audio.context.audioWorklet.addModule('./scrub.js') // your own nodes, on the same context
  445. let scrub = new AudioWorkletNode(audio.context, 'scrub')
  446. let mic = audio()
  447. mic.record({ sampleRate: 16000, channels: 1 })
  448. mic.stop()
  449. await audio.devices() // { input: [{ id, name, default }, …], output: […] }
  450. let take = audio()
  451. take.compressor().plate(0.2) // edits, heard as they come in
  452. take.record({ device: 'USB', monitor: 'Headphones' }) // by id, name, or part of a name
  453. ```
  454. ### Metering
  455. | Method                         | Description                                                                                                                         |
  456. |:--|:--|
  457. | `.meter(what, cb?)` | live per-block stats of what plays, delivered as it is heard: `rms`, `peak`, `ms`, `min`, `max`, `dc`, `clipping`, `spectrum`, or your own. Without `cb`, read `.value`. Returns `{ value, stop() }`. The same on a worker facade: measured in the worker, delivered on the page. |
  458. | Option                         | Description                                                                                                                         |
  459. |:--|:--|
  460. | `type` | stat name, array of names, or omit for all block stats. |
  461. | `channel` | `n` for one channel, `[n, m]` per-channel, or omit for scalar avg (mirrors `a.stat()`). |
  462. | `smoothing` | one-pole EMA time constant τ, in seconds. |
  463. | `hold` | peak-hold decay τ, in seconds. |
  464. | `bins`, `fMin`, `fMax` | spectrum resolution and range (when `type: 'spectrum'`). |
  465. ```js
  466. a.meter('rms', v => draw(v)) // scalar avg across channels
  467. a.meter(['rms', 'peak'], v => draw(v)) // { rms, peak }
  468. a.meter({ type: 'rms', channel: [0, 1] }, v => draw(v)) // [L, R]
  469. a.meter({ type: 'spectrum', bins: 64, smoothing: 0.15 }, drawFFT) // Float32Array of mel bins
  470. a.meter({}, ({ delta, offset }) => draw(delta)) // no type → all block stats
  471. let m = a.meter({ type: 'rms' }) // pull form
  472. requestAnimationFrame(function tick() { draw(m.value); requestAnimationFrame(tick) })
  473. m.stop() // release
  474. ```
  475. ### Analysis
  476. | Method                         | Description                                                                                                                         |
  477. |:--|:--|
  478. | `await .stat(name, opts?)` | one value; with `{ bins: n }` a `Float32Array`, the value over each of n spans of the range: where, not only how much (lists, a key and spectra come whole; `bins` sizes spectrum and cepstrum); an array of names gives an array. `{ channel: n }` one channel, `[n, m]` per channel; `{at, duration}` sub-range. |
  479. | `await .detect(opts?)` | `{ bpm, confidence, beats, onsets }` in one pass; `{ channel }` as in `stat`. |
  480. | `await .check(spec)` | pass or fail against a delivery spec: `{ pass, rules: [{ name, value, unit, min, max, pass }] }`. `'acx'` (RMS, peak, noise floor, room tone, 44.1 kHz), `'podcast'` (Apple: -16 LUFS ±1, ≤ -1 dBTP), `'spotify'` (plays at -14 LUFS, ≤ -1 dBTP), `'broadcast'` (EBU R 128: -23 ±0.2 LUFS, ≤ -1 dBTP), `'netflix'` (dialog -27 ±2 LUFS, ≤ -2 dBTP). Each limit cites its source in [fn/check.js](fn/check.js). |
  481. | Stat                         | Description                                                                                                                         |
  482. |:--|:--|
  483. | `'db'` | peak amplitude in dBFS. |
  484. | `'rms'` | RMS amplitude, linear (the CLI prints dBFS). |
  485. | `'noisefloor'` | RMS of the quietest 0.4 s, dB: the room between words (ACX Check's measure, sample-exact). |
  486. | `'print'` | the noise print of a range, as `denoise({ noise })` takes it: dB in 1025 bands 23.4375 Hz apart, 0 to 24 kHz (white noise of RMS 0.01 prints −40). |
  487. | `'peak'` | `max(\|min\|, \|max\|)`, linear. |
  488. | `'loudness'` | integrated LUFS (ITU-R BS.1770-4; surround channels weighted, LFE excluded). |
  489. | `'momentary'`, `'shortterm'` | maximum 400 ms / 3 s loudness, LUFS (EBU Tech 3341). |
  490. | `'dialog'` | loudness of the speech only, LUFS: speech found automatically (AES TD1008 dialog loudness). |
  491. | `'dc'` | DC offset. |
  492. | `'clipping'` | clipped samples, at 16-bit full scale (±32767/32768) or beyond (scalar: timestamps, binned: counts). |
  493. | `'silence'` | silent ranges as `{at, duration}`. |
  494. | `'crest'` | peak/RMS in dB. Sine ≈ 3dB, square ≈ 0dB. |
  495. | `'centroid'` | spectral centroid in Hz (brightness). |
  496. | `'flatness'` | spectral flatness: 0 tonal, 1 noise. |
  497. | `'correlation'` | L/R phase correlation, −1 to +1. Mono returns 1. |
  498. | `'max'`, `'min'` | peak envelope per bin, for waveforms. |
  499. | `'spectrum'` | mel spectrum in dB (A-weighted); of several channels, their mean power. |
  500. | `'cepstrum'` | MFCCs. |
  501. | `'bpm'` | tempo. |
  502. | `'beats'`, `'onsets'` | timestamps as `Float64Array` (seconds). |
  503. | `'hits'` | where the level jumps, up (a strike, a swell out of the ground) or down (a stop), over 10 ms and over 50 ms, as `Float64Array` (seconds): each at its attack's zero crossing, the sample to cut at, where `'onsets'` reads 23 ms blocks. |
  504. | `'notes'` | `[{time, duration, freq, midi, note, clarity}]` (pYIN + Tony note HMM); with `robust: true`, the same through noise and rooms (a neural pYIN stage 1); with `poly: true`, polyphonic `[{time, duration, freq, midi, note, velocity, bends}]` (Basic Pitch). |
  505. | `'chords'` | `[{time, duration, label, root, quality, bass, confidence}]`: Chordino on NNLS chroma (Mauch & Dixon 2010), matched to the reference plugin; labels like `'Am'`, `'G7'`, `'C/E'`, `'N'`. |
  506. | `'key'` | `{tonic, mode, label, confidence}` (Krumhansl-Schmuckler). |
  507. | `'voicing'` | share of the range voiced, 0 to 1: frames of pYIN's pitch curve with a pitch, every 10 ms. |
  508. | `'hnr'` | harmonics-to-noise ratio, dB, over the voiced frames (Boersma 1993; matches Praat's To Harmonicity (ac) frame by frame); null where none is voiced. |
  509. | `'harmonic'` | level of the periodic part, dB: the periodic share of each frame's power (Boersma's r) times the power. Unlike RMS, unmoved by noise taken away: what an edit left of a voice. |
  510. | `'similar'` | where else it sounds like `{ at, duration }` (a cough, a click, a beep): `[{ at, duration, score }]`, the range's log-mel patch, each band over its median, slid along the whole by Pearson's correlation, from `threshold` (0.7); `band` `[low, high]` Hz compares those frequencies alone. Also `a.similar(range, opts)`.<br><sub>≡ iZotope RX Find Similar</sub> |
  511. Opts: `bpm`, `beats`, `onsets` take `{ minBpm, maxBpm, delta, frameSize, hopSize }`; `notes` takes `{ minFreq, maxFreq, frameSize, hopSize, minDuration }`; `chords`, `key` take `{ frameSize, hopSize, tuning }` (frames of 16384 samples at 44.1 kHz, 0.34 to 0.51 s at other rates, every eighth of a frame; concert A read from the audio unless `tuning` in Hz is given); `chords` also `boostN` (no-chord bias, 0.1); `key` also `method: 'nnls' | 'pcp'`. `chords` needs `@audio/mir-nnls-chroma` and `@audio/mir-chordino`, `key` needs `@audio/mir-nnls-chroma`: GPL-2.0-or-later translations of the reference plugins, installed by choice (`npm i @audio/mir-nnls-chroma @audio/mir-chordino`); `key` with `method: 'pcp'` needs only the MIT `@audio/mir-chroma` and `@audio/mir-key`, installed with `audio` unless optional dependencies are skipped. `notes` with `robust: true` needs `@audio/neural-pitch` (weights inside): a network's pitch candidates in place of YIN's keep the notes where YIN loses them (Vocadito onsets F 0.76 against 0.53 at 0 dB SNR) and trail it slightly on clean audio, so YIN stays the default. `notes` with `poly: true` takes `{ minFreq, maxFreq, minDuration, onsetThreshold, frameThreshold }` and needs `@audio/neural-transcribe`, whose model downloads on first use; `bends` are cents from the note's pitch per 11.6 ms frame, in 33.3-cent steps (in-tune notes read 0).
  512. ```js
  513. let loud = await a.stat('loudness') // LUFS
  514. let [db, clips] = await a.stat(['db', 'clipping']) // multiple at once
  515. let spec = await a.stat('spectrum', { bins: 128 }) // frequency bins
  516. let [min, max] = await a.stat(['min', 'max'], { bins: 800 }) // peak envelope for canvas rendering
  517. await a.stat('rms', { channel: 0 }) // left only → number
  518. await a.stat('rms', { channel: [0, 1] }) // per-channel → [n, n]
  519. let gaps = await a.stat('silence', { threshold: -40 }) // [{at, duration}, ...]
  520. let bpm = await a.stat('bpm') // 120.5
  521. let beats = await a.stat('beats') // Float64Array [0, 0.5, 1, ...]
  522. let { bpm, confidence, beats, onsets } = await a.detect() // full pipeline, one pass
  523. let notes = await a.stat('notes') // [{time, duration, freq, midi, note: 'A4', clarity}]
  524. let chords = await a.stat('chords') // [{time, duration, label: 'Am', confidence}]
  525. let k = await a.stat('key') // {label: 'C', mode: 'major', confidence}
  526. let coughs = await a.similar({ at: 12.3, duration: 0.4 }) // [{ at, duration, score }]: the other coughs
  527. ```
  528. ### Meta
  529. | Property                         | Description                                                                                                                         |
  530. |:--|:--|
  531. | `.meta` | tags: `{title, artist, album, year, bpm, key, comment, pictures, raw, ...}`. Writable. `meta.raw` holds format-specific blocks untouched (WAV bext/iXML, ID3v2 frames, FLAC blocks). |
  532. | `.meta.pictures` | cover art `[{mime, type, description, data, url}]`. `.url` is a lazy Blob URL (browser) or data URL (Node). |
  533. | `.markers` | `[{time, label}]` in output seconds; edits shift or drop them. |
  534. | `.mark(time, label?)` | a marker at `time`, seconds of the audio as edited so far; `{at, duration}`, a region. Later edits carry it; in silence, at its distance from the sound nearest it; past the end, the end until a later edit makes the audio reach it. Chainable. |
  535. | `.regions` | `[{at, duration, label}]`; edits shift or drop them. |
  536. Parsed on decode, written on save; round-trips WAV, MP3, FLAC.
  537. ```js
  538. let a = await audio('song.mp3')
  539. a.meta.title // 'Track Name'
  540. a.meta.artist = 'Me' // mutate
  541. img.src = a.meta.pictures[0].url // lazy Blob URL
  542. a.crop({ at: 10, duration: 30 })
  543. a.markers // re-projected — outside markers dropped, inside shifted
  544. await a.save('edited.mp3') // tags + pictures preserved
  545. await a.save('stripped.wav', { meta: false }) // opt out
  546. ```
  547. ### Utility
  548. | Method                         | Description                                                                                                                         |
  549. |:--|:--|
  550. | `.on(event, fn)`, `.off(event?, fn?)` | subscribe / unsubscribe. |
  551. | `.undo(n?)` | returns the undone edit, for redo via `.run()`. |
  552. | `.run(...edits)` | apply `['type', opts]` edits: op params (`value`, `freq`, …) plus range keys. |
  553. | `.dispose()` | release resources. Supports `using`. |
  554. | Event                         | Description                                                                                                                         |
  555. |:--|:--|
  556. | `'data'` | pages decoded/pushed. Payload: `{ delta, offset, sampleRate, channels }`. |
  557. | `'change'` | any edit or undo. |
  558. | `'metadata'` | stream header decoded. Payload: `{ sampleRate, channels, estDuration }`: seconds it lasts, from the header where it says (WAV, AIFF, FLAC, an MP3's Xing or VBRI frame, a constant bitrate), else from its size; null when neither is known. |
  559. | `'timeupdate'` | playback position, as heard (~50 times a second). Payload: `currentTime`. |
  560. | `'play'` | playback started or resumed. |
  561. | `'pause'` | playback paused. |
  562. | `'volumechange'` | volume or muted changed. |
  563. | `'ended'` | playback ended: at its end, by `stop()`, or taken over by `play({ from })`; not in a loop. |
  564. | `'progress'` | during save/encode. Payload: `{ offset, total }` in seconds. |
  565. ```js
  566. a.on('data', ({ delta }) => draw(delta)) // decode progress
  567. a.on('timeupdate', t => ui.update(t)) // playback position
  568. a.run(
  569. ['gain', { value: -3, at: 10, duration: 5 }],
  570. ['crop', { at: 1, duration: 2 }],
  571. ['fade', { in: 1, curve: 'exp' }],
  572. ['insert', { source: ref, at: 2 }],
  573. )
  574. a.undo() // undo last edit
  575. b.run(...a.edits) // replay onto another file
  576. JSON.stringify(a); audio(json) // serialize / restore
  577. ```
  578. ### Plugins
  579. | Method                         | Description                                                                                                                         |
  580. |:--|:--|
  581. | `audio.use(...plugins)` | register an [@audio contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) factory, a stat `{ stat, compute }`, a codec `{ codec, test?, decode?, encode? }`, a function receiving `audio`, or a [registry](docs/plugins.md#registry) name. Registry plugins need no `use`: `a.compressor()` and `a.stat('truepeak')` load them on first use. |
  582. | `audio.op(name, descriptor)` | register an op: a `process` function or `{ params, process, plan, resolve }`. |
  583. | `audio.op(name?)` | one descriptor, or all ops. |
  584. | `audio.stat(name, descriptor)` | register a stat: `(chs, ctx) => [...]` or `{ block, reduce, query }`. |
  585. | `.plugin(ref, opts?)` | a native plugin as an edit: VST3, CLAP, Audio Unit or LV2, by file, name or id, through the optional [`@audio/host`](https://github.com/audiojs/host) (Node). Its parameters by key in its own units, functions or `{ t, v }` to automate them; its latency taken off, its tail rendered (`tail` seconds, `false` none); `notes`, `midi`, `bpm`, `timeSignature` for what it hears; `key` a sidechain; `preset`, `state`; `plugin` one of a file holding several; `params` for a parameter named like an option (`mix`, `at`); `isolate` in a process of its own. `audio --plugins` lists them, `audio --plugins NAME` one's parameters. A Web Audio Module (WAM 2.0) by its module's URL or its class, in a page too: hosted by the WAM SDK in an OfflineAudioContext (Node: `web-audio-api`), the input rendered through it before playing, its compensation delay taken off, its decay until silence; parameters by id or label, automation, `notes`, `midi` and `bpm` as WAM events.<br><sub>≡ Pedalboard `load_plugin`, a DAW's insert, a WAM host</sub> |
  586. Plugins also run without the engine: `audio/batch` over a whole signal, `audio/stream` over live chunks. [Plugin tutorial](docs/plugins.md).
  587. ```js
  588. import { compressor } from '@audio/dynamics-compressor/audio'
  589. audio.use(compressor) // bring-your-own factory
  590. a.freeverb({ room: 0.8 }) // registry plugin: loads on first render; tail composes
  591. music.ducker({ key: voice }) // sidechain via the key option
  592. await a.stat('truepeak') // stat plugins land on a.stat()
  593. audio.op('crush', { params: ['bits'], process: (input, output, ctx) => {
  594. let steps = 2 ** (ctx.bits ?? 8)
  595. for (let c = 0; c < input.length; c++)
  596. for (let i = 0; i < input[c].length; i++)
  597. output[c][i] = Math.round(input[c][i] * steps) / steps
  598. }})
  599. a.crush(4) // custom op, chainable like built-ins
  600. a.plugin('RX 12 De-click', { sensitivity: 6 }) // native plugins: npm i @audio/host
  601. a.plugin('AUDelay', { delayTime: 0.25, feedback: t => 20 * t }) // automated, on the timeline
  602. audio(4).plugin('Surge XT', { notes: [{ time: 0, duration: 1, note: 'C4' }], bpm: 96 })
  603. ```
  604. ### Worker
  605. | Call                         | Description                                                                                                                         |
  606. |:--|:--|
  607. | `audioWorker(source, opts?)` | same API, engine in a Worker; the main thread keeps a few-KB facade. |
  608. | `audio(source, { worker: true })` | same, once `audio/worker` is imported. |
  609. | `{ worker: new Worker(url) }` | your own worker entry: codecs, plugins, your own code and messages, then `audio/worker`, which talks on a port of its own. |
  610. | `expose(a)` → id, `audioWorker.adopt(id, { worker })` | hand an instance your worker made to the page as a facade. |
  611. | `audioWorker.context` | the page's AudioContext, the same as `audio.context`. |
  612. Across the boundary `clip()`, `split()`, `clone()` return promises; op errors emit `'error'`; functions don't cross, use `{t, v}` curves. `play()` renders in the worker straight into the page's AudioWorklet: the main thread can stall for seconds without a dropout. [Architecture](docs/architecture.md#worker-engine).
  613. ```js
  614. import audioWorker from 'audio/worker'
  615. let a = audioWorker('track.mp3') // decode/edits/stats/encode in a Worker
  616. a.gain(-3).fade(0.5)
  617. let [mins, maxs] = await a.stat(['min','max'], { bins: 640 }) // transferred, zero-copy
  618. a.play() // rendered in the worker, played by an AudioWorklet (Node: @audio/speaker)
  619. // your own worker: its messages stay its own, and what it makes plays on the page
  620. import audio from 'audio' // worker.js
  621. import { expose } from 'audio/worker'
  622. self.onmessage = ({ data }) => self.postMessage({ out: expose(audio(data.file).gain(-3)) })
  623. let worker = new Worker('./worker.js', { type: 'module' }) // page
  624. worker.onmessage = ({ data }) => audioWorker.adopt(data.out, { worker }).play()
  625. ```
  626. ## CLI
  627. **`npm i -g audio`**, or without installing: `npx audio …`
  628. ```sh
  629. audio [source] [transforms...] [sink] [options]
  630. ```
  631. A pipeline: a **source** produces audio, **transforms** reshape it, a **sink** consumes it. With no sink, a file opened at a terminal opens the player, paused: space plays it. Piped, redirected or `--json`, the default sink is `stat`, printing an overview. `audio record` with no sink saves `recording.wav`.
  632. ```sh
  633. # sources
  634. FILE path, URL, or glob ('*.wav' for batch)
  635. - stdin (or omit when piping)
  636. record capture from microphone
  637. # transforms (chained left-to-right)
  638. gain fade trim normalize crop
  639. clip remove reverse repeat pad
  640. speed stretch pitch insert mix
  641. crossfade remix pan split resample
  642. highpass lowpass eq lowshelf highshelf
  643. notch bandpass allpass vocals dither
  644. crossfeed shrink crossover match spectral
  645. repair copy cut paste plugin
  646. # sinks (terminate the chain — at most one)
  647. stat [NAMES...] print analysis (default)
  648. play [loop] open player UI
  649. save PATH encode and write (or `-` for stdout); `192k` bitrate, `24bit` depth
  650. # options
  651. -f --force overwrite existing output
  652. --format FMT override output format
  653. --macro FILE apply edits from JSON
  654. --cue FILE split at cue-sheet tracks (with split)
  655. --verbose show progress
  656. --help, -h help (or per-op: `audio gain --help`)
  657. --mcp serve the CLI to AI agents as an MCP tool (stdio)
  658. --plugins [NAME] the native plugins @audio/host finds, or one's parameters
  659. # a native plugin (npm i @audio/host): by name, its parameters as name:value
  660. audio voice.wav plugin "RX 12 De-click" sensitivity:6 save clean.wav
  661. # named options, after an op or sink: name:value
  662. normalize -27 lufs ceiling:-2 ducker key:voice.wav save out.m4a codec:alac
  663. # compatibility shortcuts
  664. -p ⇔ play -l ⇔ play loop -o PATH ⇔ save PATH
  665. ```
  666. ### Playback
  667. <img src="docs/player.gif" alt="Audiojs demo" width="624">
  668. <!-- ```sh
  669. audio kirtan.mp3
  670. ▶ 0:06:37 ━━━━━━━━────────────────────────────────────────── -0:36:30 ▁▂▃▄▅__
  671. ▂▅▇▇██▇▆▇▇▇██▆▇▇▇▆▆▅▅▆▅▆▆▅▅▆▅▅▅▃▂▂▂▂▁_____________
  672. 50 500 1k 2k 5k 10k 20k
  673. 48k 2ch 43:07 -0.8dBFS -30.8LUFS
  674. ``` -->
  675. <kbd>␣</kbd> pause · <kbd>←</kbd>/<kbd>→</kbd> seek ±10s · <kbd>⇧←</kbd>/<kbd>⇧→</kbd> seek ±60s · <kbd>↑</kbd>/<kbd>↓</kbd> volume · <kbd>l</kbd> loop · <kbd>s</kbd> save as · <kbd>q</kbd> quit
  676. ```sh
  677. # play full song
  678. audio song.mp3 play
  679. # play fragment
  680. audio song.mp3 10s..15s play
  681. # play and loop a hook
  682. audio song.mp3 30s..45s play loop
  683. # play with effects applied live (streamable ops)
  684. audio song.mp3 normalize broadcast highpass 80hz play
  685. ```
  686. ### Edit
  687. ```sh
  688. # clean up
  689. audio raw-take.wav trim -30db normalize podcast fade 0.3s -0.5s save clean.wav
  690. # scope a range (applies to whole chain)
  691. audio in.wav 1s..10s gain -3db save out.wav
  692. # range on a single op
  693. audio in.wav gain -3db 1s..10s save out.wav
  694. # filter chain
  695. audio in.mp3 highpass 80hz lowshelf 200hz -3db save out.wav
  696. # concat
  697. audio intro.mp3 + content.wav + outro.mp3 trim normalize fade 0.5s -2s save ep.mp3
  698. # crossfade into next
  699. audio track1.mp3 crossfade track2.mp3 2s save mixed.wav
  700. # voiceover
  701. audio bg.mp3 gain -12db mix narration.wav 2s save mixed.wav
  702. # music bed ducked under the voice (sidechain)
  703. audio bed.mp3 ducker key:voice.wav mix voice.wav save episode.wav
  704. # loudness to any target, true peak held; delivery settings
  705. audio book.wav normalize -20 lufs save book.mp3 192k
  706. # fix a video's sound, keep the picture
  707. audio talk.mp4 highpass 80hz 4 normalize podcast save talk.clean.mp4
  708. # master to a reference track (tone, width, loudness)
  709. audio mix.wav master reference.wav save master.wav
  710. # shorten pauses; the same cuts as an EDL for the video editor (.fcpxml, .otio too)
  711. audio talk.mp4 shrink 0.3 save talk.edl
  712. # split
  713. audio audiobook.mp3 split 30m 60m save 'chapter-{i}.mp3'
  714. audio album.wav split --cue album.cue save '{i} - {title}.mp3' # cue-sheet tracks, tagged
  715. # record
  716. audio record 30s save voice.wav
  717. ```
  718. ### Analysis
  719. ```sh
  720. # overview (the default sink when piped or redirected)
  721. audio speech.wav stat
  722. # range overview
  723. audio speech.wav 0..10s stat
  724. # specific stats
  725. audio speech.wav stat loudness rms
  726. # tempo / beat grid / onsets
  727. audio track.mp3 stat bpm
  728. audio track.mp3 stat beats onsets
  729. # loudness to spec: integrated, max momentary / short-term, speech only, true peak
  730. audio mix.wav stat loudness momentary shortterm dialog truepeak
  731. # pass or fail against a delivery spec (exit 1 on a fail); --json for scripts
  732. audio episode.wav normalize podcast check podcast
  733. audio chapter.wav check acx --json
  734. # pitch / chords / key
  735. audio song.mp3 stat notes
  736. audio song.mp3 stat chords
  737. audio song.mp3 stat key
  738. # spectrum / cepstrum with bin count
  739. audio speech.wav stat spectrum 128
  740. audio speech.wav stat cepstrum 13
  741. # stat after transforms (transforms apply, then stat)
  742. audio speech.wav gain -3db stat db
  743. ```
  744. ### Batch
  745. ```sh
  746. audio '*.wav' trim normalize podcast save '{name}.clean.{ext}'
  747. audio '*.wav' gain -3db save '{name}.out.{ext}'
  748. audio 'chapters/*.mp3' check acx # a whole audiobook: one line per chapter
  749. ```
  750. ### Check in CI
  751. A file that misses its spec fails the build: one line per file, ✓ or the rule it broke, exit 1 on any fail.
  752. ```yaml
  753. # .github/workflows/audio.yml
  754. on: [push, pull_request]
  755. jobs:
  756. audio:
  757. runs-on: ubuntu-latest
  758. steps:
  759. - uses: actions/checkout@v5
  760. - uses: audiojs/audio@master # or any CI: npx -y audio 'episodes/*.mp3' check podcast
  761. with:
  762. files: episodes/*.mp3
  763. spec: podcast # spotify, broadcast, netflix, acx
  764. ```
  765. ### Stdin/stdout
  766. ```sh
  767. cat in.wav | audio gain -3db save - > out.wav
  768. curl -s https://ex.com/speech.mp3 | audio normalize save clean.wav
  769. ```
  770. ### Tab completion
  771. ```sh
  772. eval "$(audio --completions zsh)" # add to ~/.zshrc
  773. eval "$(audio --completions bash)" # add to ~/.bashrc
  774. audio --completions fish | source # fish
  775. ```
  776. ## FAQ
  777. <dl>
  778. <dt>What formats are supported?</dt>
  779. <dd>Decode: WAV, MP3, FLAC, OGG Vorbis, Opus, AAC, AIFF, CAF, WebM, AMR, WMA, QOA via <a href="https://github.com/audiojs/decode">decode</a>. Encode: WAV, MP3, FLAC, Opus, OGG, AIFF via <a href="https://github.com/audiojs/encode">encode</a>. Codecs are WASM-based, lazy-loaded on first use.</dd>
  780. <dt>Does it need ffmpeg or native addons?</dt>
  781. <dd>No ffmpeg: codecs and processing are JS and WASM. Playing and recording in Node go through a small native addon, prebuilt for macOS, Linux (x64, arm64) and Windows (x64), so nothing compiles at install; elsewhere playback falls back to <code>ffplay</code>, SoX or <code>aplay</code>. For the CLI, install globally: <code>npm i -g audio</code>.</dd>
  782. <dt>How big is it?</dt>
  783. <dd>In a page: 128 KB gzipped, the whole library minified (<code>dist/audio.min.js</code>); codecs and plugins load on first use via <code>import()</code>, so unused formats aren't fetched. In Node: <code>npm i audio</code> installs 16 MB in 327 packages (every codec and plugin), nothing compiled; <code>npm i audio --omit=optional</code> 11.5 MB in 115 (the codecs and the core: filters, EQ, loudness, pitch, stretch), each plugin family then installed by name when used (<code>npm i @audio/dynamics</code>).</dd>
  784. <dt>How does it handle large files?</dt>
  785. <dd>Audio is stored in fixed-size pages. In the browser, with <code>{ storage: 'persistent' }</code> (or <code>'auto'</code>, where OPFS exists), cold pages evict to OPFS when memory exceeds budget — auto-sized from <code>navigator.storage.estimate()</code> (quota/4, 64MB..512MB), overridable via <code>{budget}</code>; each instance keeps its own store, its copies share it. Works for decoded files, <code>audio.from(pcm)</code> and pushed streams. Stats stay resident (~7 MB for 2h stereo).</dd>
  786. <dt>Are edits destructive?</dt>
  787. <dd>No. <code>a.gain(-3).trim()</code> pushes entries to an edit list — source pages aren't touched. Edits replay on <code>read()</code> / <code>save()</code> / <code>for await</code>.</dd>
  788. <dt>Can I use it in the browser?</dt>
  789. <dd>Yes, same API. See <a href="#browser">Browser</a> for bundle options and import maps.</dd>
  790. <dt>Does it need the full file before I can work with it?</dt>
  791. <dd>No. Playback, edits, and structural ops (crop, repeat, pad, insert, etc.) all stream incrementally during decode — output begins before the file finishes loading. The edit plan recompiles as data arrives, tracking a safe output boundary per op. Only ops that depend on total length (open-end reverse, negative <code>at</code>) wait for full decode.</dd>
  792. <dt>TypeScript?</dt>
  793. <dd>Yes, ships with <code>audio.d.ts</code>.</dd>
  794. <dt>Does it have feature parity with FFmpeg / SoX / librosa?</dt>
  795. <dd>Yes — the <a href="https://github.com/audiojs">audiojs</a> ecosystem covers the practical baseline of FFmpeg filters, SoX effects, librosa analysis, Pedalboard and MIREX, all as <code>@audio/*</code> plugins <code>audio</code> wires through one API (the few uncovered items are esoteric or deliberately skipped). Every effect, filter, generator and analyzer lives in the <a href="docs/plugins.md#registry">registry</a> — call one by name and it loads on first use. Coverage matrix: <a href="docs/comparison.md">docs/comparison.md</a>.</dd>
  796. <dt>How is this different from SoX / FFmpeg / Audacity / librosa / Web Audio / Tone.js?</dt>
  797. <dd>In one line: <code>audio</code> is the only one that runs the same API in Node and the browser, with non-destructive lazy edits that stream during decode. The native tools (SoX, FFmpeg) have no JS API, browser, or undo, and only FFmpeg is still faster on most raw throughput; the browser libs (Web Audio, Tone.js, Howler) are real-time graphs, not file editors. Full feature and <a href="docs/comparison.md#performance">performance</a> matrices vs pydub, librosa, aubio, essentia, Pedalboard, SoX, FFmpeg, Audacity and MATLAB are in <a href="docs/comparison.md">docs/comparison.md</a>.</dd>
  798. </dl>
  799. ## Built with
  800. * [decode](https://github.com/audiojs/decode) – codec decoding (13+ formats)
  801. * [encode](https://github.com/audiojs/encode) – codec encoding
  802. * [filter](https://github.com/audiojs/filter) – filters (weighting, EQ, auditory)
  803. * [speaker](https://github.com/audiojs/speaker) – audio output
  804. * [mic](https://github.com/audiojs/mic) – audio input
  805. * [pitch](https://github.com/audiojs/pitch) – pitch, chord, key analysis
  806. * [audio-type](https://github.com/audiojs/audio-type) – format detection
  807. * [pcm-convert](https://github.com/audiojs/pcm-convert) – PCM format conversion
  808. <p align="center"><a href="./license.md">MIT</a> · <a href="https://github.com/krishnized/license">ॐ</a></p>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.