Skip to content
Back to search

mfa-refine

SkillWorks with Claude CodeGitHub

Fix Montreal Forced Aligner TextGrids: re-transcribe, re-align in short chunks, cross-check, and report what to check by hand. CPU-friendly; includes a Claude Code skill.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/sfoley3--mfa-refine/badge.svg)](/listings/sfoley3--mfa-refine)

HTML

<a href="/listings/sfoley3--mfa-refine"><img src="/api/listings/sfoley3--mfa-refine/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

git clone https://github.com/sfoley3/mfa-refine.git
cd mfa-refine
conda env create -f environment.yml
conda activate mfa-refine
mfa model download acoustic english_us_arpa
mfa model download dictionary english_us_arpa
mfa model download g2p english_us_arpa

PyTorch must come from conda-forge, as `environment.yml` does. pip torch alongside conda MFA crashes on macOS with `OMP: Error #15`.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 128 lines
  1. # mfa-refine
  2. Fix Montreal Forced Aligner (MFA) TextGrids where words land in silence, drift off the speech, or
  3. were never aligned. Runs on CPU; Whisper uses an NVIDIA GPU when one is available.
  4. MFA aligns a whole recording against a transcript in one go. On long, pause-heavy or disordered
  5. speech (word lists, reading passages with restarts, clinical tasks) one error can pull many words
  6. off the speech. `mfa-refine` re-does the alignment in short, silence-bounded pieces and tells you
  7. where it is still unsure:
  8. 1. **transcribe** – split each file at pauses (energy VAD) and run Whisper on each chunk, so it
  9. knows which words were actually said in which chunk.
  10. 2. **prepare** – drop Whisper hallucinations, then match what Whisper heard against your
  11. transcript (the word tier of the existing TextGrid). Your transcript supplies the labels and
  12. Whisper decides where they go, so a small Whisper model is enough.
  13. 3. **align** – run `mfa align` on the short chunks (words + phones), with speaker adaptation and
  14. G2P for out-of-vocabulary words.
  15. 4. **ctc** – align the same chunks with an independent CTC aligner (torchaudio MMS_FA) as a
  16. cross-check.
  17. 5. **stitch** – rebuild full-length TextGrids, trim silent word edges (never across stop
  18. closures), and write `refine_report.csv`.
  19. Every step caches its output in `OUT/_work`, so re-running only redoes what changed.
  20. ## Install
  21. ```bash
  22. git clone https://github.com/sfoley3/mfa-refine.git
  23. cd mfa-refine
  24. conda env create -f environment.yml
  25. conda activate mfa-refine
  26. mfa model download acoustic english_us_arpa
  27. mfa model download dictionary english_us_arpa
  28. mfa model download g2p english_us_arpa
  29. ```
  30. PyTorch must come from conda-forge, as `environment.yml` does. pip torch alongside conda MFA
  31. crashes on macOS with `OMP: Error #15`.
  32. ## Run
  33. ```bash
  34. mfa-refine run \
  35. --audio data/spk1/wav --textgrids data/spk1/textgrids \
  36. --audio data/spk2/wav --textgrids data/spk2/textgrids \
  37. --out refined/ \
  38. --speaker-pattern '^(spk\d+)_'
  39. ```
  40. - Audio files and TextGrids are matched by file stem. A file without a TextGrid is labelled with
  41. Whisper's words.
  42. - `--speaker-pattern` is a regex on the file stem whose group 1 is the speaker. MFA adapts to each
  43. speaker.
  44. - `--include` / `--exclude` take globs on the file stem (repeatable), e.g. `--exclude '*_bvt*'`.
  45. - `--steps align,ctc,stitch` re-runs part of the pipeline.
  46. - `--whisper-model` defaults to `small.en`. Your transcript supplies the labels, so Whisper only
  47. has to place them, and on read word lists `small.en` matched `large-v3`. On disordered or highly
  48. repetitive speech `medium.en` recovered more of the transcript, at about 1.7× the CPU time and
  49. ~4.5 GB RAM (vs ~3 GB). Try it when the report shows many `unspoken` words.
  50. - `--device auto` (default) runs Whisper on an NVIDIA GPU (CUDA) if there is one, else on CPU, and
  51. falls back to CPU if the GPU fails to load. `--device mps` tries the Apple Silicon GPU
  52. (ctranslate2's MPS backend is experimental; check the output).
  53. MFA itself always runs on CPU.
  54. CUDA needs the NVIDIA cuBLAS/cuDNN libraries that faster-whisper requires (see its README).
  55. - For other languages, set `--language`, `--dictionary`, `--acoustic-model` and `--g2p-model` to
  56. MFA models for that language.
  57. Run `mfa-refine run -h` to see all options.
  58. ## Read the report
  59. `OUT/refine_report.csv` has one row per file:
  60. | column | meaning |
  61. |---|---|
  62. | `before_*` / `after_*` | `words_on_silence`, `words_over_1s` and `uncovered_speech_s` for the input and output word tiers |
  63. | `check_by_hand` | chunks (start-end s) where MFA and the CTC aligner disagree by more than `--check-threshold` (median word onset) |
  64. | `flagged_words` | output words that still sit mostly on silence or last over 1 s |
  65. | `ctc_unchecked_chunks` | chunks the CTC cross-check couldn't cover, e.g. words with digits or symbols. These aren't in `check_by_hand`, so look at them yourself if it matters |
  66. | `mfa_failed_chunks` | chunks MFA could not align. Words there are spread evenly and have no phones |
  67. | `untranscribed_speech` | speech Whisper produced no usable words for, e.g. coughs, laughs, or words missing from your transcript |
  68. | `ctc_disagree` | individual words whose onsets differ by more than 100 ms between the two aligners |
  69. | `transcript_changes` | every decision made about the labels, as tagged notes (below) |
  70. `transcript_changes` tags:
  71. - `rejected_chunk`, `drop_low_conf`, `strip_phrase`: Whisper output thrown away as hallucination
  72. - `heard:'x'->'y'`: Whisper heard `x`, labelled with the transcript's `y`
  73. - `repeat`: a repeated stretch, labelled with the transcript words it repeats
  74. - `extra_speech`, `added`: words Whisper heard that aren't in the transcript, kept as heard
  75. - `unlabelled_sparse`: a chunk too sparse to match, left unlabelled
  76. - `unspoken`: a transcript word not found in the audio, left out
  77. ## Fix what's flagged
  78. ```bash
  79. mfa-refine inspect --audio data/spk1/wav --textgrids data/spk1/textgrids --out refined/ spk1_task1 42.3
  80. ```
  81. This prints the chunk around 42.3 s: what Whisper heard, the labels used, the MFA and CTC word
  82. times, the VAD speech regions and a 50 ms energy profile. Then open the refined TextGrid with the
  83. audio in Praat and fix the boundaries by hand.
  84. ## Tuning
  85. | option | default | change when |
  86. |---|---|---|
  87. | `--vad-threshold` | 0.35 | quiet speech counts as silence (lower it) or noise counts as speech (raise it) |
  88. | `--min-silence` | 0.30 | chunks are too long (lower it) or words are split mid-utterance (raise it) |
  89. | `--snap-max` | 0.05 | maximum silence (s) trimmed from each word edge |
  90. | `--check-threshold` | 0.15 | too many or too few chunks are listed in `check_by_hand` |
  91. ## Using it with Claude Code
  92. `.claude/skills/mfa-refine/SKILL.md` is a Claude Code skill. Open Claude Code in this repo, or copy
  93. the skill folder to `~/.claude/skills/`, and ask it to "refine the MFA alignments in …". Claude
  94. will run the pipeline, read the report and go through the flagged chunks with you.
  95. ## Tests
  96. ```bash
  97. python -m pytest
  98. ```
  99. ## Data
  100. `.gitignore` excludes `*.wav`, `*.flac` and `*.TextGrid`. Don't commit participant recordings or
  101. annotations.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.