summaryrefslogtreecommitdiff
path: root/tools/whistle-eval/samples/README.md
blob: 28f0d122f33516ab4709c390be8df95d97c5e347 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# Test clips

The audio files are not committed. Recreate them with `edge-tts` (neural TTS)
and `ffmpeg`, or drop your own 16 kHz mono 16-bit WAV files here. When a
`<name>.txt` sidecar exists, `evaluate.py` reports the WER against it.

```sh
# English
edge-tts --voice en-US-AriaNeural \
  --text "The quick brown fox jumps over the lazy dog near the riverbank, while the scientists carefully measured the temperature of the experimental apparatus." \
  --write-media en1.mp3
# Italian
edge-tts --voice it-IT-ElsaNeural \
  --text "Il rapido volpe marrone salta sopra il cane pigro vicino alla riva del fiume, mentre gli scienziati misurano con attenzione la temperatura dell'apparecchiatura sperimentale." \
  --write-media it1.mp3
# ...then:
for f in en1 it1; do
  ffmpeg -y -i "$f.mp3" -ar 16000 -ac 1 -c:a pcm_s16le "$f.wav"
done
```

The JFK clip is the public sample from
[whisper.cpp](https://github.com/ggerganov/whisper.cpp/blob/master/samples/jfk.wav).

## References used by the committed test set

| file | reference |
|---|---|
| `jfk.wav` | And so my fellow Americans ask not what your country can do for you ask what you can do for your country |
| `en1.wav` | The quick brown fox jumps over the lazy dog near the riverbank, while the scientists carefully measured the temperature of the experimental apparatus. |
| `en2.wav` | Please call me back at four fifteen in the afternoon. The meeting is with Doctor Smith and Professor Johnson from the university. |
| `it1.wav` | Il rapido volpe marrone salta sopra il cane pigro vicino alla riva del fiume, mentre gli scienziati misurano con attenzione la temperatura dell'apparecchiatura sperimentale. |
| `it2.wav` | Per favore, richiamami alle quattro e un quarto del pomeriggio. L'incontro e con il dottor Rossi e la professoressa Bianchi dell'universita. |

`it1_noisy.wav` is `it1.wav` mixed with low-level white noise:

```sh
ffmpeg -y -i it1.wav -f lavfi -i "anoisesrc=color=white:amplitude=0.03:duration=999" \
  -filter_complex "[0:a][1:a]amix=inputs=2:duration=first:weights=1 0.35" \
  -ar 16000 -ac 1 -c:a pcm_s16le it1_noisy.wav
```