Decoding Parameters as Methodological Choices in ASR-Derived Corpus Construction

<span style="font-family:Rubik;font-size:23pt;font-weight: 700;font-style: normal;float:left;text-align: left;color:white;-webkit-text-fill-color: #bd962d;-webkit-text-stroke: 0.8px white;line-height:1.2;">Decoding Parameters as Methodological Choices in ASR-Derived Corpus Construction</span>

<p style="float:left;text-align: left;color:white;font-weight: 700;font-style: normal;-webkit-text-fill-color: #F0EADA;-webkit-text-stroke: 0px white;font-size:1.15em;line-height:1.45;">
Steven Coats<br>
University of Oulu<br>
<a href="mailto:steven.coats@oulu.fi" style="color:#F0EADA;text-decoration:none;">steven.coats@oulu.fi</a><br>
CMC2026, Oulu<br>
</p>

---

<div class="my-footer"><span>Steven Coats&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;Decoding Parameters in ASR | CMC2026</span></div>

---

## Talk Organization

1. **Context:** ASR as infrastructure for spoken-corpus construction

2. **Methodological focus:** Inference configuration as a component of corpus design

3. **Paired corpus experiment:** Beam search and sampling across 774 hours of podcast audio

4. **Results:** Corpus-scale divergence and audio-linked examples

5. **Implications:** Validation, reporting, and reproducible corpus construction

---

### ASR as Corpus Infrastructure

<div style="text-align:center; font-size:1.08em; margin:10px 0 18px 0;">
  <strong>Large audio collections</strong>&ensp;&#8594;&ensp;
  <span style="color:#901a1e; font-weight:700;">ASR and alignment</span>&ensp;&#8594;&ensp;
  <strong>Searchable spoken corpora</strong>
</div>

.pull-left[
#### What has changed
- Neural ASR systems such as wav2vec 2.0, Whisper, and Qwen now provide strong transcription performance across many settings <span class="small">Baevski et al. (2020); Radford et al. (2022); Shi et al. (2026)</span>
- Hundreds of hours of speech can be processed without complete manual transcription
- Word timing and diarization can connect searchable text back to the recording
]

.pull-right[
#### Corpus applications
- Automated sociophonetic processing <span class="small">(Coto-Solano et al., 2021)</span>
- Large web-audio corpus pipelines <span class="small">(Coats, 2025)</span>
- Variety-specific adaptation and corpus enrichment <span class="small">(Coats et al., forthcoming)</span>
]

---

### Inference Configuration as Corpus Methodology

<div style="text-align:center; font-size:1.18em; line-height:1.7; margin-top:16px;">
  <strong>Same audio</strong><br>
  &#8595;<br>
  <strong>Same ASR model</strong><br>
  &#8595;<br>
  <span style="color:#901a1e; font-weight:700;">Different decoding parameters</span><br>
  &#8595;<br>
  <strong>Different corpus transcripts</strong>
</div>

<div style="background-color:#f8f9fa; border-left:5px solid #901a1e; padding:10px 16px; margin-top:14px;">
Corpus builders routinely report the ASR model. The inference configuration that converts model scores into a transcript is often documented less fully.
</div>

---

### Beam Search, Sampling, and Temperature

.pull-left[
#### Beam search
- Maintains the five highest-scoring partial hypotheses.
- Expands and prunes them as decoding proceeds.
- Returns the highest-scoring completed sequence found by the search.
- Deterministic for a fixed model and input.
]

.pull-right[
#### Sampling
- Draws candidate continuations from the model's token-probability distribution.
- A best-of-five procedure retains the highest-scoring sampled sequence.
- **Temperature** controls the concentration of the sampling distribution.
- Lower temperature places more probability on the highest-scoring alternatives.
]

<div style="background-color:#f8f9fa; border-left:5px solid #2b6cb0; padding:10px 15px; margin-top:13px;">
  <strong>Our comparison:</strong> five-beam decoding versus low-temperature sampling at <em>T=.1</em>, with five sampled candidates.
</div>

<div style="font-size:0.62em; text-align:right; margin-top:5px;">For temperature scaling, see Hinton et al. (2015).</div>

---

### Paired Corpus Experiment

.pull-left[
#### Material
- **1,322 recordings**
- **774.02 hours**
- Six Singapore English podcast series
- Natural conversation, overlap, music, sound effects, and variable recording quality
- [YCSEP](https://ycsep.corpora.li) <span class="small">(Coats et al., 2025)</span>
]

.pull-right[
#### Held constant
- The same audio
- The same Whisper model, fine-tuned on Singapore English conversation <span class="small">(Coats et al., forthcoming)</span>
- The same thresholds and anti-loop settings
- The same alignment and diarization pipeline
]

<div style="background-color:#f8f9fa; border-left:5px solid #901a1e; padding:10px 15px; margin-top:12px;">
  <strong>Experimental contrast:</strong> five-beam decoding versus best-of-five sampling at <em>T=.1</em>.
</div>

---

### Corpus-Level Difference

<br>

| Metric | Beam search | Sampling `$(T=0.1)$` | Absolute Difference | Relative Change |
|:---|:---:|:---:|:---:|:---:|
| **Raw ASR words** | 9,719,188 | 9,817,584 | +98,396 | **+1.01%** |
| **Aligned words** (WhisperX) | 9,666,394 | 9,765,405 | +99,011 | **+1.02%** |

<br>
<div style="background-color: #f8f9fa; border-left: 5px solid #901a1e; padding: 10px 15px;">
  <strong>Paired result:</strong> Sampling at <em>T=0.1</em> produced 1.02% more aligned words than five-beam decoding (95% paired-bootstrap interval: +0.89% to +1.16%). 
</div>

---

### Example 1: Omission under Beam Search

**Context:** A presenter closes a video while music plays in the background.

.pull-left[
#### Five-Beam Output `$(T=0)$`
<div style="background-color: #ffffff; border: 1px solid #ccc; padding: 12px; border-radius: 6px; font-size: 0.82em; line-height: 1.45; font-family: 'Source Code Pro', monospace;">
  … I think that at least for now I'm leaning towards staying in Singapore …
  <span style="color:#777; font-weight:700;">[audible sentence omitted]</span> …
  hit that subscribe button … put the bell on for those notifications …
</div>
]

.pull-right[
#### Sampling Output `$(T=0.1)$`
<div style="background-color: #ffffff; border: 1px solid #ccc; padding: 12px; border-radius: 6px; font-size: 0.78em; line-height: 1.4; font-family: 'Source Code Pro', monospace;">
  … I think that at least for now I'm leaning towards staying in Singapore.
  <span style="color: #1a5fb4; font-weight: 700;">Thank you so much for watching this episode … make sure you hit that thumbs up … drop a comment …</span>
  hit that subscribe button … put the bell on for those notifications …
</div>
]

<div style="background-color: #f8f9fa; border-left: 5px solid #1a5fb4; padding: 8px 12px; margin-top: 6px; font-size: 0.82em; line-height: 1.4;">
  <strong>What the example shows:</strong> both outputs are accurate before and after the gap, but beam search omits the intervening audible sentence.<br>
  <span style="color:#555;"><strong>Plausible contributor:</strong> the background music was absent from the model's conversational fine-tuning data.</span>
</div>

---

### Example 2: Hallucination under Sampling

**Context:** Two hosts discuss the films of writer and director Alex Garland (Paper Table 3).

.pull-left[
#### Five-Beam Output `$(T=0)$`
<div style="background-color: #ffffff; border: 1px solid #ccc; padding: 14px; border-radius: 6px; font-size: 0.85em; line-height: 1.48; font-family: 'Source Code Pro', monospace;">
  … his most famous film is <em>Ex-Machina</em> lah … he wrote <em>The Beach</em> …
</div>
<p style="font-size: 0.85em; color: #2b6cb0; margin-top: 8px;"><em>(No corresponding insertion in this output)</em></p>
]

.pull-right[
#### Sampling Output `$(T=0.1)$`; Hallucinated Loop
<div style="background-color: #ffffff; border: 1px solid #ccc; padding: 14px; border-radius: 6px; font-size: 0.85em; line-height: 1.48; font-family: 'Source Code Pro', monospace;">
… his most famous film is <em>Ex-Machina</em> lah <span style="color: #c53030; font-weight: 700;">which is a movie that is set in the Middle East and it's about a guy named Alex-Garland who is a terrorist who is trying to escape from the u u u u u u u u u u u u u u</span> … he wrote <em>The Beach</em> …
</div>
<p style="font-size: 0.85em; color: #c53030; margin-top: 8px;"><em>(<span style="color: #c53030; font-weight: 700;">Red text:</span> Hallucinated insertion not supported by audio)</em></p>
]

<div style="background-color: #f8f9fa; border-left: 5px solid #c53030; padding: 10px 15px; margin-top: 10px;">
  <strong>What the example shows:</strong> <em>T=.1</em> sampling inserts an unsupported plot description and then enters a repetition loop. Neither is grounded in the audio.
</div>

---

### Interpretation and Scope of Inference

.pull-left[
#### Supported by the evidence
- The decoding configurations produced materially different corpora
- Sampling produced about **99,000 additional aligned words**
- Sampling can help genuine recovery, but also result in unsupported insertions
]

.pull-right[
#### Not established by this experiment
- More words automatically means greater accuracy
- Beam search is always better—or sampling is always better
]

<div style="background-color:#f8f9fa; border-left:5px solid #901a1e; padding:10px 15px; margin-top:12px;">
  A later sampling run at <em>T=0.01</em> produced still more words than <em>T=.1</em>. There is no simple rule that higher temperature means more output.
</div>

---

### Methodological Recommendations

1. **Target-domain validation**
   - Compare configurations on representative audio, including music, overlap, noise, and speaker changes

2. **Gold-standard error analysis**
   - Use a manageable human-transcribed sample to separate omissions, insertions, and substitutions (word error rate, WER)

3. **Complete inference reporting**
   - Model and software version, beam or sampling settings, temperature, thresholds, segmentation, voice-activity detection (VAD), and and whether previous transcript text or other textual context was supplied to the model

4. **Auditability and versioning**
   - Preserve audio links, timestamps, and versioned transcripts

<div style="background-color:#f8f9fa; border-left:5px solid #2b6cb0; padding:9px 14px; margin-top:10px;">
Inference choices can affect lexical frequencies, concordance results, retrieved examples, and ultimately linguistic interpretation.
</div>

---

### Backup: Sampling-Only Temperature Check

<div style="display:flex; gap:14px; margin:18px 0; text-align:center;">
  <div style="flex:1; background:#eef1f4; padding:14px; border-radius:7px;"><strong>Beam search</strong><br><em>T=0</em><br><span style="font-size:1.3em;">9.666m words</span></div>
  <div style="flex:1; background:#eaf4ee; padding:14px; border-radius:7px;"><strong>Sampling</strong><br><em>T=.01</em><br><span style="font-size:1.3em;">10.052m words</span></div>
  <div style="flex:1; background:#f8eeee; padding:14px; border-radius:7px;"><strong>Sampling</strong><br><em>T=.1</em><br><span style="font-size:1.3em;">9.765m words</span></div>
</div>

- A later sampling-only control used the same best-of-five procedure at `$.01$` and `$.1$`.
- Output at `$.1$` was **2.85% lower** than at `$.01$` (95% interval: 2.68--3.01% lower).
- Both sampling settings recovered the first audible exchange; only `$.1$` produced the second example's unsupported continuation.

<div style="background-color:#f8f9fa; border-left:5px solid #901a1e; padding:10px 15px; margin-top:12px;">
  <strong>Broader lesson:</strong> output cannot be predicted from the temperature number alone. The complete decoding configuration must be tested empirically.
</div>

---

### Backup: Limitations and Future Directions

.pull-left[
#### Current Study Limitations
- Primary focus on beam search at `$T=0.0$` versus sampling at `$T=0.1$`, with one sampling-only control at `$T=0.01$`.
- Corpus-level macro evaluation and illustrative segment analysis rather than exhaustive manual gold-standard annotation across all 774 hours.
- The corpus-level comparison measures output divergence, not WER, deletion rate, or insertion rate.
- In faster-whisper, `$T=0.0$` and `$T>0$` select different decoding algorithms (beam search versus sampling).
- A single seeded `$T=0.1$` run does not measure sampling variability across repeated runs.
- Evaluated on conversational English podcast speech.
]

.pull-right[
#### Future Research Roadmap
- **Gold-standard sample:** Estimate insertion, deletion, and substitution rates under both settings.
- **Repeated sampling:** Measure run-to-run variance at positive temperatures.
- **Broader configurations:** Compare model, VAD, thresholds, prompts, and segmentation as well as temperature.
- **Cross-Linguistic Validation:** Evaluating multilingual Whisper models on low-resource and dialectal varieties.
]

---

### Conclusions

- The same audio and model can produce **different corpus transcripts** under different decoding configurations

- These differences include both **omissions** and **unsupported insertions**

- No setting is automatically best: the choice must be **tested on the target material**

- ASR parameters should be **reported, justified, and treated as part of corpus design**

</div>

<div style="background-color:#f8f9fa; border-left:6px solid #901a1e; padding:12px 17px; margin-top:18px; font-size:1.05em;">
  <strong>Central claim:</strong> Inference parameters have observable consequences for ASR-derived corpora and therefore belong in corpus methodology.
</div>

---

.pull-right[
<span style="font-family:Rubik;font-size:28pt;font-weight: 700;font-style: normal;float:right;text-align: right;color:white;-webkit-text-fill-color: black;-webkit-text-stroke: 0.8px;line-height:1.2;">Thank You!</span>
]
 
<br><br><br><br><br>

<p style="float:right;text-align: right;color:white;font-weight: 700;font-style: normal;-webkit-text-fill-color: #bd962d;-webkit-text-stroke: 0.5px;font-size:1.15em;line-height:1.45;">
Steven Coats<br>
Faculty of Humanities, University of Oulu, Finland<br>
<a href="mailto:steven.coats@oulu.fi" style="color:#901a1e;text-decoration:none;">steven.coats@oulu.fi</a><br>
Questions & Discussion<br>
</p>

---

### References

Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020). [wav2vec 2.0: A framework for self-supervised learning of speech representations](https://arxiv.org/abs/2006.11477). *arXiv preprint arXiv:2006.11477*.

Coats, S. (2025). [An automatic pipeline for processing streamed content: New horizons for corpus linguistics and phonetics](https://doi.org/10.1515/9783111434018-011). In L. Cotgrove, L. Herzberg, & H. Lüngen (Eds.), *Exploring digitally-mediated communication with corpora: Methods, analyses, and corpus construction* (pp. 257–274). De Gruyter.

Coats, S., Basile, C. A., Morin, C., & Fuchs, R. (2025). [The YouTube corpus of Singapore English podcasts](https://doi.org/10.1075/eww.25018.coa). *English World-Wide, 46*(3), 274–298.

Coats, S., Basile, C. A., Morin, C., & Fuchs, R. (forthcoming). Fine-Tuning ASR for Corpus Linguistics: Singapore English. *Language Resources and Evaluation*.

Coto-Solano, R., Stanford, J. N., & Reddy, S. K. (2021). [Advances in completely automated vowel analysis for sociophonetics: Using end-to-end speech recognition systems with DARLA](https://doi.org/10.3389/frai.2021.662097). *Frontiers in Artificial Intelligence, 4*, 681466.

Hinton, G., Vinyals, O., & Dean, J. (2015). [Distilling the knowledge in a neural network](https://arxiv.org/abs/1503.02531). *arXiv preprint arXiv:1503.02531*.

Lea, C., Huang, Z., Narain, J., Tooley, L., Yee, D., Tran, D. T., Georgiou, P., Bigham, J. P., & Findlater, L. (2023). [From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognition](https://doi.org/10.1145/3544548.3581224). In *Proceedings
of the 2023 CHI Conference on Human Factors in Computing Systems*, pages 1–16.

Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022). [Robust speech recognition via large-scale weak supervision](https://arxiv.org/abs/2212.04356).

Shi, X., Wang, X., Guo, Z., Wang, Y., Zhang, P., Zhang, X., Guo, Z., Hao, H., Xi, Y., Yang, B., Xu, J., Zhou, J., & Lin, J. (2026). [Qwen3-ASR technical report](https://arxiv.org/abs/2601.21337). *arXiv preprint arXiv:2601.21337*.

]
]

---

### Backup Slide 1: Mathematical Properties of Softmax

.pull-left[
#### Invariance Under Uniform Shift
For any scalar constant `$c$`:
`$$\frac{\exp((z_i + c)/T)}{\sum_j \exp((z_j + c)/T)} = \frac{\exp(z_i/T)}{\sum_j \exp(z_j/T)}$$`
The probability distribution depends strictly on the **relative distances** between logits, not their absolute magnitudes.
]

.pull-right[
#### Limiting Behavior of Entropy
The Shannon entropy of the token distribution:
`$$H(p) = -\sum_{i} p_i(T) \log p_i(T)$$`

- As `$T \to 0$`: `$H(p) \to 0$` (Zero uncertainty, deterministic greedy selection).
- As `$T \to \infty$`: `$H(p) \to \log(|V|)$` (Maximum entropy, uniform random choice).
]

---

### Backup Slide 2: Candidate Environments for Further Validation

.pull-left[
#### Acoustically difficult environments:
1. **Overlapping Speech & Cross-Talk:** Multi-party dialogue where acoustic energy is shared across speakers.
2. **Background Acoustic Noise:** Music beds, street noise, or mic bumps.
3. **Reduced Articulation:** Fast casual speech, vowel reductions, dropped consonants.
4. **Vocal Fry & Low Intensity:** Breathy or low-volume utterance endings.
]

.pull-right[
#### Research question:
- These conditions may be unevenly distributed across:
  - speakers, genres, and recording settings;
  - formal and informal interaction;
  - conversational roles and overlap patterns.
- A stratified gold-standard sample is needed to test whether configuration-specific errors create demographic or register bias.
]