MATHEMATICAL ANALYSIS OF SPEECH FEATURE EXTRACTION FROM RAW WAVEFORMS: FROM SPECTRAL REPRESENTATIONS TO WAV2VEC 2.0
Article Sidebar
Main Article Content
Abstract
This analytical study formalizes the transformation of a sampled speech waveform into machine-usable acoustic features. The classical path is expressed through short-time framing, Hamming windowing, Fourier analysis, Mel filter banks and MFCCs, while the learned path is examined using wav2vec 2.0. A formula-based derivation shows that the seven-layer wav2vec 2.0 convolutional encoder has an overall stride of 320 samples and a receptive field of 400 samples; at 16 kHz these correspond to 20 ms feature spacing and 25 ms acoustic context. Thus, classical and learned front ends operate on comparable local time scales but differ in whether the transformation is fixed by signal-processing assumptions or learned from unlabeled speech.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
S. B. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,” IEEE Trans. Acoust., Speech, Signal Process., vol. 28, no. 4, pp. 357–366, 1980, doi: 10.1109/TASSP.1980.1163420.
Z. Kh. Abdul and A. K. Al-Talabani, “Mel Frequency Cepstral Coefficient and its applications: A review,” IEEE Access, vol. 10, pp. 122136–122158, 2022, doi: 10.1109/ACCESS.2022.3223444.
A. Baevski, H. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Adv. Neural Inf. Process. Syst., vol. 33, pp. 12449–12460, 2020.
A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 30, pp. 5998–6008, 2017.
K. Choi and E. J. Yeo, “Opening the black box of wav2vec feature encoder,” arXiv:2210.15386, 2022.