This paper presents a speaker-adaptive approach to automatic pronunciation assessment for English language learning, with a focus on Uzbek learners. The proposed methodology integrates acoustic signal processing, feature extraction, and deep learning-based modeling within a unified framework. A key contribution of the study is the introduction of dynamic speaker profiling based on fundamental frequency and energy, enabling adaptive dataset selection according to speaker characteristics such as age and gender. Mel-Frequency Cepstral Coefficients are employed for acoustic feature representation, while Wav2Vec 2.0 is utilized for deep contextual embedding and pronunciation evaluation. Experimental results demonstrate improved accuracy and efficiency compared to conventional approaches.
Adaptive Pronunciation Assessment Based on Acoustic Feature Profiling and Wav2vec 2.0 For English Language Learning
DOI:
Keywords:
Abstract
References
Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems (NeurIPS), 33, 12449–12460.
Hsu, W. N., Bolte, B., Tsai, Y. H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021). HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 3451–3460.
Pratap, V., Xu, Q., Sriram, A., Tomasello, P., & Sainath, T. N. (2020). Scaling up online speech recognition using convnets and transformers. Interspeech, 2020, 3072–3076.
Qin, J., Hu, Y., & Wan, V. (2021). End-to-end mispronunciation detection and diagnosis using deep neural networks. Speech Communication, 129, 1–12.
Feng, Y., Chen, N. F., & Li, H. (2021). Automatic pronunciation assessment: A review. IEEE Transactions on Learning Technologies, 14(3), 343–358.
Li, Y., Qian, Y., & Yu, K. (2020). Improving mispronunciation detection using deep neural network based acoustic models. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28, 239–251.
Zhang, X., & Wu, J. (2022). Speaker-adaptive speech processing using deep learning: A survey. IEEE Access, 10, 24567–24589.
Chorowski, J., Weiss, R. J., Bengio, S., & van den Oord, A. (2021). Unsupervised speech representation learning with wav2vec and beyond. IEEE Signal Processing Magazine, 38(3), 68–80.
Rao, K., Sak, H., & Prabhavalkar, R. (2020). Exploring architectures, data and units for streaming end-to-end speech recognition with RNN-transducer. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2020, 193–199.
Wang, Z., Qian, Y., & Soong, F. (2022). Speaker-aware pronunciation assessment using deep neural networks. IEEE Transactions on Audio, Speech, and Language Processing, 30, 1125–1137.