Aditya Yadavalli

selected publications

1. ACL

   SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT

   Yadavalli, Aditya*, Yadavalli, Alekhya*, and Tobin, Vera

   In Proc. ACL 2023

   Abs Bib PDF

   Second language acquisition (SLA) research has extensively studied cross-linguistic transfer, the influence of linguistic structure of a speaker’s native language [L1] on the successful acquisition of a foreign language [L2]. Effects of such transfer can be positive (facilitating acquisition) or negative (impeding acquisition). We find that NLP literature has not given enough attention to the phenomenon of negative transfer. To understand patterns of both positive and negative transfer between L1 and L2, we model sequential second language acquisition in LMs. Further, we build a Mutlilingual Age Ordered CHILDES (MAO-CHILDES) – a dataset consisting of 5 typologically diverse languages, i.e., German, French, Polish, Indonesian, and Japanese – to understand the degree to which native Child-Directed Speech (CDS) [L1] can help or conflict with English language acquisition [L2]. To examine the impact of native CDS, we use the TILT-based cross lingual transfer learning approach established by Papadimitriou and Jurafsky (2020) and find that, as in human SLA, language family distance predicts more negative transfer. Additionally, we find that conversational speech data shows greater facilitation for language acquisition than scripted speech data. Our findings call for further research using our novel Transformer-based SLA models and we would like to encourage it by releasing our code, data, and models

   @article{Yadavalli2023SLABERTTP,
     title = {SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT},
     author = {Yadavalli, Aditya* and Yadavalli, Alekhya* and Tobin, Vera},
     journal = {In Proc. ACL},
     year = {2023},
     volume = {abs/2305.19589}
   }

2. TACL

   AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR

   Olatunji, Tobi, Afonja, Tejumade,  Yadavalli, Aditya, Emezue, Chris Chinenye, Singh, Sahib, Dossou, Bonaventure F. P., Osuchukwu, Joanne, Osei, Salomey, Tonja, Atnafu Lambebo, Etori, Naome, and Mbataku, Clinton

   Transactions of the Association for Computational Linguistics Dec 2023

   Abs Bib PDF

   Africa has a very poor doctor-to-patient ratio. At very busy clinics, doctors could see 30+ patients per day—a heavy patient burden compared with developed countries—but productivity tools such as clinical automatic speech recognition (ASR) are lacking for these overworked clinicians. However, clinical ASR is mature, even ubiquitous, in developed nations, and clinician-reported performance of commercial clinical ASR systems is generally satisfactory. Furthermore, the recent performance of general domain ASR is approaching human accuracy. However, several gaps exist. Several publications have highlighted racial bias with speech-to-text algorithms and performance on minority accents lags significantly. To our knowledge, there is no publicly available research or benchmark on accented African clinical ASR, and speech data is non-existent for the majority of African accents. We release AfriSpeech, 200hrs of Pan-African English speech, 67,577 clips from 2,463 unique speakers across 120 indigenous accents from 13 countries for clinical and general domain ASR, a benchmark test set, with publicly available pre-trained models with SOTA performance on the AfriSpeech benchmark.

   @article{Olatunji2023,
     title = {AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR},
     author = {Olatunji, Tobi and Afonja, Tejumade and Yadavalli, Aditya and Emezue, Chris Chinenye and Singh, Sahib and Dossou, Bonaventure F. P. and Osuchukwu, Joanne and Osei, Salomey and Tonja, Atnafu Lambebo and Etori, Naome and Mbataku, Clinton},
     journal = {Transactions of the Association for Computational Linguistics},
     year = {2023},
     volume = {11},
     pages = {1669-1685},
     month = dec
   }

3. SLT

   How Do Phonological Properties Affect Bilingual Automatic Speech Recognition?

   Jain, Shelly*,  Yadavalli, Aditya*, Mirishkar, Ganesh, and Vuppala, Anil Kumar

   In IEEE Spoken Language Technology Workshop Dec 2022

   Abs Bib PDF

   Multilingual Automatic Speech Recognition (ASR) for Indian languages is an obvious technique for leveraging their similarities. We present a detailed analysis of how phonological similarities and differences between languages affect Time Delay Neural Network (TDNN) and End-to-End (E2E) ASR. To study this, we select genealogically similar pairs from five Indian languages and train bilingual acoustic models. We compare these against corresponding monolingual acoustic models and find similar phoneme distributions within speech to be the primary factor for improving model performance, with phoneme overlap being secondary. The influence of phonological properties on performance is visible in both cases. Word Error Rate (WER) of E2E decreased by a median of 2.35%, and upto 8.5% when the phonological similarity was greatest. WER of TDNN increased by 11.69% when the similarity was lowest. Thus, it is clear that the choice of supplementary language is important for model performance.

   @inproceedings{Shelly2022SLT,
     author = {Jain, Shelly* and Yadavalli, Aditya* and Mirishkar, Ganesh and Vuppala, Anil Kumar},
     booktitle = {IEEE Spoken Language Technology Workshop},
     title = {How Do Phonological Properties Affect Bilingual Automatic Speech Recognition?},
     year = {2022}
   }

4. Interspeech

   Multi-Task End-to-End Model for Telugu Dialect and Speech Recognition

   Yadavalli, Aditya, Mirishkar, Ganesh, and Vuppala, Anil Kumar

   In Proc. Interspeech Dec 2022

   Abs Bib PDF

   Conventional Automatic Speech Recognition (ASR) systems are susceptible to dialect variations within a language, thereby adversely affecting the ASR. Therefore, the current practice is to use dialect-specific ASRs. However, dialect-specific information or data is hard to obtain making it difficult to build dialect-specific ASRs. Furthermore, it is cumbersome to maintain multiple dialect-specific ASR systems for each language. We build a unified multi-dialect End-to-End ASR that removes the need for a dialect recognition block and the need to maintain multiple dialect-specific ASRs for three Telugu regional dialects: Telangana, Coastal Andhra, and Rayalaseema. We find that pooling the data and training a multi-dialect ASR benefits the low-resource dialect the most – an improvement of over 9.71% in relative Word Error Rate (WER). Subsequently, we experiment with multi-task ASRs where the primary task is to transcribe the audio and the secondary task is to predict the dialect. We do this by adding a Dialect ID to the output targets. Such a model outperforms naive multi-dialect ASRs by up to 8.24% in relative WER. Additionally, we test this model on a dialect recognition task and find that it outperforms strong baselines by 6.14% in accuracy.

   @inproceedings{Aditya2022Interspeech,
     author = {Yadavalli, Aditya and Mirishkar, Ganesh and Vuppala, Anil Kumar},
     booktitle = {Proc. Interspeech},
     title = {Multi-Task End-to-End Model for Telugu Dialect and Speech Recognition},
     pages = {1387--1391},
     doi = {10.21437/Interspeech.2022-10739},
     year = {2022}
   }

5. NAACL-SRW

   Exploring the Effect of Dialect Mismatched Language Models in Telugu Automatic Speech Recognition

   Yadavalli, Aditya, Mirishkar, Ganesh, and Vuppala, Anil Kumar

   In North American Chapter of the Association of Computational Linguistics Student Research Workshop Dec 2022

   Abs Bib PDF

   Previous research has found that Acoustic Models (AM) of an Automatic Speech Recognition (ASR) system are susceptible to dialect variations within a language, thereby adversely affecting the ASR. To counter this, researchers have proposed to build a dialect-specific AM while keeping the Language Model (LM) constant for all the dialects. This study explores the effect of dialect mismatched LM by considering three different Telugu regional dialects: Telangana, Coastal Andhra, and Rayalaseema. We show that dialect variations that surface in the form of a different lexicon, grammar, and occasionally semantics can significantly degrade the performance of the LM under mismatched conditions. Therefore, this degradation has an adverse effect on the ASR even when dialect-specific AM is used. We show a degradation of up to 13.13 perplexity points when LM is used under mismatched conditions. Furthermore, we show a degradation of over 9% and over 15% in Character Error Rate (CER) and Word Error Rate (WER), respectively, in the ASR systems when using mismatched LMs over matched LMs.

   @inproceedings{Aditya2022NAACLSRW,
     author = {Yadavalli, Aditya and Mirishkar, Ganesh and Vuppala, Anil Kumar},
     booktitle = {North American Chapter of the Association of Computational Linguistics Student Research Workshop},
     title = {Exploring the Effect of Dialect Mismatched Language Models in Telugu
     Automatic Speech Recognition},
     publisher = {Association for Computational Linguistics (ACL)},
     year = {2022}
   }