Aditya Yadavalli

selected publications

1. ACL

   What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels

   Aditya Yadavalli, Tiago Pimentel, Tamar I Regev, Ethan Gotlieb Wilcox, and Alex Warstadt

   In Proc. ACL 2026

   Abs Bib PDF

   Prosody—the melody of speech—conveys critical information often not captured by the words or text of a message. In this paper, we propose an information-theoretic approach to quantify how much is conveyed by prosody that is not recoverable from text alone, and, crucially, what prosody conveys. Our approach applies large speech and language models to estimate the mutual information between a particular dimension of an utterance’s meaning (e.g., its emotion) and any of its communication channels (e.g., audio or text). We then use this approach to quantify the information conveyed by audio and text about sarcasm, emotion, and questionhood, using speech from television and podcasts. We find that for sarcasm and emotion, the audio channel, and by implication the prosodic channel, transmits over an order of magnitude more information about these features than the text channel alone, at least when long-term context beyond the current sentence is unavailable. For questionhood, prosody provides comparatively less additional information. We conclude by outlining a program applying our approach to more dimensions of meaning, communication channels, and languages.

   @article{Yadavalli2026Prosody,
     title = {What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels},
     author = {Yadavalli, Aditya and Pimentel, Tiago and Regev, Tamar I and Wilcox, Ethan Gotlieb and Warstadt, Alex},
     journal = {In Proc. ACL},
     year = {2026}
   }

2. EACL Findings

   AccentFold: A Journey through African Accents for Zero-Shot ASR Adaptation to Target Accents

   Abraham Owodunni*,  Aditya Yadavalli*, Chris Emezue*, Tobi Olatunji*, and Clinton Mbataku

   In Proc. EACL Findings 2024

   Abs Bib PDF

   Despite advancements in speech recognition, accented speech remains challenging. While previous approaches have focused on modeling techniques or creating accented speech datasets, gathering sufficient data for the multitude of accents, particularly in the African context, remains impractical due to their sheer diversity and associated budget constraints. To address these challenges, we propose AccentFold, a method that exploits spatial relationships between learned accent embeddings to improve downstream Automatic Speech Recognition (ASR). Our exploratory analysis of speech embeddings representing 100+ African accents reveals interesting spatial accent relationships highlighting geographic and genealogical similarities, capturing consistent phonological, and morphological regularities, all learned empirically from speech. Furthermore, we discover accent relationships previously uncharacterized by the Ethnologue. Through empirical evaluation, we demonstrate the effectiveness of AccentFold by showing that, for out-of-distribution (OOD) accents, sampling accent subsets for training based on AccentFold information outperforms strong baselines a relative WER improvement of 4.6%. AccentFold presents a promising approach for improving ASR performance on accented speech, particularly in the context of African accents, where data scarcity and budget constraints pose significant challenges. Our findings emphasize the potential of leveraging linguistic relationships to improve zero-shot ASR adaptation to target accents

   @article{Owodunni2024AccentFoldAJ,
     title = {AccentFold: A Journey through African Accents for Zero-Shot ASR Adaptation to Target Accents},
     author = {Owodunni, Abraham* and Yadavalli, Aditya* and Emezue, Chris* and Olatunji, Tobi* and Mbataku, Clinton},
     journal = {In Proc. EACL Findings},
     year = {2024}
   }

3. ACL

   SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT

   Aditya Yadavalli*, Alekhya Yadavalli*, and Vera Tobin

   In Proc. ACL 2023

   Abs Bib PDF

   Second language acquisition (SLA) research has extensively studied cross-linguistic transfer, the influence of linguistic structure of a speaker’s native language [L1] on the successful acquisition of a foreign language [L2]. Effects of such transfer can be positive (facilitating acquisition) or negative (impeding acquisition). We find that NLP literature has not given enough attention to the phenomenon of negative transfer. To understand patterns of both positive and negative transfer between L1 and L2, we model sequential second language acquisition in LMs. Further, we build a Mutlilingual Age Ordered CHILDES (MAO-CHILDES) – a dataset consisting of 5 typologically diverse languages, i.e., German, French, Polish, Indonesian, and Japanese – to understand the degree to which native Child-Directed Speech (CDS) [L1] can help or conflict with English language acquisition [L2]. To examine the impact of native CDS, we use the TILT-based cross lingual transfer learning approach established by Papadimitriou and Jurafsky (2020) and find that, as in human SLA, language family distance predicts more negative transfer. Additionally, we find that conversational speech data shows greater facilitation for language acquisition than scripted speech data. Our findings call for further research using our novel Transformer-based SLA models and we would like to encourage it by releasing our code, data, and models

   @article{Yadavalli2023SLABERTTP,
     title = {SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT},
     author = {Yadavalli, Aditya* and Yadavalli, Alekhya* and Tobin, Vera},
     journal = {In Proc. ACL},
     year = {2023},
     volume = {abs/2305.19589}
   }

4. SLT

   How Do Phonological Properties Affect Bilingual Automatic Speech Recognition?

   Shelly Jain*,  Aditya Yadavalli*, Ganesh Mirishkar, and Anil Kumar Vuppala

   In IEEE Spoken Language Technology Workshop 2022

   Abs Bib PDF

   Multilingual Automatic Speech Recognition (ASR) for Indian languages is an obvious technique for leveraging their similarities. We present a detailed analysis of how phonological similarities and differences between languages affect Time Delay Neural Network (TDNN) and End-to-End (E2E) ASR. To study this, we select genealogically similar pairs from five Indian languages and train bilingual acoustic models. We compare these against corresponding monolingual acoustic models and find similar phoneme distributions within speech to be the primary factor for improving model performance, with phoneme overlap being secondary. The influence of phonological properties on performance is visible in both cases. Word Error Rate (WER) of E2E decreased by a median of 2.35%, and upto 8.5% when the phonological similarity was greatest. WER of TDNN increased by 11.69% when the similarity was lowest. Thus, it is clear that the choice of supplementary language is important for model performance.

   @inproceedings{Shelly2022SLT,
     author = {Jain, Shelly* and Yadavalli, Aditya* and Mirishkar, Ganesh and Vuppala, Anil Kumar},
     booktitle = {IEEE Spoken Language Technology Workshop},
     title = {How Do Phonological Properties Affect Bilingual Automatic Speech Recognition?},
     year = {2022}
   }