Meng-Ping Lin, Jen-Cheng Hou, Chia-Wei Chen
et al.
Speech enhancement (SE) aims to improve the quality and intelligibility of speech in noisy environments. Recent studies have shown that incorporating visual cues in audio signal processing can enhance SE performance. Given that human speech communication naturally involves audio, visual, and linguistic modalities, it is reasonable to expect additional improvements by integrating linguistic information. However, effectively bridging these modality gaps, particularly during knowledge transfer remains a significant challenge. In this paper, we propose a novel multi-modal learning framework, termed DLAV-SE, which leverages a diffusion-based model integrating audio, visual, and linguistic information for audio-visual speech enhancement (AVSE). Within this framework, the linguistic modality is modeled using a pretrained language model (PLM), which transfers linguistic knowledge to the audio-visual domain through a cross-modal knowledge transfer (CMKT) mechanism during training. After training, the PLM is no longer required at inference, as its knowledge is embedded into the AVSE model through the CMKT process. We conduct a series of SE experiments to evaluate the effectiveness of our approach. Results show that the proposed DLAV-SE system significantly improves speech quality and reduces generative artifacts, such as phonetic confusion, compared to state-of-the-art (SOTA) methods. Furthermore, visualization analyses confirm that the CMKT method enhances the generation quality of the AVSE outputs. These findings highlight both the promise of diffusion-based methods for advancing AVSE and the value of incorporating linguistic information to further improve system performance.
The introduction of large language models and other influential developments in AI-based language processing have led to an evolution in the methods available to quantitatively analyse language data. With the resultant growth of attention on language processing, significant challenges have emerged, including the lack of standardisation in organising and sharing linguistic data and the absence of standardised and reproducible processing methodologies. Striving for future standardisation, we first propose the Language Processing Data Structure (LPDS), a data structure inspired by the Brain Imaging Data Structure (BIDS), a widely adopted standard for handling neuroscience data. It provides a folder structure and file naming conventions for linguistic research. Second, we introduce pelican nlp, a modular and extensible Python package designed to enable streamlined language processing, from initial data cleaning and task-specific preprocessing to the extraction of sophisticated linguistic and acoustic features, such as semantic embeddings and prosodic metrics. The entire processing workflow can be specified within a single, shareable configuration file, which pelican nlp then executes on LPDS-formatted data. Depending on the specifications, the reproducible output can consist of preprocessed language data or standardised extraction of both linguistic and acoustic features and corresponding result aggregations. LPDS and pelican nlp collectively offer an end-to-end processing pipeline for linguistic data, designed to ensure methodological transparency and enhance reproducibility.
While the embedding of words has revolutionized the field of Natural Language Processing, the embedding of concepts has received much less attention so far. A dense and meaningful representation of concepts, however, could prove useful for several tasks in computational linguistics, especially those involving cross-linguistic data or sparse data from low resource languages. First methods that have been proposed so far embed concepts from automatically constructed colexification networks. While these approaches depart from automatically inferred polysemies, attested across a larger number of languages, they are restricted to the word level, ignoring lexical relations that would only hold for parts of the words in a given language. Building on recently introduced methods for the inference of partial colexifications, we show how they can be used to improve concept embeddings in meaningful ways. The learned embeddings are evaluated against lexical similarity ratings, recorded instances of semantic shift, and word association data. We show that in all evaluation tasks, the inclusion of partial colexifications lead to improved concept representations and better results. Our results further show that the learned embeddings are able to capture and represent different semantic relationships between concepts.
Large Language Models (LLMs) are commonly evaluated for robustness against paraphrased or semantically equivalent jailbreak prompts, yet little attention has been paid to linguistic variation as an attack surface. In this work, we systematically study how linguistic styles such as fear or curiosity can reframe harmful intent and elicit unsafe responses from aligned models. We construct style-augmented jailbreak benchmark by transforming prompts from 3 standard datasets into 11 distinct linguistic styles using handcrafted templates and LLM-based rewrites, while preserving semantic intent. Evaluating 16 open- and close-source instruction-tuned models, we find that stylistic reframing increases jailbreak success rates by up to +57 percentage points. Styles such as fearful, curious and compassionate are most effective and contextualized rewrites outperform templated variants. To mitigate this, we introduce a style neutralization preprocessing step using a secondary LLM to strip manipulative stylistic cues from user inputs, significantly reducing jailbreak success rates. Our findings reveal a systemic and scaling-resistant vulnerability overlooked in current safety pipelines.
El cine ha incorporado en sus historias el oficio del periodismo ya que, por la naturaleza de la profesión, permite introducir narraciones con un alto grado de interés. En el imaginario colectivo los periodistas se posicionan como el cuarto poder y tienen una responsabilidad social. En los comienzos del cine y hasta los años 90, aproximadamente, las mujeres -frente a sus compañeros masculinos- ocupaban un rol secundario, en papeles estereotipados que las mostraban como profesionales frías, implacables, dispuestas a todo para conseguir la noticia en detrimento de su vida personal. Aparecen masculinizadas y sexualizadas. Empleando una triangulación metodológica que aplica las técnicas del análisis descriptivo, el análisis fílmico y la entrevista en profundidad se estudia el largometraje El reino, de Rodrigo Sorogoyen, para determinar cómo se representa la figura de la periodista en él. Tras el estudio de los resultados se observa que las periodistas en activo, en la actualidad, no se ven representadas por la visión de su profesión que ofrece El reino ni el cine en general. Esto se debe a que se siguen perpetuando ciertos estereotipos con los que se muestra en la gran pantalla el rol de la mujer periodista.
Communication. Mass media, Journalism. The periodical press, etc.
The article questions the notions of virality and interdisciplinarity through the semiodiscursive analysis of memes of #ChatonsMignons threads in the context of the pre-campaign of the 2022 French presidential elections. The analysis underlines how mobilized interdisciplinary references serve an argumentative dimension.
Language. Linguistic theory. Comparative grammar, Communication. Mass media
This study explores linguistic differences between human and LLM-generated dialogues, using 19.5K dialogues generated by ChatGPT-3.5 as a companion to the EmpathicDialogues dataset. The research employs Linguistic Inquiry and Word Count (LIWC) analysis, comparing ChatGPT-generated conversations with human conversations across 118 linguistic categories. Results show greater variability and authenticity in human dialogues, but ChatGPT excels in categories such as social processes, analytical style, cognition, attentional focus, and positive emotional tone, reinforcing recent findings of LLMs being "more human than human." However, no significant difference was found in positive or negative affect between ChatGPT and human dialogues. Classifier analysis of dialogue embeddings indicates implicit coding of the valence of affect despite no explicit mention of affect in the conversations. The research also contributes a novel, companion ChatGPT-generated dataset of conversations between two independent chatbots, which were designed to replicate a corpus of human conversations available for open access and used widely in AI research on language modeling. Our findings enhance understanding of ChatGPT's linguistic capabilities and inform ongoing efforts to distinguish between human and LLM-generated text, which is critical in detecting AI-generated fakes, misinformation, and disinformation.
Ramaneswaran Selvakumar, Sonal Kumar, Hemant Kumar Giri
et al.
Open-vocabulary audio language models (ALMs), like Contrastive Language Audio Pretraining (CLAP), represent a promising new paradigm for audio-text retrieval using natural language queries. In this paper, for the first time, we perform controlled experiments on various benchmarks to show that existing ALMs struggle to generalize to linguistic variations in textual queries. To address this issue, we propose RobustCLAP, a novel and compute-efficient technique to learn audio-language representations agnostic to linguistic variations. Specifically, we reformulate the contrastive loss used in CLAP architectures by introducing a multi-view contrastive learning objective, where paraphrases are treated as different views of the same audio scene and use this for training. Our proposed approach improves the text-to-audio retrieval performance of CLAP by 0.8%-13% across benchmarks and enhances robustness to linguistic variation.
Ekaterina Taktasheva, Maxim Bazhukov, Kirill Koncha
et al.
Minimal pairs are a well-established approach to evaluating the grammatical knowledge of language models. However, existing resources for minimal pairs address a limited number of languages and lack diversity of language-specific grammatical phenomena. This paper introduces the Russian Benchmark of Linguistic Minimal Pairs (RuBLiMP), which includes 45k pairs of sentences that differ in grammaticality and isolate a morphological, syntactic, or semantic phenomenon. In contrast to existing benchmarks of linguistic minimal pairs, RuBLiMP is created by applying linguistic perturbations to automatically annotated sentences from open text corpora and carefully curating test data. We describe the data collection protocol and present the results of evaluating 25 language models in various scenarios. We find that the widely used language models for Russian are sensitive to morphological and agreement-oriented contrasts but fall behind humans on phenomena requiring understanding of structural relations, negation, transitivity, and tense. RuBLiMP, the codebase, and other materials are publicly available.
Somnath Banerjee, Avik Halder, Rajarshi Mandal
et al.
The integration of pretrained language models (PLMs) like BERT and GPT has revolutionized NLP, particularly for English, but it has also created linguistic imbalances. This paper strategically identifies the need for linguistic equity by examining several knowledge editing techniques in multilingual contexts. We evaluate the performance of models such as Mistral, TowerInstruct, OpenHathi, Tamil-Llama, and Kan-Llama across languages including English, German, French, Italian, Spanish, Hindi, Tamil, and Kannada. Our research identifies significant discrepancies in normal and merged models concerning cross-lingual consistency. We employ strategies like 'each language for itself' (ELFI) and 'each language for others' (ELFO) to stress-test these models. Our findings demonstrate the potential for LLMs to overcome linguistic barriers, laying the groundwork for future research in achieving linguistic inclusivity in AI technologies.
The article is devoted to controversial issues of using experimental methods (associative and semantic experiments) in cognitive linguistic research. The aim of the work is to identify differences in the goals, procedure and format of the data obtained in the course of associative and semantic experiments. The scientific novelty of the study lies in the fact that for the first time: 1) a review of the texts of doctoral dissertations over the past 16 years and several scientific and methodological works conducted within the framework of the cognitive direction was carried out to describe the linguo-cognitive methodology; 2) an attempt was made to compare cognitive interpretations of the meanings of the words треск (crack) and хруст (crunch) based on the results of associative and semantic experiments. The results obtained in the study showed that: 1) the variability of the methods of linguo-cognitive research used in the considered works is due to differences in the understanding of the linguo-cognitive method by the authors; in some cases, the problem of a detailed description of the linguo-cognitive methodology in relation to the tasks of the study is difficult to solve; 2) the composition and formulation of features included in the cognitive interpretation of the meanings of the words треск (crack) and хруст (crunch) according to the results of associative and semantic experiments are different; 3) when conducting experiments, there are errors associated with the subjectivity of the data obtained; additional experimental procedures make it possible to reduce these errors to a minimum and thereby increase the objectivity of the experimental data.
This paper analyses, if the Intertemporal Guarantee of Freedom, that was developed by the German Federal Constitutional Court (GFCC), can be used to expand the protection of human rights against the harms of climate change. The case of the Swiss Senior Women shows that there are jurisdictions, where the Intertemporal Guarantee of Freedom could be applied to improve standing and the control standard of states’ climate change action. Within international law bodies with jurisdiction over human rights treaties there are distinctive standards of protection against the harms of climate change. A major deficit within the international human rights protection against climate change lies within the focus on the positive obligations and the corresponding wide margin of appreciation granted to the states. The Intertemporal Guarantee of Freedom could provide a protection expansion in this regard, especially in the case of the European Court of Human Rights. It could also enable and legitimise present human rights concerns focused on the future actions of states following their past inaction. One considerable hurdle that is not addressed by it are procedural hurdles like the Plaumann formula applied by the European Court of Justice. The Intertemporal Guarantee of Freedom cannot solve major problems for climate change litigation like procedural hurdles. Yet, it can provide a new approach for complaints to address unambitious mitigation legislation which will lead to future human rights infringements.
Online manipulation is a pressing concern for democracies, but the actions and strategies of coordinated inauthentic accounts, which have been used to interfere in elections, are not well understood. We analyze a five million-tweet multilingual dataset related to the 2017 French presidential election, when a major information campaign led by Russia called "#MacronLeaks" took place. We utilize heuristics to identify coordinated inauthentic accounts and detect attitudes, concerns and emotions within their tweets, collectively known as socio-linguistic characteristics. We find that coordinated accounts retweet other coordinated accounts far more than expected by chance, while being exceptionally active just before the second round of voting. Concurrently, socio-linguistic characteristics reveal that coordinated accounts share tweets promoting a candidate at three times the rate of non-coordinated accounts. Coordinated account tactics also varied in time to reflect news events and rounds of voting. Our analysis highlights the utility of socio-linguistic characteristics to inform researchers about tactics of coordinated accounts and how these may feed into online social manipulation.
When listening to connected speech, human brain can extract multiple levels of linguistic units, such as syllables, words, and sentences. It has been hypothesized that the time scale of cortical activity encoding each linguistic unit is commensurate with the time scale of that linguistic unit in speech. Evidence for the hypothesis originally comes from studies using the frequency-tagging paradigm that presents each linguistic unit at a constant rate, and more recently extends to studies on natural speech. For natural speech, it is sometimes assumed that neural encoding of different levels of linguistic units is captured by the neural response tracking speech envelope in different frequency bands (e.g., around 1 Hz for phrases, around 2 Hz for words, and around 4 Hz for syllables). Here, we analyze the coherence between speech envelope and idealized responses, each of which tracks a single level of linguistic unit. Four units, i.e., phones, syllables, words, and sentences, are separately considered. It is shown that the idealized phone-, syllable-, and word-tracking responses all correlate with the speech envelope both around 3-6 Hz and below ~1 Hz. Further analyses reveal that the 1-Hz correlation mainly originates from the pauses in connected speech. The results here suggest that a simple frequency-domain decomposition of envelope-tracking activity cannot separate the neural responses to different linguistic units in natural speech.
Covert communication (also known as steganography) is the practice of concealing a secret inside an innocuous-looking public object (cover) so that the modified public object (covert code) makes sense to everyone but only someone who knows the code can extract the secret (message). Linguistic steganography is the practice of encoding a secret message in natural language text such as spoken conversation or short public communications such as tweets.. While ad hoc methods for covert communications in specific domains exist ( JPEG images, Chinese poetry, etc), there is no general model for linguistic steganography specifically. We present a novel mathematical formalism for creating linguistic steganographic codes, with three parameters: Decodability (probability that the receiver of the coded message will decode the cover correctly), density (frequency of code words in a cover code), and detectability (probability that an attacker can tell the difference between an untampered cover compared to its steganized version). Verbal or linguistic steganography is most challenging because of its lack of artifacts to hide the secret message in. We detail a practical construction in Python of a steganographic code for Tweets using inserted words to encode hidden digits while using n-gram frequency distortion as the measure of detectability of the insertions. Using the publicly accessible Stanford Sentiment Analysis dataset we implemented the tweet steganization scheme -- a codeword (an existing word in the data set) inserted in random positions in random existing tweets to find the tweet that has the least possible n-gram distortion. We argue that this approximates KL distance in a localized manner at low cost and thus we get a linguistic steganography scheme that is both formal and practical and permits a tradeoff between codeword density and detectability of the covert message.
Outside the work of a small number of pioneering Slavic linguists and historians, the work of Nikolai Marr is little known today. In this essay I argue that Marr’s writings are worth re-examining, particularly in the light of his critique of European linguistics as propagating the racist ideology of imperialism and the subjugation of colonized peoples – now characterized as Orientalism. The first part of the essay situates Marr’s work within the wider context of the division since the nineteenth century between mainstream European comparative philology based in Germany, with its hierarchical model of family trees, and the more egalitarian tradition developed in Eastern Europe that emphasized the lateral interactions of speech in social contexts, with related languages operating in their own ecosystems of geographical proximity. In the second part of the essay I consider some of the elements of Marr’s work that remain of interest: his critique of Orientalism in linguistics, of the relations between western knowledges and forms of colonial power, on language as something not to be studied in isolation but as a living part of the social ecosystem and its power relations, and his conceptual emphasis on lateral thinking through rhizomatic forms and hybridization.
: PhD thesis in Computer Science written by Alessandra Teresa Cigna-rella under the supervision of Dr. Cristina Bosco (University of Turin) and Prof. Dr. Paolo Rosso (Universitat Polit`ecnica de Val`encia). This thesis was developed under a cotutelle between the PRHLT Research Center of the Universitat Polit`ecnica de Val`encia, Spain and the Computer Science Department of the University of Turin, Italy. The thesis defense was held in Torino ( online ), Italy on October 26th, 2021. The doctoral committee was composed by: Prof. Dr. Joakim Nivre (Department of Linguistics and Philology, Uppsala University, Sweden), Dr. Saif M. Mohammad (National Research Council Canada,