Aleksandra Krasnodębska, Karolina Seweryn, Szymon Łukasik
et al.
Despite increasing efforts to ensure the safety of large language models (LLMs), most existing safety assessments and moderation tools remain heavily biased toward English and other high-resource languages, leaving majority of global languages underexamined. To address this gap, we introduce a manually annotated benchmark dataset for language model safety classification in Polish. We also create adversarially perturbed variants of these samples designed to challenge model robustness. We conduct a series of experiments to evaluate LLM-based and classifier-based models of varying sizes and architectures. Specifically, we fine-tune three models: Llama-Guard-3-8B, a HerBERT-based classifier (a Polish BERT derivative), and PLLuM, a Polish-adapted Llama-8B model. We train these models using different combinations of annotated data and evaluate their performance, comparing it against publicly available guard models. Results demonstrate that the HerBERT-based classifier achieves the highest overall performance, particularly under adversarial conditions.
The Japanese word "好きだ" (suki da, like) is a な-adjective (adjectival noun) used to express preferences. It typically appears in the structure "～が好きだ" to indicate liking someone or something. However, in practical Japanese usage, the structure "～を好きだ" is also used in certain contexts, raising questions about its grammatical variability and acceptability in modern Japanese. With advances in large-scale corpora, language research has increasingly transitioned into the era of big data, allowing for more systematic and empirical analysis of linguistic patterns. This study explores the grammatical role, frequency, and acceptance of this alternative structure, offering insights into language theory and language education.
The soft power diplomacy has been used by Japan since the end of WWII and post bubble economy, and was officialized as ‘Cool Japan’ program which includes all aspects of Japanese culture and cultural products. On the other hand, ‘city pop’, a genre popular in Japan around the 80’s, surfaced again in the 2020’s as a result of YouTube’s algorithm, and has been a trend on the internet with many global fans. The purpose of this research was to find out whether Japan city pop music trend contributes to Cool Japan program, with mixed method approach and data collecting method of online survey on Japan city pop fans in Indonesia. It is then concluded that Japan city pop has been contributing to Cool Japan by meeting it’s main goal and strategies of creating Japan’s image as a cool nation on its own.
The “breast-is-best” ideology (bonyū shinwa), which is still firmly rooted in Japanese consciousness, can cause women stress and pain, becoming a source of anxiety and even depression. When used in fiction, we can read it as a topos that highlights the great pressure experienced by mothers as primary caregivers in contemporary Japanese society.
In this paper, I analyze the representation of breastfeeding in relation to the concept of care in the novel Chibusa no kuni de (In the Country of Breasts, 2020) by Fukazawa Ushio. I argue that Fukazawa’s novel tackles the issue of “breast-is-best” discourse on several levels. On the one hand, this novel describes the discomfort that both the lack and overproduction of breastmilk can cause in women, and also depicts women who lack the so-called “maternal instinct.” On the other hand, the novel also questions the traditional view of care as something that should be performed at the individual level in the intimate sphere. My analysis of Fukazawa’s work will highlight the link between breastfeeding, care, and power, while shedding light on the interdependence between caregivers and care-receivers.
Language and Literature, Japanese language and literature
Sayantan Adak, Daivik Agrawal, Animesh Mukherjee
et al.
We investigate the knowledge of object affordances in pre-trained language models (LMs) and pre-trained Vision-Language models (VLMs). A growing body of literature shows that PTLMs fail inconsistently and non-intuitively, demonstrating a lack of reasoning and grounding. To take a first step toward quantifying the effect of grounding (or lack thereof), we curate a novel and comprehensive dataset of object affordances -- Text2Afford, characterized by 15 affordance classes. Unlike affordance datasets collected in vision and language domains, we annotate in-the-wild sentences with objects and affordances. Experimental results reveal that PTLMs exhibit limited reasoning abilities when it comes to uncommon object affordances. We also observe that pre-trained VLMs do not necessarily capture object affordances effectively. Through few-shot fine-tuning, we demonstrate improvement in affordance knowledge in PTLMs and VLMs. Our research contributes a novel dataset for language grounding tasks, and presents insights into LM capabilities, advancing the understanding of object affordances. Codes and data are available at https://github.com/sayantan11995/Text2Afford
While large language models exhibit certain cross-lingual generalization capabilities, they suffer from performance degradation (PD) on unseen closely-related languages (CRLs) and dialects relative to their high-resource language neighbour (HRLN). However, we currently lack a fundamental understanding of what kinds of linguistic distances contribute to PD, and to what extent. Furthermore, studies of cross-lingual generalization are confounded by unknown quantities of CRL language traces in the training data, and by the frequent lack of availability of evaluation data in lower-resource related languages and dialects. To address these issues, we model phonological, morphological, and lexical distance as Bayesian noise processes to synthesize artificial languages that are controllably distant from the HRLN. We analyse PD as a function of underlying noise parameters, offering insights on model robustness to isolated and composed linguistic phenomena, and the impact of task and HRL characteristics on PD. We calculate parameter posteriors on real CRL-HRLN pair data and show that they follow computed trends of artificial languages, demonstrating the viability of our noisers. Our framework offers a cheap solution for estimating task performance on an unseen CRL given HRLN performance using its posteriors, as well as for diagnosing observed PD on a CRL in terms of its linguistic distances from its HRLN, and opens doors to principled methods of mitigating performance degradation.
The article concerns the problem of prisoners of war during the Russo-Japanese war. The problem appears in the focus of modern researchers quite rarely: they usually write about political, military, and economic aspects of the war. The article describes the process of choosing the places of stay for the captured Japanese soldiers and officers in 1904–1905.
By the beginning of the war, Russia assumed a number of obligations, since, during the Conference in Hague in 1899, it signed the Convention with respect to the Laws and Customs of War on Land, prescribing the treatment of prisoners based on humanistic ideals.
The sources used during writing the work represent the latest research materials and the archival documents. These documents are stored in the Russian State Military Historical Archive (RGVIA) in Moscow.
While determining the places for the Japanese stay, the Military department faced a severe problem, since it was necessary to take into account different factors: whether it would be possible to ensure guarding the prisoners of war, whether there was a suitable building for their accommodation, whether the Japanese soldiers and officers would have opportunities to commit sabotage. The latest issue was vividly discussed in the official documents of the period: there were concerns that the Japanese would destroy railways, as they did in Manchuria.
The idea that the place for the Japanese prisoners should be in the Far East was rejected almost immediately due to its proximity to the theater of operations. Siberia also did not fit, since a railway passed through it – the most important transport artery during the war, so it was decided to place the Japanese in European Russia. The city of Penza was chosen as a collection point, from where prisoners of war were distributed to the cities of the Kazan, Moscow, Kiev, and Saint Petersburg military districts.
However, in the autumn of 1904, Emperor Nicholas II issued a decree that the Japanese should not be stationed near the passage of the Russian troops. After that it was decided to accommodate all Japanese prisoners of war in one place: in the village of Medved, Novgorod province. The barracks located there were perfect for housing a small number of Japanese prisoners, where they stayed until the end of the war.
In a code-switched (CS) scenario, the use of spoken language diarization (LD) as a pre-possessing system is essential. Further, the use of implicit frameworks is preferable over the explicit framework, as it can be easily adapted to deal with low/zero resource languages. Inspired by speaker diarization (SD) literature, three frameworks based on (1) fixed segmentation, (2) change point-based segmentation and (3) E2E are proposed to perform LD. The initial exploration with synthetic TTSF-LD dataset shows, using x-vector as implicit language representation with appropriate analysis window length ($N$) can able to achieve at per performance with explicit LD. The best implicit LD performance of $6.38$ in terms of Jaccard error rate (JER) is achieved by using the E2E framework. However, considering the E2E framework the performance of implicit LD degrades to $60.4$ while using with practical Microsoft CS (MSCS) dataset. The difference in performance is mostly due to the distributional difference between the monolingual segment duration of secondary language in the MSCS and TTSF-LD datasets. Moreover, to avoid segment smoothing, the smaller duration of the monolingual segment suggests the use of a small value of $N$. At the same time with small $N$, the x-vector representation is unable to capture the required language discrimination due to the acoustic similarity, as the same speaker is speaking both languages. Therefore, to resolve the issue a self-supervised implicit language representation is proposed in this study. In comparison with the x-vector representation, the proposed representation provides a relative improvement of $63.9\%$ and achieved a JER of $21.8$ using the E2E framework.
The glory won by Yanagita (Matsuoka) Kunio (1875-1962) is rarely attained by “real” humanitarian scholars, especially as he was dealing with such a narrow field of knowledge as ethnology. Yanagita was unknown to the general public before the war, but gained official recognition and nationwide fame in the post-war period. The reason for the wide recognition was that he studied and created reality. The reality was the people of Japan, mostly Okinawa residents.
Many scholars agree that judicious use of learners’ native language (L1) can be advantageous in the language (L2) classroom; however, the role of the L1 in students’ beliefs about self-efficacy has received little attention in the literature thus far. Through the use of a questionnaire, this paper examines the beliefs of university-level Japanese EFL students regarding the use of the L1 (Japanese) and its role in self-efficacy in L2 (English) learning. The major findings show that Japanese university-level EFL students believed the use of Japanese may help to improve their English reading and writing skills more than their speaking and listening skills, as well as for the learning of grammar and vocabulary in particular. Suggestions about what university EFL educators can do are also provided.
Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation. On a high level, given an input, a language model can be used to automatically complete the sequence in a statistically-likely way. Based on this, users prompt these models with language instructions or examples, to implement a variety of downstream tasks. Advanced prompting methods can even imply interaction between the language model, a user, and external tools such as calculators. However, to obtain state-of-the-art performance or adapt language models for specific tasks, complex task- and model-specific programs have to be implemented, which may still require ad-hoc interaction. Based on this, we present the novel idea of Language Model Programming (LMP). LMP generalizes language model prompting from pure text prompts to an intuitive combination of text prompting and scripting. Additionally, LMP allows constraints to be specified over the language model output. This enables easy adaption to many tasks while abstracting language model internals and providing high-level semantics. To enable LMP, we implement LMQL(short for Language Model Query Language), which leverages the constraints and control flow from an LMP prompt to generate an efficient inference procedure that minimizes the number of expensive calls to the underlying language model. We show that LMQL can capture a wide range of state-of-the-art prompting methods in an intuitive way, especially facilitating interactive flows that are challenging to implement with existing high-level APIs. Our evaluation shows that we retain or increase the accuracy on several downstream tasks, while also significantly reducing the required amount of computation or cost in the case of pay-to-use APIs (26-85% cost savings).
Afra Feyza Akyürek, Muhammed Yusuf Kocyigit, Sejin Paik
et al.
Researchers have devised numerous ways to quantify social biases vested in pretrained language models. As some language models are capable of generating coherent completions given a set of textual prompts, several prompting datasets have been proposed to measure biases between social groups -- posing language generation as a way of identifying biases. In this opinion paper, we analyze how specific choices of prompt sets, metrics, automatic tools and sampling strategies affect bias results. We find out that the practice of measuring biases through text completion is prone to yielding contradicting results under different experiment settings. We additionally provide recommendations for reporting biases in open-ended language generation for a more complete outlook of biases exhibited by a given language model. Code to reproduce the results is released under https://github.com/feyzaakyurek/bias-textgen.
The purpose of this work is to contribute toward the larger goal of creating a Quantum Natural Language Processing (QNLP) translator program. This work contributes original diagrammatic representations of the Japanese language based on prior work that accomplished on the English language based on category theory. The germane differences between the English and Japanese languages are emphasized to help address English language bias in the current body of research. Additionally, topological principles of these diagrams and many potential avenues for further research are proposed. Why is this endeavor important? Hundreds of languages have developed over the course of millennia coinciding with the evolution of human interaction across time and geographic location. These languages are foundational to human survival, experience, flourishing, and living the good life. They are also, however, the strongest barrier between people groups. Over the last several decades, advancements in Natural Language Processing (NLP) have made it easier to bridge the gap between individuals who do not share a common language or culture. Tools like Google Translate and DeepL make it easier than ever before to share our experiences with people globally. Nevertheless, these tools are still inadequate as they fail to convey our ideas across the language barrier fluently, leaving people feeling anxious and embarrassed. This is particularly true of languages born out of substantially different cultures, such as English and Japanese. Quantum computers offer the best chance to achieve translation fluency in that they are better suited to simulating the natural world and natural phenomenon such as natural speech. Keywords: category theory, DisCoCat, DisCoCirc, Japanese grammar, English grammar, translation, topology, Quantum Natural Language Processing, Natural Language Processing
Medical vision-and-language pre-training (Med-VLP) has received considerable attention owing to its applicability to extracting generic vision-and-language representations from medical images and texts. Most existing methods mainly contain three elements: uni-modal encoders (i.e., a vision encoder and a language encoder), a multi-modal fusion module, and pretext tasks, with few studies considering the importance of medical domain expert knowledge and explicitly exploiting such knowledge to facilitate Med-VLP. Although there exist knowledge-enhanced vision-and-language pre-training (VLP) methods in the general domain, most require off-the-shelf toolkits (e.g., object detectors and scene graph parsers), which are unavailable in the medical domain. In this paper, we propose a systematic and effective approach to enhance Med-VLP by structured medical knowledge from three perspectives. First, considering knowledge can be regarded as the intermediate medium between vision and language, we align the representations of the vision encoder and the language encoder through knowledge. Second, we inject knowledge into the multi-modal fusion model to enable the model to perform reasoning using knowledge as the supplementation of the input image and text. Third, we guide the model to put emphasis on the most critical information in images and texts by designing knowledge-induced pretext tasks. To perform a comprehensive evaluation and facilitate further research, we construct a medical vision-and-language benchmark including three tasks. Experimental results illustrate the effectiveness of our approach, where state-of-the-art performance is achieved on all downstream tasks. Further analyses explore the effects of different components of our approach and various settings of pre-training.
Native language identification (NLI) is the task of automatically identifying the native language (L1) of an individual based on their language production in a learned language. It is useful for a variety of purposes including marketing, security and educational applications. NLI is usually framed as a multi-label classification task, where numerous designed features are combined to achieve state-of-the-art results. Recently deep generative approach based on transformer decoders (GPT-2) outperformed its counterparts and achieved the best results on the NLI benchmark datasets. We investigate this approach to determine the practical implications compared to traditional state-of-the-art NLI systems. We introduce transformer adapters to address memory limitations and improve training/inference speed to scale NLI applications for production.
A number of morphology-based word embedding models were introduced in recent years. However, their evaluation was mostly limited to English, which is known to be a morphologically simple language. In this paper, we explore whether and to what extent incorporating morphology into word embeddings improves performance on downstream NLP tasks, in the case of morphologically rich Russian language. NLP tasks of our choice are POS tagging, Chunking, and NER -- for Russian language, all can be mostly solved using only morphology without understanding the semantics of words. Our experiments show that morphology-based embeddings trained with Skipgram objective do not outperform existing embedding model -- FastText. Moreover, a more complex, but morphology unaware model, BERT, allows to achieve significantly greater performance on the tasks that presumably require understanding of a word's morphology.