Abstract Background Acute respiratory distress syndrome (ARDS) and acute lung injury (ALI) are life-threatening conditions with limited effective pharmacological interventions. Sivelestat sodium, a selective neutrophil elastase inhibitor, has been extensively investigated in ARDS/ALI treatment due to its significant anti-inflammatory properties; however, its therapeutic efficacy remains controversial. As a drug developed and first approved in Japan, most previous meta-analyses have failed to incorporate Japanese literature, potentially introducing substantial language bias. Additionally, Japanese clinical practices may employ different treatment protocols that could offer novel perspectives on sivelestat sodium’s application. This systematic review aims to comprehensively evaluate sivelestat sodium’s efficacy in ARDS/ALI patients by simultaneously including both English and Japanese clinical literature. Methods We will systematically search English databases (Cochrane Library, EMBASE, PubMed) and Japanese databases (Ichushi Web, J-STAGE) for randomized controlled trials comparing sivelestat sodium with placebo or standard therapy in adult ARDS/ALI patients. Two independent reviewers will screen studies, extract data, and assess risk of bias. Primary outcomes include duration of mechanical ventilation and all-cause mortality (28–30 days, ICU, and in-hospital). If sufficient eligible studies are identified, a random-effects model will be employed for meta-analysis. Between-study heterogeneity will be assessed using the I 2 statistic, and the certainty of evidence will be evaluated using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) framework. Discussion This protocol outlines a systematic approach for evaluating sivelestat sodium’s efficacy in ARDS/ALI through the integration of English and Japanese literature. Methodological rigor will ensure high-quality evidence synthesis despite challenges in integrating diverse studies. This research addresses a significant gap, as meta-analyses incorporating Japanese publications have been absent for over a decade. The findings will provide evidence-based guidance for clinical practice, inform individualized treatment strategies, optimize sivelestat administration protocols, and identify directions for future research. Systematic review registration PROSPERO CRD420251067146
Natural disasters, such as earthquakes and tsunamis, significantly impact human existence, particularly in coastal and island environments like Indonesia and Japan. This study examines the roles of oral traditions in these two countries as cultural knowledge that raises disaster awareness and encourages environmental harmony. It focuses on two oral traditions: the Smong song from Simeulue Island in Aceh, Indonesia, and the kamuy yukara “Nokkurunka” song as represented in Tsushima sYuko’s Jakka Dofuni Umi no Kioku no Monogatari, obtained from the oral traditions of the Ainu people in Japan. Using a collective memory framework and comparative literature analysis, this qualitative research explores how oral traditions in these countries share ecological wisdom related to natural disasters and how they contribute to disaster awareness and cultural resilience. The findings reveal that the Smong functions as a collective memory of cultural knowledge passed down across generations, providing practical guidance to save lives during major earthquakes and tsunamis. Meanwhile, the kamuy yukara “Nokkurunka” serves as a medium to preserve memories of tsunami experiences and as a spiritual narrative on the relationship between humans and kamuy (gods). Thus, both traditions convey local knowledge, functioning as an early warning system for disasters.
This book provides a broad overview of Natural Language Generation (NLG), including technology, user requirements, evaluation, and real-world applications. The focus is on concepts and insights which hopefully will remain relevant for many years, not on the latest LLM innovations. It draws on decades of work by the author and others on NLG. The book has the following chapters: Introduction to NLG; Rule-Based NLG; Machine Learning and Neural NLG; Requirements; Evaluation; Safety, Maintenance, and Testing; and Applications. All chapters include examples and anecdotes from the author's personal experiences, and end with a Further Reading section. The book should be especially useful to people working on applied NLG, including NLG researchers, people in other fields who want to use NLG, and commercial developers. It will not however be useful to people who want to understand the latest LLM technology. There is a companion site with more information at https://ehudreiter.com/book/
Samuel Rothfarb, Megan C. Davis, Ivana Matanovic
et al.
Artificial intelligence is reshaping scientific exploration, but most methods automate procedural tasks without engaging in scientific reasoning, limiting autonomy in discovery. We introduce Materials Agents for Simulation and Theory in Electronic-structure Reasoning (MASTER), an active learning framework where large language models autonomously design, execute, and interpret atomistic simulations. In MASTER, a multimodal system translates natural language into density functional theory workflows, while higher-level reasoning agents guide discovery through a hierarchy of strategies, including a single agent baseline and three multi-agent approaches: peer review, triage-ranking, and triage-forms. Across two chemical applications, CO adsorption on Cu-surface transition metal (M) adatoms and on M-N-C catalysts, reasoning-driven exploration reduces required atomistic simulations by up to 90% relative to trial-and-error selection. Reasoning trajectories reveal chemically grounded decisions that cannot be explained by stochastic sampling or semantic bias. Altogether, multi-agent collaboration accelerates materials discovery and marks a new paradigm for autonomous scientific exploration.
Pre-trained language models (PLMs) are increasingly being applied to code-related tasks. Although PLMs have achieved good results, they do not take into account potential high-order data correlations within the code. We propose three types of high-order correlations in code tokens, i.e. abstract syntax tree family correlation, lexical correlation, and line correlation. We design a tokens and hyperedges generator to capture these high-order data correlations. We improve the architecture of hypergraph neural networks and combine it with adapter tuning to propose a novel hypergraph-based adapter (HGAdapter) to fine-tune PLMs. HGAdapter can encode high-order data correlations and is allowed to be inserted into various PLMs to enhance performance. Experiments were conducted on several public datasets, including six languages of code summarization and code clone detection tasks. Our methods improved the performance of PLMs in datasets to varying degrees. Experimental results validate the introduction of high-order data correlations that contribute to improved effectiveness.
Dominik Meier, Jan Philip Wahle, Paul Röttger
et al.
As large language models (LLMs) become integrated into sensitive workflows, concerns grow over their potential to leak confidential information. We propose TrojanStego, a novel threat model in which an adversary fine-tunes an LLM to embed sensitive context information into natural-looking outputs via linguistic steganography, without requiring explicit control over inference inputs. We introduce a taxonomy outlining risk factors for compromised LLMs, and use it to evaluate the risk profile of the threat. To implement TrojanStego, we propose a practical encoding scheme based on vocabulary partitioning learnable by LLMs via fine-tuning. Experimental results show that compromised models reliably transmit 32-bit secrets with 87% accuracy on held-out prompts, reaching over 97% accuracy using majority voting across three generations. Further, they maintain high utility, can evade human detection, and preserve coherence. These results highlight a new class of LLM data exfiltration attacks that are passive, covert, practical, and dangerous.
Soka Gakkai is one of the largest neo-religious organizations in Japan and is based on the ideology of Nichirenism. It was created in 1930 but became widespread and gained popularity after the Second World War. Today, it has millions of members both in Japan and abroad, it also has educational institutions from kindergartens to universities, its own media outlets, a representative office at the UN, and significant influence on the country’s domestic politics. In its modern form, the organization owes much to the efforts of Ikeda Daisaku, the third president and spiritual leader, as well as the founder of the political party Komeito. He had a colossal influence on the development of Soka Gakkai, creating a powerful bureaucratic structure in it, and brought its activities to the international arena. Gradually, D. Ikeda moved away from formal management of affairs, but, as the honorary president of the organization, he continued to direct the activities and ideology of the society. At the same time, many processes in the organization turned out to be focused on exploiting the image of D. Ikeda, who occupied a central place in all religious and secular affairs. In the official publications of Soka Gakkai, D. Ikeda appears as a writer, pacifist, intellectual, and educator. Numerous diplomatic, educational, and peacekeeping activities, the opening of schools, museums, and universities, and the writing and publication of a multi-volume collection of works are attributed to his personal initiative. The rise of D. Ikeda can be explained by the orientation of Nichirenism toward the continuity of teachings through charismatic leadership. After his death in 2023, there was no other authoritative leader in Soka Gakkai, and there was a threat of destabilization and disintegration of the organization and the loss of its political influence. At the same time, the refusal to transfer charismatic leadership to a successor, apparently, was a conscious decision made by the bureaucratic apparatus of the organization. The article examines the evolution of the personality and image of Ikeda Daisaku and his role in the activities of the Soka Gakkai at the present stage.
Large Language Models (LLMs) are increasingly used in production systems, powering applications such as chatbots, summarization, and question answering. Despite their success, controlling the length of their response remains a significant challenge, particularly for tasks requiring structured outputs or specific levels of detail. In this work, we propose a method to adapt pre-trained decoder-only LLMs for precise control of response length. Our approach incorporates a secondary length-difference positional encoding (LDPE) into the input embeddings, which counts down to a user-set response termination length. Fine-tuning with LDPE allows the model to learn to terminate responses coherently at the desired length, achieving mean token errors of less than 3 tokens. We also introduce Max New Tokens++, an extension that enables flexible upper-bound length control, rather than an exact target. Experimental results on tasks such as question answering and document summarization demonstrate that our method enables precise length control without compromising response quality.
This paper delves into the text processing aspects of Language Computing, which enables computers to understand, interpret, and generate human language. Focusing on tasks such as speech recognition, machine translation, sentiment analysis, text summarization, and language modelling, language computing integrates disciplines including linguistics, computer science, and cognitive psychology to create meaningful human-computer interactions. Recent advancements in deep learning have made computers more accessible and capable of independent learning and adaptation. In examining the landscape of language computing, the paper emphasises foundational work like encoding, where Tamil transitioned from ASCII to Unicode, enhancing digital communication. It discusses the development of computational resources, including raw data, dictionaries, glossaries, annotated data, and computational grammars, necessary for effective language processing. The challenges of linguistic annotation, the creation of treebanks, and the training of large language models are also covered, emphasising the need for high-quality, annotated data and advanced language models. The paper underscores the importance of building practical applications for languages like Tamil to address everyday communication needs, highlighting gaps in current technology. It calls for increased research collaboration, digitization of historical texts, and fostering digital usage to ensure the comprehensive development of Tamil language processing, ultimately enhancing global communication and access to digital services.
The rapid rise of Language Models (LMs) has expanded their use in several applications. Yet, due to constraints of model size, associated cost, or proprietary restrictions, utilizing state-of-the-art (SOTA) LLMs is not always feasible. With open, smaller LMs emerging, more applications can leverage their capabilities, but selecting the right LM can be challenging as smaller LMs do not perform well universally. This work tries to bridge this gap by proposing a framework to experimentally evaluate small, open LMs in practical settings through measuring semantic correctness of outputs across three practical aspects: task types, application domains, and reasoning types, using diverse prompt styles. It also conducts an in-depth comparison of 10 small, open LMs to identify the best LM and prompt style depending on specific application requirements using the proposed framework. We also show that if selected appropriately, they can outperform SOTA LLMs like DeepSeek-v2, GPT-4o, GPT-4o-mini, Gemini-1.5-Pro, and even compete with GPT-4o.
Amirhossein Abaskohi, Sara Baruni, Mostafa Masoudi
et al.
This paper explores the efficacy of large language models (LLMs) for Persian. While ChatGPT and consequent LLMs have shown remarkable performance in English, their efficiency for more low-resource languages remains an open question. We present the first comprehensive benchmarking study of LLMs across diverse Persian language tasks. Our primary focus is on GPT-3.5-turbo, but we also include GPT-4 and OpenChat-3.5 to provide a more holistic evaluation. Our assessment encompasses a diverse set of tasks categorized into classic, reasoning, and knowledge-based domains. To enable a thorough comparison, we evaluate LLMs against existing task-specific fine-tuned models. Given the limited availability of Persian datasets for reasoning tasks, we introduce two new benchmarks: one based on elementary school math questions and another derived from the entrance exams for 7th and 10th grades. Our findings reveal that while LLMs, especially GPT-4, excel in tasks requiring reasoning abilities and a broad understanding of general knowledge, they often lag behind smaller pre-trained models fine-tuned specifically for particular tasks. Additionally, we observe improved performance when test sets are translated to English before inputting them into GPT-3.5. These results highlight the significant potential for enhancing LLM performance in the Persian language. This is particularly noteworthy due to the unique attributes of Persian, including its distinct alphabet and writing styles.
Mohammed Khalilia, Sanad Malaysha, Reem Suwaileh
et al.
This paper presents an overview of the Arabic Natural Language Understanding (ArabicNLU 2024) shared task, focusing on two subtasks: Word Sense Disambiguation (WSD) and Location Mention Disambiguation (LMD). The task aimed to evaluate the ability of automated systems to resolve word ambiguity and identify locations mentioned in Arabic text. We provided participants with novel datasets, including a sense-annotated corpus for WSD, called SALMA with approximately 34k annotated tokens, and the IDRISI-DA dataset with 3,893 annotations and 763 unique location mentions. These are challenging tasks. Out of the 38 registered teams, only three teams participated in the final evaluation phase, with the highest accuracy being 77.8% for WSD and the highest MRR@1 being 95.0% for LMD. The shared task not only facilitated the evaluation and comparison of different techniques, but also provided valuable insights and resources for the continued advancement of Arabic NLU technologies.
This paper aims to investigate the motivations of Indonesian students to pursue tertiary education in Japan. Investigating the experiences and perspectives of five Indonesian students, collected through in-dept interview, pursuing a degree program in Japan, this study argued that academic factors alone proved insufficient in capturing a comprehensive picture of students' motivation. The findings revealed that while academic motivation remained as an important driving force, socio-cultural factors contributed heavily to maintaining the students’ interest towards Japan. Interestingly, the findings also revealed that the students’ consideration to continue study in Japan was heavily influenced by routine and seemingly mundane matters namely, day-to-day living experience, such as the tolerance exhibited by Japanese society towards religious practices, the geographical proximity, and safety concerns, rather than being primarily driven by academic ambitions. Thus, to gain comprehensive students’ motivations in pursuing higher education abroad, it is necessary to contextualize it within a broader socio-cultural background.
Pedro Colon-Hernandez, Henry Lieberman, Yida Xin
et al.
Contextualized or discourse aware commonsense inference is the task of generating coherent commonsense assertions (i.e., facts) from a given story, and a particular sentence from that story. Some problems with the task are: lack of controllability for topics of the inferred facts; lack of commonsense knowledge during training; and, possibly, hallucinated or false facts. In this work, we utilize a transformer model for this task and develop techniques to address the aforementioned problems in the task. We control the inference by introducing a new technique we call "hinting". Hinting is a kind of language model prompting, that utilizes both hard prompts (specific words) and soft prompts (virtual learnable templates). This serves as a control signal to advise the language model "what to talk about". Next, we establish a methodology for performing joint inference with multiple commonsense knowledge bases. Joint inference of commonsense requires care, because it is imprecise and the level of generality is more flexible. You want to be sure that the results "still make sense" for the context. To this end, we align the textual version of assertions from three knowledge graphs (ConceptNet, ATOMIC2020, and GLUCOSE) with a story and a target sentence. This combination allows us to train a single model to perform joint inference with multiple knowledge graphs. We show experimental results for the three knowledge graphs on joint inference. Our final contribution is exploring a GAN architecture that generates the contextualized commonsense assertions and scores them as to their plausibility through a discriminator. The result is an integrated system for contextual commonsense inference in stories, that can controllably generate plausible commonsense assertions, and takes advantage of joint inference between multiple commonsense knowledge bases.
Language Models (LMs) have proven their ability to acquire diverse linguistic knowledge during the pretraining phase, potentially serving as a valuable source of incidental supervision for downstream tasks. However, there has been limited research conducted on the retrieval of domain-specific knowledge, and specifically legal knowledge. We propose to explore the task of Entity Typing, serving as a proxy for evaluating legal knowledge as an essential aspect of text comprehension, and a foundational task to numerous downstream legal NLP applications. Through systematic evaluation and analysis and two types of prompting (cloze sentences and QA-based templates) and to clarify the nature of these acquired cues, we compare diverse types and lengths of entities both general and domain-specific entities, semantics or syntax signals, and different LM pretraining corpus (generic and legal-oriented) and architectures (encoder BERT-based and decoder-only with Llama2). We show that (1) Llama2 performs well on certain entities and exhibits potential for substantial improvement with optimized prompt templates, (2) law-oriented LMs show inconsistent performance, possibly due to variations in their training corpus, (3) LMs demonstrate the ability to type entities even in the case of multi-token entities, (4) all models struggle with entities belonging to sub-domains of the law (5) Llama2 appears to frequently overlook syntactic cues, a shortcoming less present in BERT-based architectures.
For a robot to personalize physical assistance effectively, it must learn user preferences that can be generally reapplied to future scenarios. In this work, we investigate personalization of household cleanup with robots that can tidy up rooms by picking up objects and putting them away. A key challenge is determining the proper place to put each object, as people's preferences can vary greatly depending on personal taste or cultural background. For instance, one person may prefer storing shirts in the drawer, while another may prefer them on the shelf. We aim to build systems that can learn such preferences from just a handful of examples via prior interactions with a particular person. We show that robots can combine language-based planning and perception with the few-shot summarization capabilities of large language models (LLMs) to infer generalized user preferences that are broadly applicable to future interactions. This approach enables fast adaptation and achieves 91.2% accuracy on unseen objects in our benchmark dataset. We also demonstrate our approach on a real-world mobile manipulator called TidyBot, which successfully puts away 85.0% of objects in real-world test scenarios.
Reinforcement Learning from Human Feedback (RLHF) facilitates the alignment of large language models with human preferences, significantly enhancing the quality of interactions between humans and models. InstructGPT implements RLHF through several stages, including Supervised Fine-Tuning (SFT), reward model training, and Proximal Policy Optimization (PPO). However, PPO is sensitive to hyperparameters and requires multiple models in its standard implementation, making it hard to train and scale up to larger parameter counts. In contrast, we propose a novel learning paradigm called RRHF, which scores sampled responses from different sources via a logarithm of conditional probabilities and learns to align these probabilities with human preferences through ranking loss. RRHF can leverage sampled responses from various sources including the model responses from itself, other large language model responses, and human expert responses to learn to rank them. RRHF only needs 1 to 2 models during tuning and can efficiently align language models with human preferences robustly without complex hyperparameter tuning. Additionally, RRHF can be considered an extension of SFT and reward model training while being simpler than PPO in terms of coding, model counts, and hyperparameters. We evaluate RRHF on the Helpful and Harmless dataset, demonstrating comparable alignment performance with PPO by reward model score and human labeling. Extensive experiments show that the performance of RRHF is highly related to sampling quality which suggests RRHF is a best-of-n learner. Codes available at https://github.com/GanjinZero/RRHF.
Much research in the functional linguistics literature suggests that the use of zero pronouns is driven by the degree of interclausal connection. Kim (1990, 1992) claims that in clause chain languages such as Korean and Japanese, zero pronouns are primarily used following an interclausal connective with a tight interclausal connection that maintains subject continuity (whether the subject referent is maintained or changed in the following clause) and action continuity (temporal sequence or action sequence is continued or interrupted in the following clause). Consistent with the claim, the results of Experiments 1 and 2 demonstrated that Korean speakers used more zero pronouns following a connective with higher subject continuity (Experiment 1) and action continuity (Experiments 1 and 2). The effect of connectives was observed when grammatical role, referential predictability, and coherence relation were controlled for (Experiment 2). These results are best explained by assuming the role of discourse continuity on referential form choice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).