Hasil untuk "Language and Literature"

J. Mcglone

K. Bruffee

eighth or ninth on a list of ten items. Last year it appeared again, first on the list. Teachers of literature have also begun to talk about collaborative learning, although not always by that name. It is viewed as a way of engaging students more deeply with the text and also as an aspect of professors' engagement with the professional community. At its 1978 convention the Modern Language Association scheduled a multi-session forum entitled "Presence, Knowledge, and Authority in the Teaching of Literature." One of the associated sessions, called "Negotiations of Literary Knowledge," included a discussion of the authority and structure (including the collaborative classroom structure) of "interpretive communities." At the 1983 MLA convention collaborative practices in reestablishing authority and value in literary studies were examined under such rubrics as "Talking to the Academic Community: Conferences as Institutions" and "How Books 11 and 12 of Paradise Lost Got to be Valuable" (changes in interpretive attitudes in the community of Miltonists). In both these contexts collaborative learning is discussed sometimes as a process that constitutes fields or disciplines of study and sometimes as a pedagogical tool that "works" in teaching composition and literature. The former discussion, often highly theoretical, usually manages to keep at bay the more

James R. Taylor, E. J. Every

J. Alderson, D. Wall

Alberto H. F. Laender, B. Ribeiro-Neto, A. D. Silva et al.

D. Royall, E. Lauterbach, J. Cummings et al.

Keivan Seyyedi, S. M. Ismail

Dominik Meier, Jan Philip Wahle, Paul Röttger et al.

As large language models (LLMs) become integrated into sensitive workflows, concerns grow over their potential to leak confidential information. We propose TrojanStego, a novel threat model in which an adversary fine-tunes an LLM to embed sensitive context information into natural-looking outputs via linguistic steganography, without requiring explicit control over inference inputs. We introduce a taxonomy outlining risk factors for compromised LLMs, and use it to evaluate the risk profile of the threat. To implement TrojanStego, we propose a practical encoding scheme based on vocabulary partitioning learnable by LLMs via fine-tuning. Experimental results show that compromised models reliably transmit 32-bit secrets with 87% accuracy on held-out prompts, reaching over 97% accuracy using majority voting across three generations. Further, they maintain high utility, can evade human detection, and preserve coherence. These results highlight a new class of LLM data exfiltration attacks that are passive, covert, practical, and dangerous.

Maria Morant Giner

El presente trabajo forma parte de una investigación más amplia dedicada al uso y reescritura de los mitos griegos en la dramaturgia chilena de los años 90 y 2000. En este periodo se observa un notable incremento, respecto a las décadas anteriores, en la reelaboración teatral del legado griego que tiene su cúspide en el año 2006. Es precisamente en la década de los 2000 cuando se escribe y estrena la primera adaptación chilena del mito de Antígona: Antígona, (historia de objetos perdidos) (2001) de Daniela Cápona. A esta le siguen pocos años después El thriller de Antígona y Hnos. S.A. La maldición de la sangre Labdácida de Ana López Montaner y Antígona en el espejo de Juan Carlos Villavicencio. Así pues, el propósito de este trabajo será ofrecer una aproximación panorámica a estas tres obras, enmarcando su lectura y análisis dentro de la tradición propia de las Antígonas latinoamericanas.

S. Gass, Evangeline Marlos Varonis

Neelabh Sinha, Vinija Jain, Aman Chadha

The rapid rise of Language Models (LMs) has expanded their use in several applications. Yet, due to constraints of model size, associated cost, or proprietary restrictions, utilizing state-of-the-art (SOTA) LLMs is not always feasible. With open, smaller LMs emerging, more applications can leverage their capabilities, but selecting the right LM can be challenging as smaller LMs do not perform well universally. This work tries to bridge this gap by proposing a framework to experimentally evaluate small, open LMs in practical settings through measuring semantic correctness of outputs across three practical aspects: task types, application domains, and reasoning types, using diverse prompt styles. It also conducts an in-depth comparison of 10 small, open LMs to identify the best LM and prompt style depending on specific application requirements using the proposed framework. We also show that if selected appropriately, they can outperform SOTA LLMs like DeepSeek-v2, GPT-4o, GPT-4o-mini, Gemini-1.5-Pro, and even compete with GPT-4o.

Amirhossein Abaskohi, Sara Baruni, Mostafa Masoudi et al.

This paper explores the efficacy of large language models (LLMs) for Persian. While ChatGPT and consequent LLMs have shown remarkable performance in English, their efficiency for more low-resource languages remains an open question. We present the first comprehensive benchmarking study of LLMs across diverse Persian language tasks. Our primary focus is on GPT-3.5-turbo, but we also include GPT-4 and OpenChat-3.5 to provide a more holistic evaluation. Our assessment encompasses a diverse set of tasks categorized into classic, reasoning, and knowledge-based domains. To enable a thorough comparison, we evaluate LLMs against existing task-specific fine-tuned models. Given the limited availability of Persian datasets for reasoning tasks, we introduce two new benchmarks: one based on elementary school math questions and another derived from the entrance exams for 7th and 10th grades. Our findings reveal that while LLMs, especially GPT-4, excel in tasks requiring reasoning abilities and a broad understanding of general knowledge, they often lag behind smaller pre-trained models fine-tuned specifically for particular tasks. Additionally, we observe improved performance when test sets are translated to English before inputting them into GPT-3.5. These results highlight the significant potential for enhancing LLM performance in the Persian language. This is particularly noteworthy due to the unique attributes of Persian, including its distinct alphabet and writing styles.

Mohammed Khalilia, Sanad Malaysha, Reem Suwaileh et al.

This paper presents an overview of the Arabic Natural Language Understanding (ArabicNLU 2024) shared task, focusing on two subtasks: Word Sense Disambiguation (WSD) and Location Mention Disambiguation (LMD). The task aimed to evaluate the ability of automated systems to resolve word ambiguity and identify locations mentioned in Arabic text. We provided participants with novel datasets, including a sense-annotated corpus for WSD, called SALMA with approximately 34k annotated tokens, and the IDRISI-DA dataset with 3,893 annotations and 763 unique location mentions. These are challenging tasks. Out of the 38 registered teams, only three teams participated in the final evaluation phase, with the highest accuracy being 77.8% for WSD and the highest MRR@1 being 95.0% for LMD. The shared task not only facilitated the evaluation and comparison of different techniques, but also provided valuable insights and resources for the continued advancement of Arabic NLU technologies.

Craig Messner, Tom Lippincott

We present a dataset of 19th century American literary orthovariant tokens with a novel layer of human-annotated dialect group tags designed to serve as the basis for computational experiments exploring literarily meaningful orthographic variation. We perform an initial broad set of experiments over this dataset using both token (BERT) and character (CANINE)-level contextual language models. We find indications that the "dialect effect" produced by intentional orthographic variation employs multiple linguistic channels, and that these channels are able to be surfaced to varied degrees given particular language modelling assumptions. Specifically, we find evidence showing that choice of tokenization scheme meaningfully impact the type of orthographic information a model is able to surface.

Xi Cao, Dolma Dawa, Nuo Qun et al.

The textual adversarial attack refers to an attack method in which the attacker adds imperceptible perturbations to the original texts by elaborate design so that the NLP (natural language processing) model produces false judgments. This method is also used to evaluate the robustness of NLP models. Currently, most of the research in this field focuses on English, and there is also a certain amount of research on Chinese. However, to the best of our knowledge, there is little research targeting Chinese minority languages. Textual adversarial attacks are a new challenge for the information processing of Chinese minority languages. In response to this situation, we propose a Tibetan syllable-level black-box textual adversarial attack called TSAttacker based on syllable cosine distance and scoring mechanism. And then, we conduct TSAttacker on six models generated by fine-tuning two PLMs (pre-trained language models) for three downstream tasks. The experiment results show that TSAttacker is effective and generates high-quality adversarial samples. In addition, the robustness of the involved models still has much room for improvement.

Siripen Pongpaichet, Boonyapat Sukosit, Chitchaya Duangtanawat et al.

Crimes result in not only loss to individuals but also hinder national economic growth. While crime rates have been reported to decrease in developed countries, underdeveloped and developing nations still suffer from prevalent crimes, especially those undergoing rapid expansion of urbanization. The ability to monitor and assess trends of different types of crimes at both regional and national levels could assist local police and national-level policymakers in proactively devising means to prevent and address the root causes of criminal incidents. Furthermore, such a system could prove useful to individuals seeking to evaluate criminal activity for purposes of travel, investment, and relocation decisions. Recent literature has opted to utilize online news articles as a reliable and timely source for information on crime activity. However, most of the crime monitoring systems fueled by such news sources merely classified crimes into different types and visualized individual crimes on the map using extracted geolocations, lacking crucial information for stakeholders to make relevant, informed decisions. To better serve the unique needs of the target user groups, this paper proposes a novel comprehensive crime visualization system that mines relevant information from large-scale online news articles. The system features automatic crime-type classification and metadata extraction from news articles. The crime classification and metadata schemes are designed to serve the need for information from law enforcement and policymakers, as well as general users. Novel interactive spatiotemporal designs are integrated into the system with the ability to assess the severity and intensity of crimes in each region through the novel Criminometer index. The system is designed to be generalized for implementation in different countries with diverse prevalent crime types and languages composing the news articles, owing to the use of deep learning cross-lingual language models. The experiment results reveal that the proposed system yielded 86%, 51%, and 67% F1 in crime type classification, metadata extraction, and closed-form metadata extraction tasks, respectively. Additionally, the results of the system usability tests indicated a notable level of contentment among the target user groups. The findings not only offer insights into the possible applications of interactive spatiotemporal crime visualization tools for proactive policymaking and predictive policing but also serve as a foundation for future research that utilizes online news articles for intelligent monitoring of real-world phenomena.

C. Friedman, Pauline Kra, Hong Yu et al.

Claire Barale, Michael Rovatsos, Nehal Bhuta

Language Models (LMs) have proven their ability to acquire diverse linguistic knowledge during the pretraining phase, potentially serving as a valuable source of incidental supervision for downstream tasks. However, there has been limited research conducted on the retrieval of domain-specific knowledge, and specifically legal knowledge. We propose to explore the task of Entity Typing, serving as a proxy for evaluating legal knowledge as an essential aspect of text comprehension, and a foundational task to numerous downstream legal NLP applications. Through systematic evaluation and analysis and two types of prompting (cloze sentences and QA-based templates) and to clarify the nature of these acquired cues, we compare diverse types and lengths of entities both general and domain-specific entities, semantics or syntax signals, and different LM pretraining corpus (generic and legal-oriented) and architectures (encoder BERT-based and decoder-only with Llama2). We show that (1) Llama2 performs well on certain entities and exhibits potential for substantial improvement with optimized prompt templates, (2) law-oriented LMs show inconsistent performance, possibly due to variations in their training corpus, (3) LMs demonstrate the ability to type entities even in the case of multi-token entities, (4) all models struggle with entities belonging to sub-domains of the law (5) Llama2 appears to frequently overlook syntactic cues, a shortcoming less present in BERT-based architectures.

Zheng Yuan, Hongyi Yuan, Chuanqi Tan et al.

Reinforcement Learning from Human Feedback (RLHF) facilitates the alignment of large language models with human preferences, significantly enhancing the quality of interactions between humans and models. InstructGPT implements RLHF through several stages, including Supervised Fine-Tuning (SFT), reward model training, and Proximal Policy Optimization (PPO). However, PPO is sensitive to hyperparameters and requires multiple models in its standard implementation, making it hard to train and scale up to larger parameter counts. In contrast, we propose a novel learning paradigm called RRHF, which scores sampled responses from different sources via a logarithm of conditional probabilities and learns to align these probabilities with human preferences through ranking loss. RRHF can leverage sampled responses from various sources including the model responses from itself, other large language model responses, and human expert responses to learn to rank them. RRHF only needs 1 to 2 models during tuning and can efficiently align language models with human preferences robustly without complex hyperparameter tuning. Additionally, RRHF can be considered an extension of SFT and reward model training while being simpler than PPO in terms of coding, model counts, and hyperparameters. We evaluate RRHF on the Helpful and Harmless dataset, demonstrating comparable alignment performance with PPO by reward model score and human labeling. Extensive experiments show that the performance of RRHF is highly related to sampling quality which suggests RRHF is a best-of-n learner. Codes available at https://github.com/GanjinZero/RRHF.

Jimmy Wu, Rika Antonova, Adam Kan et al.

For a robot to personalize physical assistance effectively, it must learn user preferences that can be generally reapplied to future scenarios. In this work, we investigate personalization of household cleanup with robots that can tidy up rooms by picking up objects and putting them away. A key challenge is determining the proper place to put each object, as people's preferences can vary greatly depending on personal taste or cultural background. For instance, one person may prefer storing shirts in the drawer, while another may prefer them on the shelf. We aim to build systems that can learn such preferences from just a handful of examples via prior interactions with a particular person. We show that robots can combine language-based planning and perception with the few-shot summarization capabilities of large language models (LLMs) to infer generalized user preferences that are broadly applicable to future interactions. This approach enables fast adaptation and achieves 91.2% accuracy on unseen objects in our benchmark dataset. We also demonstrate our approach on a real-world mobile manipulator called TidyBot, which successfully puts away 85.0% of objects in real-world test scenarios.