Hasil untuk "Language and Literature"

Qianwen Zhao, Meng Meng, Rahul Kumar et al.

Comorbidities are associated with the severity of coronavirus disease 2019 (COVID‐19). This meta‐analysis aimed to explore the risk of severe COVID‐19 in patients with pre‐existing chronic obstructive pulmonary disease (COPD) and ongoing smoking history. A comprehensive systematic literature search was carried out to find studies published from December 2019 to 22 March 2020 from five databases. The languages of literature included English and Chinese. The point prevalence of severe COVID‐19 in patients with pre‐existing COPD and those with ongoing smoking was evaluated with this meta‐analysis. Overall 11 case series, published either in Chinese or English language with a total of 2002 cases, were included in this study. The pooled OR of COPD and the development of severe COVID‐19 was 4.38 (fixed‐effects model; 95% CI: 2.34‐8.20), while the OR of ongoing smoking was 1.98 (fixed‐effects model; 95% CI: 1.29‐3.05). There was no publication bias as examined by the funnel plot and Egger's test (P = not significant). The heterogeneity of included studies was moderate for both COPD and ongoing smoking history on the severity of COVID‐19. COPD and ongoing smoking history attribute to the worse progression and outcome of COVID‐19.

B. Powell, J. McMillen, E. Proctor et al.

Antoine Bordes, S. Chopra, J. Weston

This paper presents a system which learns to answer questions on a broad range of topics from a knowledge base using few hand-crafted features. Our model learns low-dimensional embeddings of words and knowledge base constituents; these representations are used to score natural language questions against candidate answers. Training our system using pairs of questions and structured representations of their answers, and pairs of question paraphrases, yields competitive results on a recent benchmark of the literature.

S. D. Pietra, V. D. Pietra, J. Lafferty

We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the field and an iterative scaling algorithm is used to estimate the optimal values of the weights. The random field models and techniques introduced in this paper differ from those common to much of the computer vision literature in that the underlying random fields are non-Markovian and have a large number of parameters that must be estimated. Relations to other learning approaches, including decision trees, are given. As a demonstration of the method, we describe its application to the problem of automatic word classification in natural language processing.

K. Cicerone, Cynthia Dahlberg, K. Kalmar et al.

P. MacIntyre, R. Gardner

M. Paradis

Yan Xu, Lili Mou, Ge Li et al.

Relation classification is an important research arena in the field of natural language processing (NLP). In this paper, we present SDP-LSTM, a novel neural network to classify the relation of two entities in a sentence. Our neural architecture leverages the shortest dependency path (SDP) between two entities; multichannel recurrent neural networks, with long short term memory (LSTM) units, pick up heterogeneous information along the SDP. Our proposed model has several distinct features: (1) The shortest dependency paths retain most relevant information (to relation classification), while eliminating irrelevant words in the sentence. (2) The multichannel LSTM networks allow effective information integration from heterogeneous sources over the dependency paths. (3) A customized dropout strategy regularizes the neural network to alleviate overfitting. We test our model on the SemEval 2010 relation classification task, and achieve an $F_1$-score of 83.7\%, higher than competing methods in the literature.

Sebastian Ruder, Ivan Vulic, Anders Søgaard

Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages. In this survey, we provide a comprehensive typology of cross-lingual word embedding models. We compare their data requirements and objective functions. The recurring theme of the survey is that many of the models presented in the literature optimize for the same objectives, and that seemingly different models are often equivalent, modulo optimization strategies, hyper-parameters, and such. We also discuss the different ways cross-lingual word embeddings are evaluated, as well as future challenges and research horizons.

Craig Dicker, A. U. Chamot, J. M. O'malley

Joana Breidenbach

Colonized Australia presents a very grim picture of Aboriginal people in general. They are in a minority in their own land and many Aboriginal trib es and languages in fact, became extinct. Land, sacred to Aboriginals, has been lost to the c olonizers. It is only the Aboriginal consciousness that keeps them united in this predic ament in spite of their differences in language, culture, colour, region and religion. It is also this unity that makes them fight, at times with a martial spirit, against discrimination and m otivates them to assert their Aboriginal identity. In the past few decades Australia has pro duced a considerable amount of Aboriginal Literature reflecting Aboriginal struggle economic freedom, legal recognition and reforms for basic living conditions. Mudrooroo, Jack Davis, Ale xis Wright, Kim Scott, and other Aboriginal Writers represent these issues through different li terary genres of poetry, fiction and drama. In this paper I try to explain how Aboriginal form can be identified and how functions of Aboriginality can be recognized in the writings of Mudrooroo and Jack Davis writings that belong to the early phase of Aboriginal period.

Alison Mackey, Jenefer Philp

Marie-José Guilloteaux, Z. Dörnyei

R. Rogers, Elizabeth Malancharuvil-Berkes, M. Mosley et al.

C. Snow, P. Uccelli

Maria Cristina Fumagalli

En 1957 se le encargó al relativamente joven Derek Walcott (tenía entonces 27 años) una obra teatral destinada a la celebración de la Apertura del Primer Parlamento Federal de las Antillas Inglesas el 23 de abril de 1958. El resultado fue Drums and Colours, una pieza que pone en escena cuatrocientos años de la compleja y turbulenta historia del Caribe, basada, como el propio Walcott declarara, en “cuatro imágenes emblemáticas: Colón en cadenas; la pintura de Millais The Boyhood of Raleigh [La niñez de Raleigh]; el cochero de la familia Breda, Toussaint L’Ouverture, y el martirio de George William Gordon por la independencia de Jamaica”. Al llamar la atención sobre estas “imágenes emblemáticas”, Walcott revela que Drums and Colours se originó en una matriz visual, invita a considerar la obra como una extendida interpretación verbal de estímulos visuales y subraya la importancia de la interdisciplinariedad para su agenda decolonial. Al analizar estas imágenes de Walcott, argumentaré que la obra del muralista mexicano Diego Rivera es otra importante, aunque no reconocida, fuente visual de Drums and Colours, que pone de manifiesto el modo en que las ideas y la obra del joven Walcott estaban profundamente impregnadas de su sentido de fidelidad con la vasta región del mar Caribe y con la agenda emancipatoria de su “pequeña gente”.

Neelabh Sinha

Language Models (LMs) have revolutionized natural language processing, enabling high-quality text generation through prompting and in-context learning. However, models often struggle with long-context summarization due to positional biases, leading to suboptimal extraction of critical information. There are techniques to improve this with fine-tuning, pipelining, or using complex techniques, which have their own challenges. To solve these challenges, we propose QA-prompting - a simple prompting method for summarization that utilizes question-answering as an intermediate step prior to summary generation. Our method extracts key information and enriches the context of text to mitigate positional biases and improve summarization in a single LM call per task without requiring fine-tuning or pipelining. Experiments on multiple datasets belonging to different domains using ten state-of-the-art pre-trained models demonstrate that QA-prompting outperforms baseline and other state-of-the-art methods, achieving up to 29% improvement in ROUGE scores. This provides an effective and scalable solution for summarization and highlights the importance of domain-specific question selection for optimal performance.

Keliang Liu, Dingkang Yang, Ziyun Qian et al.

In recent years, training methods centered on Reinforcement Learning (RL) have markedly enhanced the reasoning and alignment performance of Large Language Models (LLMs), particularly in understanding human intents, following user instructions, and bolstering inferential strength. Although existing surveys offer overviews of RL augmented LLMs, their scope is often limited, failing to provide a comprehensive summary of how RL operates across the full lifecycle of LLMs. We systematically review the theoretical and practical advancements whereby RL empowers LLMs, especially Reinforcement Learning with Verifiable Rewards (RLVR). First, we briefly introduce the basic theory of RL. Second, we thoroughly detail application strategies for RL across various phases of the LLM lifecycle, including pre-training, alignment fine-tuning, and reinforced reasoning. In particular, we emphasize that RL methods in the reinforced reasoning phase serve as a pivotal driving force for advancing model reasoning to its limits. Next, we collate existing datasets and evaluation benchmarks currently used for RL fine-tuning, spanning human-annotated datasets, AI-assisted preference data, and program-verification-style corpora. Subsequently, we review the mainstream open-source tools and training frameworks available, providing clear practical references for subsequent research. Finally, we analyse the future challenges and trends in the field of RL-enhanced LLMs. This survey aims to present researchers and practitioners with the latest developments and frontier trends at the intersection of RL and LLMs, with the goal of fostering the evolution of LLMs that are more intelligent, generalizable, and secure.

Xudong Lu, Qi Liu, Yuhui Xu et al.

A pivotal advancement in the progress of large language models (LLMs) is the emergence of the Mixture-of-Experts (MoE) LLMs. Compared to traditional LLMs, MoE LLMs can achieve higher performance with fewer parameters, but it is still hard to deploy them due to their immense parameter sizes. Different from previous weight pruning methods that rely on specifically designed hardware, this paper mainly aims to enhance the deployment efficiency of MoE LLMs by introducing plug-and-play expert-level sparsification techniques. Specifically, we propose, for the first time to our best knowledge, post-training approaches for task-agnostic and task-specific expert pruning and skipping of MoE LLMs, tailored to improve deployment efficiency while maintaining model performance across a wide range of tasks. Extensive experiments show that our proposed methods can simultaneously reduce model sizes and increase the inference speed, while maintaining satisfactory performance. Data and code will be available at https://github.com/Lucky-Lance/Expert_Sparsity.

Gihun Lee, Minchan Jeong, Yujin Kim et al.

While learning to align Large Language Models (LLMs) with human preferences has shown remarkable success, aligning these models to meet the diverse user preferences presents further challenges in preserving previous knowledge. This paper examines the impact of personalized preference optimization on LLMs, revealing that the extent of knowledge loss varies significantly with preference heterogeneity. Although previous approaches have utilized the KL constraint between the reference model and the policy model, we observe that they fail to maintain general knowledge and alignment when facing personalized preferences. To this end, we introduce Base-Anchored Preference Optimization (BAPO), a simple yet effective approach that utilizes the initial responses of reference model to mitigate forgetting while accommodating personalized alignment. BAPO effectively adapts to diverse user preferences while minimally affecting global knowledge or general alignment. Our experiments demonstrate the efficacy of BAPO in various setups.