Hasil untuk "Language and Literature"

Shijie Wu, Ozan Irsoy, Steven Lu et al.

The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology. We release Training Chronicles (Appendix C) detailing our experience in training BloombergGPT.

Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow et al.

Abstract Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this article, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely, metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.

Wonjae Kim, Bokyung Son, Ildoo Kim

Vision-and-Language Pre-training (VLP) has improved performance on various joint vision-and-language downstream tasks. Current approaches to VLP heavily rely on image feature extraction processes, most of which involve region supervision (e.g., object detection) and the convolutional architecture (e.g., ResNet). Although disregarded in the literature, we find it problematic in terms of both (1) efficiency/speed, that simply extracting input features requires much more computation than the multimodal interaction steps; and (2) expressive power, as it is upper bounded to the expressive power of the visual embedder and its predefined visual vocabulary. In this paper, we present a minimal VLP model, Vision-and-Language Transformer (ViLT), monolithic in the sense that the processing of visual inputs is drastically simplified to just the same convolution-free manner that we process textual inputs. We show that ViLT is up to tens of times faster than previous VLP models, yet with competitive or better downstream task performance. Our code and pre-trained weights are available at https://github.com/dandelin/vilt.

Juyong Jiang, Fan Wang, Jiasi Shen et al.

Large Language Models (LLMs) have garnered remarkable advancements across diverse code-related tasks, known as Code LLMs, particularly in code generation that generates source code with LLM from natural language descriptions. This burgeoning field has captured significant interest from both academic researchers and industry professionals due to its practical significance in software development, e.g., GitHub Copilot. Despite the active exploration of LLMs for a variety of code tasks, either from the perspective of Natural Language Processing (NLP) or Software Engineering (SE) or both, there is a noticeable absence of a comprehensive and up-to-date literature review dedicated to LLM for code generation. In this survey, we aim to bridge this gap by providing a systematic literature review that serves as a valuable reference for researchers investigating the cutting-edge progress in LLMs for code generation. We introduce a taxonomy to categorize and discuss the recent developments in LLMs for code generation, covering aspects such as data curation, latest advances, performance evaluation, ethical implications, environmental impact, and real-world applications. In addition, we present a historical overview of the evolution of LLMs for code generation and provide a quantitative and qualitative comparative analysis of experimental results of code LLMs, sourced from their original papers to ensure a fair comparison on the HumanEval, MBPP, and BigCodeBench benchmarks, across various levels of difficulty and types of programming tasks, to highlight the progressive enhancements in LLM capabilities for code generation. We identify critical challenges and promising opportunities regarding the gap between academia and practical development. Furthermore, we have established a dedicated resource GitHub page (https://github.com/juyongjiang/CodeLLMSurvey) to continuously document and disseminate the most recent advances in the field.

J. Maurício, Inês Domingues, Jorge Bernardino

Transformers are models that implement a mechanism of self-attention, individually weighting the importance of each part of the input data. Their use in image classification tasks is still somewhat limited since researchers have so far chosen Convolutional Neural Networks for image classification and transformers were more targeted to Natural Language Processing (NLP) tasks. Therefore, this paper presents a literature review that shows the differences between Vision Transformers (ViT) and Convolutional Neural Networks. The state of the art that used the two architectures for image classification was reviewed and an attempt was made to understand what factors may influence the performance of the two deep learning architectures based on the datasets used, image size, number of target classes (for the classification problems), hardware, and evaluated architectures and top results. The objective of this work is to identify which of the architectures is the best for image classification and under what conditions. This paper also describes the importance of the Multi-Head Attention mechanism for improving the performance of ViT in image classification.

Shidur Rahman

The researchers of various disciplines often use qualitative and quantitative research methods and approaches for their studies. Some of these researchers like to be known as qualitative researchers; others like to be regarded as quantitative researchers. The researchers, thus, are sharply polarised; and they involve in a competition of pointing out the benefits of their own preferred methods and approaches. But, both the methods and approaches (qualitative and quantitative) have pros and cons. This study, therefore, aims to discuss the advantages and disadvantages of using qualitative and quantitative research approaches and methods in language testing and assessment research. There is a focus on ethical considerations too. The study found some strengths of using qualitative methods for language “assessment and testing” research—such as, eliciting deeper insights into designing, administering, and interpreting assessment and testing; and exploring test-takers’ behaviour, perceptions, feelings, and understanding. Some weaknesses are, for instance, smaller sample size and time consuming. Quantitative research methods, on the other hand, involve a larger sample, and do not require relatively a longer time for data collection. Some limitations are that quantitative research methods take snapshots of a phenomenon: not in-depth, and overlook test-takers’ and testers’ experiences as well as what they mean by something. Among these two research paradigms, the quantitative one is dominant in the context of language testing and assessment research.

Weilin Cai, Juyong Jiang, Fan Wang et al.

Large language models (LLMs) have garnered unprecedented advancements across diverse fields, ranging from natural language processing to computer vision and beyond. The prowess of LLMs is underpinned by their substantial model size, extensive and diverse datasets, and the vast computational power harnessed during training, all of which contribute to the emergent abilities of LLMs (e.g., in-context learning) that are not present in small models. Within this context, the mixture of experts (MoE) has emerged as an effective method for substantially scaling up model capacity with minimal computation overhead, gaining significant attention from academia and industry. Despite its growing prevalence, there lacks a systematic and comprehensive review of the literature on MoE. This survey seeks to bridge that gap, serving as an essential resource for researchers delving into the intricacies of MoE. We first briefly introduce the structure of the MoE layer, followed by proposing a new taxonomy of MoE. Next, we overview the core designs for various MoE models including both algorithmic and systemic aspects, alongside collections of available open-source implementations, hyperparameter configurations and empirical evaluations. Furthermore, we delineate the multifaceted applications of MoE in practice, and outline some potential directions for future research.

Changle Qu, Sunhao Dai, Xiaochi Wei et al.

Recently, tool learning with large language models (LLMs) has emerged as a promising paradigm for augmenting the capabilities of LLMs to tackle highly complex problems. Despite growing attention and rapid advancements in this field, the existing literature remains fragmented and lacks systematic organization, posing barriers to entry for newcomers. This gap motivates us to conduct a comprehensive survey of existing works on tool learning with LLMs. In this survey, we focus on reviewing existing literature from the two primary aspects (1) why tool learning is beneficial and (2) how tool learning is implemented, enabling a comprehensive understanding of tool learning with LLMs. We first explore the “why” by reviewing both the benefits of tool integration and the inherent benefits of the tool learning paradigm from six specific aspects. In terms of “how”, we systematically review the literature according to a taxonomy of four key stages in the tool learning workflow: task planning, tool selection, tool calling, and response generation. Additionally, we provide a detailed summary of existing benchmarks and evaluation methods, categorizing them according to their relevance to different stages. Finally, we discuss current challenges and outline potential future directions, aiming to inspire both researchers and industrial developers to further explore this emerging and promising area.

Ksenia Chmutina, Jason K von Meding

For decades sections of the academic community have been emphasizing that disasters are not natural. Nevertheless, politicians, the media, various international organizations—and, more surprisingly, many established researchers working in disaster studies—are still widely using the expression “natural disaster.” We systematically analyzed the usage of the expression “natural disaster” by disaster studies researchers in 589 articles in six key academic journals representative of disaster studies research, and found that authors are using the expression in three principal ways: (1) delineating natural and human-induced hazards; (2) using the expression to leverage popularity; and (3) critiquing the expression “natural disaster.” We also identified vulnerability themes that illustrate the context of “natural disaster” usage. The implications of continuing to use this expression, while explicitly researching human vulnerability, are wide-ranging, and we explore what this means for us and our peers. This study particularly aims to stimulate debate within the disaster studies research community and related fields as to whether the term “natural disaster” is really fit for purpose moving forward.

Oana Celia GHEORGHIU

Polonius, Romanian author Victor Cilincă’s farcical rewriting of Hamlet, is worth mentioning among the many postmodernist adaptations of Shakespeare’s works, despite the fact that it is virtually unknown to both theatregoers and critics. The play premiered in 1996, in Galati, having been ‘hidden’ in the writer’s drawer for more than a decade for fear that censorship might grasp its anti-totalitarian implications. In 2011, it was translated by Petru Iamandi for an American indie press. Prefaced by a brief overview of drawer literature and “refashioning of Shakespeare’s image along the lines of Communist ideology” (Colipcă-Ciobanu 2016: 26), in communist Romania, the paper focuses on the meta-dimension of the two-act play, as well as on the subversive aspects identifiable at the textual level.

Firda Rohmatul Aimah, Eko Suhartoyo

The discussion regarding the utilization of peer feedback necessitates careful consideration, particularly within the university context. However, the impact of using peer feedback remains indeterminate. This study investigated Indonesian EFL students' perceptions of implementing peer feedback during synchronous and asynchronous periods. A systematic literature review (SLR) method was applied in this study. The data were gathered using the Lens database, evaluated using PRISMA, and coded based on the research questions. The data were separated into two parts over ten years: synchronous (2012-2018) and asynchronous (2019-2022). The findings indicated positive and negative comments. In terms of positive comments, students claimed they got a better understanding; however, in terms of negative comments, students mentioned that the feedback was questionable. The students implemented peer feedback via online and offline media. For online media, students use social media, such as Instagram, Facebook, and websites. In terms of offline media, students used papers. The students also implemented some types of peer feedback, such as support comments, knowledge sharing, negotiation, appreciation, and criticism. The current research limitations and suggestions underscore the need for broader investigations across diverse educational levels and geographical contexts, emphasizing the imperative to enhance the study's generalizability and validity.

Mohamed Abdellatif Hussein, Hesham A. Hassan, M. Nassef

Background Writing composition is a significant factor for measuring test-takers’ ability in any language exam. However, the assessment (scoring) of these writing compositions or essays is a very challenging process in terms of reliability and time. The need for objective and quick scores has raised the need for a computer system that can automatically grade essay questions targeting specific prompts. Automated Essay Scoring (AES) systems are used to overcome the challenges of scoring writing tasks by using Natural Language Processing (NLP) and machine learning techniques. The purpose of this paper is to review the literature for the AES systems used for grading the essay questions. Methodology We have reviewed the existing literature using Google Scholar, EBSCO and ERIC to search for the terms “AES”, “Automated Essay Scoring”, “Automated Essay Grading”, or “Automatic Essay” for essays written in English language. Two categories have been identified: handcrafted features and automatically featured AES systems. The systems of the former category are closely bonded to the quality of the designed features. On the other hand, the systems of the latter category are based on the automatic learning of the features and relations between an essay and its score without any handcrafted features. We reviewed the systems of the two categories in terms of system primary focus, technique(s) used in the system, the need for training data, instructional application (feedback system), and the correlation between e-scores and human scores. The paper includes three main sections. First, we present a structured literature review of the available Handcrafted Features AES systems. Second, we present a structured literature review of the available Automatic Featuring AES systems. Finally, we draw a set of discussions and conclusions. Results AES models have been found to utilize a broad range of manually-tuned shallow and deep linguistic features. AES systems have many strengths in reducing labor-intensive marking activities, ensuring a consistent application of scoring criteria, and ensuring the objectivity of scoring. Although many techniques have been implemented to improve the AES systems, three primary challenges have been identified. The challenges are lacking of the sense of the rater as a person, the potential that the systems can be deceived into giving a lower or higher score to an essay than it deserves, and the limited ability to assess the creativity of the ideas and propositions and evaluate their practicality. Many techniques have only been used to address the first two challenges.

Diana Castilleja

En un contexto actual de creciente polarización y politización en torno a las cuestiones migratorias, se analizan cinco propuestas de cómics y narrativas gráficas que contribuyen a visibilizar voces y espacios olvidados de la migración latinoamericana hacia Estados Unidos. Se hace hincapié en el rol de la hibridez genérica al otorgar y reforzar la legitimidad del relato de historias personales y colectivas mediante la inserción de la autobiografía, el testimonio, el reportaje, el western o la ciencia ficción, entre otros. El corpus elegido: Migrar (Mateo y Martínez, 2011), Cita en Phoenix (Sandoval, 2016), La cicatriz. En la frontera entre México y Estados Unidos (Ferraris y Chiocca, 2019), Barrera (Vaughan, Martín y Vicente, 2019) y Ana (Arriaga y Ramos, 2021), da cuenta de las múltiples posibilidades que ofrecen estas narrativas gráficas que dejan de ser divertimento para participar activamente en la configuración de un discurso socialmente comprometido.

Claudia Ruiz García

El presente artículo aborda varias formas de acercarse a la problemática del suicidio frente a la sociedad, la moral y la religión en el siglo XVIII en Francia e Italia. Se trata de un tema de gran importancia que es analizado por algunas mentes ilustradas como es el caso del Barón de Montesquieu, Voltaire, Cesare Beccaria y Giacomo Casanova. Se revisa la incorporación del vocablo “suicidio” a las lenguas inglesa, francesa, española e italiana y se investigan las razones por las cuales algunos filósofos prefieren abordarlo con extrema cautela, mientras que otros lo hacen frontalmente, arriesgándose a severos castigos por parte de la censura. En cada uno de los textos de estos autores, se utilizan diferentes estrategias discursivas para acercarse a un tema tan delicado, castigado por los tribunales civiles y religiosos. Se estudian obras de ficción y tratados políticos tales como Las cartas persas y Del espíritu de las leyes de Montesquieu, El Tratado de la Tolerancia de Voltaire, De delitos y penas de Beccaria y el Discurso y diálogos sobre el suicidio e Historia de mi vida de Giacomo Casanova. A lo largo de ellos, se observa un interesante mecanismo en la historia de las mentalidades, ya que se asiste a un cambio de paradigma que desembocará, poco a poco, en una nueva mirada sobre el acto suicida, pues se desdibuja la creencia de la participación del diablo como responsable de incitar a la persona a realizarlo y se pone en tela de juicio la idea común de que se trata de un comportamiento propio de la locura. El concepto comienza, por fin, a explicarse dentro de un marco racional y absolutamente comprensible, para posteriormente lograr despenalizarlo y secularizarlo.

Monther M. Elaish, Liyana Shuib, N. Ghani et al.

Marwan H. Sallam, Elena Martín-Monje, Yan Li

Abstract This study aims to explore the current published research on Language Massive Open Online Courses (LMOOCs), outlining the types of papers, countries where studies were performed and institutions devoted to this field. Also, it intends to classify the reviewed literature following a general categorisation of MOOCs , and to identify the main trends and topics of interest for LMOOC researchers. Results show that there is still a lack of LMOOC-related articles in CALL (computer assisted language learning) journals, since most of the publications in the period reviewed (2012-18) are conference papers. The country in which most studies have been done so far is Spain and Universidad Nacional de Educación a Distancia (UNED) is currently the most active institution in this area. Within the taxonomy established, the most popular categories of studies focused on LMOOC participants or providers and case studies. Systematic review of the published literature indicated that research trends in LMOOCs studies comprise: 1) conceptualisation of LMOOCs and their distinctive features; 2) attempts to find the most suitable model for language teaching and learning beyond the xMOOC/cMOOC dichotomy; 3) suitability of LMOOCs for languages for specific purposes (LSP) courses; 4) focus on the learners and their motivation and experience throughout the course; 5) reflection on the new role of the teacher; 6) instructional design and how it affects participants’ learning and possible attrition; and 7) importance of social learning in LMOOCs. Nevertheless, LMOOCs is still an emerging field in need of further serious studies.

Jesús Duce García

Entre los motivos relacionados con los grimorios o libros mágicos que abundan en el género de los libros de caballerías, el ejemplo que aparece en el Valerián de Hungría representa una singularidad muy destacable. Se trata del libro del sabio Zenofor, posiblemente el objeto fantástico más original creado por el notario valenciano Dionís Clemente, autor de la obra. El libro de Zenofor reúne elementos maravillosos de diversa índole, relacionados, como resulta pertinente, con el sistema probatorio que deben superar los héroes caballerescos. Ahora bien, el poder más inusitado del libro de Zenofor es la capacidad de reproducir imágenes reales de personas que se encuentran en otros lugares, a modo de una conexión visual, simultánea en el tiempo, entre espacios y dimensiones diferentes. Todo lo cual otorga a este increíble objeto el dominio insospechado de la imagen y la palabra.

Fina Nabilah Layaliya, Haryadi Haryadi, Nas Haryati Setyaningsih

Penelitian ini merupakan penelitian kepustakaan yakni data-data atau bahan-bahan yang diperlukan berasal dari perpustakaan baik berupa buku, ensklopedi, kamus, jurnal, dokumen, maupun majalah. Media pembelajaran dibutuhkan dalam mata pelajaran yang diajarkan di sekolah, salah satunya mata pelajaran Bahasa Indonesia. Komponen pelajaran Bahasa Indonesia mencakup mendengarkan, berbicara, membaca, dan menulis. Keempat sampek tersebut apabila dalam proses pembelajaran memanfaatkan media pembelajaran maka akan berkesan dan menarik, sehingga siswa mendapatkan pengalaman-pengalaman yang nyata dalam proses pembelajaran. Penelitian ini dilakukan bertujuan untuk mendeskripsikan macam-macam media pembelajaran yang bisa dimanfaatkan dalam pembelajaran bahasa dan sastra.

Debangana Mishra

This article will be focusing on Ishmael Reed’s Mumbo Jumbo (1972), a vibrant postmodernist text, which offers a fresh perspective on the rise of black popular culture in the form of Jes Grew, which is largely informed by jazz and neo-hoodoo aesthetics. Jes Grew, the phenomenon which binds the multifaceted text in cohesion and brings together elements from History, Jazz and Afrofuturism, is communicated by using the metaphor of a virulent disease- the Jes Grew pandemic. The article is a work in cultural studies, attempting to map the evolution of the counter culture that Jes Grew represents and its effect on identity. This mapping is achieved by viewing the equation of the Jazz counter-culture with the Jes Grew pandemic. Jes Grew decodes the cultural and racial politics Mumbo Jumbo is invested in by destabilising the meaning and perspective attached to ‘disease’ and adapting it to an entirely new climate of cultural reclamation and celebration by deconstructing the dominant culture defined illness (Jes Grew in the text) and reinterpreting it as potentially healing and liberating. The discussion of the politics and aesthetics of this counter-culture mainly hinges on the central metaphor of the Jes Grew pandemic operating throughout the narrative. Raymond Williams’ work on culture studies and Stuart Hall’s theory on the formation and representation of cultural identities are particularly helpful in discussing the issues of culture and identity that are in dialogue with the narrative.

Olga Gilyazova

The article addresses the relationship between open and closed texts. The aim of the study is to analyze the specifics of relationship between the open and closed text in the general context of communication between the author and the reader. While supporting mainly U. Eco’s concept in our article, we furnish it with the analysis of four approaches of openness/closedness, which we have singled out. Openness/closedness is conceptualized, firstly: as an ontological perspective of the opposition between the communication parameters: The text-as-process and text-as-outcome; secondly, as interpretation procedures that are common for the articulation of all texts, regardless of their form and contents; thirdly, as the ontological and technological potential of works’ physical unfinishedness that invite the recipient to co-authorship (for example, ‘the work-in-movement’ and hypertext); fourthly, as the ability of a text to provoke multiple or unambiguous interpretations. We also demonstrate that the way of text’s functioning is determined not only by the addressee's attitude to it, but text itself, in return, stimulates this attitude, predisposing to it by its specificity (as informative or artistic, kitsch or art, hypertext or linear text, digital or analog). However, as we explain in the conclusion, the dialectic of openness/closedness can affect the distinctness of these dichotomies: The text can use its openness manipulatively, which turns it into a closed text.