Hasil untuk "Computational linguistics. Natural language processing"

Wanru Zhao, Yihong Chen, Royson Lee et al.

Pre-trained large language models (LLMs) have become a cornerstone of modern natural language processing, with their capabilities extending across a wide range of applications and languages. However, the fine-tuning of multilingual LLMs, especially for low-resource languages, faces significant challenges arising from data-sharing restrictions (the physical border) and inherent linguistic differences (the linguistic border). These barriers hinder users of various languages, particularly those in low-resource regions, from fully benefiting from the advantages of LLMs. To address these challenges, we propose the Federated Prompt Tuning Paradigm for multilingual scenarios, which utilizes parameter-efficient fine-tuning while adhering to data sharing restrictions. We design a comprehensive set of experiments and analyze them using a novel notion of language distance to highlight the strengths of our paradigm: Even under computational constraints, our method not only improves data efficiency but also facilitates mutual enhancements across languages, particularly benefiting low-resource ones. Compared to traditional local cross-lingual transfer tuning methods, our approach achieves 6.9\% higher accuracy with improved data efficiency, and demonstrates greater stability and generalization. These findings underscore the potential of our approach to promote social equality and champion linguistic diversity, ensuring that no language is left behind.

Yonatan Belinkov, James Glass

AbstractThe field of natural language processing has seen impressive progress in recent years, with neural network models replacing many of the traditional systems. A plethora of new models have been proposed, many of which are thought to be opaque compared to their feature-rich counterparts. This has led researchers to analyze, interpret, and evaluate neural networks in novel and more fine-grained ways. In this survey paper, we review analysis methods in neural language processing, categorize them according to prominent research trends, highlight existing limitations, and point to potential directions for future work.

ALMAS AKMAL

Mustansar Hussain Tarar is a famous Urdu novelist. In this article, the desolate settlement Presented in Tarar's Novel "Bahao” and its psychological study under the on human thoughts and psychology due to the destruction of its civilization has been examined. Analysis of the character of novel is presented as a psychological study under the influence of theories such as life instinct, death instant, multiple personality disorder, narcissistic culture, Freud’s theory of dream, inferiority complex, oppression, replacement, Consciousness, Un Consciousness and Nostalgia. The aim of this article is to make the student of Urdu literature understand the psychological problems faced by the characters in the story of a deserted slum.

Fatima BAHI & Youcef ATTIA

Abstract : This study aims at understanding the concept of metalanguage: its origins, definition, its perception by the Arb receiver, its translation and its interpretation. The human being is unique living being in terms of using the language to express his thoughts linguistically without even knowing that he is applying the grammar of the language. In connection to this, the philosophical thought, ancient and new, paid a significant attention to language. Therefore, trends in schools of linguistics and publications have varied introducing definitions of language and linguistics particularly at the age of globalization during which knowledge has become open access due to the services technology affords for the scientific study of language. Accordingly, research has overstepped the narrow boundaries to search deeper in metalanguage. As a result, studies in linguistics have contributed to establishing a theory of the concept of metalanguage leading to the emergence of many definitions of the term specifically after the work of Ferdinand De Saussure who was interested in language and the science of language. Immediately then a philosophical question that goes beyond the simple study and naive definitions to a deeper philosophy represented by meta-philosophy crops up. Iin other words, it questions what is meta-philosophy, its most important cognitive dimensions, the specificity of this prefix (meta), which has influenced all research and all modern studies, transforming it from the path of research into the subject to research into metaphysics, and finally how it was received by Arab studies translated, theorized, transmitted, and applied this prefix ? Keywords: meta, meta-language, beyond, tongue, linguistics.

Yambaïdjé MADJINDAYE

Résumé : Publié dans le cadre de la résidence d’écriture "Rwanda : écrire par devoir de mémoire" initiée et organisée en juillet 1998 par le Tchadien Nocky Djédanoum, Murambi, le livre des ossements de Boubacar Boris Diop apparaît comme un véritable recueil de témoignages d’honneur. Dans un langage à la fois pathétique et mélancolique, l’auteur sénégalais essaye, autant que faire se peut et tout en suivant la requête des rescapés et des victimes de l’hécatombe qui pensent qu’il ne sert à rien d’altérer les réalités du génocide à travers une mise en fiction dénaturante, de dire l’indicible, de nommer l’innommable, de narrer l’inénarrable et de décrire l’indescriptible. La présente réflexion se propose donc d’explorer la mise en fiction desdits témoignages ainsi que le mode opératoire et le style très personnels de l’auteur qui consiste à donner, à tour de rôle, la parole aux victimes et aux bourreaux. L’analyse a été, au bout du compte, amplement concluante : la mise en fiction des témoignages a été relativement un succès. La littérature de génocide a ainsi, à travers ce roman, contribué à aider les rescapés à faire le deuil de leurs proches et à se libérer du choc traumatique. Elle a, enfin, aidé à conjurer les démons du génocide afin que de telles monstruosités ne se reproduisent plus, ni au Rwanda ni dans un autre pays d’Afrique. Mots-clés : Bourreaux – génocide rwandais – fiction – Rwanda – victimes – témoignages.

Mostefa Boulanouar & Ali Krim

Abstract : Sarcasm is an inherent tendency in the dramatic conscience since the early beginnings of theatrical practice, as it is a human expressive representation through its interaction with the various events and phenomena presented to it that provoked different reactions from the human being. Since theater is a collection of presentations of various human phenomena, it was self-evident that it carries. It contains satirical tendencies about what a person rejects or deplores, whether behaviorally or intellectually. The satirical style in theater is the result of the dramatic moment, but a tendency that has its beginnings and its founders, members, and even its pioneers. Although the father of arts was of Greek origins, the same origin is satire in the theater, and it became linked to the Greek dramatist Aristotle, and he was not alone in drawing it in any way, but his contemporaries recognized his excellence in it. He enabled it within his theatre, and allowed it to emerge as a special phenomenon, and this is evident through the play “The Frogs,” which was considered the opening and beginning of the satirical activity in the theater, after which dramas continued in the same manner. Aristotle considered that his goal in writing the play “The Frogs” was not merely to arouse laughter. The reason he turned to ridicule was the corruption that prevailed in society during the Peloponnesian War, but the goal was to reform that corruption after spreading it among the public, and ridicule is a beneficial and common good. And it wasn't Algerian theatre with an innovation of saying that it pursues a satirical tendency through its dealings with the issues that were presented to the Algerian person through his historical and cultural process, this art in Algeria He started laughing / being sarcastic with Juha's play for its companions, Alalo 1926, followed by performances, sketches, and improvisational skits for the purpose/goal of criticizing the social behaviors that prevailed in Algerian society during the colonial/destiny era. Through this research paper, we will present an Algerian textual model in which satirical tendencies were a basic path for displaying its expressive foundations and pictorial references, which is the text “The Unhappy and the Sleepy” by Azzedine djelaoudji with its rejection of many of the values that have come to characterize many collective as well as individual behaviors. Keywords: Théâtre, satire, humor, social values, Intellectual representations.

Vincent Freiberger, Erik Buchmann

Natural Language Processing (NLP) plays an important role in our daily lives, particularly due to the enormous progress of Large Language Models (LLM). However, NLP has many fairness-critical use cases, e.g., as an expert system in recruitment or as an LLM-based tutor in education. Since NLP is based on human language, potentially harmful biases can diffuse into NLP systems and produce unfair results, discriminate against minorities or generate legal issues. Hence, it is important to develop a fairness certification for NLP approaches. We follow a qualitative research approach towards a fairness certification for NLP. In particular, we have reviewed a large body of literature on algorithmic fairness, and we have conducted semi-structured expert interviews with a wide range of experts from that area. We have systematically devised six fairness criteria for NLP, which can be further refined into 18 sub-categories. Our criteria offer a foundation for operationalizing and testing processes to certify fairness, both from the perspective of the auditor and the audited organization.

Ben Hutchinson

This position paper concerns the use of religious texts in Natural Language Processing (NLP), which is of special interest to the Ethics of NLP. Religious texts are expressions of culturally important values, and machine learned models have a propensity to reproduce cultural values encoded in their training data. Furthermore, translations of religious texts are frequently used by NLP researchers when language data is scarce. This repurposes the translations from their original uses and motivations, which often involve attracting new followers. This paper argues that NLP's use of such texts raises considerations that go beyond model biases, including data provenance, cultural contexts, and their use in proselytism. We argue for more consideration of researcher positionality, and of the perspectives of marginalized linguistic and religious communities.

Denisa Millo, Blerina Vika, Nevila Baci

The financial sector, a pivotal force in economic development, increasingly uses the intelligent technologies such as natural language processing to enhance data processing and insight extraction. This research paper through a review process of the time span of 2018-2023 explores the use of text mining as natural language processing techniques in various components of the financial system including asset pricing, corporate finance, derivatives, risk management, and public finance and highlights the need to address the specific problems in the discussion section. We notice that most of the research materials combined probabilistic with vector-space models, and text-data with numerical ones. The most used technique regarding information processing is the information classification technique and the most used algorithms include the long-short term memory and bidirectional encoder models. The research noticed that new specific algorithms are developed and the focus of the financial system is mainly on asset pricing component. The research also proposes a path from engineering perspective for researchers who need to analyze financial text. The challenges regarding text mining perspective such as data quality, context-adaption and model interpretability need to be solved so to integrate advanced natural language processing models and techniques in enhancing financial analysis and prediction. Keywords: Financial System (FS), Natural Language Processing (NLP), Software and Text Engineering, Probabilistic, Vector-Space, Models, Techniques, TextData, Financial Analysis.

Yucheng Hu, Yuxing Lu

Large Language Models (LLMs) have catalyzed significant advancements in Natural Language Processing (NLP), yet they encounter challenges such as hallucination and the need for domain-specific knowledge. To mitigate these, recent methodologies have integrated information retrieved from external resources with LLMs, substantially enhancing their performance across NLP tasks. This survey paper addresses the absence of a comprehensive overview on Retrieval-Augmented Language Models (RALMs), both Retrieval-Augmented Generation (RAG) and Retrieval-Augmented Understanding (RAU), providing an in-depth examination of their paradigm, evolution, taxonomy, and applications. The paper discusses the essential components of RALMs, including Retrievers, Language Models, and Augmentations, and how their interactions lead to diverse model structures and applications. RALMs demonstrate utility in a spectrum of tasks, from translation and dialogue systems to knowledge-intensive applications. The survey includes several evaluation methods of RALMs, emphasizing the importance of robustness, accuracy, and relevance in their assessment. It also acknowledges the limitations of RALMs, particularly in retrieval quality and computational efficiency, offering directions for future research. In conclusion, this survey aims to offer a structured insight into RALMs, their potential, and the avenues for their future development in NLP. The paper is supplemented with a Github Repository containing the surveyed works and resources for further study: https://github.com/2471023025/RALM_Survey.

Zhousi Chen, Mamoru Komachi

AbstractWe extend a pair of continuous combinator-based constituency parsers (one binary and one multi-branching) into a discontinuous pair. Our parsers iteratively compose constituent vectors from word embeddings without any grammar constraints. Their empirical complexities are subquadratic. Our extension includes 1) a swap action for the orientation-based binary model and 2) biaffine attention for the chunker-based multi-branching model. In tests conducted with the Discontinuous Penn Treebank and TIGER Treebank, we achieved state-of-the-art discontinuous accuracy with a significant speed advantage.

Daniela Puato

The topic of this study is the pragmatics of investment recommendations in German financial magazines. The target reader, mainly a private investor, expects a clear assessment of the quality of the investment product and a corresponding operative instruction regarding his investment decision (whether he should buy, sell or hold a certain stock, mutual fund, or bond). On the basis of a large corpus of over five thousand texts from the most important weekly magazines (Börse Online, Der Aktionär, Focus Money) it is investigated to what extent this text genre meets reader expectations. Investment recommendations show a potentially complex pragmatic structure including, among others, a headline, an evaluation of relevant arguments (pros and cons) and the operative instruction (buy/sell/hold). However, crucial elements may be missing or even give conflicting hints concerning the investment decision. Moreover, operative instructions often tend to be formulated as rather vague indirect speech acts, limiting the overall pragmatic user-friendliness. These apparent inconsistencies can be put down to a secondary pragmatic goal from the perspective of the writer: not only to give good advice, but also not to be held accountable for bad advice.

Ziheng Zeng, Kellen Tan Cheng, Srihari Venkat Nanniyur et al.

Idiomatic expression (IE) processing and comprehension have challenged pre-trained language models (PTLMs) because their meanings are non-compositional. Unlike prior works that enable IE comprehension through fine-tuning PTLMs with sentences containing IEs, in this work, we construct IEKG, a commonsense knowledge graph for figurative interpretations of IEs. This extends the established ATOMIC2020 graph, converting PTLMs into knowledge models (KMs) that encode and infer commonsense knowledge related to IE use. Experiments show that various PTLMs can be converted into KMs with IEKG. We verify the quality of IEKG and the ability of the trained KMs with automatic and human evaluation. Through applications in natural language understanding, we show that a PTLM injected with knowledge from IEKG exhibits improved IE comprehension ability and can generalize to IEs unseen during training.

Hao Zou, Zae Myung Kim, Dongyeop Kang

This survey paper provides a comprehensive review of the use of diffusion models in natural language processing (NLP). Diffusion models are a class of mathematical models that aim to capture the diffusion of information or signals across a network or manifold. In NLP, diffusion models have been used in a variety of applications, such as natural language generation, sentiment analysis, topic modeling, and machine translation. This paper discusses the different formulations of diffusion models used in NLP, their strengths and limitations, and their applications. We also perform a thorough comparison between diffusion models and alternative generative models, specifically highlighting the autoregressive (AR) models, while also examining how diverse architectures incorporate the Transformer in conjunction with diffusion models. Compared to AR models, diffusion models have significant advantages for parallel generation, text interpolation, token-level controls such as syntactic structures and semantic contents, and robustness. Exploring further permutations of integrating Transformers into diffusion models would be a valuable pursuit. Also, the development of multimodal diffusion models and large-scale diffusion language models with notable capabilities for few-shot learning would be important directions for the future advance of diffusion models in NLP.

Patrick Perrine

Large Language Models (LLMs) have been making big waves in the machine learning community within the past few years. The impressive scalability of LLMs due to the advent of deep learning can be seen as a continuation of empiricist lingusitic methods, as opposed to rule-based linguistic methods that are grounded in a nativist perspective. Current LLMs are generally inaccessible to resource-constrained researchers, due to a variety of factors including closed source code. This work argues that this lack of accessibility could instill a nativist bias in researchers new to computational linguistics, given that new researchers may only have rule-based, nativist approaches to study to produce new work. Also, given that there are numerous critics of deep learning claiming that LLMs and related methods may soon lose their relevancy, we speculate that such an event could trigger a new wave of nativism in the language processing community. To prevent such a dramatic shift and placing favor in hybrid methods of rules and deep learning, we call upon researchers to open source their LLM code wherever possible to allow both empircist and hybrid approaches to remain accessible.

Ina Lehmkuhle, Josefin Lindgren

It has been suggested that acquiring the appropriate use of referring expressions consists of a shift from an initial focus on global accessibility factors, e.g., animacy or character type, towards primarily considering local accessibility factors, such as information status, referential function and topicality. At which age this shift takes place remains an open question. The present study investigates anaphoric reference in picture-based written narratives by German-speaking 10-year-olds and adults. We analyse and compare the extent to which referential function (maintenance vs. reintroduction), a local accessibility factor, and character type (main character vs. secondary character), a global accessibility factor, influence children’s and adults’ choice of referring expression. The results show that referential function affected referential choice in both children and adults, with significantly higher proportions of pronouns in maintenance than in reintroduction. However, character type only influenced the children, who produced a significantly higher proportion of pronouns with main characters than with secondary characters. These results suggest that children’s referring expression use is not yet fully adultlike at age 10, and that adults and children weigh local and global accessibility factors differently: global factors play a role in children’s referential choice in addition to local ones, whereas adults are primarily influenced by local accessibility factors.

Anees Rashid Hashmi, Khawaja Zahid Aziz

<p class="MsoNormal" style="text-align: justify; text-indent: .5in; line-height: normal;"><em><span style="font-size: 12.0pt; font-family: &quot;Times New Roman&quot;,serif; mso-ascii-theme-font: major-bidi; mso-hansi-theme-font: major-bidi; mso-bidi-theme-font: major-bidi;">The occupation, slavery and violence are against the nature and most notorious in the life of nations who are suffered from cultural, structural, physical and potential violence which badly impacted upon all five factors of human life activities i.e., economic, environmental, political, security and societal. Man has an instinct to fight against these psychological phenomenon slavery, occupation, subjugation and deprivation of rights which requires any forceful stimulus to revoke against injustice, violence and hostility. In Kashmir, just after the downfall of Mughals the alien rulers oppressed, made inhuman, harsh and authoritarian treatment with the Kashmiris by Afghans, Sikhs and in last by Dogras. Iqbal </span></em><em><span style="font-size: 12.0pt; font-family: &quot;Times New Roman&quot;,serif; mso-ascii-theme-font: major-bidi; mso-hansi-theme-font: major-bidi; mso-bidi-theme-font: major-bidi; mso-bidi-language: ER;">having Kashmiri family background remained worry about the poor and miserable situation and substandard living of the Kashmiris. The slavery, tortures and heavy taxes with deprivation for fundamental right, no authority, no liberty, and freedom were very famous in subcontinent. Iqbal raised his voice against subjugation and inhuman treatment with the Kashmiris and also inspired Kashmiris against brutality and slavery. The lightening words of Iqbal is still working as magical impacts in the feelings, emotions, thoughts and ideas of the Kashmiris and they stood against the Dogra illegal and injustice monocracy and undemocratic autocracy in Kashmir. The resistance of Iqbal during the accidents of 1931 gave birth to impressive freedom movement after the partition of subcontinent. This is an attempt to highlight the Iqbal specific poems and verses written on the misery of Kashmir and some lines for the motivations of the Kashmiris which resulted into strong inspiration against Indian illegal Occupation since last 74 years.</span></em></p>

Abdussalam Aljadani

Abstract Cryptocurrencies are distributed digital currencies that have emerged as a consequence of financial technology advancement. In 2017, cryptocurrencies have shown a huge rise in their market capitalization and popularity. They are now employed in today’s financial systems as individual investors, corporate firms, and big institutions are heavily investing in them. However, this industry is less stable than traditional currency markets. It can be affected by several legal, sentimental, and technical factors, so it is highly volatile, dynamic, uncertain, and unpredictable, hence, accurate forecasting is essential. Recently, cryptocurrency price prediction becomes a trending research topic globally. Various machine and deep learning algorithms, e.g., Neural Networks (NN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and Bidirectional LSTM (BiLSTM) were utilized to analyze the factors influencing the prices of the cryptocurrencies and accordingly predict them. This paper suggests a five-phase framework for cryptocurrency price prediction based on two state-of-the-art deep learning architectures (i.e., BiLSTM and GRU). The current study uses three public real-time cryptocurrency datasets from “Yahoo Finance”. Bidirectional Long Short-Term Memory and Gated Recurrent Unit deep learning-based algorithms are used to forecast the prices of three popular cryptocurrencies (i.e., Bitcoin, Ethereum, and Cardano). The Grid Search approach is used for the hyperparameters optimization processes. Results indicate that GRU outperformed the BiLSTM algorithm for Bitcoin, Ethereum, and Cardano, respectively. The lowest RMSE for the GRU model was found to be 0.01711, 0.02662, and 0.00852 for Bitcoin, Ethereum, and Cardano, respectively. Experimental results proved the significant performance of the proposed framework that achieves the minimum MSE and RMSE values.

Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn et al.

To achieve lifelong language learning, pseudo-rehearsal methods leverage samples generated from a language model to refresh the knowledge of previously learned tasks. Without proper controls, however, these methods could fail to retain the knowledge of complex tasks with longer texts since most of the generated samples are low in quality. To overcome the problem, we propose three specific contributions. First, we utilize double language models, each of which specializes in a specific part of the input, to produce high-quality pseudo samples. Second, we reduce the number of parameters used by applying adapter modules to enhance training efficiency. Third, we further improve the overall quality of pseudo samples using temporal ensembling and sample regeneration. The results show that our framework achieves significant improvement over baselines on multiple task sequences. Also, our pseudo sample analysis reveals helpful insights for designing even better pseudo-rehearsal methods in the future.

Oskar van der Wal, Jaap Jumelet, Katrin Schulz et al.

Detecting and mitigating harmful biases in modern language models are widely recognized as crucial, open problems. In this paper, we take a step back and investigate how language models come to be biased in the first place. We use a relatively small language model, using the LSTM architecture trained on an English Wikipedia corpus. With full access to the data and to the model parameters as they change during every step while training, we can map in detail how the representation of gender develops, what patterns in the dataset drive this, and how the model's internal state relates to the bias in a downstream task (semantic textual similarity). We find that the representation of gender is dynamic and identify different phases during training. Furthermore, we show that gender information is represented increasingly locally in the input embeddings of the model and that, as a consequence, debiasing these can be effective in reducing the downstream bias. Monitoring the training dynamics, allows us to detect an asymmetry in how the female and male gender are represented in the input embeddings. This is important, as it may cause naive mitigation strategies to introduce new undesirable biases. We discuss the relevance of the findings for mitigation strategies more generally and the prospects of generalizing our methods to larger language models, the Transformer architecture, other languages and other undesirable biases.