H. Freire-Oliveira, E. R. F. Paiva, J. Gama
et al.
Classification in the context of multi-label data streams represents a challenge that has attracted significant attention due to its high real-world applicability. However, this task faces problems inherent to dynamic environments, such as the continuous arrival of data at high speed and volume, changes in the data distribution (concept drift), the emergence of new labels (concept evolution), and the latency in the arrival of ground truth labels. This systematic literature review presents an in-depth analysis of multi-label data stream classification proposals. We characterize the latest methods in the literature, providing a comprehensive overview, building a thorough hierarchy, and discussing how the proposals approach each problem. Furthermore, we discuss the adopted evaluation strategies and analyze the methods' asymptotic complexity and resource consumption. Finally, we identify the main gaps and offer recommendations for future research directions in the field.
This paper reviews literature pertaining to the development of data science as a discipline, current issues with data bias and ethics, and the role that the discipline of information science may play in addressing these concerns. Information science research and researchers have much to offer for data science, owing to their background as transdisciplinary scholars who apply human-centered and social-behavioral perspectives to issues within natural science disciplines. Information science researchers have already contributed to a humanistic approach to data ethics within the literature and an emphasis on data science within information schools all but ensures that this literature will continue to grow in coming decades. This review article serves as a reference for the history, current progress, and potential future directions of data ethics research within the corpus of information science literature.
The rise of smart contracts has expanded blockchain's capabilities, enabling the development of innovative decentralized applications (dApps). However, this advancement brings its own challenges, including the management of distributed architectures and immutable data. Addressing these complexities requires a specialized approach to software engineering, with blockchain-oriented practices emerging to support development in this domain. Developer Experience (DEx) is central to this effort, focusing on the usability, productivity, and overall satisfaction of tools and frameworks from the engineers' perspective. Despite its importance, research on Blockchain Developer Experience (BcDEx) remains limited, with no systematic mapping of academic and industry efforts. To bridge this gap, we conducted a Multivocal Literature Review analyzing 62 to understand the distribution of BcDEx sources, practical implementations, and their impact. Our findings revealed that academic focus on BcDEx is limited compared to the coverage in gray literature, which primarily includes blogs (41.8%) and corporate sources (21.8%). Particularly, development efficiency, multi-network support, and usability are the most addressed aspects in tools and frameworks. In addition, we found that BcDEx is being shaped through five key perspectives: complexity abstraction, adoption facilitation, productivity enhancement, developer education, and BcDEx evaluation.
A smart ring is a wearable electronic device in the form of a ring that incorporates diverse sensors and computing technologies to perform a variety of functions. Designed for use with fingers, smart rings are capable of sensing more subtle and abundant hand movements, thus making them a good platform for interaction. Meanwhile, fingers are abundant with blood vessels and nerve endings and accustomed to wearing rings, providing an ideal site for continuous health monitoring through smart rings, which combine comfort with the ability to capture vital biometric data, making them suitable for all-day wear. We collected in total of 206 smart ring-related publications and conducted a systematic literature review. We provide a taxonomy regarding the sensing and feedback modalities, applications, and phenomena. We review and categorize these literatures into four main areas: (1) interaction - input, (2) interaction - output, (3) passive sensing - in body feature, (4) passive sensing - out body activity. This comprehensive review highlights the current advancements within the field of smart ring and identifies potential areas for future research.
Materiality has emerged as a significant theme in Holocaust literature as well as in Holocaust studies scholarship, highlighting the pivotal role of physical objects. This materiality has been conceptualized in various ways in recent scholarship, including «testimonial objects», «objects of return», and «artifacts of memory». Building on this conceptual framework, the article analyzes the ways in which transgenerational memory transmission is thematized in Monica Csango’s memoir <i>Fortielser. Min jødiske familiehistorie</i> («Concealments. My Jewish Family History», 2017), investigating what memorial functions material places and objects—in particular inherited objects—serve in the transmission and representation of memory within the narrative. The central question the article addresses is: Which places and objects are central to the narrative’s representation of memory, and in what ways do they mediate memory and trauma? The article suggests that postmemory transforms physical objects and places spaces into sites of remembering and mourning, enabling transgenerational continuity and memory transmission in <i>Fortielser</i>. These findings underscore the central role of material and spatial mediums in sustaining intergenerational remembrance, suggesting that inherited artifacts and projected spaces constitute vital modes of memory transmission, or «acts of transfer», within parts of Jewish Norwegian second- and third-generation literature.
History of scholarship and learning. The humanities
This article argues that enduring legacies of colonialism undermine successful integration of non-Western immigrants in Western societies and that the currently dominant national identity model does not allow to address these legacies and their effects. The article discusses why trust is an important component of successful integration and identifies and critiques four core features of national identity models. The article then proposes an alternative way to analyze social trust which pays attention to historically grown and international structures that influence the background conditions in which trust relations take place. Such an analysis brings to light a broader picture of what influences trust and successful integration, supplementing the current focus on cultural difference and shared norms with an analysis of prejudice and boundary-making as well as inequality.
This paper examines the acceptability of Spanish adverbs in sentence-medial positions in declarative main clauses. Both simple and complex verb forms are considered. Prior work has established that some Spanish adverbs can be placed in more than one sentence-medial position. To date, research on this topic by means of formal judgments is notably sparse, so this paper aims to expand the body of available evidence. In addition to examining acceptability, this paper also investigates whether there is a preferred position when more than one is available.
Two online experimental tasks were completed by native speakers of Spanish (n = 48): an acceptability judgment task and a gap filling task. The tasks contained target sentences with one of the four adverbs that were selected for the study: completamente ‘completely’, normalmente ‘usually’, siempre ‘always’ and ya ‘already’. Half of the target sentences contained just a lexical verb, while the other half also included an auxiliary.
The acceptability data are first analysed descriptively, followed by an analysis with cumulative link mixed models. These analyses are then contrasted with a descriptive analysis of the preference data supported by binomial testing. The results show that the adverbs tested in the study display different patterns of acceptability and preferential placement. The results of each adverb are discussed individually within the current theoretical framework, addressing implications for theories that view adverbs as specifiers and those that view them as adjuncts.
Benjamin Newman, Yoonjoo Lee, Aakanksha Naik
et al.
When conducting literature reviews, scientists often create literature review tables - tables whose rows are publications and whose columns constitute a schema, a set of aspects used to compare and contrast the papers. Can we automatically generate these tables using language models (LMs)? In this work, we introduce a framework that leverages LMs to perform this task by decomposing it into separate schema and value generation steps. To enable experimentation, we address two main challenges: First, we overcome a lack of high-quality datasets to benchmark table generation by curating and releasing arxivDIGESTables, a new dataset of 2,228 literature review tables extracted from ArXiv papers that synthesize a total of 7,542 research papers. Second, to support scalable evaluation of model generations against human-authored reference tables, we develop DecontextEval, an automatic evaluation method that aligns elements of tables with the same underlying aspects despite differing surface forms. Given these tools, we evaluate LMs' abilities to reconstruct reference tables, finding this task benefits from additional context to ground the generation (e.g. table captions, in-text references). Finally, through a human evaluation study we find that even when LMs fail to fully reconstruct a reference table, their generated novel aspects can still be useful.
Systematic literature reviews (SLRs) are essential but labor-intensive due to high publication volumes and inefficient keyword-based filtering. To streamline this process, we evaluate Large Language Models (LLMs) for enhancing efficiency and accuracy in corpus filtration while minimizing manual effort. Our open-source tool LLMSurver presents a visual interface to utilize LLMs for literature filtration, evaluate the results, and refine queries in an interactive way. We assess the real-world performance of our approach in filtering over 8.3k articles during a recent survey construction, comparing results with human efforts. The findings show that recent LLM models can reduce filtering time from weeks to minutes. A consensus scheme ensures recall rates >98.8%, surpassing typical human error thresholds and improving selection accuracy. This work advances literature review methodologies and highlights the potential of responsible human-AI collaboration in academic research.
Academic literature reviews have traditionally relied on techniques such as keyword searches and accumulation of relevant back-references, using databases like Google Scholar or IEEEXplore. However, both the precision and accuracy of these search techniques is limited by the presence or absence of specific keywords, making literature review akin to searching for needles in a haystack. We present vitaLITy 2, a solution that uses a Large Language Model or LLM-based approach to identify semantically relevant literature in a textual embedding space. We include a corpus of 66,692 papers from 1970-2023 which are searchable through text embeddings created by three language models. vitaLITy 2 contributes a novel Retrieval Augmented Generation (RAG) architecture and can be interacted with through an LLM with augmented prompts, including summarization of a collection of papers. vitaLITy 2 also provides a chat interface that allow users to perform complex queries without learning any new programming language. This also enables users to take advantage of the knowledge captured in the LLM from its enormous training corpus. Finally, we demonstrate the applicability of vitaLITy 2 through two usage scenarios. vitaLITy 2 is available as open-source software at https://vitality-vis.github.io.
The paradigm of Big Data has been established as a solid field of studies in many areas such as healthcare, science, transport, education, government services, among others. Despite widely discussed, there is no agreed definition about the paradigm although there are many concepts proposed by the academy and industry. This work aims to provide an analytical view of the studies conducted and published regarding the Big Data paradigm. The approach used is the systematic map of the literature, combining bibliometric analysis and content analysis to depict the panorama of research works, identifying patterns, trends, and gaps. The results indicate that there is still a long way to go, both in research and in concepts, such as building and defining adequate infrastructures and standards, to meet future challenges and for the paradigm to become effective and bring the expected benefits.
Anirudh Ajith, Mengzhou Xia, Alexis Chevalier
et al.
Literature search questions, such as "Where can I find research on the evaluation of consistency in generated summaries?" pose significant challenges for modern search engines and retrieval systems. These questions often require a deep understanding of research concepts and the ability to reason across entire articles. In this work, we introduce LitSearch, a retrieval benchmark comprising 597 realistic literature search queries about recent ML and NLP papers. LitSearch is constructed using a combination of (1) questions generated by GPT-4 based on paragraphs containing inline citations from research papers and (2) questions manually written by authors about their recently published papers. All LitSearch questions were manually examined or edited by experts to ensure high quality. We extensively benchmark state-of-the-art retrieval models and also evaluate two LLM-based reranking pipelines. We find a significant performance gap between BM25 and state-of-the-art dense retrievers, with a 24.8% absolute difference in recall@5. The LLM-based reranking strategies further improve the best-performing dense retriever by 4.4%. Additionally, commercial search engines and research tools like Google Search perform poorly on LitSearch, lagging behind the best dense retriever by up to 32 recall points. Taken together, these results show that LitSearch is an informative new testbed for retrieval systems while catering to a real-world use case.
Abstract Background Increasingly, counter‐radicalisation interventions are using case management approaches to structure the delivery of tailored services to those at risk of engaging in, or engaged in, violent extremism. This review sets out the evidence on case management tools and approaches and is made up of two parts with the following objectives. Objectives Part I: (1) Synthesise evidence on the effectiveness of case management tools and approaches in interventions seeking to counter radicalisation to violence. (2) Qualitatively synthesise research examining whether case management tools and approaches are implemented as intended, and the factors that explain how they are implemented. Part II: (3) Synthesise systematic reviews to understand whether case management tools and approaches are effective at countering non‐terrorism related interpersonal or collective forms of violence. (4) Qualitatively synthesise research analysing whether case management tools and approaches are implemented as intended, and what influences how they are implemented. (5) Assess the transferability of tools and approaches used in wider violence prevention work to counter‐radicalisation interventions. Search Methods Search terms tailored for Part I and Part II were used to search research repositories, grey literature sources and academic journals for studies published between 2000 and 2022. Searches were conducted in August and September 2022. Forward and backward citation searches and consultations with experts took place between September 2022 and February 2023. Studies in English, French, German, Russian, Swedish, Norwegian and Danish were eligible. Selection Criteria Part I: Studies had to report on a case management intervention, tool or approach, or on specific stages of the case management process. Only experimental and stronger quasi‐experimental studies were eligible for inclusion in the analysis of effectiveness. The inclusion criteria for the analysis of implementation allowed for other quantitative designs and qualitative research. Part II: Systematic reviews examining a case management intervention, tool or approach, or stage(s) of the case management process focused on countering violence were eligible for inclusion. Data Collection and Analysis Part I: 47 studies were eligible for Part I. No studies met the inclusion criteria for Objective 1; all eligible studies related to Objective 2. Data from these studies was synthesised using a framework synthesis approach and presented narratively. Risk of bias was assessed using the CASP (for qualitative research) and EPHPP (for quantitative research) checklists. Part I: Eight reviews were eligible for Part II. Five reviews met the inclusion criteria for Objective 3, and seven for Objective 4. Data from the studies was synthesised using a framework synthesis approach and presented narratively. Risk of bias was assessed using the AMSTAR II tool. Findings Part I: No eligible studies examined effectiveness of tools and approaches. Seven studies examined the implementation of different approaches, or the assumptions underpinning interventions. Clearly defined theories of change were absent, however these interventions were assessed as being implemented in line with their own underlying logic. Forty‐three studies analysed the implementation of tools during individual stages of the case management process, and forty‐one examined the implementation of this process as‐a‐whole. Factors which influenced how individual stages and the case management process as a whole were implemented included strong multi‐agency working arrangements; the inclusion of relevant knowledge and expertise, and associated training; and the availability of resources. The absence of these facilitators inhibited implementation. Additional implementation barriers included overly risk‐oriented logics; public and political pressure; and broader legislation. Twenty‐eight studies identified moderators that shaped how interventions were delivered, including delivery context; local context; standalone interventions; and client challenges. Part II: The effectiveness of two interventions – mentoring and multi‐systemic therapy – in reducing violent outcomes were each assessed by one systematic review, whilst three reviews analysed the impact that the use of risk assessment tools (n = 2) and polygraphs (n = 1) had on outcomes. All these reviews reported mixed results. Comparable factors to those identified in Part I, such as staff training and expertise and delivery context, were found to shape implementation. On the basis of this modest sample, the research on interventions to counter non‐terrorism related violence was assessed to be transferable to counter‐radicalisation interventions. Authors' Conclusions The effectiveness of existing case management tools and approaches is poorly understood, and research examining the factors that influence how different approaches are implemented is limited. However, there is a growing body of research on the factors which facilitate or generate barriers to the implementation of case management interventions. Many of the factors and moderators relevant to countering radicalisation to violence also impact how case management tools and approaches used to counter other forms of violence are implemented. Research in this wider field seems to have transferable insights for efforts to counter radicalisation to violence. This review provides a platform for further research to test the impact of different tools, and the mechanisms by which they inform outcomes. This work will benefit from using the case management framework as a way of rationalising and analysing the range of tools, approaches and processes that make up case managed interventions to counter radicalisation to violence.
In sponsored search advertising (SSA), keywords serve as the basic unit of business model, linking three stakeholders: consumers, advertisers and search engines. This paper presents an overarching framework for keyword decisions that highlights the touchpoints in search advertising management, including four levels of keyword decisions, i.e., domain-specific keyword pool generation, keyword targeting, keyword assignment and grouping, and keyword adjustment. Using this framework, we review the state-of-the-art research literature on keyword decisions with respect to techniques, input features and evaluation metrics. Finally, we discuss evolving issues and identify potential gaps that exist in the literature and outline novel research perspectives for future exploration.
Insights are often considered the ideal outcome of visual analysis sessions. However, there is no single definition of what an insight is. Some scholars define insights as correlations, while others define them as hypotheses or aha moments. This lack of a clear definition can make it difficult to build visualization tools that effectively support insight discovery. In this paper, we contribute a comprehensive literature review that maps the landscape of existing insight definitions. We summarize key themes regarding how insight is defined, with the goal of helping readers identify which definitions of insight align closely with their research and tool development goals. Based on our review, we also suggest interesting research directions, such as synthesizing a unified formalism for insight and connecting theories of insight to other critical concepts in visualization research.
James D Kiper, Simon Sultana, Brent Auernheimer
et al.
Software engineering (SE) evolves rapidly, with changing technology and industry expectations. The curriculum review bodies (e.g., ACM and IEEE-CS working groups) respond well but can have refresh cycles measured in years. For Computer Science and SE educators to be agile, predictive, and adapt to changing technology trends, judicious use of gray literature (GL) can be helpful. Other fields have found GL useful in bridging academic research and industry needs. GL can be extended to SE to aid faculty preparing students for industry. We address two questions: first, given the velocity of technical change, do current curricular guidelines accurately reflect industry practice and need for our graduates? Second, how can we track current and emerging trends to capture relevant competencies? We argue a study of the scholarly literature will have a limited impact on our understanding of current and emerging trends and curriculum designers would do well to utilize GL. We close with recommendations for SE educators.
Talieh Sadeghi, Joakim Finne, Ida Bring Løberg
et al.
Abstract Public employment services are increasingly becoming digital. Previous literature has highlighted potential and intersectional risks associated with digitalizing these services. To address this, the current study explores the relationships between traditional and digital follow-up on the one hand, and trust and satisfaction on the other, among a sample of 1195 young people between the ages 18–25 registered with the Norwegian Labour and Welfare Administration. Furthermore, the study aimed to investigate any variations in associations between different groups of young people who require varying levels of support (low, medium and high). Results for the whole sample suggested that both digital activity plans and face-to-face meetings were related to trust and satisfaction. For the group with low support needs, digital follow-up was positively associated with both trust and satisfaction with employment services. Among those with high support needs, both face-to-face meetings and digital follow-up were linked to trust and satisfaction. In sum, our results reveal that digital forms of follow-up are not inferior to the more traditional types of follow-up with regards to trust and satisfaction.
Social pathology. Social and public welfare. Criminology
Dates, periods and timespans are described in archaeological datasets using a number of different textual patterns for which myriad variations exist, rendering direct automated comparison difficult. The issue can occur even within records from the same dataset and is further compounded when attempting to integrate multilingual data – particularly where dates may be expressed in words rather than numbers. The same problem can be found in temporal metadata, whether manually entered or generated via Natural Language Processing (NLP) techniques from reports and grey literature. Resolving and normalizing dates and periods to internationally agreed standard formats enables efficient data integration, interchange, search, comparison and visualization. This paper reports on the design and implementation of a tool to normalize temporal expressions to a numerical time axis and reflects on key issues. Textual patterns for seven categories of temporal expression have been normalized: Ordinal named or numbered centuries; Year spans; Single year (with tolerance); Decades; Century spans; Single year with prefix; Named periods. The following languages are currently supported: Dutch, English, French, German, Italian, Norwegian, Spanish, Swedish, Welsh. Methods are described together with an (open source) normalization tool developed in Python and four applications of the method are discussed, together with limitations and future work. Results are presented from diverse data sets and languages. The input is a temporal text string and a language code (ISO639-1). The output is a tab delimited text file with start/end years (in ISO 8601 format), relative to Common Era (CE). The normalized outputs are provided as additional attributes along with the original text expression for consuming software to employ in end-user applications.
Rickrolling is an Internet cultural phenomenon born in the mid 2000s. Originally confined to Internet fora, it has spread to other channels and media. In this paper, we hypothesize that rickrolling has reached the formal academic world. We design and conduct a systematic experiment to survey rickrolling in the academic literature. As of March 2022, there are 23 academic documents intentionally rickrolling the reader. Rickrolling happens in footnotes, code listings, references. We believe that rickrolling in academia proves inspiration and facetiousness, which is healthy for good science. This original research suggests areas of improvement for academic search engines and calls for more investigations about academic pranks and humor.