FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Sewon Min, Kalpesh Krishna, Xinxi Lyu
et al.
Evaluating the factuality of long-form text generated by large language models (LMs) is non-trivial because (1) generations often contain a mixture of supported and unsupported pieces of information, making binary judgments of quality inadequate, and (2) human evaluation is time-consuming and costly. In this paper, we introduce FACTSCORE, a new evaluation that breaks a generation into a series of atomic facts and computes the percentage of atomic facts supported by a reliable knowledge source. We conduct an extensive human evaluation to obtain FACTSCOREs of people biographies generated by several state-of-the-art commercial LMs -- InstructGPT, ChatGPT, and the retrieval-augmented PerplexityAI -- and report new analysis demonstrating the need for such a fine-grained score (e.g., ChatGPT only achieves 58%). Since human evaluation is costly, we also introduce an automated model that estimates FACTSCORE using retrieval and a strong language model, with less than a 2% error rate. Finally, we use this automated metric to evaluate 6,500 generations from a new set of 13 recent LMs that would have cost $26K if evaluated by humans, with various findings: GPT-4 and ChatGPT are more factual than public models, and Vicuna and Alpaca are some of the best public models. FACTSCORE is available for public use via `pip install factscore`.
1194 sitasi
en
Computer Science
Data feminism
Natalia V. Kovalyova
All data have biographies: they are collected, cleaned, and complied before they could support powerful decisions. Every junction on this trajectory matters. Yet, according to Catherine D’Ignazio a...
On the origin of species by means of natural selection: Or the preservation of the favoured races in the struggle for life.
C. Darwin
5717 sitasi
en
History, Biology
Analyzing Qualitative Data
G. Gibbs
3216 sitasi
en
Sociology, Computer Science
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
Maria De-Arteaga, Alexey Romanov, Hanna M. Wallach
et al.
We present a large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives. We analyze the potential allocation harms that can result from semantic representation bias. To do so, we study the impact on occupation classification of including explicit gender indicators---such as first names and pronouns---in different semantic representations of online biographies. Additionally, we quantify the bias that remains when these indicators are "scrubbed," and describe proxy behavior that occurs in the absence of explicit gender indicators. As we demonstrate, differences in true positive rates between genders are correlated with existing gender imbalances in occupations, which may compound these imbalances.
553 sitasi
en
Computer Science, Mathematics
FMA: A Dataset for Music Analysis
Michaël Defferrard, Kirell Benzi, P. Vandergheynst
et al.
We introduce the Free Music Archive (FMA), an open and easily accessible dataset suitable for evaluating several tasks in MIR, a field concerned with browsing, searching, and organizing large music collections. The community's growing interest in feature and end-to-end learning is however restrained by the limited availability of large audio datasets. The FMA aims to overcome this hurdle by providing 917 GiB and 343 days of Creative Commons-licensed audio from 106,574 tracks from 16,341 artists and 14,854 albums, arranged in a hierarchical taxonomy of 161 genres. It provides full-length and high-quality audio, pre-computed features, together with track- and user-level metadata, tags, and free-form text such as biographies. We here describe the dataset and how it was created, propose a train/validation/test split and three subsets, discuss some suitable MIR tasks, and evaluate some baselines for genre recognition. Code, data, and usage examples are available at this https URL
589 sitasi
en
Computer Science
THE GO-ALONG AS ETHNOGRAPHIC RESEARCH TOOL
Margarethe Kusenbach
Archiving Uncertainty: Leveraging Crowdsourcing Methodology in Documenting Personal Experiences of COVID-19
Sanita Reinsone, Ilze Ļaksa-Timinska, Haralds Matulis
et al.
The COVID-19 pandemic, declared a global crisis by the World Health Organization on 11 March 2020, dramatically altered society’s daily routines, social interactions, and digital media consumption. This paper aims to evaluate how leveraging crowdsourcing techniques in combination with digital archiving strategies can effectively capture real-time autobiographical reflections during a period of crisis. The study focuses on the formation of the Latvian Pandemic Diary Collection, initiated through an open-call crowdsourcing initiative that was launched on 17 March 2020 to collect people’s recordings of their personal thoughts and emotional responses in this period of profound change and uncertainty. Methodologically, the project employed a versatile rapid-response crowdsourcing framework and dynamic digital archiving strategy, coupled with an intense outreach and social media campaign to encourage diary submissions.
The diary project has resulted in an open-access digital collection of 2,333 daily entries by 238 participants, offering unique insights into the societal shift towards digital modes of professional, social, and educational interaction in response to the pandemic’s constraints. Although the methodology applied proved effective in eliciting relatively widespread public engagement, difficulties became apparent in relation to the scalability of the approach and getting different segments of society to participate in the project through the crowdsourced-driven digital collection of autobiographical narratives in a rapidly changing social context.
Biography, Literature (General)
An Unpublished Article by Medievalist Historian V.T. Sirotenko About Gogol
Igor A. Vinogradov
Based on an authorized archival copy, an article about Gogol by medievalist historian Vasily Trofimovich Sirotenko (1915–2006), a professor at Perm, Dnepropetrovsk and Armavir universities, is being published. The article by V.T. Sirotenko, “The Early Middle Ages in the light of N.V. Gogol,” was written in 1978 and deposited in 1990 but is currently inaccessible to the reader. A professional historian notes Gogol’s extensive knowledge of the history of the European Middle Ages and evaluates the writer’s significant contribution to the study of world history. The article is a continuation of the works of numerous researchers and historians, F.A. Vitberg, S.A. Vengerov, G.P. Georgievsky, S.I. Mashinsky, M.P. Alekseev, V.F. Semenov and others, who noted, despite the cartoonish reviews of Gogol by some of his students at St. Petersburg University (in 1834–1835), the writer’s thorough historical erudition, the high scientific level of his university lectures, as well as works of art from world and national history. The publication contributes to the return into scientific circulation of an article by Professor V.T. Sirotenko, unknown to modern researchers, and can serve as a supplement to our article “The Unknown Book of N.V. Gogol 1834: Plan, Context, Reminiscences,” published in the first issue of the journal “Literary Fact” for 2022.
Literature (General), Slavic languages. Baltic languages. Albanian languages
As Camp as a Row of Pink Tents: Stephen’s Portrait of Mr W. S.
Samuel Slote
In the ‘Scylla and Charybdis’ episode of Ulysses, Stephen Dedalus presents a theory about Shakespeare’s biographical motivations for writing Hamlet, which he ultimately claims, perhaps disingenuously, to not believe. Stephen’s apparent disbelief in his own theory echoes Oscar Wilde’s ‘Portrait of Mr W. H.’, which is referenced within ‘Scylla’, and which also propounds a theory of Shakespeare’s artistic production in terms of his biography. Furthermore, like the various characters in Wilde’s story, Stephen’s theory is propelled primarily from the internal evidence of Shakespeare’s texts. In this article, I will analyse the playful and learned insincerity of both theories through the optic of camp in order to tease out the implications that Stephen’s argument about Shakespeare has for James Joyce’s aesthetics.
History of scholarship and learning. The humanities
The Elements of Tale in Contemporary Iraqi Painting (Khlaif Mahmoud As a Model)
Nour Khalif Mahmoud
The research dealt with the elements of the tale in contemporary Iraqi painting (Khlaif Mahmoud as a model). Where the research problem was identified by answering the following question: - Are the elements of the tale achieved in the works of the artist Khalif Mahmoud?
The research included four chapters, the first chapter dealt with the research problem, the research objective, and their objective, temporal and spatial limits. It followed by the terminology and procedural definition.
Chapter two included the theoretical framework that contained three sections. The first dealt with the concept of the tale and its components, the second one dealt with the tale from linguistic narration to narration in contemporary Iraqi painting, while the third dealt with the foundations of the tale in painting and its perspectives for the artist Khlaif Mahmoud.
Chapter three dealt with a presentation of the research procedures. As for chapter four, it included what was achieved of research procedures to reach the goals in light of the analysis of the approved samples, and the results came in which the artist relied on the style of modern realism and expressionism related to the popular heritage (the tale). The researcher also reached a number of conclusions, including that the tale is the product of a popular and cultural heritage that has been inherited by generations since the inception of creation and ended with the path and the biography of the artist.
Last fight of Alyosha Popovich
M. N. Kozlov, E. E. Boitsova
Importance. Despite the fact that in our time the interest of Russian scientists in the history of Ancient Rus has significantly increased, some of its pages remain almost unexplored. For example, Alyosha Popovich, repeatedly mentioned in Ancient Rus’ chroniques and bylinas, remains for many modern researchers only a folklore character. Meanwhile, a detailed analysis of the key moments of the biography of the historical prototype of the bylina’s bogatyr will allow modern scientists to more deeply explore the historical processes that took place in the Ancient Rus’ society of the pre-Mongol era.Materials and methods. In the course of the research, special methods of cognition (historicalanalytical and problem-chronological) were used. The materials that form the basis of the research have been studied and analyzed taking into account the chronology of events, the need to obtain historical information from the scientific sources being studied. When comparing different theoretical views on the analyzed problems, the method of comparative and retrospective analysis was used. Chroniques and chronographs of the 15th–17th centuries, as well as Ancient Rus’ bylinas, the main character of which was Alyosha Popovich, became the basis of the source base of the research.Results and Discussion. Based on the analysis of Ancient Rus’ chroniques and chronographs, as well as the text of the folk bylina “Alyosha Popovich and Tugarin the Serpent”, the authors refuted the hypothesis that recently dominated in Russian science that Alyosha Popovich is a collective image of the ancient defenders of native land. The historical reconstruction of the biography of the most probable historical prototype of the bylina’s bogatyr is also presented.Conclusion. The main historical prototype of bylina’s bogatyr Alyosha Popovich was the Rostov warrior Alexander Popovich. He served in the druzhinas of the Old Russian knyazs Vsevolod Yuryevich and Konstantin Vsevolodovich and under their command participated in military campaigns and internecine wars.
Education (General), Philology. Linguistics
„Je est un autre“
Alfred Prédhumeau
France and the French play an important role in Döblin’s biography, as his adoption of French nationality shows. This contribution, which is based on a few works from the exile period, will show how the author’s relationship to French alterity is reflected in his literary production. Our reading will focus on the relevant passages from the Babylonische Wandrung, the Schicksalsreise and November 1918. Compared to the earlier texts published between 1917 and 1921, we can see that Döblin slowly underwent a significant paradigm shift: his changed French view on Germany is characterized by feelings of alterity; he himself had become an Other.
Locating the Storyteller in Silent Waters: Sabiha Sumar’s Cinematic Tale of Shared Histories and Divided Identities
Rahat Imran
In her multi award-winning feature film Silent Waters (2003), Pakistani woman filmmaker Sabiha Sumar connects the socio-political traumas of the Partition of India and creation of Pakistan (1947) with the onset of military dictator General Zia-ul-Haq’s Islamization period (1977-1988) in Pakistan. Presenting a story based on real-life events, the film focuses on the impact of religious fundamentalism and nationalism on women in particular. Examining Silent Waters as an example of “history on film/film on history” (Rosenstone 2013), and film as an “agent, product, and source of history” (Ferro 1983), the discussion identifies and analyzes the filmmaker’s own tacitly embedded location and participation in the filmic narrative as an experiential ‘auto/bio-historiographer’, arguing for the value of this new paradigm in Cinema Studies.
Visual arts, Communication. Mass media
Nicola Abbagnano, un padre fondatore della sociologia italiana nel secondo dopoguerra
Alberto Baldissera
Nicola Abbagnano (1901-1990) was the greatest Italian historian of philosophy in twentieth century and one of the greatest in Europe and the West. In his memory Rosanna Panelli Marvulli dedicated an extensive biography. With Franco Ferrarotti, Abbagnano founded “Quaderni di Sociologia”, this journal, in 1951. Both had intense contacts with American philosophical and sociological culture. The aim was to claim the legitimacy of sociology as a social science; reject the criticisms of Benedetto Croce and the Marxists; outline a “rigorously individualistic conception of social life”; reject any determinism, therefore positivist and Marxist theories. It was not by chance that the initiative was heavily criticized by philosophers with a spiritualistic background. Although not a sociologist, Abbagnano was not only, more than others, a great supporter of Italian sociology after the Second World War. It has also guided its development. For this reason he should be recognized, more than others, as a founding father of this discipline in Italy.
Professor M. V. Vynokurov: stages of freight wagons creation (1930-1950)
Nataliia Ustiak
The urgent task of modern historical science is the comprehensive study of the personalities of prominent scientists and engineers who made a significant contribution to the formation of science and technology. The article is devoted to the analysis of the activity of professor M. V. Vynokurov in the field of the creation of freight wagons. In the history of science and technology. M. V. Vynokurov pointed out that it had been decided to produce new wagons using light-alloy steel, which greatly reduced the weight of the dead load. Such a solution was based on the technical experience of the US wagon industry. Particular attention was paid to the unification of the parts and assemblies subjected to the process of wear and damage to the most, were replaced with the current and periodic repairs of the wagon. This important measure has significantly reduced the cost of manufacturing and was important for the organization of repair because it simplified the ability to replace worn parts with spare ones. M. V. Vynokurov is known as a prominent specialist in the field of rolling stock, he devoted his life to the development of carriages and rolling stock. In preparing this article, chronological, typological, comparative methods of historical knowledge, classification and systematization of historical sources and bibliographic material were used that allowed to systematize and critically evaluate the sources used in relation to the question of the stages of the creation of freight wagons. The role of professor M. V. Vynokurov in this process is shown, covering the 1930-1950 years, the most productive years of a scientist and engineer. It was established that due to the personal contribution of M. V. Vynokurov in the unification of parts for different types of freight wagons, there was a reduction in operating costs of railways. This was achieved due to the massive production of various types of freight wagons. Study of the development of freight wagons through the biography of the scientist-engineer M. V. Vynokurov, which is an integral part of the complex of knowledge, implies the application of a systematic approach as a methodological means of scientific knowledge.
History (General) and history of Europe, Science (General)
Watching God in the electronic noise: Philip K. Dick, Bill Viola, and the (video) art of speculative fiction
Mattia Petricola
This study represents as an attempt to make sense of a rather curious coincidence. In Philip K. Dick's novel A scanner darkly (1973-1977), a character named Tony Amsterdam experiences a mystical vision after an acid trip. God appears to him in the form of «showers of colored sparks, like when something goes wrong with your TV set». In the same year 1973, a young Bill Viola creates Information, a work of video art featuring showers of colored sparks that are eerily similar to the ones seen by Tony Amsterdam.
On the one hand, this article investigates the cultural substratum that the two works have in common vis-à-vis their construction and use of abstract video; it is upon such a substratum that this bizarre coincidence depends. On the other hand, it aims to explore how the speculative use of screens and telecommunication technologies can mediate the religious, the sacred, and, most of all, the mystical.
The first two sections examine Tony Amsterdam’s experience in the context of A scanner darkly, discussing its relation to mysticism, to other episodes in the novel and to Dick’s biography. In section three, I interpret Information as a self-reflexive work of meta-video, proposing to read its aesthetic effect as the result of an electronic ouroboros: a circular video system in which input and output are coincident. Section four bridges the two works by reading them in the light of Jeffrey Sconce’s notion of ‘electronic elsewhere’ (2000) and John Durham Peter’s reflections on the idea of communication (1999). Finally, section five proposes a parallel between the semiotics of visual perception in A scanner darkly and the semiotics of Viola’s abstract video art
Geography. Anthropology. Recreation, Language. Linguistic theory. Comparative grammar
«Do rei exemplar: alguns ecos bíblicos na oratoria do Lancelot en prose»
Rafaela Câmara Simões da Silva
Resumen: O Lancelot en Prose, extenso romance que desenvolve a biografia de Lancelot, insere-se no grande ciclo de romances arturianos escrito em França durante a década de vinte do séc. XIII. Este texto testemunha, no nosso entender, uma clara exploração das potencialidades bíblicas na literatura medieval. A nossa análise incidirá essencialmente num momento discursivo que consideramos fundamental no romance, a repreensão dirigida a Artur por um preudome que se apresenta perante a corte do rei. Procuraremos identificar, através do confronto entre este excerto do Lancelot en Prose e as Escrituras, quais os elementos bíblicos nele retido e de que modo foram tratados. Esperaremos poder, assim, averiguar quais as intenções autorais no uso da fonte bíblica.
Palabras clave: Lancelot en Prose, Bíblia, teoria da realeza, romance arturiano.
Abstract: Lancelot en Prose, an extensive romance where Lancelot’s biography is developed, is integrated into the wide cycle of Arthurian romances that was written in France during the second decade of the thirteenth century. This text testifies, as long as we are concerned, a clear exploration of the biblical potentials in medieval literature. Our analysis will focus essentially on a discursive moment that we consider to be central to the romance: the reprimand directed to Arthur by a preudome that presents himself before the king’s court. We will try to identify, through the confrontation between the referred excerpt of Lancelot en Prose and the Scripture which are the elements withheld in it and how they were treated. By this confrontation we hope to ascertain the authorial intentions in the use of the biblical source.
Keywords: Lancelot en Prose, Bible, theory of kingship, Arthurian romance.
Medieval history, Language and Literature
The Islamic Ethics in the poetry of ‘Abdullah bin al-Mubarak
Muhammad Ismail bin Abdul Salam
‘Abdullah bin al-Mubark was born in Marw’ one of the prime cities in Khurasan, (nowadays in the surroundings of Afghanistan and Central Asia), in the year 118 AH. In addition to his many talents, achievements and abilities, ‘Abdullah bin al-Mubarak was also gifted in literacy, particularly in the art of poetry. He held an eloquent tongue which was recognized by all who conversed with him and his language displayed the nature of someone who had been taught well. Most of the poetry which has been recorded from him is actually his advice to others, whether they were close friends or high-ranking Caliphs and Rulers. The topics spoken of concerned the common issues which had arisen in his time (e.g. matters pertaining to theology, politics, the worldview, the community etc) and as always, they contained much wisdom and hence the books of history have sealed them and recorded them. This research article discussed Biography of Abdullah ibn Al Mubarak, The Islamic Ethics in his poetry, Impact of Rhetoric on his poetry with special concentration on the four kinds i.e. citation, impact of Quranic words, Quranic pictorial and Quranic style on his poetry.
Islam, Religions. Mythology. Rationalism
Valentim Fernandes and Jacobo Cromberger: the pioneers in the biography of portuguese copyright and law of literature in the portuguese language
Víctor Gameiro Drummond
This paper describes and critically analyzes the first attributions of privileges occurred in Portuguese-speaking countries in the 16th century, especially Portugal, as well as the reflections of these facts in the history of Copyright in this country and those that historically are related to it, which means historically the beginning of a law of literature in the Portuguese language.
French literature - Italian literature - Spanish literature - Portuguese literature, Law in general. Comparative and uniform law. Jurisprudence