Analyzing Bias in False Refusal Behavior of Large Language Models for Hate Speech Detoxification
Kyuri Im, Shuzhou Yuan, Michael Färber
While large language models (LLMs) have increasingly been applied to hate speech detoxification, the prompts often trigger safety alerts, causing LLMs to refuse the task. In this study, we systematically investigate false refusal behavior in hate speech detoxification and analyze the contextual and linguistic biases that trigger such refusals. We evaluate nine LLMs on both English and multilingual datasets, our results show that LLMs disproportionately refuse inputs with higher semantic toxicity and those targeting specific groups, particularly nationality, religion, and political ideology. Although multilingual datasets exhibit lower overall false refusal rates than English datasets, models still display systematic, language-dependent biases toward certain targets. Based on these findings, we propose a simple cross-translation strategy, translating English hate speech into Chinese for detoxification and back, which substantially reduces false refusals while preserving the original content, providing an effective and lightweight mitigation approach.
SACRED: A Faithful Annotated Multimedia Multimodal Multilingual Dataset for Classifying Connectedness Types in Online Spirituality
Qinghao Guan, Yuchen Pan, Donghao Li
et al.
In religion and theology studies, spirituality has garnered significant research attention for the reason that it not only transcends culture but offers unique experience to each individual. However, social scientists often rely on limited datasets, which are basically unavailable online. In this study, we collaborated with social scientists to develop a high-quality multimedia multi-modal datasets, \textbf{SACRED}, in which the faithfulness of classification is guaranteed. Using \textbf{SACRED}, we evaluated the performance of 13 popular LLMs as well as traditional rule-based and fine-tuned approaches. The result suggests DeepSeek-V3 model performs well in classifying such abstract concepts (i.e., 79.19\% accuracy in the Quora test set), and the GPT-4o-mini model surpassed the other models in the vision tasks (63.99\% F1 score). Purportedly, this is the first annotated multi-modal dataset from online spirituality communication. Our study also found a new type of connectedness which is valuable for communication science studies.
Understanding recovery in women with substance use disorder through interpretative phenomenological analysis
Amina Saleem, Elizabeth Schwaiger
Abstract Aims There has been a rise in the number of women who engage in risky use of substances. Although research explaining the difference between substance use practices of men and women exist, research understanding substance usage and recovery in women has not been conducted in Pakistan. Hence, the aim of the present study was to understand the experience of women who engage in risky use of substances, specifically the reasons they seek recovery. Design For this purpose, semi-structured interviews in the native language of the participants (i.e. Urdu) were conducted with five participants. The average time of the initial interview conducted was 32 min. A Purposive Sampling technique was used within the social constructivism framework. The interviews were transcribed in Urdu and translated into English by the primary researcher. The results were analyzed using Interpretative Phenomenological Analysis (IPA). The themes were first elicited manually and then further analyzed using the software Taguette. Participants The study was conducted on five participants recruited from a private sector rehabilitation centre. Results There were four themes elicited related to the reasons women seek recovery: wanting to care for their children, desiring a greater sense of self-worth, religion, and concerns for the impact on their family of origin. Findings Family, religion, children, and perception about self were leading factors that motivated women to seek recovery. Previous research has not investigated these factors, but this study has highlighted the importance of these factors in women’s recovery journey. Conclusion A single theme cannot be highlighted as the main factor for recovery. Instead, understanding the themes as a whole including the context is important to understand the process of recovery. The results of this study can help in raising awareness among the society and designing therapy protocols for women that take into account their reasons for moving toward recovery.
Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approach
Aditya Tomar, Rudra Murthy, Pushpak Bhattacharyya
Bias and stereotypes in language models can cause harm, especially in sensitive areas like content moderation and decision-making. This paper addresses bias and stereotype detection by exploring how jointly learning these tasks enhances model performance. We introduce StereoBias, a unique dataset labeled for bias and stereotype detection across five categories: religion, gender, socio-economic status, race, profession, and others, enabling a deeper study of their relationship. Our experiments compare encoder-only models and fine-tuned decoder-only models using QLoRA. While encoder-only models perform well, decoder-only models also show competitive results. Crucially, joint training on bias and stereotype detection significantly improves bias detection compared to training them separately. Additional experiments with sentiment analysis confirm that the improvements stem from the connection between bias and stereotypes, not multi-task learning alone. These findings highlight the value of leveraging stereotype information to build fairer and more effective AI systems.
The condensed homotopy type of a scheme
Peter J. Haine, Tim Holzschuh, Marcin Lara
et al.
We study a condensed version of the étale homotopy type of a scheme, which refines both the usual étale homotopy type of Friedlander-Artin-Mazur and the proétale fundamental group of Bhatt-Scholze. In the first part of this paper, we prove that this condensed homotopy type satisfies descent along integral morphisms and that the expected fiber sequences hold. We also provide explicit computations, for example, for rings of continuous functions. A key ingredient in many of our arguments is a description of the condensed homotopy type using the Galois category of a scheme introduced by Barwick-Glasman-Haine. In the second part, we focus on the fundamental group of the condensed homotopy type in more detail. We show that, unexpectedly, the fundamental group of the condensed homotopy type of the affine line $\mathbf{A}^1_{\mathbf{C}}$ over the complex numbers is nontrivial. Nonetheless, its Noohi completion recovers the proétale fundamental group of Bhatt-Scholze. Moreover, we show that a mild correction, passing to the quasiseparated quotient, fixes most of this group's quirks. Surprisingly, this quotient is often a topological group.
The Correction of Two Mistakes in the Biography of Mohammad Hossein Hosseini-e Shirazi, Poet and Mystic in Qajar Era
Masume Sadeqi, Mahin Panahi, Mahdi Nikmanesh
IntroductionThe effort of researchers in biography writing has led to a better knowledge of poets’ lives and works, but sometimes there are mistakes in the information. Therefore, it is necessary to review them to create a secure foundation by correcting errors or eliminating deficiencies. One of them is about Hosseini-e Shirazi.Hosseini Shirazi was a poet and mystic in Qajar era. He studied with his father, but the superficial sciences were little for him. He traveled searching for true knowlege and a spiritual teacher until he became Mirza Abo al-Qasem Sokut’s disciple in Shiraz and gained grace. After the death of Hosseini's father and the master, he became the successor and started preaching. Hosseini also composed Khamseh/Panj-Ganj (Vamaq and Azra, Mehr and Mah, Oshtornameh, Ilahinameh, and Vasf al-Hal). He also had a poem collection that is unavailable now. He died in 1249 AH and was buried in Shiraz.In the research about Hosseini's life and works, the authors found that Hosseini's birth year and mystical orientation were wrongly recorded in some biographies and books. There are different sayings about his hometown. In this research, an attempt has been made to answer the following questions: why is Hosseini's birth year in some books and certificates incorrect? Based on Hosseini's works, how can we find it and what are the opinions about his birthplace? Also, how close to the truth is Hosseini's attribution to Nematullahi in some newer texts? In the older sources and biographies, do they consider him a main member or follower of Nematullahi? In Hosseini's works, what signs and reasons are there to reject or prove this issue?Literature ReviewLimited research works have been published about Hosseini:Khamseh-e Hosseini (1324 AH), lithography of the first three Mathnavis of Khamseh, by Abdolhosein Zolreyasatein (1251-1332 AH). The correction of four mathnavis of Vamaq and Azra (1382), Mehr and Mah (1386), Oshtornameh (1386), and Vasf al-Hal (1392) from Khamseh-e Hosseini along with an introduction about the poet by Kavoos Hassanli and Kavoos Rezaei. Vasf al-Hal is the poet’s autobiography in verse, which is useful for knowing the his life. Hassanli and Rezaei have also written articles about Hosseini’s mathnavis. Jooya Jahanbakhsh has edited “The Explanation of Mowlavi’s Prose Prologues in Mathnavi” and added an explanation to that. “Introducing Ilahinameh by Hosseini-e Shirazi and the Influence of Mathnavi-e Ma’navi on It” (1401) is written by Masumeh Sadeqi and Mahin Panahi. The only PhD thesis about Hosseini is by Masumeh Sadeqi entitled Manuscript Correction and Structural Analysis of Ilahinameh Composed by Mohammad Hossein Hosseini-e Shirazi. Also, some literary biographies contain a brief description of Hosseini's biography and parts of his poems.In the introductions written by the editors on the four mathnavis of Vamaq and Azra, Mehr and Mah, Oshtornameh, and Vasf al-Hal, Hosseini's birth year is 1184 AH and his age is 65. In the articles, 1184 is recorded but the researches show that incorrect. Hosseini's birthplace is also mentioned Karbala, while in some sources it is Shiraz. In the introduction to the revision of Oshtornameh and Vasf al-Hal citing Persian Poets and Scientists (Roknzadeh Adamiyat, 1337), Hosseini is considered “the head of the Akhawan Nematullahi dynasty”, while studies show it incorrect, too. MethodologyIn this research, with a descriptive-analytical and critical method, the poet's birth year has been determined by enumerating the definitive reasons with a careful examination of the first-hand sources about the poet's life and the documents obtained from Hosseini's works, especially Ilahinameh and Vasf al-Hal and the origin of the mistakes recorded in some biographies and books. To calculate the year, attention was paid to the chronograms of Ilahinameh and other signs placed in the text. The existing statements and the differences in the statements about the poet’s birthplace and the attribution of Hosseini to Nematullahi with solid reasons closer to the truth are mentioned. DiscussionThe year of Hosseini's birth is not mentioned in these works: Riaz al-Arefin, Majma al-Fosaha, Hadiqat al-Shoara, Tarayeq al-Haqayeq, Hosseini's Khamseh lithographic preface, Farsnameh Naseri, Al-Dhariyya, Reihanat al-Adab. Ebn-e Yusuf Shirazi, in The Manuscripts List of the Sepah-Salar High School Library, based on the chronogram of Ilahinameh and the age of the poet mentioned in the verses near the end of Ilahinameh considers the birth year to be 1184 AH; the mistake in the calculation of the numbers corresponding to the chronogram is quite obvious here and the most important reason for this mistake. According to the chronogram at the end of Ilahinameh, which indicates the year 1239, and the mentioning of 45 as the age of the poet, about 200 verses before the end, and also the record of 1237 as the date of composition at the end of the manuscript of Ilahinameh, it can be said that the poet’s birthdate is 1192 or 1194. The final verses of Ilahinameh mourning Abu al-Qasem Sokut, show that 1194 is more accurate.Hosseini's birthplace is recorded as Karbala in most of the biographies, but the poet spoke in such a way that it can be assumed to be Shiraz. Both DivanBeygi in Hadiqat al-Sho’ara and based on it Hassan Emdad in The Image of Persian Poets in a Thousand Years have recorded it as Shiraz; obviously, these are not enough reasons to prove that.Regarding Hosseini's mystical orientation, from the biography texts, only in Persian Poets and Scientists, Hosseini has been called “the head of the Nematullahi dynasty”. Hassanli and Rezaei have also mentioned it in the introduction to the revision of Oshtornameh and Vasf-al-Hal; however, none of the authors among the elders of the Nematullahi in other reliable sources such as Riaz al-Arefin, preface of Khamseh-e Hosseini and Tarayeq al-Haqayeq mention Hosseini as the head of the Nematullahi and they all introduce Hosseini as Abo al-Qasem Sokut’s disciple.According to Hosseini's statement in his works – and biographies – he had remained faithful in the devotion of Abo al-Qasem until his death. In the works written about Sokut, there have been hints about him as a member of Noorbakhshiya dynasty or Owaisiya and it is not far from the mind that Hosseini was also on the way of his master, or – as they have also said about Sokut – he was not a follower of any dynasty. Nematullahi’s followers were close to the Hosseini family, and perhaps the mistake of attributing Hosseini to Nematullahi was for that.ConclusionHosseini-e-Shirazi, a mystical poet of the Qajar era, was born in 1194 AH, according to the signs he placed in his works, especially Ilahinameh and Vasf al-Hal. He adhered to the principles of the Sharia, a mystic and the Twelver Shiite religion, and he was Abo al-Qasem Sokut’s disciple and remained loyal to him until his death.In the end, due to the existence of some mistakes and deficiencies in the registration and recording of information in the biographies, the necessity of revising the biographies and literary research about the lives of poets and writers is emphasized.
CLASP: Contrastive Language-Speech Pretraining for Multilingual Multimodal Information Retrieval
Mohammad Mahdi Abootorabi, Ehsaneddin Asgari
This study introduces CLASP (Contrastive Language-Speech Pretraining), a multilingual, multimodal representation tailored for audio-text information retrieval. CLASP leverages the synergy between spoken content and textual data. During training, we utilize our newly introduced speech-text dataset, which encompasses 15 diverse categories ranging from fiction to religion. CLASP's audio component integrates audio spectrograms with a pre-trained self-supervised speech model, while its language encoding counterpart employs a sentence encoder pre-trained on over 100 languages. This unified lightweight model bridges the gap between various modalities and languages, enhancing its effectiveness in handling and retrieving multilingual and multimodal data. Our evaluations across multiple languages demonstrate that CLASP establishes new benchmarks in HITS@1, MRR, and meanR metrics, outperforming traditional ASR-based retrieval methods that rely on transcribing speech into text for subsequent text retrieval, especially in specific scenarios.
BanStereoSet: A Dataset to Measure Stereotypical Social Biases in LLMs for Bangla
Mahammed Kamruzzaman, Abdullah Al Monsur, Shrabon Das
et al.
This study presents BanStereoSet, a dataset designed to evaluate stereotypical social biases in multilingual LLMs for the Bangla language. In an effort to extend the focus of bias research beyond English-centric datasets, we have localized the content from the StereoSet, IndiBias, and Kamruzzaman et. al.'s datasets, producing a resource tailored to capture biases prevalent within the Bangla-speaking community. Our BanStereoSet dataset consists of 1,194 sentences spanning 9 categories of bias: race, profession, gender, ageism, beauty, beauty in profession, region, caste, and religion. This dataset not only serves as a crucial tool for measuring bias in multilingual LLMs but also facilitates the exploration of stereotypical bias across different social categories, potentially guiding the development of more equitable language technologies in Bangladeshi contexts. Our analysis of several language models using this dataset indicates significant biases, reinforcing the necessity for culturally and linguistically adapted datasets to develop more equitable language technologies.
MAFIA: Multi-Adapter Fused Inclusive LanguAge Models
Prachi Jain, Ashutosh Sathe, Varun Gumma
et al.
Pretrained Language Models (PLMs) are widely used in NLP for various tasks. Recent studies have identified various biases that such models exhibit and have proposed methods to correct these biases. However, most of the works address a limited set of bias dimensions independently such as gender, race, or religion. Moreover, the methods typically involve finetuning the full model to maintain the performance on the downstream task. In this work, we aim to modularly debias a pretrained language model across multiple dimensions. Previous works extensively explored debiasing PLMs using limited US-centric counterfactual data augmentation (CDA). We use structured knowledge and a large generative model to build a diverse CDA across multiple bias dimensions in a semi-automated way. We highlight how existing debiasing methods do not consider interactions between multiple societal biases and propose a debiasing model that exploits the synergy amongst various societal biases and enables multi-bias debiasing simultaneously. An extensive evaluation on multiple tasks and languages demonstrates the efficacy of our approach.
On the Use of Proxies in Political Ad Targeting
Piotr Sapiezynski, Levi Kaplan, Alan Mislove
et al.
Detailed targeting of advertisements has long been one of the core offerings of online platforms. Unfortunately, malicious advertisers have frequently abused such targeting features, with results that range from violating civil rights laws to driving division, polarization, and even social unrest. Platforms have often attempted to mitigate this behavior by removing targeting attributes deemed problematic, such as inferred political leaning, religion, or ethnicity. In this work, we examine the effectiveness of these mitigations by collecting data from political ads placed on Facebook in the lead up to the 2022 U.S. midterm elections. We show that major political advertisers circumvented these mitigations by targeting proxy attributes: seemingly innocuous targeting criteria that closely correspond to political and racial divides in American society. We introduce novel methods for directly measuring the skew of various targeting criteria to quantify their effectiveness as proxies, and then examine the scale at which those attributes are used. Our findings have crucial implications for the ongoing discussion on the regulation of political advertising and emphasize the urgency for increased transparency.
Open Questions about the Visualization of Sociodemographic Data
Florent Cabric, Margrét Vilborg Bjarnadóttir, Anne-Flore Cabouat
et al.
This paper collects a set of open research questions on how to visualize sociodemographic data. Sociodemographic data is a common part of datasets related to people, including institutional censuses, health data systems, and human-resources fles. This data is sensitive, and its collection, sharing, and analysis require careful consideration. For instance, the European Union, through the General Data Protection Regulation (GDPR), protects the collection and processing of any personal data, including sexual orientation, ethnicity, and religion. Data visualization of sociodemographic data can reinforce stereotypes, marginalize groups, and lead to biased decision-making. It is, therefore, critical that these visualizations are created based on good, equitable design principles. In this paper, we discuss and provide a set of open research questions around the visualization of sociodemographic data. Our work contributes to an ongoing refection on representing data about people and highlights some important future research directions for the VIS community. A version of this paper and its fgures are available online at osf.io/a2u9c.
MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models
Sepehr Janghorbani, Gerard de Melo
Recent breakthroughs in self supervised training have led to a new class of pretrained vision language models. While there have been investigations of bias in multimodal models, they have mostly focused on gender and racial bias, giving much less attention to other relevant groups, such as minorities with regard to religion, nationality, sexual orientation, or disabilities. This is mainly due to lack of suitable benchmarks for such groups. We seek to address this gap by providing a visual and textual bias benchmark called MMBias, consisting of around 3,800 images and phrases covering 14 population subgroups. We utilize this dataset to assess bias in several prominent self supervised multimodal models, including CLIP, ALBEF, and ViLT. Our results show that these models demonstrate meaningful bias favoring certain groups. Finally, we introduce a debiasing method designed specifically for such large pre-trained models that can be applied as a post-processing step to mitigate bias, while preserving the remaining accuracy of the model.
Decolonial AI Alignment: Openness, Viśe\d{s}a-Dharma, and Including Excluded Knowledges
Kush R. Varshney
Prior work has explicated the coloniality of artificial intelligence (AI) development and deployment through mechanisms such as extractivism, automation, sociological essentialism, surveillance, and containment. However, that work has not engaged much with alignment: teaching behaviors to a large language model (LLM) in line with desired values, and has not considered a mechanism that arises within that process: moral absolutism -- a part of the coloniality of knowledge. Colonialism has a history of altering the beliefs and values of colonized peoples; in this paper, I argue that this history is recapitulated in current LLM alignment practices and technologies. Furthermore, I suggest that AI alignment be decolonialized using three forms of openness: openness of models, openness to society, and openness to excluded knowledges. This suggested approach to decolonial AI alignment uses ideas from the argumentative moral philosophical tradition of Hinduism, which has been described as an open-source religion. One concept used is viśe\d{s}a-dharma, or particular context-specific notions of right and wrong. At the end of the paper, I provide a suggested reference architecture to work toward the proposed framework.
Moderation Patterns of Pesantren in Indonesia: A Study on the Perceptions and Responses of Kyai, Teachers and Santri
Andy Hadiyanto, Yusuf Hanafi, Rudy Muhamad Barnannsyah
et al.
Religious moderation is actually an effort to restore understanding and application of religious teachings that are sincere, friendly, and oriented towards improving the quality of life and humanity. The realization of religious moderation is expected to further strengthen the role of religion as the spirit of civilization. Pesantren as one of the oldest religious educational institutions in Indonesia, from the beginning has played a role in creating Indonesian Islamic civilization with its characteristics that are open, tolerant, and humanist. Thus, pesantren is an Islamic educational institution that has long played a role in mainstreaming moderation in Indonesia. In line with that, this study seeks to dig deeper into the mode of religious moderation that is maintained in the pesantren environment based on the perceptions and roles of the main elements of the pesantren, such as kyai, teachers, and santri. This study uses a qualitative approach by conducting a literature study and documentation of partners which can be used as a valid data source. Theoretically, the mode of religious moderation in pesantren can be seen from theological aspects, sharia, state politics, and relations with others. The results show that the manifestation of religious moderation in the theological aspect which is manifested by the aqidah ahlussunnah wal Jama'ah (aswaja), is related to the problem of human and God relations, the limitations of Mu'min Kafir, and the position of reason and revelation. In the Sharia aspect, religious moderation appears in the acceptance of mu'tabarah schools of jurisprudence, although there are still Shafii schools and reject mixing (talfiiq). In the aspect of state politics, religious moderation of kyai, ustaz, and santri can be seen in their acceptance of the Unitary State of the Republic of Indonesia (NKRI) and Pancasila as the basis of the state, and in them against the politicization of religion. As for the relationship with the moderating attitude of pesantren exponents from their acceptance of the concept seen as different attitudes and readiness to blend in without having to negate each other and lose their identity.
The effect of socioeconomic factors on quality of life of elderly in Jaffna district of Sri Lanka
Sathees Santhalingam, Sivayogan Sivagurunathan, Shamini Prathapan
et al.
Globally, the proportion of the elderly is increasing. In comparison to other Southeast Asian countries, Sri Lanka’s population is rapidly aging. The elderly are a vulnerable age group that requires special attention to live a long and healthy life. As, there was a scarcity of data on the elderly’s quality of life, studying the level of quality of life and the associated factors of the elderly in the Jaffna district will provide insight into how to plan interventions to improve the elderly’s overall well-being in Jaffna District and Sri Lanka as well. The study aimed to determine the quality of life of the elderly in the Jaffna district of Sri Lanka and to study the association of socioeconomic factors with the quality of life. This cross-sectional study was conducted among 813 community-dwelling elderly in the Jaffna District of Sri Lanka. Socio-economic characteristics were recorded by way of a structured questionnaire. The WHOQOL-Bref questionnaire was used to assess quality of life in four domains: physical health, psychological, social participation and the environment. The statistical Package of Social Science Software (SPSS) version 21 was used to analyse the data. Univariate, bivariate, and multivariate analyses were applied, p-value less than 0.05 was considered statistically significant. Among the four QOL domains, the mean (SD) score for an environmental domain was (12.1±2.1), (12.0±2.8) for the psychological domain, (11.8±2.3) for the physical health domain, and (10.1±3.0) for the social relationship domain. Factors significantly associated with all domains of QOL included marital status, level of education, living arrangement, employment, level of income, income adequacy and ownership of the house. Furthermore, age, sex, religion, number of children, and presence of monthly income, were significantly associated with at least one domain of QOL of the elderly. According to these findings, the QOL of the elderly in the Jaffna district of Sri Lanka seems low. And it was associated with multiple socio-economic factors. Interventions to improve the QOL of the elderly are anticipated with a higher emphasis on social relationship for the elderly.
Public aspects of medicine
Acceptance of evolution by high school students: Is religion the key factor?
Graciela da Silva Oliveira, Giuseppe Pellegrini, Leonardo Augusto Luvison Araújo
et al.
The idea of biological evolution is not accepted by many people around the world, with a large disparity amongst countries. Some factors may act as obstacles to the acceptance of evolution, such as religion, a lack of openness to experience, and not understanding the nature of science. Although the strength of the association between evolution acceptance and non-scientific factors varies among studies, it is often assumed that resistance to evolution is the byproduct of a religious background. Some studies are even more specific and try to associate the acceptance of evolution with precise religious affiliations. We aimed to explore the strength of associations among nationality, religion, and the acceptance of evolution by students using multiple correspondence analysis (MCA) and statistical tools, with nationwide samples from two different countries. Here, we show that wider sociocultural factors predict the acceptance of evolution to a higher degree than a religious background. We carried out two nationwide data collections that allowed us to compare differences in the acceptance of evolution in Italy and Brazil by high school students who declare to belong to the same religion in the two countries. Roman Catholic students showed significant differences between the two countries, and the gap between them was wider than between Catholics and non-Catholic Christians within Brazil. Our conclusions support those who argue that religious affiliation is not the main factor in predicting the level of evolution acceptance. The sociocultural environment and the level of evolutionary knowledge seem to be more important in this regard. These results open up new interpretative perspectives and provide a better understanding of attitudes towards evolution.
TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes
Sherzod Hakimov, Gullal S. Cheema, Ralph Ewerth
The detection of offensive, hateful content on social media is a challenging problem that affects many online users on a daily basis. Hateful content is often used to target a group of people based on ethnicity, gender, religion and other factors. The hate or contempt toward women has been increasing on social platforms. Misogynous content detection is especially challenging when textual and visual modalities are combined to form a single context, e.g., an overlay text embedded on top of an image, also known as meme. In this paper, we present a multimodal architecture that combines textual and visual features in order to detect misogynous meme content. The proposed architecture is evaluated in the SemEval-2022 Task 5: MAMI - Multimedia Automatic Misogyny Identification challenge under the team name TIB-VA. Our solution obtained the best result in the Task-B where the challenge is to classify whether a given document is misogynous and further identify the main sub-classes of shaming, stereotype, objectification, and violence.
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
Tianxiang Sun, Junliang He, Xipeng Qiu
et al.
Automatic evaluation metrics are crucial to the development of generative systems. In recent years, pre-trained language model (PLM) based metrics, such as BERTScore, have been commonly adopted in various generation tasks. However, it has been demonstrated that PLMs encode a range of stereotypical societal biases, leading to a concern on the fairness of PLMs as metrics. To that end, this work presents the first systematic study on the social bias in PLM-based metrics. We demonstrate that popular PLM-based metrics exhibit significantly higher social bias than traditional metrics on 6 sensitive attributes, namely race, gender, religion, physical appearance, age, and socioeconomic status. In-depth analysis suggests that choosing paradigms (matching, regression, or generation) of the metric has a greater impact on fairness than choosing PLMs. In addition, we develop debiasing adapters that are injected into PLM layers, mitigating bias in PLM-based metrics while retaining high performance for evaluating text generation.
Counterfactual Multi-Token Fairness in Text Classification
Pranay Lohia
The counterfactual token generation has been limited to perturbing only a single token in texts that are generally short and single sentences. These tokens are often associated with one of many sensitive attributes. With limited counterfactuals generated, the goal to achieve invariant nature for machine learning classification models towards any sensitive attribute gets bounded, and the formulation of Counterfactual Fairness gets narrowed. In this paper, we overcome these limitations by solving root problems and opening bigger domains for understanding. We have curated a resource of sensitive tokens and their corresponding perturbation tokens, even extending the support beyond traditionally used sensitive attributes like Age, Gender, Race to Nationality, Disability, and Religion. The concept of Counterfactual Generation has been extended to multi-token support valid over all forms of texts and documents. We define the method of generating counterfactuals by perturbing multiple sensitive tokens as Counterfactual Multi-token Generation. The method has been conceptualized to showcase significant performance improvement over single-token methods and validated over multiple benchmark datasets. The emendation in counterfactual generation propagates in achieving improved Counterfactual Multi-token Fairness.
Christianity on display: a semiotic study of two museums of world religions (Glasgow, Taipei)
Min-Hsiu Liao
This article regards museums of world religions as intersemiotic sites where the knowledge of individual religions as well as religion as a broad concept is socially constructed. It examines the role of verbal interpretations in co-constructing knowledge of religion with other visual and spatial semiotics. The case study is based on a comparison of the text panels and the display cases on Christianity in two museums: St Mungo Museum of Religious Life and Art (SMM) in Glasgow, and Museum of World Religions (MWR) in Taipei. The methodology combines the micro-level analysis of theme-rheme pattern in information progression, logical-semantic relations in verbal-visual interaction, and a pragmatic account of the two epistemic communities in which the museums are situated. The results suggest that through the interaction between the text panels, labels, and individual objects, each museum has construed its own material definition of religion. Specifically, Christianity is construed as a phenomenon perceived by Christians in SMM, whereas in MWR, the knowledge of Christianity develops from the holy scriptures.
Philosophy of religion. Psychology of religion. Religion in relation to other subjects, Communication. Mass media