Matthew Varona, Karen Bonilla, Maryam Hedayati
et al.
Research in visualization literacy explores the skills required to engage with visualizations. This state-of-the-art report surveys the current literature in visualization literacy to provide a comprehensive overview of the field. We propose a taxonomy of visualization literacy that organizes the field into competency themes and research categories. To address ambiguity surrounding the term ``visualization literacy'', we provide a framework for operationalizing visualization literacy based on application contexts (including domain, scenario, and audience) and relevant competencies, which are categorized under consumption, construction, critique, and connection. Research contributions are organized into five categories: ontology, assessment, mechanisms, populiteracy, and intervention. For each category, we identify key trends, discuss which competencies are addressed, highlight open challenges, and examine how advancements within these areas inform and reinforce each other, driving progress in the field.
This paper focuses on a key challenge in visual emotion understanding: given an art image, the model pinpoints pixel regions that trigger a specific human emotion, and generates linguistic explanations for it. Despite advances in general segmentation, pixel-level emotion understanding still faces a dual challenge: first, the subjectivity of emotion limits general segmentation models like SAM to adapt to emotion-oriented segmentation tasks; and second, the abstract nature of art expression makes it hard for captioning models to balance pixel-level semantics and emotion reasoning. To solve the above problems, this paper proposes the Emotion stimuli Segmentation and Explanation Model (EmoSEM) model to endow the segmentation framework with emotion comprehension capability. First, to enable the model to perform segmentation under the guidance of emotional intent well, we introduce an emotional prompt with a learnable mask token as the conditional input for segmentation decoding. Then, we design an emotion projector to establish the association between emotion and visual features. Next, more importantly, to address emotion-visual stimuli alignment, we develop a lightweight prefix adapter, a module that fuses the learned emotional mask with the corresponding emotion into a unified representation compatible with the language model. Finally, we input the joint visual, mask, and emotional tokens into the language model and output the emotional explanations. It ensures that the generated interpretations remain semantically and emotionally coherent with the visual stimuli. Our method realizes end-to-end modeling from low-level pixel features to high-level emotion interpretation, delivering the first interpretable fine-grained framework for visual emotion analysis. Extensive experiments validate the effectiveness of our model. Code will be made publicly available.
Visual Analytics (VA) integrates humans, data, and models as key actors in insight generation and data-driven decision-making. This position paper values and reflects on 16 VA process models and frameworks and makes nine high-level observations that motivate a fresh perspective on VA. The contribution is the HDMI Canvas, a perspective to VA that complements the strengths of existing VA process models and frameworks. It systematically characterizes diverse roles of humans, data, and models, and how these actors benefit from and contribute to VA processes. The descriptive power of the HDMI Canvas eases the differentiation between a series of VA building blocks, rather than describing general VA principles only. The canvas includes modern human-centered methodologies, including human knowledge externalization and forms of feedback loops, while interpretable and explainable AI highlight model contributions beyond their conventional outputs. The HDMI Canvas has generative power, guiding the design of new VA processes and is optimized for external stakeholders, improving VA outreach, interdisciplinary collaboration, and user-centered design. The utility of the HDMI Canvas is demonstrated through two preliminary case studies.
This article examines the portrayal of Father Colin MacInnes in the Scottish Gaelic documentary Gleanntan Ecuador (Jan Pester, 1994). A Catholic priest from South Uist in the Scottish Outer Hebrides, MacInnes moved to Ecuador in 1984 and settled in Comité del Pueblo, a shantytown located in the outskirts of Quito, where he spent over twenty years contributing to local initiatives such as the provision of running water, hospitals, and churches. Central to the film is MacInnes’ efforts to secure funding for a water supply project, achieved through transnational solidarity networks. This article constitutes the first academic approach to the film, bringing together studies in architecture and urban development, anthropology, ethnography, and religious and cultural studies. It argues that, by prioritising MacInnes as a spokesperson for the locals, the film inadvertently reproduces “civilisation and progress” tropes commonly associated with cinematic portrayals of religious missionaries in Ecuador and Latin America. This statement is supported by identifying onscreen binaries between the precariousness of the Ecuadorian township and the idyllic landscapes of the Scottish Isles, emphasising the charity of Scottish parishioners. The film also reiterates the hostility of local communist leaders, highlighting intimidation tactics and extorsion, which speak of its positionality within a postcommunist world.
A Cinemateca do Museu de Arte Moderna do Rio de Janeiro surgiu em um momento de intensas transformações no Brasil. Já em sua ata de fundação, o MAM Rio previa um setor destinado ao cinema, o que demonstra o entendimento de que a atividade cinematográfica é uma arte moderna. O presente artigo visa refletir o significado das ações de difusão da Cinemateca, em particular, as quatro edições da Mostra Internacional de Arte Cinematográfica organizadas por ela, a saber, os festivais A História do Cinema Americano, em 1958; A História do Cinema Francês, em 1959; A História do Cinema Italiano, em 1960, e A História do Cinema Russo e Soviético, em 1961-1962. A hipótese é que tais Mostras exerceram uma função fundamental na consolidação da Cinemateca, justamente por terem sido eventos de grande porte no cenário cultural da cidade, algo até então inédito no âmbito cinematográfico do Rio de Janeiro. Em seguida, tais Mostras foram transtornadas pelas reviravoltas políticas do país e pelas mudanças estéticas do próprio cinema. A pesquisa foi realizada no acervo da Cinemateca do MAM Rio, assim como em consulta a jornais da época.
Durante mucho tiempo las lesbianas fueron prácticamente invisibles en las ciudades de la península ibérica, difícilmente tenían lugares o cartografías, y apenas se las asociaba con imágenes o imaginarios. En el Estado Español, el franquismo contribuyó enormemente a esta invisibilización. Pero la ocupación del espacio urbano por una identidad es esencial para que ésta pueda construirse como sujeto político y social. A partir de la Transición, con los primeros grupos de activistas lesbofeministas, y durante las décadas de 80 y 90, las lesbianas empiezan un proceso de transformación de su condición de invisibilidad, acompañado de un impulso de autorrepresentación, en las calles de ciudades como Bilbao, Barcelona o Madrid. Estas fotografías se encuentran mayoritariamente en posesión de las fotógrafas y fotografiadas, en cajones y fondos de armario. En este artículo empezamos a sacarlas a la luz, a analizar y reflexionar sobre sus efectos hoy, buscando abrir contra imaginarios urbanos en la Península
Visual Question Answering (VQA) is a challenging task that requires systems to provide accurate answers to questions based on image content. Current VQA models struggle with complex questions due to limitations in capturing and integrating multimodal information effectively. To address these challenges, we propose the Rank VQA model, which leverages a ranking-inspired hybrid training strategy to enhance VQA performance. The Rank VQA model integrates high-quality visual features extracted using the Faster R-CNN model and rich semantic text features obtained from a pre-trained BERT model. These features are fused through a sophisticated multimodal fusion technique employing multi-head self-attention mechanisms. Additionally, a ranking learning module is incorporated to optimize the relative ranking of answers, thus improving answer accuracy. The hybrid training strategy combines classification and ranking losses, enhancing the model's generalization ability and robustness across diverse datasets. Experimental results demonstrate the effectiveness of the Rank VQA model. Our model significantly outperforms existing state-of-the-art models on standard VQA datasets, including VQA v2.0 and COCO-QA, in terms of both accuracy and Mean Reciprocal Rank (MRR). The superior performance of Rank VQA is evident in its ability to handle complex questions that require understanding nuanced details and making sophisticated inferences from the image and text. This work highlights the effectiveness of a ranking-based hybrid training strategy in improving VQA performance and lays the groundwork for further research in multimodal learning methods.
This Special Issue concerns the artistic practice of autotheory and its associated af-fordances and risks as undertaken by artists, art writers, and those interested in the stakes of the practice of autotheory [...]
Recently, uncertainty-aware deep learning methods for multiclass labeling problems have been developed that provide calibrated class prediction probabilities and out-of-distribution (OOD) indicators, letting machine learning (ML) consumers and engineers gauge a model's confidence in its predictions. However, this extra neural network prediction information is challenging to scalably convey visually for arbitrary data sources under multiple uncertainty contexts. To address these challenges, we present ScatterUQ, an interactive system that provides targeted visualizations to allow users to better understand model performance in context-driven uncertainty settings. ScatterUQ leverages recent advances in distance-aware neural networks, together with dimensionality reduction techniques, to construct robust, 2-D scatter plots explaining why a model predicts a test example to be (1) in-distribution and of a particular class, (2) in-distribution but unsure of the class, and (3) out-of-distribution. ML consumers and engineers can visually compare the salient features of test samples with training examples through the use of a ``hover callback'' to understand model uncertainty performance and decide follow up courses of action. We demonstrate the effectiveness of ScatterUQ to explain model uncertainty for a multiclass image classification on a distance-aware neural network trained on Fashion-MNIST and tested on Fashion-MNIST (in distribution) and MNIST digits (out of distribution), as well as a deep learning model for a cyber dataset. We quantitatively evaluate dimensionality reduction techniques to optimize our contextually driven UQ visualizations. Our results indicate that the ScatterUQ system should scale to arbitrary, multiclass datasets. Our code is available at https://github.com/mit-ll-responsible-ai/equine-webapp
Noptanit Chotisarn, Sarun Gulyanon, Tianye Zhang
et al.
The past decade has witnessed rapid progress in AI research since the breakthrough in deep learning. AI technology has been applied in almost every field; therefore, technical and non-technical end-users must understand these technologies to exploit them. However existing materials are designed for experts, but non-technical users need appealing materials that deliver complex ideas in easy-to-follow steps. One notable tool that fits such a profile is scrollytelling, an approach to storytelling that provides readers with a natural and rich experience at the reader's pace, along with in-depth interactive explanations of complex concepts. Hence, this work proposes a novel visualization design for creating a scrollytelling that can effectively explain an AI concept to non-technical users. As a demonstration of our design, we created a scrollytelling to explain the Siamese Neural Network for the visual similarity matching problem. Our approach helps create a visualization valuable for a short-timeline situation like a sales pitch. The results show that the visualization based on our novel design helps improve non-technical users' perception and machine learning concept knowledge acquisition compared to traditional materials like online articles.
Taxi drivers often take much time to navigate the streets to look for passengers, which leads to high vacancy rates and wasted resources. Empty taxi cruising remains a big concern for taxi companies. Analyzing the pick-up point selection behavior can solve this problem effectively, providing suggestions for taxi management and dispatch. Many studies have been devoted to analyzing and recommending hot-spot regions of pick-up points, which can make it easier for drivers to pick up passengers. However, the selection of pick-up points is complex and affected by multiple factors, such as convenience and traffic management. Most existing approaches cannot produce satisfactory results in real-world applications because of the changing travel demands and the lack of interpretability. In this paper, we introduce a visual analytics system, T-PickSeer, for taxi company analysts to better explore and understand the pick-up point selection behavior of passengers. We explore massive taxi GPS data and employ an overview-to-detail approach to enable effective analysis of pick-up point selection. Our system provides coordinated views to compare different regularities and characteristics in different regions. Also, our system assists in identifying potential pick-up points and checking the performance of each pick-up point. Three case studies based on a real-world dataset and interviews with experts have demonstrated the effectiveness of our system.
Visual arts refer to art experienced primarily through vision. 3D visual optical art is one of them. Artists use their rich imagination and experience to combine light and objects to give viewers an unforgettable visual experience. However, the design process involves much trial and error; therefore, it is often very time-consuming. This has prompted many researchers to focus on proposing various algorithms to simplify the complicated design processes and help artists quickly realize the arts in their minds. To help computer graphics researchers interested in creating 3D visual optical art, we first classify and review relevant studies, then extract a general framework for solving 3D visual optical art design problems, and finally propose possible directions for future research.
The application of data visualization in public art attracts increasing attention. In this paper, we present the design and implementation of a visualization method for sunlight data collected over a long period of time with an industrial camera. The proposed method makes use of the saturation and value information of collected sunlight image data in Hue Saturation Value color model to show the variation of the mood of the sunlight. Specifically, we create visual patterns with a rotating planet gear, which has an intuitively consistent geometric meaning with HSV color model and the planetary motion. Due to the variation of the sunlight data over time, the generated visual pattern presents a periodic variation that corresponds to the changing mood of the sunlight. Furthermore, we also use the sunlight data to generate music as another form of data representation. Two public artworks have been created with the above visualization and auralization methods and displayed on an exhibition held at China Resources Tower, Shenzhen, China. This work is a typical practice of creating public installations with data visualization technology, giving a glimpse into the many ways science and art intersect.
We present a machine learning system that can quantify fine art paintings with a set of visual elements and principles of art. This formal analysis is fundamental for understanding art, but developing such a system is challenging. Paintings have high visual complexities, but it is also difficult to collect enough training data with direct labels. To resolve these practical limitations, we introduce a novel mechanism, called proxy learning, which learns visual concepts in paintings though their general relation to styles. This framework does not require any visual annotation, but only uses style labels and a general relationship between visual concepts and style. In this paper, we propose a novel proxy model and reformulate four pre-existing methods in the context of proxy learning. Through quantitative and qualitative comparison, we evaluate these methods and compare their effectiveness in quantifying the artistic visual concepts, where the general relationship is estimated by language models; GloVe or BERT. The language modeling is a practical and scalable solution requiring no labeling, but it is inevitably imperfect. We demonstrate how the new proxy model is robust to the imperfection, while the other models are sensitively affected by it.
Gli archivi del Moderno costituiscono una preziosa fonte per indagare non solo l’evoluzione con cui il linguaggio grafico ha accompagnato e raccontato il progetto di architettura in un particolare momento della sua storia, ma, specie se riferiti all’orizzonte delle opere non realizzate, rappresentano anche uno straordinario strumento di lettura di quelle potenzialità, talvolta inespresse o di quelle utopie urbane che hanno segnato l’evoluzione del pensiero sulla città del XX secolo. Testimonianza preziosa in tal senso, è data dai disegni custoditi nell’Archivio privato di Frediano Frediani, che raccontano l’utopia urbana di una Napoli inedita, estremamente d’avanguardia e allo stesso tempo, profondamente radicata nella sua identità territoriale. Nel copioso materiale d’archivio qui si intende indagare su alcune architetture non realizzate di Frediani, in cui l’autore propone soluzioni sul waterfront che raccontano del rapporto del capoluogo campano con il mare, e che si inseriscono in un tratto di costa centrale nelle dinamiche urbane quotidiane.
Denise C. Nelson-Hurwitz, Michelle Tagorda, Uday Patil
et al.
Undergraduate students balance course requirements for the university, college, school, and major. Each set of requirements, including degree-specific curriculum, is intended to promote synergistic interaction of competence, skills, and knowledge, beyond serving as a collection of individual courses. Understanding of curriculum is important for program recruitment as undergraduate students are more informed when deciding between bachelor's degrees options. Among cohorted programs, this understanding is also helpful in communicating and promoting common intellectual experiences. Comprehension of curriculum is especially important for persistence when students are better able to articulate the connections between course and competencies needed to advance in coursework. To improve universal design for learning within program advising, visual curriculum maps were created as infographics to support student understanding of Bachelor of Arts in Public Health degree requirements and specific capstone course pathways. This map is printed as a small booklet and has been pilot tested among prospective students with positive feedback, then implemented in routine advising sessions. Visual maps of capstone requirements were well-received in concept, however constructive student feedback during pilot testing necessitated further revision. Student feedback also encouraged the application of culturally appropriate visuals and analogies to celebrate student diversity. Visual aids such as these may improve access to information among students through universal design, and also improve recruitment, retention efforts, and student buy-in to degree curricula.
In the aesthetic study, semiotic interpretation is an intellectual approach to deciphering the hidden meaning of an aesthetic achievement's taste. The study's goals include comprehending the artwork's structure, which necessitates an in-depth interpretation of the language by translating it into official symbols. This research method uses a linear formation sample with semantic attributes attached to the Kufic calligraphy of artworks. The findings reveal that the emergence of geometric lines in a linear composition has a decorative abstraction that may be used to argue that the artist intended to place it there and that this is a symbol of the cosmos and its existence. This study adds to our understanding of compositional insight in Arabic calligraphy as an expressive aesthetic expression that alludes to the text in the sense of meaning that lies underneath the direct meaning
While people with visual impairments are interested in artwork as much as their sighted peers, their experience is limited to few selective artworks that are exhibited at certain museums. To enable people with visual impairments to access and appreciate as many artworks as possible at ease, we propose an online art gallery that allows users to explore different parts of a painting displayed on their touchscreen-based devices while listening to corresponding verbal descriptions of the touched part on the screen. To investigate the scalability of our approach, we first explored if anonymous crowd who may not have expertise in art are capable of providing visual descriptions of artwork as a preliminary study. Then we conducted a user study with 9 participants with visual impairments to explore the potential of our system for independent artwork appreciation by assessing if and how well the system supports 4 steps of Feldman Model of Criticism. The findings suggest that visual descriptions of artworks produced by an anonymous crowd are sufficient for people with visual impairments to interpret and appreciate paintings with their own judgments which is different from existing approaches that focused on delivering descriptions and opinions written by art experts. Based on the lessons learned from the study, we plan to collect visual descriptions of a greater number of artwork and distribute our online art gallery publicly to make more paintings accessible for people with visual impairments.
This paper presents VisMCA, an interactive visual analytics system that supports deepening understanding in ML results, augmenting users' capabilities in correcting misclassification, and providing an analysis of underlying patterns, in response to the VAST Challenge 2020 Mini-Challenge 2. VisMCA facilitates tracking provenance and provides a comprehensive view of object detection results, easing re-labeling, and producing reliable, corrected data for future training. Our solution implements multiple analytical views on visual analysis to offer a deep insight for underlying pattern discovery.