Sebe Vanbrabant, Gustavo Rovelo Ruiz, Davy Vanacken
While the increased integration of AI technologies into interactive systems enables them to solve an increasing number of tasks, the black-box problem of AI models continues to spread throughout the interactive system as a whole. Explainable AI (XAI) techniques can make AI models more accessible by employing post-hoc methods or transitioning to inherently interpretable models. While this makes individual AI models clearer, the overarching system architecture remains opaque. This challenge not only pertains to standard XAI techniques but also to human examination and conversational XAI approaches that need access to model internals to interpret them correctly and completely. To this end, we propose conceptually representing such interactive systems as sequences of structural building blocks. These include the AI models themselves, as well as control mechanisms grounded in literature. The structural building blocks can then be explained through complementary explanatory building blocks, such as established XAI techniques like LIME and SHAP. The flow and APIs of the structural building blocks form an unambiguous overview of the underlying system, serving as a communication basis for both human and automated agents, thus aligning human and machine interpretability of the embedded AI models. In this paper, we present our flow-based approach and a selection of building blocks as MATCH: a framework for engineering Multi-Agent Transparent and Controllable Human-centered systems. This research contributes to the field of (conversational) XAI by facilitating the integration of interpretability into existing interactive systems.
Large Language Models (LLMs) have transformed software engineering, but their application to physical engineering domains remains underexplored. This paper evaluates LLMs' capabilities in high-powered rocketry design through RocketBench, a benchmark connecting LLMs to high-fidelity rocket simulations. We test models on two increasingly complex design tasks: target altitude optimization and precision landing challenges. Our findings reveal that while state-of-the-art LLMs demonstrate strong baseline engineering knowledge, they struggle to iterate on their designs when given simulation results and ultimately plateau below human performance levels. However, when enhanced with reinforcement learning (RL), we show that a 7B parameter model outperforms both SoTA foundation models and human experts. This research demonstrates that RL-trained LLMs can serve as effective tools for complex engineering optimization, potentially transforming engineering domains beyond software development.
This article introduces MCNP-GO (https://github.com/afriou/mcnpgo), a Python package designed to manipulate and assemble MCNP input files, allowing users to assemble a set of independent objects, each described by a valid MCNP file, into a single cohesive file. This tool is particularly useful for applications where precise modeling and positioning of equipment are crucial. The package addresses the challenges of managing large databases of MCNP input files, ensuring reliability and traceability through configuration management systems. MCNP-GO provides functionalities such as renumbering, extracting subsets of files, transforming files, and assembling files while managing collisions and materials. It also keeps track of the operations performed on files, enhancing traceability and ease of modification. The article demonstrates the package's capabilities through a practical example of assembling an MCNP input file for a tomographic experiment, highlighting its efficiency and user-friendliness. MCNP-GO is designed for users with minimal Python knowledge.
Often software engineering classes have the student concentrate on designing and planning the project but stop short of actual student team development of code. This leads to criticism by employers of new graduates that they are missing skills in working in teams and coordinating multiple overlapping changes to a code base. Additionally, students that are not actively experiencing team development are unprepared to understand and modify existing legacy-code bases written by others. This paper presents a new approach to teaching undergraduate software engineering that emphasizes not only software engineering methodology but also experiencing development as a member of a team and modifying a legacy code base. Our innovative software engineering course begins with learning the fundamentals of software engineering, followed by examining an existing framework of a social media application. The students are then grouped into multiple software teams, each focusing on a different aspect of the app. The separate teams must define requirements, design, and provide documentation on the services. Using an Agile development approach, the teams incrementally add to the code base and demonstrate features as the application evolves. Subsequent iterations of the class pick up the prior students code base, providing experience working with a legacy code base. Preliminary results of using this approach at the university are presented in this paper including quantitative analysis. Analysis of student software submissions to the cloud-based code repository shows student engagement and contributions over the span of the course. Positive student evaluations show the effectiveness of applying the principles of software engineering to the development of a complex solution in a team environment. Keywords: Software engineering, teaching, college computer science, innovative methods, agile.
Context: Experiment replications play a central role in the scientific method. Although software engineering experimentation has matured a great deal, the number of experiment replications is still relatively small. Software engineering experiments are composed of complex concepts, procedures and artefacts. Laboratory packages are a means of transfer-ring knowledge among researchers to facilitate experiment replications. Objective: This paper investigates the experiment replication process to find out what information is needed to successfully replicate an experiment. Our objective is to propose the content and structure of laboratory packages for software engineering experiments. Method: We evaluated seven replications of three different families of experiments. Each replication had a different experimenter who was, at the time, unfamiliar with the experi-ment. During the first iterations of the study, we identified experimental incidents and then proposed a laboratory package structure that addressed these incidents, including docu-ment usability improvements. We used the later iterations to validate and generalize the laboratory package structure for use in all software engineering experiments. We aimed to solve a specific problem, while at the same time looking at how to contribute to the body of knowledge on laboratory packages. Results: We generated a laboratory package for three different experiments. These packages eased the replication of the respective experiments. The evaluation that we conducted shows that the laboratory package proposal is acceptable and reduces the effort currently required to replicate experiments in software engineering. Conclusion: We think that the content and structure that we propose for laboratory pack-ages can be useful for other software engineering experiments.
Background: Previous research highlights that common misconceptions about developer productivity lead to harmful and inaccurate evaluations of software work, pointing to the need for organizations to differentiate between measures of production, productivity, and performance as an important step that helps to suggest improvements to how we measure the success of engineering teams. Methodology: Using a card sort, we explored how a Three Layer Productivity Framework was used by 16 software engineers at a Software Engineering focused conference to rank measures of success, first in the current practice of their organization and second in their individual beliefs about the best ways to measure engineering success. Results and discussion: Overall, participants preferred organizations to 1) continue their prioritized focus on performance layer metrics, 2) increase the focus on productivity metrics, and 3) decrease their focus on production metrics. When asked about the current metrics of their organizations, while all roles reported a current focus on performance metrics, only ICs reported a strong focus on production metrics. When asked about metrics they would prefer, all roles preferred more performance metrics but only leaders and ICs also wanted productivity metrics. While all participants were aligned on performance metrics being a top preference, there was misalignment on which specific metrics are used. Our findings show that when measuring developer success, organizations should continue measurement using performance metrics, consider an increased focus on productivity metrics, and consider a decreased focus on production metrics.
This systematic literature review aims to investigate the impact of artificial intelligence (AI) on the labour force in software engineering, with a particular focus on the skills needed for future software engineers, the impact of AI on the demand for software engineering skills, and the future of work for software engineers. The review identified 42 relevant publications through a comprehensive search strategy and analysed their findings. The results indicate that future software engineers will need to be competent in programming and have soft skills such as problem-solving and interpersonal communication. AI will have a significant impact on the software engineering workforce, with the potential to automate many jobs currently done by software engineers. The role of a software engineer is changing and will continue to change in the future, with AI-assisted software development posing challenges for the software engineering profession. The review suggests that the software engineering profession must adapt to the changing landscape to remain relevant and effective in the future.
This paper reports on a study of transferring a desktop-based model-based engineering tool to a web application. The study has been conducted in the WEBMODEL project where the well-established technology stack around the Eclipse platform and the Eclipse Modeling Framework was lifted into a cloud-based environment. As results, a modeling language independent tooling kernel for web-based modeling tools and a minimal prototypical web-based implementation of the AutoFOCUS 3 model-based engineering tool are presented. Furthermore, the report documents experiences and implementation advises gained during the implementation.
Manuel De Stefano, Fabiano Pecorelli, Dario Di Nucci
et al.
Quantum computing is no longer only a scientific interest but is rapidly becoming an industrially available technology that can potentially overcome the limits of classical computation. Over the last years, all major companies have provided frameworks and programming languages that allow developers to create their quantum applications. This shift has led to the definition of a new discipline called quantum software engineering, which is demanded to define novel methods for engineering large-scale quantum applications. While the research community is successfully embracing this call, we notice a lack of systematic investigations into the state of the practice of quantum programming. Understanding the challenges that quantum developers face is vital to precisely define the aims of quantum software engineering. Hence, in this paper, we first mine all the GitHub repositories that make use of the most used quantum programming frameworks currently on the market and then conduct coding analysis sessions to produce a taxonomy of the purposes for which quantum technologies are used. In the second place, we conduct a survey study that involves the contributors of the considered repositories, which aims to elicit the developers' opinions on the current adoption and challenges of quantum programming. On the one hand, the results highlight that the current adoption of quantum programming is still limited. On the other hand, there are many challenges that the software engineering community should carefully consider: these do not strictly pertain to technical concerns but also socio-technical matters.
Software engineers need relevant and up-to-date architectural knowledge (AK), in order to make well-founded design decisions. However, finding such AK is quite challenging. One pragmatic approach is to search for AK on the web using traditional search engines (e.g. Google); this is common practice among software engineers. Still, we know very little about what AK is retrieved, from where, and how useful it is. In this paper, we conduct an empirical study with 53 software engineers, who used Google to make design decisions using the Attribute-Driven-Design method. Based on how the subjects assessed the nature and relevance of the retrieved results, we determined how effective web search engines are to find relevant architectural information. Moreover, we identified the different sources of AK on the web and their associated AK concepts.
Masfiqur Rahaman, MD. Nazmul Hasan Sakib, Nafisa Islam
et al.
Using low-cost piezoelectric sensors to sense structural vibration exhibits great potential in augmenting structural engineering, which is yet to be explored in the literature to the best of our knowledge. Examples of such unexplored augmentation include classifying diverse structures (such as building, flyover, foot over-bridge, etc.), and relating the extent of vibration generated at different height of a structure and the associated height. Accordingly, to explore these cases, we develop a low-cost piezoelectric sensor-based vibration sensing system aiming to remotely collect real vibration data from diversified civil structures. We dig into our collected sensed data to classify five different types of structures through rigorous statistical and machine-learning-based analyses. Our analyses achieve a classification accuracy of up to 97% with an F1 score of 0.97. Nonetheless, in the rarely explored time domain, our analyses reveal a novel modality of relation between vibration generated at different heights of a structure and the associated height, which was explored in the frequency domain earlier in the literature with expensive sensors.
Hennie Huijgens, Ayushi Rastogi, Ernst Mulders
et al.
In 2014, a Microsoft study investigated the sort of questions that data science applied to software engineering should answer. This resulted in 145 questions that developers considered relevant for data scientists to answer, thus providing a research agenda to the community. Fast forward to five years, no further studies investigated whether the questions from the software engineers at Microsoft hold for other software companies, including software-intensive companies with different primary focus (to which we refer as software-defined enterprises). Furthermore, it is not evident that the problems identified five years ago are still applicable, given the technological advances in software engineering.
Alberto Gutierrez-Torre, Josep Ll. Berral, David Buchaca
et al.
Maritime traffic emissions are a major concern to governments as they heavily impact the Air Quality in coastal cities. Ships use the Automatic Identification System (AIS) to continuously report position and speed among other features, and therefore this data is suitable to be used to estimate emissions, if it is combined with engine data. However, important ship features are often inaccurate or missing. State-of-the-art complex systems, like CALIOPE at the Barcelona Supercomputing Center, are used to model Air Quality. These systems can benefit from AIS based emission models as they are very precise in positioning the pollution. Unfortunately, these models are sensitive to missing or corrupted data, and therefore they need data curation techniques to significantly improve the estimation accuracy. In this work, we propose a methodology for treating ship data using Conditional Restricted Boltzmann Machines (CRBMs) plus machine learning methods to improve the quality of data passed to emission models. Results show that we can improve the default methods proposed to cover missing data. In our results, we observed that using our method the models boosted their accuracy to detect otherwise undetectable emissions. In particular, we used a real data-set of AIS data, provided by the Spanish Port Authority, to estimate that thanks to our method, the model was able to detect 45% of additional emissions, of additional emissions, representing 152 tonnes of pollutants per week in Barcelona and propose new features that may enhance emission modeling.
Mohit Kumar Ahuja, Mohamed-Bachir Belaid, Pierre Bernabé
et al.
Trustworthiness is a central requirement for the acceptance and success of human-centered artificial intelligence (AI). To deem an AI system as trustworthy, it is crucial to assess its behaviour and characteristics against a gold standard of Trustworthy AI, consisting of guidelines, requirements, or only expectations. While AI systems are highly complex, their implementations are still based on software. The software engineering community has a long-established toolbox for the assessment of software systems, especially in the context of software testing. In this paper, we argue for the application of software engineering and testing practices for the assessment of trustworthy AI. We make the connection between the seven key requirements as defined by the European Commission's AI high-level expert group and established procedures from software engineering and raise questions for future work.
Jan Bosch, Ivica Crnkovic, Helena Holmström Olsson
Artificial intelligence (AI) and machine learning (ML) are increasingly broadly adopted in industry, However, based on well over a dozen case studies, we have learned that deploying industry-strength, production quality ML models in systems proves to be challenging. Companies experience challenges related to data quality, design methods and processes, performance of models as well as deployment and compliance. We learned that a new, structured engineering approach is required to construct and evolve systems that contain ML/DL components. In this paper, we provide a conceptualization of the typical evolution patterns that companies experience when employing ML as well as an overview of the key problems experienced by the companies that we have studied. The main contribution of the paper is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions and an overview of open items that need to be addressed by the research community at large.
Processes of propagation and interaction of nonlinear gravity-capillary waves on the free surface of a deep non-conducting liquid with high dielectric constant under the action of a tangential electric field are numerically simulated. The computational method is based on the time-dependent conformal transformation of the region occupied by the fluid into a half-plane. In the limit of a strong electric field, when the gravitational and capillary forces are negligibly small, there exists an exact analytical solution of the electro-hydrodynamic equations describing propagation without distortions of nonlinear surface waves along (or against) the electric field direction. In the situation where gravity and capillarity are taken into account, the results of numerical simulations indeed show that, for large external field, the waves traveling in a given direction tend to preserve their shape. In the limit of a strong electric field, the interaction of counter-propagating waves leads to the formation of regions, where the electrostatic and dynamic pressures undergo a discontinuity, and the curvature of the surface increases infinitely. The Fourier spectrum of the surface perturbations tends to the power-law distribution (k^-2). In the case of a finite electric field, the wave interaction results in a radiation of massive cascade of small-scale capillary waves that causes the chaotic behavior of the system. The investigated mechanism of interaction between oppositely-traveling waves can enhance development of the capillary turbulence of the fluid surface.
Daniel Graziotin, Fabian Fagerholm, Xiaofeng Wang
et al.
Recent research in software engineering supports the "happy-productive" thesis, and the desire of flourishing happiness among programmers is often expressed by industry practitioners. Recent literature has suggested that a cost-effective way to foster happiness and productivity among workers could be to limit unhappiness of developers due to its negative impact. However, possible negative effects of unhappiness are still largely unknown in the software development context. In this paper, we present the first results from a study exploring the consequences of the unhappy developers. Using qualitative data analysis of the survey responses given by 181 participants, we identified 49 potential consequences of unhappiness while developing software. These results have several implications. While raising the awareness of the role of moods, emotions and feelings in software development, we foresee that our classification scheme will spawn new happiness studies linking causes and effects, and it can act as a guideline for developers and managers to foster happiness at work.