This paper presents Nova, a real-time scheduling framework for serving agentic vision-language models (VLMs) on a single GPU with balanced per-request latency and overall request process throughput. Our design begins by enabling effective pipelining across vision encode, LLM prefill, and LLM decode stages of VLMs, by exploiting their heterogeneous resource demands during execution and incorporating elastic GPU spatial partitioning among stages to maximally utilize the compute and memory resources. Building on this, we introduce a real-time scheduling algorithm that adaptively calibrates resource allocation among stages based on a Pareto-optimal analysis of the latency-throughput trade-off, allowing the system to sustain responsiveness and resource efficiency under dynamic request loads. To further alleviate GPU memory pressure, we design a lightweight weight offloading strategy for vision encoders that preserves inference efficiency with minimized memory overhead. Extensive evaluations on both synthetic and real-world agent workloads demonstrate that Nova consistently outperforms the state-of-the-art baselines, improving the maximum latency by up to 23.3%, while keeping competitive throughput.
Memory tiering in datacenters does not achieve its full potential due to hotness fragmentation -- the intermingling of hot and cold objects within memory pages. This fragmentation prevents page-based reclamation systems from distinguishing truly hot pages from pages containing mostly cold objects, fundamentally limiting memory efficiency despite highly skewed accesses. We introduce address-space engineering: dynamically reorganizing application virtual address spaces to create uniformly hot and cold regions that any page-level tiering backend can manage effectively. HADES demonstrates this frontend/backend approach through a compiler-runtime system that tracks and migrates objects based on access patterns, requiring minimal developer intervention. Evaluations across ten data structures achieve up to 70% memory reduction with 3% performance overhead, showing that address space engineering enables existing reclamation systems to reclaim memory aggressively without performance degradation.
The network interface adapter (NIC) is a critical component of a cloud server occupying a unique position. Not only is network performance vital to efficient operation of the machine, but unlike compute accelerators like GPUs, the network subsystem must react to unpredictable events like the arrival of a network packet and communicate with the appropriate application end point with minimal latency. Current approaches to server stacks navigate a trade-off between flexibility, efficiency, and performance: the fastest kernel-bypass approaches dedicate cores to applications, busy-wait on receive queues, etc. while more flexible approaches appropriate to more dynamic workload mixes incur much greater software overhead on the data path. However, we reject this trade-off, which we ascribe to an arbitrary (and sub-optimal) split in system state between the OS and the NIC. Instead, by exploiting the properties of cache-coherent interconnects and integrating the NIC closely with the OS kernel, we can achieve something surprising: performance for RPC workloads better than the fastest kernelbypass approaches without sacrificing the robustness and dynamic adaptation of kernel-based network subsystems.
Gernot Heiser, Ivan Velickovic, Peter Chubb
et al.
We present LionsOS, an operating system for security- and safety-critical embedded systems. LionsOS is based on the formally verified seL4 microkernel and designed with verification in mind. It uses a static architecture and features a highly modular design driven by strict separa- tion of concerns and a focus on simplicity. We demonstrate that LionsOS achieves excellent performance on system-call intensive workloads.
The storage stack in the traditional operating system is primarily optimized towards improving the CPU utilization and hiding the long I/O latency imposed by the slow I/O devices such as hard disk drivers (HDDs). However, the emerging storage media experience significant technique shifts in the past decade, which exhibit high bandwidth and low latency. These high-performance storage devices, unfortunately, suffer from the huge overheads imposed by the system software including the long storage stack and the frequent context switch between the user and kernel modes. Many researchers have investigated huge efforts in addressing this challenge by constructing a direct software path between a user process and the underlying storage devices. We revisit such novel designs in the prior work and present a survey in this paper. Specifically, we classify the former research into three categories according to their commonalities. We then present the designs of each category based on the timeline and analyze their uniqueness and contributions. This paper also reviews the applications that exploit the characteristics of theses designs. Given that the user-space storage is a growing research field, we believe this paper can be an inspiration for future researchers, who are interested in the user-space storage system designs.
This paper studies three fundamental aspects of an OS that impact the performance and energy efficiency of network processing: 1) batching, 2) processor energy settings, and 3) the logic and instructions of the OS networking paths. A network device's interrupt delay feature is used to induce batching and processor frequency is manipulated to control the speed of instruction execution. A baremetal library OS is used to explore OS path specialization. This study shows how careful use of batching and interrupt delay results in 2X energy and performance improvements across different workloads. Surprisingly, we find polling can be made energy efficient and can result in gains up to 11X over baseline Linux. We developed a methodology and a set of tools to collect system data in order to understand how energy is impacted at a fine-grained granularity. This paper identifies a number of other novel findings that have implications in OS design for networked applications and suggests a path forward to consider energy as a focal point of systems research.
Classic test-and-test (TS) mutual exclusion locks are simple, and enjoy high performance and low latency of ownership transfer under light or no contention. However, they do not scale gracefully under high contention and do not provide any admission order guarantees. Such concerns led to the development of scalable queue-based locks, such as a recent Compact NUMA-aware (CNA) lock, a variant of another popular queue-based MCS lock. CNA scales well under load and provides certain admission guarantees, but has more complicated lock handover operations than TS and incurs higher latencies at low contention. We propose Fissile locks, which capture the most desirable properties of both TS and CNA. A Fissile lock consists of two underlying locks: a TS lock, which serves as a fast path, and a CNA lock, which serves as a slow path. The key feature of Fissile locks is the ability of threads on the fast path to bypass threads enqueued on the slow path, and acquire the lock with less overhead than CNA. Bypass is bounded (by a tunable parameter) to avoid starvation and ensure long-term fairness. The result is a highly scalable NUMA-aware lock with progress guarantees that performs like TS at low contention and like CNA at high contention.
Almost all modern hardware, from phone SoCs to high-end servers with accelerators, contain memory translation and protection hardware like IOMMUs, firewalls, and lookup tables which make it impossible to reason about, and enforce protection and isolation based solely on the processor's MMUs. This has led to numerous bugs and security vulnerabilities in today's system software. In this paper we regain the ability to reason about and enforce access control using the proven concept of a reference monitor mediating accesses to memory resources. We present a fine-grained, realistic memory protection model that makes this traditional concept applicable today, and bring system software in line with the complexity of modern, heterogeneous hardware. Our design is applicable to any operating system, regardless of architecture. We show that it not only enforces the integrity properties of a system, but does so with no inherent performance overhead and it is even amenable to automation through code generation from trusted hardware specifications.
Software-controlled heterogeneous memory systems have the potential to improve performance, efficiency, and cost tradeoffs in emerging systems. Delivering on this promise requires an efficient operating system (OS) mechanisms and policies for data management. Unfortunately, modern OSes do not support efficient tiering of data between heterogeneous memories. While this problem is known (and is being studied) for application-level data pages, the question of how best to tier OS kernel objects has largely been ignored. We show that careful kernel object management is vital to the performance of software-controlled tiered memory systems. We find that the state-of-the-art OS page management research leaves considerable performance on the table by overlooking how best to tier, migrate, and manage kernel objects like inodes, dentry caches, journal blocks, network socket buffers, etc., associated with the filesystem and networking stack. In response, we characterize hotness, reuse, and liveness properties of kernel objects to develop appropriate tiering/migration mechanisms and policies. We evaluate our proposal using a real-system emulation framework on large-scale workloads like RocksDB, Redis, Cassandra, and Spark and achieve 1.4X to 4X higher throughput compared to the prior art.
Over-estimation of worst-case execution times (WCETs) of real-time tasks leads to poor resource utilization. In a mixed-criticality system (MCS), the over-provisioning of CPU time to accommodate the WCETs of highly critical tasks may lead to degraded service for less critical tasks. In this paper, we present PAStime, a novel approach to monitor and adapt the runtime progress of highly time-critical applications, to allow for improved service to lower criticality tasks. In PAStime, CPU time is allocated to time-critical tasks according to the delays they experience as they progress through their control flow graphs. This ensures that as much time as possible is made available to improve the Quality-of-Service of less critical tasks, while high-criticality tasks are compensated after their delays. In this paper, we integrate PAStime with Adaptive Mixed-criticality (AMC) scheduling. The LO-mode budget of a high-criticality task is adjusted according to the delay observed at execution checkpoints. This is the first implementation of AMC in the scheduling framework Using LITMUS-RT, which is extended with our PAStime runtime policy and tested with real-time Linux applications such as object classification and detection. We observe in our experimental evaluation that AMC-PAStime significantly improves the utilization of the low-criticality tasks while guaranteeing service to high-criticality tasks.
With the emergence of Non-Volatile Memories (NVMs) and their shortcomings such as limited endurance and high power consumption in write requests, several studies have suggested hybrid memory architecture employing both Dynamic Random Access Memory (DRAM) and NVM in a memory system. By conducting a comprehensive experiments, we have observed that such studies lack to consider very important aspects of hybrid memories including the effect of: a) data migrations on performance, b) data migrations on power, and c) the granularity of data migration. This paper presents an efficient data migration scheme at the Operating System level in a hybrid DRAMNVM memory architecture. In the proposed scheme, two Least Recently Used (LRU) queues, one for DRAM section and one for NVM section, are used for the sake of data migration. With careful characterization of the workloads obtained from PARSEC benchmark suite, the proposed scheme prevents unnecessary migrations and only allows migrations which benefits the system in terms of power and performance. The experimental results show that the proposed scheme can reduce the power consumption up to 79% compared to DRAM-only memory and up to 48% compared to the state-of-the art techniques.
The hardware/software boundary in modern heterogeneous multicore computers is increasingly complex, and diverse across different platforms. A single memory access by a core or DMA engine traverses multiple hardware translation and caching steps, and the destination memory cell or register often appears at different physical addresses for different cores. Interrupts pass through a complex topology of interrupt controllers and remappers before delivery to one or more cores, each with specific constraints on their configurations. System software must not only correctly understand the specific hardware at hand, but also configure it appropriately at runtime. We propose a formal model of address spaces and resources in a system that allows us to express and verify invariants of the system's runtime configuration, and illustrate (and motivate) it with several real platforms we have encountered in the process of OS implementation.
Real-time scheduling algorithms proposed in the literature are often based on worst-case estimates of task parameters. The performance of an open-loop scheme can be degraded significantly if there are uncertainties in task parameters, such as the execution times of the tasks. Therefore, to cope with such a situation, a closed-loop scheme, where feedback is exploited to adjust the system parameters, can be applied. We propose an optimal control framework that takes advantage of feeding back information of finished tasks to solve a real-time multiprocessor scheduling problem with uncertainty in task execution times, with the objective of minimizing the total energy consumption. Specifically, we propose a linear programming based algorithm to solve a workload partitioning problem and adopt McNaughton's wrap around algorithm to find the task execution order. The simulation results illustrate that our feedback scheduling algorithm can save energy by as much as 40% compared to an open-loop method for two processor models, i.e. a PowerPC 405LP and an XScale processor.
Nota do tradutor (Nt): o referido texto foi redigido originalmente em francês, em julho de 2002; logo foi traduzido para o inglês e publicado. Mais tarde, em 2007, foi retomado em coletânea, também em língua inglesa (vide referências abaixo). Após o falecimento do autor, a 12 de janeiro de 2010, uma obra póstuma reuniu uma série de textos esparsos, juntando conferências e variadas participações em publicações coletivas. Com o título de La Politique comme art stratégique (Paris: Syllepse, 2011), o volume foi organizado e prefaciado pelo pensador socialista e homem político Antoine Artous.
Heterogeneous multicore architectures are becoming increasingly popular due to their potential of achieving high performance and energy efficiency compared to the homogeneous multicore architectures. In such systems, the real-time scheduling problem becomes more challenging in that processors have different speeds. A job executing on a processor with speed $x$ for $t$ time units completes $(x \cdot t)$ units of execution. Prior research on heterogeneous multiprocessor real-time scheduling has focused on hard real-time systems, where, significant processing capacity may have to be sacrificed in the worst-case to ensure that all deadlines are met. As meeting hard deadlines is overkill for many soft real-time systems in practice, this paper shows that on soft real-time heterogeneous multiprocessors, bounded response times can be ensured for globally-scheduled sporadic task systems with no utilization loss. A GEDF-based scheduling algorithm, namely GEDF-H, is presented and response time bounds are established under both preemptive and non-preemptive GEDF-H scheduling. Extensive experiments show that the magnitude of the derived response time bound is reasonable, often smaller than three task periods. To the best of our knowledge, this paper is the first to show that soft real-time sporadic task systems can be supported on heterogeneous multiprocessors without utilization loss, and with reasonable predicted response time.
A etnologia das Terras Baixas sul americanas, nas últimas décadas, através dos trabalhos de Descola (1986; 2005), Viveiros e Castro (1996; 2009) e Santos-Granero (2009), só para citar alguns dentre vários outros possíveis nomes, evidenciou a convivência de múltiplos agentes na experiência da cotidianidade dos povos ameríndios. Frente a essa proliferação, faz-se importante levar a sério tais agências e acompanhar as trajetórias seguidas pelos diferentes vetores. Ao longo desse texto, portanto, iremos acompanhar essas múltiplas conexões, as quais agem em um processo de recíproca ressonância, ou seja, a oscilação ou o movimento de uma, influencia a oscilação da outra. Na descrição que segue, porei em evidência como estes produzem o movimento dos outros: músicas, falas formais, marimbondos, plantas e pessoas, mas também pinturas corporais e animais. Dessa forma, quero evidenciar a possibilidade de observar o conjunto, composto por estes elementos em contínuo movimento, como um sistema que constantemente dinamiza movimentos de adaptação, os quais permitem trazer à vista uma multiplicidade de relações que abrangem diferentes âmbitos da cotidianidade Xikrin.
This paper presents Flashmon version 2, a tool for monitoring embedded Linux NAND flash memory I/O requests. It is designed for embedded boards based devices containing raw flash chips. Flashmon is a kernel module and stands for "flash monitor". It traces flash I/O by placing kernel probes at the NAND driver level. It allows tracing at runtime the 3 main flash operations: page reads / writes and block erasures. Flashmon is (1) generic as it was successfully tested on the three most widely used flash file systems that are JFFS2, UBIFS and YAFFS, and several NAND chip models. Moreover, it is (2) non intrusive, (3) has a controllable memory footprint, and (4) exhibits a low overhead (<6%) on the traced system. Finally, it is (5) simple to integrate and used as a standalone module or as a built-in function / module in existing kernel sources. Monitoring flash memory operations allows a better understanding of existing flash management systems by studying and analyzing their behavior. Moreover it is useful in development phase for prototyping and validating new solutions.
Ardalan Amiri Sani, Kevin Boos, Min Hong Yun
et al.
Mobile systems are equipped with a diverse collection of I/O devices, including cameras, microphones, sensors, and modems. There exist many novel use cases for allowing an application on one mobile system to utilize I/O devices from another. This paper presents Rio, an I/O sharing solution that supports unmodified applications and exposes all the functionality of an I/O device for sharing. Rio's design is common to many classes of I/O devices, thus significantly reducing the engineering effort to support new I/O devices. Our implementation of Rio on Android consists of 6700 total lines of code and supports four I/O classes with fewer than 450 class-specific lines of code. Rio also supports I/O sharing between mobile systems of different form factors, including smartphones and tablets. We show that Rio achieves performance close to that of local I/O for audio, sensors, and modems, but suffers noticeable performance degradation for camera due to network throughput limitations between the two systems, which is likely to be alleviated by emerging wireless standards.
This is the era of High Performance Computing (HPC). There is a great demand of the best performance evaluation techniques for the file and storage systems. The task of evaluation is both necessary and hard. It gives in depth analysis of the target system and that becomes the decision points for the users. That is also helpful for the inventors or developers to find out the bottleneck in their systems. In this paper many performance evaluation techniques are described for file and storage system evaluation and the main stress is given on the important one that is replay traces. A survey has been done for the performance evaluation techniques used by the researchers and on the replay traces. And the taxonomy of the replay traces is described. The some of the popular replay traces are just like, Tracefs [1], //Trace [2], Replayfs [3] and VFS Interceptor [12]. At last we have concluded all the features that must be considered when we are going to develop the new tool for the replay traces. The complete work of this paper shows that the storage system developers must care about all the techniques which can be used for the performance evaluation of the file systems. So they can develop highly efficient future file and storage systems.
Multi- and many-core processors are becoming increasingly popular in embedded systems. Many of these processors now feature hardware virtualization capabilities, such as the ARM Cortex A15, and x86 processors with Intel VT-x or AMD-V support. Hardware virtualization offers opportunities to partition physical resources, including processor cores, memory and I/O devices amongst guest virtual machines. Mixed criticality systems and services can then co-exist on the same platform in separate virtual machines. However, traditional virtual machine systems are too expensive because of the costs of trapping into hypervisors to multiplex and manage machine physical resources on behalf of separate guests. For example, hypervisors are needed to schedule separate VMs on physical processor cores. In this paper, we discuss the design of the Quest-V separation kernel, that partitions services of different criticalities in separate virtual machines, or sandboxes. Each sandbox encapsulates a subset of machine physical resources that it manages without requiring intervention of a hypervisor. Moreover, a hypervisor is not needed for normal operation, except to bootstrap the system and establish communication channels between sandboxes.