Small Language Models in the Real World: Insights from Industrial Text Classification
Lujun Li, Lama Sleem, Niccolo' Gentile
et al.
With the emergence of ChatGPT, Transformer models have significantly advanced text classification and related tasks. Decoder-only models such as Llama exhibit strong performance and flexibility, yet they suffer from inefficiency on inference due to token-by-token generation, and their effectiveness in text classification tasks heavily depends on prompt quality. Moreover, their substantial GPU resource requirements often limit widespread adoption. Thus, the question of whether smaller language models are capable of effectively handling text classification tasks emerges as a topic of significant interest. However, the selection of appropriate models and methodologies remains largely underexplored. In this paper, we conduct a comprehensive evaluation of prompt engineering and supervised fine-tuning methods for transformer-based text classification. Specifically, we focus on practical industrial scenarios, including email classification, legal document categorization, and the classification of extremely long academic texts. We examine the strengths and limitations of smaller models, with particular attention to both their performance and their efficiency in Video Random-Access Memory (VRAM) utilization, thereby providing valuable insights for the local deployment and application of compact models in industrial settings.
Design and Implementation of Novel H-Shaped Self-Diplexing SIW Rectangular Cavity-Backed Antenna with Harmonic Suppression for Terrestrial Communications
Ravindiran Asaithambi, Rajkishor Kumar
A novel and low profile, planar, rectangular cavity-backed self-diplexing substrate integrated waveguide (SIW) antenna with H-shaped slot for dual-band wireless services was designed and demonstrated. The proposed antenna structure radiates from H-shaped slot, which is etched on top of the SIW rectangular cavity, and is excited by two separate 50 Ω microstrip feed lines. The H-shaped slot is a combination of two vertical slots and one horizontal slot; because of that the presented antenna radiates at two distinct frequency bands around 8.95 GHz and 10 GHz, simultaneously. The design methodology results show that the H-shaped slot is significantly more effective than various other slots in the proposed geometry to suppress the unwanted harmonics, attaining good impedance matching and bandwidths and achieving better isolation between these two ports. Hence, the complete design mechanism helped to achieve self-diplexing characteristics. Furthermore, a self-diplexing H-shaped SIW rectangular cavity-backed antenna was fabricated and characterized for the complete demonstration purpose and found good covenants between the simulated one. Measured results show that the presented designed has impedance bandwidths for the lower and upper frequency bands of around 2.0% (8.89–9.03 GHz) and 3.1% (10.01–10.32 GHz), respectively, and obtained maximum measured gain of 5.11 dBi and 5.41 dBi at 8.95 GHz and 10.15 GHz, respectively. The proposed self-diplexing SIW rectangular cavity-backed structure shows that front-to-back ratios (FTBRs) are more than 21 dB, and on the other side, it provides good isolation between the two ports, which is more than 20 dB.
Electrical engineering. Electronics. Nuclear engineering, Cellular telephone services industry. Wireless telephone industry
Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarization
David Thulke, Yingbo Gao, Rricha Jalota
et al.
This paper explores the rapid development of a telephone call summarization system utilizing large language models (LLMs). Our approach involves initial experiments with prompting existing LLMs to generate summaries of telephone conversations, followed by the creation of a tailored synthetic training dataset utilizing stronger frontier models. We place special focus on the diversity of the generated data and on the ability to control the length of the generated summaries to meet various use-case specific requirements. The effectiveness of our method is evaluated using two state-of-the-art LLM-as-a-judge-based evaluation techniques to ensure the quality and relevance of the summaries. Our results show that fine-tuned Llama-2-7B-based summarization model performs on-par with GPT-4 in terms of factual accuracy, completeness and conciseness. Our findings demonstrate the potential for quickly bootstrapping a practical and efficient call summarization system.
A Practical Evaluation of Commercial Industrial Augmented Reality Systems in an Industry 4.0 Shipyard
Oscar Blanco-Novoa, Tiago M Fernandez-Carames, Paula Fraga-Lamas
et al.
The principles of the Industry 4.0 are guiding manufacturing companies towards more automated and computerized factories. Such principles are also applied in shipbuilding, which usually involves numerous complex processes whose automation will improve its efficiency and performance. Navantia, a company that has been building ships for 300 years, is modernizing its shipyards according to the Industry 4.0 principles with the help of the latest technologies. Augmented Reality (AR), which when utilized in an industrial environment is called Industrial AR (IAR), is one of such technologies, since it can be applied in numerous situations in order to provide useful and attractive interfaces that allow shipyard operators to obtain information on their tasks and to interact with certain elements that surround them. This article first reviews the state of the art on IAR applications for shipbuilding and smart manufacturing. Then, the most relevant IAR hardware and software tools are detailed, as well as the main use cases for the application of IAR in a shipyard. Next, it is described Navantia's IAR system, which is based on a fog-computing architecture. Such a system is evaluated when making use of three IAR devices (a smartphone, a tablet and a pair of smart glasses), two AR SDKs (ARToolKit and Vuforia) and multiple IAR markers, with the objective of determining their performance in a shipyard workshop and inside a ship under construction. The results obtained show remarkable performance differences among the different IAR tools and the impact of factors like lighting, pointing out the best combinations of markers, hardware and software to be used depending on the characteristics of the shipyard scenario.
Biology and Technology Interaction: Study identifying the impact of robotic systems on fish behaviour change in industrial scale fish farms
Linn Danielsen Evjemo, Qin Zhang, Hanne-Grete Alvheim
et al.
The significant growth in the aquaculture industry over the last few decades encourages new technological and robotic solutions to help improve the efficiency and safety of production. In sea-based farming of Atlantic salmon in Norway, Unmanned Underwater Vehicles (UUVs) are already being used for inspection tasks. While new methods, systems and concepts for sub-sea operations are continuously being developed, these systems generally does not take into account how their presence might impact the fish. This abstract presents an experimental study on how underwater robotic operations at fish farms in Norway can affect farmed Atlantic salmon, and how the fish behaviour changes when exposed to the robot. The abstract provides an overview of the case study, the methods of analysis, and some preliminary results.
A Single-Layer Circularly Polarized Reflectarray Antenna with High Aperture Efficiency for Microwave Power Transmission
Huaiqing Zhang, Yuxin Wang, Jiapeng Wang
et al.
In this article, a single-layer circularly polarized reflectarray antenna (RA) with a linearly polarized feed is proposed for microwave power transmission. The unit cell of the reflectarray is composed of a rectangular patch surrounded by four groups of E-shaped structures, which feature a very low level of cross-polarization. Simulation results demonstrate that the phase of its reflection coefficient can be tuned continuously in a range of 500° by adjusting the lengths of the E-shaped structures. In response to a normally incident plane wave, the phase values of the reflection coefficient associated with two orthogonal linear polarization directions of the reflected wave can be tuned independently, which makes it possible to convert a linearly polarized incident wave to a circularly polarized beam. A reflectarray with 15 × 15 unit cells is fabricated and tested. The measurement results demonstrate a 3-dB axial ratio bandwidth of more than 2 GHz around the center frequency of 5.8 GHz. The measured gain value of the fabricated reflectarray is 25.4 dBi, corresponding to an aperture efficiency of 52.5%.
Electrical engineering. Electronics. Nuclear engineering, Cellular telephone services industry. Wireless telephone industry
Prompt Tuned Embedding Classification for Multi-Label Industry Sector Allocation
Valentin Leonhard Buchner, Lele Cao, Jan-Christoph Kalo
et al.
Prompt Tuning is emerging as a scalable and cost-effective method to fine-tune Pretrained Language Models (PLMs), which are often referred to as Large Language Models (LLMs). This study benchmarks the performance and computational efficiency of Prompt Tuning and baselines for multi-label text classification. This is applied to the challenging task of classifying companies into an investment firm's proprietary industry taxonomy, supporting their thematic investment strategy. Text-to-text classification is frequently reported to outperform task-specific classification heads, but has several limitations when applied to a multi-label classification problem where each label consists of multiple tokens: (a) Generated labels may not match any label in the label taxonomy; (b) The fine-tuning process lacks permutation invariance and is sensitive to the order of the provided labels; (c) The model provides binary decisions rather than appropriate confidence scores. Limitation (a) is addressed by applying constrained decoding using Trie Search, which slightly improves classification performance. All limitations (a), (b), and (c) are addressed by replacing the PLM's language head with a classification head, which is referred to as Prompt Tuned Embedding Classification (PTEC). This improves performance significantly, while also reducing computational costs during inference. In our industrial application, the training data is skewed towards well-known companies. We confirm that the model's performance is consistent across both well-known and less-known companies. Our overall results indicate the continuing need to adapt state-of-the-art methods to domain-specific tasks, even in the era of PLMs with strong generalization abilities. We release our codebase and a benchmarking dataset at https://github.com/EQTPartners/PTEC.
DOA Estimation Method of Weak Signal under the Compound Background of Strong Interference and Colored Noise
Bin Lin, Guoping Hu, Hao Zhou
et al.
The traditional algorithm performing direction of arrival (DOA) estimation under the background of strong interference and colored noise has the problems of low estimation accuracy and small measurement targets. Based on the construction of a fourth-order cumulant (FOC) matrix to suppress colored noise, this paper adopts the extended noise subspace (ENS) algorithm and the fixed projection blocking (FPB) algorithm to estimate the DOA of weak targets. Firstly, a FOC matrix of the received signal vector is established to curb the noise component, and the eigenvalue decomposition is performed. Then, two approaches of weak signal DOA estimation are proposed. One approach is to merge the space where the strong interference steering vector lies into the noise subspace to construct an extended noise subspace, and then, the multisignal classification (MUSIC) algorithm is used to obtain the DOA estimation of the weak signal on the basis of the extended noise subspace. Another approach is to build the orthogonal projection matrix of the interference subspace as the interference blocking matrix, and the receiving array signal is preprocessed, and on the basis of it, the eigen decomposition is performed again to obtain the DOA information of the weak signal. Both algorithms make breakthroughs in the aperture limitation of the traditional algorithm, effectively expand the aperture, and promote the accuracy of estimation. The simulation tests the effectiveness of the proposed method.
Electrical engineering. Electronics. Nuclear engineering, Cellular telephone services industry. Wireless telephone industry
Textile UWB Antenna with Metamaterial for Healthcare Monitoring
S. Parameswari, C. Chitra
A new metamaterial-based UWB band-notched textile antenna for body area network (BAN) with an operational frequency range of 3 GHz to 11 GHz is created in this paper. The ultra-ide band (UWB) frequency band is covered by the antenna (3.1 GHz to 10.6 GHz). The antennas are smaller because of the usage of denim (jeans) material, which has a permittivity of 1.67. To increase the impedance transmission capacity, the ground plane is reduced to a partly rectangular conductive substance. The hexagonal cut on the bottom side is utilised to boost bandwidth by enhancing the electric field dispersion at the edges. The fabrication is built of a 1 mm thick denim (jeans) substrate, and the feed is a traditional microstrip feed. The return loss and gain characteristics of the proposed antenna are investigated. The performance of a specified antenna is investigated step by step with variable feed length, feed breadth, and substrate properties.
Electrical engineering. Electronics. Nuclear engineering, Cellular telephone services industry. Wireless telephone industry
A Compact Wideband Circularly Polarized Microstrip Slot Antenna with Parasitic Elements
Chenhui Xia, Shuo Diao, Wenting Yin
et al.
A compact wideband circularly polarized (CP) microstrip slot antenna (MSA) with parasitic elements is designed in this letter. The CP MSA comprises a square-loop sequential-phase (SP) feeding configuration, four rotated rectangular patches, and four L-shaped slots embedded in the ground plane. The square-loop SP feeding structure comprises a square loop and an arc-shaped strip, which could provide a 270° phase difference. Four rotated rectangular patches are placed at the edge of the square-loop feeding configuration using a capacitively tightly coupled feeding method to stimulate the CP resonant mode. After selecting these elements and tuning proper dimensions, the broad operating bandwidths of 4.38–5.25 GHz (18%) for |S11| <–10 dB and 4.65–5.31 GHz (13.2%) for AR <3 dB could be realized. Hence, the designed CP MSA has a potential application value in wireless communication.
Electrical engineering. Electronics. Nuclear engineering, Cellular telephone services industry. Wireless telephone industry
A Path to Smart Radio Environments: An Industrial Viewpoint on Reconfigurable Intelligent Surfaces
Ruiqi Liu, Qingqing Wu, Marco Di Renzo
et al.
With both the standardization and commercialization completed in an unforeseen pace for the 5th generation (5G) wireless network, researchers, engineers and executives from the academia and the industry have turned their sights on candidate technologies to support the next generation wireless networks. Reconfigurable intelligent surfaces (RIS), sometimes referred to as intelligent reflecting surfaces (IRS), have been identified to be potential components of the future wireless networks because they can reconfigure the propagation environment for wireless signals with low-cost passive devices. In doing so, the coverage of a cell can be expected to increase significantly as well as the overall throughput of the network. RIS has not only become an attractive research area but also triggered a couple of projects to develop appropriate solutions to enable the set-up of hardware demonstrations and prototypes. In parallel, technical discussions and activities towards standardization already took off in some regions. Promoting RIS to be integrated into future commercial networks and become a commercial success requires significant standardization work taken place both at regional level standards developing organizations (SDO) and international SDOs such as the 3rd Generation Partnership Project (3GPP). While many research papers study how RIS can be used and optimized, few effort is devoted to analyzing the challenges to commercialize RIS and how RIS can be standardized. This paper intends to shed some light on RIS from an industrial viewpoint and provide a clear roadmap to make RIS industrially feasible.
Deep Feature CycleGANs: Speaker Identity Preserving Non-parallel Microphone-Telephone Domain Adaptation for Speaker Verification
Saurabh Kataria, Jesús Villalba, Piotr Żelasko
et al.
With the increase in the availability of speech from varied domains, it is imperative to use such out-of-domain data to improve existing speech systems. Domain adaptation is a prominent pre-processing approach for this. We investigate it for adapt microphone speech to the telephone domain. Specifically, we explore CycleGAN-based unpaired translation of microphone data to improve the x-vector/speaker embedding network for Telephony Speaker Verification. We first demonstrate the efficacy of this on real challenging data and then, to improve further, we modify the CycleGAN formulation to make the adaptation task-specific. We modify CycleGAN's identity loss, cycle-consistency loss, and adversarial loss to operate in the deep feature space. Deep features of a signal are extracted from an auxiliary (speaker embedding) network and, hence, preserves speaker identity. Our 3D convolution-based Deep Feature Discriminators (DFD) show relative improvements of 5-10% in terms of equal error rate. To dive deeper, we study a challenging scenario of pooling (adapted) microphone and telephone data with data augmentations and telephone codecs. Finally, we highlight the sensitivity of CycleGAN hyper-parameters and introduce a parameter called probability of adaptation.
Multimedia Multicast Services in 5G Networks: Subgrouping and Non-Orthogonal Multiple Access Techniques
J. Montalbán, Pasquale Scopelliti, M. Fadda
et al.
84 sitasi
en
Computer Science
High Isolation and Ideal Correlation Using Spatial Diversity in a Compact MIMO Antenna for Fifth-Generation Applications
Doae El Hadri, Alia Zakriti, Asmaa Zugari
et al.
This paper presents a compact Multiple Input Multiple Output antenna with high isolation and low envelope correlation (ECC) for fifth-generation applications using spatial diversity technique. The proposed MIMO antenna consists of two single antennas, each having size of 13 × 12.8 mm2, symmetrically arranged next to each other. The single and MIMO antennas are simulated and analyzed. To verify the simulated results, the prototype antennas were fabricated and measured. A good agreement between measurements and simulations is obtained. The proposed antenna covers the 28 GHz band (27.5–28.35 GHz) allocated by the FCC for 5G applications. Moreover, the isolation is more than 35 dB and the ECC is less than 0.0004 at the operating band, which means that the mutual coupling between the two elements is negligible. The MIMO parameters, such as diversity gain (DG), total active reflection coefficient (TARC), realized gain, and efficiency, are also studied. Thus, the results demonstrate that our antenna is suitable for 5G MIMO applications.
Electrical engineering. Electronics. Nuclear engineering, Cellular telephone services industry. Wireless telephone industry
A Distributed Power Control Algorithm for Energy Efficiency Maximization in Wireless Cellular Networks
Rojin Aslani, Mehdi Rasti
In this paper, we propose a distributed power control algorithm for addressing the global energy efficiency (GEE) maximization problem subject to satisfying a minimum target SINR for all user equipments (UEs) in wireless cellular networks. We state the problem as a multi-objective optimization problem which targets minimizing total power consumption and maximizing total throughput, simultaneously, while a minimum target SINR is guaranteed for all UEs. We propose an iterative scheme executed in the UEs to control their transmit power using individual channel state information (CSI) such that the GEE is maximized in a distributed manner. We prove the convergence of the proposed iterative algorithm to its corresponding unique fixed point also shown by our numerical results. Additionally, simulation results demonstrate that our proposed scheme outperforms other algorithms in the literature and performs like the centralized algorithm executed in the base station and maximizes the GEE using the global CSI.
Probabilistic Cellular Automata for Granular Media in Video Games
Jonathan Devlin, Micah D. Schuster
Granular materials are very common in the everyday world. Media such as sand, soil, gravel, food stuffs, pharmaceuticals, etc. all have similar irregular flow since they are composed of numerous small solid particles. In video games, simulating these materials increases immersion and can be used for various game mechanics. Computationally, full scale simulation is not typically feasible except on the most powerful hardware and tends to be reduced in priority to favor other, more integral, gameplay features. Here we study the computational and qualitative aspects of side profile flow of sand-like particles using cellular automata (CA). Our CA uses a standard square lattice that updates via a custom, modified Margolus neighborhood. Each update occurs using a set of probabilistic transitions that can be tuned to simulate friction between particles. We focus on the look of the sandpile structure created from an hourglass shape over time using different transition probabilities and the computational impact of such a simulation.
Wireless Feedback Control with Variable Packet Length for Industrial IoT
Kang Huang, Wanchun Liu, Yonghui Li
et al.
The paper considers a wireless networked control system (WNCS), where a controller sends packets carrying control information to an actuator through a wireless channel to control a physical process for industrial-control applications. In most of the existing work on WNCSs, the packet length for transmission is fixed. However, from the channel-encoding theory, if a message is encoded into a longer codeword, its reliability is improved at the expense of longer delay. Both delay and reliability have great impact on the control performance. Such a fundamental delay-reliability tradeoff has rarely been considered in WNCSs. In this paper, we propose a novel WNCS, where the controller adaptively changes the packet length for control based on the current status of the physical process. We formulate a decision-making problem and find the optimal variable-length packet-transmission policy for minimizing the long-term average cost of the WNCSs. We derive a necessary and sufficient condition on the existence of the optimal policy in terms of the transmission reliabilities with different packet lengths and the control system parameter.
Wireless Network Virtualization With SDN and C-RAN for 5G Networks: Requirements, Opportunities, and Challenges
E. J. Kitindi, Shu Fu, Yunjian Jia
et al.
Wireless network virtualization (WNV) has drawn attention from the researchers ranging from academia to industry as one of the significant technologies in the cellular network communication. It is considered as a pioneer to achieve effective resource utilization with decreased operating expenses and capital expenses by decoupling the networks functionalities of coexisting virtual networks. It facilitates fast deployment of new services and novel technologies. WNV paradigm is in the early stages, and there is a large room for the research community to develop new architectures, systems, and applications. The availability of software-defined networking (SDN) and cloud/centralized radio access network (C-RAN) steers up the hope for the WNV realization. This paper surveys WNV along with the recent developments in SDN and C-RAN technologies. Based on these technologies and WNV concepts, we identify the requirements and opportunities of future cellular networks. We then propose a general architectural framework for the WNV based on SDN. In-depth discussion of challenges and research issues as well as promising approaches for future networks communication improvements are also proposed. Finally, we give several promising candidates of future network services for residential customers and business customers.
91 sitasi
en
Computer Science
Dynamic Service Composition Orchestrated by Cognitive Agents in Mobile & Pervasive Computing
Oscar J. Romero
Automatic service composition in mobile and pervasive computing faces many challenges due to the complex nature of the environment. Common approaches address service composition from optimization perspectives which are not feasible in practice due to the intractability of the problem, limited computational resources of smart devices, service host's mobility, and time constraints. Our main contribution is the development of a cognitively-inspired agent-based service composition model focused on bounded rationality rather than optimality, which allows the system to compensate for limited resources by selectively filtering out continuous streams of data. The evaluation of our approach shows promising results when compared against state-of-the-art service composition models.
NLSC: Unrestricted Natural Language-based Service Composition through Sentence Embeddings
Oscar J. Romero, Ankit Dangi, Sushma A. Akoju
Current approaches for service composition (assemblies of atomic services) require developers to use: (a) domain-specific semantics to formalize services that restrict the vocabulary for their descriptions, and (b) translation mechanisms for service retrieval to convert unstructured user requests to strongly-typed semantic representations. In our work, we argue that effort to developing service descriptions, request translations, and matching mechanisms could be reduced using unrestricted natural language; allowing both: (1) end-users to intuitively express their needs using natural language, and (2) service developers to develop services without relying on syntactic/semantic description languages. Although there are some natural language-based service composition approaches, they restrict service retrieval to syntactic/semantic matching. With recent developments in Machine learning and Natural Language Processing, we motivate the use of Sentence Embeddings by leveraging richer semantic representations of sentences for service description, matching and retrieval. Experimental results show that service composition development effort may be reduced by more than 44\% while keeping a high precision/recall when matching high-level user requests with low-level service method invocations.