This paper investigates the automatic exploration problem under the unknown environment, which is the key point of applying the robotic system to some social tasks. The solution to this problem via stacking decision rules is impossible to cover various environments and sensor properties. Learning-based control methods are adaptive for these scenarios. However, these methods are damaged by low learning efficiency and awkward transferability from simulation to reality. In this paper, we construct a general exploration framework via decomposing the exploration process into the decision, planning, and mapping modules, which increases the modularity of the robotic system. Based on this framework, we propose a deep reinforcement learning-based decision algorithm that uses a deep neural network to learning exploration strategy from the partial map. The results show that this proposed algorithm has better learning efficiency and adaptability for unknown environments. In addition, we conduct the experiments on the physical robot, and the results suggest that the learned policy can be well transferred from simulation to the real robot.
Contacts are central to most manipulation tasks as they provide additional dexterity to robots to perform challenging tasks. However, frictional contacts leads to complex complementarity constraints. Planning in the presence of contacts requires robust handling of these constraints to find feasible solutions. This paper presents PY ROBO COP which is a lightweight Python-based package for control and optimization of robotic systems described by nonlinear Differential Algebraic Equations (DAEs). In particular, the proposed optimization package can handle systems with contacts that are described by complementarity constraints. We also present a general framework for specifying obstacle avoidance constraints using complementarity constraints. The package performs direct transcription of the DAEs into a set of nonlinear equations by performing orthogonal collocation on finite elements. The resulting optimization problem belongs to the class of Mathematical Programs with Complementarity Constraints (MPCCs). MPCCs fail to satisfy commonly assumed constraint qualifications and require special handling of the complementarity constraints in order for NonLinear Program (NLP) solvers to solve them effectively. PY ROBO COP provides automatic reformulation of the complementarity constraints that enables NLP solvers to perform optimization of robotic systems. The package is interfaced with ADOL-C for obtaining sparse derivatives by automatic differentiation and IPOPT for performing optimization. We provide extensive numerical examples for various different robotic systems with collision avoidance as well as contact constraints represented using complementarity constraints. We provide comparisons with other open source optimization packages like CasADi and Pyomo. The code is open sourced and available at https://github.com/merlresearch/PyRoboCOP. Note to Practitioners—PY ROBO COP is intended to be an easy-to-use software package written in Python which can be used for optimization, estimation and control for a large class of robotic systems. Including, in particular, contact-rich applications to deal with complex scenarios that arise when making and breaking contacts during a task. Typical problems that can be solved with our work are trajectory and control sequence optimization, parameter estimation. To make the proposed software package easier for practitioners, the paper provides access to the package and a large number of example problems. Furthermore, the package also provides a guide describing the details of all the methods a user might have to implement for their own system. Compared to some of the other packages, PY ROBO COP works with NumPy object arrays which is the native computing package in Python. We believe that this will make it much easier to learn and use compared to some of the other optimal control packages.
This paper investigates the problem of fuzzy adaptive finite-time inverse optimal control for active suspension systems (ASSs). The fuzzy logic systems (FLSs) are utilized to learn the unknown non-linear dynamics and an auxiliary system is established. Based on the finite-time stability theory and inverse optimal theory, a fuzzy adaptive inverse finite-time inverse optimal control method is proposed. It is proven that the formulated control approach guarantees the stability of the controlled systems, while ensuring that errors converge to a small neighborhood of zero within finite time. Moreover, the optimized control performance can be achieved. Eventually, the simulation results demonstrate the effectiveness of the proposed fuzzy adaptive finite-time inverse optimal control scheme.
Control engineering systems. Automatic machinery (General), Electronic computers. Computer science
Learning is a key function in the brain to be able to achieve the activity patterns required to perform various activities. While specific behaviors are determined by activity in localized regions, the interconnections throughout the entire brain play a key role in enabling its ability to exhibit desired activity. To mimic this setup, this paper examines the use of reservoir computing to control a linear-threshold network brain model to a desired trajectory. We first formally design open- and closed-loop controllers that achieve reference tracking under suitable conditions on the synaptic connectivity. Given the impracticality of evaluating closed-form control signals, particularly with growing network complexity, we provide a framework where a reservoir of a larger size than the network is trained to drive the activity to the desired pattern. We illustrate the versatility of this setup in two applications: selective recruitment and inhibition of neuronal populations for goal-driven selective attention, and network intervention for the prevention of epileptic seizures.
Control engineering systems. Automatic machinery (General), Technology
Mohammad Alali, Armita Kazeminajafabadi, Mahdi Imani
Advances in technology have enabled the use of sensors with varied modalities to monitor different parts of systems, each providing diverse levels of information about the underlying system. However, resource limitations and computational power restrict the number of sensors/data that can be processed in real-time in most complex systems. These challenges necessitate the need for selecting/scheduling a subset of sensors to obtain measurements that guarantee the best monitoring objectives. This paper focuses on sensor scheduling for systems modeled by hidden Markov models. Despite the development of several sensor selection and scheduling methods, existing methods tend to be greedy and do not take into account the long-term impact of selected sensors on monitoring objectives. This paper formulates optimal sensor scheduling as a reinforcement learning problem defined over the posterior distribution of system states. Further, the paper derives a deep reinforcement learning policy for offline learning of the sensor scheduling policy, which can then be executed in real-time as new information unfolds. The proposed method applies to any monitoring objective that can be expressed in terms of the posterior distribution of the states (e.g. state estimation, information gain, etc.). The performance of the proposed method in terms of accuracy and robustness is investigated for monitoring the security of networked systems and the health monitoring of gene regulatory networks.
Control engineering systems. Automatic machinery (General), Systems engineering
In this article, the fuzzy dynamic event-triggered tracking control problem of semilinear parabolic systems (SLPSs) with time-varying delay is investigated. First, T-S fuzzy partial differential equation models are introduced to describe the SLPSs. Second, a less conservative and more general fuzzy dynamic event-triggered strategy (DETS) is proposed to reduce communication consumption and avoid unnecessary continuous signal monitoring. Since the dynamic threshold is closely related to the currently sampled signal and the latest successfully transmitted signal, it can be promptly dynamically adjusted. In addition, on the basis of a reasonable assumption, a novel linear matrix inequality (LMI) relax technique is introduced to deal with the mismatched premise variables between the fuzzy systems and the fuzzy controller. By constructing the appropriate Lyapunov–Krasovskii candidate functional, the criterion that the SLPSs can asymptotically track the target systems is derived, and the desired dynamic event-triggered control gains can be obtained by solving a set of LMIs. The DETS reduces effectively communication resource consumption. Finally, the control problem of temperature distribution of catalytic rod in practical engineering application is given to verify the effectiveness and superiority of the proposed control scheme.
An emerging branch of control theory specialises in certificate learning, concerning the specification of a desired (possibly complex) system behaviour for an autonomous or control model, which is then analytically verified by means of a function-based proof. However, the synthesis of controllers abiding by these complex requirements is in general a non-trivial task and may elude the most expert control engineers. This results in a need for automatic techniques that are able to design controllers and to analyse a wide range of elaborate specifications. In this paper, we provide a general framework to encode system specifications and define corresponding certificates, and we present an automated approach to formally synthesise controllers and certificates. Our approach contributes to the broad field of safe learning for control, exploiting the flexibility of neural networks to provide candidate control and certificate functions, whilst using SMT-solvers to offer a formal guarantee of correctness. We test our framework by developing a prototype software tool, and assess its efficacy at verification via control and certificate synthesis over a large and varied suite of benchmarks.
Abstract This paper focuses on the global prespecifiable fixed‐time control problem of uncertain strict‐feedback nonlinear system with input quantization. In contrast to the existing fixed‐time control results, the signs of the control gains can be unknown in our paper. Unlike the traditional fixed‐time control schemes, some exponential functions are added into the virtual controllers to accelerate the convergence rate. Besides, some dynamic switching functions are also introduced to compensate the unknown control signs. Based on the novel regulate law, the switching functions can be regulated online to ensure the fixed‐time stability for the considered system. Furthermore, the settling time can be easily adjusted by selecting the controller parameters. Simulation results are provided to illustrate the effectiveness of the proposed control scheme.
Control engineering systems. Automatic machinery (General)
The detection and tracking of small and weak maneuvering radar targets in complex electromagnetic environments is still a difficult problem to effectively solve. To address this problem, this paper proposes a dynamic programming tracking-before-detection method based on long short-term memory (LSTM) network value function(VL-DP-TBD). With the help of the estimated posterior probability provided by the designed LSTM network, the calculation of the posterior value function of the traditional DP-TBD algorithm can be more accurate, and the detection and tracking effect achieved for maneuvering small and weak targets is improved. Utilizing the LSTM network to model the posterior probability estimation of the target motion state, the posterior probability moving features of the maneuvering target can be learned from the noisy input data. By incorporating these posterior probability estimation values into the traditional DP-TBD algorithm, the accuracy and robustness of the calculation of the posterior value function can be enhanced, so that the improved architecture is capable of effectively recursively accumulating the movement trend of the target. Simulation results show that the improved architecture is able to effectively reduce the aggregation effect of a posterior value function and improve the detection and tracking ability for non-cooperative nonlinear maneuvering dim small target.AbbreviationsLSTM: Long short-term memory; DP-TBD: Dynamic programming-based tracking before detection; DBT: Detection before tracking; TBD: Tracking before detection; HT-TBD: Tracking-before-detection algorithm based on the Hough transform; PF-TBD: Tracking-before-detection algorithm based on particle filtering; RFS-TBD: Tracking-before-detection algorithm based on random finite sets; SNR: Signal-to-noise ratio; DP: Dynamic programming; EVT: Extreme value theory; EVT: Generalized extreme value theory; GLRT: Generalized likelihood ratio detection; KT: Keystone transformation; PGA: Phase gradient autofocusing; CFAR: Constant false-alarm rate; J-CA-CFAR: Joint intensity-spatial CFAR; MF: Merit function; CP-DP-TBD: Candidate plot-based DP-TBD; CIT: Coherent integration time; RNN: Recurrent neural network; CS: Current statistical; Pd: Detection probability; Pt: Tracking probability.
Control engineering systems. Automatic machinery (General), Systems engineering
Semi-global bipartite fault-tolerant containment control framework on antagonistic communication networks is proposed in this article for heterogeneous multiagent systems (MASs) under the influence of input saturation and actuator faults. An observer is constructed to estimate the leaders’ states on signed digraph, where the communication networks are antagonistic. A fully distributed virtual control approach is developed to acquire the containment trajectory. Based on the observer, a semi-global containment control method is developed to compensate for the detrimental impacts of both input saturation and actuator faults. Besides, the dynamics and state-space dimensions of the agents can be different. The proposed framework overcomes two drawbacks of the conventional containment control: 1) the containment trajectory is obtained under general antagonistic communication networks, which is more general in engineering applications and 2) both actuator faults and input saturation are solved for heterogeneous agents, which relaxes the limitation of homogeneous dynamics. Finally, a simulation example is conducted to test and verify the feasibility of the proposed method framework.
Rotor dynamic balancing is essential in rotor industrial. The conventional balancing methods, including the influence coefficients method and modal balancing method, are effective, but lack economy and sufficient usage of the data. To overcome the disadvantages of the conventional balancing methods, a balancing method using unsupervised deep learning without weight trails had been proposed. The proposed network could identify the unbalanced forces from the data observed from just one run of the rotor and without labels. To validate the novel balancing method, an experimental rig is well-designed and established. Experimental validation and comparison with influence coefficients method are conducted. The experimental results show that the proposed balancing method gives consideration to both cost and accuracy. Compared with influence coefficients method, no extra weight trail process is needed and balancing performances are comparative. The experimental rig can be used for proving the scheme and for further same kind of research.
Control engineering systems. Automatic machinery (General), Technology (General)
Complex systems seen either in general engineering practice or economics are subjected to ever increased uncertainties that are mostly represented as random variables or parameters, and the characteristics of random variables are represented by their probability density functions (PDFs). Controlling their PDFs means to shape their stochastic distributions and in general it would provide a full treatment for system analysis and operational control and optimization. This leads to the development of stochastic distribution control (SDC) systems theory in the past decades, where the original aim of the controller design is to realize a shape control of the distributions of certain random variables in their PDFs sense for some engineering processes. Indeed, once the PDFs of these random variables or parameters are used to describe their distribution characters, the control task is to obtain control signals so that the output PDFs of stochastic systems are made to follow their target PDFs. The subject of SDC was initially originated for non‐Gaussian stochastic control systems design but has found a wide spectrum of applications in general systems in terms of data‐driven modeling, analysis, signal processing (filtering), data mining via multivariable statistics, decision‐making (optimization) for systems subjected to uncertainties and even in economics. In this context, SDC constitutes an effective primer tool for complex system analysis, control and operational optimizations. In this review paper, a detailed survey of the developments on the research of SDC systems will be made together with their wide spectrum applications and future perspectives.
Abstract Swarm robots are a branch of robotics that draws inspiration from biological swarms to mimic their collective behavior. Automatic design methods are part of swarm engineering, depend on artificial intelligence algorithms to produce the collective behavior of robots. In general, they follow two-approach evolutionary algorithms like practical swarm optimization and reinforcement learning. This paper studies these approaches, illustrating the effect of modifications and enhancements of algorithms for both directions, showing important parameters considered for the best performance of the swarm, and explaining the methods and advantages of using deep learning to reinforcement learning.
Originating in the artificial intelligence literature, optimistic planning (OP) is an algorithm that generates near-optimal control inputs for generic nonlinear discrete-time systems whose input set is finite. This technique is, therefore, relevant for the near-optimal control of nonlinear switched systems for which the switching signal is the control, and no continuous input is present. However, OP exhibits several limitations, which prevent its desired application in a standard control engineering context, as it requires, for instance, that the stage cost takes values in $[0, 1]$, an unnatural prerequisite, and that the cost function is discounted. In this article, we modify OP to overcome these limitations, and we call the new algorithm ${\rm OP}_{\text{min}}$. We then analyze ${\rm OP}_{\text{min}}$ under general stabilizability and detectability assumptions on the system and the stage cost. New near-optimality and performance guarantees for ${\rm OP}_{\text{min}}$ are derived, which have major advantages compared to those originally given for OP. We also prove that a system whose inputs are generated by ${\rm OP}_{\text{min}}$ in a receding-horizon fashion exhibits stability properties. As a result, ${\rm OP}_{\text{min}}$ provides a new tool for the near-optimal, stable control of nonlinear switched discrete-time systems for generic cost functions.
Global orbits connect the saddle points in an infinite period through the homoclinic and heteroclinic types of manifolds. Different from the periodic movement analysis, it requires special strategies to obtain expression of the orbit and detect the associated profound dynamic behaviors, such as chaos. In this paper, a global dynamic frequency method is applied to detect the homoclinic and heteroclinic bifurcation of the complicated nonlinear systems. The so-called dynamic frequency refers to the newly introduced frequency that varies with time t , unlike the usual static variable. This new method obtains the critical bifurcation value as well as the analytic expression of the orbit by using a standard five-step hyperbolic function-balancing procedure, which represents the influence of the higher harmonic terms on the global orbit and leads to a significant reduction of calculation workload. Moreover, a new homoclinic manifold analysis maps the periodic excitation onto the target global manifold that transfers the chaos discussion of non-autonomous systems into the orbit computation of the general autonomous system. That strategy unifies the global bifurcation analysis into a standard orbit approximation procedure. The numerical simulation results are shown to compare with the predictions.
Control engineering systems. Automatic machinery (General), Acoustics. Sound
Controlling complex systems to desired states is of primary importance in science and engineering. In the classical control framework, the plants to be controlled usually do not have their own payoff or objective functions; however, this is not the case in many practical situations in, for examples, social, economic, and “intelligent” engineering systems. This motivates our introduction of the game-based control system (GBCS), which has a hierarchical decision-making structure: one regulator and multiple agents. The regulator is regarded as the global controller that makes decision first, and then, the agents try to optimize their respective objective functions to reach a possible Nash equilibrium as a result of noncooperative dynamic game. A fundamental issue in the GBCS is: Is it possible for the regulator to change the macrostates by regulating the Nash equilibrium formed by the agents at the lower level? This leads to the investigation of controllability of the Nash equilibrium of the GBCS. In this paper, we will first formulate this new problem in a general nonlinear framework and then focus on linear systems. Some explicit necessary and sufficient algebraic conditions on the controllability of the Nash equilibrium are given for a linear GBCS, by solving the controllability problem of the associated forward and backward dynamic equations, which is a key technical issue and has rarely been explored in the literature.
The increasing penetration of renewable and distributed energy resources in distribution networks calls for real-time and distributed voltage control. In this article, we investigate local Volt/VAR control with a general class of control functions, and show that the power system dynamics with nonincremental local voltage control can be seen as a distributed algorithm for solving a well-defined optimization problem (reverse engineering). The reverse engineering further reveals a fundamental limitation of the nonincremental voltage control: the convergence condition is restrictive and prevents better voltage regulation at equilibrium. This motivates us to design two incremental local voltage control schemes based on the subgradient and pseudo-gradient algorithms, respectively, for solving the same optimization problem (forward engineering). The new control schemes decouple the dynamical property from the equilibrium property, and have much less restrictive convergence conditions. This article presents another step toward developing a new foundation—network dynamics as optimization algorithms—for distributed real-time control and optimization of future power networks.
Abstract Mastering the fusion of information and communication technologies with physical systems to cyber-physical automation systems is of main concern to engineers in the industrial automation domain. The engineering of these systems is challenging as their distributed nature and the heterogeneity of stakeholders and tools involved in their engineering contradict the need for the simultaneous engineering of their cyber and physical parts over their life cycle. This paper presents a novel approach based on the virtual engineering method, which provides support for the simultaneous engineering of the cyber and physical parts of automation systems. The approach extends and integrates the life cycle centered view mandated by current conceptual architectures and the digital twin paradigm with an integrated, iterative engineering method. The benefits of the approach are highlighted in a case study related to the engineering of the control logic of a cyber physical automation system originating from the process engineering domain. We describe for the first time a modular domain ontology, which formally describes the cyber and physical part of the system. We present cyber services built on top of the ontology layer, which allow to automatically verify different control logic types and simultaneously verify cyber and physical parts of the system in an incremental manner.