Keywords: Manufacturing systems and automation, Learning
Abstract: Manufacturing has undergone significant changes over the past five-ten years thanks to technological advancements that have been leveraged to meet a diverse set of customer requirements driven by global and societal needs. Conventional manufacturing control strategies were typically designed for robustness and speed within a controlled and well-regulated environment. However, recent demands for customization and agility coupled with big data investments have provided an opportunity for more learning-based methods to be introduced. Data driven strategies have long provided a means of harnessing information to enhance the performance of these complex systems. This talk is motivated by real-world interest from industry in understanding how to combine data-based learning and experiential knowledge to make intelligent decisions that can save time, money, and resources.

In this talk, we examine which aspects of manufacturing processes lend themselves to learning strategies and which bring additional challenges. We also explore cases in which learning has been applied in different ways to additive manufacturing processes in order to improve process knowledge and performance. Opportunities for additional integration of learning into the manufacturing domain will be discussed and open research questions for control-theoretic advancements will be highlighted.

Keywords: Hybrid systems, Optimal control
Abstract: This paper presents an algorithm for solving the optimization problem associated with hybrid model predictive control for a class of discretized hybrid control systems. The proposed approach consists of reformulating the optimal control problem as a mixed integer quadratic problem (MIQP), which can be solved using well-established algorithms in the literature. Specifically, the given discretized hybrid control system is transformed into a mixed logical dynamical (MLD) system that, for the class of discretized hybrid control systems considered, gives rise to an MIQP. The MLD model is obtained through an intermediate step that transforms the discretized hybrid control system into a discrete-time control system.

Keywords: Hybrid systems, Predictive control for linear systems
Abstract: This paper presents identities for calculating robust successor and precursor sets of discrete-time linear hybrid systems. The proposed technique relies on generating a set containing all possible state transitions of a dynamic system over a region of interest, named the state-update set. Forward and backward reachability can then be performed using only projection, intersection, and Minkowski difference set operations with the state-update set. It is shown how state-update sets may be defined as hybrid zonotopes for mixed logical dynamical systems and linear systems in closed loop with model predictive control

Keywords: Hybrid systems, Nonlinear systems, Robust control
Abstract: In this paper, we show that a hybrid integrator-gain system (HIGS) is a nonlinear negative imaginary (NNI) system. We prove that the positive feedback interconnection of a linear negative imaginary (NI) system and a HIGS is asymptotically stable. We apply the HIGS to a MEMS nanopositioner, as an example of a linear NI system, in a single-input single-output framework. We analyze the stability and the performance of the closed-loop interconnection in both time and frequency domains through simulations and demonstrate the applicability of HIGS as an NNI controller to a linear NI system.

Keywords: Hybrid systems, Observers for Linear systems, Lyapunov methods
Abstract: We propose an observer design method for hybrid systems with linear maps and known jump times based on decomposing the state into parts exhibiting different kinds of observability properties. Using a series of transformations depending on the time elapsed since the previous jump, the state may be decomposed into up to three parts, where the first one is instantaneously observable during flows from the flow output, the second one detectable at jumps from the jump output thanks to the combination of flows and jumps, and the remaining part naturally detectable at jumps still thanks to this combination of flows and jumps but implicitly from the flow output. An observer is then designed to estimate each part, relying on a flow-based Kalman-like observer exploiting the flow output for the first part, a jump-based observer exploiting the jump output for the second, and a jump-based observer exploiting a fictitious output for the third. Global exponential stability of the estimation error is proven using Lyapunov analysis.

Keywords: Hybrid systems, Stability of hybrid systems
Abstract: Infinitesimal contraction analysis, wherein global convergence results are obtained from properties of local dynamics, is a powerful analysis tool. In this paper, we generalize infinitesimal contraction analysis to hybrid systems in which state-dependent guards trigger transitions defined by reset maps between modes that may have different norms and need not be of the same dimension. In contrast to existing literature, we do not restrict mode sequence or dwell time. We work in settings where the hybrid system flow is differentiable almost everywhere and its derivative is the solution to a jump-linear-time-varying differential equation whose jumps are defined by a saltation matrix determined from the guard, reset map, and vector field. Our main result shows that if the vector field is infinitesimally contracting, and if the saltation matrix is non-expansive, then the intrinsic distance between any two trajectories decreases exponentially in time. When bounds on dwell time are available, our approach yields a bound on the intrinsic distance between trajectories regardless of whether the dynamics are expansive or contractive. We illustrate our results using wo examples: a constrained mechanical system and an electrical circuit with an ideal diode.

Keywords: Hybrid systems, Maritime control, Stability of hybrid systems
Abstract: This paper generalizes results on synergistic hybrid feedback control. Specifically, we propose a generalized definition of synergistic Lyapunov functions and feedbacks which allows the logic variable in traditional synergistic control, denoted the synergy variable, to be vector-valued and change during flows. Moreover, we introduce synergy gaps relative to components of product sets, which enables us to define jump conditions in the form of synergy gaps for different components of the synergy variable. In particular, this enables us to formulate existing hybrid output feedback control schemes within the synergistic control framework. Furthermore, we show that our generalized definition is amenable to backstepping. Finally, we give an example of how traditional synergistic control can be combined with ship maneuvering control with discrete path dynamics.

Keywords: Adaptive control, Optimization algorithms, Constrained control
Abstract: We introduce an algorithm that we call Practically Safe Extremum Seeking (PSfES), which seeks to minimize an unknown objective function while avoiding unsafe regions of state space, except for a possible minor violation of the safety boundaries, which can be arbitrarily reduced using the algorithm's design parameters, such as the perturbation frequency and amplitude. We allow the metric of safety---a barrier function---to be functionally unknown. Only the value of the barrier function is assumed measured. We introduce dynamic filters which emulate, in an average and singularly perturbed sense, the feedback law of a standard quadratic programming (QP) and control barrier function (CBF) based safety filter, acting on a nominal extremum seeking (ES) controller. These filters have the effect of enabling the use of the averaging and singular perturbation theorems, which enable us to guarantee convergence (practical) to near a point in the safe set and safety (practical) during the transient, both in a local sense. We present a design for multiple dimensions but provide an analysis in one dimension. Finally, the behavior of our controller is demonstrated in simulation for two examples.

Keywords: Adaptive control, Nonlinear systems identification, Power systems
Abstract: A novel adaptive Koopman based model predictive control (MPC) algorithm for wind farm control is presented. Using the data-driven Koopman approach the highly non-linear wake effects governing wind farm dynamics can be efficiently modelled. An update rule is presented to enable online learning only when new information is available. Moreover, to provide sufficient excitation of all relevant model frequencies in closed loop, a small test signal is superimposed on the control input while the Koopman model is updated. Simulation studies in the WFSim environment illustrate excellent accuracy for wind speed estimation. In closed-loop, the adaptive online-update strategy tracks reference farm yield well, considerably outperforming recently presented non-adaptive schemes.

Keywords: Adaptive control, Optimal control
Abstract: This paper addresses neural network (NN) based optimal adaptive regulation of uncertain nonlinear discrete-time systems in affine form using output feedback via lifelong concurrent learning. First, an adaptive NN observer is introduced to estimate both the state vector and control coefficient matrix, and its NN weights are adjusted using both output error and concurrent learning term to relax the persistency excitation (PE) condition. Next, by utilizing an actor-critic framework for estimating the value functional and control policy, the critic network weights are tuned via both temporal different error and concurrent learning schemes through a replay buffer. The actor NN weights are tuned using control policy errors. To attain lifelong learning for performing effectively during multiple tasks, an elastic weight consolidation term is added to the critic NN weight tuning law. The state estimation, regulation, and the weight estimation errors of the observer, actor and critic NNs are demonstrated to be bounded when performing tasks by using Lyapunov analysis. Simulation results are carried out to verify the effectiveness of the proposed approach on a Vander Pol Oscillator. Finally, extension to optimal tracking is given briefly.

Keywords: Flight control, Adaptive control, Control applications
Abstract: This letter considers the problem of global asymptotic position and heading tracking for multirotors. We propose a hybrid adaptive feedback control law that globally asymptotically tracks a position and heading reference in the presence of unknown constant disturbances in both the translational and rotational dynamics. By employing a tuning function-based backstepping approach, the number of parameter estimates are minimized. Moreover, we propose a novel control law for the translational subsystem, which leads to a simpler virtual control law when backstepping. Global asymptotic heading tracking is achieved through a novel construction of the desired rotation matrix. The theory is verified through experiments on a quadrotor.

Keywords: Lyapunov methods, Adaptive control, Large-scale systems
Abstract: We solve the problem of decentralized adaptive stabilization for infinite networks of hierarchically intercon- nected switched systems in strict-feedback form with unknown switching signals and with unknown control directions. We assume that each node is interconnected with only finite set of its neighbors and the number of neighbors of each node has a uniform upper boundary. The trajectories of the closed-loop system with our switching-independent adaptive feedback law converge to the origin with respect to the l∞-norm, and this property holds uniformly with respect to the unknown switching signals.

Keywords: Machine learning, Adaptive control, Identification
Abstract: This paper defines geometric criteria which are then used to establish sufficient conditions for persistency of excitation with vector functions constructed from single hidden-layer neural networks with step or ReLU activation functions. We show that these conditions hold when employing reference system tracking, as is commonly done in adaptive control. We demonstrate the results numerically on a system with linearly parameterized activations of this type and show that the parameter estimates converge to the true values with the sufficient conditions met.

Keywords: Robotics, Autonomous robots, Autonomous vehicles
Abstract: In this work, we extend a recently proposed methodology to construct artificial vector fields for robot navigation in n-dimensional spaces to track and circulate time-varying curves. Now, we incorporate the ability to deviate from obstacles that might be static or dynamic by constructing a collision-free vector field. It can be considered a reactive approach in which the obstacles can be locally sensed and then circumnavigated at a fixed distance. The novel vector field that allows the circumnavigation of obstacles and the traversal of a target curve is built upon the consideration of the closest points on the obstacles, which can assume a generic shape, and the closest point on the curve. In order to validate our method, we present ROS-based computational simulations and real world experiments with a wheeled robot and an UAV quadcopter.

Keywords: Robotics, Healthcare and medical systems, Optimal control
Abstract: The paper addresses the rehabilitation of disabled patients using a lower-limb fully-actuated exoskeleton. We propose a numerical method to replan the current step without jeopardizing stability. Stability is evaluated in the light of a simple linear time-invariant surrogate model. The method's core is the analysis of an input-constrained optimal control problem with state specified at an unspecified terminal time. A detailed study of the extremals given by Pontryagin Maximum Principle is sufficient to characterize its feasibility. This allows a fast replanning strategy. The efficiency of the numerical algorithm (resolution time below 1,ms) yields responsiveness to the patient's request. Realistic simulations on a full-body model of the patient-exoskeleton system stress that cases of practical interest for physiotherapists are well-addressed.

Keywords: Robotics, Predictive control for nonlinear systems
Abstract: This paper presents a two-stage prediction-based control scheme for embedding the environment's geometric properties into a collision-free Pythagorean Hodograph spline, and subsequently finding the optimal path within the parameterized free space. The ingredients of this approach are twofold: First, we present a novel spatial path parameterization applicable to any arbitrary curve without prior assumptions in its adapted frame. Second, we identify the appropriateness of Pythagorean Hodograph curves for a compact and continuous definition of the path-parametric functions required by the presented spatial model. This dual-stage formulation results in a motion planning approach, where the geometric properties of the environment arise as states of the prediction model. Thus, the presented method is attractive for motion planning in dense environments. The efficacy of the approach is evaluated according to an illustrative example.

Keywords: Robotics, Nonlinear systems, Biomedical
Abstract: Task-invariant approaches are desirable in exoskeleton control design as they have the potential of providing consistent assistance across locomotor tasks. Different from traditional trajectory-tracking approaches that are specific to tasks and users, task-invariant control approaches do not replicate normative joint kinematics, which could eliminate the need for task detection and allow more flexibility for human users. In this paper, we propose a task-invariant control paradigm for lower-limb exoskeletons to alter the human user’s centroidal momentum, i.e., a sum of projected limb momenta onto the human’s center of mass. We design a virtual reference model based on human user’s self-selected gaits to provide a reference centroidal momentum for the exoskeleton to track and make it adaptable to changes in gait patterns. Mathematically, the proposed approach reduces the control design problem into a lower-dimensional space. With the number of actuators being greater than the dimension of the centroidal momentum vector, we can guarantee the existence of a centroidal momentum shaping law for underactuated systems through optimization. Simulation results on a human-like biped show that the proposed shaping strategy can produce beneficial results on assisting human locomotion, such as metabolic cost reduction.

Keywords: Robotics, Large-scale systems, Distributed parameter systems
Abstract: This work studies a robot-assisted crowd evacuation problem where we control a small group of robots to guide a large human crowd to safe locations. The challenge lies in how to model human-robot interactions and design robot controls to indirectly control a human population that significantly outnumbers the robots. To address the challenge, we treat the crowd as a continuum and formulate the evacuation objective as driving the crowd density to target locations. We propose a novel mean-field model which consists of a family of microscopic equations that explicitly model how human motions are locally guided by the robots and an associated macroscopic equation that describes how the crowd density is controlled by the navigation velocity fields generated by all robots. Then, we design density feedback controllers for the robots to dynamically adjust their states such that the generated navigation velocity fields drive the crowd density to a target density. Stability guarantees of the proposed controllers are proven. Agent-based simulations are included to evaluate the proposed evacuation algorithms.

Keywords: Robotics, Distributed control, Stability of nonlinear systems
Abstract: Quadratic programming (QP)-based nonlinear controllers have gained increasing popularity in the legged locomotion community. This paper presents a formal foundation to systematically decompose QP-based centralized nonlinear controllers into a network of lower-dimensional local QPs, with application to legged locomotion. The proposed approach formulates a feedback structure between the local QPs and assumes a one-step communication delay protocol. The properties of local QPs are analyzed, wherein it is established that their steady-state solutions on periodic orbits (representing gaits) coincide with that of the centralized QP. The asymptotic convergence of local QPs' solutions to the steady-state solution is studied via Floquet theory. The effectiveness of the analytical results is evaluated through rigorous numerical simulations and various experiments on a quadrupedal robot, with the result being robust locomotion on different terrains and in the presence of external disturbances. The paper shows that the proposed distributed QPs have considerably less computation time and reduced noise propagation sensitivity than the centralized QP.

Keywords: Neural networks, Machine learning, Large-scale systems
Abstract: Implicit neural networks (INNs) are a class of learning models that use implicit algebraic equations as layers and have been shown to exhibit several notable benefits over traditional feedforward neural networks (FFNNs). In this paper, we use interval reachability analysis to study robustness of INNs and compare them with FFNNs. We first introduce the notion of tight inclusion function and use it to provide the tightest rectangular over-approximation of the neural network's input-output map. We also show that tight inclusion functions lead to sharper robustness guarantees than the well-studied robustness measures of Lipschitz constants. Like exact Lipschitz constants, tight inclusions functions are computationally challenging to obtain, and thus we develop a framework based upon mixed monotonicity and contraction theory to estimate the tight inclusion functions for INNs. We show that our approach performs at least as well as, and generally better than, state-of-the-art interval-bound propagation methods for INNs. Finally, we design a novel optimization problem for training robust INNs and we provide empirical evidence that suitably-trained INNs can be more robust than comparably-trained FFNNs.

Keywords: Machine learning, Neural networks, Distributed parameter systems
Abstract: We consider the controllability problem for the continuity equation, corresponding to neural ordinary differential equations (ODEs), which describes how a probability measure is pushedforward by the flow. We show that the controlled continuity equation has very strong controllability properties. Particularly, a given solution of the continuity equation corresponding to a bounded Lipschitz vector field defines a trajectory on the set of probability measures. For this trajectory, we show that there exist piecewise constant training weights for a neural ODE such that the solution of the continuity equation corresponding to the neural ODE is arbitrarily close to it. As a corollary to this result, we establish that the continuity equation of the neural ODE is approximately controllable on the set of compactly supported probability measures that are absolutely continuous with respect to the Lebesgue measure.

Keywords: Neural networks, Uncertain systems
Abstract: This paper presents a new reachability analysis approach to compute interval over-approximations of the output set of feedforward neural networks with input uncertainty. We adapt to neural networks an existing mixed-monotonicity method for the reachability analysis of dynamical systems and apply it to each partial network within the main network. This ensures that the intersection of the obtained results is the tightest interval over-approximation of the output of each layer that can be obtained using mixed-monotonicity on any partial network decomposition. Unlike other tools in the literature focusing on small classes of piecewise-affine or monotone activation functions, the main strength of our approach is its generality: it can handle neural networks with any Lipschitz-continuous activation function. In addition, the simplicity of our framework allows users to very easily add unimplemented activation functions, by simply providing the function, its derivative and the global argmin and argmax of the derivative. Our algorithm is compared to five other interval-based tools (Interval Bound Propagation, ReluVal, Neurify, VeriNet, CROWN) on both existing benchmarks and two sets of small and large randomly generated networks for four activation functions (ReLU, TanH, ELU, SiLU).

Keywords: Machine learning, Lyapunov methods, Stability of nonlinear systems
Abstract: We develop a versatile deep neural network architecture, called Lyapunov-Net, to approximate Lyapunov functions of dynamical systems in high dimensions. Lyapunov-Net guarantees positive definiteness, and thus can be easily trained to satisfy the negative orbital derivative condition, which only renders a single term in the empirical risk function in practice. This significantly simplifies parameter tuning and results in greatly improved convergence during network training and approximation quality. We also provide comprehensive theoretical justifications on the approximation accuracy and certification guarantees of Lyapunov-Nets. We demonstrate the efficiency of the proposed method on nonlinear dynamical systems in high dimensional state spaces, and show that the proposed approach significantly outperforms the state-of-the-art methods.

Keywords: Learning, Predictive control for nonlinear systems, Lyapunov methods
Abstract: We present a learning-based predictive control methodology using the differentiable programming framework with probabilistic Lyapunov-based stability guarantees. The neural Lyapunov differentiable predictive control (NLDPC) learns the policy by constructing a computational graph encompassing the system dynamics, state and input constraints, and the necessary Lyapunov certification constraints, and thereafter using the automatic differentiation to update the neural policy parameters. In conjunction, our approach jointly learns a Lyapunov function that certifies the regions of state-space with stable dynamics. We also provide a sampling-based statistical guarantee for the training of NLDPC from the distribution of initial conditions. Our offline training approach provides a computationally efficient and scalable alternative to classical explicit model predictive control solutions. We substantiate the advantages of the proposed approach with simulations to stabilize the double integrator model and on an example of controlling an aircraft model.

Keywords: Learning, Observers for nonlinear systems, Numerical algorithms
Abstract: For non-autonomous multivariable discrete-time nonlinear systems, we address the state estimation problem using a Kazantzis-Kravaris-Luenberger (KKL) observer. We aim to build a mapping that transforms a nonlinear dynamics into a stable linear system modulo an output injection and to design an asymptotic observer. However, this mapping is difficult to compute and its numerical approximation may be badly conditioned during the transient phase. We propose an algorithm based on ensemble learning techniques to improve the numerical approximation of the mapping and its extension in the transient phase. This ensures a good asymptotic convergence of the observer and avoids peaking phenomena. The algorithm demonstrates good performance in high-dimensional and multi-input-multi-output examples.

Keywords: Subspace methods, Predictive control for linear systems, Identification for control
Abstract: Direct data-driven control has attracted substantial interest since it enables optimization-based control without the need for a parametric model. This paper presents a new Instrumental Variable (IV) approach to Data-enabled Predictive Control (DeePC) that results in favorable noise mitigation properties, and demonstrates the direct equivalence between DeePC and Subspace Predictive Control (SPC). The methodology relies on the derivation of the characteristic equation in DeePC along the lines of subspace identification algorithms. A particular choice of IVs is presented that is uncorrelated with future noise, but at the same time highly correlated with the data matrix. A simulation study demonstrates the improved performance of the proposed algorithm in the presence of process and measurement noise.

Keywords: Predictive control for linear systems, Stochastic optimal control, Optimal control
Abstract: The implementation of data-driven predictive control schemes based on Willems' fundamental lemma often relies on a single-shooting approach, i.e., it uses one large Hankel matrix to cover the entire optimization horizon. However, the numerical solution is fostered by the use of multiple segmented horizons which require less data in smaller Hankel matrices. This paper extends the segmentation idea towards multiple shooting for data-driven optimal control of stochastic LTI systems. Using a stochastic variant of the fundamental lemma and polynomial chaos expansions, we propose a multiple-shooting formulation which combines trajectory segmentation and moment matching. We show that, for LTI systems subject to Gaussian noise of finite variance, our formulation is without loss of optimality while it allows for a significant reduction of the problem dimension in Gaussian and non-Gaussian settings. We draw upon a numerical example to compare the proposed framework to the usual single-shooting approach.

Keywords: Predictive control for nonlinear systems, Neural networks, Output regulation
Abstract: This paper deals with the design of nonlinear MPC controllers that provide offset-free setpoint tracking for models described by Neural Nonlinear AutoRegressive eXogenous (NNARX) networks. The NNARX model is identified from input-output data collected from the plant, and can be given a state-space representation with known measurable states made by past input and output variables, so that a state observer is not required. In the training phase, the Incremental Input-to-State Stability (δISS) property can be forced when consistent with the behavior of the plant. The δISS property is then leveraged to augment the model with an explicit integral action on the output tracking error, which allows to achieve offset-free tracking capabilities to the designed control scheme. The proposed control architecture is numerically tested on a water heating system and the achieved results are compared to those scored by another popular offset-free MPC method, showing that the proposed scheme attains remarkable performances even in presence of disturbances acting on the plant.

Keywords: Machine learning, Predictive control for nonlinear systems, Chemical process control
Abstract: We use Koopman theory for data-driven model reduction of nonlinear dynamical systems with controls. We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. We present a deep-learning approach to train the proposed models. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.

Keywords: Pattern recognition and classification, Statistical learning, Machine learning
Abstract: The scenario approach is a technique for data-driven decision making that has found application in a variety of fields including systems and control design. Although initially conceived in the context of worst-case optimization, the scenario approach has progressively evolved into a general methodology that allows one to keep control on the risk of solutions designed from data according to complex decision processes. In a recent contribution, the theory of compression schemes (a paradigm that plays a fundamental role in statistical learning theory) has been deeply revisited in the wake of the scenario approach, which has led to unprecedentedly sharp generalization and risk quantification results. In this paper, we build on these achievements to gain insight on a classification paradigm called Guaranteed Error Machine (GEM). First, by leveraging the theory of reproducing kernels Hilbert spaces, we introduce a new, more flexible, GEM algorithm, which allows for complex classification geometries. The proposed scheme is then shown to fit into the new compression theory, from which new sharp results for the probability of GEM misclassification are derived in a distribution-free context.

Keywords: Machine learning, Uncertain systems, Robust control
Abstract: Learning-based control offers the potential to tackle challenging control problems and hence receives a lot of attention, both from the control and machine learning communities. To provide rigorous control-theoretic guarantees like stability or other safety-related properties, many of these approaches require a quantification of the %remaining uncertainty. uncertainty that is inevitable when learning from real data. Unfortunately, many existing methods rely on unrealistic assumptions to derive such uncertainty bounds, preventing the real-world usage of learning-based control. We focus on the important regression setting and propose a new approach that only needs assumptions that can be derived from reasonable engineering knowledge. In order to achieve this goal, we combine the recently introduced Hard Shape Constrained Kernel Machines with geometric assumptions expressing prior model knowledge. The resulting algorithms can compute both nominal predictions with prescribed properties and rather tight uncertainty sets. Numerical experiments, including an illustrative control example, demonstrate the feasibility and performance of our approach.

Keywords: Identification
Abstract: Two identification algorithms for state space modeling from binary output measurements are presented in this paper. The first algorithm is based on the estimation of a high order FIR model used to recover the high resolution output signal. The second algorithm is based on an iterative scheme allowing the construction of state sequence and then the recovering of the high resolution output signal. Implementation conditions are discussed and illustrated. Algorithms are shown to be efficient by numerical simulations.

Keywords: Identification, Uncertain systems, Statistical learning
Abstract: This paper presents a method for jointly estimating the state, input, and parameters of linear systems in an online fashion. The method is specially designed for measurements that are corrupted with non-Gaussian noise or outliers, which are commonly found in engineering applications. In particular, it combines recursive, alternating, and iteratively-reweighted least squares into a single, one-step algorithm, which solves the estimation problem online and benefits from the robustness of least-deviation regression methods. The convergence of the iterative method is formally guaranteed. Numerical experiments show the good performance of the estimation algorithm in presence of outliers and in comparison to state-of-the-art methods.

Keywords: Identification, Time-varying systems, Lyapunov methods
Abstract: High-order tuners are algorithms that show promise in achieving greater efficiency than classic gradient-based algorithms in identifying the parameters of parametric models and/or in facilitating the progress of a control or optimization algorithm whose adaptive behavior relies on such models. For high-order tuners, robust stability properties, namely uniform global asymptotic (and exponential) stability, currently rely on a persistent excitation (PE) condition. In this work, we establish such stability properties with a novel analysis based on a Matrosov theorem and then show that the PE requirement can be relaxed via a concurrent learning technique driven by sampled data points that are sufficiently rich. We show numerically that concurrent learning may greatly improve efficiency. We incorporate reset methods that preserve the stability guarantees while providing additional improvements that may be relevant in applications that demand highly accurate parameter estimates at relatively low additional cost in computation.

Keywords: Closed-loop identification, Nonlinear systems identification, Estimation
Abstract: We investigate the usage of Moving Horizon Estimation (MHE) for state and parameter estimation for partially non-detectable systems with measurements corrupted by outliers. We propose a GGN-based arrival cost update formula and illustrate how it can be generalized to nonconvex loss functions that can be effectively used for outlier rejection. Moreover, we propose an adaptive regularization scheme for the arrival cost which introduces forgetting as well as additional pseudo-measurements to the arrival cost update. We illustrate the performance of the proposed algorithms on a longitudinal vehicle state and parameter estimation problem.

Keywords: Closed-loop identification, Identification for control
Abstract: This paper provides a method for obtaining a continuous-time model of a target system in closed-loop from input-output data alone, in the case where no knowledge of the controllers nor excitation signals is available and I/O data may suffer from unknown offsets. The proposed method is based on a fixed-pole observer model, which is a reasonable continuous-time version corresponding to the innovation model in discrete-time and allows the identification of unstable target systems. Furthermore, it is shown that the proposed method can be attributed to a convex optimization problem by fixing the observer poles. The method is within the framework of the stabilized output error method and shares usability advantages such as robustness to noise with complex dynamics and applicability to a wide class of models. The effectiveness of the method is illustrated through numerical examples.

Keywords: Observers for nonlinear systems, Variable-structure/sliding-mode control, Control applications
Abstract: In this letter, a novel scheme is proposed to identify in finite time the value of an unknown output parameter for a class of nonlinear dynamical systems. Inspired by the Super-Twisting Sliding Mode Algorithm (STA), the identification problem is solved in an innovative way, consisting of two steps. An STA observer is firstly designed to track in finite time the system output. Exploiting the observer behaviour during the sliding motion, a second scheme, still inspired by the STA, is used to estimate the unknown value of the output parameter, thus solving the identification problem in finite time. The numerical simulations based on district heating systems validate the effectiveness of the proposed identification method.

Keywords: Large-scale systems, Hybrid systems, Network analysis and control
Abstract: In this paper, we focus on the problem of compositional synthesis of controllers enforcing signal temporal logic (STL) tasks over a class of continuous-time nonlinear interconnected systems. By leveraging the idea of funnel-based control, we show that a fragment of STL specifications can be formulated as assume-guarantee contracts. A new concept of contract satisfaction is then defined to establish our compositionality result, which allows us to guarantee the satisfaction of a global contract by the interconnected system when all subsystems satisfy their local contracts. Based on this compositional framework, we then design closed-form continuous-time feedback controllers to enforce local contracts over subsystems in a decentralized manner. Finally, we demonstrate the effectiveness of our results on a numerical example.

Keywords: Formal Verification/Synthesis
Abstract: Contract theory is a powerful tool to reason on systems that are interacting with an external environment, possibly made of other systems. Formally, a contract is usually given by assumptions and guarantees, which specify the expected behavior of the system (the guarantees) in a certain context (the assumptions). In this work, we present a verification framework for discrete-time dynamical systems with assume-guarantee contracts. We first introduce a class of assume-guarantee contracts with their satisfaction semantics parameterized by a time-horizon over which assumptions are evaluated. We then show that the problem of verifying whether such contracts are satisfied is equivalent to show the existence of a positive invariant set for an auxiliary system. This allows us to leverage the extensive literature on invariant set computation. A simple illustrative example is provided to show the effectiveness of our approach.

Keywords: Autonomous robots, Robotics, Autonomous systems
Abstract: When designing autonomous systems, we need to consider multiple trade-offs at various abstraction levels, and the choices of single (hardware and software) components need to be studied jointly. In this work we consider the problem of designing the control algorithm as well as the platform on which it is executed. In particular, we focus on vehicle control systems, and formalize state-of-the-art control schemes as monotone feasibility relations. We then show how, leveraging a monotone theory of co-design, we can study the embedding of control synthesis problems into the task-driven co-design problem of a robotic platform. The properties of the proposed approach are illustrated by considering urban driving scenarios. We show how, given a particular task, we can efficiently compute Pareto optimal design solutions.

Keywords: Formal Verification/Synthesis, Linear systems, Large-scale systems
Abstract: We present assume-guarantee contracts for continuous-time linear dynamical systems with inputs and outputs. These contracts are used to express specifications on the dynamic behaviour of a system. Contrary to existing approaches, we use simulation to compare the dynamic behaviour of two systems. This has the advantage of being supported by efficient numerical algorithms for verification as well as being related to the rich literature on (bi)simulation based techniques for verification and control, such as those based on (discrete) abstractions. Using simulation, we define contract implementation and a notion of contract refinement. We also define a notion of series composition for contracts, which allows us to reason about the series interconnection of systems on the basis of the contracts on its components. Together, the notions of refinement and composition allow contracts to be used for modular design and analysis of interconnected systems.

Keywords: Power systems, Decentralized control, Network analysis and control
Abstract: In this paper, we propose a design procedure of data-adaptive power system stabilizers (PSSs) in the framework of retrofit control. The proposed procedure is modular in the sense that both design and implementation processes of PSSs can be performed using only a local subsystem model and local measurement. In particular, we consider online identification of a dynamical feedback effect between the states of a generator of interest and the main grid to make the PSS adaptive to the variation of grid characteristics depending on power flow distributions. The main theoretical contribution is to show that the same retrofit controller as that developed for linear systems in the literature works properly even for nonlinear power systems where an operating point of interest varies depending on power flow distributions. In addition, we propose an online identification algorithm of the grid characteristics that makes use of the physical structure of power systems. We demonstrate the efficacy of the proposed data-adaptive PSS by a numerical simulation on the IEEE 9-bus test power system.

Keywords: Computer-aided control design, Automotive control
Abstract: We present a compositional control synthesis method based on assume-guarantee contracts with application to correct-by-construction design of vehicular mission plans. In our approach, a mission-level specification expressed in a fragment of signal temporal logic (STL) is decomposed into predicates defined on non-overlapping time intervals. The STL predicates are then mapped to an aggregation of contracts defined via piecewise continuously differentiable time-varying control barrier functions. The barrier functions are used to constrain the lower-level control synthesis problem, which is solved via quadratic programming. Our approach can avoid the conservatism of previous methods for task-driven control based on under-approximations. We illustrate its effectiveness on a case study motivated by vehicular mission planning under safety constraints as well as constraints imposed by traffic regulations, vehicle-to-vehicle, and vehicle-to-infrastructure communication.

Keywords: Resilient Control Systems, Agents-based systems, Autonomous robots
Abstract: For efficient and robust task accomplishment in multi-agent systems, an agent must be able to distinguish cooperative agents from non-cooperative (i.e., uncooperative and adversarial) agents. In this paper, we first develop a trust metric based on which each agent forms its own belief of how cooperative the other agents are, i.e., of how much the other agents contribute to maintaining safety. With safety encoded as Control Barrier Functions (CBFs), the trust metric is in turn used to adjust the rate at which the CBFs allow the system trajectories to approach the boundary of the safe set. This is achieved via a novel form of a CBF, called the Rate-Tunable CBF, which yields less conservative performance compared to an identity-agnostic implementation, where cooperative and non-cooperative agents are treated similarly. The proposed adaptation and control method is evaluated via simulations on heterogeneous multi-agent systems including non-cooperative agents.

Keywords: Resilient Control Systems, Optimal control, Power systems
Abstract: A key aspect of a resilient dynamical system is its ability to recover from large disruptions. In this paper, we define and characterize recovery energy as a quantitative measure of the minimum effort required by a control dynamical system to return to a safe operating region after large disruptions. The problem of steering the system state to a pre-specified safe region with minimum recovery energy is posed as an optimization problem which turns out to be a nonconvex due to the nonlinear nature of the system dynamics. To this end, we use the Koopman operator theoretic framework to obtain a convex reformulation in a lifted observable space. Finally, we demonstrate the approach on an IEEE 123-bus test feeder and quantify its ability to recover from disruptions.

Keywords: Resilient Control Systems, Robotics, Networked control systems
Abstract: We consider a class of problems where robots gather observations of each other to assess the legitimacy of their peers. Previous works propose accurate detection of malicious robots when robots are able to extract observations of each other for a long enough time. However, they often consider static networks where the set of neighbors a robot observes remains the same. Mobile robots experience a dynamic set of neighbors as they move, making the acquisition of adequate observations more difficult. We design a stochastic policy that enables the robots to periodically gather observations of every other robot, while simultaneously satisfying a desired robot distribution over an environment modeled by sites. We show that with this policy, any pre-existing or new malicious robot in the network will be detected in a finite amount of time, which we minimize and also characterize. We derive bounds on the time needed to obtain the desired number of observations for a given topological map and validate these bounds in simulation. We also show and verify in a hardware experiment that the team is able to successfully detect malicious robots, and thus estimate the true distribution of cooperative robots per site, in order to converge to the desired robot distribution over sites.

Keywords: Optimization, Communication networks, Optimization algorithms
Abstract: This paper considers a distributed optimization problem in a multi-agent system where a fraction of the agents is compromised and acts in a malicious manner. Specifically, the compromised agents steer the network of agents away from the optimal solution by sending relaying false information to their neighbors and consume significant bandwidth in the available communication channels. We propose a distributed gradient-based optimization algorithm in which the non-malicious agents exchange quantized information with one another. We prove convergence of the solution to a neighborhood of the optimal solution, explore the convergence attributes of the algorithm and characterize the solutions obtained under the communication-constrained environment and presence of malicious agents. Numerical simulations to illustrate the results are also presented.

Keywords: Optimization algorithms, Optimization, Network analysis and control
Abstract: We study the problem of inference attack in distributed optimization, with adversarial agents aiming to obtain the sensitive information of some critical agent in a network. Different from existing privacy-preserving and resilient distributed optimization algorithms, we propose inference algorithms from the perspective of launching well-designed attacks to help infer the sensitive local information. The key idea is that by utilizing the critical agent's neighborhood information and the predefined update protocol, adversarial agents can not only interpolate the gradient of its local objective function, but also manipulate it to converge to its own local minimizer. The proposed algorithms can thus obtain the approximation of the gradient or the minimizer of the local objective of the critical agent. We characterize the performance through interpolation errors, as well as distances to the optimal value and optimal point of the local objective. Numerical simulations are presented to verify the effectiveness of these algorithms.

Keywords: Vision-based control, Resilient Control Systems, Cyber-Physical Security
Abstract: Autonomous Cyber-Physical Systems (CPS) fuse proprioceptive sensors such as GPS and exteroceptive sensors including Light Detection and Ranging (LiDAR) and cameras for state estimation and environmental observation. It has been shown that both types of sensors can be compromised by malicious attacks, leading to unacceptable safety violations. We study the problem of safety-critical control of a LiDAR-based system under sensor faults and attacks. We propose a framework consisting of fault tolerant estimation and fault tolerant control. The former reconstructs a LiDAR scan with state estimations, and excludes the possible faulty estimations that are not aligned with LiDAR measurements. We also verify the correctness of LiDAR scans by comparing them with the reconstructed ones and removing the possibly compromised sector in the scan. Fault tolerant control computes a control signal with the remaining estimations at each time step. We prove that the synthesized control input guarantees system safety using control barrier certificates. We validate our proposed framework using a UAV delivery system in an urban environment. We show that our proposed approach guarantees safety for the UAV whereas a baseline fails.

Keywords: Cooperative control, Emerging control applications, Traffic control
Abstract: In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection by integrating a safety layer using control barrier functions. First, in our motion planning module, each CAV computes the optimal control trajectory using simple vehicle dynamics. The trajectory does not make any of the state, control, and safety constraints active. A vehicle-level tracking controller employs a combined feedforward-feedback control law to track the resulting optimal trajectory from the motion planning module. Then, a barrier-certificate module, acting as a middle layer between the vehicle-level tracking controller and physical vehicle, receives the control law from the vehicle-level tracking controller and using realistic vehicle dynamics ensures that none of the state, control, and safety constraints becomes active. The latter is achieved through a quadratic program, which can be solved efficiently in real time. We demonstrate the effectiveness of our extended framework through a numerical simulation.

Keywords: Cooperative control
Abstract: This work presents a verifiable condition for the selection of a sufficient number of pursuers to capture a faster evader. The condition is based on the tracking performance of a multi-agent control scheme. Trajectory tracking results are provided for both the effects of the multi-agent control topology and its execution by the pursuers in the context of input saturation. To that end, nonlinear contraction theory is leveraged because it provides a unifying framework for the analysis of systems subject to bounded disturbances. Simulation experiments are performed to validate the proposed condition for sufficient pursuers selection.

Keywords: Cooperative control, Fault tolerant systems, Resilient Control Systems
Abstract: Accurate state measurement is important to ensure the reliable operation of distributed multi-agent systems (MAS). Existing fault-tolerant control methods generally assume the sensor faults to be bounded and uncorrelated. In this paper, we study the ramifications of allowing the sensor attack injections to be unbounded and correlated. Concerted malicious behaviors may bypass existing attack-detection methods and threaten the synchronization performance and even system stability of the distributed networked MAS. In particular, we consider networked heterogeneous MAS in the face of correlated sensor attacks and unforeseen actuator faults. We propose a distributed attack-resilient synchronization control framework to guarantee the uniform ultimate boundedness of the overall dynamical system. A simulation example illustrates the efficacy of the proposed attack-resilient method.

Keywords: Cooperative control, Optimization, Distributed control
Abstract: In this study, we discuss a primal-dual distributed algorithm for online convex optimization with a time-varying coupled constraint on unbalanced directed graphs. A group of agents exchanges the estimation variable for the dual optimizer and the scaling variable, which are used for compensating the unbalanced information flow. Then, each agent updates the primal and dual variables using the projected subgradient methods. We confirm that the regret of the cost function and the cumulative error of the constraint violation achieve sublinearity. A numerical example of a distributed economic dispatch problem demonstrates that the estimation of each agent approaches the optimal strategy under the coupled inequality constraint.

Keywords: Cooperative control, Lyapunov methods, Stability of nonlinear systems
Abstract: We address the agreement-based coordination of first-order multi-agent systems interconnected over arbitrary connected undirected graphs and under transient and steady-state constraints. The system is in a leader-follower configuration where only a part of the agents, the leaders, are directly controlled via an external control input, in addition to the agreement protocol. We propose a control law for the leaders, based on the gradient of a potential function, that achieves consensus and guarantees that the trajectories of the inter-agent distances of the entire system remain bounded by a performance function. Relying on the edge-agreement framework and Lyapunov’s first method, we establish strong stability results in the sense of asymptotic stability of the consensus manifold and, in the leaderless case, nonuniform-in-time input-to-state stability with respect to additive disturbances. A numerical simulation illustrates the effectiveness of the proposed approach.

Keywords: Cooperative control, Distributed control
Abstract: This paper presents a distributed protocol for a heterogeneous MAS to track the global maximum of multiple time-varying references. We define this as the global max-tracking problem. Compared to the existing max-consensus problems, the global max-tracking problem has more challenges, as the references are time-varying, and no agent has any knowledge about their global maximum a priori. Under the proposed protocol, each agent tracks the global maximum of continuously time-varying references irrespective of their time-derivatives. Thus, the current protocol works for a wide range of time-varying references. A discussion with the numerical simulations exemplifies the efficacy of the proposed global max-tracking protocol.

Keywords: Stochastic optimal control, Filtering, Learning
Abstract: We study an approximation method for partially observed Markov decision processes (POMDPs) with continuous spaces. Belief MDP reduction, which has been the standard approach to study POMDPs, which, due to its uncountable state space and strict regularity properties however, requires rigorous approximation methods for practical applications. In this work, we focus on an approximation procedure via discretizing the observation space and constructing a fully observed finite MDP model using a finite length history of the discrete observations and control actions. We show that the resulting policy is nearly optimal under some regularity assumptions on the channel, and under certain controlled filter stability requirements for the hidden state process. We also provide a Q learning algorithm that uses a finite memory of discretized information variables, and prove its convergence to the optimality equation of the finite fully observed MDP constructed using the approximation method.

Keywords: Stochastic optimal control, Markov processes, Learning
Abstract: Traditional reinforcement learning (RL) aims to maximize the expected total reward, while the risk of uncertain outcomes needs to be controlled to ensure reliable performance in a risk-averse setting. In this paper, we consider the problem of maximizing dynamic risk of a sequence of rewards in infinite-horizon Markov Decision Processes (MDPs). We adapt the Expected Conditional Risk Measures (ECRMs) to the infinite-horizon risk-averse MDP and prove its time consistency. Using a convex combination of expectation and conditional value-at-risk (CVaR) as a special one-step conditional risk measure, we reformulate the risk-averse MDP as a risk-neutral counterpart with augmented action space and manipulation on the immediate rewards. We further prove that the related Bellman operator is a contraction mapping, which guarantees the convergence of any value-based RL algorithms. Accordingly, we develop a risk-averse deep Q-learning framework, and our numerical studies based on two simple MDPs show that the risk-averse setting can reduce the variance and enhance robustness of the results.

Keywords: Stochastic systems, Machine learning, Learning
Abstract: We propose Deterministic Sequencing of Exploration and Exploitation (DSEE) algorithm with interleaving exploration and exploitation epochs for model-based RL problems that aim to simultaneously learn the system model, i.e., a Markov decision process (MDP), and the associated optimal policy. During exploration, DSEE explores the environment and updates the estimates for expected reward and transition probabilities. During exploitation, the latest estimates of the expected reward and transition probabilities are used to obtain a robust policy with high probability. We design the lengths of the exploration and exploitation epochs such that the cumulative regret grows as a sub-linear function of time.

Keywords: Stochastic systems, Randomized algorithms, Machine learning
Abstract: In this paper, we give a simple and direct proof of the convergence of the stochastic approximation algorithm under suitable conditions. The main result here can be com- pared to that in Borkar and Meyn (2000), which is based on the ODE method, that is, showing that the sample paths of the algorithm converge towards the deterministic trajectories of an associated ODE. In contrast, the present proof is based on martingale theory, first proposed in Gladyshev (1965). Consequently, there are fewer assumptions here compared to previous papers. An important part of the proof is a new converse Lyapunov theorem for global exponential stability. Aside from its application to stochastic approximation theory, this new converse Lyapunov theorem would be useful for researchers in stability theory.

Keywords: Stochastic optimal control, Statistical learning, Uncertain systems
Abstract: This paper proposes a new structured method for a moving agent to predict the paths of dynamically moving obstacles and avoid them using a risk-aware model predictive control (MPC) scheme. Given noisy measurements of the a priori unknown obstacle trajectory, a bootstrapping technique predicts a set of obstacle trajectories. The bootstrapped predictions are incorporated in the MPC optimization using a risk-aware methodology so as to provide probabilistic guarantees on obstacle avoidance. We validate our methods using simulations of a multi-rotor drone that avoids various moving obstacles.

Keywords: Statistical learning, Machine learning, Estimation
Abstract: The paper introduces a method to construct confidence bands for bounded, band-limited functions based on a finite sample of input-output pairs. The approach is distribution-free w.r.t. the observation noises and only the knowledge of the input distribution is assumed. It is nonparametric, that is, it does not require a parametric model of the regression function and the regions have non-asymptotic guarantees. The algorithm is based on the theory of Paley-Wiener reproducing kernel Hilbert spaces. The paper first studies the fully observable variant, when there are no noises on the observations and only the inputs are random; then it generalizes the ideas to the noisy case using gradient-perturbation methods. Finally, numerical experiments demonstrating both cases are presented.

Keywords: Robust control, Sampled-data control, Stochastic optimal control
Abstract: We tackle safe trajectory planning under Gaussian mixture model (GMM) uncertainty. Specifically, we use a GMM to model the multimodal behaviors of obstacles' uncertain states. Then, we develop a mixed-integer conic approximation to the chance-constrained trajectory planning problem with deterministic linear systems and polyhedral obstacles. When the GMM moments are estimated via finite samples, we develop a tight concentration bound to ensure the chance constraint with a desired confidence. Moreover, to limit the amount of constraint violation, we develop a Conditional Value-at-Risk (CVaR) approach corresponding to the chance constraints and derive a tractable approximation for known and estimated GMM moments. We verify our methods with state-of-the-art trajectory prediction algorithms and autonomous driving datasets.

Keywords: Robust control, Nonlinear systems, Constrained control
Abstract: This paper concentrates on the fragility margins of discrete-time Piecewise Affine (PWA) closed-loop dynamics. Starting from the case where the nominal trajectories are controlled by a PWA controller using a positioning mechanism within a binary search tree (BST), we are interested in preserving the properties of the nominal dynamics, particularly the positive invariance in the presence of perturbations in the control law representation. As the main result, we define (and effectively construct) the fragility margin for a hyperplane defining the partition of the nominal PWA controller. This hyperplane will be identified through a node in a BST, and the proposed margin characterizes the degrees of freedom in the perturbation of its coefficients as it might result from a quantization operation. From the mathematical standpoint, the fragility margin is a set in the coefficients’ space, and the properties of this set are formally described.

Keywords: Robust control, Optimal control, Linear systems
Abstract: Recent developments in cyber-physical systems and event-triggered control have led to an increased interest in the impact of sparse disturbances on dynamical processes. We study Linear Quadratic Regulator (LQR) control under sparse disturbances by analyzing three distinct policies: the blind online policy, the disturbance-aware policy, and the optimal offline policy. We derive the two-dimensional recurrence structure of the optimal disturbance-aware policy, under the assumption that the controller has information about future disturbance values with only a probabilistic model of their locations in time. Under mild conditions, we show that the disturbance-aware policy converges to the blind online policy if the number of disturbances grows sublinearly in the time horizon. Finally, we provide a finite-horizon regret bound between the blind online policy and optimal offline policy, which is proven to be quadratic in the number of disturbances and in their magnitude. This provides a useful characterization of the suboptimality of a standard LQR controller when confronted with unexpected sparse perturbations.

Keywords: Robust control, Predictive control for nonlinear systems, Predictive control for linear systems
Abstract: This paper exploits the efficient computation of reachable sets for monotone systems to formulate model predictive control approaches with guaranteed constraint satisfaction and recursive feasibility in the presence of uncertainty. To include all possible realizations of the uncertainties, the reachable sets are approximated as hyperrectangles in the problem formulation. The presented approach is extended to include recourse, i.e., the knowledge about the presence of further measurement information in the prediction horizon. By dividing the reachable sets, multiple regions are obtained, for which the future inputs can be chosen separately. The applicability of the proposed approaches are shown by the means of a temperature control example.

Keywords: Robust control, Uncertain systems, Large-scale systems
Abstract: We present a new, scalable alternative to the structured singular value, which we call nu, provide a convex upper bound, study their properties and compare them to ell_1 robust control. The analysis relies on a novel result on the relationship between robust control of dynamical systems and non-negative constant matrices.

Keywords: Robust control, Uncertain systems, Large-scale systems
Abstract: We develop convex parameterizations for L1-robust controllers in the System Level Synthesis (SLS) framework, allowing us to find robust decentralized controllers that preserve design constraints on sparsity, locality, and delay. The parameterizations have interpretable relationships with the set of nominally stabilizing SLS controllers. We show that the state feedback robust performance SLS problem can be posed exactly and end-to-end as a linear program applicable to large-scale distributed control. We generalize these results to unstructured uncertainties in the output feedback setting, providing new decentralized robust performance guarantees in this setting.

Keywords: Linear systems, Robust control
Abstract: This paper considers the robustness of event-triggered control of general linear systems against additive frequency-domain uncertainties. It is revealed that in the event-triggered mechanisms, the sampling errors are images of affine operators acting on the sampled outputs. Though not belonging to mathcal {RH}_{infty}, these operators are finite-gain mathcal L_2 stable with operator-norm depending on the triggering conditions and the norm bound of the uncertainties. Moreover, the robust event-triggered controller design problem can then be transformed into a standard H_{infty} synthesis problem of a linear system having the same order as the controlled plant. Algorithms are also provided to construct the event-triggered robust dynamic output-feedback controllers.

Keywords: Networked control systems, Discrete event systems, Learning
Abstract: We present an approach to reduce the communication of information needed on a Distributed Q-Learning system inspired by Event Triggered Control (ETC) techniques. We consider a baseline scenario of a Distributed Q-Learning problem on a Markov Decision Process (MDP). Following an event-based approach, N agents sharing a value function explore the MDP and compute a trajectory-dependent triggering signal which they use distributedly to decide when to communicate information to a central learner in charge of computing updates on the action-value function. These decision functions form an Event Based distributed Q learning system (EBd-Q), and we derive convergence guarantees resulting from the reduction of communication. We then apply the proposed algorithm to a cooperative path planning problem, and show how the agents are able to learn optimal trajectories communicating a fraction of the information. Additionally, we discuss what effects (desired and undesired) these event-based approaches have on the learning processes studied, and how they can be applied to more complex multi-agent systems.

Keywords: Resilient Control Systems, Agents-based systems, Cyber-Physical Security
Abstract: In this paper, we consider the problem of multiagent consensus in the presence of mobile adversaries. Faulty agents try to prevent the coordination among the regular agents and moreover are mobile in the sense that they can change their identities over time. Our approach towards resilient consensus is to extend the so-called mean subsequence reduced (MSR) algorithms to reduce the necessary communication based on two measures: The information exchanged by the agents takes the form of ternary data in each message and furthermore selftriggered method is used to keep the transmission frequency limited. Certain features are introduced to address issues specific to the mobile nature of the adversarial agents. We verify the effectiveness of the proposed algorithm by means of a numerical example.

Keywords: Machine learning, Sensor networks, Statistical learning
Abstract: We present a practical asynchronous data fusion model for networked agents to perform distributed Bayesian learning without sharing raw data. Our algorithm employs unadjusted Langevin dynamics with a gossip-based protocol for sampling, coupled with an event-triggered mechanism to further reduce communication between gossiping agents. These mechanisms drastically reduce communication overhead and help avoid bottlenecks commonly experienced with distributed algorithms. In addition, the algorithm is expected to increase resilience to occasional link failure. We establish mathematical guarantees for our algorithm and demonstrate its effectiveness via a numerical experiment.

Keywords: Optimal control, Markov processes, Discrete event systems
Abstract: Given a partially observable Markov decision process (POMDP) with finite state, input and measurement spaces, and costly measurements and control, we consider the problem of when to sample and actuate. Both sampling and actuation are modeled as control actions in a framework encompassing estimation and intervention problems. The process evolves freely between two consecutive control action times. Control actions are assumed to reset the conditional distribution of the state given the measurements to one of a finite number of distributions. We tackle the problem of deciding when control actions should occur in order to minimize an average cost that penalizes states and the rate of control actions. The problem is first shown to boil down to a stopping time problem. While the latter can be solved optimally, the complexity of the optimal policy is intractable. Thus, we propose two approximate methods. The first is inspired by relaxed dynamic programming, and it is within an additive cost factor of the optimal policy. The second is inspired by consistent event-triggered control and ensures that the cost is smaller than that of periodic control for the same control rate. We conclude that the latter policy can deal with large dimensional problems, as demonstrated in the context of precision agriculture.

Keywords: Machine learning, Uncertain systems, Adaptive control
Abstract: In this paper, we present a trajectory tracking control law for a class of partially unknown nonlinear systems that combines backstepping and event-triggered online learning. We employ Gaussian processes to learn the unknown system model using measurement data collected online, while the proposed control law is active. Our approach uses an efficient event-triggered online learning scheme that exclusively collects informative data to update the estimated model used for control. The resulting control law guarantees that the tracking error is globally uniformly ultimately bounded. The inter-event time is shown to be lower-bounded by a positive constant. Moreover, we also discuss how to obtain a trade-off between the cardinality of the collected training data and the size of the ultimate tracking error bound. In a simulation example, our approach is shown to outperform a state-of-the-art offline learning-based approach both in terms of tracking performance and data efficiency.

Keywords: Predictive control for nonlinear systems, Automotive control, Switched systems
Abstract: Gasoline Controlled Auto-Ignition (GCAI) is a novel combustion technology that promises higher efficiency and lower pollutant emissions than conventional combustion engines. These advantages come at the cost of a high sensitivity to disturbances and instabilities in combustion phasing. Closed-loop control methods such as linear model predictive control are able to stabilize the process, although the operating range remains limited. Nonlinear model predictive control and moving horizon estimation are able to optimize the fully nonlinear system model and promise higher prediction and estimation accuracy. Nonlinear models based on differential equations offer high physical detail and flexibility, however they pose a high computational burden on the control and estimation algorithms. This paper proposes an efficient, real-time capable MPC and MHE scheme based on the Real Time Iteration (RTI) for a continuous-time GCAI model. We achieve a feedback delay of below 1 ms and RTI preparation time of below 18 ms in numerical simulations on the embedded hardware of an existing GCAI engine test setup. Thereby the proposed scheme satisfies the computational constraints imposed by the engine speed by a safety factor of 4.

Keywords: Predictive control for nonlinear systems, Maritime control, Autonomous systems
Abstract: Underactuated agents can achieve energy-efficient navigation in the air and oceans by hitchhiking complex flows. However, in real flows the forecast error can be larger than the actuation of the agent which poses a challenge for reliable navigation. In this paper, we propose a closed-loop control schema in the spirit of Model Predictive Control which allows time-optimal replanning at every step with one computation per forecast. We us the recent Multi-Time Hamilton-Jacobi Reachability formulation to obtain a value function which is used for closed-loop control. We evaluate the reliability of our method empirically over a large set of multi-day start-target missions in the ocean currents of the Gulf of Mexico with realistic forecast errors. Our method outperforms the baselines significantly and achieves high reliability, measured as the success rate of navigating from start to target within a maximum allowed time, at low computational cost.

Keywords: Predictive control for nonlinear systems, Flexible structures, Power generation
Abstract: The current trend in the evolution of wind turbines is to increase their rotor size in order to capture more power. This leads to taller, slender and more flexible towers, which thus experience higher dynamical loads due to the turbine rotation and environmental factors. It is hence compelling to deploy advanced control methods that can dynamically counteract such loads, especially at tower positions that are more prone to develop cracks or corrosion damages. Still, to the best of the authors’ knowledge, little to no attention has been paid in the literature to load mitigation at multiple tower locations. Furthermore, there is a need for control schemes that can balance load reduction with optimization of power production. In this paper, we develop an Economic Model Predictive Control (eMPC) framework to address such needs. First, we develop a linear modal model to account for the tower flexural dynamics. Then we incorporate it into an eMPC framework, where the dynamics of the turbine rotation are expressed in energy terms. This allows us to obtain a convex formulation, that is computationally attractive. Our control law is designed to avoid the “turn-pike” behavior and guarantee recursive feasibility. We demonstrate the performance of the proposed controller on a 5MW reference WT model: the results illustrate that the proposed controller is able to reduce the tower loads at multiple locations, without significant effects to the generated power.

Keywords: Predictive control for nonlinear systems, Optimal control, Aerospace
Abstract: This paper considers model predictive control of a tandem-rotor helicopter. The error is formulated using the matrix Lie group SE2(3). A reference trajectory to a target is calculated using a quartic guidance law, leveraging the differentially flat properties of the system, and refined using a finite-horizon linear quadratic regulator. The nonlinear system is linearized about the reference trajectory enabling the formulation of a quadratic program with control input, attitude keep-in zone, and attitude error constraints. A non-uniformly spaced prediction horizon is leveraged to capture the multi-timescale dynamics while keeping the problem size tractable. Monte-Carlo simulations demonstrate robustness of the proposed control structure to initial conditions, model uncertainty, and environmental disturbances.

Keywords: Predictive control for nonlinear systems, Robotics, Algebraic/geometric methods
Abstract: This paper proposes a Nonlinear Model-Predictive Control (NMPC) method capable of finding and converging to energy-efficient regular oscillations, which require no control action to be sustained. The approach builds up on the recently developed Eigenmanifold theory, which defines the sets of line-shaped oscillations of a robot as an invariant two-dimensional submanifold of its state space. By defining the control problem as a nonlinear program (NLP), the controller is able to deal with constraints in the state and control variables and be energy-efficient not only in its final trajectory but also during the convergence phase. An initial implementation of this approach is proposed, analyzed, and tested in simulation.

Keywords: Predictive control for nonlinear systems, Computational methods, Fluid flow systems
Abstract: For the analysis of dynamical systems, it is fundamental to determine all states that can be reached at any given time. In this work, we obtain and apply new governing equations for reachability analysis over multiple start and terminal times all at once, and for systems operating in time-varying environments with dynamic obstacles and any other relevant dynamic fields. The theory and schemes are developed for both backward and forward reachable tubes with time-varying target and start sets. The resulting value functions elegantly capture not only the reachable tubes but also time-to-reach and time-to-leave maps as well as start time vs. duration plots and other useful secondary quantities for optimal control. We discuss the numerical schemes and computational efficiency. We first verify our results in an environment with a moving target and obstacle where reachability tubes can be analytically computed. We then consider the Dubin’s car problem extended with a moving target and obstacle. Finally, we showcase our multi-time reachability in a non-hydrostatic bottom gravity current system. Results highlight the novel capabilities of exact multi-time reachability in dynamic environments.

Keywords: Emerging control applications, Modeling, Biomolecular systems
Abstract: Understanding the process of protein unfolding plays a crucial role in various applications such as design of folding-based protein engines. Using the well-established kinetostatic compliance (KCM)-based method for modeling of protein conformation dynamics and a recent nonlinear control theoretic approach to KCM-based protein folding, this paper formulates protein unfolding as a destabilizing control analysis/synthesis problem. In light of this formulation, it is shown that the Chetaev instability framework can be used to investigate the KCM-based unfolding dynamics. In particular, a Chetaev function for analysis of unfolding dynamics under the effect of optical tweezers and a class of control Chetaev functions for synthesizing control inputs that elongate protein strands from their folded conformations are presented. Based on the presented control Chetaev function, an unfolding input is derived from the Artstein-Sontag universal formula and the results are compared against optical tweezer-based unfolding.

Keywords: Mechatronics, Observers for nonlinear systems, Control applications
Abstract: In this paper, we present nonlinear proportional integrator (N-PI) disturbance observers (DOBs) for enhancing the motion tracking performance of a surface-mounted permanent magnet synchronous motor in rapidly varying speed regions. By presenting an N-PI-DOB for load torque estimation with a torque modulation technique, we show that the electromechanical system, including the motion controller, disturbance observers, and inverter controller, can be presented in a three-cascaded form, in which the stability analysis of the closed-loop system can be easily performed. After analyzing the disturbances of the current tracking error dynamics, we designed another N-PI-DOB and a Lyapunov-based nonlinear controller for the current loop to enhance the motion tracking performance. With these N-PI-DOBs and the motion controller, we analyzed the stability of the motion tracking error dynamics and estimation error dynamics. We experimentally performed a comparative study with and without N-PI-DOB to validate the effectiveness of the proposed method under the conditions of unknown load torque and rapid speed variation.

Keywords: Neural networks, Machine learning, Nonlinear output feedback
Abstract: In this work, we show the potential of the universal approximation property of neural networks in the design of interconnection and damping assignment passivity-based controllers (IDA-PBC) for stabilizing nonlinear underactuated mechanical systems of degree one. Towards this end, we refor- mulate the IDA-PBC design methodology as a neural supervised learning problem that approximates the solution of the partial differential matching equations, which fulfills the equilibrium assignment and stability conditions. The output of the neural learning process has clear physical and control-theoretic inter- pretations in terms of energy, passivity and Lyapunov stability. The proposed approach is numerically evaluated in two well- known underactuated systems: the inverted pendulum on a cart and inertial wheel pendulum, whose analytic IDA-PBC solutions are non-trivial to obtain.

Keywords: Mechatronics, Robotics, Robust control
Abstract: When designing controllers for piezoelectric systems with hysteresis, usually simplified models are used. This can lead to inaccuracies in the closed-loop system response. In this paper, we pose the problem of tracking stabilization of piezoelectric systems using the generalized Bouc-Wen model, a highly non-linear system rarely used to design controllers. Besides, we consider only partial knowledge of one hysteresis system parameter and external disturbances in all the system states. We propose an interconnected control composed of three parts for the solution: an observer, a virtual hysteresis control, and an actuator control. It is demonstrated that the closed-loop system converges in finite time. Simulation experiments were carried out, demonstrating the effectiveness of our approach despite exogenous and unknown disturbances.

Keywords: Mechatronics, Grey-box modeling, Machine learning
Abstract: Unknown nonlinear dynamics often limit the tracking performance of feedforward control. The aim of this paper is to develop a feedforward control framework that can compensate these unknown nonlinear dynamics using universal function approximators. The feedforward controller is parametrized as a parallel combination of a physics-based model and a neural network, where both share the same linear autoregressive (AR) dynamics. This parametrization allows for efficient output-error optimization through Sanathanan-Koerner (SK) iterations. Within each SK-iteration, the output of the neural network is penalized in the subspace of the physics-based model through orthogonal projection-based regularization, such that the neural network captures only the unmodelled dynamics, resulting in interpretable models.

Keywords: Mechatronics
Abstract: This paper investigates the output feedback tracking control of the motion system driven by an electromagnetic actuator with uncertain nonlinearities and unknown gap variation, considering only the measured output position. The output feedback control is based on two cascade high-gain observers combined with a full state feedback control that is based on the backstepping approach. In this work, the observer of two cascaded high-gain observers with different speeds is considered; the faster one estimates the output position and velocity of the motion system and feeds a virtual nonlinear output to estimate the magnetic flux and nonlinear hysteresis of the reluctance actuator. We show that the equilibrium point of the full state feedback control system under full knowledge of the system information is globally asymptotically stable. The simulation results show that the output feedback control achieves the tracking control objective and recovers the performance of the state feedback control. Also, the simulation results show the robustness of the output feedback control for unknown variations in the air gap and unknown external disturbances.

Keywords: Game theory, Hierarchical control, Smart grid
Abstract: In modern buildings renewable energy generators and storage devices are spreading, and consequently the role of the users in the power grid is shifting from passive to active. We design a demand response scheme that exploits the prosumers' flexibility to provide ancillary services to the main grid. We propose a hierarchical scheme to coordinate the interactions between the distribution system operator and a community of smart prosumers. The framework inherits characteristics from price-based and incentive-based schemes and it retains the advantages of both. We cast the problem as a Stackelberg game with the prosumers as followers and the distribution system operator as leader. We solve the resulting bilevel optimization program via a KKT reformulation, proving the existence and the convergence to a local Stackelberg equilibrium.

Keywords: Game theory, Distributed control, Optimization algorithms
Abstract: We consider a set of autonomous agents with LTI dynamics subject to constant external disturbance. Each agent seeks to minimize a coupled cost function in steady-state subject to equality constraints. We propose a novel control methodology to solve the problem that captures much the past work done on Nash equilibrium seeking for dynamic agents. We show that using our methodology, the problem reduces to one of designing a set of decentralized stabilizing controllers. Example controller designs are provided for two cases.

Keywords: Game theory, Cooperative control, Decentralized control
Abstract: Learning in games has been widely used to solve many cooperative multi-agent problems such as coverage control, consensus, self-reconfiguration or vehicle-target assignment. One standard approach in this domain is to formulate the problem as a potential game and to use an algorithm such as log-linear learning to achieve the stochastic stability of globally optimal configurations. Standard versions of such learning algorithms are asynchronous, i.e., only one agent updates its action at each round of the learning process. To enable faster learning, we propose a synchronization strategy based on decentralized random prioritization of agents, which allows multiple agents to change their actions simultaneously when they do not affect each other's utility or feasible actions. We show that the proposed approach can be integrated into any standard asynchronous learning algorithm to improve the convergence speed while maintaining the limiting behavior (e.g., stochastically stable configurations). We support our theoretical results with simulations in a coverage control scenario.

Keywords: Game theory, Cyber-Physical Security, Resilient Control Systems
Abstract: We introduce a game in which two players with opposing objectives seek to repeatedly takeover a common resource. The resource is modeled as a discrete time dynamical system over which a player can gain control after spending a state-dependent amount of energy at each time step. We use a FlipIT-inspired deterministic model that governs which player is in control at every time step. A player's policy is the probability with which it should spend energy to gain control of the resource at a given time step. Our main results are three-fold. First, we present analytic expressions for the cost-to-go as a function of the hybrid state of the system, i.e., the physical state of the dynamical system and the binary FlipDyn state for any general system with arbitrary costs. These expressions are exact when the physical state is also discrete and has finite cardinality. Second, for a continuous physical state with linear dynamics and quadratic costs, we derive expressions for Nash equilibrium (NE). For scalar physical states, we show that the NE depends only on the parameters of the value function and costs, and is independent of the state. Third, we derive an approximate value function for higher dimensional linear systems with quadratic costs. Finally, we illustrate our results through a numerical study on the problem of controlling a linear system in a given environment in the presence of an adversary.

Keywords: Game theory, Decentralized control, Control over communications
Abstract: Motivated by recent works in the communication and psychology literature, we model and study the role social identity -- a person's sense of belonging to a group -- plays in human information consumption. A hallmark of Social Identity Theory (SIT) is the notion of `status', i.e., an individual's desire to enhance their and their `in-group's' utility textit{relative} to that of an `out-group'. In the context of belief formation, this comes off as a desire to believe positive news about the in-group and negative news about the out-group, which has been empirically shown to support belief in misinformation and false news.

We model this phenomenon as a Stackelberg game being played over an information channel between a news-source (sender) and news-consumer (receiver), with the receiver incorporating the `status' associated with social identity in their utility, in addition to accuracy. We characterize the strategy that must be employed by the sender to ensure that its message is trusted by receivers of all identities while maximizing accuracy of information. We show that, as a rule, this optimal quality of information at equilibrium decreases when a receiver's sense of identity increases. Our work supports the perspective that identity based reasoning among receivers is a motivating factor in encouraging media slant. We further demonstrate how extensions of our model can be used to quantitatively estimate the level of importance given to identity in a population.

Keywords: Game theory, Distributed control, Networked control systems
Abstract: We analyze convergence of decentralized fictitious play (DFP) in near-potential games, where agents participate in a game in which the change in utility functions are closely aligned with a potential function. In DFP, agents take actions that maximize their expected utilities computed based on local estimates of empirical frequencies of other agents. These local estimates are updated by averaging estimates received from neighbors in a time-varying communication network. Given near-potential games with finitely many Nash equilibria that are distant enough from each other, we show that the empirical frequencies converge near a single Nash Equilibrium. This result establishes that DFP maintains the properties of standard fictitious play (FP) in near-potential games.

Keywords: Output regulation, Adaptive control, Power systems
Abstract: We propose a novel internal model-based controller to solve the reference tracking problem for an uncertain plant and a sinusoidal reference of unknown frequency. A third order self-synchronizing synchronverter model is used as internal model, leading to remarkable results. We only require that the plant mathbf{P} is stable and linear, and that an ``initial guess'' omega_n on the reference frequency omega_r is available. Under these assumptions, we ensure tracking for omega_rin[0.25omega_n,4omega_n]. Moreover, among other features, our controller is able to withstand (at steady-state) large step jumps in omega_r (approx 300% frequency jumps) and in the dynamics of mathbf{P} (e.g., mathbf{P} is allowed to change sign during operation). Extensive numerical experiments are provided to illustrate these impressive features.

Keywords: Output regulation, Robust adaptive control, Flight control
Abstract: The paper deals with the problem of adaptive output regulation of nonlinear systems. Building on the design technique recently proposed in [1], we present a new approach where the regulator’s internal model is adapted online via Gaussian Processes. Adaptation is performed by following an "event-triggered" logic and hybrid tools are used to analyse the resulting closed-loop system. Unlike the approach of [1], where the steady-state control policy is supposed to belong to a specific finite-dimensional model set, here we only require smoothness. The paper also presents numerical simulations showing how the proposed method outperforms previous approaches.

Keywords: Optimal control, Cooperative control, Output regulation
Abstract: In this paper, a novel adaptive optimal control strategy is proposed to achieve the cooperative optimal output regulation of continuous-time linear multi-agent systems based on adaptive dynamic programming (ADP). The proposed method is different from those in the existing literature of ADP and cooperative output regulation in the sense that the knowledge of the exosystem dynamics is not required in the design of the exostate observers for those agents with no direct access to the exosystem. Moreover, an optimal control policy is obtained without the prior knowledge of the modeling information of any agent while achieving the cooperative output regulation. Instead, we use the state/input information along the trajectories of the underlying dynamical systems and the estimated exostates to learn the optimal control policy. Simulation results show the efficacy of the proposed algorithm, where both estimation errors of exosystem matrix and exostates, and the tracking errors converge to zero in an optimal sense, which solves the cooperative optimal output regulation problem.

Keywords: Output regulation, Uncertain systems
Abstract: This paper deals with the problem of disturbance rejection for uncertain LTI SISO systems perturbed by an emph{unmeasurable} external disturbance under the framework of output regulation. The system is assumed to be minimum phase and internally stable, but the model parameters are completely unknown. In addition, no knowledge of the external disturbance, including frequency, amplitude and phase is required to be known in advance. A novel high-order sliding mode-based Unknown Input Observer(UIO) is developed to stabilize the system and reconstruct the external disturbance. The main feature distinguishing the proposed method from the existing ones is that we do not need to integrate a frequency estimator into the adaptive controller or update the frequency estimation in a hybrid manner. Instead, the disturbance is directly duplicated by the aforementioned unknown input observer. The boundedness of states and asymptotic convergence properties are rigorously proved. Finally, the effectiveness of the proposed technique is illustrated by a numerical example.

Keywords: Output regulation, Optimization algorithms, Constrained control
Abstract: We consider the problem of designing a feedback controller which robustly regulates an LTI system to an optimal operating point in the presence of unmeasured disturbances. A general design framework based on so-called optimality models was previously put forward for this class of problems, effectively reducing the problem to that of stabilization of an associated nonlinear plant. This paper presents several simple and fully constructive stabilizer designs to accompany the optimality model designs from [1]. The designs are based on a low-gain integral control approach, which enforces time-scale separation between the exponentially stable plant and the controller. We provide explicit formulas for controllers and gains, along with LMI-based methods for the computation of robust/optimal gains. The results are illustrated via an academic example and an application to power system frequency control.

Keywords: Feedback linearization, Output regulation, Optimization
Abstract: We discuss a control strategy to handle singularity of input gain matrix in the input-output linearization of a dynamical system with multiple outputs. The key idea is to prioritize the outputs and designate available control inputs to the outputs in the order of priority, by which tracking control for higher priority outputs is possibly maintained even in the presence of the singularity. This strategy of prioritization (called the prioritized control) allows to successfully deal with critical control objectives (such as stability and/or collision avoidance) by assigning higher priority to the relevant outputs. In this paper, we first explain how the prioritization can be applied to the process of input-output linearization and provide a definition of the prioritized control law on the framework of the multi-objective optimization with the lexicographical ordering. Then, a simple case study is performed for a two-link manipulator along with simulation results, where we further discuss an intrinsic degenerate property of prioritization that the prioritized control law can be discontinuous.

Keywords: Healthcare and medical systems, Biomedical, Metabolic systems
Abstract: Controlling postprandial glucose excursions in type 1 diabetes (T1D) still remains a challenge for most automated insulin delivery (AID) algorithms. The main limiting factor has been the slow absorption of current insulin analogs that prevents the controller from responding aggressively enough to significant glucose excursions without increasing the risk for insulin stacking and, consequently, late hypoglycemia. Faster insulin analogs represent an appealing solution to this problem, because they would enable better alignment between meal and insulin rates of appearance. In this paper, we discuss about another way to overcome this fundamental performance limitation that consists in allowing the controller to properly anticipate likely disturbances through a robust model predictive control (MPC) strategy.

Keywords: Healthcare and medical systems, Emerging control applications, Human-in-the-loop control
Abstract: In this paper, we present a model free approach to calculate long-acting insulin doses for Type 2 Diabetic (T2D) subjects in order to bring their blood glucose (BG) concentration to be within a safe range. The proposed strategy tunes the parameters of a proposed control law by using a zeroth-order online stochastic optimization approach for a defined cost function. The strategy uses gradient estimates obtained by a Recursive Least Square (RLS) scheme in an adaptive moment estimation based approach named AdaBelief. Additionally, we show how the proposed strategy with a feedback rating measurement can accommodate for a phenomena known as relative hypoglycemia or pseudo-hypoglycemia (PHG) in which subjects experience hypoglycemia symptoms depending on how quick their BG concentration is lowered. The performance of the insulin calculation strategy is demonstrated and compared with current insulin calculation strategies using simulations with three different models.

Keywords: Healthcare and medical systems, Identification for control, Simulation
Abstract: In type 2 diabetes (T2D) treatment, finding a safe and effective basal insulin dose is a challenge. The dose-response is highly individual and to ensure safety, people with T2D “titrate” by slowly increasing the daily insulin dose to meet treatment targets. This titration can take months. To ease and accelerate the process, we use short-term artificial pancreas (AP) treatment tailored for initial titration and apply it as a diagnostic tool. Specifically, we present a method to automatically estimate a personalized daily dose of basal insulin from closed-loop data collected with an AP. Based on AP-data from a stochastic simulation model, we employ the continuous-discrete extended Kalman filter and a maximum likelihood approach to estimate parameters in a simple dose-response model for 100 virtual people. With the identified model, we compute a daily dose of basal insulin to meet treatment targets for each individual. We test the personalized dose and evaluate the treatment outcomes against clinical reference values. In the tested simulation setup, the proposed method is feasible. However, more extensive tests will reveal whether it can be deemed safe for clinical implementation.

Keywords: Healthcare and medical systems, Predictive control for linear systems, Process Control
Abstract: Hybrid Model Predictive Control (HMPC) is presented as a decision-making tool for novel behavioral interventions to increase physical activity in sedentary adults, such as Just Walk. A broad-based HMPC formulation for mixed logical dynamical (MLD) systems relevant to problems in behavioral medicine is developed and illustrated on a representative participant model arising from the Just Walk study. The MLD model is developed based on the requirement of granting points for meeting daily step goals and categorical input variables. The algorithm features three degrees-of-freedom tuning for setpoint tracking, measured and unmeasured disturbance rejection that facilitates controller robustness; disturbance anticipation further improves performance for upcoming events such as weekends and weather forecasts. To avoid the corresponding mixed-integer quadratic problem (MIQP) from becoming infeasible, slack variables are introduced in the objective function. Simulation results indicate that the proposed HMPC scheme effectively manages hybrid dynamics, setpoint tracking, disturbance rejection, and the transition between the two phases of the intervention (initiation and maintenance) and is suitable for evaluation in clinical trials.

Keywords: Human-in-the-loop control, Healthcare and medical systems, Robotics
Abstract: Wearable devices such as rehabilitation robots, power augmentation exoskeletons, and haptic manipulators feature operation modes that need to mirror the user's action as naturally as possible. By tracking such motions, the human-robot physical interaction is minimized, or, in other words, the device displays effusive mechanical transparency. The present work applies a transparency control framework based on inertial measurement units fixed on the user to a series elastic exoskeleton joint. This approach guarantees the robot's transparent behavior without explicitly relying on force or human-robot interaction sensors. Instead, we used a state estimator to provide the human-related quantities needed for feedforward and feedback control. The proposed controller considers the dynamics of the series elastic actuator for two motor driver configurations: torque-based and velocity-based low-level control. Using a novel definition of the human-robot transparency impedance, we designed and experimentally evaluated the performance of both controllers.

Keywords: Healthcare and medical systems, Biomedical, Control applications
Abstract: This paper investigates the potential of model-based extended Kalman filtering (EKF) for hemodynamic monitoring in a hemorrhage resuscitation-sedation case study. To the best of our knowledge, it may be the first model-based state estimation study conducted in the context of hemodynamic monitoring. Built upon a grey-box mathematical model with parametric uncertainty as process noise, the EKF can estimate cardiac output (CO) and total peripheral resistance (TPR) continuously from mean arterial pressure (AP) measurements against inter-individual physiological and pharmacological variability. Its unique practical strengths include: it does not require AP waveform as in existing AP-based pulse-contour CO (PCCO) monitors; and it can estimate CO and TPR with explicit account for the effect of sedative drugs. The efficacy of the EKF-based hemodynamic monitoring was evaluated based on a large number of plausible virtual patients generated using a collective inference algorithm, which demonstrated that it has significant advantage over open-loop pure prediction, and that its accuracy is comparable to PCCO.

Keywords: Hybrid systems, Sampled-data control, Discrete event systems
Abstract: In this work, we propose a data-driven approach for the construction of finite abstractions (a.k.a., symbolic models) for discrete-time deterministic control systems with unknown dynamics. We leverage notions of so-called alternating bisimulation functions (ABF), as a relation between each unknown system and its symbolic model, to quantify the mismatch between state behaviors of two systems. Accordingly, one can employ our proposed results to perform formal verification and synthesis over symbolic models and then carry the results back over unknown original systems. In our data-driven setting, we first cast the required conditions for constructing ABF as a robust optimization program (ROP). Solving the provided ROP is not tractable due to the existence of unknown models in the constraints of ROP. To tackle this difficulty, we collect finite numbers of data from trajectories of unknown systems and propose a scenario optimization program (SOP) corresponding to the original ROP. By establishing a probabilistic relation between optimal values of SOP and ROP, we formally construct ABF between unknown systems and their symbolic models based on the number of data and a required confidence level. We verify the effectiveness of our data-driven results over two physical case studies with unknown models including (i) a DC motor and (ii) a nonlinear jet engine compressor. We construct symbolic models from data as appropriate substitutes of original systems and synthesize policies maintaining states of unknown systems in a safe set within infinite time horizons with some guaranteed confidence levels.

Keywords: Hybrid systems, Modeling, Automata
Abstract: Hybrid automata are a powerful model class to analyze and control technical systems with nonlinear continuous dynamics and discrete-event dynamics. However, due to their complexity, controllability is generally hard to achieve in hybrid automata, although it is a desirable property of engineered systems. In the present paper, we assess a novel model class of controllable deterministic hybrid automata, which we call Flat Hybrid Automata. This model class is based on the concept of differential flatness for the nonlinear continuous dynamics and on the property of determinism and controllability for the discrete event dynamics. We review the formal definition and we give the respective conditions and a proof for its controllability. An electrical network example illustrates the methodology and shows its potential regarding modelling, control and system design.

Keywords: Hybrid systems, Nonholonomic systems, Robotics
Abstract: Why do we move forward when we walk? Our legs undergo periodic motion and thus possess no net change in position; however, our bodies do possess a net change in position and we are propelled forward. From a geometric perspective, this phenomenon of periodic input producing non-periodic output is holonomy. To obtain non-zero holonomy and propel forward, we must alternate which leg is in contact with the ground; a non-zero net motion can be obtained by concatenating arcs that would individually produce no net motion. We develop a framework for computing the holonomy group of hybrid systems and analyze their behavior in the limit as the number of impacts goes to infinity.

Keywords: Hybrid systems
Abstract: This paper proposes a rapidly-exploring random trees (RRT) algorithm to solve the motion planning problem for hybrid systems. At each iteration, the proposed algorithm, called HyRRT, randomly picks a state sample and extends the search tree by flow or jump, which is also chosen randomly when both regimes are possible. Through a definition of concatenation of functions defined on hybrid time domains, we show that HyRRT is probabilistically complete, namely, the probability of failing to find a motion plan approaches zero as the number of iterations of the algorithm increases. This property is guaranteed under mild conditions on the data defining the motion plan, which include a relaxation of the usual positive clearance assumption imposed in the literature of classical systems. The motion plan is computed through the solution of two optimization problems, one associated with the flow and the other with the jumps of the system. The proposed algorithm is applied to a walking robot so as to highlight its generality and computational features.

Keywords: Hybrid systems, Stochastic systems, Sampled-data control
Abstract: This paper is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness guarantees. The proposed scheme is based on notions of stochastic bisimulation functions (SBF) to capture the probabilistic distance between state trajectories of an unknown stochastic system and those of finite MDP. In our proposed setting, we first reformulate corresponding conditions of SBF as a robust convex program (RCP). We then propose a scenario convex program (SCP) associated to the original RCP by collecting a finite number of data from trajectories of the system. We ultimately construct an SBF between the data-driven finite MDP and the unknown stochastic system with a given confidence level by establishing a probabilistic relation between optimal values of the SCP and the RCP. We also propose two different approaches for the construction of finite MDPs from data. We illustrate the efficacy of our results over a nonlinear jet engine compressor with unknown dynamics. We construct a data-driven finite MDP as a suitable substitute of the original system to synthesize controllers maintaining the system in a safe set with some probability of satisfaction and a desirable confidence level.

Keywords: Hybrid systems, Stochastic systems, Sampled-data control
Abstract: In this work, we develop a data-driven framework with formal confidence bounds for the estimation of infinitesimal generators for continuous-time stochastic hybrid systems with unknown dynamics. The proposed approximation scheme employs both time discretization and sampling from the solution process, and estimates the infinitesimal generator of the solution process via a set of data collected from trajectories of systems. We assume some mild continuity assumptions on the dynamics of the system and quantify the closeness between the infinitesimal generator and its approximation while ensuring an a-priori guaranteed confidence bound. To provide a reasonable closeness precision, we discuss significant roles of both time discretization and number of data in our approximation scheme. In particular, for a fixed number of data, variance of the estimation converges to infinity when the time discretization goes to zero. The proposed approximation framework guides us how to jointly select a suitable data size and a time discretization parameter to cope with this counter-intuitive behavior. We demonstrate the effectiveness of our proposed results by applying them to a nonlinear jet engine compressor with unknown dynamics.

Keywords: Adaptive systems, Biologically-inspired methods, Time-varying systems
Abstract: We analyze a class of high-frequency, high-amplitude oscillatory systems in which periodicity occurs on two distinct time scales and establish the convergence of its trajectories to a suitably averaged system by recursively applying the averaging theorem. Moreover, we introduce a novel bio-inspired 3D source seeking algorithm for rigid bodies with a collocated sensor and prove its practical stability under typical assumptions on the source signal strength field by combining our averaging results with singular perturbation.

Keywords: Adaptive systems, Estimation, Learning
Abstract: There are many parameter adaptation/learning algorithms (PALA) used in adaptive control, system identification and neural networks (Nesterov, Conjugate gradients, Momentum back propagation, Averaged gradient, Integral+ proportional+derivative, ...). For most of these algorithms unfortunately there are no results available for the choice of the various coefficients (weights) allowing to guarantee the stability of the estimator for any value of the learning rate and for any initial conditions. All these algorithms are in fact particular cases of a general structure for PALA which is introduced in this paper. This structure is characterized by the presence of an embedded ARMA (poles-zeros) filter. Taking into account the inherent feedback structure of these adaptation/learning algorithms, passivity approach is used for addressing the stability issue. Conditions which will assure the stability of this general structure will be provided and then particularized for the specific algorithms described in the paper. The impact of the MA and AR terms of the embedded filter upon the performance of the algorithms will be emphasized through simulation.

Keywords: Lyapunov methods, Stability of linear systems, Adaptive systems
Abstract: Stability analyses of discrete-time adaptive control algorithms are generally predicated on quadratic Lyapunov-based frameworks that result in unavoidable complexity due to the resulting terms in the Lyapunov difference equations. This prevents generalizations of valuable continuous-time adaptive control results to their discrete-time settings. To this end, one important generalization is the consideration of actuator dynamics, which is present in any uncertain dynamical system. To address this problem, we propose a novel discrete-time adaptive control architecture predicated on the hedging method and a new projection operator. A logarithmic Lyapunov function is used for proving the asymptotic stability of the error between uncertain dynamical system states and hedging-based reference model states. An illustrative numerical example is then presented to demonstrate the efficacy of the proposed architecture.

Keywords: Direct adaptive control, Biological systems, Optimization
Abstract: Recently, several Newton-based extremum seeking strategies have been proposed, which provide fast convergence. In this study, we focus attention on perturbation-based proportional-integral extremum seeking (PIES) and block-oriented model recursive least squares extremum seeking (BOM-RLSES), and we discuss stability and convergence, highlighting the impact of the quality of the Hessian estimation. A bioprocess application is used to compare the performance of PI and RLS Newton-based strategies with respect to the blockoriented RLSES. The results confirm the faster convergence of the Newton-based formulations for an input range contained in the region of attraction of the extremum.

Keywords: Direct adaptive control, Stability of nonlinear systems
Abstract: In this work, we are concerned with the design of direct data-driven controllers for nonlinear affine systems without explicit dynamical models. To this end, we adopt the Koopman operator to approximately reformulate the nonlinear systems into bilinear forms, based on which we propose a “simple” static controller in the linear form of the preset function dictionary. Then, we show how to obtain the feedback gain matrix and establish the stability condition of the closed- loop system using only the off-line collected data. Finally, numerical results demonstrate the effectiveness and robustness of this data-driven controller.

Keywords: Direct adaptive control, Uncertain systems
Abstract: In this work, we propose a direct-adaptive MRAC for relative-degree-unity SISO systems with unknown control direction. The proposed scheme, employing an original construction of the control law and the use of an adaptive observer, achieves the long-searched objective of injecting, through the input, the unmeasurable derivative of the output error. The output derivative injection is performed by a smart construction of the control input that features a Parameter-dependent Input Normalization (PIN). The PIN scheme does not make use of Nussbaum functions usually invoked in the direct-adaptive setting, does not require persistence of excitation of indirect adaptive schemes does not require switching between multiple models does not suffer from singularities, and does not require to know a-priori bounds on the norm of the high-frequency gain and on the parameters. Effectiveness of the algorithm is illustrated by a numerical example.

Keywords: Robotics, Estimation, Biologically-inspired methods
Abstract: This paper presents a novel bioinspired underwater target detection method with distributed mobile sensors. Inspired by the fish's lateral line system, distributed mobile sensors are proposed to measure their respective local flow pressure and collectively estimate the parameters (e.g., position and dimension) of underwater targets with dynamic sensor trajectory planning. This paper formulates an optimal trajectory planning problem for a group of mobile sensors using information entropy as the cost function, with an aim to maximize the confidence level in the estimated target parameters. This paper in particular investigates the detection of a circular target in a uniform flow. A Bayesian filter is adopted to assimilate the distributed pressure measurements dynamically to estimate the position and radius of given circular targets. To verify the effectiveness of the proposed method, simulation is conducted with both fixed and moving circular targets in uniform flow. Simulation results are presented and analyzed.

Keywords: Robotics, Adaptive control, Learning
Abstract: This paper introduces a new event-sampled intelligent learning method to solve tracking control for motion constrained robotic system. In many applications, motion constraints on joint movements of a robot are needed for its safe operation or to accommodate limitations arising from its mechanical structures. To handle the control design problem, a state-transformation method is employed to guarantee motion constraints of the robot and obtain a corresponding unconstrained tracking error system. Next, the constrained control problem is transformed into a general optimal tracking control problem for the unconstrained error system. Finally, we implement neural networks based learning structures to obtain the optimal solution. An important highlight of our proposed algorithm is that we have integrated a dynamic event-sampled approach to the learning-based controller design, thus reducing the system state sampling times. Using the proposed event-sampled learning control method, the stability of the closed-loop system and the convergence of the weights of neural networks are both established. Lastly, the effectiveness of the proposed intelligent control method is verified via simulation.

Keywords: Stability of nonlinear systems, Robotics, Formal Verification/Synthesis
Abstract: Linear-quadratic regulators (LQR) are a well known and widely used tool in control theory for both linear and nonlinear dynamics. For nonlinear problems, an LQR-based controller is usually only locally viable, thus, raising the problem of estimating the region of attraction (ROA). The need for good ROA estimations becomes especially pressing for underactuated systems, as a failure of controls might lead to unsafe and unrecoverable system states. Known approaches based on optimization or sampling, while working well, might be too slow in time critical applications and are hard to verify formally. In this work, we propose a novel approach to estimate the ROA based on the analytic solutions to linear ODEs for the torque limited simple pendulum. In simulation and physical experiments, we compared our approach to a Lyapunov-sampling baseline approach and found that our approach was faster to compute, while yielding ROA estimations of similar phase space area.

Keywords: Distributed parameter systems, Robotics, Filtering
Abstract: Compared with traditional rigid-body robots, soft robots not only exhibit unprecedented adaptation and flexibility but also present novel challenges in their modeling and control because of their infinite degrees of freedom. Most of the existing approaches have mainly relied on approximated models so that the well-developed finite-dimensional control theory can be exploited. However, this may bring in modeling uncertainty and performance degradation. Hence, we propose to exploit infinite-dimensional analysis for soft robotic systems. Our control design is based on the increasingly adopted Cosserat rod model, which describes the kinematics and dynamics of soft robotic arms using nonlinear partial differential equations (PDE). We design infinite-dimensional state feedback control laws for the Cosserat PDE model to achieve trajectory tracking (consisting of position, rotation, linear and angular velocities) and prove their uniform tracking convergence. We also design an infinite-dimensional extended Kalman filter on Lie groups for the PDE system to estimate all the state variables (including position, rotation, strains, curvature, linear and angular velocities) using only position measurements. The proposed algorithms are evaluated using simulations.

Keywords: Nonholonomic systems, Adaptive control, Robotics
Abstract: This work introduces a new single-stage adaptive controller for Euler-Lagrange systems with nonholonomic constraints. The proposed mechanism provides a simpler design philosophy compared to double-stage mechanisms (that address kinematics and dynamics in two steps), while achieving analogous stability properties, i.e. stability of both original and internal states. Meanwhile, we do not require direct access to the internal states as required in state-of-the-art single-stage mechanisms. The proposed approach is studied via Lyapunov analysis, validated numerically on wheeled mobile robot dynamics and compared to a standard double-stage approach.

Keywords: Observers for nonlinear systems, Stability of nonlinear systems, Robotics
Abstract: Pfaffian (velocity) constraints are encountered commonly in mechanical systems. In many cases, the measurements for feedback control are obtained from an exteroceptive sensor system, which is not only of low rate, but also suffers from physical discontinuities. This negatively affects controller performance and places severe limitations on the choice of control parameters. To this end, a novel framework comprised of a rigid Body Observer (BObs) and a pose regulator is proposed. During the inter-sampling periods, the observer propagates the state based on an internal model to provide continuous estimates, which are exploited by the pose regulator to stabilize equilibria. We prove uniform asymptotic stability for the closed loop. Furthermore, we validate the proposed framework through simulation and also the BObs experimentally.

Keywords: Neural networks, Markov processes, Switched systems
Abstract: Neural networks (NNs) are emerging as powerful tools to represent the dynamics of control systems with complicated physics or black-box components. Due to complexity of NNs, however, existing methods are unable to synthesize complex behaviors with guarantees for NN dynamic models (NNDMs). This work introduces a control synthesis framework for stochastic NNDMs with performance guarantees. The focus is on specifications expressed in linear temporal logic interpreted over finite traces (LTLf), and the approach is based on finite abstraction. Specifically, we leverage recent techniques for convex relaxation of NNs to formally abstract a NNDM into an interval Markov decision process (IMDP). Then, a strategy that maximizes the probability of satisfying a given specification is synthesized over the IMDP and mapped back to the underlying NNDM. We show that the process of abstracting NNDMs to IMDPs reduces to a set of convex optimization problems, hence guaranteeing efficiency. We also present an adaptive refinement procedure that makes the framework scalable. On several case studies, we illustrate the our framework is able to provide non-trivial guarantees of correctness for NNDMs with architectures of up to 5 hidden layers and hundreds of neurons per layer.

Keywords: Neural networks, Optimization algorithms, Machine learning
Abstract: It was shown in the literature that, for a fully connected neural network (NN), the gradient descent algorithm converges to zero. Motivated by that work, we provide here general conditions under which we can derive the convergence of the gradient descent algorithm from the convergence of the gradient flow, in the case of NNs, in a systematic way. Our approach is based on an analysis of the error in Euler's method in the case of NNs, and relies on the concept of local strong convexity. Unlike existing approaches in the literature, our approach allows to provide convergence guarantees without making any assumptions on the number of hidden nodes of the NN or the number of training data points. A numerical example is proposed, showing the merits of our approach.

Keywords: Neural networks, Machine learning, LMIs
Abstract: This paper is concerned with the training of neural networks (NNs) under semidefinite constraints, which allows for NN training with robustness and stability guarantees. In particular, we focus on Lipschitz bounds for NNs. Exploiting the banded structure of the underlying matrix constraint, we set up an efficient and scalable training scheme for NN training problems of this kind based on interior point methods. Our implementation allows to enforce Lipschitz constraints in the training of large-scale deep NNs such as Wasserstein generative adversarial networks (WGANs) via semidefinite constraints. In numerical examples, we show the superiority of our method and its applicability to WGAN training.

Keywords: Neural networks, Machine learning, Optimization
Abstract: Artificial neural networks have recently been utilized in many feedback control systems and introduced new challenges regarding the safety of such systems. This paper considers the safe verification problem for a dynamical system with a given feedforward neural network as the feedback controller by using a constrained zonotope-based approach. A novel set-based method is proposed to compute both exact and over-approximated reachable sets for neural feedback systems with linear models, and linear program-based sufficient conditions are presented to verify whether the trajectories of such a system can avoid unsafe regions represented as constrained zonotopes. The results are also extended to neural feedback systems with nonlinear models. The computational efficiency and accuracy of the proposed method are demonstrated by two numerical examples where a comparison with state-of-the-art methods is also provided.

Keywords: Neural networks, Nonlinear systems, Optimization
Abstract: Neural networks have seen a recent increased use in control feedback systems. However, providing robustness guarantees on these feedback systems has proven challenging and to combat these issues, there is a significant amount of current research. One of the biggest shortcoming of neural networks is how sensitive they are to adversarial inputs. Given that feedback systems are usually subject to external perturbations, this issue surrounding neural networks must be overcome before they can be used in safety-critical applications. One method to tackle this problem is to compute outer-approximations of the reachable sets, through bounding the activation functions in the neural network controller. Our approach is to use these bounds in a sparse polynomial optimisation framework in conjunction with the Positivstellensatz. The sparsity property is able to exploit the natural cascading structure of the neural network to allow for tractable solve times. The Positivstellensatz is able to provide more accurate bounds over similar methods by asserting the emptiness of a semi-algebraic set. We show through examples that our method can provide tighter bounds over similar methods, with reasonable computational time. Our approach is also able to deal with non-linear polynomial dynamics due to the polynomial optimisation framework we use, while other methods need to use alternative work-around solutions to incorporate the non-linearities.

Keywords: Neural networks, Optimization, Optimization algorithms
Abstract: Recent works in deep learning have demonstrated impressive performance using “implicit deep models,” wherein conventional architectures composed of forward-propagating, differentiable parametric layers are replaced by more expressive models composed of an implicitly defined fixed-point equation together with a prediction equation. Methods for training implicit deep models are currently restricted to end-to-end optimization, which relies on solving a matrix-variable fixed-point equation to compute the gradient and an expensive projection step at every iteration. In this work, we extend the idea of greedy layer-wise training, an approach found to yield state-of-the-art performance in conventional deep learning, to a sequential greedy training algorithm for implicit deep models with a strictly upper block triangular structure. We show that such implicit models can be regarded as generalized dense block modules of Dense Convolutional Networks (DenseNets), and thus inherit the underlying parameter efficiency property. For models trained with the Euclidean loss, we develop an alternating minimization subroutine for our sequential optimization algorithm, which consists of alternating between efficiently solvable least squares problems and single hidden-layer training problems. Furthermore, we theoretically prove that training a non-strictly upper triangular ReLU implicit model is equivalent to training a strictly upper block triangular one, allowing for the application of our algorithm to even more general models. Experiments on smooth and nonsmooth function interpolation, and on MNIST and Fashion-MNIST classification tasks, show that our algorithm consistently converges to models that out-perform state-of-the-art end-to-end implicit learning.

Keywords: Uncertain systems, Linear systems
Abstract: In this paper, a method to represent every input-output trajectory of a continuous-time linear system in terms of previously collected data is presented. This corresponds to a continuous-time version of the well-known Willems' lemma. The result is obtained by sampling the continuous signals at regular intervals, and constructing Hankel-like structures that closely resemble their discrete-time counterparts. Then, it is shown how to use measured persistently excited data to design a time-varying vector of parameters that allows the generation of arbitrary piecewise differentiable trajectories. A class of input signals that satisfies the conditions for persistence of excitation is also provided.

Keywords: Optimal control, Neural networks
Abstract: We address the problem of designing stabilizing control policies for nonlinear systems in discrete-time, while minimizing an arbitrary cost function. When the system is linear and the cost is convex, the System Level Synthesis (SLS) approach offers an effective solution based on convex programming. Beyond this case, a globally optimal solution cannot be found in a tractable way, in general. In this paper, we develop a parametrization of all and only the control policies stabilizing a given time-varying nonlinear system in terms of the combined effect of 1) a strongly stabilizing base controller and 2) a stable SLS operator to be freely designed. Based on this result, we propose a Neural SLS (Neur-SLS) approach guaranteeing closed-loop stability during and after parameter optimization, without requiring any constraints to be satisfied. We exploit recent Deep Neural Network (DNN) models based on Recurrent Equilibrium Networks (RENs) to learn over a rich class of nonlinear stable operators, and demonstrate the effectiveness of the proposed approach in numerical examples.

Keywords: Estimation, Lyapunov methods, Observers for Linear systems
Abstract: This paper proposes a data-driven input reconstruction method from outputs (IRO) based on the Willems' Fundamental Lemma. Given only output measurements, the unknown inputs estimated recursively by the IRO asymptotically converge to the true input without knowing the initial conditions. A recursive IRO and a moving-horizon IRO are developed based respectively on Lyapunov conditions and Luenberger-observer-type feedback, and their asymptotic convergence properties are studied. An experimental study is presented demonstrating the efficacy of the moving-horizon IRO for estimating the occupancy of a building on the EPFL campus via measured carbon dioxide levels.

Keywords: Observers for Linear systems, Learning, Estimation
Abstract: This paper proposes a method for learning optimal state estimators from input/output data for linear discrete-time stochastic systems. We show that this problem can be expressed in the reinforcement learning framework, suitably adapted to the peculiar problem structure. In particular, we introduce the specific Bellman equation for the state estimation problem and use temporal differences to solve it. We show in simulations that the resulting data-driven method for state estimation converges to the optimal observer.

Keywords: Identification for control, Robust control, LMIs
Abstract: In this paper we investigate data-based analysis and design problems without making the prevailing assumption that there exists a common Lyapunov function for all systems unfalsified by data. In particular, we provide necessary and sufficient conditions under which a given set of state data are informative for stability, in the sense that all systems explaining the data are stable. These conditions are derived by making use of the celebrated Kalman-Yakubovich-Popov lemma. We also explain the potential of extending these results to other analysis problems like data-based stabilizability, and to the design of stabilizing controllers. The results are applied to analyze the stability of a large-scale network, which highlights the tractability of the provided framework in comparison to previous conditions involving linear matrix inequalities.

Keywords: Identification, Identification for control, Statistical learning
Abstract: Complementing data-driven models of dynamic systems with certificates of reliability and safety is of critical importance in many applications, such as in the design of robust control policies for unknown or uncertain systems. In this paper, we propose an efficient method to construct finite-sample confidence regions for the parameters of unknown linear systems in state-space form. The proposed procedure builds on the Sign-Perturbed Sums (SPS) paradigm and returns regions that are provably exact, i.e., contain the true parameters with the desired probability, using finite data and under minimal assumptions on the noise distribution. In particular, the noise distribution is only required to be symmetric about the origin, thus encompassing as special cases commonly studied noise settings, such as zero-mean Gaussian or Laplacian noise. The performance of our procedure is tested across different noise scenarios and compared to that of alternative approaches.

Keywords: Identification for control, Closed-loop identification, Sampled-data control
Abstract: Frequency-domain representations of multirate systems are essential for controller design and performance evaluation of multirate systems and sampled-data control. The aim of this paper is to develop a time-efficient closed-loop identification approach for multirate systems in the frequency domain. The developed method utilizes local polynomial modeling for lifted representations of LPTV systems, which enables direct identification of closed-loop multirate systems in a single identification experiment. Unlike LTI identification techniques, the developed method does not suffer from bias due to ignored LPTV dynamics. The developed approach is demonstrated on a multirate example, resulting in accurate and fast identification in the frequency domain.

Keywords: Identification, Stochastic optimal control, Optimization
Abstract: In this paper, we define and solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic Gaussian (LQG) and the linear-quadratic sensorimotor (LQS) control model. These Stochastic Optimal Control (SOC) models are state-of-the-art approaches describing human movements. The LQG ISOC problem consists of finding the unknown weighting matrices of the quadratic cost function and the covariance matrices of the additive Gaussian noise processes based on ground truth trajectories observed from the human in practice. The LQS ISOC problem aims at additionally finding the covariance matrices of the signal-dependent noise processes characteristic for the LQS model. We propose a solution to both ISOC problems which iteratively estimates cost function and covariance matrices via two bi-level optimizations. Simulation examples show the effectiveness of our developed algorithm. It finds parameters that yield trajectories matching mean and variance of the ground truth data.

Keywords: Identification, Switched systems, Identification for control
Abstract: For the identification of switched systems with measured states and a measured switching signal, this work aims to analyze the effect of switching strategies on the estimation error. The data is assumed to be collected from globally asymptotically or marginally stable switched systems under switches that are arbitrary or subject to an average dwell time constraint. Then the switched system is estimated by the least-squares (LS) estimator. To capture the effect of the parameters of the switching strategies on the LS estimation error, finite-sample error bounds are developed in this work. The obtained error bounds show that the estimation error is logarithmic of the switching parameters when there are only stable modes; however, when there are unstable modes, the estimation error bound can increase linearly as the switching parameter changes. This suggests that in the presence of unstable modes, the switching strategy should be properly designed to avoid the significant increase of the estimation error.

Keywords: Neural networks, Identification for control, Nonlinear systems identification
Abstract: This paper presents a framework for physics-informed energy-based neural network (NN) design to learn models of interconnected systems under the port-Hamiltonian (pH) formalism. In particular, this paper focuses on mechanical systems and incorporates the physical knowledge of Lagrangians into the neural networks to facilitate learning of equations of motion from the data. Moreover, the transformation from the Lagrangian mechanics to the Hamiltonian mechanics is incorporated into the NN architecture and learned from the data such that the learned model is compatible with the pH framework. Then, the structure of input-state-output pH models is imposed on the NN, which guarantees the dissipativity of the learned model. Furthermore, modeling interconnected systems is facilitated by the compositionality property of the pH systems. Additionally, the consistency between the Hamiltonian and Lagrangian is employed for the energy estimation to enable energy-based control. The proposed approach is shown to be computationally more efficient than the existing Lagrangian-based NN design approaches. Furthermore, the learned models with energy estimation are employed for energy-based model predictive control (MPC) design purpose. Experimental results using single (and double) inverted pendulum on carts show that the proposed learning-based approach can achieve an improved performance of model identification compared to the Lagrangian neural networks, accurate estimation of energies and strong control performance.

Keywords: Identification for control, Predictive control for linear systems
Abstract: In this paper, we propose a novel approach to map experimental measurements directly onto explicit predictive control laws, without the need of deriving the system matrices first. To this aim, we resort to Willems’ fundamental lemma, through which we obtain the explicit formulas by suitably elaborating the resulting constrained optimization problem. To tackle the pervading presence of noise in real data, we propose a noise handling strategy guaranteeing to asymptotically recover the noiseless performance. The effectiveness of the proposed method is supported by numerical simulations on two benchmark examples.

Keywords: Identification, Indirect adaptive control, Robust adaptive control
Abstract: In this work, we propose a novel parameter identifier for linear time-invariant systems, which has the fastest possible convergence in principal (i.e., dead-beat). The identifier applies where states and derivatives of states can be measured. The identifier is applied to the certainty-equivalence (indirect) model reference adaptive control that relaxes the conventional assumptions about open-loop stability and persistent excitation in MRAC. Furthermore, in conventional MRAC the parameter convergence rate is proportional to the tracking error, whereas our proposed adaptation law has dead-beat dynamics. This method is previously applied to a Delta robot. In this paper, we present a theoretical analysis of the method by solving the identifier dynamics to get its closed-form solution and examine the effect of perturbations such as white additive noise, uncertainty, and disturbance. The linearized model of an actuated inverted pendulum with unknown parameters is used as a benchmark example.

Keywords: Game theory, Optimization algorithms
Abstract: A major challenge in multi-agent systems is that the system complexity grows dramatically with the number of agents as well as the size of their action spaces, which is typical in real world scenarios such as autonomous vehicles, robotic teams, network routing, etc. It is hence in imminent need to design decentralized or independent algorithms where the update of each agent is only based on their local observations without the need of introducing complex communication/coordination mechanisms. In this work, we study the finite-time convergence of independent entropy-regularized natural policy gradient (NPG) methods for potential games, where the difference in an agent's utility function due to unilateral deviation matches exactly that of a common potential function. The proposed entropy-regularized NPG method enables each agent to deploy symmetric, decentralized, and multiplicative updates according to its own payoff. We show that the proposed method converges to the quantal response equilibrium (QRE)---the equilibrium to the entropy-regularized game---at a sublinear rate, which is independent of the size of the action space and grows at most sublinearly with the number of agents. Appealingly, the convergence rate further becomes independent with the number of agents for the important special case of identical-interest games, leading to the first method that converges at a dimension-free rate. Our approach can be used as a smoothing technique to find an approximate Nash equilibrium (NE) of the unregularized problem without assuming that stationary policies are isolated.

Keywords: Adaptive control, Optimal control, Linear systems
Abstract: This paper proposes model-based and model-free inverse reinforcement learning (RL) control algorithms for multiplayer game systems described by linear continuous-time differential equations. Both algorithms find the learner the same optimal control policies and trajectories as the expert, by inferring the unknown expert players' cost functions from the expert's trajectories. This paper first discusses a model-based inverse RL policy iteration that consists of 1) policy evaluation for cost matrices using a Lyapunov equation, 2) state-reward weight improvement using inverse optimal control (IOC), and 3) policy improvement using optimal control. Based on the model-based algorithm, an online data-driven inverse RL algorithm is proposed without knowing system dynamics or expert control gains. Rigorous convergence and stability analysis of these algorithms are provided. Finally, a simulation example verifies our approach.

Keywords: Mean field games, Learning, Large-scale systems
Abstract: Independent learners naively employ single-agent learning algorithms in multi-agent systems, oblivious to the effect of other strategic agents present in their environment. This paper studies partially observed N-player mean-field games from a decentralized learning perspective with two primary objectives: (i) to study the convergence properties of independent learners, and (ii) to identify structural properties that can guide algorithm design. Toward the first objective, we study the learning iterates obtained by independent learners, and find that these iterates converge under mild conditions. We then present a notion of subjective equilibrium suitable for analyzing independent learners. Toward the second objective, we study policy updating processes subject to a so-called epsilon-satisficing condition: agents who are subjectively epsilon-best-responding at a given joint policy do not change their policy. After establishing structural results for such processes, we develop an independent learning algorithm for N-player mean-field games. Exploiting the aforementioned structural results, we give guarantees of convergence to subjective epsilon-equilibrium under self-play.

Keywords: Game theory, Markov processes, Learning
Abstract: This paper presents a learning dynamic with almost sure convergence guarantee for any stochastic game with turn-based controllers (on state transitions) as long as stage-payoffs have stochastic fictitious-play-property. For example, two-player zero-sum and n-player potential strategic-form games have this property. Note also that stage-payoffs for different states can have different structures such as they can sum to zero in some states and be identical in others. The dynamics presented combines the classical stochastic fictitious play with value iteration for stochastic games. There are two key properties: (i) players play finite horizon stochastic games with increasing lengths within the underlying infinite-horizon stochastic game, and (ii) the turn-based controllers ensure that the auxiliary stage-games (induced from the continuation payoff estimated) have the stochastic fictitious-play-property.

Keywords: Markov processes, Machine learning, Agents-based systems
Abstract: In this work we investigate an importance sampling approach for evaluating policies for a structurally time-varying factored Markov decision process (MDP), i.e. the policy's value is estimated with a high-probability confidence interval. In particular, we begin with a multi-agent MDP controlled by a known policy but with unknown transition dynamics. One agent is then removed from the system - i.e. the system experiences node dropout - forming a new MDP of the remaining agents, with a new state space, action space, and new transition dynamics. We assume that the effect of removing an agent corresponds to the marginalization of its factor in the transition dynamics. The reward function may likewise be marginalized, or it may be entirely redefined for the new system. Robust policy importance sampling is then used to evaluate candidate policies for the new system, and estimated values are presented with probabilistic confidence bounds. This computation is completed with no observations of the new system, meaning that a safe policy may be found before dropout occurs. The utility of this approach is demonstrated in simulation and compared to Monte Carlo simulation of the new system.

Keywords: Game theory, Learning, Emerging control applications
Abstract: How can a social planner adaptively incentivize selfish agents who are learning in a strategic environment to induce a socially optimal outcome in the long run? We propose a two-timescale learning dynamics to answer this question in games. In our learning dynamics, players adopt a class of learning rules to update their strategies at a faster timescale, while a social planner updates the incentive mechanism at a slower timescale. In particular, the update of the incentive mechanism is based on each player's externality, which is evaluated as the difference between the player's marginal cost and the society's marginal cost in each time step. We show that any fixed point of our learning dynamics corresponds to the optimal incentive mechanism such that the corresponding Nash equilibrium also achieves social optimality. We also provide sufficient conditions for the learning dynamics to converge to a fixed point so that the adaptive incentive mechanism eventually induces a socially optimal outcome. Finally, as an example, we demonstrate that the sufficient conditions for convergence are satisfied in Cournot competition with finite players.

Keywords: Formal Verification/Synthesis, Agents-based systems, Network analysis and control
Abstract: We define μTGL, a spatio-temporal logic with fixed points and first-order agent quantification whose expressive power allows the definition of topological properties of networks of communicating agents. The existence of temporal operators and fixed points requires particular care when defining its semantics. We demonstrate the logic’s usefulness on an example, where we monitor a complex property that ensures resilient consensus.

Keywords: Formal Verification/Synthesis, Autonomous robots
Abstract: Modular robots are highly versatile due to their ability to reconfigure and change their mechanical properties. This ability makes them optimal for scenarios that require different types of tasks. However, task allocation and cooperation become a combinatorial problem when the number of modules increases. To tackle this problem, we propose a high-level planner for reconfigurable robots with heterogeneous capabilities, e.g., aerial motion and tool operation. Modules can attach and detach to create configurations that manipulate tools satisfying temporal and logic-constrained tasks. The mission is specified using Metric Temporal Logic (MTL) which offers the capacity to not only account for where and who needs to satisfy a task but also when and for how long. We model the problem using a Mixed Integer Linear Problem (MILP) approach, capturing cost for reconfiguration, satisfying a task, and motion in the environment in a specific configuration. Additionally, we consider that not all configurations can satisfy every task. We find trajectories for modular robots that guarantee mission satisfaction. Finally, we show the performance in simulations with multiple tasks and requirements in an environment.

Keywords: Formal Verification/Synthesis, Estimation, Autonomous systems
Abstract: This work considers the problem of resource-performance trade-off analysis for a system consisting of an active agent and a sensor network. Specifically, we focus on Event-triggered (ET) estimation, which allows communication of measurements only when deemed useful, e.g., when Kalman filter innovations exceed some threshold. We introduce a framework for ET-threshold strategy synthesis that optimally trades off resource consumption with task performance. Our approach is based on a novel belief space discretization technique that abstracts a continuous-space dynamics model for ET estimation to a finite Markov decision process. We show this abstraction method is more efficient than alternatives and also benefits from interpretability. Then, we leverage off-the-shelf tools to compute the set of all optimal trade-offs between resource consumption and task performance. Given a choice from this set, we synthesize an ET-threshold strategy that achieves the desirable behavior. Simulated results show our approach identifies non-trivial trade-offs between performance and energy savings, with only modest computational effort.

Keywords: Formal Verification/Synthesis, Optimal control, Autonomous systems
Abstract: We introduce a metric that can quantify the temporal relaxation of Signal Temporal Logic (STL) specifications and facilitate resilient control synthesis in the face of infeasibilities. The proposed metric quantifies a cumulative notion of relaxation among the subtasks, and minimizing it yields to structural changes in the original STL specification by i) modifying time-intervals, ii) removing subtasks entirely if needed. To this end, we formulate an optimal control problem that extracts state and input sequences by minimally violating the temporal requirements while achieving the desired predicates. We encode this problem in the form of a computationally efficient mixed-integer program. We show some theoretical results on the properties of the new metric. Finally, we present a case study of a robot that minimally violates the time constraints of desired tasks in the face of an infeasibility.

Keywords: Formal Verification/Synthesis, Neural networks, Uncertain systems
Abstract: The increasing prevalence of neural networks (NNs) in safety-critical applications calls for methods to certify their behavior and guarantee safety. This paper presents a backward reachability approach for safety verification of neural feedback loops (NFLs), i.e., closed-loop systems with NN control policies. While recent works have focused on forward reachability as a strategy for safety certification of NFLs, backward reachability offers advantages over the forward strategy, particularly in obstacle avoidance scenarios. Prior works have developed techniques for backward reachability analysis for systems without NNs, but the presence of NNs in the feedback loop presents a unique set of problems due to the nonlinearities in their activation functions and because NN models are generally not invertible. To overcome these challenges, we use existing forward NN analysis tools to find affine bounds on the control inputs and solve a series of linear programs (LPs) to efficiently find an approximation of the backprojection (BP) set, i.e., the set of states for which the NN control policy will drive the system to a given target set. We present an algorithm to iteratively find BP set estimates over a given time horizon and demonstrate the ability to reduce conservativeness in the BP set estimates by up to 88% with some extra computational cost. We use numerical results from a double integrator model to verify the efficacy of these algorithms and demonstrate the ability to certify safety for a linearized ground robot model in a collision avoidance scenario where forward reachability fails.

Keywords: Formal Verification/Synthesis, Neural networks
Abstract: In this paper, we introduce NNSynth, a new framework that uses machine learning techniques to guide the design of abstraction-based controllers with correctness guarantees. NNSynth utilizes neural networks (NNs) to guide the search over the space of controllers. The trained neural networks are ``projected'' and used for constructing a ``local'' abstraction of the system. An abstraction-based controller is then synthesized from such ``local'' abstractions. If a controller that satisfies the specifications is not found, then the best found controller is ``lifted'' to a neural network for further training. Our experiments show that this neural network-guided synthesis leads to more than 50x or even 100x speedup in high dimensional systems compared to the state-of-the-art.

Keywords: Networked control systems, Cyber-Physical Security, Embedded systems
Abstract: Fully Homomorphic Encryption (FHE) has made it possible to perform addition and multiplication operations on encrypted data. Using FHE in control thus has the advantage that control effort for a plant can be calculated remotely without ever decrypting the exchanged information. FHE in its current form is however not practically applicable for real-time control as its computational load is very high compared to traditional encryption methods. In this paper a reformulation of the Gentry FHE scheme is proposed and applied on an FPGA to solve this problem. It is shown that the resulting FHE scheme can be implemented for real-time stabilization of an inverted double pendulum using discrete time control.

Keywords: Networked control systems, Control over communications, Cooperative control
Abstract: We use the homomorphic properties of the Paillier cryptosystem to implement decoupling control over a wireless network without disclosing any information of a control agents state to neighboring agents. The results from a hardware in the loop implementation reveal the primary bottlenecks regarding the computational load on the local agents. Nevertheless they demonstrate a basic usability of Micropython for encrypted control with viable key sizes.

Keywords: Networked control systems, Control over communications, Estimation
Abstract: The problem of state estimation based on information received over a finite bit-rate channel gives rise to the study of minimal bit rate above which state can be estimated with any desired accuracy. In the past few years, researchers have studied the minimal average bit rate which is sufficient enough for state estimation such that the estimation error stays within a given factor of its initial value. The notion of restoration entropy characterizes this type of bit rate. Recent results proposed numerical schemes to estimate restoration entropy by the computation of singular values of the linearized systems. Such schemes are either complex to implement or suffer severely from computational complexity and the size of the state dimension. In this paper, we describe a modular approach to compute an upper bound of the restoration entropy of a large network by decomposing the network to an interconnection of smaller subsystems. Then, we formulate a distributed optimization problem which is solved for each subsystem separately and then their optimization results are composed to get an upper bound of the restoration entropy for the overall network. We illustrate the effectiveness of our results by two examples.

Keywords: Networked control systems, Communication networks, Linear systems
Abstract: In networked environments, adversaries may exploit location information to perform carefully crafted attacks on cyber-physical systems. To prevent such security breaches, this letter develops a network localization and navigation (NLN) paradigm that accounts for network secrecy in the control of mobile agents. We consider a scenario in which a mobile agent is tasked with maneuvering through an adversarial network, based on a nominal control policy, and we aim to reduce the ability of the adversarial network to infer the mobile agent’s position. Specifically, the Fisher information of the agent’s position obtained by the adversarial network is adopted as a secrecy metric. We propose a new control policy that results from an optimization problem and achieves a compromise between maximizing location secrecy and minimizing the deviation from the nominal control policy. Results show that the proposed optimization-based control policy significantly improves the secrecy of the mobile agent.

Keywords: Networked control systems, Cyber-Physical Security, Estimation
Abstract: This note studies state estimation in wireless networked control systems with secrecy against eavesdropping. Specifically, a sensor transmits a system state information to the estimator over a legitimate user link, and an eavesdropper overhears these data over its link independent of the user link. Each connection may be affected by packet losses and is modeled by a finite-state Markov channel (FSMC), an abstraction widely used to design wireless communication systems. This paper presents a novel concept of optimal mean square expected secrecy over FSMCs and delineates the design of a secrecy parameter requiring the user mean square estimation error (MSE) to be bounded and eavesdropper MSE unbounded. We illustrate the developed results on an example of an inverted pendulum on a cart whose parameters are estimated remotely over a wireless link exposed to an eavesdropper.

Keywords: Networked control systems, Cyber-Physical Security, Optimization
Abstract: The data-driven attack strategies recently have received much attention when the full knowledge of the system model is unknown or difficult to be obtained for the adversary. Note that despite the critical parameters of the system model being unavailable for the adversary, the existing data-driven attack methods still depend on the linearity of the unknown system model. In this paper, we design a completely model-free online attack strategy where the adversary with limited capability aims to compromise state variables such that the output value follows the expected trajectory. Specifically, we first construct a zeroth-order feedback optimization framework and uninterruptedly use probing signals for real-time measurements. Then, we iteratively update the attack signals along the composite direction of the gradient estimates of the objective function evaluations and the projected gradients. These objective function evaluations can be obtained only by real-time measurements. Furthermore, we characterize the optimality of these solutions via the optimality gap, which is affected by the dimensions of the attack signal, the iterations of solutions, and the convergence rate of the system. Extensive simulations are conducted to show the effectiveness of the proposed attack strategy.

Keywords: Stochastic optimal control, Optimal control, Markov processes
Abstract: This paper aims to provide a Dynamic Programming (DP) approach to solve the Mission-Wide Chance-Constrained Optimal Control Problems (MWCC-OCP). The mission-wide chance constraint guarantees that the probability that the entire state trajectory lies within a constraint/safe region is higher than a prescribed level, and is different from the stage-wise chance constraints imposed at individual time steps. The control objective is to find an optimal policy sequence that achieves both (i) satisfaction of a mission-wide chance constraint, and (ii) minimization of a cost function. By transforming the stage-wise chance-constrained problem into an unconstrained counterpart via Lagrangian method, standard DP can then be deployed. Yet, for MWCC-OCP, this methods fails to apply, because the mission-wide chance constraint cannot be easily formulated using stage-wise chance constraints due to the time-correlation between the latter (individual states are coupled through the system dynamics). To fill this gap, firstly, we detail the conditions required for a classical DP solution to exist for this type of problem; secondly, we propose a DP solution to the MWCC-OCP through state augmentation by introducing an additional functional state variable.

Keywords: Stochastic optimal control, Stochastic systems, Optimal control
Abstract: We formulate and solve the problem of finite horizon minimum control effort steering of the state probability distribution between prescribed endpoint joints for a finite population of networked noisy nonuniform Kuramoto oscillators. We consider both the first and second order stochastic Kuramoto models. For numerical solution of the associated stochastic optimal control, we propose combining certain measure-valued proximal recursions and the Feynman-Kac path integral computation. We illustrate the proposed framework via numerical examples.

Keywords: Stochastic optimal control, Stochastic systems, Linear systems
Abstract: After introducing a risk-aware stability definition, this paper investigates the problems of the event- and self-triggered control for discrete-time linear stochastic systems in the view of risk. In particular, the risk is quantified by the worst-case Conditional Value-at-Risk (CVaR) using the first two moments of the uncertainty probability distributions. This allows us to design triggering mechanisms considering the risk that the estimated worst-case tail behaviors of the system are worse than a given threshold. Numerical examples are included to illustrate the performances of the proposed approaches.

Keywords: Stochastic optimal control, Stochastic systems, Linear systems
Abstract: We propose a policy iteration algorithm for solving the multiplicative noise linear quadratic output feedback design problem. The algorithm solves a set of coupled Riccati equations for estimation and control arising from a partially observable Markov decision process (POMDP) under a class of linear dynamic control policies. We show in numerical experiments far faster convergence than a value iteration algorithm, formerly the only known algorithm for solving this class of problem. The results suggest promising future research directions for policy optimization algorithms in more general POMDPs, including the potential to develop novel approximate data-driven approaches when model parameters are not available.

Keywords: Stochastic optimal control, Optimal control, Stochastic systems
Abstract: In contrast to the classical concept of a Carnot engine that alternates contact between heat baths of different temperatures, naturally occurring processes usually harvest energy from anisotropy, being exposed simultaneously to chemical and thermal fluctuations of different intensities. In these cases, the enabling mechanism responsible for transduction of energy is the presence of a non-equilibrium steady state (NESS). A suitable stochastic model for such a phenomenon is the Brownian gyrator -- a two-degree of freedom stochastically driven system that exchanges energy and heat with the environment. In this context, we present a geometric view of the energy harvesting mechanism, from a stochastic control perspective, that entails a forced periodic trajectory of the system state on the thermodynamic manifold. Dissipation and work output are expressed accordingly as path integrals of a controlled process, and fundamental limitations on power and efficiency are expressed in geometric terms via a relationship to an isoperimetric problem. The theory is presented for high-order systems far from equilibrium and beyond the linear response regime.

Keywords: Stochastic optimal control, Optimization algorithms, Constrained control
Abstract: This paper focuses on the pathologies of common gradient-based algorithms for solving optimization problems under probability constraints. These problems are both important and difficult to handle. We provide a thorough analysis of what we believe is the main pathology of the distributional formulation of the problem, using both examples and rigorous proofs. With this knowledge, we then present an alternative formulation that we call the quantile formulation of the problem and prove that this formulation is exempt of the numerical pathologies of the original formulation. Our theoretical framework is developed for univariate distributions. To complete the work, we present generalizations to a class of multivariate problems motivated by the design of a robotic medical device to guide micro-surgeries, which must satisfy a specific structural reliability

Keywords: Robust control
Abstract: In this paper, we give an upper bound on the deviation from H-infinity optimality of a class of controllers as a function of the deviation from symmetry in the state matrix. We further suggest a scalar measure of symmetry which is shown to be directly relevant for estimating nearness to optimality. In connection to this, we give a simple analytical solution to a class of Lyapunov equations for two dimensional state matrices. Finally, we demonstrate how a well-chosen symmetric part for nearly symmetric state matrices may lead not only to near-optimality, but also to controller sparsity, a desirable property for large-scale systems.

In the special case that the state matrix is symmetric and Hurwitz, our main result simplifies to give an H-infinity optimal controller with several benefits, a result which has recently appeared in the literature. In this sense, the above is a significant generalization which considers a much wider class of systems, yet allows one to retain the benefits of symmetric state matrices, while offering means of quantifying the effect of this on the H-infinity norm.

Keywords: Robust control, Uncertain systems, Output regulation
Abstract: Robust output-feedback with active disturbance rejection control (ADRC) proved to be a very effective control paradigm for highly uncertain perturbed systems. The main practical limitation of the ADRC results from an application of a conventional high-gain observer which may cause significant amplification of a high-frequency noise corrupting the output measurements. We show that this main limitation can be significantly reduced by augmenting a conventional ADRC system with an auxiliary on-line estimator of the closed-loop tracking error dynamics which contains key residual information, useful for improving a resultant tracking control performance in the augmented ADRC system. It is shown that the tracking control improvement, resulting from an application of the proposed augmentation mechanism, prevents an excessive amplification of feedback noises and keeps a control cost on an acceptable level, especially for systems with higher-order dynamics. The proposed concept has been illustrated by selected results of extensive numerical simulations.

Keywords: Robust control, Uncertain systems, Power systems
Abstract: This paper proposes a virtual resistance-based nonlinear control to stabilize and robustify the current layer of inverter-based resources, subject to the grid voltage disturbances, and the grid parameter uncertainties. A class of virtual resistances is proposed and analyzed using concepts from dissipative systems theory. Moreover, specific nonlinear virtual resistance-based controllers are derived, with their corresponding performance analytically bounded. The theoretical and simulation results show that the proposed nonlinear virtual resistance-based controllers significantly reduce the mathcal{L}_2 gain of the closed-loop error system to grid voltage variation and parametric uncertainties. Significant improvements of the transient and steady-state current responses are also demonstrated over linear virtual resistance counterparts.

Keywords: Robust control, Uncertain systems, Stochastic optimal control
Abstract: This paper considers a class of discrete time, linear, stochastic uncertain systems defined in terms of a nominal Gaussian state-space model; the uncertainty is described by a relative entropy tolerance for each time increment of the dynamic model. For this class of systems, a problem of worstcase robust performance analysis with respect to a quadratic cost functional is solved. The solution takes the form of a risk-sensitive cost with a time-varying risk-sensitive parameter. Finally, a numerical example is presented to illustrate the methodology.

Keywords: Robust adaptive control, Fault tolerant systems, Learning
Abstract: Multiple model adaptive control (MMAC) is an adaptive control method designed for plant parameter uncertainty given both linear and non-linear plant models. For a system subject to varying operating conditions, the number of controllers necessary to guarantee stable control under nominal-plant uncertainty, or under multiple operating conditions, are both unknown. We propose a learning-based controller synthesis approach that can guarantee stability of a system subject to varying operating conditions. We adopt a convex hull (CH)-based multiple-model controller estimation algorithm that only requires N+M+1 controllers, where N is the dimension of a compact nominal uncertainty parameter set, and M is the number of additive faults. We empirically validate this result for a quadcopter, which is subject to faults in rotors and sensors as well as to adverse wind conditions.

Keywords: Robust adaptive control, Variable-structure/sliding-mode control
Abstract: This paper propose an adaptive time-varying approach for stabilization for a class of perturbed chains of integrators with unknown control coefficient. To drive the trajectories initiating far from the origin, a time-varying feedback with adaptation is used, ensuring that the closed-loop system’s solutions reach a prescribed vicinity of the origin within a finite-time, which is upper bounded by a prescribed time moment. Once the prescribed neighborhood is reached, a barrier function-based controller guarantees ultimate boundedness of the system’s solutions despite the presence of perturbations with unknown upper bound. Numerical simulations on the trajectory tracking problem of a 1-DOF torsional system and a stabilization problem of a third-order academic example show the effectiveness of the approach.

Keywords: Discrete event systems, Supervisory control, Cyber-Physical Security
Abstract: This tutorial paper studies the effect of deception attacks on compromised sensors and actuators at the supervisory control layer of cyber-physical control systems. The problem is modeled and analyzed in the framework of the theories of state estimation, diagnosability, and supervisory control of discrete event systems, where discrete transition models are used. Both attacks and defense against attacks are considered. First, robust estimation and diagnosis in the presence of sensor attacks is analyzed. Next, the problem of synthesizing covert attacks is formulated and its solution is discussed in different contexts. Then, necessary and sufficient conditions on the existence of resilient supervisors are presented in the context of a general attack model. Finally, the problem of synthesizing supervisors that are resilient to covert attacks on sensors and actuators is studied. This paper accompanies the tutorial session on this topic presented at the IEEE CDC 2022.

Keywords: Discrete event systems, Automata, Fault diagnosis
Abstract: The talk starts by discussing recursive algorithms for state estimation and event inference, both of which are key tasks for monitoring and control of discrete event systems. We also discuss various pertinent properties of interest and, in particular, diagnosability (i.e., the ability to detect within finite time the occurrence/type of a fault). We then analyze how such state estimation and event inference tasks might be compromised when a malicious attacker utilizes its knowledge of the underlying system model and its tampering capabilities (such as deletions, insertions, or substitutions of observed symbols) to influence the sequence of observations generated by the system. Assuming that each tampering action by the attacker is associated with a positive cost, we discuss approaches to describe matching sequences of observations, as well as to systematically obtain corresponding (``robustified") state estimates. We also develop techniques for verifying tamper-tolerant diagnosability under constraints on the total number of deletions, insertions, and substitutions (or, more generally, the total budget) that the attacker has at its disposal.

Keywords: Discrete event systems, Supervisory control, Cyber-Physical Security
Abstract: We start with a brief review of relevant results from the theory of supervisory control of discrete event systems. We then describe modeling techniques that account for potential deception attacks on sensors in the feedback loop of supervisory control. Next, we consider the point of view of the attacker. How can the attacker inflict damage to the system without being detected? Finally, we outline a methodology for synthesizing person-in-the-middle attacks on communication protocols.

Keywords: Discrete event systems, Supervisory control, Cyber-Physical Security
Abstract: We provide an intuitive introduction to a few recent theories about resilient supervisory control against stealthy attacks. By analyzing the closed-loop control mechanism in the Ramadge-Wonham (RW) supervisory control paradigm, we explain how a stealthy attack may exploit weakness of the RW paradigm. Upon such intuition, we introduce two defense strategies against stealthy sensor attacks and stealthy actuator attacks, respectively. We first identify the minimum ``risk'' information in a given plant model that ensures the existence of a stealthy sensor attack. Then we explain how to eliminate such risk information from the plant model, upon which we show that the existence of a resilient supervisor against all possible stealthy sensor attacks is decidable under some mild assumptions on the plant model. Considering the decidability of the existence of a resilient supervisor against all possible stealthy actuator attacks is still open, we introduce a heuristic supervisor obfuscation strategy, which relies on the fact that, given a plant and a supervisor, the existence of a stealthy actuator attack is decidable. We explain how to obfuscate control information in the supervisor via supervisor reduction so that a resulting control-equivalent obfuscated supervisor may become resilient against all stealthy actuator attacks. Simple examples are provided to illustrate introduced ideas.

Keywords: Predictive control for nonlinear systems, Robust control, Optimal control
Abstract: We propose a robust tube-based Model Predictive Control (MPC) paradigm for nonlinear systems whose dynamics can be expressed as a difference of convex functions. The approach exploits the convexity properties of the system model to derive convex conditions that govern the evolution of robust tubes bounding predicted trajectories. These tubes allow an upper bound on a performance cost to be minimised subject to state and control constraints as a convex program, the solution of which can be used to update an estimate of the optimal state and control trajectories. This process is the basis of an iteration that solves a sequence of convex programs at each discrete time step. We show that the algorithm is recursively feasible, converges asymptotically to a fixed point of the iteration and ensures closed loop stability. The algorithm can be terminated after any number of iterations without affecting stability or constraint satisfaction. A case study is presented to illustrate an application of the algorithm.

Keywords: Predictive control for nonlinear systems, Uncertain systems
Abstract: In real-world applications, we often require reliable decision making under dynamics uncertainties using noisy high-dimensional sensory data. Recently, we have seen an increasing number of learning-based control algorithms developed to address the challenge of decision making under dynamics uncertainties. These algorithms often make assumptions about the underlying unknown dynamics and, as a result, can provide safety guarantees. This is more challenging for other widely used learning-based decision making algorithms such as reinforcement learning. Furthermore, the majority of existing approaches assume access to state measurements, which can be restrictive in practice. In this paper, inspired by the literature on safety filters and robust output-feedback control, we present a robust predictive output-feedback safety filter (RPOF-SF) framework that provides safety certification to an arbitrary controller applied to an uncertain nonlinear control system. The proposed RPOF-SF combines a robustly stable observer that estimates the system state from noisy measurement data and a predictive safety filter that renders an arbitrary controller safe by (possibly) minimally modifying the controller input to guarantee safety. We show in theory that the proposed RPOF-SF guarantees constraint satisfaction despite disturbances applied to the system. We demonstrate the efficacy of the proposed RPOF-SF algorithm using an uncertain mass-spring-damper system.

Keywords: Process Control, Predictive control for nonlinear systems, Iterative learning control
Abstract: This paper presents an iterative way of computing a control algorithm with the aim of enabling reference tracking for an unknown nonlinear system. The method consists of three blocks: iterative learning control (ILC), robust model predictive control (MPC), and a linear approximation of the Koopman operator. The method proceeds in iterations, where at the end of an iteration, two steps are performed. First, the trajectories of the system obtained from previous iterations are used to build the linear approximation of the Koopman operator. Second, the linear model is used to compute the ILC signal. While these steps are executed in an offline manner, during an iteration, the control actions are computed online using the robust tube-based MPC. The tubes are defined by constraint tightening sets that compensate for the discrepancy between the true dynamics and its linear approximation. We demonstrate our method on the reference tracking for a 4 tank system.

Keywords: Predictive control for nonlinear systems, Predictive control for linear systems, Machine learning
Abstract: Machine learning techniques such as neural networks bear the potential to improve the performance and applicability of model predictive control to real-world systems. However, they also bear the danger of erratic-unpredictable behavior and malfunctioning of machine learning approaches. Neural networks might fail to predict system behavior as it is often impossible to provide strict performance or uncertainty bounds. While this challenge can be tackled using robust model predictive control approaches that span a safety net around the machine learning supported predictions, this can lead to significant performance degradation and infeasibility. To tackle this challenge, a safe neural network-supported learning tube model predictive control scheme is proposed, which allows to bound the worst-case performance in case of a malfunctioning of the machine learning component, yet enables decreasing the conservatism. The basic idea is to enforce the neural network to stay in the vicinity of a presumed given nominal model with the error dynamics directly incorporated into the neural network output function. Therefore, the error dynamics do not require an additional control input, resulting in the omission of input constraint tightening. Constraint fulfillment is guaranteed, robust set stability for a particular learning function class is established, and an upper bound for the performance of a malfunctioning neural network is given. The method is evaluated in simulations considering a rover operating in an uncertain environment.

Keywords: Predictive control for nonlinear systems, Computational methods, Identification for control
Abstract: We present an analytical and computational framework using the theory of Koopman operator to design dual-mode model predictive control (MPC) for nonlinear control systems. Dual-mode MPC incorporates stability and performance constraints in the controller and remains a challenging problem to solve for systems involving nonlinear dynamics. We exploit the spectral properties of the Koopman operator to provide a systematic approach for the design of optimal control, guaranteeing the stability of the feedback control system. The optimal cost function is also used for characterizing the maximal invariant set, which serves as the terminal set in the design of finite-horizon MPC. The finite horizon MPC relies on lifting nonlinear system dynamics using Koopman eigenfunctions to provide a linear approach for the design of MPC with state and input constraints.

Keywords: Predictive control for nonlinear systems, Constrained control
Abstract: We study tracking control for nonlinear systems with known relative degree and stable internal dynamics by the recently introduced technique of Funnel MPC. The objective is to achieve the evolution of the tracking error within a prescribed performance funnel. We propose a novel stage cost for Funnel MPC, extending earlier designs to the case of arbitrary relative degree, and show that the control objective as well as initial and recursive feasibility are always achieved - without requiring any terminal conditions or a sufficiently long prediction horizon. We only impose an additional feasibility constraint in the optimal control problem.

Keywords: Aerospace, Mechatronics, Robotics
Abstract: This work addresses a fully-actuated Hexa-rotor's control under exogenous position and attitude disturbances. For that aim, we propose a nonlinear and robust control based on a backstepping technique for position dynamics and a geometric control for attitude dynamics. We demonstrate that the closed-loop system is globally exponentially stable. To validate the reached stability, we present simulation results using Matlab/Simulink and software in the loop simulations using the well-known PX4 firmware in a realistic scenario on a virtual environment in Gazebo. Finally, to ultimately demonstrate the effectiveness of our algorithm, we implement it on an own-developed fully-actuated Hexa-rotor platform. Our contributions can be summarized as 1) the design of a robust controller for the fully-actuated Hexa-rotor not presented in the literature, 2) the publication of the controller's code for the PX4 autopilot software, and 3) drone's construction and presentation of real flight experiments.

Keywords: Aerospace, Control applications, Simulation
Abstract: This study proposes a descent control system for a space lander with a large amount of fuel, to protect the lander from overturning when landing on microgravity objects. As sloshing is a major problem during descent, it is desirable to construct a control system that considers the sloshing dynamics. We derive a model of the lander considering fuel dynamics, determine the fuel-optimal trajectory for guidance, design a fuel state estimator for navigation, and design a tracking controller. The effectiveness of the proposed method is evaluated using a physical simulator. The proposed method achieves simultaneous stabilization of the lander and fuel, and is shown to be effective in preventing overturning.

Keywords: Aerospace, Computational methods, PID control
Abstract: The evolution of technology has made increasingly advantageous the introduction of Unmanned Aerial Systems (UASs) in various applications, especially by exploiting their ability for autonomous flight. This paper presents an innovative approach to simulating UAS maneuvers that integrates a Computational Fluid Dynamics (CFD) model and a closed-loop control algorithm for both position and attitude dynamics. We chose the Proportional-Integrative-Derivative (PID) controller for this preliminary research activity because of its simple implementation and widespread employment in commercial autopilot systems. The numerical simulation of the UAS aerodynamics allows for performing an accurate analysis in critical situations. These include, for example, ground effect or wind gusts scenarios, which require an enhanced propulsive model to capture the interaction between vehicle dynamics, aerodynamics, and environmental conditions. The coupled CFD/PID framework can be a virtual testing environment for UAS platforms. Here we report on its validation. The paper compares such an innovative in-the-loop CFD approach and a classical simplified propulsive model that adopts constant thrust and torque coefficients.

Keywords: Visual servo control, Flight control, Autonomous vehicles
Abstract: This paper presents the design of an image-based visual servoing developed under a nonsingular fast terminal adaptive sliding mode strategy. The approach considers a quadrotor UAV executing a dynamic target tracking against wind turbulence. The image projection of the target is extracted through a virtual camera method to establish an image features vector that allows controlling the position and heading of the rotorcraft through a nonsingular fast terminal adaptive sliding mode controller, which provides robustness against model uncertainties and bounded external disturbances; and chattering reduction due to the nonoverestimation of its gain. Furthermore, a Lyapunov stability analysis guarantees the practical finite-time convergence of the closed-loop system. Finally, simulation results demonstrate the feasibility and advantages of the proposed visual-based control scheme.

Keywords: Power generation, Predictive control for nonlinear systems, Fluid flow systems
Abstract: Wind turbine (WT) controllers are often geared towards maximum power extraction, while suitable operating constraints should be guaranteed such that WT components are protected from failures. Control strategies can be also devised to reduce the generated power, for instance to track a power reference provided by the grid operator. They are called down- regulation strategies and allow to balance power generation and grid loads, as well as to provide ancillary grid services, such as frequency regulation. Although this balance is limited by the wind availability and grid demand, the quality of wind energy can be improved by introducing down-regulation strategies that make use of the kinetic energy of the turbine dynamics. This paper shows how the kinetic energy in the rotating components of turbines can be used as an additional degree-of-freedom by different down-regulation strategies. In particular we explore the power tracking problem based on convex model predictive control (MPC) at a single wind turbine. The use of MPC allows us to introduce a further constraint that guarantees flow stability and avoids stall conditions. Simulation results are used to illustrate the performance of the developed down- regulation strategies. Notably, by maximizing rotor speeds, and thus kinetic energy, the turbine can still temporarily guarantee tracking of a given power reference even when occasional saturation of the available wind power occurs. In the study case we proved that our approach can guarantee power tracking in saturated conditions for 10 times longer than with traditional down-regulation strategies.

Keywords: Power systems, Smart grid
Abstract: The adoption of distributed energy resources such as photovoltaics (PVs) has increased dramatically during the previous decade. The increased penetration of PVs into distribution networks (DNs) can cause voltage fluctuations that have to be mitigated. One of the key utility assets employed to this end are step-voltage regulators (SVRs). It is desirable to include tap selection of SVRs in optimal power flow (OPF) routines, a task that turns out to be challenging because the resultant OPF problem is nonconvex with added complexities stemming from accurate SVR modeling. While several convex relaxations based on semi-definite programming (SDP) have been presented in the literature for optimal tap selection, SDP based schemes do not scale well and are challenging to implement in large-scale planning or operational frameworks. This paper deals with the optimal tap selection (OPTS) problem for wye-connected SVRs using linear approximations of power flow equations. Specifically, the LinDist3Flow model is adopted and the effective SVR ratio is assumed to be continuous–enabling the formulation of a problem called LinDist3Flow-OPTS, which amounts to a linear program. The scalability and optimality gap of LinDist3Flow-OPTS are evaluated with respect to existing SDP-based and nonlinear programming techniques for optimal tap selection in three standard feeders, namely, the IEEE 13-bus, 123-bus, and 8500-node DNs. For all DNs considered, LinDist3Flow-OPTS achieves an optimality gap of approximately 1% or less while significantly lowering the computational burden.

Keywords: Game theory, Large-scale systems, Network analysis and control
Abstract: IIn this letter, we deal with evolutionary game-theoretic learning processes for population games on networks with dynamically evolving communities. Specifically, we propose a novel mathematical framework in which a deterministic, continuous-time replicator equation on a community network is coupled with a closed dynamic flow process between communities, in turn governed by an environmental feedback mechanism. When such a mechanism is independent of the game-theoretic learning process, a closed-loop system of differential equations is obtained. Through a direct analysis of the system, we study its asymptotic behavior. Specifically, we prove that, if the learning process converges, it converges to a (possibly restricted) Nash equilibrium of the game, even when the dynamic flow process does not converge. Moreover, for a class of population games —two-strategy matrix games— a Lyapunov argument is employed to establish an evolutionary folk theorem that guarantees convergence to a subset of Nash equilibria, that is, the evolutionary stable states of the game. Numerical simulations are provided to illustrate and corroborate our findings.

Keywords: Game theory, Learning, Optimization algorithms
Abstract: This paper investigates the impact of feedback quantization on multi-agent learning. In particular, we analyze the equilibrium convergence properties of the well-known “follow the regularized leader” (FTRL) class of algorithms when players can only observe a quantized (and possibly noisy) version of their payoffs. In this information-constrained setting, we show that coarser quantization triggers a qualitative shift in the convergence behavior of FTRL schemes. Specifically, if the quantization error lies below a threshold value (which depends only on the underlying game and not on the level of uncertainty entering the process or the specific FTRL variant under study), then (i) FTRL is attracted to the game’s strict Nash equilibria with arbitrarily high probability; and (ii) the algorithm’s asymptotic rate of convergence remains the same as in the non-quantized case. Otherwise, for larger quantization levels, these convergence properties are lost altogether: players may fail to learn anything beyond their initial state, even with full information on their payoff vectors. This is in contrast to the impact of quantization in continuous optimization problems, where the quality of the obtained solution degrades smoothly with the quantization level.

Keywords: Game theory, Learning, Control of networks
Abstract: Social learning refers to the process by which networked strategic agents learn an unknown state of the world by observing state-related private signals as well as other agents' actions. In the classic study of social learning by Bikhchandani, Hirshleifer, and Welch, it was shown that in this setting, information cascades occur, in which agents blindly imitate others' behavior and as a result learning stops for the whole community. Several proposals have been forwarded to mitigate this detrimental phenomenon.

In this paper we consider the introduction of an information coordinator to mitigate information cascades. The coordinator commits to a contract and agents choose to enter the mechanism or not. If they enter they pay a fee and inform the coordinator of their private information (not necessarily truthfully). The coordinator, in turn, suggests an action to the agents based on his cumulative knowledge. We study a class of mechanisms that possess properties such as individual rationality for agents (i.e., agents want to enter the mechanism), truth telling, and profit maximization for the coordinator. We show the existence of such a mechanism which strictly improves social welfare, and results in strictly positive profit for the coordinator, so that agents and the coordinator are willing to adopt this approach. Furthermore, we analyze the performance of this mechanism and show significant gains on both aforementioned metrics.

Keywords: Game theory, Network analysis and control, Simulation
Abstract: We consider the diffusion of innovation in social networks using a game-theoretic approach. In our model, each individual plays a coordination game with its neighbors and decides what alternative product to adopt to maximize its payoff. As products are used in conjunction with others and through repeated interactions, individuals are more interested in their long-term benefits and often tend to show trustworthiness to others in order to maximize their long-term payoffs. To capture such trustworthy behavior, we deviate from the expected utility theory and use a new notion of rationality based on the limited-trust equilibrium. By incorporating such rational behavior into the product diffusion model, we analyze the convergence of emerging dynamics to their equilibrium points using a mean-field approximation. In particular, we study the convergence rate of the diffusion process using the absorption probability and the expected absorption time of a reduced-size Markov chain. Simulation shows that when individuals behave trustworthily, their long-term payoffs will increase significantly compared to the case when they are solely self-interested.

Keywords: Game theory, Optimization algorithms, Network analysis and control
Abstract: We consider decentralized learning dynamics for agents in an anti-coordination network game. In the anti-coordination network game, there is a preferred action in the absence of neighbors' actions, and the utility an agent receives from the preferred action decreases as more of its neighbors select the preferred action, potentially causing the agent to select a less desirable action. The decentralized dynamics that is based on the iterated elimination of dominated strategies converge for the considered game. Given a convergent action profile, we measure anti-coordination by the number of edges in the underlying graph that have at least one agent in either end of the edge not taking the preferred action. The maximum anti-coordination (MAC) problem seeks to find an optimal set of agents to control under a finite budget so that the overall network disconnect is maximized on game convergence as a result of the dynamics. In this paper we show that the MAC is approximately submodular in line networks for any realization of the utility constants in the population. Utilizing this result, we provide a performance guarantee for the greedy agent selection algorithm for MAC. Finally, we use a computational study to show the effectiveness of greedy node selection strategies to solve MAC on general bipartite networks.

Keywords: Game theory, Network analysis and control, Agents-based systems
Abstract: We consider linear-quadratic-Gaussian (LQG) network games in which agents have quadratic payoffs that depend on their individual and neighbors' actions, and an unknown payoff-relevant state. An information designer determines the fidelity of information revealed to the agents about the payoff state to maximize the social welfare. Prior results show that full information disclosure is optimal under certain assumptions on the payoffs, i.e., it is beneficial for the average individual. In this paper, we provide conditions for general network structures based on the strength of the dependence of payoffs on neighbors' actions, i.e., competition, under which a rational agent is expected to benefit, i.e., receive higher payoffs, from full information disclosure. We find that all agents benefit from information disclosure for the star network structure when the game is homogeneous. We also identify that the central agent benefits more than a peripheral agent from full information disclosure unless the competition is strong and the number of peripheral agents is small enough. Despite the fact that all agents expect to benefit from information disclosure ex-ante, a central agent can be worse-off from information disclosure in many realizations of the payoff state under strong competition, indicating that a risk-averse central agent can prefer uninformative signals ex-ante.