Keywords: Observers for Linear systems
Abstract: A typical multi-agent system is composed of a follower system consisting of multiple subsystems and a leader system whose output is to be tracked by the output of each subsystem of the follower. What makes the control of a multi-agent system interesting is that the control law has to be distributed in that it must satisfy time-varying communication constraints. In the special case where all the subsystems of the follower can access the full information of the leader, one can design, for each follower subsystem, a conventional control law based on the information of the leader and this follower subsystem. The collection of these conventional control laws constitutes the so-called purely decentralized control law for the multi-agent system. Nevertheless, the purely decentralized control is in general not feasible as it violates the communication constraints. In this talk, we will elucidate a framework for designing a distributed control law by cascading a purely decentralized control law and a so-called distributed observer for the leader system, which is a distributed dynamic compensator capable of estimating the information of the leader and transmitting the estimated information to each follower subsystem over the communication network. The core of this design framework is the distributed observer for the leader system, which was initiated in 2010 and has experienced three phases of developments. In the first phase, the distributed observer only aimed at estimating the state of the leader assuming that every follower subsystem knows the dynamics of the leader. In the second phase, which started in 2015, the distributed observer was rendered the capability of estimating not only the state but also the dynamics of the leader assuming that only the children of the leader know the information of the leader. The distributed observer was further enhanced in 2018 for linear leader systems containing unknown parameters, thus entering its third phase of the development. Such a distributed observer not only estimates the state but also the unknown parameters of the leader. The talk will offer an overview on the development of the three phases of the distributed observer together with its extensions and applications.

Keywords: Agents-based systems
Abstract: A multi-agent system should be capable of fast and flexible decision-making if it is to successfully manage the uncertainty, variability, and dynamic change encountered when operating in the real world. Decision-making is fast if it breaks indecision as quickly as indecision becomes costly. This requires fast divergence away from indecision in addition to fast convergence to a decision. Decision-making is flexible if it adapts to signals important to successful operations, even if they are weak or rare. This requires tunable sensitivity to input for modulating regimes in which the system is ultra-sensitive and in which the system is robust. Nonlinearity and feedback in the multi-agent decision-making dynamics are necessary to meet these requirements.

I will present theoretical principles, analytical results, and applications of a general model of decentralized, multi-agent, and multi-option, nonlinear opinion dynamics that enables fast and flexible decision-making. I will explain how the critical features of fast and flexible multi-agent decision-making depend on nonlinearity, feedback, and the structure of the inter-agent communication network and a belief system network. And I will show how the theory and results provide a principled and systematic means for designing and analyzing multi-agent decision-making in systems ranging from multi-robot teams to social networks.

Keywords: Stochastic systems, Game theory
Abstract: ct This talk will provide an introduction to the tutorial session by first recalling the basic elements and assumptions of the standard LQG model, and then identifying several variations around it. One of these variations is placement of memory restrictions on the controllers, which brings in a signaling element into controller design. Here linearity of optimal controller is retained for some classes of such problems (which will be identified), but for most it does not (also to be discussed). Another variation is failure of channels according to some probability law, yet another one is adversarial intrusion into transmission of information (which leads to a stochastic zero-sum game), and yet a third one is introduction of bandwidth limitations, which necessitates quantization of the signals transmitted. Optimal designs for all these variations will be presented. Also to be discussed is what is known as rational expectation models (arising particularly in economics) where the evolution of a decision process depends on future expectations of a decision-maker on that evolution, which can also be handled within the linear quadratic framework with Gaussian statistics, though quite different from the basic LQG framework. Yet another topic to be covered is incentive designs, again in the linear-quadratic framework, where now there are (at least) two decision-makers with different objective functions, where one has the capability to (partially) shape the objective of the other one toward his own benefit. The talk will conclude with variations in the direction of other types of stochastic dynamic games as a segue to the next talk in the session.

Keywords: Stochastic systems, Game theory, Mean field games
Abstract: Since the seminal paper of Fleming and Souganidis, stochastic differential games have been playing a central role in mathematical control theory, as they can be applied to model the general decision-making process between interacting players under stochastic uncertainties. Two different types of stochastic differential games can be formulated depending on the role of the interacting players. Specifically, when the interaction of the players can be described in a symmetric way, it is called the Nash differential game. On the other hand, the Stackelberg differential game can be used to formulate the nonsymmetric leader-follower hierarchical decision-making process between the players.

This talk consists of two parts, studying various recent results on LQ stochastic Nash and Stackelberg differential games. In the first part, the rigorous mathematical formulation on LQ stochastic Nash and Stackelberg differential games will be covered within various different frameworks, including systems with random-coefficients, games of mean-field type, Markov-jump systems, and systems with delay, where we will also provide several different notions of Nash and Stackelberg equilibria depending on the underlying information structures. In the second part, we will address the detailed mathematical approaches to and analyses of the LQ stochastic differential games formulated in the first part, and present their explicit Nash/Stackelberg equilibrium solutions expressed by Riccati differential equations. Some examples including numerical solvability of the corresponding Riccati differential equations will also be discussed to illustrate the theoretical results of this talk.

Keywords: Networked control systems, Optimal control
Abstract: The linear quadratic (LQ) control is the core of modern control theory, which has attracted great attention since the 1950s. However, there still exist some challenging fundamental problems that need to be addressed, which have also posed obstacles in areas of networked control systems (NCSs) and other related areas. For example, the stochastic LQR with time-delay has never been properly solved even for the most basic case of single input delay, although the delay-free case has been fully addressed by J. Bismut in the 70’s. As a result, control of NCSs faces fundamental difficulty in the cases of simultaneous packet loss and delay of control or state. Another problem is irregular LQ control, i. e., the related Riccati equation is irregular which leads to the LQ controller not being solvable even though it exists. This arises when the weighting matrix of control is semi-positive or indefinite. Irregular LQR has been a long-standing problem since the 70's.

In this talk, we will first identify the challenges in the LQ control; second, we will address these challenges by establishing the unity of LQ control with forward and backward differential/difference equations (FBDEs). Finally, by solving the FBDEs, we will present the solutions to various problems identified, including irregular LQ control, stochastic LQ control with delay, optimal control and stabilization in NCSs with asymmetric information and/or with simultaneous packet loss and delay.

(This talk is based on joint work of the speaker with Juanjuan Xu of Shandong University)

Keywords: Stochastic systems, Game theory
Abstract: For linear-quadratic (LQ, for short) optimal control problems, there are several notions closely related: Existence of optimal control, two-point boundary value problem (optimality system), and (differential/algebraic) Riccati equation. An impression that people have is that these three notions are roughly equivalent. However, it is too vague.

The purpose of this two-part talk is to clarify the relation among the above notions for stochastic LQ problems with deterministic coefficients. We introduce open-loop and closed-loop solvability of LQ problems. The following results will be presented: (i) The open-loop solvability is equivalent to solvability of the optimality system (which is now a forward-backward stochastic differential equation) plus the convexity of the cost functional; (ii) The closed-loop solvability is equivalent to the solvability of the Riccati equation; (iii) Open-loop and closed-loop solvability are not equivalent, in general, and they are equivalent for the infinite horizon LQ problems; (iv) There are corresponding results for two-person zero-sum differential games of LQ setting in terms of open-loop and closed-loop saddle points; (v) If the LQ optimal control problem has a closed-loop optimal strategy, and the problem also has an open-loop optimal control which admits a closed-loop representation, then the open-loop optimal control coincides with the outcome of the closed-loop optimal strategy; (vi) The conclusion of (v) continue to hold for open-loop and closed-loop saddle points of two-person zero-sum LQ differential games; However, it is not true, in general, for Nash equilibria of non-zero-sum differential games. This talk is divided into two parts: Part I will cover (i)—(iii), and Part II will cover (iv)—(vi).

(This talk is based on the joint work of the speaker with Jingrui Sun of Southern University of Science and Technology, China.)

Keywords: Optimal control, Machine learning, Computer-aided control design
Abstract: This paper develops a framework for differentiating sparse optimal control inputs with respect to cost parameters. The difficulty lies in the non-smoothness induced by a sparsity-enhancing regularizer. To avoid this, we identify the optimal inputs as a unique zero point of a function using the proximal technique. This enables us to characterize the differentiability and employ the implicit function theorem. We also demonstrate the effectiveness of our approach using a numerical example of inverse optimal control.

Keywords: Iterative learning control, Data driven control, Optimal control
Abstract: In this paper, an off-policy reinforcement learning algorithm is designed to solve the continuous-time linear quadratic regulator (LQR) problem using only input-state data measured from the system. Different from other algorithms in the literature, we propose the use of a specific persistently exciting input as the exploration signal during the data collection step. We then show that, using this persistently excited data, the solution of the matrix equation in our algorithm is guaranteed to exist and to be unique at every iteration. Convergence of the algorithm to the optimal control input is also proven. Moreover, we formulate the policy evaluation step as the solution of a Sylvester-transpose equation, which increases the efficiency of its solution. A method to determine an initial stabilizing policy using only measured data is proposed. Finally, the advantages of the proposed method are tested via simulation.

Keywords: Optimization
Abstract: Bayesian optimization (BO) has emerged as a data-efficient method for global optimization of expensive black-box functions, which commonly arise in learning-based control applications. Recent work has shown that BO can be augmented with gradient measurements to further improve its convergence behavior. These approaches mostly rely on standard acquisition functions and indirectly incorporate gradient information into a probabilistic surrogate model of the performance function to improve its local predictions. This paper presents a new strategy to simultaneously exploit performance (zeroth-order) and gradient (first-order) data within a single constrained acquisition optimization. This is done by enforcing a set of black-box constraints that mimic the necessary optimality conditions for the original global optimization problem. We establish how the incorporation of these constraints restricts the allowable search space of BO, leading to less exploration than zeroth-order BO. The performance of the proposed method is demonstrated for closed-loop policy search via reinforcement learning on a benchmark LQR problem.

Keywords: Uncertain systems, Randomized algorithms, Statistical learning
Abstract: The scenario approach, originally developed as a computational tool for robust problems, has through the years developed into a solid, general, framework for data-driven decision making and design. One main driving force that has fostered this process has certainly been the increasing generality of the considered schemes. In this paper, we move a further step forward in this process. By leveraging some recent results in the wake of the so-called wait-and-judge paradigm, we fully develop a scheme for scenario optimization with constraint relaxation in a non-convex setup, so greatly expanding previous achievements valid under a convexity assumption. We show that a purely data-driven, and yet tight and informative, quantification of the solution robustness is possible regardless of the mechanism through which uncertainty is generated. The generality of this new non-convex setup provides an extremely versatile scheme for data-driven design that can be applied to a variety of problems ranging from mixed-integer optimization to design in abstract spaces.

Keywords: Data driven control, Optimal control, Adaptive control
Abstract: In this paper, we address a data-driven linear quadratic optimal control problem in which the regulator design is performed on-policy by resorting to approaches from reinforcement learning and model reference adaptive control. In particular, a continuous-time identifier of the value function is used to generate online a reference model for the adaptive stabilizer. By introducing a suitably selected dithering signal, the resulting policy is shown to achieve asymptotic convergence to the optimal gain while the controlled plant reaches asymptotically the behavior of the optimal closed-loop system.

Keywords: Machine learning, Distributed control, Stochastic optimal control
Abstract: The theory and application of mean field games has grown significantly since its origins less than two decades ago. This paper considers a special class in which the game is cooperative, and the cost includes a control penalty defined by Kullback-Leibler divergence, as commonly used in reinforcement learning and other fields. Its use as a control cost or regularizer is often preferred because this leads to an attractive solution. This paper considers a particular control paradigm called Kullback-Leibler Quadratic (KLQ) optimal control, and arrives at the following conclusions: 1. in application to distributed control of electric loads, a new modeling technique is introduced to obtain a simple Markov model for each load (the `agent' in mean field theory). 2. It is argued that the optimality equations may be solved using Monte-Carlo techniques---a specialized version of stochastic gradient descent (SGD). 3. The use of averaging minimizes the asymptotic covariance in the SGD algorithm; the form of the optimal covariance is identified for the first time.

Keywords: Constrained control, Hybrid systems
Abstract: Koopman liftings have been successfully used to learn high dimensional linear approximations for autonomous systems for prediction purposes, or for control systems for leveraging linear control techniques to control nonlinear dynamics. In this paper, we show how learned Koopman approximations can be used for state-feedback correct-by-construction control. To this end, we introduce the Koopman over-approximation, a (possibly hybrid) lifted representation that has a simulation-like relation with the underlying dynamics. Then, we prove how successive application of controlled predecessor operation in the lifted space leads to an implicit backward reachable set for the actual dynamics. Finally, we demonstrate the approach on two nonlinear control examples with unknown dynamics.

Keywords: Uncertain systems, Optimal control, Predictive control for nonlinear systems
Abstract: We study the convex hulls of reachable sets of nonlinear systems with bounded disturbances. Reachable sets play a critical role in control, but remain notoriously challenging to compute, and existing over-approximation tools tend to be conservative or computationally expensive. In this work, we exactly characterize the convex hulls of reachable sets as the convex hulls of solutions of an ordinary differential equation from all possible initial values of the disturbances. This finite-dimensional characterization unlocks a fast sampling-based method to accurately over-approximate reachable sets. We give applications to neural feedback loop analysis and robust model predictive control.

Keywords: Formal Verification/Synthesis, Discrete event systems, Computational methods
Abstract: A logical zonotope, which is a new set representation for binary vectors, is introduced in this paper. A logical zonotope is constructed by XOR-ing a binary vector with a combination of other binary vectors called generators. Such a zonotope can represent up to 2^n binary vectors using only n generators. It is shown that logical operations over sets of binary vectors can be performed on the zonotopes' generators and, thus, significantly reduce the computational complexity of various logical operations (e.g., XOR, NAND, AND, OR, and semi-tensor products). Similar to traditional zonotopes' role in the formal verification of dynamical systems over real vector spaces, logical zonotopes can efficiently analyze discrete dynamical systems defined over binary vector spaces. We illustrate the approach and its ability to reduce the computational complexity in two use cases: (1) encryption key discovery of a linear feedback shift register and (2) safety verification of a road traffic intersection protocol.

Keywords: Machine learning, Neural networks, Robust control
Abstract: Neural networks (NNs) have been shown to learn complex control laws successfully, often with performance advantages or decreased computational cost compared to alternative methods. Neural network controllers (NNCs) are, however, highly sensitive to disturbances and uncertainty, meaning that it can be challenging to make satisfactory robustness guarantees for systems with these controllers. This problem is exacerbated when considering multi-agent NN-controlled systems, as existing reachability methods often scale poorly for large systems. This paper addresses the problem of finding overapproximations of forward reachable sets for discrete-time uncertain multi-agent systems with distributed NNC architectures. We first reformulate the dynamics, making the system more amenable to reachablility analysis. Next, we take advantage of the distributed architecture to split the overall reachability problem into smaller problems, significantly reducing computation time. We use quadratic constraints, along with a convex representation of uncertainty in each agent's model, to form semidefinite programs, the solutions of which give overapproximations of forward reachable sets for each agent. Finally, the methodology is tested on two realistic examples: a platoon of vehicles and a power network system.

Keywords: Optimization, Statistical learning
Abstract: We present a novel framework for distributionally robust optimization (DRO), called cost-aware DRO (CADRO). The key idea of CADRO is to exploit the cost structure in the design of the ambiguity set to reduce conservatism. Particularly, the set specifically constrains the worst-case distribution along the direction in which the expected cost of an approximate solution increases most rapidly. We prove that CADRO provides both a high-confidence upper bound and a consistent estimator of the out-of-sample expected cost, and show empirically that it produces solutions that are substantially less conservative than existing DRO methods, while providing the same guarantees.

Keywords: Randomized algorithms, Optimization algorithms, Nonlinear systems
Abstract: Efficient methods to provide sub-optimal solutions to non-convex optimization problems with knowledge of their sub-optimality have been long sought after. To that end, by leveraging recent work in risk-aware verification, we provide two algorithms to (1) probabilistically bound the optimality gaps of solutions reported by novel percentile optimization techniques, and (2) probabilistically bound the maximum optimality gap reported by percentile approaches for repetitive applications, e.g. Model Predictive Control (MPC). Notably, our results work for a large class of optimization problems. We showcase the efficacy and repeatability of our results on a few, benchmark non-convex optimization problems and the utility of our results for controls in a Nonlinear MPC setting.

Keywords: Attack Detection, Estimation, Transportation networks
Abstract: Connected vehicles have great advantages in driving safety and energy efficiency under the support of vehicle-to-everything (V2X) networks, while they are also vulnerable to malicious cyber-attacks. To enhance the cyber security of connected vehicles, a cyber-attack detection framework is proposed based on multi-source information fusion specifically for the vehicle localization system. In this framework, an iterative unbiased finite impulse response (UFIR) filter is utilized to estimate the vehicle position with low computational load, based on the vehicle dynamics model and information from the inertial measurement system (IMU), GPS, and V2X networks. In addition, a discriminator module is developed to analyze the residuals between estimations and position information from different sources for cyber-attack detection. Finally, multiple simulation cases are implemented to validate the effectiveness of the proposed framework.

Keywords: Traffic control, Optimal control, Machine learning
Abstract: Highway merging scenarios featuring mixed traffic conditions pose significant modeling and control challenges for connected and automated vehicles (CAVs) interacting with incoming on-ramp human-driven vehicles (HDVs). In this paper, we present an approach to learn an approximate information state (AIS) model of CAV-HDV interactions. Thus, the CAV learns the behavior of an incoming HDV using the AIS model and uses it to generate a control strategy for merging. First, we validate the efficacy of this framework on real-world data by using it to predict the behavior of an HDV in situations with other HDVs extracted from the Next-Generation Simulation repository. Then, we generate simulation data for HDV-CAV interactions in a highway merging scenario using a standard inverse reinforcement learning approach. Without assuming a prior knowledge of the generating model, we show that our AIS model learns to predict the future trajectory of the HDV using only observations. Subsequently, we generate safe merging control policies for a CAV when merging with HDVs that demonstrate a spectrum of driving behaviors, from aggressive to conservative. We establish the effectiveness of the proposed approach by performing numerical simulations.

Keywords: Traffic control, Control applications, Autonomous vehicles
Abstract: This work investigates traffic control via controlled connected and automated vehicles (CAVs) using novel controllers derived from the linear-quadratic regulator (LQR) theory. CAV-platoons are modeled as moving bottlenecks impacting the surrounding traffic with their speeds as control inputs. An iterative controller algorithm based on the LQR theory is proposed along with a variant that allows for penalizing abrupt changes in platoon speeds. The controllers use the Lighthill-Whitham-Richards (LWR) model implemented using an extended cell transmission model (CTM) which considers the capacity drop phenomenon for a realistic representation of traffic in congestion. The effectiveness of the proposed traffic control algorithms is tested using a traffic control example and compared with existing proportional-integral (PI)- and model predictive control (MPC)- based controllers from the literature. A case study using the TransModeler traffic microsimulation software is conducted to test the usability of the proposed controller in a realistic setting. It is observed that the proposed controller works well in both settings to mitigate the impact of the jam caused by a fixed bottleneck. The computation time required by the controller is also small making it suitable for real-time control.

Keywords: Traffic control, Optimal control
Abstract: This article studies the optimal control of autonomous vehicles over a given time horizon to smooth traffic. We model the dynamics of a mixed-autonomy platoon as a system of non-linear ODEs, where the acceleration of human-driven vehicles is governed by a car-following model, and the acceleration of autonomous vehicles is to be controlled. We formulate the car-following task as an optimal control problem and propose a computational method to solve it. Our approach uses an adjoint formulation to compute gradients of the optimization problem explicitly, resulting in more accurate and efficient numerical computations. The gradients are then used to solve the problem using gradient-based optimization solvers. We consider an instance of the problem with the objective of improving the fuel efficiency of the vehicles in the platoon. The effectiveness of the proposed approach is demonstrated through numerical experiments. We apply the proposed approach to different scenarios of lead vehicle trajectories and platoon sizes. The results suggest that introducing an AV can produce significant energy savings for the platoon. It also reveals that the solution is agnostic to the platoon size thus the fuel saving is mainly due to optimizing the trajectory of the AV.

Keywords: Agents-based systems, Automotive systems, Smart cities/houses
Abstract: Motivated by the ever growing use of location-based services, we present a decentralised class of algorithms called Tree-Proof-of-Position (T-PoP). Most of the current proofs of location are centralised, thus forgoing verifiablity and privacy for the users. Decentralised solutions also exist, but they suffer from drawbacks that make them unsuitable for realistic, adversarial use cases. T-PoP algorithms rely on the web of interconnected devices in a smart city to establish how likely it is that an agent is in their claimed position. T-PoP operates under adversarial assumptions, where some agents are incentivised to be dishonest. We present a theoretical model for T-PoP and its security properties, and we validate this model through a large number of Monte-Carlo simulations. We specifically focus on two instances of T-PoP and analyse their security and reliability properties under a range of adversarial conditions. Use-cases and applications are discussed towards the end of the paper.

Keywords: Traffic control, Game theory, Mean field games
Abstract: The morning commute bottleneck congestion problem has classically been modelled as a static game in which commuters act strategically based on their immediate Value of Time (VOT). This has restricted existing congestion mitigation techniques to rely on essentially monetary incentives to affect the static costs of the commuters. In contrast, a dynamic model enables characterizing the strategic trade-off between immediate and future resource access rights and inspires the design of new classes of fair, non-monetary congestion mitigation schemes. In this paper, we show how the recently proposed Dynamic Population Game (DPG) framework can be leveraged to study a non-monetary economy for bottleneck congestion management based on karma, a non-tradable mobility credit. Our DPG model allows to consider an elastic demand of commuters that only travel if congestion is reduced, and we show that a Stationary Nash Equilibrium (SNE) is guaranteed to exist despite of the dynamic participation of these commuters. Through numerical case studies we illustrate how our tools can assist policy makers in taking informed decisions about complex policy outcomes. In particular, we show how the dynamic karma scheme is robust to a potentially detrimental rebound effect that would manifest in a static monetary scheme.

Keywords: Optimization, Decentralized control, Machine learning
Abstract: Recently, decentralized optimization over the Stiefel manifold has attracted tremendous attentions due to its wide range of applications in various fields. Existing methods rely on the gradients to update variables, which are not applicable to the objective functions with non-smooth regularizers, such as sparse PCA. In this paper, to the best of our knowledge, we propose the first decentralized algorithm for non-smooth optimization over Stiefel manifolds. Our algorithm approximates the non-smooth part of objective function by its Moreau envelope, and then existing algorithms for smooth optimization can be deployed. We establish the convergence guarantee with the iteration complexity of O (epsilon^{-4}). Numerical experiments conducted under the decentralized setting demonstrate the effectiveness and efficiency of our algorithm.

Keywords: Optimization algorithms, Machine learning, Distributed control
Abstract: The stochastic Gradient Tracking (GT) method for distributed optimization, is known to be robust against the inter-client variance caused by data heterogeneity. However, the stochastic GT method can be communication-intensive, requiring a large number of communication rounds of message exchange for convergence. To address this challenge, this paper proposes a new communication-efficient stochastic GT algorithm called the Local Stochastic GT (LSGT) algorithm, which adopts the local stochastic gradient descent (local SGD) technique in the GT method. With LSGT, each agent can perform multiple SGD updates locally within each communication round. Although it is not known previously whether the stochastic GT method can benefit from the local SGD, we establish the conditions under which our proposed LSGT algorithm enjoys the linear speedup brought by local SGD. Compared with the existing work, our analysis requires less restrictive conditions on the mixing matrix and algorithm stepsize. Moreover, it reveals that the local SGD does not only reserve the resilience of the stochastic GT method against the data heterogeneity but also speeds up reducing the tracking error reduction in the optimization process. The experimental results demonstrate that the proposed LSGT exhibits improved convergence speed and robust performance in various heterogeneous environments.

Keywords: Optimization, Machine learning, Decentralized control
Abstract: Decentralized stochastic optimization has become a crucial tool for addressing large-scale machine learning and control problems. In decentralized algorithms, all computing nodes are connected through a network topology, and each node communicates only with its direct neighbors. Decentralized algorithms can significantly reduce communication overhead by eliminating the need for global communication. However, existing research on the linear speedup analysis of decentralized stochastic algorithms is limited to the condition of network-dependent learning rates, which rarely holds in practice since the network connectivity is typically unknown to each node. As a result, it remains an open question whether a linear speedup bound can be achieved using network-independent learning rates. This paper provides an affirmative answer. By utilizing a new analysis framework, we prove D-SGD and Exact-Diffusion, two representative decentralized stochastic algorithms, can achieve linear speedup with network-independent learning rates. Simulations are provided to validate our theories.

Keywords: Optimization algorithms, Communication networks, Machine learning
Abstract: This paper develops a fully stochastic proximal primal-dual (FSPPD) algorithm for distributed convex optimization. At each iteration, the distributed algorithm has agents communicating on a randomly drawn graph and applies random sparsification on the transmitted messages, while the agents only have access to a stochastic gradient oracle. To our best knowledge, this is the first compression-enabled distributed stochastic gradient algorithm on random graphs utilizing the primal-dual framework. With diminishing step size, we show that the FSPPD algorithm converges almost surely to an optimal solution of the strongly convex optimization problem. Numerical experiments are provided to verify our results.

Keywords: Optimization algorithms, Cyber-Physical Security, Machine learning
Abstract: We consider Byzantine-robust distributed learning with asynchronous participation of clients at a certain probability, where Byzantine clients can send malicious messages to the server. Instead of relying on traditional robust aggregation rules, such as Krum and Median, that can only tolerate a limited proportion of Byzantine clients, we propose an asynchronous Byzantine-robust stochastic aggregation method that employs regularization-based techniques to mitigate Byzantine attacks, and adopts variance-reduced techniques to eliminate the effect of stochastic noise of gradient sampling. Leveraging a properly designed Lyapunov function, we show that the proposed algorithm converges linearly to an error ball that is independent of stochastic gradient variance. Extensive experiments are conducted to show its efficacy in dealing with Byzantine attacks compared to the existing counterparts.

Keywords: Predictive control for linear systems, Optimization algorithms, Constrained control
Abstract: In this paper, we consider a Model Predictive Control (MPC) problem of a continuous-time linear time-invariant system subject to continuous-time path constraints on the states and the inputs. By leveraging the concept of differential flatness, we can replace the differential equations governing the system with linear mapping between the states, inputs, and flat outputs (including their derivatives). The flat outputs are then parameterized by piecewise polynomials, and the model predictive control problem can be equivalently transformed into a Semi-Definite Programming (SDP) problem via Sum-of-Squares (SOS), ensuring constraint satisfaction at every continuous-time interval. We further note that the SDP problem contains a large number of small-size semi-definite matrices as optimization variables. To address this, we develop a Primal-Dual Hybrid Gradient (PDHG) algorithm that can be efficiently parallelized to speed up the optimization procedure. Simulation results on a quadruple-tank process demonstrate that our formulation can guarantee strict constraint satisfaction, while the standard MPC controller based on the discretized system may violate the constraint inside a sampling period. Moreover, the computational speed superiority of our proposed algorithm is collaborated by numerical simulation.

Keywords: Estimation, Biologically-inspired methods, Observers for Linear systems
Abstract: The accurate estimation of the noise covariance matrix (NCM) in a dynamic system is critical for state estimation and control, as it has a major influence in their optimality. Although a large number of NCM estimation methods have been developed, most of them assume the noises to be white. However, in many real-world applications, the noises are colored (e.g., they exhibit temporal autocorrelations), resulting in suboptimal solutions. Here, we introduce a novel brain-inspired algorithm that accurately and adaptively estimates the NCM for dynamic systems subjected to colored noise. Particularly, we extend the Dynamic Expectation Maximization algorithm to perform both online noise covariance and state estimation by optimizing the free energy objective. We mathematically prove that our NCM estimator converges to the global optimum of this free energy objective. Using randomized numerical simulations, we show that our estimator outperforms nine baseline methods with minimal noise covariance estimation error under colored noise conditions. Notably, we show that our method outperforms the best baseline (Variational Bayes) in joint noise and state estimation for high colored noise. We foresee that the accuracy and the adaptive nature of our estimator make it suitable for online estimation in real-world applications.

Keywords: Estimation, Control applications, Robust control
Abstract: LiDAR is a standard sensor choice for self-localization and SLAM of indoor autonomous robots. While there are many methods to estimate a robot's location using LiDAR measurements, most rely on algorithms that solve a generic LiDAR scan matching problem. When safety is a concern, these algorithms must provide a bound on the localization error to enable safety enforcing controllers, such as those based on Control Barrier Functions. Unfortunately, most existing scan matching algorithms offer no formal guarantees, and are tailored to structured, high-resolution 3D point clouds.

In this paper, we present an improved theoretical analysis for a low-cost alternative to these methods named PASTA (Provably Accurate Simple Transformation Alignment), originally introduced in [8]. We provide a formal worst-case error guarantee on the localization error and show, experimentally, that it is tight. This characterization of the localization error simplifies the use of high-dimensional perception data for safety-critical control algorithms.

Keywords: Estimation, Hybrid systems
Abstract: The design of a robust observer under denial-of-service attacks is addressed for linear time invariant systems in the bounded-error framework. The cyber-attacks occur between the output of the sensors localized on the physical plant and the cyber part embedding the observer. The data required by the observer are thus available at sporadic measurement time instants. In this setting, an interval impulsive observer is synthesized. The stability analysis of the dynamics of the state estimation error is done leveraging finite-gain L1 stability theory for hybrid systems. The observer L1 gain is computed by combining interval analysis and the resolution of algebraic inequalities that greatly reduces the synthesis complexity when compared to the state-of-the-art approaches that usually rely on solving many bilinear matrix inequalities. A numerical example illustrates the approach and the performance of the designed robust observer.

Keywords: Estimation, Hybrid systems, Machine learning
Abstract: Lithium-sulfur (Li-S) is a promising battery chem- istry for applications demanding high energy densities, such as electrified aircraft and heavy-duty trucks, among others. A critical challenge in modeling the Li-S chemistry lies in the use of differential algebraic (DAE) equations for representing the electrochemical dynamics. Due to their constrained and stiff nature, these equations are not conducive to real-time state estimation. In this study, we propose a novel approach to constrained state estimation for Li-S batteries by integrating a piecewise affine (PWA) model into a moving horizon estimation (MHE) framework. We begin by deriving the PWA model using a linear tree algorithm based on data obtained from simulations of a calibrated DAE model. We further leverage the unique structural advantages of the proposed PWA model to formulate a real-time state estimation algorithm grounded in a mixed-integer quadratic program. Overall, our initial findings, based on a single constant current trajectory, demonstrate that our approach offers an accurate and computationally efficient method for modeling and state estimation of Li-S batteries. The coupled PWA-MHE framework effectively captures the dynamics of the DAE system, even in the presence of high observational noise (20mV).

Keywords: Estimation, Filtering, Time-varying systems
Abstract: The increasing availability of sensing techniques provides a great opportunity for engineers to design state estimation methods, which are optimal for the system under observation and the observed noise patterns. However, these patterns often do not fulfill the assumptions of existing methods. We provide a direct method using samples of the noise to create a moving horizon observer for linear time-varying and nonlinear systems, which is optimal under the empirical noise distribution. Moreover, we show how to enhance the observer with distributional robustness properties in order to handle unmodeled components in the noise profile, as well as different noise realizations. We prove that, even though the design of distributionally robust estimators is a complex minmax problem over an infinite-dimensional space, it can be transformed into a regularized linear program using a system level synthesis approach. Numerical experiments with the Van der Pol oscillator show the benefits of not only using empirical samples of the noise to design the state estimator, but also of adding distributional robustness. We show that our method can significantly outperform state-of-the-art approaches under challenging noise distributions, including multi-modal and deterministic components.

Keywords: Observers for nonlinear systems, Nonlinear output feedback, Filtering
Abstract: This study is concerned with observer design for a class of Lipschitz nonlinear systems. A high-gain observer with a straightforward structure is proposed. As opposed to the well-known high gain observers, dynamic gains obtained are used to reduce the effect of peaking. In addition, the injection term of the observer is passed through a linear filter to reduce its sensitivity to noise. It is shown that the suggested observer is peaking free with respect to the initial conditions, while achieving the input to state stability with respect to measurement noise, as a HGO. The analysis of the steady-state response also shows that the proposed observer performs better in the presence of high-frequency noise. The simulation results compare the performance of the proposed method with some existing observers.

Keywords: Game theory, Agents-based systems, Linear systems
Abstract: A zero-sum tax/subsidy approach and a necessary condition for stabilizing unstable Nash equilibria in pseudo-gradient-based noncooperative dynamical systems with vector-valued payoff functions are proposed. Specifically, we first present a necessary and sufficient condition for the Nash equilibrium of the noncooperative game with vector-valued payoff functions to be bounded. Then we give a sufficient condition for such Nash equilibrium to be stable. After that, we develop a framework where a system manager constructs a zero-sum tax/subsidy incentive structure by collecting taxes from one agent and giving the same amount of subsidy to the other agent to make the incentivized Nash equilibrium stable and bounded, which can make the trajectories converge to the interior of original Nash equilibrium set. Finally, we present a numerical example to illustrate the utility of the zero-sum tax/subsidy approach.

Keywords: Game theory, Agents-based systems, Markov processes
Abstract: We consider a periodic double auction (PDA) setting where buyers of the auction have multiple (but finite)opportunities to procure multiple but fixed units of a commodity. The goal of each buyer participating in such auctions is to reduce their cost of procurement by planning their purchase across multiple rounds of the PDA. Formulating such optimal bidding strategies in a multi-agent periodic double auction setting is a challenging problem as such strategies involve planning across current and future auctions. In this work, we consider one such setup wherein the composite supply curve is known to all buyers. Specifically, for the complete information setting, we model the PDA as a Markov game and derive Markov perfect Nash equilibrium (MPNE) solution to devise an optimal bidding strategy for the case when each buyer is allowed to make one bid per round of the PDA. Thereafter, the efficacy of the Nash policies obtained is demonstrated with numerical experiments.

Keywords: Game theory, Agents-based systems, Uncertain systems
Abstract: The emergent behavior of a distributed system is conditioned by the information available to the local decision makers. Therefore, one may expect that providing decision makers with more information will improve system performance; in this work, we find that this is not necessarily the case. In multi-agent maximum coverage problems, we find that even when agents’ objectives are aligned with the global welfare, informing agents about the realization of the resource’s random values can reduce equilibrium performance by a factor of 1/2. This affirms an important aspect of designing distributed systems: information need be shared carefully. We further this understanding by providing lower and upper bounds on the ratio of system welfare when information is (fully or partially) revealed and when it is not, termed the value-of-informing. We then identify a trade-off that emerges when optimizing the performance of the best-case and worst-case equilibrium.

Keywords: Game theory, Algebraic/geometric methods, Discrete event systems
Abstract: We introduce a decentralized mechanism for pricing and exchanging alternatives constrained by transaction costs. We characterize the time-invariant solutions of a heat equation involving a (weighted) Tarski Laplacian operator, defined for max-plus matrix-weighted graphs, as approximate equilibria of the trading system. We study algebraic properties of the solution sets as well as convergence behavior of the dynamical system. We apply these tools to the "economic problem"' of allocating scarce resources among competing uses. Our theory suggests differences in competitive equilibrium, bargaining, or cost-benefit analysis, depending on the context, are largely due to differences in the way that transaction costs are incorporated into the decision-making process. We present numerical simulations of the synchronization algorithm (RRAggU), demonstrating our theoretical findings.

Keywords: Game theory, Communication networks, Uncertain systems
Abstract: In this article, we consider a problem of strategic communication between a sender (Alice) and a receiver (Bob) akin to the now-traditional model of Bayesian Persuasion intro- duced by Kamenica & Gentzkow, with the crucial difference that Bob is not assumed Bayesian. In lieu of the Bayesian assumption, Alice assumes that Bob behaves “almost like” a Bayesian agent, in some sense, without resorting to any specific model.

Under this assumption, we study Alice’s strategy when both utilities are quadratic and the prior is scalar. We show that, contrary to the Bayesian case, Alice’s optimal response may be more subtle than revealing “all or nothing.” More precisely, Alice reveals the state of the world when it lies outside a specific interval, and nothing otherwise. This interval increases (and the amount of information shared decreases) as Bob further departs from Bayesianity, much to his detriment.

Keywords: Game theory, Control of networks, Learning
Abstract: A conventional strategic classification problem takes on a Stackelberg form: a decision maker commits to a decision rule (e.g., in the form of a binary classifier) and agents best respond to the published decision rule by deciding on an effort level so as to maximize their chance of getting a favorable decision less the cost of the effort. This problem becomes significantly more complex when we allow agents access to two types of effort: honest (improvement actions) and dishonest (or cheating/gaming). While the former improves an agent's underlying unobservable states (e.g., certain types of qualification), the latter merely improves an agent's outward observable feature, serving as input to the classifier. Under the natural assumption that honest effort is more costly than cheating, prior work has shown that the decision maker has limited ability to mitigate cheating by simply adjusting the decision rule. In this paper, we consider a collaboration mechanism, which the decision maker establishes at a cost and offers to the agents together with the decision rule. In this case, an agent best responds by choosing not only its effort but also whether to participate in the mechanism and if so, with which other agents it wishes to form a connection or collaboration relation. While agents outside the mechanism remain independent of each other, those inside the mechanism are connected to a group of collaborators and enjoy positive externality in the form of a boost in their observable features and consequently enhanced probability of a favorable decision outcome. We show how the collaboration mechanism can induce agents to participate and take improvement actions over gaming and how it can benefit both sides. We also discuss the social values of the system, including social welfare, social qualification status, and the mechanism surplus.

Keywords: Optimal control, Adaptive control, Constrained control
Abstract: Q-learning is a promising method for solving optimal control problems for uncertain systems without the explicit need for system identification. However, approaches for continuous-time Q-learning have limited provable safety guarantees, which restrict their applicability to real-time safety-critical systems. This paper proposes a safe Q-learning algorithm for partially unknown linear time-invariant systems to solve the linear quadratic regulator problem with user-defined state constraints. We frame the safe Q-learning problem as a constrained optimal control problem using reciprocal control barrier functions and show that such an extension provides a safety-assured control policy. To the best of our knowledge, Q-learning for continuous-time systems with state constraints has not yet been reported in the literature.

Keywords: Optimal control, Autonomous systems, Optimization
Abstract: In this paper we study an infinite-horizon persistent monitoring problem in a two-dimensional mission space containing a finite number of statically placed targets, at each of which we assume a constant rate of uncertainty accumulation. Equipped with a sensor of finite range, the agent is capable of reducing the uncertainty of nearby targets. We derive a steady-state minimum time periodic trajectory over which each of the target uncertainties is driven down to zero during each visit. A hierarchical decomposition leads to purely local optimal control problems, coupled via boundary conditions. We optimize both the local trajectory segments as well as the boundary conditions in an on-line bilevel optimization scheme.

Keywords: Optimal control, Communication networks, Autonomous systems
Abstract: We propose a method for providing communication network infrastructure in autonomous multi-agent teams. In particular, we consider a set of communication agents that are placed alongside regular agents from the system. In order to find the optimal positions to place such agents, we define a flexible performance function that adapts to network requirements for different systems. We provide an algorithm based on shadow prices of a related convex optimization problem in order to arrive at a local maximum. We run the algorithm for three different performance functions associated to three practical scenarios, in which we show both the performance of the algorithm and the flexibility for different network requirements.

Keywords: Optimal control, Computational methods
Abstract: An efficient approach for the construction of separable approximations of optimal value functions from interconnected optimal control problems is presented. The approach is based on assuming decaying sensitivities between subsystems, enabling a curse-of-dimensionality free approximation, for instance by deep neural networks.

Keywords: Optimal control, Control applications, Smart grid
Abstract: Pumping in water distribution systems (WDSs) consumes a significant amount of power from the grid and may incur large electricity cost. WDSs with solar panels and batteries can greatly reduce the electricity cost by displacing the use of grid-based electricity with solar or stored energy, while also utilising water storage elements to allow selective pumping. A novel economic model predictive control (EMPC) scheme is proposed in this paper to facilitate optimal operation of pumps, batteries and solar panels in the WDSs. The proposed EMPC controller seeks to minimize the energy cost for water pumping while keeping the water levels in tanks and the battery state of charge within restricted limits. The EMPC is applied to an EPANET model of the Richmond Pruned Network, a Doyle Fuller Newman model of a lithium-ion battery, and simulated output from solar panels to to determine the efficacy of the proposed scheme.

Keywords: Optimal control, Cyber-Physical Security, Autonomous systems
Abstract: We consider a setting where a supervisor delegates an agent to perform a certain control task, while the agent is incentivized to deviate from the given policy to achieve its own goal. In this work, we synthesize the optimal deceptive policies for an agent who attempts to hide its deviations from the supervisor's policy. We study the deception problem in the continuous-state discrete-time stochastic dynamics setting and, using motivations from hypothesis testing theory, formulate a Kullback-Leibler control problem for the synthesis of deceptive policies. This problem can be solved using backward dynamic programming in principle, which suffers from the curse of dimensionality. However, under the assumption of deterministic state dynamics, we show that the optimal deceptive actions can be generated using path integral control. This allows the agent to numerically compute the deceptive actions via Monte Carlo simulations. Since Monte Carlo simulations can be efficiently parallelized, our approach allows the agent to generate deceptive control actions online. We show that the proposed simulation-driven control approach asymptotically converges to the optimal control distribution.

Keywords: Optimization algorithms, Computer-aided control design
Abstract: While many approaches were developed for obtaining worst-case complexity bounds for first-order optimization methods in the last years, there remain theoretical gaps in cases where no such bound can be found. In such cases, it is often unclear whether no such bound exists (e.g., because the algorithm might fail to systematically converge) or simply if the current techniques do not allow finding them.

In this work, we propose an approach to automate the search for cyclic trajectories generated by first-order methods. This provides a constructive approach to show that no appropriate complexity bound exists, thereby complementing the approaches providing sufficient conditions for convergence. Using this tool, we provide ranges of parameters for which some of the famous heavy-ball, Nesterov accelerated gradient, inexact gradient descent, and three-operator splitting algorithms fail to systematically converge, and show that it nicely complements existing tools searching for Lyapunov functions.

Keywords: Optimization algorithms, Cooperative control, Agents-based systems
Abstract: We propose a flexible gradient tracking approach with adjustable computation and communication steps for solving distributed stochastic optimization problems over networks. The proposed method allows each node to perform multiple local gradient updates and multiple inter-node communications in each round, aiming to strike a balance between computation and communication costs according to the properties of objective functions and network topology in non-i.i.d. settings. Leveraging a properly designed Lyapunov function, we derive both the computation and communication complexities for achieving arbitrary accuracy on smooth and strongly convex objective functions. Our analysis demonstrates sharp dependence of the convergence performance on graph topology and properties of objective functions, highlighting the trade-off between computation and communication. Numerical experiments are conducted to validate our theoretical findings.

Keywords: Optimization algorithms, Distributed control, Networked control systems
Abstract: In this paper, we propose a novel distributed algorithm for consensus optimization over networks. The key idea is to achieve dynamic consensus on the agents' average and on the global descent direction by iteratively solving an online auxiliary optimization problem through the Alternating Direction Method of Multipliers (ADMM). Such a mechanism is suitably interlaced with a local proportional action steering each agent estimate to the solution of the original consensus optimization problem. The analysis uses tools from system theory to prove the linear convergence of the scheme with strongly convex costs. Finally, some numerical simulations confirm our findings and show the robustness of the proposed scheme.

Keywords: Optimization algorithms, Distributed control, Agents-based systems
Abstract: This study addresses a distributed optimization with a novel class of coupling of variables, called clique-wise coupling. A clique is a node set of a complete subgraph of an undirected graph. This setup is an extension of pairwise coupled optimization problems (e.g., consensus optimization) and allows us to handle coupling of variables consisting of more than two agents systematically. To solve this problem, we propose a clique-based linearized ADMM algorithm, which is proved to be distributed. Additionally, we consider objective functions given as a sum of nonsmooth and smooth convex functions and present a more flexible algorithm based on the FLiP-ADMM algorithm. Moreover, we provide convergence theorems of these algorithms. Notably, all the algorithmic parameters and the derived condition in the theorems depend only on local information, which means that each agent can choose the parameters in a distributed manner. Finally, we apply the proposed methods to a consensus optimization problem and demonstrate their effectiveness via numerical experiments.

Keywords: Optimization algorithms, Distributed control, Stochastic systems
Abstract: Most of the current literature focused on centralized learning is centered around the celebrated average-consensus paradigm and less attention is devoted to scenarios where the communication between the agents may be imperfect. This paper presents three different algorithms of Decentralized Federated Learning (DFL) in the presence of imperfect information sharing modeled as noisy communication channels. The first algorithm, Federated Noisy Decentralized Learning (FedNDL1) comes from the literature, where the noise is added to the algorithm parameters to simulate the scenario of the presence of noisy communication channels. This algorithm shares parameters to form a consensus with the clients based on a communication graph topology through a noisy communication channel. The proposed second algorithm (FedNDL2) is similar to the first algorithm but with added noise to the parameters and it performs the gossip averaging before the gradient optimization. The proposed third algorithm (FedNDL3), on the other hand, shares the gradients through noisy communication channels instead of the parameters. Theoretical and experimental results show that under imperfect information sharing, the third scheme that mixes gradients is more robust in the presence of a noisy channel compared with the algorithms from the literature that mix the parameters.

Keywords: Optimization algorithms, Distributed control
Abstract: This work studies the distributed nonconvex optimization problem in bandwidth-limited communication environments. We develop a communication-efficient algorithm based on the gradient-tracking based distributed optimization method, where each computation node is equipped with a new event-triggered communication scheduler. Such scheduler approves the broadcasting only when the innovation of exchanged variables exceeds the change of decision variables in two consecutive updates. Compared to the conventional scheduler with time-dependent vanishing thresholds, the proposed one adapts better to the optimization dynamics and thus leads to more significant communication reduction. Finally, we prove the convergence of the algorithm and illustrate its performance via numerical examples.

Keywords: Machine learning, Algebraic/geometric methods, Predictive control for nonlinear systems
Abstract: Extended Dynamic Mode Decomposition (eDMD) is a powerful tool to generate data-driven surrogate models for the prediction and control of nonlinear dynamical systems in the Koopman framework. In eDMD a compression of the lifted system dynamics on the space spanned by finitely many observables is computed, in which the original space is embedded as a low-dimensional manifold. While this manifold is invariant for the infinite-dimensional Koopman operator, this invariance is typically not preserved for its eDMD-based approximation. Hence, an additional (re-)projection step is often tacitly incorporated to improve the prediction capability. We propose a novel framework for consistent reprojectors respecting the underlying manifold structure. Further, we present a new geometric reprojector based on maximum-likelihood arguments, which significantly enhances the approximation accuracy and preserves known finite-data error bounds.

Keywords: Machine learning, Autonomous vehicles, Data driven control
Abstract: Trajectory planners of autonomous vehicles usually rely on physical models to predict the vehicle behavior. However, despite their suitability, physical models have some shortcomings. On the one hand, simple models suffer from larger model errors and more restrictive assumptions. On the other hand, complex models are computationally more demanding and depend on environmental and operational parameters. In each case, the drawbacks can be associated to a certain degree to the physical modeling of the yaw rate dynamics. Therefore, this paper investigates the yaw rate prediction based on conditional neural processes (CNP), a data-driven meta-learning approach, to simultaneously achieve low errors, adequate complexity and robustness to varying parameters. Thus, physical models can be enhanced in a targeted manner to provide accurate and computationally efficient predictions to enable safe planning in autonomous vehicles. High fidelity simulations for a variety of driving scenarios and different types of cars show that CNP makes it possible to employ and transfer knowledge about the yaw rate based on current driving dynamics in a human-like manner, yielding robustness against changing environmental and operational conditions.

Keywords: Machine learning, Control over communications, Optimization algorithms
Abstract: For machine learning in situations where data is scattered and cannot be aggregated, federated learning, in which aggregators and agents send and receive model parameters, is one of the most promising methods. The scheduling problem of deciding which agents to communicate with has been studied in various ways, but it is not easy to solve due to its combinatorial optimization nature. In this letter, we attempt to solve this scheduling problem using combinatorial optimization theory. Specifically, we propose an efficient exact solution method based on dynamic programming and a greedy method whose superiority is confirmed by numerical examples. We also discuss the applicability of the proposed methods to a more realistic federated learning setting.

Keywords: Machine learning, Data driven control, Constrained control
Abstract: Safety is a primary concern when applying reinforcement learning to real-world control tasks, especially in the presence of external disturbances. However, existing safe reinforcement learning algorithms rarely account for external disturbances, limiting their applicability and robustness in practice. To address this challenge, this paper proposes a robust safe reinforcement learning framework that tackles worst-case disturbances. First, this paper presents a policy iteration scheme to solve for the robust invariant set, i.e., a subset of the safe set, where persistent safety is only possible for states within. The key idea is to establish a two-player zero-sum game by leveraging the safety value function in Hamilton-Jacobi reachability analysis, in which the protagonist (i.e., control inputs) aims to maintain safety and the adversary (i.e., external disturbances) tries to break down safety. This paper proves that the proposed policy iteration algorithm converges monotonically to the maximal robust invariant set. Second, this paper integrates the proposed policy iteration scheme into a constrained reinforcement learning algorithm that simultaneously synthesizes the robust invariant set and uses it for constrained policy optimization. This algorithm tackles both optimality and safety, i.e., learning a policy that attains high rewards while maintaining safety under worst-case disturbances. Experiments on classic control tasks show that the proposed method achieves zero constraint violation with learned worst-case adversarial disturbances, while other baseline algorithms violate the safety constraints substantially. Our proposed method also attains comparable performance as the baselines even in the absence of the adversary.

Keywords: Machine learning, Identification for control, Predictive control for nonlinear systems
Abstract: Nonlinear model predictive control (MPC) is an established control framework that not only provides a systematic way to handle state and input constraints, but also offers the flexibility to incorporate data-driven models. With the proliferation of machine learning techniques, there is an uptrend in the development of learning-based MPC, with neural networks (NN) being an important cornerstone. Although it has been shown that NNs are expressive enough to model the dynamics of complex systems and produce accurate state predictions, these predictions often do not include uncertainty estimates or have practical finite sample guarantees. In contrast to existing work that either requires the data samples to be exchangeable or relies on properties of the underlying data distribution, we propose an approach that utilizes weighted conformal prediction to alleviate these assumptions and to synthesize provably valid, finite-sample uncertainty estimates for data-driven dynamics models, in a distribution-free manner. These uncertainty estimates are generated online and incorporated into a novel uncertainty-aware learning-based MPC framework. Through a case study with a cartpole system controlled by a state-of-the-art learning-based MPC framework, we demonstrate that our approach not only provides well-calibrated uncertainty estimates, but also enhances the closed-loop performance of the system.

Keywords: Machine learning, Large-scale systems, Communication networks
Abstract: Federated learning (FL) has recently gained much attention due to its effectiveness in speeding up supervised learning tasks under communication and privacy constraints. However, whether similar speedups can be established for reinforcement learning remains much less understood theoretically. Towards this direction, we study a federated policy evaluation problem where agents communicate via a central aggregator to expedite the evaluation of a common policy. To capture typical communication constraints in FL, we consider finite capacity up-link channels that can drop packets based on a Bernoulli erasure model. Given this setting, we propose and analyze QFedTD - a quantized federated temporal difference learning algorithm with linear function approximation. Our main technical contribution is to provide a finite-sample analysis of QFedTD that (i) highlights the effect of quantization and erasures on the convergence rate; and (ii) establishes a linear speedup w.r.t. the number of agents under Markovian sampling. Notably, while different quantization mechanisms and packet drop models have been extensively studied in the FL, distributed optimization, and networked control systems literature, our work is the first to provide a non-asymptotic analysis of their effects in multi-agent and federated reinforcement learning.

Keywords: Agents-based systems, Autonomous systems, Distributed control
Abstract: A new approach to solving the linear algebraic equation Ax = b is presented by introducing a leaderless clustered multi-agent system. Each agent is associated with a certain submatrix of A and a vector obtained from b by a certain decomposition process. A distributed algorithm has each agent processing information solely with its own information about A and b, but sharing a time-varying estimate of part of the solution of Ax = b with its neighbors, as defined by a graphical structure overlaying the agents. This graphical structure divides the agents into clusters, with agents in one cluster being associated with one block column of A, and different rows of that block column; with each intra-cluster graph being connected. The update law uses consensus within individual clusters, utilizing their estimated states together with a process of inter-cluster conservation aimed at maintaining certain necessary constraints. Unlike previous literature that uses clustered multi-agent systems or double-layered networks, this framework does not require the presence of an aggregator or central node in the network’s clusters. The algorithm demonstrates exponential convergence, evidenced by both theoretical proof and numerical simulations.

Keywords: Agents-based systems, Algebraic/geometric methods, Network analysis and control
Abstract: Stable polarization of multi-agent systems has been shown to exist over Rn and highly symmetric nonlinear spaces, especially the n-sphere S^n. Toward a more generalized setting without assuming linearity or symmetry, our previous work established the same type of emergent behavior over general hypersurfaces, subsuming the n-sphere case. In this paper, we discuss our ongoing work of extending our previous hypersurface results to study the stability of polarized equilibria of multi-agent gradient flows evolving on general Riemannian manifolds. The aim is to provide sufficient conditions in terms of the manifold geometry. Special attention is paid to two nonlinear manifolds of interest, the Stiefel manifold and the Grassmannian. While the polarization of the former share similar traits to that of the n-sphere, the latter is shown to have distinct polarization behaviors.

Keywords: Agents-based systems, Decentralized control, Nonlinear systems
Abstract: This paper proposes a novel constructive barrier feedback for reactive collision avoidance between two agents. It incorporates this feature in a formation tracking control strategy for a group of 2nd-order dynamic robots defined in three-dimensional space. Using only relative measurements between neighboring agents, we propose an elegant decentralized controller as the sum of a nominal tracking controller and the constructive barrier feedback for leader-follower formations under a directed single-spanning tree graph topology. The key ingredient is the use of divergent flow as a dissipative term, which slows down the relative velocity in the direction of the neighboring robots without compromising the nominal controller’s performance. Compared to traditional barrier function-based optimization controllers, the proposed constructive barrier feedback avoids feasibility issues and results in more computationally efficient control algorithms with systematic equilibrium analysis.

Keywords: Agents-based systems, Discrete event systems, Network analysis and control
Abstract: In this paper, we present a new bounded-confidence model of opinion dynamics where agents with a substantially different opinion may still be able to influence each other along a static graph. Our intention is to account for hard and fast ties present due to physical or social proximity. This additional feature allows weak but persistent interaction between disagreeing agents. Albeit simple, the model remains difficult to analyse due to its separation of time scales. We introduce an appropriate notion of stability, give some properties of the system and formulate a conjecture supported by numerical simulations.

Keywords: Agents-based systems, Distributed control, Cooperative control
Abstract: This paper presents a resilient distributed algorithm for solving a system of linear algebraic equations over a multi-agent network in the presence of Byzantine agents capable of arbitrarily introducing untrustworthy information in communication. It is shown that the algorithm causes all non-Byzantine agents' states to converge to the same least squares solution exponentially fast, provided appropriate levels of graph redundancy and objective redundancy are established. An explicit convergence rate is also provided.

Keywords: Agents-based systems, Distributed control, Robotics
Abstract: In this paper, we develop a strategy for arranging a team of agents in orbits of arbitrary shape that pass through a common point that may move. In particular, the shapes are defined by star-shaped sets. The solution is based on its transformation into the traditional circular formation problem and subsequent reconversion. Given the large number of solutions to such a problem, the developed algorithm is well suited as a plug & play command generator to extend these solutions to the promising field of aerial robotic swarms. Two examples of such an application show its ease of implementation.

Keywords: Cooperative control, Adaptive control, Identification
Abstract: We propose a new method for simultaneous synchronization and topology identification of a complex dynamical network that relies on the edge-agreement framework and on adaptive-control approaches by design of an auxiliary network. Our method guarantees the identification of the unknown topology and it guarantees that once the topology is identified the complex network achieves synchronization. Under our identification algorithm we are able to provide stability results for the estimation errors in the form of uniform semiglobal practical asymptotic stability. Finally, we demonstrate the effectiveness of our approach with an illustrating example.

Keywords: Cooperative control, Agents-based systems, Cyber-Physical Security
Abstract: The interaction topology plays a significant role in the collaboration of multi-agent systems. How to preserve the topology against inference attacks has become an imperative task for security concerns. In this paper, we propose a distributed topology-preserving algorithm for second-order multiagent systems by adding noisy inputs. The major novelty is that we develop a strategic compensation approach to overcome the noise accumulation issue in the second-order dynamic process while ensuring the exact second-order consensus. Specifically, we design two distributed compensation strategies that make the topology more invulnerable against inference attacks. Furthermore, we derive the relationship between the inference error and the number of observations by taking the ordinary least squares estimator as a benchmark. Extensive simulations are conducted to verify the topology-preserving performance of the proposed algorithm.

Keywords: Cooperative control, Adaptive systems, Uncertain systems
Abstract: This paper proposes an adaptive distributed observer for a class of discrete-time uncertain linear leader systems. The leader system is assumed to be neutrally stable with unknown parameters in the system matrix. Such a leader system can produce multi-tone sinusoidal signals with unknown frequencies, magnitudes, and phases. Under the assumption that the digraph of the communication network is a spanning tree with the leader system as the root, the proposed adaptive distributed observer is shown to be capable of estimating over the communication network not only the leader's state, but also the unknown parameters of the leader's system matrix.

Keywords: Cooperative control, Agents-based systems, Autonomous robots
Abstract: This paper presents a collaboration strategy that enables heterogeneous agents, i.e., different capabilities and dynamics, to accomplish tasks by working together. The collaboration between multiple agents is inspired by the ecological concept known as a mutualism, an interaction between two or more species that benefits everyone involved. A collaborative act is made possible through the composition of barrier functions, which allows the heterogeneous agents to work together safely. Moreover, a measure of collaborative potential is established to assess the merit of agents interacting with each other. Furthermore, the collaboration framework is provided for a general multi-agent setting. Finally, the collaboration framework's efficacy is demonstrated in two case studies that necessitate collaboration between the heterogeneous agents to complete their respective tasks.

Keywords: Cooperative control, Agents-based systems, Communication networks
Abstract: This paper investigates the bipartite flocking behavior of multi-agent systems with coopetition interactions, where communications between agents are described by signed digraphs. The scenario with switching topologies due to the movement of agents, and time delays caused by the limited data transmission capability, is considered comprehensively. Nonlinear weight functions are designed to describe the relationship between the communication distance of agents and the coopetition degree in real biological networks. A distributed update rule based on the neighbors' information and the designed weight functions is proposed. By the aid of the graph theory and sub-stochastic matrix properties, the effectiveness of the proposed update rule is proved theoretically, and the algebraic conditions for achieving the bipartite flocking behavior are obtained. Finally, the theoretical results are verified by numerical simulations.

Keywords: Cooperative control, Adaptive control, Identification for control
Abstract: This paper presents a finite-time topology identification method for complex dynamical networks. This method prevents the difficulty of verifying linear independence conditions and ensures the success of accurate topology identification. The topology identification scheme first renders the error dynamics between the networks and reference signals zero in finite time, and afterward, the topology is estimated by building an auxiliary network. The identification of topology is achieved once a relaxed excitation condition holds. The excitation condition is guaranteed by the proposed tracking control scheme. A finite-time topology identification and synchronization scheme for complex systems is further proposed where synchronization is realized by removing the exciting signals after the identification of the topology. At last, the simulation results verify the feasibility of the proposed method.

Keywords: Closed-loop identification, Large-scale systems, Networked control systems
Abstract: For identification of a single module in a linear dynamic network with correlated disturbances different methods are available in a prediction error setting. While indirect methods fully rely on the presence of a sufficient number of external excitation signals for achieving data-informativity, the local direct method with a MIMO predictor model can exploit also non-measured disturbance signals for data-informativity. However, a simple two-node example shows that this local direct method can also be conservative in terms of the number of external excitation signals that is required. Inspired by a recently introduced multi-step method for full network identification, we present a multi-step least squares method for single module identification. In a first indirect step a model is estimated that is used to reconstruct the innovation on a set of output signals, which in a second step is used to directly estimate the module dynamics with a MISO predictor model. The resulting path based conditions for data-informativity show that the multi-step method requires a smaller number of excitation signals for data-informativity than the local direct method.

Keywords: Network analysis and control, Stochastic systems, Identification
Abstract: Over the last decade, there has been a significant increase in interest for techniques that can infer the connectivity structure of a network of dynamic systems. This article examines a flexible class of network systems and reviews various methods for reconstructing their underlying graph. However, these techniques typically only guarantee consistent reconstruction if additional assumptions on the model are made, such as the network topology being a tree, the dynamics being strictly causal, or the absence of directed loops in the network. The central theme of the article is to reinterpret these methodologies under a unified framework where a graphical notion of separation between nodes of the underlying graph corresponds to a probabilistic notion of separation among associated stochastic processes. This duality property enables the creation of algorithms for reconstructing the network topology. The article also endeavors to connect the considered class of networks with other semantically similar network models.

Keywords: Network analysis and control, Identification
Abstract: We analyze the problem of network identifiability with nonlinear functions associated with the edges. We consider a static model for the output of each node and by assuming a perfect identification of the function associated with the measurement of a node, we provide conditions for the identifiability of the edges in a specific class of functions. First, we analyze the identifiability conditions in the class of all nonlinear functions and show that even for a path graph, it is necessary to measure all the nodes except by the source. Then, we consider analytic functions satisfying f(0)=0 and we provide conditions for the identifiability of paths and trees. Finally, by restricting the problem to a smaller class of functions where none of the functions is linear, we derive conditions for the identifiability of directed acyclic graphs. Some examples are presented to illustrate the results.

Keywords: Boolean control networks and logic networks, Identification, Large-scale systems
Abstract: Dynamic network reconstruction aims to infer network structure from input-output data. Dynamical structure functions (DSFs) have been introduced to represent structural information between observable nodes of linear time-invariant systems. However, reconstructing large-scale DSFs can be difficult since most existing methods do not scale. Instead of inferring large DSFs directly, an alternative approach is to reconstruct many small-scale DSFs that are easier to infer. Given a sparsity constraint on the network, this paper proposes a necessary and sufficient condition for perfect reconstruction of the Boolean network using collapsed small-scale networks. For sparse networks, such as gene regulatory networks, this method can significantly reduce time and computational costs of Boolean network inference for most links in the network, especially when using parallel computing.

Keywords: Estimation, Power systems, Optimization
Abstract: Observability of all bus voltages in a power network enables overall monitoring and fault detection of power flow. Information on this voltage state is often a combination of voltage, current measurements obtained by Phasor Measurement Units (PMUs) and state estimation techniques that use admittance information of the connections between nodes within the power network. Voltage state estimation is a challenge for a power network in which limited PMUs need to be combined with partially know network admittance information. The challenge lies in choosing locations of PMUs such that full voltage state reconstruction is possible, despite the lack of full knowledge on network admittance information. This paper proposes a methodology of placing PMUs across a network with incomplete network admittance information that guarantees complete observability of the voltage states. The methodology separates network nodes in distinct nodal sets based on voltage, current and admittance information. Permutations of the sets are uses to establish the minimum number of PMUs required for full voltage state observability for a power network with partially known admittance information. Subsequently, an additional optimal placement can be used to further minimize the variance of the estimated voltage states. The proposed PMU placement approach is tested on a modified IEEE-14 bus with incomplete network admittance information.

Keywords: Identification, Networked control systems, Statistical learning
Abstract: Networked dynamic systems are ubiquitous in various domains such as industrial processes, social networks, and biological systems. These systems produce high-dimensional data that reflect the complex interactions among the network nodes with rich sensor measurements. In this paper, we propose a novel algorithm for latent dynamic networked system identification that leverages the network structure and performs dimension reduction for each node via dynamic latent variables (DLV). The algorithm assumes that the DLVs of each node have an auto-regressive model with exogenous input and interactions from other nodes. The DLVs of each node are extracted to capture the most predictable latent variables in the high dimensional data, while the residual factors are not predictable. The advantage of the proposed framework is demonstrated on an industrial process network for system identification and dynamic data analytics.

Keywords: Biological systems, Stability of nonlinear systems, Agents-based systems
Abstract: It is known that the effect of species' density on species' growth is non-additive in real ecological systems. This challenges the conventional Lokta-Volterra model, where the interactions are always pairwise and their effects are additive. To address this challenge, we introduce HOIs (Higher-Order Interactions) and are able to capture, for example, the indirect effect of one species on a second one correlating to a third species. Towards this end, we propose a purely cooperative higher-order Lokta-Volterra model and a higher-order Lokta-Volterra two-faction competition model. By utilizing the theory of monotone systems, we provide stability conditions for both models. The stability analysis further shows that small HOIs usually promote the coexistence of all species, while the extinction of some species is usually caused by a huge difference among the higher-order competitive terms. Finally, illustrative numerical examples are provided to highlight our contributions.

Keywords: Game theory, Hybrid systems, Optimization
Abstract: We consider a large population of decision makers that choose their evolutionary strategies based on simple pairwise imitation rules. We describe such a dynamic process by the replicator dynamics. Differently from the available literature, where the payoffs signals are assumed to be updated continuously, we consider a more realistic scenario where they are updated occasionally. Our main technical contribution is to devise two event-triggered communication schemes with asymptotic convergence guarantees to a Nash equilibrium. Finally, we show how our proposed approach is applicable as an efficient distributed demand response mechanism.

Keywords: Control of networks, Distributed control, Optimal control
Abstract: We analyze the optimal Linear Exponential Quadratic Gaussian (LEQG) control synthesis of a spatially distributed system with a shift invariance in its spatial coordinate, perturbed by additive white Gaussian noise. We refer to such a system as spatially invariant. The LEQG framework accounts for the risk attitude of the controller in its synthesis by appropriate selection of the value of a free parameter, providing the possibility to continuously tune the degree of risk awareness of the controller. We prove important structural properties of the optimal LEQG control problem for spatially invariant systems, namely that: (i) the optimal LEQG control gain is spatially invariant itself; (ii) the LEQG control synthesis problem is equivalent to a family of decoupled LEQG optimization problems of smaller dimension; and (iii) under some further assumptions, the optimal LEQG control gain is spatially localized. Through a case study, we illustrate how the risk attitude of the controller tunes the degree of spatial localization of the optimal control gain. We argue that the proven structural properties can be leveraged to reduce the computational complexity of obtaining the optimal LEQG control gain in large-scale systems and to design distributed risk-aware controller implementations.

Keywords: Game theory, Nonlinear systems, Adaptive control
Abstract: Controlling evolutionary game-theoretic dynamics is a problem of paramount importance for the systems and control community, with several applications spanning from social science to engineering. Here, we study a population of individuals who play a generic 2-action matrix game, and whose actions evolve according to a replicator equation - a nonlinear ordinary differential equation that captures salient features of the collective behavior of the population. Our objective is to steer such a population to a specified equilibrium that represents a desired collective behavior - e.g., to promote cooperation in the prisoner's dilemma. To this aim, we devise an adaptive-gain controller, which regulates the system dynamics by adaptively changing the entries of the payoff matrix of the game. The adaptive-gain controller is tailored according to distinctive features of the game, and conditions to guarantee global convergence to the desired equilibrium are established.

Keywords: Network analysis and control, Agents-based systems, Communication networks
Abstract: This paper addresses the problem of propagation of opinions in a Signed Friedkin-Johnsen (SFJ) model, i.e., an opinion dynamics model in which the agents are stubborn and the interaction graph is signed. We provide sufficient conditions for the stability of the SFJ model and for convergence to consensus of a concatenation of such SFJ models.

Keywords: Game theory, Network analysis and control, Optimization algorithms
Abstract: In the field of international security, understanding the strategic interactions between countries within a networked context is crucial. Our previous research has introduced a “games-on-signed graphs” framework to analyze these interactions. While the framework is intended to be basic and general, there is much left to be explored, particularly in capturing the complexity of strategic scenarios in international relations. Our paper aims to fill this gap in two key ways. First, we modify the existing preference axioms to allow for a more nuanced understanding of how countries pursue self-survival, defense of allies, and offense toward adversaries. Second, we introduce a novel algorithm that proves the existence of a pure strategy Nash equilibrium for these revised games. To validate our model, we employ historical data from the year 1940 as the game input and predict countries’ survivability. Our contributions thus extend the real-world applicability of the original framework, offering a more comprehensive view of strategic interactions in a networked security environment.

Keywords: Adaptive control, Aerospace
Abstract: Crucial phases in aerial transportation and delivery of suspended payloads are the clasping and unclasping of the payload to the cable. During these phases, along with the uncertainties in the quadrotor and in the environment, the inevitable payload swings induced by the human interaction or by other external interaction will create additional state-dependent uncertainties; such uncertainties pose a significant challenge in terms of control. If they continue unabated, these uncertainties can cause safety hazard for the quadrotor, the payload and, most importantly, for the human operating the clasping/unclasping tasks. As the state-of-the-art adaptive controllers cannot tackle such uncertainties or considers them as bounded terms, this paper presents an adaptive anti-swing controller where all uncertainties are taken in a state-dependent form. This choice is made to better capture uncertain clasping and unclasping operations of the suspended payload. The closed-loop stability is studied analytically and the real-time experiments confirm significant performance improvements for the proposed scheme over the state of the art.

Keywords: Adaptive control, Autonomous vehicles, Cooperative control
Abstract: An externally positive system has the property of giving a nonnegative output for any nonnegative input. By making the inter-vehicle spacing a nonnegative output, this system property is significant for collision avoidance in platooning. Yet, existing platooning results based on external positivity just apply to adaptive cruise control (ACC): as ACC uses on-board sensing only, these results do not apply when onboard sensing is integrated with inter-vehicle communication, as in cooperative adaptive cruise control (CACC). This work provides an integrated external positivity design for CACC. When unreliable communication requires transitions between CACC and ACC, the design still guarantees graceful degradation in terms of collision avoidance and disturbance rejection. Such graceful transitions can be attained also in the presence of vehicle parameter uncertainty, via a suitable adaptive control design.

Keywords: Adaptive control, Closed-loop identification, Nonlinear systems
Abstract: Transient performance improvement in adaptive backstepping control is beneficial for the stability and robustness of control systems. In addition, parameter convergence in classical adaptive control is dependent on a stringent condition named persistent excitation (PE). This paper proposes a least squares-based composite learning backstepping control (LS-CLBC) strategy with high-order tuners for strict-feedback uncertain nonlinear systems such that exponential stability with parameter convergence is achieved under interval excitation (IE) or even partial IE that is strictly weaker than PE. In the LS-CLBC, the storage and forgetting of online historical data are determined by the exciting strength of a novel excitation matrix consisting of only active regressor channels, such that excitation information of regressor channels is exploited more effectively and efficiently to achieve parameter estimation. The learning rate is adjusted online based on LS and integrated into a high-order tuner to obtain the high-order time derivatives of parameter estimates. The closed-loop system is proven exponentially stable. Simulation results have demonstrated the superiority of the proposed approach.

Keywords: Adaptive control, Distributed parameter systems, Autonomous vehicles
Abstract: This paper extends recent results on model reference adaptive control using reproducing kernel Hilbert space (RKHS) learning techniques for some general cases of multi-input systems. We leverage recent results on error bounds for nonlinear observers in a vector-valued RKHS to design adaptive model reference adaptive controllers (MRAC) that are induced by operator-valued kernels. This paper formulates a model reference adaptive control strategy based on a dead zone robust modification, and derives for this case conditions for the ultimate boundedness of the tracking error. The RKHS setting allows the control designer to influence the ultimate bound by selection and placement of operator-valued kernels. As in the scalar-valued setting, closed-form expressions are obtained for the ultimate upper bound. But in this case the upper bound depends on a generalization of the power function for an operator-valued kernel space. Finally, we provide a detailed illustration our results in practice for the case of attitude control of a streamlined tailed-controlled underwater vehicle.

Keywords: Adaptive control, Estimation, Aerospace
Abstract: This paper presents an online parameter update algorithm in the context of composite model reference adaptive control based on intermittent signal holding to improve convergence properties of the parameters representing the unstructured uncertainties in the absence of persistent excitation. The present study extends the algorithm which was previously developed by considering only the structured uncertainties for which the basis functions are known a priori. The proposed extension utilises the Gaussian radial basis function neural network as the model for the uncertainty assuming appropriate placement of the local basis functions in the state space. A notable distinction from the case with full knowledge of the features constituting the linearly-parameterised uncertainty model is that the extended algorithm introduces a robustifying modification in the earlier phase of operation to deal with the inevitable learning residual.

Keywords: Adaptive control, Identification for control, Linear systems
Abstract: The theory of adaptive learning-based control currently suffers from a mismatch between its empirical performance and the theoretical characterization of its performance, with consequences for, e.g., the understanding of sample efficiency, safety, and robustness. The linear quadratic regulator with unknown dynamics is a fundamental adaptive control setting with significant structure in its dynamics and cost function, yet even in this setting the ratio between the best-known regret or estimation error upper bounds and their corresponding best-known lower bounds is unbounded due to polylogarithmic factors in T. This gap has not been closed in any of the many papers theoretically studying the linear quadratic regulator with unknown dynamics, and indeed similar gaps have plagued other areas of theoretical online learning such as reinforcement learning. The contribution of this paper is to close that gap by establishing a novel regret upper-bound of O_p(sqrt{T}) , and simultaneously establishes an estimation error bound on the dynamics of O_p(T^{-1/4}). The two keys to our improved proof technique are (1) a more precise upper- and lower-bound on the system Gram matrix by establishing exact rates of eigenvalues from different sub-spaces and (2) a self-bounding argument for the expected estimation error of the optimal controller. Our technique may shed light on removing polylogarithmic factors in other adaptive learning problems.

Keywords: Power systems, Power electronics, Identification
Abstract: A fast and accurate grid impedance measurement of three-phase power systems is crucial for online assessment of power system stability and adaptive control of grid-connected converters. Existing grid impedance measurement approaches typically rely on pointwise sinusoidal injections or sequential wideband perturbations to identify a nonparametric grid impedance curve via fast Fourier computations in the frequency domain. This is not only time-consuming, but also inaccurate during time-varying grid conditions, while on top of that, the identified nonparametric model cannot be immediately used for stability analysis or control design. To tackle these problems, we propose to use parametric system identification techniques (e.g., prediction error or subspace methods) to obtain a parametric impedance model directly from time-domain current and voltage data. Our approach relies on injecting wideband excitation signals in the converter's controller and allows to accurately identify the grid impedance in closed loop within one injection and measurement cycle. Even though the underlying parametric system identification techniques are well-studied in general, their utilization in a grid impedance identification setup poses specific challenges, is vastly underexplored, and has not gained adequate attention in urgent and timely power systems applications. To this end, we demonstrate in numerical experiments how the proposed parametric approach can accomplish a significant improvement compared to prevalent nonparametric methods.

Keywords: Power systems, Smart grid, Network analysis and control
Abstract: The identification of distribution network topology and parameters is a critical problem that lays the foundation for improving network efficiency, enhancing reliability, and increasing its capacity to host distributed energy resources. Network identification problems often involve estimating a large number of parameters based on highly correlated measurements, resulting in an ill-conditioned and computationally demanding estimation process. We address these challenges by proposing two admittance matrix estimation methods. In the first method, we use the eigendecomposition of the admittance matrix to generalize the notion of stationarity to electrical signals and demonstrate how the stationarity property can be used to facilitate a maximum a posteriori estimation procedure. We relax the stationarity assumption in the second proposed method by employing Linear Minimum Mean Square Error (LMMSE) estimation. Since LMMSE estimation is often ill-conditioned, we introduce an approximate well-conditioned solution. Our quantitative results demonstrate the improvement in computational efficiency compared to the state-of-the-art methods while preserving the estimation accuracy.

Keywords: Optimization, Power systems, Data driven control
Abstract: To solve unmodeled optimization problems with hard constraints, this paper proposes a novel zeroth-order approach called Safe Zeroth-order Optimization using Linear Programs (SZO-LP). The SZO-LP method solves a linear program in each iteration to find a descent direction, followed by a step length determination. We prove that, under mild conditions, the iterates of SZO-LP have an accumulation point that is also the primal of a KKT pair. We then apply SZO-LP to solve an Optimal Power Flow (OPF) problem on the IEEE 30-bus system. The results demonstrate that SZO-LP requires less computation time and samples compared to state-of-the-art approaches.

Keywords: Optimization, Modeling, Power systems
Abstract: The deployment of battery energy storage systems (BESS) is necessary to integrate terawatts of renewable generation while supporting grid resilience and reliability efforts. Optimizing battery dispatch requires predictive battery models that accurately characterize the battery state of charge (SOC) to ensure that the battery operates within the energy and power limits and avoids unexpected saturation effects. Furthermore, most BESS are unable to simultaneously charge and discharge, which begets an additional, non-convex complementary constraint. This paper presents and compares recently developed predictive battery models that side-step the non-convexity while providing supporting analysis on modeling error and optimal parameter selection. Specifically, insights for four different predictive BESS formulations are presented, including non-linear, mixed-integer, linear convex relaxation, and linear robust formulations. Additionally, two two-stage approaches are also considered. Analysis is conducted on optimal parameter selection for two of the methods, as well, as providing a new and improved SOC error bound on the relaxed formulation and the role of sustainability constraints on the robust formulation. Through the lens of relevant BESS use-cases, the paper discusses optimality and feasibility guarantees between the different models and provides extensive simulation-based analysis.

Keywords: Power systems, Learning
Abstract: Ensuring the frequency stability of electric grids with increasing renewable resources is a key problem in power system operations. In recent years, a number of advanced controllers have been designed to optimize frequency control. These controllers, however, almost always assume that the net load in the system remains constant over a sufficiently long time. Given the intermittent and uncertain nature of renewable resources, it is becoming important to explicitly consider net load that is time-varying.

This paper proposes an adaptive approach to frequency control in power systems with significant time-varying net load. We leverage the advances in short-term load forecasting, where the net load in the system can be accurately predicted using weather and other features. We integrate these predictions into the design of adaptive controllers, which can be seamlessly combined with most existing controllers including conventional droop control and emerging neural network-based controllers. We prove that the overall control architecture achieves frequency restoration decentralizedly. Case studies verify that the proposed method improves both transient and frequency-restoration performances compared to existing approaches.

Keywords: Power systems, Power generation, Machine learning
Abstract: Power systems Unit Commitment (UC) problem determines the generator commitment schedule and dispatch decisions for power networks based on forecasted electricity demand. However, with the increasing penetration of renewables and stochastic demand behaviors, it becomes challenging to solve the large-scale, multi-interval UC problem in an efficient manner. The main objective of this paper is to propose a fast and reliable scheme to eliminate a set of redundant or inactive physical constraints in the high-dimensional, multi-interval, mixed-integer UC problem, while the reduced problem is equivalent to the original full problem in terms of commitment decisions. Our key insights lie on pre-screening the constraints based on the load distribution and considering the physical feasibility regions of multi-interval UC problem. For the multi-step UC formulation, we overcome screening conservativeness by utilizing the multi-step ramping relationships, and can reliably screen out more constraints compared to current practice. Extensive simulations on both specific load samples and load regions validate the proposed technique can screen out more than 80% constraints while preserving the feasibility of multi-interval UC problem.

Keywords: Data driven control, Autonomous systems, Behavioural systems
Abstract: We define persistency of excitation and informativity for system identification for the class of 2D state- representable autonomous systems. We characterize informativity for system identification in terms of properties of a matrix constructed from the restrictions of a system trajectory on successive consecutive lines. We state a procedure to compute arbitrary trajectories from a "sufficiently rich" one.

Keywords: Data driven control, Constrained control, Robust control
Abstract: This paper presents a direct data-driven approach for computing robust control invariant (RCI) sets and their associated state-feedback control laws for linear time-invariant systems affected by bounded disturbances. The proposed method utilizes a single state-input trajectory generated from the system, to compute a polytopic RCI set with a desired complexity and an invariance-inducing feedback controller, without the need to identify a model of the system. The problem is formulated in terms of a set of sufficient linear matrix inequality conditions that are then combined in a semi-definite program to maximize the volume of the RCI set while respecting the state and input constraints. We demonstrate through a numerical case study that the proposed data-driven approach can generate RCI sets that are of comparable size to those obtained by a model-based method in which exact knowledge of the system matrices is assumed.

Keywords: Data driven control, Behavioural systems, Identification
Abstract: The paper studies conical, convex, and affine models in the framework of behavioral systems theory. We investigate basic properties of such behaviors and address the problem of constructing models from measured data. We prove that closed, shift-invariant, conical, convex, and affine models have the intersection property, thereby enabling the definition of most powerful unfalsified models based on infinite-horizon measurements. We then provide necessary and sufficient conditions for representing conical, convex, and affine finite-horizon behaviors using raw data matrices, expressing persistence of excitation requirements in terms of non-negative rank conditions. The applicability of our results is demonstrated by a numerical example arising in population ecology.

Keywords: Data driven control, Learning, Optimal control
Abstract: In this paper we provide direct data-driven expressions for the Linear Quadratic Regulator (LQR), the Kalman filter, and the Linear Quadratic Gaussian (LQG) controller using a finite dataset of noisy input, state, and output trajectories. We show that our data-driven expressions are consistent, since they converge as the number of experimental trajectories increases, we characterize their convergence rate, and quantify their error as a function of the system and data properties. These results complement the body of literature on data-driven control and finite-sample analysis, and provide new ways to solve canonical control and estimation problems that do not assume, nor require the estimation of, a model of the system and noise and do not rely on solving implicit equations.

Keywords: Data driven control, Learning, Optimization
Abstract: This paper considers adjusting a fully parametrized model predictive control (MPC) scheme to approximate the optimal policy for a system as accurately as possible. By adopting MPC as a function approximator in reinforcement learning (RL), the MPC parameters can be adjusted using Q-learning or policy gradient methods. However, each method has its own specific shortcomings when used alone. Indeed, Q-learning does not exploit information about the policy gradient and therefore may fail to capture the optimal policy, while policy gradient methods miss any cost function corrections not affecting the policy directly. The former is a general problem, whereas the latter is an issue when dealing with economic problems specifically. Moreover, it is notoriously difficult to perform second-order steps in the context of policy gradient methods while it is straightforward in the context of Q-learning. This calls for an organic combination of these learning algorithms, in order to fully exploit the MPC parameterization as well as speed up convergence in learning.

Keywords: Data driven control, Linear systems, Stability of linear systems
Abstract: This paper presents a novel approach for solving the pole placement and eigenstructure assignment problems through data-driven methods. By using open-loop data alone, the paper shows that it is possible to characterize the allowable eigenvector subspaces, as well as the set of feedback gains that solve the pole placement problem. Additionally, the paper proposes a closed-form expression for the feedback gain that solves the eigenstructure assignment problem. Finally, the paper discusses a series of optimization problems aimed at finding sparse feedback gains for the pole placement problem.

Keywords: Nonlinear output feedback, Robotics, Vision-based control
Abstract: We present a method for synthesizing dynamic, reduced-order output-feedback polynomial control policies for control-affine nonlinear systems which guarantees runtime stability to a goal state, when using visual observations and a learned perception module in the feedback control loop. We leverage Lyapunov analysis to formulate the problem of synthesizing such policies. This problem is nonconvex in the policy parameters and the Lyapunov function that is used to prove the stability of the policy. To solve this problem approximately, we propose two approaches: the first solves a sequence of sum-of-squares optimization problems to iteratively improve a policy which is provably-stable by construction, while the second directly performs gradient-based optimization on the parameters of the polynomial policy, and its closed-loop stability is verified a posteriori. We extend our approach to provide stability guarantees in the presence of observation noise, which realistically arises due to errors in the learned perception module. We evaluate our approach on several underactuated nonlinear systems, including pendula and quadrotors, showing that our guarantees translate to empirical stability when controlling these systems from images, while baseline approaches can fail to reliably stabilize the system.

Keywords: Nonlinear systems, Aerospace
Abstract: Controllability and feasaiblity measures are used to determine whether a given system can achieve its specified objective. However, for nonlinear systems with space constraints, the controllable and feasible sets may be highly sensitive to minor perturbations in the system's constraints, initial states and parameters. This becomes particularly important in codesign of hypersonic vehicles, where functions governing the dynamics must be estimated from expensive computational fluid dynamics simulations, and poor initialization can lead to significant wasted resources. By relaxation of the constraints and introduction of a surrogate cost, we provide a method for detection and quantification of which constraints are violating feasibility. To demonstrate the method in a concrete example, we apply the technique to simulation of hypersonic vehicle trajectories.

Keywords: Nonlinear systems, Algebraic/geometric methods
Abstract: In a recent paper it has been shown that the existence, for a MIMO nonlinear system, of normal forms with a special structure that proves to be useful in the design of feedback laws is implied by an assumption introduced a long time ago by Hirschorn in his work on systems invertibility. In this paper, we provide an alternative viewpoint and prove that a necessary and sufficient condition for the existence of such kind of normal forms can be identified in a special feature of the so-called maximal controlled invariant distribution algorithm.

Keywords: Nonlinear systems, Algebraic/geometric methods
Abstract: The relations between a control system with delays given by a nonlinear input-output equation and its realization are addressed. The algebraic formalism based on rings of polynomials over the rings associated with the considered systems and modules of differential one-forms is used to show the relations between submodules corresponding to the input-output equation and its realization.

Keywords: Nonlinear systems, Fluid flow systems, Data driven control
Abstract: We consider the problem of the transport of a density of states from an initial state distribution to a desired final state distribution through a dynamical system with actuation. In particular, we consider the case where the control signal is a function of time, but not space; that is, the same actuation is applied at every point in the state space. This is motivated by several problems in fluid mechanics, such as mixing and manipulation of a collection of particles by a global control input such as a uniform magnetic field, as well as by more general control problems where a density function describes an uncertainty distribution or a distribution of agents in a multi-agent system. We formulate this problem using the generators of the Perron-Frobenius operator associated with the drift and control vector fields of the system. By considering finite-dimensional approximations of these operators, the density transport problem can be expressed as a control problem for a bilinear system in a high-dimensional, lifted state. With this system, we frame the density control problem as a problem of driving moments of the density function to the moments of a desired density function, where the moments of the density can be expressed as an output which is linear in the lifted state. This output tracking problem for the lifted bilinear system is then solved using differential dynamic programming, an iterative trajectory optimization scheme.

Keywords: Nonlinear systems, Estimation, Statistical learning
Abstract: This paper derives rates of convergence of approximations of the deterministic Koopman operator using a framework based on estimating solutions of inverse problems. By restricting the domain of the Koopman operator, simple sufficient conditions are derived that ensure that the resulting Koopman operator is compact when acting on a suitable reproducing kernel Hilbert space (RKHS). Approximations of the Koopman operator, or its inverse, are derived in terms of Galerkin approximations of solutions to an associated inverse problem which depends on noisy data. The resulting bounds on accuracy of approximations to the Koopman operator then take a classical form: we obtain explicit representations of the contributions of the approximation error and the generalization error in terms of the reduced dimension and noise level. As the reduced dimension of the approximations increases, the approximation error decreases, while the generalization error increases. The generalization error increases with an increase in the noise level. In the case of a discrete evolution over a smooth, compact, connected, Riemannian manifold, we show that these two contributions to the error can be bounded in terms of the fill distance of centers of approximation and samples in the manifold

Keywords: Linear systems, Estimation, Identification
Abstract: In this paper, we analyze "sensitivity identifiability" of initial states and parameters in affinely parametrized linear systems. If the true parameter is sensitivity unidentifiable (non-SI), optimization-based estimation algorithms may face computational problems. Thus, it is important to detect whether the parameter is non-SI a priori. To this aim, we address a problem to find the condition that the parameter is non-SI for any initial state and input. Then, we obtain a sufficient condition for the problem. The condition is given by algebraic equations and is expected to be the foundation of structural conditions. We show systems that satisfy the condition and are observable and controllable.

Keywords: Linear systems, Estimation
Abstract: We consider the problems of system invertibility and input reconstruction for linear time-invariant (LTI) systems using only measured data. The two problems are connected in the sense that input reconstruction is possible provided that the system is left invertible. To verify the latter property without model knowledge, we leverage behavioral systems theory and develop two data-driven algorithms: one based on input/state/output data and the other based only on input/output data. We then consider the problem of input reconstruction for both noise-free and noisy data settings. In the case of noisy data, a statistical approach is leveraged to formulate the problem as a maximum likelihood estimation (MLE) problem. The proposed approaches are finally illustrated with numerical examples that show: exact input reconstruction in the noise-free setting; and the better performance of the MLE-based approach compared to the standard least-norm solution.

Keywords: Linear systems
Abstract: This paper presents a technique for designing two-parameter compensators that stabilize a plant and provide offset-free tracking of set-points without overshooting or undershooting. We first represent the impulse response of linear systems using combinatorial polynomials, based on which a new set of conditions is derived for the system to be externally positive. This result is then used in control synthesis to achieve monotonic tracking. In contrast to the methods available in the literature, the proposed technique always gives a solution whenever the problem is feasible, can yield as small a settling time as desired, and provides the freedom to choose the closed-loop poles arbitrarily inside the unit circle, all obtained by low-degree controllers.

Keywords: Linear systems, Identification for control, Stability of linear systems
Abstract: The exact pole placement problem concerns computing a static feedback law for a linear dynamical system that will assign its poles at a set of pre-specified locations. This is a classic problem in feedback control and numerous methodologies have been proposed in the literature for cases where a model of the system to control is available. In this paper, we study the problem of computing feedback laws for pole placement (and, more generally, eigenstructure assignment) directly from experimental data. Interestingly, we show that the closed-loop poles can be placed exactly at arbitrary locations without relying on any model description but by using only finite-length trajectories generated by the open-loop system. In turn, these findings imply that classical control goals, such as feedback stabilization or meeting transient transient performance specifications, can be achieved directly from data without first identifying a system model. Numerical experiments demonstrate the benefits of the data-driven pole-placement approach compared to its model-based counterpart.

Keywords: Linear systems
Abstract: A new notion of phase of multi-input multi-output (MIMO) systems was recently defined and studied, leading to new understandings in various fronts including a formulation of small phase theorem, a performance criterion named mathcal{H}_{infty} phase sector, and a sectored real lemma, etc. In this paper, we define a new notion of mathcal{H}_2^T-dissipativity and show the connection between the phase of a multivariable linear time-invariant (LTI) system and the mathcal{H}_2^T-dissipativity. The mathcal{H}_2^T-dissipativity, roughly speaking, is dissipativity restricted to the time-domain mathcal{H}_2 space which consists of mathcal{L}_2 signals with only positive frequency components. In addition, by exploiting the newly defined mathcal{H}_2^T-dissipativity, we also study the phase of a feedback system and provide a physical interpretation of the sectored real lemma.

Keywords: Linear systems, Lyapunov methods, Filtering
Abstract: Drawing samples from a given target probability distribution is a fundamental task in many science and engineering applications. A commonly used method for sampling is the Markov chain Monte Carlo (MCMC) which simulates a Markov chain whose stationary distribution coincides with the target one. In this work, we study the convergence and complexity of MCMC algorithms from a dynamic system point of view. We focus on the special cases with Gaussian target distributions and provide a Lyapunov perspective to them using tools from linear control theory. In particular, we systematically analyze two popular MCMC algorithms: Langevin Monte Carlo (LMC) and kinetic Langevin Monte Carlo (KLMC). By applying Lyapunov theory we derive tight complexity bounds to these algorithms. Our analysis also highlights subtle differences between sampling and optimization that could inform the more challenging task to sample from general distributions. Overall, our findings offer valuable insights for improving MCMC algorithms.

Keywords: Biological systems, Robust adaptive control, Estimation
Abstract: The control of neuronal networks, whether biological or neuromorphic, relies on tools for estimating parameters in the presence of model uncertainty. In this work, we explore the robustness of adaptive observers for neuronal estimation. Inspired by biology, we show that decentralization and redundancy help recover the performance of a centralized recursive mean square algorithm in the presence of uncertainty and mismatch on the internal dynamics of the model.

Keywords: Biological systems, Linear systems, Stochastic systems
Abstract: In this paper, we consider stable stochastic linear systems modeling whole-brain resting-state dynamics. We parametrize the state matrix of the system (effective connectivity) in terms of its steady-state covariance matrix (functional connectivity) and a skew-symmetric matrix S. We examine how the matrix S influences some relevant dynamic properties of the system. Specifically, we show that a large S enhances the degree of stability and excitability of the system, and makes the latter more responsive to high-frequency inputs.

Keywords: Cellular dynamics, Systems biology, Identification
Abstract: The immune system response to pathogens is organized by a network of cells communicating through expression of a variety of proteins and signaling molecules. A high number of genes are involved in encoding these communicating agents, but the relatively low number of data points is a major challenge in modelling the gene expression response. In this work we propose a feature-selection approach based on gene expression distributions at the single-cell level that improves dynamics identification at the population level. We investigate common approaches to differential expression analysis and show that Earth Mover's Distance (EMD) is a relatively robust measure for gene selection as reflected by the coefficient of variation as well as accuracy of a naive Bayes classifier based on the selected genes. We ultimately propose the bootstrap standard deviation metric as an estimate of state uncertainty and show that statistically significant signals in pathogen response can be recovered in the reduced state space constructed with the selected genes.

Keywords: Biomedical, Estimation, Optimization
Abstract: This paper presents an extended moving horizon observer to estimate both the states and the pharmacodynamic(PD) parameters of an anesthesia model, based on real data. The inputs of this model are the injection rates of Propofol and Remifentanil. The states represent the concentration of the anesthetic agents in different compartments of the human body (muscles, fat, blood) and in the effect site. The considered output is the Bispectral index (BIS) which is derived from the electroencephalogram (EEG). The observer is designed such that the parameters are estimated during the anesthesia induction phase, and then almost frozen for the rest of the surgery. The estimator is validated on real data that were extracted from the VitalDB database (Lee et al., 2022).

Keywords: Biomedical, Stability of hybrid systems, Stability of nonlinear systems
Abstract: Functional electrical stimulation (FES)-cycling is an effective method of rehabilitation for people with neuromuscular disorders. Muscle stimulation and electric motor inputs are designed to complement the rider’s volitional pedaling, but open challenges remain in the analysis of the stability and robustness of the human-machine system under the influence of switching between muscle and motor inputs. Discontinuous switching between muscle stimulation inputs and motor input motivates the use of a hybrid systems analysis, reducing gain conditions compared to a switched systems analysis and yielding robustness to disturbances. In this paper, repetitive learning control (RLC)-based feedforward terms for each muscle group and electric motor are designed to improve cadence tracking and reduce high-gain feedback terms that can cause chattering effects. Muscle stimulation limits are systematically considered for the safety and comfort of the rider, and a cadence controller is designed integrating RLC and robust control terms to account for input saturation. A passivity-based analysis ensures the hybrid system is flow output strictly passive from the rider’s volitional effort to the tracking error output. Moreover, the position and cadence tracking errors are shown to asymptotically converge based on a Lyapunov-like stability analysis.

Keywords: Healthcare and medical systems, Biological systems, Systems biology
Abstract: In 2022, worldwide mpox outbreaks have called attention to mpox virus infection and treatment opportunities using the drugs cidofovir and tecovirimat, which target different stages of in-host viral proliferation, respectively production and shedding. We propose a new model of in-host viral infection dynamics that distinguishes between the two stages, so as to explore the distinct effects of the two drugs, and we analyse the model properties and behaviour. Reducing the model order via timescale separation is shown to lead to the classical target-cell limited model, with a lumped viral proliferation rate depending on both production and shedding. We explicitly introduce the effect of the two drugs and we exemplify how to formulate and solve an optimal control problem that leverages the model dynamics to schedule optimal combined treatments.

Keywords: Constrained control, Adaptive control, Feedback linearization
Abstract: This paper extends our previous study on an explicit saturated control for a quadcopter, which ensures both constraint satisfaction and stability thanks to the linear representation of the system in the flat output space. The novelty here resides in the adaptivity of the controller’s gain to enhance the system’s performance without exciting its parasitic dynamics and avoid lavishing the input actuation with excessively high gain parameters. Moreover, we provide a thorough robustness analysis of the proposed controller when additive disturbances are affecting the system behavior. Finally, simulation and experimental tests validate the performances of the proposed controller.

Keywords: Constrained control, Adaptive control
Abstract: Maglev trains are levitated by magnetic forces to maintain a desired air gap between guideways and magnets. Although every two magnets are mechanically coupled via a levitation bogie, the existing controllers are usually designed only for each individual magnet, which may result in unstable air gaps or levitation failures. Differently, this paper proposes an adaptive safe backstepping scheme to collaboratively control two magnets of the same bogie. In particular, safety constraints are introduced to adaptive backstepping control via quadratic programs, which ensures the air gap within a permissible range and significantly reduces overshoots. Finally, simulation results validate the effectiveness of the proposed collaborative controller.

Keywords: Constrained control, Autonomous robots, Nonlinear systems
Abstract: We develop a novel adaptation-based technique for safe control design in the presence of multiple state constraints. Specifically, we introduce an approach for synthesizing any number of candidate control barrier functions (CBFs), each encoding a different state constraint, into one consolidated CBF (C-CBF) candidate. We then propose a parameter adaptation law for the weights of the C-CBF's constituent functions such that its controllable dynamics are non-vanishing. We prove that the adaptation law certifies the consolidated CBF candidate as valid for a class of nonlinear, control-affine, multi-agent systems, which permits its use in a quadratic program based control law. We highlight the success of our approach in simulation on a multi-robot goal-reaching problem in a warehouse environment, and further demonstrate its efficacy via a laboratory study with an AION ground rover operating amongst other vehicles behaving both aggressively and conservatively.

Keywords: Constrained control, Energy systems, Distributed control
Abstract: We coordinate interconnected agents where the control input of each agent is limited by the control input of others. In that sense, the systems have to share a limited resource over a network. Such problems can arise in different areas and it is here motivated by a district heating example. When the shared resource is insufficient for the combined need of all systems, the resource will have to be shared in an optimal fashion. In this scenario, we want the systems to automatically converge to an optimal equilibrium. The contribution of this paper is the proposal of a control architecture where each separate system is controlled by a local PI controller. The controllers are then coordinated through a global rank-one anti- windup signal. It is shown that the equilibrium of the proposed closed-loop system minimizes the infinity-norm of stationary state deviations. A proof of linear-domain passivity is given, and a numerical example highlights the benefits of the proposed method with respect to the state-of-the-art.

Keywords: Constrained control, Hybrid systems, Aerospace
Abstract: This paper presents extensions of control barrier function (CBF) and control Lyapunov function (CLF) theory to systems wherein all actuators cause impulsive changes to the state trajectory, and can only be used again after a minimum dwell time has elapsed. These rules define a hybrid system, wherein the controller must at each control cycle choose whether to remain on the current state flow or to jump to a new trajectory. We first derive a sufficient condition to render a specified set forward invariant using extensions of CBF theory. We then derive related conditions to ensure asymptotic stability in such systems, and apply both conditions online in an optimization-based control law with aperiodic impulses. We simulate both results on a spacecraft docking problem with multiple obstacles.

Keywords: Constrained control, Hybrid systems, Stability of nonlinear systems
Abstract: In this paper, we study the relationship between systems controlled via Control Barrier Function (CBF) approaches and a class of discontinuous dynamical systems, called Projected Dynamical Systems (PDSs). In particular, under appropriate assumptions, we show that the vector field of CBF-controlled systems is a Krasovskii-like perturbation of the set-valued map of a differential inclusion, that abstracts PDSs. This result provides a novel perspective to analyze and design CBF-based controllers. Specifically, we show how, in certain cases, it can be employed for designing CBF-based controllers that, while imposing safety, preserve asymptotic stability and do not introduce undesired equilibria or limit cycles. Finally, we briefly discuss about how it enables continuous implementations of certain projection-based controllers, that are gaining increasing popularity.

Keywords: Stochastic optimal control, Adaptive control, Machine learning
Abstract: The paper introduces the first formulation of convex Q-learning for Markov decision processes with function approximation. The algorithms and theory rest on a relaxation of a dual of Manne's celebrated linear programming characterization of optimal control. The main contributions firstly concern properties of the relaxation, described as a deterministic convex program: we identify conditions for a bounded solution, a significant connection between the solution to the new convex program, and the solution to standard Q-learning with linear function approximation. The second set of contributions concern algorithm design and analysis: (i) A direct model-free method for approximating the convex program for Q-learning shares properties with its ideal. In particular, a bounded solution is ensured subject to a simple property of the basis functions; (ii) The proposed algorithms are convergent and new techniques are introduced to obtain the rate of convergence in a mean-square sense;

(iii) The approach can be generalized to a range of performance criteria, and it is found that variance can be reduced by considering ``relative'' dynamic programming equations; (iv) The theory is illustrated with an application to a classical inventory control problem.

Keywords: Stochastic optimal control, Control over communications, Networked control systems
Abstract: We address the asymptotic problem of signalling information from one controller to another controller, in a series network, consisting of control system 1 (CS-1) and control system 2 (CS-2), as shown in the Figure, first analyzed in [1] for finite-horizon. Controller 2 of CS-2 has access to feedback information from its output, while controller 1 of CS-1 does not have access to feedback information from its output. Under suitable detectability and stabilizability conditions of matrix algebraic Riccati equations (AREs), it is shown that, if the rate of generating information by CS-1 is below the asymptotic control-coding (CC) capacity of CS-2, then we can synthesize, i) a controller-encoder for CS-2 that simultaneously controls the CS-2 and encodes the state of the CS-1, and operates at the CC capacity of CS-2, ii) a decoder for CS-2 that is optimal with respect to a mean-square error (MSE) criterion, and iii) a controller for CS-1, which acts on the decoder output, and it is optimal with respect to the pay-off of CS-1. Compared to [1], this paper includes bounds to MSE and Error probability of communicating digital messages.

Keywords: Stochastic optimal control, Iterative learning control, Nonlinear output feedback
Abstract: This paper studies the learning-to-control problem under process and sensing uncertainties for dynamical systems. In our previous work, we developed a data-based generalization of the iterative linear quadratic regulator (iLQR) to design closed-loop feedback control for high-dimensional dynamical systems with partial state observation. This method required perfect simulation rollouts which are not realistic in real applications. In this work, we briefly introduce this method and explore its efficacy under process and sensing uncertainties. We prove that in the fully observed case where the system dynamics are corrupted with noise but the measurements are perfect, it still converges to the global minimum. However, in the partially observed case where both process and measurement noise exist in the system, this method converges to a biased ``optimum". Thus multiple rollouts need to be averaged to retrieve the true optimum. The analysis is verified in two nonlinear robotic examples simulated in the above cases.

Keywords: Stochastic optimal control, Iterative learning control, Optimal control
Abstract: Optimal control is notoriously difficult for stochastic nonlinear systems. Ren et.al. 2022 introduced Spectral Dynamics Embedding for developing reinforcement learning methods for controlling an unknown system. It uses an infinite-dimensional feature to linearly represent the state-value function and exploits finite-dimensional truncation approximation for practical implementation. However, the finite-dimensional approximation properties in control have not been investigated even when the model is known. In this paper, we provide a tractable stochastic nonlinear control algorithm that exploits the nonlinear dynamics upon the finite-dimensional feature approximation, Spectral Dynamics Embedding Control (SDEC), with an in-depth theoretical analysis to characterize the approximation error induced by the finite-dimension truncation and statistical error induced by finite-sample approximation in both policy evaluation and policy optimization. We also empirically test the algorithm and compare the performance with Koopman-based methods and iLQR methods on the pendulum swingup problem.

Keywords: Stochastic optimal control, Large-scale systems, Decentralized control
Abstract: We consider a large-population optimal control problem involving a major agent and a large number of minor agents. By starting with a centralized optimal control problem, we employ a re-scaling method to derive decentralized control laws. This re-scaling method is further used to obtain a tight upper bound of O(1/N) for the performance loss resulting from decentralized control. This improves upon known results of O(1/sqrt{N}) in the literature for similar models.

Keywords: Stochastic optimal control, Machine learning, Iterative learning control
Abstract: We revisit the standard formulation of tabular actor-critic algorithm as a two time-scale stochastic approximation with value function computed on a faster time-scale and policy computed on a slower time-scale. This emulates policy iteration. We begin by observing that reversal of the time scales will in fact emulate value iteration and is a legitimate algorithm. We provide a proof of convergence and compare the two empirically with and without function approximation (with both linear and nonlinear function approximators) and observe that our proposed critic-actor algorithm performs on par with actor-critic in terms of both accuracy and computational effort.

Keywords: Cyber-Physical Security, Attack Detection, Machine learning
Abstract: We introduce the problem of model-extraction attacks in cyber-physical systems in which an attacker attempts to estimate (or extract) the feedback controller of the system. Extracting (or estimating) the controller provides an unmatched edge to attackers since it allows them to predict the future control actions of the system and plan their attack accordingly. Hence, it is important to understand the ability of the attackers to perform such an attack. In this paper, we focus on the setting when a Deep Neural Network (DNN) controller is trained using Reinforcement Learning (RL) algorithms and is used to control a stochastic system. We play the role of the attacker that aims to estimate such an unknown DNN controller, and we propose a two-phase algorithm. In the first phase, also called the offline phase, the attacker uses side-channel information about the RL-reward function and the system dynamics to identify a set of candidate estimates of the unknown DNN. In the second phase, also called the online phase, the attacker observes the behavior of the unknown DNN and uses these observations to shortlist the set of final policy estimates. We provide theoretical analysis of the error between the unknown DNN and the estimated one. We also provide numerical results showing the effectiveness of the proposed algorithm.

Keywords: Cyber-Physical Security, Autonomous vehicles, Traffic control
Abstract: Recent studies reveal that Autonomous Vehicles (AVs) can be manipulated by hidden backdoors, causing them to perform harmful actions when activated by physical triggers. However, it is still unclear how these triggers can be activated while adhering to traffic principles. Understanding this vulnerability in a dynamic traffic environment is crucial. This work addresses this gap by presenting physical trigger activation as a reachability problem of controlled dynamic system. Our technique identifies security-critical areas in traffic systems where trigger conditions for accidents can be reached, and provides intended trajectories for how those conditions can be reached. Testing on typical traffic scenarios showed the system can be successfully driven to trigger conditions with near 100% activation rate. Our method benefits from identifying AV vulnerability and enabling effective safety strategies.

Keywords: Cyber-Physical Security, Computer/Network Security, Estimation
Abstract: In this paper, we investigate an optimal strategy for malicious agents to compromise remote state estimators where only a portion of the transmitted packets are secured. First, the analysis of the performance evolution and stealthiness properties of innovation-based linear attacks that can compromise unsafe transmission channels is provided. An optimal attack policy is then derived by numerically solving an optimization problem step by step. Different from the scenario where all links are vulnerable to cyber-attacks, the existence of secured channels poses a tighter stealthiness constraint and thus can significantly reduce the worst-case attack impact. Additionally, it is shown that the well-studied flipping-sign-attack in existing work cannot remain stealthy. Finally, a numerical example and comparative studies are included to verify the effectiveness of the proposed method.

Keywords: Cyber-Physical Security, Game theory, Fault tolerant systems
Abstract: We consider the observation of a random, binary environment state via a set of sensing nodes connected through a ring lattice. Each node obtains a correct observation with a positive probability and broadcasts its observation to its neighbors. A system operator selects a consensus threshold for the number of consistent observations, and a consensus is reached when any node has accumulated sufficient consistent observations. A Byzantine attacker can manipulate a certain number of nodes to broadcast misleading information, and thus prohibit a correct consensus. We formulate this problem as a zero-sum game and analyze the equilibria. We show that the attacker has a dominant strategy for selecting the nodes to manipulate and the information to broadcast/block. We show that, unless the attacker's budget is abundant, the system operator can select an optimal consensus threshold to balance between the chance of a correct consensus and the risk of a wrong consensus. We also use the equilibrium structure to characterize the network resiliency, i.e., the minimal number of Byzantine nodes that would eliminate the chance of a correct consensus, given the size and connectivity of the ring lattice. The results are relevant for hardware surveillance, infrastructure inspection, disaster response, etc.

Keywords: Cyber-Physical Security, Networked control systems, Kalman filtering
Abstract: Security issues are of significant importance for cyber-physical systems (CPS), where the attack design is a major concern. Most related studies on attack design implicitly consider that the control period and detection period are the same. However, the two periods could be different in practical systems with remote detection such as SCADA systems, which could lead to new vulnerabilities for attackers. In this paper, we consider the design of innovation-based linear attack strategies for CPS when the control period and detection period are inconsistent. Specifically, we propose an attack framework that consists of attack strategies for detection and non-detection instants under the period discrepancy. On this basis, we design the optimal stealthy innovation-based linear attack strategies for state estimation and LQG control to maximize the estimation error or control cost, respectively. Simulations are given to demonstrate the effectiveness of the proposed attack strategies.

Keywords: Cyber-Physical Security, Networked control systems, Sampled-data control
Abstract: This paper focuses on cyber-security issues of networked control systems in closed-loop forms from the perspective of quantized sampled-data systems. As sampling can introduce non-minimum phase zeros in discretized systems, we consider zero dynamics attacks, which is a class of false data injection attacks. Quantization of control inputs disables such attacks to be made exactly, resulting in certain errors in the system output. Specifically, we characterize a trade-off relation between attack performance and stealthiness, and then show that the attacker can reduce the output error with a modified approach by considering the quantization error of the attack signal. We provide a numerical example to demonstrate the effectiveness of the proposed approaches.

Keywords: Stability of linear systems, Robust control
Abstract: This paper is concerned with event-triggered robust static output-feedback stabilization of the second-order linear uncertain systems by a fast-varying square wave with high gain. Recently, a constructive time-delay approach for designing such a fast-varying output-feedback controller was suggested by using continuous measurements. In the present paper, we employ an event-trigger (ET) based on switching approach that determines the measurement transmission instants for this design. For the resulting switching system, we construct appropriate coordinate transformations that cancel the high gains and apply the time-delay approach to periodic averaging of the system in new coordinates. By employing appropriate Lyapunov functionals, we derive linear matrix inequalities (LMIs) for finding an efficient upper bound on the square wave frequency that guarantees the stability of the original systems. Numerical examples illustrate the efficiency of the method.

Keywords: Hybrid systems, Networked control systems, Control over communications
Abstract: We present rules to stabilize the origin of a networked system, where data exchanges between the plant and the controller only occur when an output-dependent inequality has been satisfied for a given amount of time. This strategy, called Event-Holding Control (EHC), differs from time-regularized event-triggered control (ETC) techniques, which generate transmissions as soon as a triggering condition is verified and the time elapsed since the last transmission is larger than a given bound. Indeed, the clock involved in EHC is not running continuously after each transmission instant, but only when a criterion is verified. We propose an output-based design of these triggering mechanisms that are robust to additive measurement noise and ensure an input-to-state stability (ISS) property. This EHC scheme naturally has a positive lower bound on the transmission interval. Additionally, we show via an example that, in presence of measurement noise, Zeno-like behavior, where events are generated near the minimum inter-event time consistently, may occur when the system is close to the attractor. We introduce space-regularization to mitigate this issue, resulting in an input-to-state practical stability (ISpS) property rather than ISS.

Keywords: Supervisory control, Hybrid systems, Fault accomodation
Abstract: For a constrained nonlinear control system, an automated supervisor is proposed that determines switching between a barrier function–certified controller and an uncertified controller. The switching strategy allows for properties of the uncertified controller to be exploited while preserving the forward invariance that is guaranteed by the barrier function for the certified controller. Tunable threshold functions determine regions of the state space where the supervisor switches between controllers. Conditions are given to prevent chattering by establishing a positive minimum time between switches. An example illustrates achieving forward invariance despite using an uncertified MPC controller with delayed computations.

Keywords: Discrete event systems, Hybrid systems, Nonlinear systems
Abstract: In balancing safety with the nominal control objectives, e.g., stabilization, it is desirable to reduce the time period when safety filters are in effect. Inspired by traditional spacecraft maneuvers, and with the ultimate goal of reducing the duration when safety is of concern, this paper proposes an event-triggered control framework with switching state-based triggers. Our first trigger in the scheme monitors safety constraints encoded by barrier functions, and thereby ensures safety without the need to alter the nominal controller---and when the boundary of the safety constraint is approached, the controller drives the system to the region where control actions are not needed. The second trigger condition determines if the safety constraint has improved enough for the success of the first trigger. We begin by motivating this framework for impulsive control systems, e.g., a satellite orbiting an asteroid. We then expand the approach to more general nonlinear system through the use of safety filtered controllers. Simulation results demonstrating satellite orbital maneuvers illustrate the utility of the proposed event-triggered framework.

Keywords: Discrete event systems, Kalman filtering, Networked control systems
Abstract: This paper addresses the problem of remote state estimation subject to packet dropouts, focusing on the use of an event-triggered sensor scheduler to conserve communication resources. However, packet dropouts introduce significant challenges, as the remote estimator cannot distinguish between packet loss caused by poor channel conditions and packet loss due to the event trigger. To overcome this issue, we propose a novel formulation that incorporates error-detecting codes. We prove that the Gaussian property, commonly assumed in the literature, does not hold in this setup, and instead, the system state follows an extended Gaussian mixture model (GMM). We present an exact minimum mean-squared error (MMSE) estimator and an approximate estimator, which significantly reduces algorithm complexity without sacrificing performance. Our simulation results show that the approximate estimator achieves nearly the same performance as the exact estimator while requiring much less computation time. Moreover, the proposed event trigger outperforms existing schedulers in terms of estimation accuracy.

Keywords: Networked control systems, Stochastic systems
Abstract: We investigate the scenario where a controller communicates with a nonlinear plant via a wireless erasure channel. We present an event-based control strategy to stabilize the plant while sporadically using the unreliable wireless network. In particular, control packets may be lost at any time with a certain probability. Consequently, stability is ensured in a stochastic sense. We then compare the proposed event-based policy with a baseline policy that transmits according to the age of information, i.e., the time elapsed since the last successful reception. For any given baseline policy, we show how to design an event-based policy that ensures the same guaranteed control performance while leading, on average, to a strictly smaller channel utilization. Numerical simulations suggest that the achieved channel utilization may in fact be significantly smaller.

Keywords: Stability of nonlinear systems, Neural networks, Optimization
Abstract: This paper investigates stability conditions of continuous-time Hopfield and firing-rate neural networks by leveraging contraction theory. First, we present a number of useful general algebraic results on matrix polytopes and products of symmetric matrices. Then, we give sufficient conditions for strong and weak Euclidean contractivity, i.e., contractivity with respect to the ell_2 norm, of both models with symmetric weights and (possibly) non-smooth activation functions. Our contraction analysis leads to contraction rates which are log-optimal in almost all symmetric synaptic matrices. Finally, we use our results to propose a firing-rate neural network model to solve a quadratic optimization problem with box constraints.

Keywords: Stability of nonlinear systems, Variational methods, LMIs
Abstract: Recently, the concept of p-dominance has been proposed as a unified framework to study rich behaviors of nonlinear systems. In this paper, we consider finding a set of linear dynamic output feedback controllers rendering the closed-loop systems p-dominant. We first derive an existence condition. Based on this condition, we then provide a parametrization of controllers. For Lure's systems, the proposed method can be applied only by solving a finite family of linear matrix inequalities, which is illustrated by achieving multi-stabilization and stabilization of a limit cycle.

Keywords: Observers for nonlinear systems, Estimation, Autonomous vehicles
Abstract: Navigation and exploration within unknown environments are typical examples in which simultaneous localization and mapping (SLAM) algorithms are applied. When mobile agents deploy only range sensors without bearing information, the agents must estimate the bearing using the online distance measurement for the localization and mapping purposes. In this paper, we propose a scalable dynamic bearing estimator to obtain the relative bearing of the static landmarks in the local coordinate frame of a moving agent in real-time. Using contraction theory, we provide convergence analysis of the proposed range-only bearing estimator and present upper and lower-bound for the estimator gain. Numerical simulations demonstrate the effectiveness of the proposed method.

Keywords: Switched systems, Stability of hybrid systems, Hybrid systems
Abstract: Incremental input-to-state stability plays an important role in the analysis of nonlinear systems, as it opens up the possibility for accurate performance characterizations beyond classical approaches. In this paper, we are interested in deriving conditions for incremental stability of a specific class of discontinuous dynamical systems containing a so-called hybrid integrator. Recently, it was shown that hybrid integrators have the potential for overcoming fundamental performance limitations of linear time-invariant control, thereby making them interesting for use in, e.g., high-precision motion control applications. The main contribution of this paper is to show that these hybrid integrators have incremental input-to-state stability properties, and that, under an incremental small-gain condition, the feedback interconnection of a hybrid integrator and a linear time-invariant plant is incrementally input-to-state stable.

Keywords: Switched systems, Differential-algebraic systems, Stability of nonlinear systems
Abstract: In this article the solvability analysis of discrete-time nonlinear singular switched systems with restricted switching signals is studied. We provide necessary and sufficient conditions for the solvability analysis under fixed switching signals and fixed mode sequences. The so-called surrogate systems (ordinary systems that have the equivalent behavior to the original switched systems) are introduced for solvable switched systems. Incremental stability, which ensures that all solution trajectories converge with each other, is then studied by utilizing these surrogate systems. Sufficient (and necessary) conditions are provided for this stability analysis using single and switched Lyapunov function approaches.

Keywords: Stability of nonlinear systems
Abstract: This paper studies the celebrated Kuramoto-Sakaguchi model of coupled oscillators adopting two recent concepts. First, we consider appropriately-defined subsets of the n-torus called winding cells. Second, we analyze the semicontractivity of the model, i.e., the property that the distance between trajectories decreases when measured according to a seminorm.

This paper establishes the local semicontractivity of the Kuramoto-Sakaguchi model, which is equivalent to the local contractivity for the reduced model. The reduced model is defined modulo the rotational symmetry. The domains where the system is semicontracting are convex phase-cohesive subsets of winding cells. Our sufficient conditions and estimates of the semicontracting domains are less conservative and more explicit than in previous works. Based on semicontraction on phase-cohesive subsets, we establish the "at most uniqueness" of synchronous states within these domains, thereby characterizing the multistability of this model.

Keywords: Delay systems, Agents-based systems, Adaptive control
Abstract: Active inference (AIF) as a comprehensive theory has been proven that it is promising in state estimation and adaptive control of uncertain systems. However, the input delay in the controller was ignored in the normal framework. When taking input delay into consideration in the uncertain system, the optimal estimation state in the normal AIF differs greatly from the real state of the system due to the accumulation of the delay effect. Therefore, delay feedback active inference(D- AIF) is proposed in this paper. Different from normal AIF, a predictive state based on the delay state becomes the expectation of the state of normal AIF. Meanwhile, the expectation of the controller becomes an epitaxial delayed feedback Proportional- Integral (PI) control. The variational free energy (VFE) is extended by adding a quadratic of control consumption, In order to minimize the variational free energy, the model uncertainty and measurement uncertainty are compensated by approximate Gaussian distributions. Surprisingly, the state estimation does not depend on the given target state in D-AIF. When AIF and D-AIF are applied for trajectory tracking of an unmanned aircraft with input delay, it can be seen from the results that delay feedback active inference control (D-AIFC) can accurately estimate the current state and has stronger robustness when dealing with sudden disturbance than active inference control (AIFC).

Keywords: Delay systems, Autonomous vehicles, Stability of nonlinear systems
Abstract: We construct a nonlinear predictor-feedback Cooperative Adaptive Cruise Control (CACC) design for homogeneous vehicular platoons subject to actuators delays, which achieves: i) positivity of vehicles' speed and spacing states, ii) mathcal{L}_{infty} string stability of the platoon, iii) stability of each individual vehicular system, and iv) tracking of a constant reference speed (dictated by the leading vehicle) and spacing. The design relies on a nominal, nonlinear control law, which guarantees i)--iv) in the absence of actuator delay, and nonlinear predictor feedback. We consider a second-order, nonlinear vehicle model with input delay. The proofs of the theoretical guarantees i)--iv) rely on derivation of explicit estimates on solutions (both during open-loop and closed-loop operation), capitalizing on the ability of predictor feedback to guarantee complete delay compensation after the dead-time interval has elapsed, and derivation of explicit conditions on initial conditions and parameters of the nominal control law. We also present consistent simulation results.

Keywords: Delay systems, Data driven control, Linear systems
Abstract: We analyze in this paper the effect of the well-known intelligent proportional controller on the stability of linear control systems. Inspired by the literature on neutral time-delay systems and advanced-type systems, we derive sufficient conditions on the order of the control system, under which, the used controller fails to achieve exponential stability. Furthermore, we obtain conditions, relating the system’s and the control parameters, such that the closed-loop system is either unstable or not exponentially stable. After that, we provide cases where the used controller achieves exponential stability. The obtained results are illustrated on an experimental benchmark that consists of an electronic throttle valve.

Keywords: Delay systems, Distributed control, LMIs
Abstract: This paper investigates the synchronization problem for generic linear multi-agent systems with known or unknown heterogeneous input and communication delays. We propose two protocols that consist of consensus-based internal controller states and decentralized controllers. This kind of distributed dynamic control methodology is able to circumvent the interactive nature of two delays by translating the synchronization problem of agents into the stability of a set of delay differential equations. We examine the synchronization problem for two distinct cases, namely, known delays and unknown delays. When the delays are known, the stability criteria are satisfied by the feasibility of an input-delay-dependent linear matrix inequality and a communication-delay-dependent coupling strength bound. The margin on the communication delay is dependent not only on the network topology but also on the system matrix, which does not have any eigenvalues with positive real parts. We also develop a distributed dynamic control protocol that can handle unknown input and communication delays, and the stability criteria are realized by using the feasibility of a linear matrix inequality and a positive coupling strength. Synchronization is guaranteed even if the unknown communication delays are arbitrarily large but bounded and the upper bound on the heterogeneous input delays is known. The proposed control methodology guarantees that inaccurate measurements of the actual states of a particular agent will not lead to an irretrievable failure of the mission.

Keywords: Delay systems, Mean field games, Stochastic optimal control
Abstract: In this study, we consider linear-quadratic (LQ) mean-field social control problems for a class of stochastic systems with ordinary control input and delay control input. We define a stabilization problem via a memoryless static output feedback (SOF) strategy and then solve the problem of minimizing the upper bound of the cost function using guaranteed cost control theory. It is found that the minimization of the upper bound of the cost function cannot be attained if only a delay control input exists. Futhermore, it is proved that it is impossible to implement a mean-field SOF strategy to solve the minimization problem, and the input matrix must have the same dimension as the state matrix. To solve this minimiztion problem, the necessary conditions for the sub-optimality are established via stochastic cross-coupled matrix equations (SCCMEs) using the Karush-Kuhn-Tucker condition and the state feedback strategy. Finally, the performance and usefulness of the proposed strategy are investigated using an order-reduced scheme based on the direct method.

Keywords: Delay systems, Nonlinear systems, Stability of nonlinear systems
Abstract: For a second-order system with time delays and power nonlinearity of the degree higher than one, which does not contain a velocity-proportional damping term, the conditions of local asymptotic stability of the zero solution are proposed. The result is based on application of the Lyapunov-Razumikhin approach, and it is illustrated by simulations. Our local stability conditions for nonlinear systems are less restrictive than stability conditions of the corresponding linear models.

Keywords: Statistical learning, Identification, Estimation
Abstract: Machine learning methods are being integrated at an ever-increasing rate into domains that have traditionally been within the purview of controls. There is a wide range of examples, including perception-based control, agile robotics, and autonomous driving and racing. As exciting as these developments may be, they have been most pronounced on the experimental and empirical sides. To deploy these systems safely, stably, and robustly into the real world, we argue that a principled and integrated theoretical understanding of a) fundamental limitations and b) statistical optimality is needed. In the past few years, a host of new techniques have been introduced to our field. Existing results in this area are relatively inaccessible to a typical first- or second-year graduate student in control theory, as they require sophisticated mathematical tools not typically included in a control theorist’s training (e.g., high-dimensional statistics and learning theory). In the first talk, we present the scope of our tutorial, which is to introduce recently developed non-asymptotic methods in the theory of (mainly linear) system identification to a wider audience. Our aim is to give simple proofs of the main developments and to highlight and collect the key technical tools to arrive at these results. We present the roadmap of the tutorial, offering a high-level preview of these tools, which lie at the intersection of Non-asymptotic Statistics and Control Theory. We emphasize that existing results for linear systems rely on the non asymptotic analysis of the least squares estimator under time-dependent data.

Keywords: Statistical learning, Identification
Abstract: In this talk we introduce the basic technical machinery - concentration inequalities- underlying much of modern statistical learning theory. We discuss how the Chernoff trick and notions such as sub-Gaussianity allow for a relatively satisfying non-asymptotic analogue to classical asymptotics. We conclude by introducing a more advanced concentration inequality, the Hanson- Wright inequality, which provides estimate of the deviation of certain random quadratic forms. This inequality will be key in the following talks.

Keywords: Statistical learning, Estimation, Identification
Abstract: In this talk we present a streamlined approach to control the lower spectrum of certain empirical covariance matrices of Causal Linear sub-Gaussian processes. The lower tail of this empirical covariance matrix arises as one of the two key terms in the canonical error decomposition of the least squares estimator. Put differently, this technique allows to quantify persistency of excitation in a non-asymptotic fashion for host of interesting processes, including for instance linear dynamical systems. The proof technique we present here is novel to the literature and only uses the Hanson-Wright Inequality and basic covering arguments.

Keywords: Statistical learning, Estimation, Identification
Abstract: In this talk, we present a brief overview of the theory of self-normalized processes as used in the analysis of least squares over causally dependent data. The self-normalized martingale arises as one of two key terms in the decomposition of the least squares parameter estimation error. We first review the relevant definitions and assumptions required to derive a (almost) time-scale invariant bound on this self-normalized martingale term. We then provide an interpretation of the result, along with a roadmap for how it is applied in the analysis of system identification. We conclude by sketching a proof for the result by employing an approach known as pseudo-maximization.

Keywords: Statistical learning, Identification, Estimation
Abstract: In this talk, we analyze well-known linear system identification algorithms that rely on the least squares algorithm. First, we focus on the identification of autoregressive systems with exogenous inputs, also known as ARX systems. By utilizing the tools presented in previous talks and contained in our tutorial paper, we establish a non-asymptotic, high-probability bound on the performance of the least-squares estimator. The bound describes how the system identification accuracy changes with the number of samples, the required confidence, and system-theoretic properties such as system size/dimension and signal-to-noise ratio. We also establish non-asymptotic persistency of excitation, that is, excitation of all the modes of the system with high probability, which is crucial for system identification. Next, we analyze the system identification of Markov parameters of statespace systems by reducing the problem to (approximate) ARX identification, which is a fundamental step for many subspace identification algorithms.

Keywords: Statistical learning, Identification, Estimation
Abstract: In this talk we outline some recent advances beyond the linear setting, focusing on finite hypothesis classes for simplicity. We illustrate how the two key concepts—persistency of excitation and control of a random walk-type quantity—for proving learning rates with the square loss function generalize to the nonlinear setting. In the nonlinear setting, the lower uniform estimate can be seen as a generalized notion of persistency of excitation, whereas a variational perspective of the empirical least squares error allows for a development very much in parallel to the self-normalized theory discussed in the fourth talk.

Keywords: Statistical learning, Machine learning, Learning
Abstract: Safety is an essential asset when learning control policies for physical systems, as violating safety constraints during training can lead to expensive hardware damage. In response to this need, the field of safe learning has emerged with algorithms that can provide probabilistic safety guarantees without knowledge of the underlying system dynamics. Those algorithms often rely on Gaussian process inference. Unfortunately, Gaussian process inference scales cubically with the number of data points, limiting applicability to high-dimensional and embedded systems. In this paper, we propose a safe learning algorithm that provides probabilistic safety guarantees but leverages the Nadaraya-Watson estimator instead of Gaussian processes. For the Nadaraya-Watson estimator, we can reach logarithmic scaling with the number of data points. We provide theoretical guarantees for the estimates, embed them into a safe learning algorithm, and show numerical experiments on a simulated seven-degrees-of-freedom robot manipulator.

Keywords: Learning, Nonlinear output feedback, Robust control
Abstract: This paper presents a policy parameterization for learning-based control on nonlinear, partially-observed dynamical systems. The parameterization is based on a nonlinear version of the Youla parameterization and the recently proposed Recurrent Equilibrium Network (REN) class of models. We prove that the resulting Youla-REN parameterization automatically satisfies stability (contraction) and user-tunable robustness (Lipschitz) conditions on the closed-loop system. This means it can be used for safe learning-based control with no additional constraints or projections required to enforce stability or robustness. We test the new policy class in simulation on two reinforcement learning tasks: 1) magnetic suspension, and 2) inverting a rotary-arm pendulum. We find that the Youla-REN performs similarly to existing learning-based and optimal control methods while also ensuring stability and exhibiting improved robustness to adversarial disturbances.

Keywords: Robust adaptive control, Adaptive control, Robust control
Abstract: We present an online learning analysis of minimax adaptive control for the case where the uncertainty includes a finite set of linear dynamical systems. Precisely, for each system inside the uncertainty set, we define the model-based regret by comparing the state and input trajectories from the minimax adaptive controller against that of an optimal controller in hindsight that knows the true dynamics. We then define the total regret as the worst case model-based regret with respect to all models in the considered uncertainty set. We study how the total regret accumulates over time and its effect on the adaptation mechanism employed by the controller. Moreover, we investigate the effect of the disturbance on the growth of the regret over time and draw connections between robustness of the controller and the associated regret rate.

Keywords: Machine learning, Learning, Uncertain systems
Abstract: Humans have the ability to deviate from their natural behavior when necessary, which is a cognitive process called response inhibition. Similar approaches have independently received increasing attention in recent years for ensuring the safety of control. Realized using control barrier functions or predictive safety filters, these approaches can effectively ensure the satisfaction of state constraints through an online adaptation of nominal control laws, e.g., obtained through reinforcement learning. While the focus of these realizations of inhibitory control has been on risk-neutral formulations, human studies have shown a tight link between response inhibition and risk attitude. Inspired by this insight, we propose a flexible, risk-sensitive method for inhibitory control. Our method is based on a risk-aware condition for value functions, which guarantees the satisfaction of state constraints. We propose a method for learning these value functions using common techniques from reinforcement learning and derive sufficient conditions for its success. By enforcing the derived safety conditions online using the learned value function, risk-sensitive inhibitory control is effectively achieved. The effectiveness of the developed control scheme is demonstrated in simulations.

Keywords: Networked control systems, Linear systems, Stochastic systems
Abstract: A learning-based safety filter is developed for discrete-time linear time-invariant systems with unknown models subject to Gaussian noises with unknown covariance. Safety is characterized using polytopic constraints on the states and control inputs. The empirically learned model and process noise covariance with their confidence bounds are used to construct a robust optimization problem for minimally modifying nominal control actions to ensure safety with high probability. The optimization problem relies on tightening the original safety constraints. The magnitude of the tightening is larger at the beginning since there is little information to construct reliable models, but shrinks with time as more data becomes available.

Keywords: Optimization
Abstract: This paper is concerned with the formal synthesis of safety controllers

for partially-observable discrete-time control systems with unknown mathematical models. Given a state estimator with unknown dynamics but a known upper bound on the estimation error, we propose a data-driven approach to compute controllers that render the partially-observable systems with unknown dynamics safe. Our proposed method is based on the

construction of control barrier certificates, where we first formulate the barrier-based safety problem as a robust program (RP). The proposed

RP is not tractable since the unknown model of the estimator appears in

one of its constraints. To tackle this issue, we collect a set of data from the black-box system and its estimator and replace the original RP

with a scenario program (SP). Due to the existence of a max-min constraint in the SP, we construct an analogous scenario program, denoted by SP^alpha, in which the max-min constraint is replaced with a single inequality constraint. The control barrier certificates together with their corresponding controllers can then be computed by solving SP^alpha via the collected data. By connecting the feasible solutions of SP^alpha and SP, the safety of the partially-observable

system equipped with the synthesized controller can be guaranteed with 100% confidence. We show the effectiveness of our results by synthesizing a safety controller for a partially-observable Van der Pol

oscillator with unknown dynamics.

Keywords: Optimization, Numerical algorithms
Abstract: Finite-sum minimization is a fundamental optimization problem in signal processing and machine learning. This paper proposes a variance-reduced shuffling gradient descent with Nesterov’s momentum for smooth convex finite-sum optimization. We integrate an explicit variance reduction into the shuffling gradient descent to deal with the variance introduced by shuffling gradients. The proposed algorithm with a unified shuffling scheme converges at a rate of O(1/T), where T is the number of epochs. The convergence rate independent of gradient variance is better than most existing shuffling gradient algorithms for convex optimization. Finally, numerical simulations demonstrate the convergence performance of the proposed algorithm.

Keywords: Network analysis and control, Optimization algorithms, Cooperative control
Abstract: This paper considers online distributed convex constrained optimization

over a time-varying multi-agent network. Agents in this network cooperate to minimize the global objective function through information

exchange with their neighbors and local computation. Since the capacity

or bandwidth of communication channels often is limited, a random quantizer is introduced to reduce the transmission bits. Through incorporating this quantizer, we develop a quantized distributed online projection-free optimization algorithm, which can achieve the saving of communication resources and computational costs. The dynamic

regret bound mathcal{O}( max{T^{1-gamma}, T^{gamma}(1+H_T) } +D_T) of the proposed algorithm is rigorously established, which depends on the total time T, function variation H_T, gradient variation D_T, and the parameter 0

Keywords: Optimization algorithms, Game theory, Predictive control for linear systems
Abstract: This paper presents a distributed methodology to produce collision-free control laws for an Unmanned Aerial Vehicles (UAVs) fleet. We use a game theoretic framework, where UAVs accommodate for individual and fleet goals, while respecting safety requirements. The method combines Control Barrier Functions (CBFs) and a primal-dual algorithm for Nash equilibrium (NE) seeking in generalized games. Feedback is introduced by Model Predictive Control (MPC) and we analyze its stability properties. The combination of these tools allows for a distributed, collision-free pointwise equilibrium solution, despite the agents’ coupling, due to common target tracking and the collision avoidance constraints. Our algorithmic results are supported theoretically and its efficacy is demonstrated via extensive numerical simulations.

Keywords: Game theory
Abstract: In this paper, the generalized Nash equilibrium (GNE) seeking problem for continuous games with coupled affine inequality constraints is investigated in a partial-decision information scenario, where each player can only access its neighbors' information through local communication although its cost function possibly depends on all other players' strategies. To this end, a novel decentralized primal-dual algorithm based on consensus and dual diffusion methods is devised for seeking the variational GNE of the studied games. This paper also provides theoretical analysis to show that the designed algorithm converges linearly for the last-iterate, which, to our best knowledge, is the first to propose a linearly convergent GNE seeking algorithm under coupled affine inequality constraints. Finally, a numerical example is presented to demonstrate the efficiency of the obtained theoretical results.

Keywords: Optimization algorithms, Optimization
Abstract: Existing asynchronous distributed optimization algorithms often use diminishing step-sizes that cause slow practical convergence, or fixed step-sizes that depend on an assumed upper bound of delays. Not only is such a delay bound hard to obtain in advance, but it is also large and therefore results in unnecessarily slow convergence. This paper develops asynchronous versions of two distributed algorithms, DGD and DGD-ATC, for solving consensus optimization problems over undirected networks. In contrast to alternatives, our algorithms can converge to the fixed point set of their synchronous counterparts using step-sizes that are independent of the delays. We establish convergence guarantees under both partial and total asynchrony. The practical performance of our algorithms is demonstrated by numerical experiments.

Keywords: Numerical algorithms, Computational methods, Optimization algorithms
Abstract: This paper presents PIQP, a high-performance toolkit for solving generic sparse quadratic programs (QP). Combining an infeasible Interior Point Method (IPM) with the Proximal Method of Multipliers (PMM), the algorithm can handle ill-conditioned convex QP problems without the need for linear independence of the constraints. The open-source implementation is written in C++ with interfaces to C, Python, Matlab, and R leveraging the Eigen3 library. The method uses a pivoting-free factorization routine and allocation-free updates of the problem data, making the solver suitable for embedded applications. The solver is evaluated on the Maros-Mészáros problem set and optimal control problems, demonstrating state-of-the-art performance for both small and large-scale problems, outperforming commercial and open-source solvers.

Keywords: Traffic control, Delay systems, Lyapunov methods
Abstract: While the connected automated vehicle (CAV) has been applied in vehicle-based traffic control that aims to stabilize a string of car-following human-driven vehicles (HV), the safety impact of a CAV controller on the overall traffic flow system is still an open question. In this paper, we propose a Robust Safety-critical Traffic Control (RSTC) design to impart safety for the mixed autonomy traffic system, in which the speed disturbance from a leading HV and input delay of the stabilizing CAV controller are considered. We employ Control Barrier Function (CBF) to impose safety constraints on a nominal CAV control input that achieves string stability. The key challenge lies in incorporating effect of the input delay and external disturbances into the CBF constraints. The predicted speed and spacing gap in the robust CBF design is obtained using a delay-compensating state predictor. The forward invariance of the safe set is proved given the derivative of speed disturbance, i.e., acceleration of the leading vehicle, is bounded. The safety improvement of RSTC over the nominal controller is then validated via numerical simulation.

Keywords: Distributed parameter systems, Backstepping, Traffic control
Abstract: This paper designs an observer–based output feedback controller for traffic flow with stop–and–go waves and disturbances in order to dissipate traffic congestion. The macroscopic traffic flow dynamics in the congestion regime is described by the linearized Aw–Rascle–Zhang (ARZ) traffic flow model over a time–varying moving spatial domain, and according to the Rankine–Hugoniot condition and the characteristic velocities of the ARZ model, a novel propagation model of the stop–and–go waves is proposed. To stabilize the traffic state and the stop–and–go waves, and to suppress traffic disturbances, an observer–based output feedback controller is designed by using the PDE backstepping method. The controller utilizes the estimated state of an observer, which is constructed based on boundary measurements only. The exponential stability of the closed–loop system in the H1 norm is proved by the Lyapunov analysis. Finally, the effectiveness of the controller for stabilizing the traffic state with stop-and-go waves and disturbances is verified by numerical simulations.

Keywords: Autonomous vehicles, Constrained control, Traffic control
Abstract: Connected automated vehicles have shown great potential to improve the efficiency of transportation systems in terms of passenger comfort, fuel economy, stability of driving behavior and mitigation of traffic congestions. Yet, to deploy these vehicles and leverage their benefits, the underlying algorithms must ensure their safe operation. In this paper, we address the safety of connected cruise control strategies for longitudinal car following using control barrier function (CBF) theory. In particular, we consider various safety measures such as minimum distance, time headway and time to conflict, and provide a formal analysis of these measures through the lens of CBFs. Additionally, motivated by how stability charts facilitate stable controller design, we derive safety charts for existing connected cruise controllers to identify safe choices of controller parameters. Finally, we combine the analysis of safety measures and the corresponding stability charts to synthesize safety-critical connected cruise controllers using CBFs. We verify our theoretical results by numerical simulations.

Keywords: Traffic control, Robust control, Autonomous vehicles
Abstract: Connected and automated vehicles (CAVs) technologies promise to attenuate undesired traffic disturbances. However, in mixed traffic where human-driven vehicles (HDVs) also exist, the nonlinear human-driving behavior has brought critical challenges for effective CAV control. This paper employs the policy iteration method to learn the optimal robust controller for nonlinear mixed traffic systems. Precisely, we consider the H_{infty} control framework and formulate it as a zero-sum game, the equivalent condition for whose solution is converted into a Hamilton–Jacobi inequality with a Hamiltonian constraint. Then, a policy iteration algorithm is designed to generate stabilizing controllers with desired attenuation performance. Based on the updated robust controller, the attenuation level is further optimized in sum of squares program by leveraging the gap of the Hamiltonian constraint. Simulation studies verify that the obtained controller enables the CAVs to dampen traffic perturbations and smooth traffic flow.

Keywords: Transportation networks, Traffic control
Abstract: Routing control is one of important traffic man- agement strategies against urban congestion. However, it could be compromised by heterogeneous driver non-compliance with routing instructions. In this article, we model the compliance in a stochastic manner and investigate its impacts on routing control. We consider traffic routing for two parallel links. Particularly, we consider two scenarios: one ignores congestion spillback while the other one considers it. We formulate the problem as a Markov chain, given random drivers’ adherence. Then, we propose the stability and instability conditions to reveal when the routing is able or unable to stabilize the traffic. We show that for links without congestion spillback there exists a necessary and sufficient stability criterion. For links admiting congestion propagation, we present one stability condition and one instability condition. These stability conditions allow us to quantify the impacts of driver non-compliance on the two-link network in terms of throughput. Finally, we illustrate the results with a set of numerical examples.

Keywords: Adaptive control, Traffic control, Intelligent systems
Abstract: Traffic signal control (TSC) is a complex and important task that affects the daily lives of millions of people. Reinforcement Learning (RL) has shown promising results in optimizing traffic signal control, but current RL-based TSC methods are mainly trained in simulation and suffer from the performance gap between simulation and the real world. In this paper, we propose a simulation-to-real-world (sim-to-real) transfer approach called UGAT, which transfers a learned policy trained from a simulated environment to a real-world environment by dynamically transforming actions in the simulation with uncertainty to mitigate the domain gap of transition dynamics. We evaluate our method on a simulated traffic environment and show that it significantly improves the performance of the transferred RL policy in the real world.

Keywords: Control Systems Privacy, Cooperative control, Decentralized control
Abstract: We propose a new dynamic average consensus algorithm that is robust to information-sharing noise arising from differential-privacy design. Not only is dynamic average consensus widely used in cooperative control and distributed tracking, it is also a fundamental building block in numerous distributed computation algorithms such as multi-agent optimization and distributed Nash equilibrium seeking. We propose a new dynamic average consensus algorithm that is robust to persistent and independent information-sharing noise added for the purpose of differential-privacy protection. In fact, the algorithm can ensure both provable convergence to the exact average reference signal and rigorous ϵ-differential privacy (even when the number of iterations tends to infinity), which, to our knowledge, has not been achieved before in average consensus algorithms. Given that channel noise in communication can be viewed as a special case of differential-privacy noise, the algorithm can also be used to counteract communication imperfections. Numerical simulation results confirm the effectiveness of the proposed approach.

Keywords: Optimization, Optimization algorithms
Abstract: We consider the problem of minimizing the sum of cost functions pertaining to agents over a network whose topology is captured by a directed graph (i.e., asymmetric communication). We cast the problem into the ADMM setting, via a consensus constraint, for which both primal subproblems are solved inexactly. In specific, the computationally demanding local minimization step is replaced by a single gradient step, while the averaging step is approximated in a distributed fashion. Furthermore, partial participation is allowed in the implementation of the algorithm. Under standard assumptions on strong convexity and Lipschitz continuous gradients, we establish linear convergence and characterize the rate in terms of the connectivity of the graph and the conditioning of the problem. Our line of analysis provides a sharper convergence rate compared to Push-DIGing. Numerical experiments corroborate the merits of the proposed solution in terms of superior rate as well as computation and communication savings over baselines.

Keywords: Optimization, Game theory
Abstract: This paper studies a class of strongly monotone games involving non-cooperative agents that optimize their own time-varying cost functions. We assume that the agents can observe other agents' historical actions and choose actions that best respond to other agents' previous actions; we call this a best response scheme. We start by analyzing the convergence rate of this best response scheme for standard time-invariant games. Specifically, we provide a sufficient condition on the strong monotonicity parameter of the time-invariant games under which the proposed best response algorithm achieves exponential convergence to the static Nash equilibrium. We further illustrate that this best response algorithm may oscillate when the proposed sufficient condition fails to hold, which indicates that this condition is tight. Next, we analyze this best response algorithm for time-varying games where the cost functions of each agent change over time. Under similar conditions as for time-invariant games, we show that the proposed best response algorithm stays asymptotically close to the evolving equilibrium. We do so by analyzing both the equilibrium tracking error and the dynamic regret. Numerical experiments on economic market problems are presented to validate our analysis.

Keywords: Optimization algorithms, Optimization, Lyapunov methods
Abstract: In this paper, we study a distributed optimization problem, where decision variables of agents need to satisfy different local constraints and the consensus constraint to minimize the sum of local cost functions. We propose a novel method to remove all these constraints by employing the exact penalty so that the derived equivalent unconstrained problem can be directly solved by subgradient descent type differential inclusions. The algorithm achieves O(1/t) rate of convergence when the cost functions are convex. Moreover, it achieves exponential convergence when the cost functions are strongly convex. Our method needs no dual variable to deal with the constraints so that computation and communication resources are saved in comparison with primal-dual methods. In addition, the method overcomes a divergence problem arising from an existing exponentially convergent distributed algorithm based on the exact penalty when the local constraints are different.

Keywords: Optimization, Distributed control, Fault tolerant systems
Abstract: We study the problem of Byzantine fault tolerance in a distributed optimization setting, where there is a group of N agents communicating with a trusted centralized coordinator. Among these agents, there is a subset of f agents that may not follow a prescribed algorithm and may share arbitrarily incorrect information with the coordinator. The goal is to find the optimizer of the aggregate cost functions of the honest agents. We will be interested in studying the local gradient descent method, also known as federated learning, to solve this problem. However, this method often returns an approximate value of the underlying optimal solution in the Byzantine setting. Recent work showed that by incorporating the so-called comparative elimination (CE) filter at the coordinator, one can provably mitigate the detrimental impact of Byzantine agents and precisely compute the true optimizer in the convex setting. The focus of the present work is to provide theoretical results to show the convergence of local gradient methods with the CE filter in a nonconvex setting. We will also provide a number of numerical simulations to support our theoretical results.