Keywords: Machine learning, Modeling, Robotics
Abstract: Accurately predicting the dynamics of robotic systems is crucial for model-based control and reinforcement learning. The most common way to estimate dynamics is by fitting a one-step ahead prediction model and using it to recursively propagate the predicted state distribution over long horizons. Unfortunately, this approach is known to compound even small prediction errors, making long-term predictions inaccurate. In this paper, we propose a new parametrization to supervised learning on state-action data to stably predict at longer horizons – that we call a trajectory-based model. This trajectory-based model takes an initial state, a future time index, and control parameters as inputs, and directly predicts the state at the future time index. Experimental results in simulated and real-world robotic tasks show that trajectory-based models yield significantly more accurate long term predictions, improved sample efficiency, and the ability to predict task reward. With these improved prediction properties, we conclude with a demonstration of methods for using the trajectory-based model for control.

Keywords: Machine learning, Identification, Statistical learning
Abstract: For tasks where the dynamics of multiple agents are physically coupled, the coordination between the individual agents becomes crucial, which requires exact knowledge of the interaction dynamics. This problem is typically addressed using centralized estimators, which can negatively impact the flexibility and robustness of the overall system. To overcome this shortcoming, we propose a novel distributed learning framework for the exemplary task of cooperative manipulation by applying Bayesian principles. Using only local state information each agent obtains an estimate of the object dynamics and grasp kinematics. These local estimates are combined using dynamic average consensus. Due to the strong probabilistic foundation of the method, each estimate of the object dynamics and grasp kinematics is accompanied by a measure of uncertainty, which allows to guarantee a bounded prediction error with high probability. Moreover, the Bayesian principles directly allow iterative learning with constant complexity, such that the proposed learning method can be used online in real-time applications. The effectiveness of the approach is demonstrated in a simulated cooperative manipulation task.

Keywords: Machine learning, Nonlinear systems, Grey-box modeling
Abstract: Recently there has been increasing interest in the use of learning techniques to model dynamical systems directly from data for scientific and engineering applications. However, in many contexts explicit data collection is expensive and learning algorithms must be data-efficient to be practicable. This suggests using additional qualitative information about the system, which is often available from prior experiments or domain-specific knowledge. We propose an approach to learning a vector field of differential equations using sparse Gaussian Processes that allows us to combine data and additional structural information, like Lie Group symmetries and fixed points. We show that this combination improves extrapolation and long-term behaviour, and reduces computational cost.

Keywords: Statistical learning, Identification, Learning
Abstract: In this paper, we investigate when system identification is statistically easy or hard, in the finite sample regime. Statistically easy to learn linear system classes have sample complexity that is polynomial with the system dimension. Most prior research in the finite sample regime falls in this category, focusing on systems that are directly excited by process noise. Statistically hard to learn linear system classes have worst-case sample complexity that is at least exponential with the system dimension, regardless of the identification algorithm. Using tools from minimax theory, we show that classes of linear systems can be hard to learn. Such classes include, for example, under-actuated or under-excited systems with weak coupling among the states. Having classified some systems as easy or hard to learn, a natural question arises as to what system properties fundamentally affect the hardness of system identifiability. Towards this direction, we characterize how the controllability index of linear systems affects the sample complexity of identification. More specifically, we show that the sample complexity of robustly controllable linear systems is upper bounded by an exponential function of the controllability index. This implies that identification is easy for classes of linear systems with small controllability index and potentially hard if the controllability index is large. Our analysis is based on recent statistical tools for finite sample analysis of system identification as well as a novel lower bound that relates controllability index with the least singular value of the controllability Gramian.

Keywords: Machine learning, Neural networks, Distributed control
Abstract: We consider a distributed Bayesian parameter inference problem where a networked set of agents collaboratively infer the posterior distribution of unknown parameters in a partial differential equation (PDE) based on their noisy measurements of the PDE solution. Given the unknown parameters residing in a known compact set, we assume that a physics-informed neural network (PINN) has already been trained as the prior model, which is valid for all possible parameters within the set. PINNs incorporate PDEs as training constraints for better generalization even with less training samples. We introduce a distributed Langevin Markov Chain Monte-Carlo algorithm that employs the trained PINN model and the agents' noisy measurements to approximate the posterior distribution of the unknown parameters. We establish convergence properties of the algorithm and demonstrate the effectiveness of the proposed approach through numerical simulations.

Keywords: Machine learning, Predictive control for nonlinear systems, Optimization algorithms
Abstract: This paper aims to improve the reliability of optimal control using models constructed by machine learning methods. Optimal control problems based on such models are generally non-convex and difficult to solve online. In this paper, we propose a model that combines the Hammerstein-Wiener model with input convex neural networks, which have recently been proposed in the field of machine learning. An important feature of the proposed model is that resulting optimal control problems are effectively solvable exploiting their convexity and partial linearity while retaining flexible modeling ability. The practical usefulness of the method is examined through its application to the modeling and control of an engine airpath system.

Keywords: Algebraic/geometric methods, Nonlinear systems, Optimization algorithms
Abstract: The design of fixed point algorithms is at the heart of monotone operator theory, convex analysis, and of many modern optimization problems arising in machine learning and control. This tutorial reviews recent advances in understanding the relationship between Demidovich conditions, one-sided Lipschitz conditions, and contractivity theorems. We review the standard contraction theory on Euclidean spaces as well as little-known results for Riemannian manifolds. Special emphasis is placed on the setting of non-Euclidean norms and the recently introduced weak pairings for the L1 and L∞ norms. We highlight recent results on explicit and implicit fixed point schemes for non-Euclidean contracting systems.

Keywords: Machine learning, Nonlinear systems, Adaptive systems
Abstract: While much progress has been achieved over the last decades in neuro-inspired machine learning, there are still fundamental problems in gradient-based learning using combinations of neurons, especially away from the infinite width limit.

These problems, such as saddle points and suboptimal plateaus of the cost function, can lead in theory and practice to failures of learning. In addition, because of their global geometry, Rectified Linear Units (ReLUs) do not allow local adjustment of a pre-learned global network, making the incorporation of new data inefficient.

This paper describes an alternative local learning approach for arbitrary piece-wise linear continuous functions. Neurons are not superimposed, but rather are spatially separated. Also, contrary to classical approximations based e.g. on radial basis functions or finite elements, there is no pre-fixed set of basis functions, but location and support are all adaptable.

The approach yields a new multi-layer algorithm based on learning a MinMax combination of linear neurons, which is applicable directly to the high-dimensional case. Global exponential convergence of the algorithm is established using Lyapunov analysis and contraction theory. It is shown that the algorithm corresponds to a set of separate exponentially stable linear regressions, whose combination is Lyapunov or contraction stable under observability conditions.

Learning is fundamentally local, which allows simple progres- sive adjustment of a previously learned network as new data becomes available. Unused neurons are pruned, avoiding unnecessary over-parameterization. Conversely, new neurons are created in areas where better approximation of the true function is needed. Overall stability is shown for this combination of pruning and creation mechanisms.

Keywords: Adaptive control, Nonlinear systems
Abstract: A key assumption in the theory of nonlinear adaptive control is that the uncertainty of the system can be expressed in the linear span of a set of known basis functions. While this assumption leads to efficient algorithms, it limits applications to very specific classes of systems. We introduce a novel nonparametric adaptive algorithm that learns an infinite-dimensional parameter density to cancel an unknown disturbance in a reproducing kernel Hilbert space. Surprisingly, the resulting control input admits an analytical expression that enables its implementation despite its underlying infinite-dimensional structure. While this adaptive input is rich and expressive -- subsuming, for example, traditional linear parameterizations -- its computational complexity grows linearly with time, making it comparatively more expensive than its parametric counterparts. Leveraging the theory of random Fourier features, we provide an efficient randomized implementation which recovers the computational complexity of classical parametric methods while provably retaining the expressiveness of the nonparametric input. In particular, our explicit bounds only depend polynomially on the underlying parameters of the system, allowing our proposed algorithms to efficiently scale to high-dimensional systems. As an illustration of the method, we demonstrate the ability of the algorithm to learn a predictive model for a 60-dimensional system consisting of ten point masses interacting through Newtonian gravitation.

Keywords: Predictive control for nonlinear systems, Autonomous robots, Constrained control
Abstract: The recent proliferation of model predictive control (MPC) in safety-critical systems has placed additional emphasis on developing algorithms that have strict performance guarantees despite the presence of modeling error or external disturbances. This tutorial summarizes key results of combining contraction theory with MPC to enable provably-safe motion planning for robotic and aerospace systems. After a brief review of control contraction metrics, we summarize the fundamental result that any closed-loop contracting system has an associated invariant tube centered around a desired trajectory. It is then shown how these tubes can be systematically incorporated into the motion planning problem as an additional safety margin for systems with uncertain dynamics. Finally, several future research directions are discussed.

Keywords: Machine learning, Robust control, Stability of nonlinear systems
Abstract: This paper presents a theoretical overview of a Neural Contraction Metric (NCM): a neural network model of an optimal contraction metric and corresponding differential Lyapunov function, the existence of which is a necessary and sufficient condition for incremental exponential stability of non-autonomous nonlinear system trajectories. Its innovation lies in providing formal robustness guarantees for learning-based control frameworks, utilizing contraction theory as an analytical tool to study the nonlinear stability of learned systems via convex optimization. In particular, we rigorously show in this paper that, by regarding modeling errors of the learning schemes as external disturbances, the NCM control is capable of obtaining an explicit bound on the distance between a time-varying target trajectory and perturbed solution trajectories, which exponentially decreases with time even under the presence of deterministic and stochastic perturbation. These useful features permit simultaneous synthesis of a contraction metric and associated control law by a neural network, thereby enabling real-time computable and probably robust learning-based control for general control-affine nonlinear systems.

Keywords: Identification, Nonlinear systems identification, Machine learning
Abstract: This tutorial paper provides an introduction to recently developed tools for machine learning, especially learning dynamical systems (system identification), with stability and robustness constraints. The main ideas are drawn from contraction analysis and robust control, but adapted to problems in which large-scale models can be learnt with behavioural guarantees. We illustrate the methods with applications in robust image recognition and system identification.

Keywords: Neural networks, Formal Verification/Synthesis, Computer-aided control design
Abstract: While conventional reinforcement learning focuses on designing agents that can perform one task, meta-learning aims, instead, to solve the problem of designing agents that can generalize to different tasks (e.g., environments, obstacles, and goals) that were not considered during the design or the training of these agents. In this paper, we consider the problem of training a provably safe Neural Network (NN) controller for uncertain nonlinear dynamical systems that can generalize to new tasks that were not present in the training data while preserving strong safety guarantees. Our approach is to learn a set of NNs during the training phase. When the task becomes available at runtime, our framework will carefully select a subset of NNs and compose them to form the final NN controller. Critical to our approach is the ability to compute a finite-state abstraction of the nonlinear dynamical system. This abstract model captures the behavior of the closed-loop system under all possible NN weights, and is used to train the NNs and compose them when the task becomes available. We provide theoretical guarantees on the correctness of the resulting NN controller and show the efficacy of our approach via simulations.

Keywords: Optimal control, Learning, Nonlinear systems
Abstract: We consider the problem of Reinforcement Learning for nonlinear stochastic dynamical systems. We show that in the RL setting, there is an inherent ``Curse of Variance" in addition to Bellman's infamous ``Curse of Dimensionality", in particular, we show that the variance in the solution grows factorial-exponentially in the order of the approximation. A fundamental consequence is that this precludes the search for anything other than ``local" feedback solutions in RL, in order to control the explosive variance growth, and thus, ensure accuracy. We further show that the deterministic optimal control has a perturbation structure, in that the higher-order terms do not affect the calculation of lower order terms, which can be utilized in RL to get accurate local solutions.

Keywords: Learning, Machine learning, Robust control
Abstract: We develop new foundations for Robust Reinforcement Learning for control, by exploring analytically the relation between the KL-regularized Reinforcement Learning and the Risk-sensitive Control “exponential of integral” criteria. We establish that the maximization of the risk-sensitive exponential criterion is equivalent to maximizing the KL-regularized objective jointly over the policy and the reference policy parameters. We show that the iterative procedure for optimizing the KL-regularized objective, by substituting the reference policy at each iteration with the optimal policy parameter obtained from the previous iteration, starting from some initial value, which is at the core of a number of well-known Reinforcement Learning algorithms, is an iterative approach for optimizing the risk-sensitive control exponential of integral criterion. We provide an interpretation of this iterative optimization procedure as the use of Minorization-Maximization (MM) algorithms. We offer a probabilistic interpretation of the iterative optimization procedure using Probabilistic Graphical Models to motivate the improved performance of such risk-sensitive objectives.

Keywords: Learning, Optimal control, Constrained control
Abstract: In this paper, we study the Instantaneously Constrained Reinforcement Learning (ICRL) problem, in which we are tasked to find a policy maximizing a reward while satisfying certain constraints at each time step. We first extend a result on strong duality of Constrained Markov Decision Process (CMDP) in the literature and propose a sufficient condition for strong duality of the ICRL problem. Inspired by the Augmented Lagrangian Method in constrained optimization, we propose a new surrogate objective function for ICRL, which could be efficiently optimized by common policy-gradient based RL algorithms. We show theoretically that a feasible and optimal policy could be obtained by optimizing this surrogate function, under certain conditions related to the feasible policy set. Our empirical results on a tabular Markov Decision Process and two nonlinear optimal control problems, a constrained pendulum and a constrained half-cheetah, justify our analysis, and suggest that our method could promote safety during learning and converge in a smaller number of iterations compared to the existing algorithms.

Keywords: Predictive control for nonlinear systems, Markov processes, Sampled-data control
Abstract: In this work, we propose a Model Predictive Control (MPC)-based Reinforcement Learning (RL) method for Autonomous Surface Vehicles (ASVs). The objective is to find an optimal policy that minimizes the closed-loop performance of a simplified freight mission, including collision-free path following, autonomous docking, and a skillful transition between them. We use a parametrized MPC-scheme to approximate the optimal policy, which considers path-following/docking costs and states (position, velocity)/inputs (thruster force, angle) constraints. The Least Squares Temporal Difference (LSTD)-based Deterministic Policy Gradient (DPG) method is then applied to update the policy parameters. Our simulation results demonstrate that the proposed MPC-LSTD-based DPG method could improve the closed-loop performance during learning for the freight mission problem of ASV.

Keywords: Transportation networks, Machine learning, Autonomous vehicles
Abstract: Autonomous mobility-on-demand (AMoD) systems represent a rapidly developing mode of transportation wherein travel requests are dynamically handled by a coordinated fleet of robotic, self-driving vehicles. Given a graph representation of the transportation network - one where, for example, nodes represent areas of the city, and edges the connectivity between them - we argue that the AMoD control problem is naturally cast as a node-wise decision-making problem. In this paper, we propose a deep reinforcement learning framework to control the rebalancing of AMoD systems through graph neural networks. Crucially, we demonstrate that graph neural networks enable reinforcement learning agents to recover behavior policies that are significantly more transferable, generalizable, and scalable than policies learned through other approaches. Empirically, we show how the learned policies exhibit promising zero-shot transfer capabilities when faced with critical portability tasks such as inter-city generalization, service area expansion, and adaptation to potentially complex urban topologies.

Keywords: Identification, Closed-loop identification, Estimation
Abstract: Continuous-time system identification has primarily dealt with sampled input and output data for constructing continuous-time models. However, sampled signals can lead to inaccurate models if their intersample behavior is not addressed appropriately. In this paper, this effect is explored in detail with respect to the SRIVC and CLSRIVC estimators, which are some of the most popular methods for open and closed-loop continuous-time system identification respectively. Based on our consistency analysis, we propose an algorithm that alleviates the asymptotic bias of these methods for arbitrary input excitations and provide an alternative procedure to achieve consistent estimates for band-limited signals. Simulation examples show the effectiveness of our approach.

Keywords: Identification, Closed-loop identification, Learning
Abstract: We study the identification of a linear time- invariant dynamical system affected by large-and-sparse disturbances modeling adversarial attacks or faults. Under the assumption that the states are measurable, we develop sufficient conditions for the recovery of the system matrices by solving a constrained lasso-type optimization problem. In the settings without control input or when the input is sub-Gaussian with a known matrix B, we characterize the type of disturbance that does not affect the estimation of the matrix A. We furthermore analyze the case when A and B are estimated simultaneously, and study how to design the input of the system to properly excite the system and make the identification possible in the presence of adversarial attacks. We introduced the key notion of ∆-spaced disturbance and element-wise identifiability to study the success of the constrained lasso estimator. The superiority of the proposed technique is demonstrated in numerical experiments.

Keywords: Identification, Closed-loop identification
Abstract: This paper discusses a kernel regularization method in the frequency domain. In particular, this paper proposes a new kernel which encodes prior knowledge on the rate of low frequency decay. Since the proposed kernel is designed for frequency response rather than impulse response, it becomes possible to estimate unstable systems. A numerical example with unstable system is shown to demonstrate the effectiveness of the proposed method.

Keywords: Closed-loop identification, Networked control systems, Identification
Abstract: This work focuses on the generic identifiability of dynamical networks with partial excitation and measurement: a set of nodes are interconnected by transfer functions according to a known topology, some nodes are excited, some are measured, and only a part of the transfer functions are known. Our goal is to determine whether the unknown transfer functions can be generically recovered based on the input-output data collected from the excited and measured nodes.

We propose a decoupled version of generic identifiability that is necessary for generic local identifiability and might be equivalent as no counter-example to sufficiency has been found yet in systematic trials. This new notion can be interpreted as the generic identifiability of a larger network, obtained by duplicating the graph, exciting one copy and measuring the other copy. We establish a necessary condition for decoupled identifiability in terms of vertex-disjoint paths in the larger graph, and a sufficient one.

Keywords: Closed-loop identification, Predictive control for nonlinear systems, Optimization
Abstract: This paper proposes an algorithm that combines Fast Moving Horizon Parameter Estimation and Model Predictive Control subject to an observability constraint designed to ensure a lower bound on the performance of the parameter estimator. Output-feedback stability is proved through input-to-state stability of the state/error system under a small noise and initial error assumption. Numerical experiments have been carried out in the case of Active Simultaneous Localisation and Mapping (SLAM).

Keywords: Identification for control, Closed-loop identification, Predictive control for linear systems
Abstract: This paper considers system identification in the presence of an unmeasured, unknown, and unmatched multi-tone harmonic disturbance with completely unknown spectrum. It is shown that the identified model possesses spurious poles at the disturbance frequencies that are cancelled by coincident, spurious zeros. Although the presence of the spurious poles is expected, this paper shows that the free response of the identified model is identical-in frequencies, amplitudes, and phases-to the free-plus-forced response of the true system. Consequently, by retaining-rather than cancelling-the coincident, spurious poles and zeros, the identified model has the ability to forecast the future response to an unknown harmonic input over a prediction horizon during which the harmonic disturbance persists. A numerical example illustrates the usefulness of this property to model predictive control with concurrent system identification for rejecting unmeasured, unknown, and unmatched harmonic disturbances with completely unknown spectrum.

Keywords: Markov processes, Optimal control, Stochastic optimal control
Abstract: We derive equivalent linear and dynamic programs for infinite horizon risk-sensitive control for minimization of the asymptotic growth rate of the cumulative cost.

Keywords: Statistical learning, Mean field games, Stochastic optimal control
Abstract: In this paper, we consider learning of discrete-time mean-field games under an average cost criterion. We propose a Q-iteration algorithm via Banach Fixed Point Theorem to compute the mean-field equilibrium when the model is known. We then extend this algorithm to the learning setting by using fitted Q-iteration and establish the probabilistic convergence of the proposed learning algorithm. Our work on learning in average-cost mean-field games appears to be the first in the literature.

Keywords: Stochastic systems, Filtering, Estimation
Abstract: We consider the problem of sampling and statistical inference in probabilistic generative models, where the latent object is a finite-dimensional diffusion process. In general, it is difficult to obtain exact expressions for the log-likelihood, so one has to resort to so-called variational inference, where a change of measure is used to come up with a tractable upper bound. We first show, using W. Fleming’s logarithmic transformation, that the problem of constructing a variational approximation to the log-likelihood can be interpreted as an optimal control problem, where the choice of a variational approximation amounts to adding a drift to the original diffusion. We then analyze this class of control problems using the formalism of conditioned stochastic differential equations due to F. Baudoin. We discuss the relation of this problem to entropic optimal transport and to the stochastic maximum principle.

Keywords: Stochastic optimal control, Optimal control, Agents-based systems
Abstract: In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method.

Keywords: Stochastic optimal control, Filtering
Abstract: The common saying, that information is power, takes a rigorous form in stochastic thermodynamics, where a quantitative equivalence between the two helps explain the paradox of Maxwell’s demon in its ability to reduce entropy. In the present paper, we build on earlier work on the interplay between the relative cost and benefits of information in producing work in cyclic operation of thermodynamic engines (by Sandberg etal. 2014). Specifically, we study the general case of overdamped particles in a time-varying potential (control action) in feedback that utilizes continuous measurements (nonlinear filtering) of a thermodynamic ensemble, to produce suitable adaptations of the second law of thermodynamics that involve information.

Keywords: Identification, Pharmaceutical processes, Biological systems
Abstract: We study the problem of designing an input to a dynamical system that is optimal at estimating unknown parameters in the system's model. We take the A and D optimality criteria on the Fisher Information Matrix associated with the estimation problem as our optimization objective. Our main motivation is the estimation of the physiological parameters that appear in pharmacokinetic dynamics using a relatively short set of measurements. In this context, model inputs correspond to the intravenous injection of drugs and input selection needs to consider safety constraints that include max-min instantaneous injection rates and total dosage amount. We divide the time interval available for the experiment into learning and optimization stages. We use the initial learning stage to obtain a preliminary estimate for the system's model. Then we find an optimal input for the optimization stage so that we can improve upon this initial estimate.

Keywords: Game theory, Modeling
Abstract: We introduce and study a group formation game in which individuals/agents, driven by self-interest, team up in disjoint groups so as to be in groups of high collective strength. This strength could be group identity, reputation, or protection, and is equally shared by all group members. The group's access to resources, obtained from its members, is traded off against the geographic dispersion of the group: spread-out groups are more costly to maintain. We seek to understand the stability and structure of such partitions. We define two types of equilibria: (1) Acceptance Equilibria (AE), in which no agent will unilaterally change group affiliation, either because the agent cannot increase her utility by switching, or because the intended receiving group is unwilling to accept her (i.e., the utility of existing members would decrease if she joined); and (2) Strong Acceptance Equilibria (SAE), in which no subset of any group will change group affiliations (move together) for the same reasons given above.

We show that under natural assumptions on the group utility functions, both an AE and SAE always exist and that any sequence of improving deviations by agents (resp., subsets of agents in the same group) converges to an AE (resp., SAE). We then characterize the properties of the AEs. We show that an "encroachment" relationship - which groups have members in the territory of other groups - always gives rise to a directed acyclic graph (DAG); conversely, given any DAG, we can construct a game with suitable conditions on the utility function that has an AE with the encroachment structure specified by the given graph.

Keywords: Game theory, Optimal control, Aerospace
Abstract: This paper studies a target defense game between two defenders and a faster moving invader. We propose a novel approach that solves the saddle point strategy from a family of time-to-capture isochrons. We show that the game can be viewed as a pursuit-evasion game between the invader and the isochrons, where the latter are controlled by the defenders. The optimal defending strategy is the one that forces the invader to isochrons with shorter time-to-capture most efficiently. The proposed method is validated in a special case with zero capture range, where we prove that the saddle point strategy is optimal regardless of other players' behaviors. In addition, the isochrons are shown to be semipermeable, and the barrier of the target defense game is solved.

Keywords: Game theory, Optimization
Abstract: This brief proposes a novel form of continuous-time evolutionary game dynamics for generalized Nash equilibrium seeking in equality-constrained population games. Using Lyapunov stability theory and duality theory, we provide sufficient conditions to guarantee the asymptotic stability, non-emptiness, compactness, and optimality of the equilibria set of the proposed dynamics for certain population games. Moreover, we illustrate our theoretical developments through a numerical simulation of an equality-constrained congestion game.

Keywords: Game theory, Optimization algorithms, Agents-based systems
Abstract: We study multi-agent optimization problems described by ordinal potential games. The objective is to reach a Nash equilibrium (NE) in a distributed manner. It is well known that an asynchronous best response dynamics (ABRD) will always converge to a pure NE in this case. However, computing the exact best response at every step of the algorithm may be computationally heavy, if not impossible. Therefore, instead of computing the exact best response, we propose an algorithm that performs a ``better response", which decreases the local cost rather than minimizing it. Basically, the agents perform a distributed asynchronous gradient descent algorithm, in which only a finite number of iterations of the gradient descent are performed by each player. We prove that this algorithm always converges to a NE and demonstrate via simulations that the computational time to reach the NE can be much shorter than the classical ABRD. Taking into account the time required for each agent to communicate, the proposed algorithm is shown to also outperform a distributed synchronous gradient descent in simulations.

Keywords: Game theory, Optimization algorithms, Learning
Abstract: We study a distributed approach for seeking a Nash equilibrium in n-cluster games with strictly monotone mappings. Each player within each cluster has access to the current value of its own smooth local cost function estimated by a zero-order oracle at some query point. We assume the agents to be able to communicate with their neighbors in the same cluster over some undirected graph. The goal of the agents in the cluster is to minimize their collective cost. This cost depends, however, on actions of agents from other clusters. Thus, a game between the clusters is to be solved. We present a distributed gradient play algorithm for determining a Nash equilibrium in this game. The algorithm takes into account the communication settings and zero-order information under consideration. We prove almost sure convergence of this algorithm to a Nash equilibrium given appropriate estimations of the local cost functions’ gradients.

Keywords: Game theory, Optimization algorithms, Large-scale systems
Abstract: The aim of this paper is to find the distributed solution of the generalized Nash equilibrium problem (GNEP) for a group of players that can communicate with each other over a connected communication network. Each player tries to minimize a local objective function of its own that may depend on the other players' decisions, and collectively all the players' decisions are subject to some globally shared resource constraints. After reformulating the local optimization problems, we introduce the notion of network Lagrangian and recast the GNEP as the zero finding problem of a properly defined operator. Utilizing the Douglas-Rachford operator splitting method, a distributed algorithm is proposed that requires only local information exchanges between neighboring players in each iteration. The convergence of the proposed algorithm to an exact variational generalized Nash equilibrium is established under two different sets of assumptions. The effectiveness of the proposed algorithm is demonstrated using the example of a Nash-Cournot production game.

Keywords: Optimization algorithms, Communication networks, Machine learning
Abstract: We study the so-called distributed two-time-scale gradient method for solving convex optimization problems over a network of agents when the communication bandwidth between the nodes is limited. Therefore, information exchanged between the nodes must be quantized. Our main contribution is to provide a novel analysis, resulting in an improved convergence rate of this method compared to existing works. In particular, we show that the method converges at a rate mathcal{O}(log^2(k)/sqrt{k}) to the optimal solution when the underlying objective function is strongly convex and smooth. The essential technique in our analysis is to consider a Lyapunov function that simultaneously captures the coupling of the consensus and optimality errors generated by the method.

Keywords: Transportation networks, Optimization algorithms, Modeling
Abstract: The express delivery network (EDN) is a complex logistics collaboration system, where massive parcels are shipped from their origins to destinations through certain routes in a transportation network. In EDN, the routing of parcels and the design of shipment resources are coupled together, and the scale of problems is usually large. This complexity and the mutual dependency of network components impose significant challenges in practical decision-making. In this paper, we propose a novel two-layer multipartite-graph representation for EDN and design an iterative solution framework to improve an ongoing network. This graph structure captures the decentralized decision-making nature of EDN and is easy to extract subgraphs for a diagnosed problem. In the solution framework, a recovery process is first implemented to obtain the initial graph, and then, graph-based MIP models are instantiated for extracted subgraphs to produce executable local modifications in the iterative process. Numerical experiments validate the efficiency and interpretability advantages of the graph structure as well as the solution framework. This method also performs well in an actual EDN with more than 30,000 nodes, leading to a million-dollar-level cost reduction for the related company.

Keywords: Decentralized control, Distributed control, Optimization
Abstract: We study the problem of distributed optimization over a network of agents where the agents strive to minimize the sum of local objective functions through an exchange of information between the nodes based on an underlying communication topology. Motivated by the need for low communication algorithms with better convergence rate in broadcast settings, we propose a subgradient method based on a state-dependent gossip algorithm. The state-dependent gossip algorithm operates by averaging the edge with the maximum disagreement over the network. We prove that agents employing the state-dependent subgradient method achieve consensus on an optimal solution. By exploiting the convergence properties of a Lyapunov function, we obviate the need for results on time-normalized information flow between any node pairs.

Keywords: Optimization, Optimization algorithms, Machine learning
Abstract: This paper revisits the well-known family of sequential convex programming methods. We adopt the difference of convex programming technique to relax a wide variety of nonconvex optimization problems into convex programs. We extend this approach to a sequential convex programming algorithm that can generate a convergent sequence of feasible points whose objective values monotonically improve. As an improvement upon the existing sequential methods, we prove that under certain assumptions, the proposed algorithm reaches feasibility within a finite number of rounds, as opposed to asymptotic feasibility. The effectiveness of the proposed approach is corroborated through experiments on the problem of robust linear regression.

Keywords: Optimization, Optimization algorithms, Power systems
Abstract: We propose an approach for verifying that a given feasible point for a polynomial optimization problem is globally optimal. The approach relies on the Lasserre hierarchy and the result of Lasserre regarding the importance of the convexity of the feasible set as opposed to that of the individual constraints. By focusing solely on certifying global optimality and relaxing the Lasserre hierarchy using necessary conditions for positive semidefiniteness based on matrix determinants, the proposed method is implementable as a computationally tractable linear program. We demonstrate this method via application to several instances of polynomial optimization, including the optimal power flow problem used to operate electric power systems.

Keywords: Optimization algorithms, Randomized algorithms, Agents-based systems
Abstract: In this paper, an unconstrained collaborative optimization of a sum of convex functions is considered where agents make decisions using local information from their neighbors. The communication between nodes are described by a random sequence of possibly state-dependent weighted networks. It is shown that the state-dependent weighted random operator of the graph has quasi-nonexpansivity property, and therefore the operator does not need the distribution of random communication topologies. Hence, it includes random networks with/without asynchronous protocols. As an extension of the problem, a more general mathematical optimization problem than that of the literature is defined, namely minimization of a convex function over the fixed-value point set of a quasi-nonexpansive random operator. A discrete-time algorithm using diminishing step size is given which can converge almost surely to the global solution of the optimization problem under suitable assumptions. Consequently, as a special case, the algorithm reduces to a totally asynchronous algorithm without requiring distribution dependency or B-connectivity assumption for the distributed optimization problem. The algorithm still works in the case where weighted matrix of the graph is periodic and irreducible in a synchronous protocol.

Keywords: Stability of nonlinear systems, Biological systems, Network analysis and control
Abstract: Remote synchronization describes a fascinating phenomenon where oscillators that are not directly connected via physical links evolve synchronously. This phenomenon is thought to be critical for distributed information processing in the mammalian brain, where long-range synchronization is empirically observed between neural populations belonging to spatially distant brain regions. Inspired by the growing belief that this phenomenon may be prompted by intermediate mediating brain regions, such as the thalamus, in this paper we derive a novel mechanism to achieve remote synchronization. This mechanism prescribes remotely synchronized oscillators to be stably connected to a cohesive relay in the network -- a group of tightly connected oscillators mediating the distant ones. Remote synchronization unfolds whenever the stability of the subnetwork formed by relays and remotely synchronized oscillators is not affected by the rest of the oscillators. In accordance with our results, we find that remotely-synchronized cortico-thalamo-cortical circuits in the brain posses strong in- terconnection profiles. Finally, we demonstrate that the absence of cohesive relays prevents stable remote synchronization in a large class of cases, further validating our results.

Keywords: Compartmental and Positive systems, Stability of nonlinear systems, Systems biology
Abstract: A time-varying nonlinear dynamical system is called a totally positive differential system (TPDS) if its Jacobian admits a special sign pattern: it is tri-diagonal with positive entries on the super- and sub-diagonals. If the vector field of a TPDS is T-periodic then every bounded trajectory converges to a T-periodic solution. In particular, when the vector field is time-invariant every bounded trajectory of a TPDS converges to an equlbrium. Here, we use the spectral theory of oscillatory matrices to analyze the behavior near a periodic solution of a TPDS. This yields information on the perturbation directions that lead to the fastest and slowest convergence to or divergence from the periodic solution. We demonstrate the theoretical results using a model from systems biology called the ribosome flow model.

Keywords: Stability of nonlinear systems, Linear systems, Optimization
Abstract: Scaled relative graphs were recently introduced to analyze the convergence of optimization algorithms using two dimensional Euclidean geometry. In this paper, we connect scaled relative graphs to the classical theory of input/output systems. It is shown that the Nyquist diagram of an LTI system on L_2 is the convex hull of its scaled relative graph under a particular change of coordinates. The SRG may be used to visualize approximations of static nonlinearities such as the describing function and quadratic constraints, allowing system properties to be verified or disproved. Interconnections of systems correspond to graphical manipulations of their SRGs. This is used to provide a simple, graphical proof of the classical incremental passivity theorem.

Keywords: Stability of nonlinear systems, LMIs, Optimization
Abstract: This paper presents a method for calculating Region of Attraction of a target set (not necessarily an equilibrium) for controlled polynomial dynamical systems, using a hierarchy of semidefinite programming problems (SDPs). Our approach builds on previous work and addresses its main issue, the fast-growing memory demands for solving large-scale SDPs. The main idea in this work is in dissecting the original resource-demanding problem into multiple smaller, interconnected, and easier to solve problems. This is achieved by spatio-temporal splitting akin to methods based on partial differential equations. We show that the splitting procedure retains the convergence and outer-approximation guarantees of the previous work, while achieving higher precision in less time and with smaller memory footprint.

Keywords: Stability of nonlinear systems, Lyapunov methods, Predictive control for nonlinear systems
Abstract: This paper develops a dissipativity-based framework for synthesis of stabilizing controllers for discrete-time nonlinear systems subject to state/input constraints. Firstly, we revisit dissipation inequalities for discrete-time nonlinear systems that involve general storage and supply functions. We prove that positive definite storage functions and cyclically negative supply functions yield asymptotic Lyapunov stability. Secondly, we define control dissipation functions (CDFs) and we construct stabilizing receding horizon controllers by minimizing CDFs subject to a cyclically negative supply condition. The effectiveness of stabilizing controllers based on CDFs is demonstrated on stabilization of interconnected synchronous generators with nonlinear coupling and state/input constraints.

Keywords: Lyapunov methods, PID control, Stability of nonlinear systems
Abstract: This paper investigates a generalized homogeneous PI (Proportional-Integral) control of MIMO linear plant. A Lyapunov function for analysis of the closed-loop system is designed. For the case of negative homogeneity degree, the obtained Lyapunov function becomes a strict Lyapunov function allowing an advanced analysis to be provided. In particular, a class of the disturbances to be rejected by the control law is characterized, a maximum control magnitude and the settling-time of the closed-loop are estimated.

Keywords: Switched systems, Optimization algorithms, Stability of hybrid systems
Abstract: This work finds a lower bound on the average dwell-time (ADT) of switching signals such that a continuous-time, graph-based, switched system is globally asymptotically stable, input-to-state stable, or integral input-to-state stable. We first formulate the lower bound on the ADT as a nonconvex optimization problem with bilinear matrix inequality constraints. Because this formulation is independent of the choice of Lyapunov functions, its solution gives a less conservative lower bound than previous Lyapunov-function-based approaches. We then design a numerical iterative algorithm to solve the optimization based on sequential convex programming with a convex-concave decomposition of the constraints. We analyze the convergence properties of the proposed algorithm, establishing the monotonic evolution of the estimates of the average dwell-time lower bound. Finally, we demonstrate the benefits of the proposed approach in two examples and compare it against other baseline methods.

Keywords: Switched systems, Stability of hybrid systems, Lyapunov methods
Abstract: This paper deals with the stability analysis problem of discrete-time switched linear systems with ranged dwell time. A novel concept called L-switching-cycle is proposed, which contains sequences of multiple activation cycles satisfying the prescribed ranged dwell time constraint. Based on L-switching-cycle, two sufficient conditions are proposed to ensure the global uniform asymptotic stability of discrete-time switched linear systems. It is noted that two conditions are equivalent in stability analysis with the same L-switching-cycle. These two sufficient conditions can be viewed as generalizations of the clock-dependent Lyapunov and multiple Lyapunov function methods, respectively. Furthermore, it has been proven that the proposed L-switching-cycle can eventually achieve the nonconservativeness in stability analysis as long as a sufficiently long L-switching-cycle is adopted. A numerical example is provided to illustrate our theoretical results.

Keywords: Switched systems, Stability of hybrid systems, Randomized algorithms
Abstract: We consider discrete-time switching systems composed of a finite family of affine sub-dynamics. First, we recall existing results and present further analysis on the stability problem, the existence and characterization of compact attractors, and the relations these problems have with the joint spectral radius of the set of matrices composing the linear part of the subsystems. Second, we tackle the problem of providing probabilistic certificates of stability along with the existence of forward invariant sets, assuming no knowledge on the system data but only observing a finite number of sampled trajectories. Some numerical examples illustrate the advantages and limits of the proposed conditions.

Keywords: Switched systems, Stability of nonlinear systems, Lyapunov methods
Abstract: This paper deals with stability of continuous-time switched nonlinear systems whose switching signals obey pre specified restrictions on admissible switches between the subsystems and admissible dwell times on the subsystems. Given a family of systems, possibly containing unstable dynamics, and the restrictions on the set of admissible switching signals, we identify a class of switching signals that obeys these restrictions and preserves input/output to-state stability (IOSS) of the resulting switched system. The primary apparatus for our analysis is multiple Lyapunov-like functions. Input-to-state stability (ISS) and global asymptotic stability (GAS) of switched systems under pre-specified restrictions on switching signals are special cases of our results when no outputs (resp., also inputs are considered.

Keywords: Switched systems, Observers for nonlinear systems, Control applications
Abstract: Global adaptive output feedback stabilization for a class of switched nonlinear time-delay systems under arbitrary switching is achieved in this paper. A new high gain observer is designed with which the non-differentiable unknown output function can be tackled and the explicit linear-like controller is fairly simple. By choosing proper common Lyapunov-Krasovskii functionals, the common controller is proved to be able to globally stabilize the switched system. A real application is given to demonstrate the effectiveness of the proposed scheme.

Keywords: Uncertain systems, LMIs, Algebraic/geometric methods
Abstract: Peak estimation bounds extreme values of a function of state along trajectories of a dynamical system. This paper focuses on extending peak estimation to continuous and discrete settings with time-independent and time-dependent uncertainty. Techniques from optimal control are used to incorporate uncertainty into an existing occupation measure-based peak estimation framework, which includes special consideration for handling switching-type (polytopic) uncertainties. The resulting infinite-dimensional linear programs can be solved approximately with Linear Matrix Inequalities arising from the moment-SOS hierarchy.

Keywords: Robust control, Linear systems, Uncertain systems
Abstract: Robust controller synthesis attracts reviving research interest, driven by the rise of learning-based systems where uncertainty and perturbation are ubiquitous. Facing an uncertain situation, a robustly stabilizing controller should maintain stability while operating under a perturbed system deviating from its nominal specification. There have been numerous results for robust controller synthesis in multiple forms and with various goals, including mu-synthesis, robust primal-dual Youla, robust input-output, and robust system level parameterizations. However, their connections with one another are not clear, and we lack a general approach to robust controller analysis and synthesis.

To serve this purpose, we derive robust stability conditions for general systems and formulate the general robust controller synthesis problem. The conditions hinge on the realization-stability lemma, a recent analysis tool that unifies existing controller synthesis methods. Not only can the conditions infer a wide range of existing robust results, but they also lead to easier derivations of new ones. Together, we demonstrate the effectiveness of the conditions and provide a unified approach to robust controller analysis and synthesis.

Keywords: Robust control, Nonlinear systems, Distributed control
Abstract: This paper gives a global solution to an H-infinity control problem for systems with symmetric state matrix and state-dependent input matrix. A simple, closed-form expression for the minimizing controller is obtained. This is in contrast to already established theory in which nonlinear H-infinity problems are solved locally. The result is then illustrated through an example, and the potential for controller sparsity is highlighted. Applications to large-scale systems are discussed.

Keywords: Robust control, Optimal control, LMIs
Abstract: H2-conic controller design seeks to minimize the closed-loop H2 norm for a nominal linear system while satisfying the Conic Sector Theorem for nonlinear stability. This problem has only been posed with limited design freedom, as opposed to fixed-order design where all controller parameters except the number of state estimates are free variables. Here, the fixed-order H2-conic design problem is reformulated as a convergent series of convex approximations using iterative convex overbounding. A synthesis algorithm and various initializations are proposed. The synthesis is applied to a passivity-violated system with uncertain parameters and compared to benchmark controller designs.

Keywords: Robust control, Optimization algorithms, LMIs
Abstract: A novel convex state-space solution to the robust output feedback synthesis problem based on dynamic integral quadratic constraints for a particular class of systems is presented. Convexification rests upon the assumption that the control-to-uncertainty block in the generalized plant commutes with the off-diagonal block of the multiplier. We demonstrate that this commutation property is valid for several by themselves interesting concrete scenarios, such as in extremum control, a generalization of the convex design of first-order optimization algorithms involving filtered gradient evaluations.

Keywords: Robust control, Predictive control for linear systems, Optimal control
Abstract: Robust tube-based model predictive control (MPC) methods address constraint satisfaction by leveraging an a priori determined tube controller in the prediction to tighten the constraints. This letter presents a system level tube-MPC (SLTMPC) method derived from the system level parameterization (SLP), which allows optimization over the tube controller online when solving the MPC problem, which can significantly reduce conservativeness. We derive the SLTMPC method by establishing an equivalence relation between a class of robust MPC methods and the SLP. Finally, we show that the SLTMPC formulation naturally arises from an extended SLP formulation and show its merits in a numerical example.

Keywords: Robust control, Sampled-data control, Quantized systems
Abstract: This paper deals with the robust quantized sampled--data leaderless consensus tracking problem for nonlinear multi-agent systems (MASs), affected by actuation disturbances and observation errors, over strongly connected networks. The input-to-state stability redesign methodology is used in order to design a new quantized sampled--data control term able to arbitrarily attenuate the effects of bounded actuation disturbances and bounded observation errors. The quantization of both input/output channels is simultaneously considered. It is proved that, by suitably fast sampling and accurately quantizing the continuous--time protocol at hand, the leaderless consensus tracking is ensured, regardless of the above disturbances and errors provided that: (i) the bounds of the actuation disturbances and of the observation errors are a--priori known; (ii) the observation errors do not affect or affect marginally the new added control term. Possible discontinuities in the function describing the protocol are also managed. The cases of time--varying sampling intervals and of non--uniform quantization in the input/output channels are included in the theory here developed. Furthermore, the stability analysis of the inter-sampling system behaviour is performed. The stabilization in the sample-and-hold sense theory is used as a tool for proving the results. The provided results are validated through an application concerning the formation control problem of unmanned aerial vehicles.

Keywords: Linear parameter-varying systems, Game theory, Decentralized control
Abstract: In this paper, infinite horizon Stackelberg games with a large follower population for stochastic linear parameter-varying (LPV) systems is studied. Players are assumed to have their own local dynamics and they are coupled with each other through a mean field term in the cost functionals. It is assumed that followers adopt Nash equilibrium strategies and can only access their own local state information. First, the conditions for the existence of a centralized Stackelberg strategy set and its solutions are obtained by using the higher-order cross-coupled matrix inequalities (CCMIs) and cross-coupled matrix equations (CCMEs), respectively. Then, the conditions for the existence of a decentralized Stackelberg strategy set and its solution are obtained by using the reduced-order CCMIs and CCMEs respectively. It is proved that, when the follower population size is sufficiently large, the centralized Stackelberg strategy set approximates to the decentralized strategy set and the approximation is evaluated by using the weakly-coupled stochastic systems theory. Finally, a simple example is computed to demonstrate the validity and usefulness of the proposed approach.

Keywords: Linear parameter-varying systems, LMIs, Observers for nonlinear systems
Abstract: This paper addresses an unknown input observer design to estimate simultaneously the 3D depth of a tracked image feature and the camera linear velocity using a low cost monocular camera and inertial sensor. The camera kinematic model is at first, augmented via the dynamic extension approach then described as a quasi-Linear Parameter Varying (qLPV) model. Further, the qLPV system is transformed into Takagi-Sugeno (T-S) form with unmeasured premise variables. The error convergence analysis is performed based on Lyapunov theory and Input to State Stability (ISS) property to ensure the boundedness of the state estimation error. Gains that guarantee the asymptotic stability of the estimation error can be properly computed by means of Linear Matrix Inequalities (LMIs). Finally the proposed approach is validated using synthetic data.

Keywords: Linear parameter-varying systems, Nonlinear systems, Nonlinear output feedback
Abstract: Unlike for Linear Time-Invariant (LTI) systems, for nonlinear systems, there exists no general framework for systematic convex controller design which incorporates performance shaping. The Linear Parameter-Varying (LPV) framework sought to bridge this gap by extending convex LTI synthesis results such that they could be applied to nonlinear systems. However, recent literature has shown that naive application of the LPV framework can fail to guarantee the desired asymptotic stability guarantees for nonlinear systems. Incremental dissipativity theory has been successfully used in the literature to overcome these issues for Continuous-Time (CT) systems. However, so far no solution has been proposed for output-feedback based incremental control for the Discrete-Time (DT) case. Using recent results on convex analysis of incremental dissipativity for DT nonlinear systems, in this paper, we propose a convex output-feedback controller synthesis method to ensure closed-loop incremental dissipativity of DT nonlinear systems via the LPV framework. The proposed method is applied on a simulation example, demonstrating improved stability and performance properties compared to a standard LPV controller design.

Keywords: Linear parameter-varying systems, Stability of linear systems, Time-varying systems
Abstract: We study the problem of sensor-less vibrational control through periodic, parametric forcing. We introduce a new analysis technique based on a lifting procedure for time-periodic systems, combined with a root locus analysis to determine stabilization or destabilization by parametric forcing. Our technique is based on several equivalences, the first one being a decomposition of the time-periodic system into a feedback between a Linear Time Invariant (LTI) system and a periodically-time varying memoryless gain. The second is an equivalence with a lifted system in which both feedback components are LTI, but with infinite-dimensional input and output spaces. The third equivalence is with a Single-Input-Single-Output (SISO) LTI system with an infinite, but regular pattern of poles and zeros in feedback with a static gain that serves as the root locus parameter. Stability criteria can then be easily obtained from root locus plots. Like the root locus, our technique is simple to apply to higher order systems, and not only to second order systems. We illustrate the technique in this paper through the Kapitza pendulum example, for which the stability boundaries in parameter space predicted by this root locus are found to be fairly accurate in comparison with classical methods like Floquet theory or dynamical averaging. We conclude with some remarks on the accuracy and potential generalizations of the method.

Keywords: Lyapunov methods, Linear parameter-varying systems, Optimization
Abstract: Polyhedral Lyapunov functions can approximate any norm arbitrarily well. Because of this, they are used to study the stability of Linear Time Varying and Linear Parameter Varying systems without being conservative. However, the computational cost of finding them grows unbounded as the size of their representation increases. We present an algorithm that attempts to find polyhedral functions while keeping the size of the representation fixed, allowing us to fine-tune the trade-off between conservativeness and computational complexity. The algorithm first measures the gap from contraction for a given polyhedral function. The solution is then used to find perturbations on the polyhedral function that reduce the contraction gap. The process is repeated until a valid polyhedral Lyapunov function is obtained. Because the algorithm is rooted in linear programming, additional linear constraints and objectives can be directly included. This leads to a flexible approach that can tackle both analysis and control synthesis problems.

Keywords: Model/Controller reduction, Linear parameter-varying systems
Abstract: In this paper, we propose a model reduction method for LPV systems. We consider LPV state-space representations with an affine dependence on the scheduling variables. The main idea behind the proposed method is to compute the reduced order model in such a manner that its frequency domain transfer function coincides with that of the original model for some frequencies. The proposed method uses Loewner-like matrices, which can be calculated from the frequency domain representation of the system. The contribution of the paper represents an extension of the well-established Loewner framework to LPV models.

Keywords: Distributed parameter systems, Optimal control, Uncertain systems
Abstract: A finite-dimensional approximation and convergence theory for the closed-loop continuous-time linear quadratic Gaussian (LQG) control of random abstract parabolic systems is developed. We apply Galerkin-based finite-dimensional approximation to the weak formulation of the state equation and establish convergence of the approximating compensators and the associated Riccati operators and functional control and observer gains to their infinite-dimensional analogs using results from linear semigroup theory. The distribution of the random parameters is assumed to either be known a-priori or estimated from population data. In addition to the convergence theory, the results of our numerical studies demonstrating the efficacy, practicality, and performance of our approach are presented and discussed.

Keywords: Delay systems, Observers for nonlinear systems, Variable-structure/sliding-mode control
Abstract: This paper provides a new stabilization control method for linear time-invariant systems subject to known time-varying measurement delays and matched unknown nonlinear disturbances that may represent actuator faults. Part of the state vector is assumed to be unmeasured in current time. Hence, the proposed method utilizes an open-loop predictor associated with a state observer based on the Super-Twisting Algorithm in order to compensate the delays and estimate the unmeasured state variables. In particular, this nonlinear observer-based structure allows for the reconstruction of the non-modeled fault signals which, unlike the existing literature, are not supposed to be generated by a known exogenous dynamic system, being also robust to parametric uncertainties, whereas the predictor advances in time the delayed output signal. Then, a sliding mode control law is developed to achieve an ideal sliding mode and guarantee global stabilization even in the presence of a more general class of perturbations, non-modeled disturbances, parametric uncertainties and delays due to the inclusion of the Super-Twisting observer. Numerical simulations illustrate the efficiency of the proposed approach.

Keywords: Distributed parameter systems, Stability of linear systems, Observers for Linear systems
Abstract: Abstract—We consider optimal control problem governed by a an acooustic equation with thermal relaxation. This leads to a third order hyperbolic system which is non-characteristic. Questions of boundary stabilizability and feedback control are the main themes. On line control will be implemented by Gain Feedback based on nonsstandard Riccati equation with unbounded coefficients. While optimal control problems with smooth controls have been considered in the recent literature, our new challenge is to consider ”rough controls ” L2 forcings which then lead to nonsmooth optimization problem and non-standard Riccati equations.

Keywords: Distributed parameter systems, Sampled-data control, Process Control
Abstract: This work presents a model-based approach for feedback control of spatially-distributed systems described by nonlinear parabolic PDEs subject to discretely-sampled state measurements, bounded measurement errors, and bounded model uncertainty. The controller is designed based on a suitable finite-dimensional approximation that captures the dominant dynamics of the infinite-dimensional system, and includes an inter-sample model predictor that compensates for the unavailability of continuous state measurements. The state of the model predictor is updated using the available state measurements at the sampling times. The sampled-data closed-loop system is analyzed and a sufficient condition for closed-loop stability is derived. The analysis leads to an explicit characterization of the closed-loop stability region in terms of the sampling period, the measurement error bound, the parametric uncertainty bounds, and the controller design parameters. A simulation case study is presented to demonstrate how the results can be used to mitigate the impact of measurement errors on closed-loop stability.

Keywords: Distributed parameter systems, Automotive systems
Abstract: This paper considers a class of cyber-attacks wherein an attacker has the ability to hack into a subset of vehicles driving on a highway, and inject subtle changes into their driving parameter(s). As a consequence of such changes, these vehicles (referred to as malicious vehicles) are able to destabilize the traffic even while the non-malicious (normal) vehicles continue to drive with their original parameter values. The traffic system comprising the mix of malicious and normal vehicles are modeled using a two-species, fourth order system of Partial Differential Equations (PDEs). On this system of PDEs, a stability analysis is performed and this is used to partition the traffic parameter space into stable and unstable regimes. Simulations are presented to demonstrate the validity of the results.

Keywords: Distributed parameter systems, Optimal control, Numerical algorithms
Abstract: A new method for optimal actuator design in vibration control is presented. The optimal actuator shape is found by means of shape calculus and a topological derivative of the LQR cost. An abstract framework is proposed based on the theory for infinite-dimensional optimization of both the actuator shape and the associated control problem. A numerical realization of the optimality condition is developed for the actuator shape using a level-set method for topological derivatives. A numerical example illustrating the design of actuator for Euler-Bernoulli beam model is provided.

Keywords: Optimization algorithms, Distributed parameter systems, Machine learning
Abstract: In a multi-agent network, we consider the problem of minimizing an objective function that is expressed as the sum of private convex and smooth functions, and a (possibly) non-differentiable convex regularizer. We propose a novel distributed second-order method based on the framework of Alternating Direction Method of Multipliers (ADMM), by invoking approximate Newton iterations to the primal update corresponding to the differentiable part. In order to achieve a distributed implementation, the total Hessian matrix is split into a diagonal component (locally computable) and an off-diagonal component (that requires communication between neighboring agents). Subsequently, the Hessian inverse is approximated by a truncation of the Taylor expansion to K terms: this amounts to fully distributed updates entailing K distributed communication rounds. We establish global linear convergence to the primal-dual optimal solution under the assumption that the private functions are strongly convex and have Lipschitz continuous gradient. Numerical experiments demonstrate the merits of the approach comparatively with state-of-the-art methods.

Keywords: Optimization, Optimization algorithms
Abstract: We consider a zeroth-order distributed optimization problem, where the global objective function is a black-box function and, as such, its gradient information is inaccessible to the local agents. Instead, the local agents can only use the values of the objective function to estimate the gradient and update their local decision variables. In this paper, we also assume that these updates are done asynchronously. To solve this problem, we propose an asynchronous zeroth-order distributed optimization method that relies on a one-point residual feedback to estimate the unknown gradient. We show that this estimator is unbiased under asynchronous updating, and theoretically analyze the convergence of the proposed method. We also present numerical experiments that demonstrate that our method outperforms two-point methods under asynchronous updating. To the best of our knowledge, this is the first asynchronous zeroth-order distributed optimization method that is also supported by theoretical guarantees.

Keywords: Optimization algorithms, Optimization
Abstract: Distributed resource allocation is a central task in network systems such as smart grids, water distribution networks, and urban transportation systems. When solving such problems in practice it is often important to have non-asymptotic feasibility guarantees for the iterates, since over-allocation of resources easily causes systems to break down. In this paper, we develop a distributed resource reallocation algorithm where every iteration produces a feasible allocation. The algorithm is fully distributed in the sense that nodes communicate only with neighbors over a given communication network. We prove that under mild conditions the algorithm converges to a point arbitrarily close to the optimal resource allocation. Numerical experiments demonstrate the competitive practical performance of the algorithm.

Keywords: Optimization algorithms, Communication networks, Machine learning
Abstract: We consider the distributed learning problem where a network of n agents seeks to minimize a global function F. Agents have access to F through noisy gradients, and they can locally communicate with their neighbors a network. We study the Decentralized Local SDG method, where agents perform a number of local gradient steps and occasionally exchange information with their neighbors. Previous algorithmic analysis efforts have focused on the specific network topology (star topology) where a leader node aggregates all agents' information. We generalize that setting to an arbitrary network by analyzing the trade-off between the number of communication rounds and the computational effort of each agent. We bound the expected optimality gap in terms of the number of iterates T, the number of workers n, and the spectral gap of the underlying network. Our main results show that by using only R=Omega(n) communication rounds, one can achieve an error that scales as O({1}/{nT}), where the number of communication rounds is independent of T and only depends on the number of agents.

Keywords: Optimization algorithms, Optimization, Distributed control
Abstract: We consider a distributed stochastic optimization problem that is solved by a decentralized network of agents with only local communication between neighboring agents. The goal of the whole system is to minimize a global objective function given as a sum of local objectives held by each agent. Each local objective is defined as an expectation of a convex smooth random function and the agent is allowed to sample stochastic gradients for this function. For this setting we propose the first accelerated (in the sense of Nesterov's acceleration) method that simultaneously attains optimal up to a logarithmic factor communication and oracle complexity bounds for smooth strongly convex distributed stochastic optimization. We also consider the case when the communication graph is allowed to vary with time and obtain complexity bounds for our algorithm, which are the first upper complexity bounds for this setting in the literature.

Keywords: Decentralized control, Optimal control, Large-scale systems
Abstract: We consider quadratically-optimal control synthesis for systems in which controls and measurements are spatially distributed. We impose a structural constraint that the feedback can only use measurements within an a priori fixed distance from the control action. Such constraints generally lead to non-convex optimal control problems. In the context of spatially invariant systems such a constraint amounts to restricting the feedback spatial convolution kernel to have a pre-specified compact support. For systems where the state is a scalar-valued spatio-temporal field, we show that this structured controller design problem can be reformulated as a convex optimization problem, thereby identifying a new class of systems for which the optimal structured control synthesis is convex. We apply our method to design an optimal structured controller for a diffusion process on the real line.

Keywords: Distributed control, Game theory, Optimization
Abstract: This paper considers a networked aggregative game (NAG) where the players are distributed over a communication network. By only communicating with a subset of players, the goal of each player in the NAG is to minimize an individual cost function that depends on its own action and the aggregate of all the players’ actions. To this end, we design a novel distributed algorithm that jointly exploits the ideas of the consensus algorithm and the conditional projection descent. Under strongly monotone assumption on the pseudo-gradient mapping, the proposed algorithm with fixed step-sizes is proved to converge linearly to the unique Nash equilibrium of the NAG. Then the theoretical results are validated by numerical experiments.

Keywords: Distributed control, Control of networks, Network analysis and control
Abstract: This paper aims to address the distributed Nash equilibrium seeking problem for noncooperative games over multi-agent networks in the presence of communication uncertainties. Compared with the existing results on distributed Nash equilibrium seeking, the underlying communication network of the agents considered in this paper is subjected to unknown communication uncertainties. Specifically, the coupling weights on communication channels are perturbed by bounded uncertain signals. To achieve the goal of Nash equilibrium seeking, two distributed adaptive Nash equilibrium seeking algorithms based on the cooperative tracking protocol and estimate feedback respectively are proposed by designing adaptive coupling weights to overcome the effect of the communication uncertainties on Nash equilibrium seeking. The convergence of the proposed algorithms is theoretically analyzed and the effectiveness of the algorithms is illustrated by simulation results.

Keywords: Distributed control, Optimization algorithms, Energy systems
Abstract: In this paper, we investigate the problem of coordination between economic dispatch (ED) and demand response(DR) in multi-energy systems (MESs), aiming to improve the economic utility and reduce the waste of energy in MESs. Since multiple energy sources are coupled through energy hubs (EHs), the supply-demand constraints are nonconvex. To deal with this issue, we propose a linearization method to transform the coordination problem to a convex social welfare optimization one. Then a decentralized algorithm based on parallel Alternating Direction Method of Multipliers (ADMM) and dynamic average tracking protocol is developed, where each agent could only make decisions based on information from their neighbors. Moreover, by using variational inequality and Lyapunov-based techniques, we show that our algorithm could always converge to the global optimal solution. Finally, a case study on the modified IEEE 14-bus network verifies the feasibility and effectiveness of our algorithm.

Keywords: Networked control systems, Nonlinear systems, Neural networks
Abstract: This paper studies event-triggered consensus control for heterogenous nonlinear multi-agent systems. We present a new distributed nonlinear event-triggered control algorithm integrating basic radial basis function neural network with event-based control. We show that it can handle any unknown dynamics linear in the control input, achieving practical consensus without Zeno behaviour. A numerical example is provided to highlight the effectiveness of the proposed algorithm in terms of learning the unknown nonlinear dynamics.

Keywords: Optimization algorithms, Networked control systems, Distributed control
Abstract: In this work, we consider the asynchronous distributed optimization problem in which each node has its own convex cost function and can communicate directly only with its neighbors, as determined by a directed communication topology (directed graph or digraph). First, we reformulate the optimization problem so that Alternating Direction Method of Multipliers (ADMM) can be utilized. Then, we propose an algorithm, herein called Asynchronous Approximate Distributed Alternating Direction Method of Multipliers (AsyAD-ADMM), using finite-time asynchronous approximate ratio consensus, to solve the multi-node convex optimization problem, in which every node performs iterative computations and exchanges information with its neighbors asynchronously. More specifically, at every iteration of AsyAD-ADMM, each node solves a local convex optimization problem for the one of the primal variables and utilizes a finite-time asynchronous approximate consensus protocol to obtain the value of the other variable which is close to the optimal value, since the cost function for the second primal variable is not decomposable. If the individual cost functions are convex, but not-necessarily differentiable, the proposed algorithm converges at a rate of O(1/k), where k is the iteration counter. The efficacy of AsyAD-ADMM is exemplified via a proof-of-concept distributed least square optimization problem with different performance-influencing factors investigated.

Keywords: Machine learning, Optimization algorithms, Cooperative control
Abstract: In this paper, we propose Push-SAGA, a decentralized stochastic first-order method for finite-sum minimization over a directed network of nodes. Push-SAGA combines node-level variance reduction to remove the uncertainty caused by stochastic gradients, network-level gradient tracking to address the distributed nature of the data, and push-sum consensus to tackle directed information exchange. We show that Push-SAGA achieves linear convergence to the exact solution for smooth and strongly convex problems and is thus the first linearly-convergent stochastic algorithm over arbitrary strongly connected directed graphs. We also characterize the regime in which Push-SAGA achieves a linear speed-up compared to its centralized counterpart and achieves a network-independent convergence rate. We illustrate the behavior and convergence properties of Push-SAGA with the help of numerical experiments on strongly convex and non-convex problems.

Keywords: Information theory and control, Automata, Intelligent systems
Abstract: In this paper, we introduce complexity-aware planning for finite-horizon deterministic finite automata with rewards as outputs, based on Kolmogorov complexity. Kolmogorov complexity is considered since it can detect computational regularities of deterministic optimal policies. We present a planning objective yielding an explicit trade-off between a policy's performance and complexity. It is proven that maximising this objective is non-trivial in the sense that dynamic programming is infeasible. We present two algorithms obtaining low-complexity policies, where the first algorithm obtains a low-complexity optimal policy, and the second algorithm finds a policy maximising performance while maintaining local (stage-wise) complexity constraints. We evaluate the algorithms on a simple navigation task for a mobile robot, where our algorithms yield low-complexity policies that concur with intuition.

Keywords: Information theory and control, Computational methods, Computer-aided control design
Abstract: We consider the task of remote state estimation and stabilization of disturbed linear plants via noisy communication channels. In 2007 Matveev and Savkin established a surprising link between this problem and Shannon’s theory of zero-error communication. By applying very recent results of computability of the channel reliability function and computability of the zero-error capacity of noisy channels by Boche and Deppe, we analyze if, on the set of linear timeinvariant systems paired with a noisy communication channel, it is uniformly decidable by means of a Turing machine whether remote state estimation and stabilization is possible. The answer to this question largely depends on whether the plant is disturbed by random noise or not. Our analysis incorporates scenarios both with and without channel feedback, as well as a weakened form of state estimation and stabilization. In the broadest sense, our results yield a fundamental limit to the capabilities of computer-aided design and autonomous systems, assuming they are based on real-world digital computers.

Keywords: Stochastic systems, Kalman filtering, Information theory and control
Abstract: We derive a new formulation of NRDF for time-varying multivariate Gauss-Markov processes driven by correlated noise modeled as a first order autoregressive moving average (ARMA(1,1)) process with MSE distortion constraint. To arrive to this formulation, we prove a key result, namely, the Gauss-Markov process with correlated noise can be equivalently written as a linear functional of the sufficient statistic of the correlated noise and its orthogonal innovations process. We then obtain our formulation under the assumption that the past samples of the sufficient statistic of the correlated noise process are known at the encoder and the decoder of our multi-input multi-output system. For jointly Gaussian processes, we characterize the problem, find its optimal realization (including its optimization coefficients) and solve it by showing that is semidefinite representable. Interestingly, the optimal realization of this problem reveals that our original assumption can be relaxed to one that only considers the past sufficient statistics available at the decoder but not at the encoder. For scalar-valued processes, we also derive a new analytical expression. The generality of our results (both for vector and scalar processes) is demonstrated by recovering known results obtained for independent noise processes.

Keywords: Information theory and control, Stochastic systems
Abstract: We consider the problem of characterizing the flow of information in stochastic systems. Recently, several measures of partial information decomposition (PID) have been proposed which, for a fixed target variable, can distinguish unique, redundant, and synergistic contributions from the predictor variables. We study how each of those partial informations travel in a Markov chain, entering at one variable, passing through several variables, and eventually exiting downstream. Our work is agnostic to specific partial information decomposition (PID) measures. We investigate partial information flow among variables relating to overflow events in a river system.

Keywords: Stability of nonlinear systems, Information theory and control, Quantized systems
Abstract: We study topological entropy of a nonlinear system represented as an interconnection of smaller subsystems. Under suitable assumptions on the Jacobian matrices characterizing the interconnection, we obtain an explicit upper bound on the entropy of the overall system, and show that it can be related to upper bounds on the entropies of the subsystems. We also analyze in detail the special case of a cascade connection of two subsystems, establishing an upper bound on the entropy which is more tightly linked to individual entropy bounds for the subsystems.

Keywords: Autonomous systems, Information theory and control, Filtering
Abstract: The performance of tracking algorithms that use bearing measurements is dependent on the sensor-target geometry. As a result, a sensor can maneuver to maximize the accuracy of estimates. The Renyi divergence has been shown to be an effective metric for selecting guidance commands that maximize information gain, thereby increasing the accuracy of estimates. However, there is no closed form solution to the Renyi divergence in applications with nonlinear measurements. As a result, the Renyi divergence is computed for a set of discretized guidance commands. Computing the Renyi divergence for each admissible command is computationally expensive, and there is no clear guidance on how many or which discrete guidance commands should be evaluated. We address multitarget tracking scenarios where a mobile sensor estimates the trajectory of one and three targets using the probability hypothesis density (PHD) filter. We show that for a PHD filter implementation the Renyi divergence need be computed only for guidance commands (heading) in a specific closed subset. The results from our analysis greatly reduce computational time, as commands outside of the subset need not be examined.

Keywords: Cyber-Physical Security, Control over communications, Linear systems
Abstract: Cyber security is nowadays an important research area in cyber-physical systems (CPS) to prevent possible disastrous physical consequences. Stealthy attacks such as covert attacks represent the most dangerous type of cyber attacks because they prevent any countermeasures and the attacked plant can become completely uncontrollable. In this paper, we consider stealthy targeted local covert attacks, which take into account the target of the adversary explicitly and need fewer disruption resources than standard covert attacks. We show conditions when the local attack can achieve its target and simultaneously remain stealthy. The provided conditions help to analyze the CPS on potential fraud and enhance the cyber security. Moreover, it is shown how the attacker may design a controller to generate a stealthy targeted local covert attack. Experimental results on the well-established benchmark system, the three-tank system, are provided to illustrate the main results.

Keywords: Cyber-Physical Security, Control Systems Privacy, Kalman filtering
Abstract: In this paper, we investigate the confidentiality of the reference signal of a feedback system under sensor attacks. In particular, we analyze the conditions for when an attacker with perfect model knowledge and access to the sensor measurements can obtain an unbiased estimate of the reference signal in a feedback control system such that the estimate's error covariance converges to zero. We call such an estimate a perfect estimate. Under the assumption of linear dynamics for the plant, the controller, and the reference signal, we show that the attacker can perfectly estimate the reference signal if and only if the reference dynamics do not have eigenvalues outside the unit circle. This implies that an attacker is able to perfectly estimate common reference signals such as step functions and sinusoidal signals from noisy measurements. The convergence rate of the estimate is, however, not exponentially fast for common reference signals and depends on the reference signal to be estimated as well as the controller used. We verify our results numerically with a simulation of a three-tank system.

Keywords: Cyber-Physical Security, Optimization, Optimal control
Abstract: This paper addresses the issue of data injection attacks on control systems. We consider attacks which aim at maximizing system disruption while staying undetected in the finite horizon. The maximum possible disruption caused by such attacks is formulated as a non-convex optimization problem whose dual problem is a convex semi-definite program. We show that the duality gap is zero using S-lemma. To determine the optimal attack vector, we formulate a soft-constrained optimization problem using the Lagrangian dual function. The framework of dynamic programming for indefinite cost functions is used to solve the soft-constrained optimization problem and determine the attack vector. Using the Karush–Kuhn–Tucker conditions, we also provide necessary and sufficient conditions under which the obtained attack vector is optimal to the primal problem. Finally, we illustrate the results through numerical examples.

Keywords: Attack Detection, Cyber-Physical Security, Observers for Linear systems
Abstract: We study the decentralized resilient state-tracking problem in which each node in a network has the objective of tracking the state of a linear dynamical system based on its local measurements and information exchanged with its neighboring nodes, despite an attack on some of the nodes. We propose a novel algorithm that solves the decentralized resilient state-tracking problem by relating it to the dynamic average consensus problem. Compared with existing solutions in the literature, our algorithm provides a solution for the most general class of decentralized resilient state-tracking problem instances.

Keywords: Cyber-Physical Security, Game theory, Sensor networks
Abstract: We study a game-theoretic model of the interactions between a Cyber-Physical System's (CPS) operator (the defender) against an attacker who launches stepping-stone attacks to reach critical assets within the CPS. We consider that, in addition to optimally allocating its security budget to protect the assets, the defender may choose to modify the CPS through network design interventions. In particular, we propose and motivate four ways in which the defender can introduce additional nodes in the CPS; these nodes may be intended as additional safeguards, be added for functional or structural redundancies, or introduce additional functionalities in the system. We analyze the security implications of each of these design interventions, and evaluate their impacts on the security of an automotive network as our case study. We motivate the choice of the attack graph for this case study and elaborate how the parameters in the resulting security game are selected using the CVSS metrics and the ISO-26262 ASIL ratings as guidance. We then use numerical experiments to verify and evaluate how our proposed network interventions may be used to guide improvements in automotive security.

Keywords: Energy systems, Power systems, Smart grid
Abstract: This paper proposes a new linear power flow model for distribution system with accurate voltage magnitude estimates. The new model can be seen as a generalization of LinDistFlow model to multiphase distribution system with generic network topology (radial or meshed) around arbitrary linearization point. We have shown that the approximation quality of the proposed model strictly dominates that of the fixed-point linearization (FPL) method, a popular linear power flow model for distribution system analysis, when both are linearized around zero injection point. Numerical examples using standard IEEE test feeders are provided to illustrate the effectiveness of the proposed model as well as the improvement in accuracy over existing methods when linearized around non-zero injection points.

Keywords: Power systems, Cooperative control, Distributed control
Abstract: With increasing integration of distributed energy resources (DERs), distribution systems (DS) with DERs are expected to provide proactive grid services. The result is that power references, or known as dispatch signals, required for DS can become faster changing than in legacy power system. Consensus-based integral controls have been proposed for the purpose of coordinating power generations of DERs in DS to match the power references. These existing controls assume the integral control signal to be sufficiently slow or constant, and ignore the potential dynamics of the integral controller when tracking more varying power references. Therefore, in this paper we present an improvement for such controls by deriving the stability condition utilizing generalized Nyquist criterion. The stability condition is quantified by a set of integral gains that guarantee stability in closed-loop system without assuming a constant integral signal. A rule-of-thumb criterion is also derived to instruct the design of the consensus topology that can provide faster convergence rate for the closed-loop system. The stability and convergence improvements developed in this paper are demonstrated and verified through numerical examples.

Keywords: Power systems, Large-scale systems
Abstract: Interconnecting power systems has a number of advantages such as better electric power quality, increased reliability of power supply, economies of scales through production and reserve pooling and so forth. Simultaneously, it may jeopardize the overall system stability with the emergence of so-called inter-area oscillations, which are coherent oscillations involving groups of rotating machines separated by large distances up to thousands of kilometers. These often weakly damped modes may have harmful consequences for grid operation, yet despite decades of investigations, the mechanisms that generate them are still poorly understood, and the existing theories are based on assumptions that are not satisfied in real power grids where such modes are observed. Here we construct a matrix perturbation theory of large interconnected power systems that clarifies the origin and the conditions for the emergence of inter-area oscillations. We show that coherent inter-area oscillations emerge from the zero-modes of a multi-area network Laplacian matrix, which hybridize only weakly with other modes, even under significant capacity of the inter-area tie-lines, i.e. even when the standard assumption of area partitioning is not satisfied. The general theory is illustrated on a two-area system, and numerically applied to the well-connected PanTaGruEl model of the synchronous grid of continental Europe.

Keywords: Power systems, Optimization, Energy systems
Abstract: This paper focuses on a day-ahead scheduling problem on generating power of thermal power plants and charge/discharge power of multiple storage batteries based on the information of the interval prediction of photovoltaic (PV) /demand power. Our purpose is to obtain an exact range of each optimal operation schedule for any possible PV/demand profiles of the interval prediction. To find this range efficiently, it is necessary to focus on the monotonicity of the solution with respect to the parameter, which corresponds to the PV/demand. If the solution is monotonically increasing or decreasing with respect to the parameter, we can find the exact range of the solution in a finite number of trials. The monotonicity has been analyzed in various power settings, and the exact range of the optimal operation schedule has been calculated so far. However, we could not treat multiple storage batteries in the previous work. To adapt to future power system problems, it is necessary to analyze the monotonicity in the case where the supplier has multiple storage batteries. In this paper, we show sufficient conditions for the monotonicity of the optimal generating power for a thermal power plant taking into account existence of two storage batteries.

Keywords: Power systems, Formal Verification/Synthesis, Switched systems
Abstract: In this paper, we present a provably correct controller synthesis approach for switched stochastic control systems with metric temporal logic (MTL) specifications with provable probabilistic guarantees. We first present the stochastic control bisimulation function for switched stochastic control systems, which bounds the trajectory divergence between the switched stochastic control system and its nominal deterministic control system in a probabilistic fashion. We then develop a method to synthesize the optimal control input signals for the nominal deterministic system with calculated robustness margins, and the same input signals can be applied to the switched stochastic control system with a lower bound guarantee for satisfying the MTL specifications. We implement our robust stochastic controller synthesis approach on both a four- bus power system and a nine-bus power system under generation loss disturbances, with MTL specifications expressing requirements for the grid frequency deviations, wind turbine generator rotor speed variations and the power flow constraints at different power lines.

Keywords: Power systems, Kalman filtering, Hybrid systems
Abstract: In this paper, we model the generalized power system state estimation problem as a hybrid estimation problem in which the network configuration and voltage phasor states are treated as discrete and continuous states, respectively. We propose the Multiple model Adaptive Power system State Estimator (MAPSE), which implements a multiple model adaptive estimator with a bank of Unscented Kalman Filters (UKFs) to provide a running estimate of the voltage phasor states and a probability distribution for the candidate network configurations. The MAPSE updates the network configuration probability distribution by applying a Bayesian recursion to the measurement residuals from each of the UKFs at each timestep. Simulation on a modified version of the 33 node distribution network demonstrates successful network configuration detection and accurate voltage phasor state estimation.

Keywords: Biological systems, Computational methods, Estimation
Abstract: The COVID-19 pandemic that has swept the world has shown a wide variety of behavior across countries, ranging from a gentle rise and gentle fall in India, to multiple waves with each peak higher than its predecessor in the USA, UK, and other countries. It is therefore a challenge to present a unified model that can manifest this wide range of behavior. In this paper, we present a new mathematical model called SUTRA for pandemics that have undetected (asymptomatic) patients, and demonstrate that it does indeed accurately predict the variety of behavior observed during the COVID-19 pandemic. The acronym SUTRA stands for Susceptible, Undetected, Tested (positive), and Removed Approach. There are several novel features of our proposed model. First, whereas previous pa- pers have divided the patient population into Asymptomatic and Infected, we divide patients instead into Tested (T ) and Undetected (U ). This explicitly accounts for the fact that, due to contact tracing and other such protocols, some fraction of asymptomatic patients do get detected. Second, we introduce a parameter called “reach,” to take into account the spatial spread of apandemic over time. Third, we present numerically stable methods for estimating the parameters in our model. We have applied our model to predict the progression of the COVID-19 pandemic in several countries, displaying a variety of behaviors. In all cases, the predictions closely match the actually observed outcomes. In the interests of brevity, we present only the predictions for India.

Keywords: Systems biology, Optimal control, Uncertain systems
Abstract: We consider a class of epidemiological models in which a compartmental linear system, including various categories of infected individuals (e.g. asymptomatic, symptomatic, quarantined), is fed back by a positive feedback, representing contagion. The positive feedback gain decreases (in a sort of negative feedback) as the epidemic evolves, due to the decrease in the number of susceptible individuals. We first propose a convergence result based on a special copositive Lyapunov function. Then, we address a major problem for this class of systems: the deep uncertainty affecting parameter values. We face the problem adopting techniques from optimal and robust control theory to assess the sensitivity of the model. For this class of systems, the optimal control solution has a peculiar decoupling property that no shooting procedure is required. Finally, we exploit the obtained bounds to assess the effectiveness of possible epidemic control strategies, including intermittent restrictions adopted during the COVID-19 pandemic.

Keywords: Network analysis and control, Stability of nonlinear systems, Predictive control for nonlinear systems
Abstract: This paper is concerned with the design of intermittent non-pharmaceutical strategies to mitigate the spread of the COVID-19 epidemic exploiting network epidemiological models. Specifically, by studying a variational equation for the dynamics of the infected, we derive, using contractivity arguments, a condition that can be used to guarantee that the effective reproduction number is less than unity. This condition (i) is easily computable, (ii) is interpretable, being directly related to the model parameters, and (iii) can be used to enforce a scalability condition that prohibits the amplification of disturbances within the network system. We then include satisfaction of such a condition as a constraint in a Model Predictive Control problem so as to mitigate (or suppress) the spread of the epidemic while minimizing the economic impact of the interventions. A data-driven model of Italy as a network of three macro-regions (North, Center, and South), whose parameters are identified from real data, is used to illustrate and evaluate the effectiveness of the proposed control strategy.

Keywords: Nonlinear systems identification, Estimation, Optimization
Abstract: We address the model identification and the computation of optimal vaccination policies for the coronavirus disease 2019 (COVID-19). We consider a stochastic Susceptible--Infected--Removed (SIR) model that captures the effect of multiple vaccine treatments, each requiring a different number of doses and providing different levels of protection against the disease. We show that the inclusion of vaccination data enables the estimation of the state of the model and key model parameters that are otherwise not identifiable. These estimates can, in turn, be used to design strategic approaches to vaccination that aim at minimizing the number of deaths and the economic cost of the disease. We illustrate these results with numerical examples.

Keywords: Biological systems, Modeling, Network analysis and control
Abstract: In this paper, we describe some of the most important objectives and needs in pandemic control. We identify the main open problems in the different stages of the decision making process, as well as the most significant challenges to overcome them, leading to promising future research directions. We provide a concise review of the most recent literature describing such challenges, highlighting the main results, achievements and methodologies that can be employed to address them. In particular, we discuss some promising recent techniques that have been successfully applied to the Covid-19 pandemic and could be very valuable in the design of novel methodologies to face similar challenges in future pandemics.

Keywords: Biological systems, Adaptive control, Model/Controller reduction
Abstract: In order to control highly-contagious and prolonged outbreaks, public health authorities intervene to institute social distancing, lock-down policies, and other Non-Pharmaceutical Interventions (NPIs). Given the high social, educational, psychological, and economic costs of NPIs, authorities tune them, alternatively tightening up or relaxing rules, with the result that, in effect, a relatively flat infection rate results. For example, during the summer of 2020 in parts of the United States, daily COVID-19 infection numbers dropped to a plateau. This letter approaches NPI tuning as a control-theoretic problem, starting from a simple dynamic model for social distancing based on the classical SIR epidemics model. Using a singular-perturbation approach, the plateau becomes a Quasi-Steady-State (QSS) of a reduced two-dimensional SIR model regulated by adaptive dynamic feedback. It is shown that the QSS can be assigned and it is globally asymptotically stable. Interestingly, the dynamic model for social distancing can be interpreted as a nonlinear integral controller. Problems of data fitting and parameter identifiability are also studied for this model. This letter also discusses how this simple model allows for a meaningful study of the effect of population size, vaccinations, and the emergence of second waves.

Keywords: Autonomous robots, Machine learning, Estimation
Abstract: Estimating and reacting to external disturbances is of fundamental importance for robust control of quadrotors. Existing estimators typically require significant tuning or training with a large amount of data, including the ground truth, to achieve satisfactory performance. This paper proposes a data-efficient differentiable moving horizon estimation (DMHE) algorithm that can automatically tune the MHE parameters online and also adapt to different scenarios. We achieve this by deriving the analytical gradient of the estimated trajectory from MHE with respect to the tuning parameters, enabling end-to-end learning for auto-tuning. Most interestingly, we show that the gradient can be calculated efficiently from a Kalman filter in a recursive form. Moreover, we develop a model-based policy gradient algorithm to learn the parameters directly from the trajectory tracking errors without the need for the ground truth. The proposed DMHE can be further embedded as a layer with other deep neural networks for joint optimization. Finally, we demonstrate the effectiveness of the proposed method via simulation, where challenging scenarios such as ground effect, square-wave and sinusoidal disturbances are examined.

Keywords: Autonomous robots
Abstract: Achieving collision avoidance or rendezvous between moving objects are important objectives in robotic systems. In performing collision avoidance or rendezvous maneuvers, the relative shapes of the objects play an important role. The literature largely models the shapes of the objects as circles, and this can make the maneuvers very conservative, especially when the objects are of elongated shape. In this paper, we model the shapes of the objects using quadric surfaces. A collision/rendezvous cone concept is employed. A computationally efficient approach to compute the collision/rendezvous cone between finite-sized quadric surfaces (which are not necessarily of the same shape) is presented. Nonlinear analytical guidance laws that perform collision avoidance or rendezvous maneuvers, are subsequently developed. Simulation results demonstrating the working of the developed guidance laws are presented.

Keywords: Learning, Autonomous robots, Machine learning
Abstract: We present a method for contraction-based feedback motion planning of locally incrementally exponentially stabilizable systems with unknown dynamics that provides probabilistic safety and reachability guarantees. Given a dynamics dataset, our method learns a deep control-affine approximation of the dynamics. To find a trusted domain where this model can be used for planning, we obtain an estimate of the Lipschitz constant of the model error, which is valid with a given probability, in a region around the training data, providing a local, spatially-varying model error bound. We derive a trajectory tracking error bound for a contraction-based controller that is subjected to this model error, and then learn a controller that optimizes this tracking bound. With a given probability, we verify the correctness of the controller and tracking error bound in the trusted domain. We then use the trajectory error bound together with the trusted domain to guide a sampling-based planner to return trajectories that can be robustly tracked in execution. We show results on a 4D car, a 6D quadrotor, and a 22D deformable object manipulation task, showing our method plans safely with learned models of high-dimensional underactuated systems, while baselines that plan without considering the tracking error bound or the trusted domain can fail to stabilize the system and become unsafe.

Keywords: Optimization, Optimal control, Autonomous robots
Abstract: We consider problems in which a mobile robot samples an unknown function defined over its operating space, so as to find a global optimum of this function. The path travelled by the robot matters, since it influences energy and time requirements. We consider a branch-and-bound algorithm called deterministic optimistic optimization, and extend it to the path-aware setting, obtaining emph{path-aware optimistic optimization} (OOPA). In this new algorithm, the robot decides how to move next via an optimal control problem that maximizes the long-term impact of the robot trajectory on lowering the upper bound, weighted by bound and function values to focus the search on the optima. An online version of value iteration is used to solve an approximate version of this optimal control problem. OOPA is evaluated in extensive experiments in two dimensions, where it does better than path-unaware and local-optimization baselines.

Keywords: Autonomous robots, Robotics, Flight control
Abstract: Due to the high demand on the actuators, physical components, and onboard computers, performing aggressive maneuvers autonomously is challenging for miniature fixed-wing unmanned aerial vehicles(UAVs), especially for one with a ducted-fan. In this paper, we proposed an integrated maneuver command generation and tracking control(MCGTC) scheme that enables fixed-wing UAVs to finish agile acrobatics. There are two key parts of our algorithm: 1)A maneuver command generator which is learned from a few demonstrations. Through a new paradigm of coordinate-free invariant, the geometric characteristics of the pilot's demonstrated trajectory are preserved 2)A tracking control structure consists of an incremental dynamic inversion(INDI) based airspeed controller, and a nonlinear dynamic inversion(NDI) based attitude controller. The feasibility and effectiveness of our proposed MCGTC scheme are verified through flight tests. Using a 2.5kg fixed-wing drone with a 70mm ducted fan, we are able to perform an agile Immelman maneuver in the physical world.

Keywords: Autonomous robots, Agents-based systems, Learning
Abstract: This paper presents an multi-agent motion planning algorithm for human-like navigation in dynamic environments. A cognitive hierarchy approach is used to model the motion of autonomous agents. We discuss potential levels of rationality and introduce a method to predict them in real-time. The rationality level prediction is achieved by observing the kinodynamic distance (KD) of other agents. An offline training phase is required to learn the maximum KD from multiple boundary value problems. Collision avoidance is ensured by introducing artificial obstacles in the environment based on the predicted levels of rationality. The motion planning is then carried out using RRT-QX. The effectiveness of the bounded rational motion planning algorithm is illustrated in simulations.

Keywords: Nonlinear systems
Abstract: “You only learn the shape of the river; and you learn it with such absolute certainty that you can always steer by the shape that’s in your head, and never mind the one that’s before your eyes” (Mark Twain, apprentice as a Mississippi river pilot, 1983).

This Twain’s intuition hides the well-known Internal Model Principle, a paradigm for any learning or adapting process. Creating, storing and updating an internal representation of the outside world is what permits an agent (e.g. a human being, an animal, an organism or a robot) to engage with the surrounding environment and smoothly operate in it, by taking advantage of earlier experiences and yet taking into account deviations from the nominal context and uncertainties never experienced before. It explains how humans and animals can operate safely in an energy efficient way, in some cases allowing them to successfully accomplish complex tasks in presence of limited computational and sensory capabilities.

The talk addresses the subject from several perspectives. I will start by overviewing how internal model principles are conceptually cast in many fields of science, ranging from neuroscience, biology, and cognitive science. Then, I will introduce the control community formalism, which more than others was successful in abstracting and making the concept rigorous. The presentation will stand on the pioneering contributions given for linear systems in the early 70s, which formalized Twain’s intuition with the paradigm that any robust regulator necessarily incorporates a model of the dynamics generating all the ideal steady state control signals associated to the perfect execution of the task. By riding on this principle, I will review how the concepts of ”model”, “robustness”, “steady state”, “incorporation”, “perfect execution” and “task” have been developed and refined during the last 30 years of research for continuous-time and hybrid systems. A special emphasis will be reserved to nonlinear dynamics for which I will present recent research outcomes. The talk will end with some open research challenges on the subject showing that Twain’s ability to navigate the Mississippi river eyes-closed has still many aspects not fully captured by systems theory.

Keywords: Discrete event systems
Abstract: The control theory of discrete-event systems (DESs) is a modeling framework for capturing the ordering of events or actions. Discrete-event systems modeling can be complementary to traditional continuous-time systems modeling or can be used alongside or in concert with continuous-time modeling in hybrid systems. Since decision-making is tantamount to prescribing which actions should or should not happen or which actions should happen before others, the body of work in DES theory is well-positioned to allow us to tackle security problems in cyber-physical systems. In this talk we present different approaches in DES control theory that address various problems in the security of systems and networks.

In particular, we examine the notion of opacity, which is the property of ensuring that secret states or secret sequences of events are not discernible from non-secret states or events to a hostile agent. We also look at cases where systems are attacked by adversarial agents that manipulate sensor outputs (i.e., event occurrences generated by a plant) so that a supervisor (i.e., a DES controller) is fooled into thinking the system is in some state that it is not in. We discuss the challenges of modeling security and secrecy problems using discrete-event systems.

Keywords: Learning, Manufacturing systems and automation, Iterative learning control
Abstract: Learning-based control methods utilize run-time data from the underlying process to improve the controller performance under model mismatch and unmodeled disturbances. This is beneficial for optimizing industrial processes, where the dynamics are difficult to model, and the repetitive nature of the process can be exploited. In this work, we develop an iterative approach for repetitive precision motion control problems where the objective is to follow a reference geometry with minimal tracking error. Our method utilizes a nominal model of the process and learns the mismatch using Gaussian Process Regression (GPR). The control input and the GPR data are updated after each iteration to improve the performance in a run-to-run fashion. We provide a preliminary convergence analysis, implementation details of the proposed controller for minimizing different error types, and a case study where we demonstrate improved tracking performance with simulation and experimental results.

Keywords: Neural networks, LMIs, Lyapunov methods
Abstract: In this paper, we analyze the stability of feedback interconnections of a linear time-invariant system with a neural network nonlinearity in discrete time. Our analysis is based on abstracting neural networks using integral quadratic constraints (IQCs), exploiting the sector-bounded and slope-restricted structure of the underlying activation functions. In contrast to existing approaches, we leverage the full potential of dynamic IQCs to describe the nonlinear activation functions in a less conservative fashion. To be precise, we consider multipliers based on the full-block Yakubovich / circle criterion in combination with acausal Zames-Falb multipliers, leading to linear matrix inequality based stability certificates. Our approach provides a flexible and versatile framework for stability analysis of feedback interconnections with neural network nonlinearities, allowing to trade off computational efficiency and conservatism. Finally, we provide numerical examples that demonstrate the applicability of the proposed framework and the achievable improvements over previous approaches.

Keywords: Machine learning, Distributed control, Control system architecture
Abstract: When designing large-scale distributed controllers, the information sharing constraints between sub-controllers, as defined by a communication topology interconnecting them, are as important as the controller itself. Controllers implemented using dense topologies typically outperform those implemented using sparse topologies, but it is also desirable to minimize the cost of controller deployment. Motivated by the above, we introduce a compact but expressive graph recurrent neural network (GRNN) parameterization of distributed controllers that is well suited for distributed controller and communication topology co-design. Our proposed parameterization enjoys a local and distributed architecture, similar to previous Graph Neural Network (GNN)-based parameterizations, while further naturally allowing for joint optimization of the distributed controller and communication topology needed to implement it. We show that the distributed controller/communication topology co-design task can be posed as an ell_1-regularized empirical risk minimization problem that can be efficiently solved using stochastic gradient methods. We run extensive simulations to study the performance of GRNN-based distributed controllers and show that (a) they achieve performance comparable to GNN-based controllers while having fewer free parameters, and (b) our method allows for performance/communication density tradeoff curves to be efficiently approximated.

Keywords: Machine learning, Lyapunov methods, Learning
Abstract: Inferring the intent of an intelligent agent from demonstrations and subsequently predicting its behavior, is a critical task in many collaborative settings. A common approach to solve this problem is the framework of inverse reinforcement learning (IRL), where the observed agent, e.g., a human demonstrator, is assumed to behave according to an intrinsic cost function that reflects its intent and informs its control actions. In this work, we reformulate the IRL inference problem to learning control Lyapunov functions (CLF) from demonstrations by exploiting the inverse optimality property, which states that every CLF is also a meaningful value function. Moreover, the derived CLF formulation directly guarantees stability of the system under the inferred control policies. We show the flexibility of our proposed method by learning from goal-directed movement demonstrations in a continuous environment.

Keywords: Uncertain systems, Predictive control for nonlinear systems, Machine learning
Abstract: A fundamental challenge in control applications stems from the lack of a sufficiently detailed system model that can be used for systematic controller design and tuning under uncertainty. This paper presents a fully data-driven robust controller design approach based on any finite set of closed-loop performance measures. We first present a method for emulating the unknown plant dynamics using a Gaussian process (GP) model learned from input-output data. By running closed-loop simulations under realizations of the GP model using posterior sampling, the impact of system uncertainties on the closed-loop performance measures is quantified. We then formulate the robust controller design problem as a constrained Bayesian optimization (CBO) problem defined in terms of the GP-emulated performance measures. To ensure a sufficient number of samples are used to estimate the worst-case performance measures, we derive a bound on the joint probability of violation that is independent of the number or probability distribution of the uncertainties. The advantages of the proposed approach are illustrated on a benchmark control problem, which demonstrates guaranteed probabilistic estimates on the worst-case performance measures are provided at every iteration of CBO.

Keywords: Machine learning, Predictive control for linear systems, Optimization
Abstract: We propose a data-driven online convex optimization algorithm for controlling dynamical systems. In particular, the control scheme makes use of an initially measured input-output trajectory and behavioral systems theory which enable it to handle unknown discrete-time linear time-invariant systems as well as a priori unknown time-varying cost functions. Further, only output feedback instead of full state measurements is required for the proposed approach. Analysis of the closed loop's performance reveals that the algorithm achieves sublinear regret if the variation of the cost functions is sublinear. The effectiveness of the proposed algorithm, even in the case of noisy measurements, is illustrated by a simulation example.

Keywords: Estimation, Smart grid, Power systems
Abstract: The increasing integration of intermittent renewable generation, especially at the distribution level, necessitates advanced planning and optimisation methodologies contingent on the knowledge of the admittance matrix, capturing the topology and line parameters of an electric network. However, a reliable estimate of the admittance matrix may either be missing or quickly become obsolete for temporally varying grids. In this work, we propose a data-driven identification method utilising voltage and current measurements collected from micro-PMUs. More precisely, we first present a maximum likelihood approach and then move towards a Bayesian framework, leveraging the principles of maximum a posteriori estimation. In contrast with most existing contributions, our approach not only factors in measurement noise on both voltage and current data, but is also capable of exploiting available a priori information such as sparsity patterns and known line admittances. Simulations conducted on benchmark cases demonstrate that, compared to other algorithms, our method can achieve greater accuracy.

Keywords: Power systems, Learning, Smart grid
Abstract: Under voltage load shedding has been considered as a standard and effective measure to recover the voltage stability of the electric power grid under emergency and severe conditions. However, this scheme usually trips a massive amount of load, which can be unnecessary and harmful to customers. Recently, deep reinforcement learning (RL) has been regarded and adopted as a promising approach that can significantly reduce the amount of load shedding. However, like most existing machine learning (ML)-based control techniques, RL control usually cannot guarantee the safety of the systems under control. In this paper, we introduce a novel safe RL method for load shedding of power systems, that can enhance the safe voltage recovery of the electric power grid after experiencing faults. Unlike the standard RL method, we integrate into the reward function a Barrier function that goes to minus infinity when the system state goes to the safety bounds. Consequently, the resulting optimal control policy, that maximizes the reward function, can render the power system to avoid the safety bounds. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark is performed to demonstrate the effectiveness of the proposed safe RL emergency control, as well as its adaptive capability to faults not seen in the training.

Keywords: Energy systems, Machine learning
Abstract: With the increase in penetration of renewable energy resources, there is a growing concern for their ability to provide predictable power that matches the forecasted power demand. Wind generation is a direct function of wind speed/direction and, in contrast to conventional generation systems, is not easily dispatchable. With an accurate estimation of generation, bids for energy to the electric grid can be planned accordingly. This paper proposes a framework for power output prediction of wind farms for different time scales. Physics-based models like FLORIS, developed by the National Renewable Energy Laboratory, can estimate power produced for a given time series atmospheric data. This paper shows that these estimates can have errors outside the acceptable bounds required for grid operation. This issue is addressed by introducing a learned correction term using LSTMs in addition to the output of FLORIS. The effectiveness of this approach is demonstrated via numerical studies of short-term and day-ahead forecasts for a utility-scale farm.

Keywords: Energy systems, Power systems, Optimization
Abstract: Parameters involved in the formulation of optimization problems are often partially unknown or random. A popular way to mitigate the effect of uncertainty is using joint chance constraints, which guarantee constraint satisfaction with high probability but are challenging to solve. In this paper, we analyze an approach for joint chance constrained problems that involves iteratively tuning problem parameters. We first show that existing naive approaches to tuning can lead to solutions without feasibility guarantees. We then introduce a two-step approach, where a tuning-based solution generation step is followed by an a posteriori solution verification step.

A main challenge of the two-step approach is to guarantee that the solution generated in the first step has a high probability of being verified as feasible in the second step. We therefore analyze how the relationship between the feasibility criteria used in each step impacts the probability of obtaining a feasible solution.

We demonstrate our results in a numerical case study of the optimal power flow problem.

Keywords: Machine learning, Control applications, Smart grid
Abstract: Synchronized data provide unprecedented opportunities for inferring voltage frequencies and rates of change of frequencies (ROCOFs) across the buses of a power system. Aligned to this goal, this work puts forth a novel framework for learning dynamics after small-signal disturbances by leveraging the tool of Gaussian processes (GPs). We extend results on inferring the input and output of a linear time-invariant system using GPs to the multi-input multi-output setup by exploiting power system swing dynamics. This physics-aware learning technique captures time derivatives in continuous time, accommodates data streams sampled potentially at different rates, and can cope with missing data and heterogeneous levels of accuracy. While Kalman filter-based approaches require knowing all system inputs, the proposed framework handles readings of system inputs, outputs, their derivatives, and combinations thereof on an arbitrary subset of buses. Relying on minimal system information, it further provides uncertainty quantification in addition to point estimates for dynamic grid signals. The required spatiotemporal covariances are obtained by exploring the statistical properties of approximate swing dynamics driven by ambient disturbances. Numerical tests verify that this technique can infer frequencies and ROCOFs at non-metered buses under (non)-ambient disturbances for a linearized dynamic model of the IEEE 300-bus benchmark.

Keywords: Power systems, Machine learning, Stability of nonlinear systems
Abstract: Transient stability of power systems is becoming increasingly important because of the growing integration of renewable resources. These resources lead to a reduction in mechanical inertia but also provide increased flexibility in frequency responses. Namely, their power electronic interfaces can implement almost arbitrary control laws. To design these controllers, reinforcement learning (RL) has emerged as a powerful method in searching for optimal non-linear control policy parameterized by neural networks.

A key challenge is to enforce that a learned controller must be stabilizing. This paper proposes a Lyapunov regularized RL approach for optimal frequency control for transient stability in lossy networks. Because the lack of an analytical Lyapunov function, we learn a Lyapunov function parameterized by a neural network. The losses are specially designed with respect to the physical power system. The learned neural Lyapunov function is then utilized as a regularization to train the neural network controller by penalizing actions that violate the Lyapunov conditions. Case study shows that introducing the Lyapunov regularization enables the controller to be stabilizing and achieve smaller losses.

Keywords: Chemical process control, Control applications, Statistical learning
Abstract: Partial least squares (PLS) has gained popularity in many domains such as industrial internet of things, bioinformatics, and econometrics due to its ability to deal with limited data, collinearity, and relevance to supervised machine learning. PLS has also been applied to system identification and subspace identification where the model is high order, high dimension, or collinear due to the lack of rich excitation. However, all PLS algorithms to date are iterative in calculating the sequence of latent variables, unlike other related methods such as principal component regression. The iterative PLS estimation has made it difficult to perform statistical analysis. In this paper, we propose a novel non-iterative PLS algorithm based on the Krylov sequence used in PLS algorithms. Only a singular value decomposition is needed to obtain an equivalent PLS model for multiple PLS latent factors. The non-iterative PLS algorithm extracts the same latent space as the conventional PLS, which are demonstrated with a couple of industrial application cases.

Keywords: Chemical process control, Statistical learning, Modeling
Abstract: In this paper, we propose a novel latent vector autoregressive (LaVAR) modeling algorithm with a canonical correlation analysis (CCA) objective to estimate a fully-interacting reduced dimensional dynamic model. This algorithm is an advancement of the dynamic inner canonical correlation analysis (DiCCA) algorithm, which builds univariate latent autoregressive models that are non-interacting. The dynamic latent variable scores of the proposed algorithm are enforced to be orthogonal, similar to those of DiCCA. An application case study on an industrial dataset is given to illustrate the superiority of the proposed algorithm. The reduced-dimensional latent dynamic model has potential applications for prediction, control, and diagnosis of systems with rich sensors, such as industrial internet of things.

Keywords: Statistical learning, Machine learning, Randomized algorithms
Abstract: We study the problem of nonstochastic bandits with expert advice, extending the setting from finitely many experts to any countably infinite set: A learner aims to maximize the total reward by taking actions sequentially based on bandit feedback while benchmarking against a set of experts. We propose a variant of Exp4.P that, for finitely many experts, enables inference of correct expert rankings while preserving the order of the regret upper bound. We then incorporate the variant into a meta-algorithm that works on infinitely many experts. We prove a high-probability upper bound of tilde{mathcal{O}}(i^*K + sqrt{KT}) on the regret, up to polylog factors, where i^* is the unknown position of the best expert, K is the number of actions, and T is the time horizon. We also provide an example of structured experts and discuss how to expedite learning in such case. Our meta-learning algorithm achieves optimal regret up to polylog factors when i^* = tilde{mathcal{O}}(sqrt{T/K}). If a prior distribution is assumed to exist for i^*, the probability of optimality increases with T, the rate of which can be fast.

Keywords: Pattern recognition and classification, Statistical learning, Randomized algorithms
Abstract: In this paper we suggest two statistical hypothesis tests for the regression function of binary classification based on conditional kernel mean embeddings. The regression function is a fundamental object in classification as it determines both the Bayes optimal classifier and the misclassification probabilities. A resampling based framework is presented and combined with consistent point estimators of the conditional kernel mean map, in order to construct distribution-free hypothesis tests. These tests are introduced in a flexible manner allowing us to control the exact probability of type I error for any sample size. We also prove that both proposed techniques are consistent under weak statistical assumptions, namely, the type II error probabilities pointwise converge to zero.

Keywords: Estimation, Statistical learning, Uncertain systems
Abstract: This work introduces an active learning approach for hypothesis testing with uncertain likelihood models. Uncertain models appear when hypotheses' parameters are built from finite and limited training data. As a result, hypothesis testing performance is limited by the dearth of training data even as the number of observations increases asymptotically. Even with large amounts of observational data, decision-making at desired error rates can be impossible. This work proposes various active learning methods to collect as little additional training data as possible and still guarantee desired error bounds. These methods attempt to reduce the amount of observational and training data required sequentially and adaptively for each hypothesis until only one hypothesis is accepted. Finally, various active learning methods are compared in terms of their data collection costs to achieve the desired false rejection rate through simulations.

Keywords: Machine learning, Statistical learning, Markov processes
Abstract: The options framework for hierarchical reinforcement learning has increased its popularity in recent years and has made improvements in tackling the scalability problem in reinforcement learning. Yet, most of these recent successes are linked with a proper options initialization or discovery. When an expert is available, the options discovery problem can be addressed by learning an options-type hierarchical policy directly from expert demonstrations. This problem is referred to as hierarchical imitation learning and can be handled as an inference problem in a Hidden Markov Model, which is done via an Expectation-Maximization type algorithm. In this work, we propose a novel online algorithm to perform hierarchical imitation learning in the options framework. Further, we discuss the benefits of such an algorithm and compare it with its batch version in classical reinforcement learning benchmarks. We show that this approach works well in both discrete and continuous environments and, under certain conditions, it outperforms the batch version.

Keywords: Identification for control, Distributed control, Robust control
Abstract: The increase in available data and complexity of dynamical systems has sparked the research on data-based system performance analysis and controller design. In this paper, we extend a recent data-based approach for guaranteed performance analysis to distributed analysis of interconnected linear systems. We present a new set of sufficient LMI conditions based on noisy input-state data that guarantees mathcal{H}_infty performance and has a structure that is applicable to distributed controller synthesis from data. Sufficient LMI conditions based on noisy data are provided for the existence of a dynamic distributed controller that achieves mathcal{H}_infty performance. The presented approach enables scalable analysis and control of large-scale interconnected systems from noisy input-state data.

Keywords: Identification for control, Machine learning
Abstract: We introduce an approach to efficiently tune LQR controllers for linear time-invariant systems to match a prescribed closed-loop behavior, such as the one given by a reference model. The proposed approach is able to efficiently tune the LQR controller, even for high dimensional systems and is superior in terms of achieved tracking performance and other criteria with respect to global optimization methods commonly used for black-box, simulation-based, automated tuning.

Keywords: Grey-box modeling, Identification for control, Neural networks
Abstract: We introduce Neural Dynamical Systems (NDS), a method of learning dynamical models in various gray-box settings which incorporates prior knowledge in the form of systems of ordinary differential equations. NDS uses neural networks to estimate free parameters of the system, predicts residual terms, and numerically integrates over time to predict future states. A key insight is that many real dynamical systems of interest are hard to model because the dynamics may vary across rollouts. We mitigate this problem by taking a trajectory of prior states as the input to NDS and train it to dynamically estimate system parameters using the preceding trajectory.

We find that NDS learns dynamics with higher accuracy and fewer samples than a variety of deep learning methods that do not incorporate the prior knowledge and methods from the system identification literature which do. We demonstrate these advantages first on synthetic dynamical systems and then on real data captured from deuterium shots from a nuclear fusion reactor. Finally, we demonstrate that these benefits can be utilized for control in small-scale experiments.

Keywords: Identification, Identification for control, Adaptive control
Abstract: Online system identification entails optimization of a sequence of cost functions that are updated by data as it becomes available. The goal is to obtain a sequence of minimizers that converge to the true parameters. If the true parameters are the unique common minimizer of all of the costs in the sequence, then convergence of a sequence of minimizers to the true parameters can be viewed as convergence of the state of a dynamical system to an equilibrium. This paper investigates global asymptotic stability of the equilibrium of a dynamical system defined by online gradient-based optimization of a sequence of quadratic cost functions that have a unique common minimizer, but may individually have multiple global minimizers due to rank deficiency. Under a weak persistency-type condition, it is shown that global asymptotic stability can be guaranteed for this class of costs. These results are specialized to the case of least squares costs and illustrated by examples.

Keywords: Iterative learning control, Identification for control
Abstract: Data-driven iterative learning control can achieve high performance for systems performing repeating tasks without the need for modeling. The aim of this paper is to develop a fast data-driven method for iterative learning control that is suitable for massive MIMO systems through the use of efficient unbiased gradient estimates. A stochastic conjugate gradient descent algorithm is developed that uses dedicated experiments to determine the conjugate search direction and optimal step size at each iteration. The approach is illustrated on a multivariable example, and it is shown that the method is superior to both the earlier stochastic gradient descent and deterministic conjugate gradient descent methods.

Keywords: Learning, Identification for control, Uncertain systems
Abstract: This paper presents a barrier-certified joint model-learning/control framework for uncertain linear systems. An experience replay-based model-learning approach is presented to learn the uncertain system’s dynamic exponentially fast with a prescribed exponential rate. The approximated model and the exponential envelope at each time instant during model-learning are leveraged in forming a novel adaptive robustified control barrier function (ARCBF) to ensure safety specifications of the system despite uncertainty. Incorporation of the bound of approximation error into the ARCBF makes safety guarantee independent of the model-learning accuracy, and thus, it is employed to only apply safe policies to the system for further data acquisition, resulting in a safe model learning procedure. As the learning progresses, the set of feasible actions expands which results in a fast reduction of conservatism of the controller. Effect of uncertainty on Lyapunov-based controller is investigated. It is shown that the experience replay model learning error can be considered as a vanishing perturbation to the approximated model and therefore, stability of the system is preserved under a mild condition, resulting in non-conservative safe and stable model-learning and control. The efficacy of the method is shown in the simulation.

Keywords: Uncertain systems, Stochastic optimal control, Flight control
Abstract: Generalized Polynomial Chaos (gPC) theory has been widely used for representing parametric uncertainty in a system, thanks to its ability to propagate uncertainty evolution. In an optimal control context, gPC can be combined with several optimization techniques to achieve a control policy that handles effectively this type of uncertainty. Such a suitable method is Differential Dynamic Programming (DDP), leading to an algorithm that inherits the scalability to high-dimensional systems and fast convergence nature of the latter. In this paper, we expand this combination aiming to acquire probabilistic guarantees on the satisfaction of nonlinear constraints. In particular, we exploit the ability of gPC to express higher order moments of the uncertainty distribution - without any Gaussianity assumption - and we incorporate chance constraints that lead to expressions involving the state covariance. Furthermore, we demonstrate that by implementing our algorithm in a receding horizon fashion, we are able to compute control policies that effectively reduce the accumulation of uncertainty on the trajectory. The applicability of our method is verified through simulation results on a differential wheeled robot and a quadrotor that perform obstacle avoidance tasks.

Keywords: Stochastic optimal control, Decentralized control, Stochastic systems
Abstract: In this paper, we investigate a sequential dynamic team problem consisting of two agents with a nested information structure. We use a combination of the person-by-person and prescription approach to derive structural results for optimal control strategies for the team. We then use these structural results to present a dynamic programming (DP) decomposition to derive the optimal control strategies for a finite time horizon. We show that our DP utilizes the nested information structure to simplify the computation of the optimal control laws for the team at the final time step.

Keywords: Stochastic optimal control, Randomized algorithms, Energy systems
Abstract: Simultaneous perturbation stochastic approximation was previously shown to provide good results for optimizing resource consumption under dynamically changing workload. We employ this approach to develop dynamic voltage-frequency scaling (DVFS) governor for Android OS. We design experimental methodology and show our algorithm performance compared to common Android DVFS governors. A link to the source code is also provided.

Keywords: Stochastic optimal control, Sampled-data control, Stochastic systems
Abstract: The presence of terminal state constraints in terms of expectations is studied for steering problems of partially observed linear stochastic systems. Three scenarios for the observation process are considered, namely, (i) continuous-time exact observations of the state, (ii) discrete-time exact observations of the state, and (iii)~discrete-time exact observations of the state accompanied by continuous-time noisy observations of the state. Closed form expressions are presented for the optimal inputs enforcing the terminal state constraint under these information structures, which are expressed in terms of controllability Gramians and solutions of Riccati and Lyapunov equations. Numerical examples are provided to illustrate the results.

Keywords: Stochastic systems, Game theory, Stochastic optimal control
Abstract: We consider security risks in the form of advanced persistent threats (APTs) and their detection using dynamic information flow tracking (DIFT). We model the tracking and the detection as a stochastic game between the attacker and the defender. Compared to the state of the art, our approach applies to a wider set of scenarios with arbitrary (not only acyclic) information-flow structure. Moreover, multidimensional rewards allow us to formulate and answer questions related to trade-offs between resource efficiency of the tracking and efficacy of the detection. Finally, our algorithm provides results with probably approximately correct (PAC) guarantees, in contrast to previous (possibly arbitrarily imprecise) learning-based approaches.

Keywords: Machine learning, Stochastic optimal control, Optimization algorithms
Abstract: Markov Decision Processes are classically solved using Value Iteration and Policy Iteration algorithms. Recent interest in Reinforcement Learning has motivated the study of methods inspired by optimization, such as gradient ascent. Among these, a popular algorithm is the Natural Policy Gradient, which is a mirror descent variant for MDPs. This algorithm forms the basis of several popular RL algorithms such as Natural actor-critic, TRPO, PPO, etc, and so is being studied with growing interest. It has been shown that Natural Policy Gradient with constant step size converges with a sublinear rate of O(1/k) to the global optimal. In this paper, we present improved finite time convergence bounds, and show that this algorithm has geometric (also known as linear) asymptotic convergence rate. We further improve this convergence result by introducing a variant of Natural Policy Gradient with adaptive step sizes. Finally, we compare different variants of policy gradient methods experimentally.

Keywords: Game theory, Queueing systems, Behavioural systems
Abstract: We study an information design problem for a non-atomic service scheduling game. The service starts at a random time and there is a continuum of agent population who have a prior belief about the service start time but do not observe the actual realization of it. The agents want to make decisions of when to join the queue in order to avoid long waits in the queue or not to arrive earlier than the service has started. There is a planner who knows when the service starts and makes suggestions to the agents about when to join the queue through an obedient direct signaling strategy, in order to minimize the average social cost. We characterize the full information and the no information equilibria and we show in what conditions it is optimal for the planner to reveal the full information to the agents. Further, by imposing appropriate assumptions on the model, we formulate the information design problem as a generalized problem of moments (GPM) and use computational tools developed for such problems to solve the problem numerically.

Keywords: Game theory, Queueing systems
Abstract: We consider a queue with an unobservable backlog by the incoming users. There is an information designer that observes the queue backlog and makes recommendations to the users arriving at the queue whether to join or not to join the queue. The arriving users have payoff relevant private types. The users, upon arrival, send a message, that is supposed to be their type, to the information designer if they are willing to hear a recommendation. The information designer then creates a recommendation for that specific type of user. The users have to pay a tax in exchange for the information they receive. In this setting, the information designer has two types of commitments. The first commitment is the recommendation policy and the second commitment is the tax function. We combine mechanism design and information design to study a queuing system with heterogeneous users. In this setting, the information designer is a sender of the information in the information design aspect and a receiver in the mechanism design aspect of the model. We formulate an optimization problem that characterizes the solution of the joint design problem. We characterize the tax functions and provide structural results for the recommendation policy of the information designer.

Keywords: Game theory, Queueing systems
Abstract: We analyse a coalition formation game between strategic service providers of a congestible service. The key novelty of our formulation is that it is a constant sum game, i.e., the total payoff across all service providers (or coalitions of providers) is fixed, and dictated by the total size of the market. The game thus captures the tension between resource pooling (to benefit from the resulting statistical economies of scale) and competition between coalitions over market share. In a departure from the prior literature on resource pooling for congestible services, we show that the grand coalition is in general not stable, once we allow for competition over market share. Instead, the stable configurations are duopolies, where the dominant coalition exploits its economies of scale to corner a disproportionate market share. We analyse the stable duopolies that emerge from this interaction.

Keywords: Learning, Game theory, Stochastic systems
Abstract: We consider a dynamic Colonel Blotto game (CBG) in which one of the players is the learner and has limited troops (budget) to allocate over a finite time horizon. At each stage, the learner strategically determines the budget distribution among the battlefields based on past observations. The other player is the adversary, who chooses its budget allocation strategies randomly from some fixed unknown distribution. The learner's objective is to minimize its regret, which is the difference between the payoff of the best mixed strategy and the realized payoff by following a learning algorithm. The dynamic CBG is analyzed under the framework of combinatorial bandit and bandit with knapsacks. We first convert the dynamic CBG with budget constraint to a path planning problem on a graph. We then devise an efficient dynamic policy for the learner that uses a combinatorial bandit algorithm Edge on the path planning graph as a subroutine for another algorithm LagrangeBwK. It is shown that under the proposed policy, the learner's regret is bounded with high probability by a term sublinear in time horizon T and polynomial with respect to other parameters.

Keywords: Game theory, Stability of nonlinear systems, Agents-based systems
Abstract: The generalized stability results of a Nash equilibrium in two-agent noncooperative systems are investigated with loss-averse consideration and quadratic payoffs. In this system, each agent is driven by the pseudo-gradient dynamics with piecewise constant sensitivity parameters for the situations of losing payoffs and gaining payoffs. Based on the mode analysis, the sufficient and necessary conditions under which agents' state converge to the Nash equilibrium are derived. We make a bifurcation analysis for the loss-aversion-based noncooperative system to observe how the loss-aversion behaviors change the stability property of the Nash equilibrium. It is found that the loss-aversion behaviors may destabilize the Nash equilibrium. We present a sufficient condition of robust stability under which the loss-aversion behaviors never destabilize the Nash equilibrium for any sensitivity parameters. A numerical example is provided to illustrate the effectiveness of our results.

Keywords: Game theory, Optimization algorithms, Large-scale systems
Abstract: We consider a class of multi-agent optimization problems, where each agent has a local objective function that depends on its own decision variables and the aggregate of others, and is willing to cooperate with other agents to minimize the sum of the local objectives. After associating each agent with an auxiliary variable and the related local estimates, we conduct primal decomposition to the globally coupled problem and reformulate it so that it can be solved distributedly. Based on the Douglas-Rachford method, an algorithm is proposed which ensures the exact convergence to a solution of the original problem. The proposed method enjoys desirable scalability by only requiring each agent to keep local estimates whose number grows linearly with the number of its neighbors. We illustrate our proposed algorithm by numerical simulations on a commodity distribution problem over a transport network.

Keywords: Optimization algorithms, Optimization
Abstract: This paper proposes a novel modification of the Riemannian steepest descent (R-SD) method with the Armijo backtracking line search. Although the proposed approach is simple because it only modifies the step lengths satisfying the Armijo condition, the results of the numerical experiments show that the approach significantly improves the performance of the standard R-SD method. Furthermore, it can be guaranteed that the global convergence and locally linear convergence properties of the original standard R-SD method are maintained in the proposed method.

Keywords: Optimization, Optimization algorithms
Abstract: Binary optimization is a long-time problem ubiquitous in many engineering applications, e.g., automatic control, cyber-physical systems and machine learning. From a mathematical viewpoint, binary optimization is NP-hard, to solve which one can find some suboptimal strategies in the literature. Among the most popular approaches, semidefinite relaxation has attracted much attention in the last years. In contrast, this work proposes and analyzes a non-convex regularization approach, through which we obtain a relaxed problem whose global minimum corresponds to the true binary solution of the original problem. Moreover, because the problem is non-convex, we propose an adaptive regularization that promotes the descent towards the global minimum. We provide both theoretical results that characterize the proposed model and numerical experiments that prove its effectiveness with respect to state-of-the-art methods.

Keywords: Optimization algorithms, Distributed control, Network analysis and control
Abstract: We describe a parameterized family of first-order distributed optimization algorithms that enable a network of agents to collaboratively calculate a decision variable that minimizes the sum of cost functions at each agent. These algorithms are emph{self-healing} in that their convergence to the correct optimizer can be guaranteed even if they are initialized randomly, agents join or leave the network, or local cost functions change. We also present simulation evidence that our algorithms are self-healing in the case of dropped communication packets. Our algorithms are the first single-Laplacian methods for distributed convex optimization to exhibit all of these characteristics. We achieve self-healing by sacrificing internal stability, a fundamental trade-off for single-Laplacian methods.

Keywords: Neural networks, Optimization
Abstract: In this paper, a robust optimization framework is developed to train shallow neural networks based on reachability analysis of neural networks. To characterize noises of input data, the input training data is disturbed in the description of interval sets. Interval-based reachability analysis is then performed for the hidden layer. With the reachability analysis results, a robust optimization training method is developed in the framework of robust least-square problems. Then, the developed robust least-square problem is relaxed to a semi-definite programming problem. It has been shown that the developed robust learning method can provide better robustness against perturbations at the price of loss of training accuracy to some extent. At last, the proposed method is evaluated on a robot arm model learning example.

Keywords: Optimization algorithms, Queueing systems, Fluid flow systems
Abstract: We describe an efficient implementation of a recent simplex-type algorithm for the exact solution of separated continuous linear programs, and compare it with linear programming approximation of these problems obtained via discretization of the time horizon. The implementation overcomes many numerical pitfalls often neglected in theoretical analysis allowing better accuracy or acceleration up to several orders of magnitude both versus previous implementation of the simplex-type algorithms and versus a state-of-the-art LP solver using discretization. Numerical study includes medium, large, and very large examples of scheduling problems and problems of control of fluid processing networks. We discuss online and offline optimization settings for various applications and outline future research directions.

Keywords: Predictive control for nonlinear systems, Optimization, Numerical algorithms
Abstract: In this paper, we study the convergence properties of an iterative algorithm for fast nonlinear model predictive control (MPC) of quasi-linear parameter-varying systems without inequality constraints. Compared to previous works considering this algorithm, we contribute conditions under which the iterations are guaranteed to converge. Furthermore, we show that the algorithm converges to suboptimal solutions and propose an optimality-preserving variant with moderately increased computational complexity. Finally, we compare both variants in terms of quality of solution and computational performance with a state-of-the-art solver for nonlinear MPC in two simulation benchmarks.

Keywords: Stability of nonlinear systems, Networked control systems, Agents-based systems
Abstract: In this paper we study input-to-state stability with respect to measurement functions for discrete time networked systems. In such a networked system, the trajectory of each subsystem is affected by another in each time step, and this constraining system mat change from step to step. We derive small gain conditions of input-to-state stability with respect to measurement functions for this type of discrete time networked system relying on the construction of dissipative-form finite time Lyapunov function. These conditions loosen those presented in [1] and [2]. Finally, we demonstrate the applicability of our results by applying them to the stability analysis of a distributed graph algorithm which does not satisfy the conditions of [1] and [2].

Keywords: Stability of nonlinear systems, Nonlinear systems, Uncertain systems
Abstract: We present a method for computing reachable sets and forward invariant sets for systems with dynamics that include unknown components. Our main assumption is that, given any hyperrectangle of states, lower and upper bounds for the unknown components that hold with high probability are available. We then show that this assumption is well-suited when the unknown terms are modeled as state-dependent Gaussian processes. Under this assumption, we leverage the theory of mixed monotone systems and propose an efficient method for computing a hyperrectangular set that over-approximates the reachable set of the system with high probability. We then show a related approach that leads to sufficient conditions for identifying sets that are forward invariant for the dynamics with high probability. These theoretical results lead to practical algorithms for efficient computation of high probability reachable sets and invariant sets. A major advantage of our approach is that it leads to tractable computations for systems up to moderately high dimension that are subject to low dimensional uncertainty modeled as Gaussian Processes, a class of systems that appears often in practice. We demonstrate our results on an example of a six-dimensional model of a multirotor aerial vehicle.

Keywords: Stability of nonlinear systems, Stochastic systems, Nonlinear systems
Abstract: This paper considers the problem of stabilizing a discrete-time non-linear stochastic system over a finite capacity noiseless channel. In the literature, it has been established that under technical assumptions, the channel capacity must not be smaller than the logarithm of the determinant of the system linearization, averaged over noise and ergodic state measures. However, this result is not tight for many systems as stable directions may hide the volume growth arising from unstable directions. In this paper, utilizing a stochastic and geometric analysis, we introduce a method which allows us to consider stochastic growth among subsets of coordinates. Accordingly, under the asymptotic ergodicity stability criterion, we establish a lower bound on information transmission requirements for the existence of a coding and control policy which renders the closed-loop system stochastically stable. We establish that for systems with a stochastically stable component, it suffices to consider only the unstable dimensions, providing a refinement on the general channel capacity bound for a large class of systems.

Keywords: Nonlinear systems, Stability of nonlinear systems, Control applications
Abstract: The broadly studied long-existing problem of the varying length pendulum swing damping using the Coriolis force is approached here via the novel two-step approach. First, convenient virtual nonholonomic constraints on the swing angle, swing angular velocity and the string length ensuring the most efficient and realistic damping are designed. Then these constraints are enforced by the input being the force applied along the string length. Efficiency of enforcement and theoretical stability proof are facilitated using finite time stability feedback for the integrator chain. Besides careful theoretical analysis including proper mathematical proofs, our novel approach was tested both in simulations and using the laboratory model.

Keywords: Nonlinear systems, Stability of nonlinear systems, Lyapunov methods
Abstract: The homogeneity concept is extended to the area of descriptor systems. A homogeneity-based admissible control is designed for stabilization of linear descriptor systems in a finite time. The presented feedback homogenizes a linear system with a specified negative degree and stabilizes it in a finite time. The tuning procedure is formalized in LMI form. The theoretical results are supported by numerical simulations.

Keywords: Nonlinear systems, Stability of nonlinear systems, Networked control systems
Abstract: The flow of contracting systems contracts 1-dimensional polygons (i.e. lines) at an exponential rate. One reason for the usefulness of contracting systems is that many interconnections of contracting sub-systems yield an overall contracting system. A recent generalization of contracting systems is called k-contracting systems, where k in {1,dots,n}. The flow of such systems contracts k-dimensional polygons at an exponential rate, and in particular they reduce to contracting systems when k=1. Here, we analyze serial interconnections of 1-contracting and 2-contracting systems. We provide conditions guaranteeing that such interconnections have a well-ordered asymptotic behaviour, and demonstrate the theoretical results using several examples.

Keywords: Observers for nonlinear systems, Network analysis and control, Switched systems
Abstract: The paper considers observability of switched Boolean control networks (SBCNs), for two different observability definitions. First, utilizing the semi-tensor product (STP) method, a coupled system based on the initial SBCN is constructed. Then the observability conditions corresponding to diverse definitions are converted into the corresponding set controllability goals for the coupled system. By adopting the set controllability technique of SBCNs, easily verified conditions for observability of SBCNs are obtained. Finally, an example is given to show the effectiveness of the results.

Keywords: Switched systems, Estimation, Hybrid systems
Abstract: An observer, based on the well-known structural conditions of state and mode observability for switched linear systems, using a high-order sliding-mode (HOSM) differentiator, is proposed to recover both the active mode and the continuous state in Finite-time. The estimation of both the mode and the state is ensured, under some mild additional conditions, even for the case when the measurable output is affected by perturbations. Moreover, a continuous state observer for the case when the mode is unobservable, or the mode estimation is negligible, is presented considering some more relaxed structural conditions ensuring the continuous state estimation disregarding the mode.

Keywords: Lyapunov methods, Switched systems, Observers for nonlinear systems
Abstract: In this paper, we study finite-time stability and stabilization of switched systems in the presence of unstable modes. In contrast to asymptotic or exponential stability where the system trajectories reach the equilibrium point as time tends to infinity, the notion of finite-time stability requires the trajectories to reach the equilibrium within a finite amount of time. We present sufficient conditions in terms of multiple Lyapunov functions for the origin of a class of switched systems to be finite-time stable. More specifically, we show that even if the value of the Lyapunov function increases in between two switches, i.e., if there are unstable modes in the system, finite-time stability can still be guaranteed if the finite-time convergent mode is active for a long enough cumulative time duration. Then, we present a method for the synthesis of a finite-time stabilizing switching signal. As a case study, we design a finite-time stable, output-feedback controller for a linear switched system, in which only one of the modes is both controllable and observable. We present one numerical example to demonstrate the efficacy of the proposed method.

Keywords: Nonlinear systems, Switched systems, Delay systems
Abstract: A scattering transformation technique is extended to the case of switched dissipative nonlinear systems with quadratic supply rates. Specifically, under a relaxed form of the liveness condition, a construction of the scattering transformation is presented for a switched nonlinear dissipative system which allows for uniform assignment of a prescribed matrix gain with a common storage function. A scattering-based stabilization method for networks of switched dissipative systems with communication delays is subsequently developed.

Keywords: Filtering, Switched systems, Linear systems
Abstract: In this paper we make a further foray on Riccati equation. Besides the interest in its own right, the Riccati differential equation (RDE) studied here has an important bearing on an optimal linear filtering problem for Markov jump linear systems with partial observations of the Markov chain. A peculiar feature of this class of RDE is that controlla- bility holds, but not detectability. Via some diagonal dominant matrices properties in conjunction with Gershgorin’s circle theorem, it is shown that this particular class of non-detectable RDE has a unique positive semidefinite solution that converges to a solution of the associated algebraic Riccati equation (ARE). Although we first consider the case with time-invariant weight of the quadratic term, it is shown that under certain conditions, extension for the time-variant case is also possible . These results has an immediate impact on the deduction of the corresponding stationary filters, and may be relevant for the optimal linear control problem. Finally, we also carry out several numerical simulations, providing a visual endorsement to the theoretical results obtained in this work.

Keywords: Nonlinear output feedback, Switched systems
Abstract: We consider the problem of dynamic output feedback stabilization at an unobservable target point. The challenge lies in according the antagonistic nature of the objective and the properties of the system: the system tends to be less observable as it approaches the target. In the literature, switching techniques rapidly appeared as a suitable approach to deal with this issue. On a case of systems with linear conservative dynamics and nonlinear output, this approach is used in conjunction with an embedding into bilinear systems that admit observers with dissipative error. Combining these two elements, global stabilization by means of a dynamic periodic time-varying output feedback is proved, and numerical simulations are provided.

Keywords: Robust control, Uncertain systems, Stability of linear systems
Abstract: This paper presents necessary and sufficient conditions under which a linear system of relative degree either one or two is state feedback equivalent to a negative imaginary (NI) system. More precisely, we show for a class of linear time-invariant strictly proper systems, that such a system can be rendered minimal and NI using full state feedback if and only if it is controllable and weakly minimum phase. A strongly strict negative imaginary state feedback equivalence result is also provided. The NI state feedback equivalence result is then applied in a robust stabilization problem for an uncertain system with a strictly negative imaginary uncertainty.

Keywords: Robust control, Uncertain systems, LMIs
Abstract: This paper addresses the problem of model match control for uncertain continuous-time linear systems by means of fixed order dynamic output-feedback. The H-infinity norm is used as performance metric for the approximation error and the synthesis conditions are formulated in terms of LMIs, which are solved iteratively. The novelty of the approach with respect to previous results is that the matrices of the controller, as well as the matrices of the reference model, appear affinely in the conditions, allowing the immediate treatment of structured controllers and providing extra degrees of freedom for the reference model, in the sense that some of the entries of the matrices do not need to be fixed a priori and, therefore, can be determined by the synthesis conditions. This latter feature allows an extension of the method to address the problem of order reduction of uncertain systems. The results are illustrated by an example based on a model borrowed from the literature.

Keywords: Robust control, Uncertain systems, Time-varying systems
Abstract: This letter quantifies the effect of random model uncertainty on finite horizon linear time-varying (LTV) systems. Mean and standard deviation field are approximated with high accuracy and efficiency by a Hilbert space technique called polynomial chaos expansion (PCE). The deterministic expansion coefficients of the generalized Fourier series are determined via orthogonal projection, also known as Galerkin projection. We propose the projection of uncertain systems in linear fractional representation (LFR), which can have computational benefits. The technique is benchmarked on a two-link robotic manipulator.

Keywords: Stability of nonlinear systems, Robust control, LMIs
Abstract: This paper investigates the problem of stability analysis and output-feedback stabilization of discrete-time Lur'e systems where the nonlinearity is odd and slope bounded. Using the linear matrix inequality (LMI) conditions from the literature to handle the l-1 norm and positive realness constraints, an iterative algorithm based on LMIs is constructed to assess stability through the existence of a Zames-Falb multiplier of any given order based on independent positive definite matrices for the l-1 norm and positive realness. More important, the method can also deal with output-feedback stabilization. Numerical examples illustrate the performance of the proposed approach when compared with other methods.

Keywords: Uncertain systems, Robust control, Stochastic optimal control
Abstract: In this paper, we study the robust linear quadratic regulator (LQR) problem for a class of discrete-time dynamical systems composed of several uncertain players with unknown or ambiguous distribution information. A distinctive feature of the assumed model is that each player is prescribed by a nominal probability distribution and categorized according to an uncertainty level of confidence. Our approach is based on minimax optimization. By following a dynamic programming approach a closed-form expression of the robust control policy is derived. The effect of ambiguity on the performance of the LQR is studied via a sequential hierarchical game with one leader and several followers. The equilibrium solution is obtained through a maximizing, time-varying probability distribution characterizing each player’s optimal policy. The behavior of the proposed method is demonstrated through an application to a drop-shipping retail fulfillment model.

Keywords: Control applications, Lyapunov methods, Robust control
Abstract: A robust nonlinear control method is developed for fluid flow velocity tracking, which formally addresses the inherent challenges in practical implementation of closed-loop active flow control systems. A key challenge being addressed here is flow control design to compensate for model parameter variations that can arise from actuator perturbations. The control design is based on a detailed reduced-order model of the actuated flow dynamics, which is rigorously derived to incorporate the inherent time-varying uncertainty in the both the model parameters and the actuator dynamics. To the best of the authors' knowledge, this is the first robust nonlinear closed-loop active flow control result to prove exponential tracking control of a reduced-order actuated flow dynamic model, which formally incorporates input-multiplicative time-varying parametric uncertainty and nonlinear coupling between the state and control signal. A rigorous Lyapunov-based stability analysis is utilized to prove semi-global exponential tracking of a desired flow field velocity profile over a given spatial domain. A detailed comparative numerical study is provided, which demonstrates the performance improvement that is achieved using the proposed robust nonlinear flow control method to compensate for model uncertainty and uncertain actuator dynamics.

Keywords: Observers for Linear systems, Estimation, Uncertain systems
Abstract: This letter proposes an unknown input zonotopic Kalman filter-based interval observer for discrete-time linear time-invariant systems. In such contexts, a change of coordinates decoupling the state and the unknown inputs is often used. Here, the dynamics are rewritten into a discrete-time linear time-invariant descriptor system by augmenting the state vector with the unknown inputs. A zonotopic outer approximation of the feasible state set is then obtained with a prediction-correction strategy using the information from the system dynamics, known inputs and outputs. Bounds for both the state and unknown inputs are obtained from this zonotopic set. The efficiency of the proposed interval observer is assessed with numerical simulations.

Keywords: Observers for Linear systems, Hybrid systems, Estimation
Abstract: This paper presents a novel approach to design interval estimators for uncertain linear impulsive systems. We consider situations where model disturbances and measurement noises are unknown but lie in given intervals. We propose a new architecture providing more degrees of freedom than standard interval observer structures for linear impulsive systems. We test the efficiency of the proposed methodology through numerical simulations.

Keywords: Observers for Linear systems, Estimation
Abstract: This work proposes a framework to design observers for systems that present low observability. It is shown that, in these scenarios, the estimation problem becomes ill-posed, which drastically limits the performance of standard observers, specially in the presence of noise. Consequently, this paper presents a method to design an observer that optimizes some potential function to be defined by the designer. This allows to implicitly regularize the estimation and recover a well-posed problem. The proposed technique is validated in a set of weakly-observable systems and the performance is compared with a common Kalman filter-like observer.

Keywords: Observers for Linear systems, Linear parameter-varying systems, Uncertain systems
Abstract: This paper proposes a new interval observer for joint estimation of the state and unknown inputs of a discrete-time linear parameter-varying (LPV) system with an unmeasurable parameter vector. This system is assumed to be subject to unknown inputs and unknown but bounded disturbances and measurement noise, while the parameter-varying matrices are elementwise bounded. Considering the unknown inputs as auxiliary states, the dynamics are rewritten as discrete-time LPV descriptor dynamics. A new structure of interval observer is then used, providing more degrees of freedom than the classical change of coordinates-based structure. The observer gains are computed by solving linear matrix inequalities derived from cooperativity condition and L_{infty} norm. Numerical simulations are run to show the efficiency of the proposed observer.

Keywords: Observers for Linear systems, Linear systems, Communication networks
Abstract: In this paper, the distributed state estimation problem of linear systems under switching communication networks is investigated. A distributed Luenberger observer is developed that can achieve distributed unbiased asymptotic state estimation if the communication networks are uniformly connected and the observed system is jointly observable. Compared with the existing results, the salient feature of this work is that the developed approach is more resilient to unreliable communication.

Keywords: Observers for Linear systems, Uncertain systems, Fault diagnosis
Abstract: One of the necessary conditions for the existence of unknown input observers (UIOs) is a matrix rank condition. For the plants for which this matrix rank condition is satisfied, a number of UIO architectures were reported in the literature. In this paper, the proposed estimator architectures are for the plants for which the matrix rank condition for the existence of UIOs is not satisfied. To construct the proposed estimators, (δ+1) observations are collected that are then used to form an augmented system that satisfies the matrix rank condition, where δ is a design parameter. The design of the unknown input and output disturbance estimators are given in terms of linear matrix inequalities (LMIs). The unknown input and output disturbance estimation errors are guaranteed to be l_∞-stable with computable performance level.

Keywords: Distributed parameter systems, Biological systems, Filtering
Abstract: We derive gain-tuning rules for the positive and negative spatial-feedback loops of a spatially-distributed filter to change the resolution of its spatial band-pass characteristic accordingly to a wavelet zoom, while preserving temporal stability. The filter design is inspired by the canonical spatial feedback structure of the primary visual cortex and is motivated by understanding attentional control of visual resolution. Besides biology, our control-theoretical design strategy is relevant for the development of neuromorphic multiresolution distributed sensors through the feedback interconnection of elementary spatial transfer functions and gain tuning.

Keywords: Distributed parameter systems, Lyapunov methods, Stability of nonlinear systems
Abstract: In this paper, we establish for any pin (1,2) and rin [p,+infty] the input-to-state stability (ISS) and integral input-to-state stability (iISS) in the spatial L^p-norm and the spatial W^{1,p}_0-norm, with respect to distributed in-domain disturbances in L^{r}_{text{loc}}([0,+infty);L^p(0,1)) and L^{r}_{text{loc}}([0,+infty);W^{1,p}_0(0,1)), respectively, for a class of 1-D nonlinear parabolic partial differential equations (PDEs) with variable coefficients under homogeneous Dirichlet conditions by using approximations of Lyapunov functionals. The obtained results fill a gap in ISS analysis for parabolic PDEs.

Keywords: Distributed parameter systems, Observers for nonlinear systems, Identification
Abstract: The problem of estimation of the unknown potential in a 1-dimensional wave equation via state observers is considered in this work. The potential is supposed to depend on the space variable only and be polynomial. The main observation information is the value of the solution of the wave equation in a subinterval of the domain, including also some of its higher-order spatial derivatives. The method we propose to estimate the potential includes turning it into a new state as in finite-dimensional parameter estimation approaches. However, in this infinite dimensions setting, this requires an indirect approach that is introduced, including an infinite-dimensional state transformation. Sufficient conditions allow the design of an internal semilinear observer for the resulting cascade system, corresponding to the observed subinterval, which estimates the potential everywhere in an exponentially fast manner.

Keywords: Distributed parameter systems, Stability of nonlinear systems, Optimal control
Abstract: Local exponential stabilization of nonlinear distributed parameter systems (DPS) by linear state feedbacks is adressed. The results rely on an adapted concept of Fréchet differentiability which is in general easier to deal with. As main contribution, it is first shown how to link the Fréchet differentiability of the nonlinear semigroup generated by the operator dynamics with the Fréchet differentiability of the closed-loop semigroup obtained by injecting a linear state feedback into the dynamics. As a second result, an appropriately stabilizing state feedback for the linearized system around any equilibrium is proved to be locally stabilizing for the nonlinear system, under some boundedness assumption on the control operator. A class of controlled systems satisfying the required assumptions is then identified. The theoretical results are illustrated for the state regulation of a diffusion equation perturbed by a nonlinear term.

Keywords: Control applications, Distributed parameter systems
Abstract: The hyperbolic distributed-parameter system under consideration comprises a second-order partial differential equation that is bidirectionally coupled with a first-order ordinary differential equation. The system models the transmission at a pneumatic test bench with boundary control and collocated measurement, yet stands in place for several other lossy or loss-free transmission problems. In the paper, an observer-based output feedback tracking controller is designed using the concept of flatness. It is a design based on normal forms, as both the state feedback tracking controller and the state observer are derived using canonical coordinates, which in turn rely on special parametrizations of the system's solutions. Additionally, a new transformation between the coordinates of the so-called hyperbolic controller and observer canonical form is presented that allows for the implementation of the observer-based controller. Overall, the closed-loop system is exponentially stable. Simulation results illustrate the tracking behavior.

Keywords: Adaptive control, Distributed parameter systems, Neural networks
Abstract: This paper is focused on the reference-tracking control problem of distributed parameter systems modeled by a class of parabolic partial differential equations (PDEs) with uncertain nonlinear dynamics. An adaptive tracking control scheme is developed by utilizing radial basis function neural networks (RBF NNs) to deal with nonlinear system uncertainties. Specifically, the Galerkin method is first employed to derive a reduced-order ordinary differential equation (ODE) model to approximate the original PDE system. Based on this, an adaptive tracking control scheme is developed based on the singular perturbation theory and Lyapunov stability theory. With the control scheme implemented on the original PDE system, the system output can be guaranteed to track a prescribed reference trajectory with desired system stability and tracking accuracy. Simulation study on a representative transport-reaction process is conducted to demonstrate the effectiveness of the proposed approach.

Keywords: Transportation networks, Optimization, Numerical algorithms
Abstract: This paper presents a Wasserstein attraction approach for solving dynamic mass transport problems over networks. In the transport problem over networks, we start with a distribution over the set of nodes that needs to be "transported" to a target distribution accounting for the network topology. We exploit the specific structure of the problem, characterized by the computation of implicit gradient steps, and formulate an approach based on discretized flows. As a result, our proposed algorithm relies on the iterative computation of constrained Wasserstein barycenters. We show how the proposed method finds approximate solutions to the network transport problem, taking into account the topology of the network, the capacity of the communication channels, and the capacity of the individual nodes.

Keywords: Power systems, Optimization algorithms, Optimal control
Abstract: To maintain voltage security with limited system model information, we develop a model-free optimal voltage control algorithm based on projected primal-dual gradient dynamics and continuous-time zeroth-order method (extremum seeking control). This proposed algorithm i) operates purely based on voltage measurements and does not require any other model information (model-free), ii) drives the voltage magnitudes back to the acceptable range while satisfying the power capacity constraints all the time (safety), and iii) minimizes the total operating cost (optimality). Moreover, this algorithm is implemented in a decentralized fashion where the privacy of controllable devices is preserved and plug-and-play operation is enabled. We prove that the proposed algorithm is semi-globally practically asymptotically stable and is structurally robust to small measurement noises. Lastly, the performance of this algorithm is further demonstrated via numerical simulations.

Keywords: Networked control systems, Adaptive control, Agents-based systems
Abstract: We propose an online adaptive synchronization method for leader-follower networks of heterogeneous agents. Distributed Model Reference Adaptive Control (DMRAC-RL) achieves synchronization that enables the improved performance of Reinforcement Learning (RL)-trained policies on a reference model. The leader observes the performance of the reference model, and the followers observe the states and actions of the agents they are connected to, but not the reference model. Notably, both the leader and followers models might differ from the reference model the RL-control policy was trained. DMRAC-RL uses an internal loop that adjusts the learned policy for the agents in the form of an augmented input to solve the distributed control problem. We show that the DMRAC-RL controller synchronizes the network of heterogeneous agents asymptotically. Numerical examples of the synchronization of a network of inverted pendulums support our theoretical findings.