Keywords: Kalman filtering, Estimation
Abstract: State estimation techniques are applied to two different plasma dynamical systems: spacecraft electric propulsion and plasma processing devices. Physical constraints are accounted for in the extended Kalman filter by adapting the process and measurement noise covariances to ensure that the model remains consistent with the physical processes. This modified filter is first used to investigate discharge current oscillations in a Hall effect thruster. Then, the filter is applied to an argon global model of a plasma discharge to estimate the absorbed power input from experimental data. Both applications demonstrate that plasma density fluctuations can be used to estimate the dynamics of other plasma properties in the system, such as electron temperature and reaction rate coefficients.

Keywords: Aerospace, Reduced order modeling, Computational methods
Abstract: This tutorial paper presents the surrogate modeling based on Gaussian process regression (GPR) and its application in the hypersonic aero-thermo-servo-elastic analysis, a key ingredient in the design of hypersonic vehicles as well as its guidance, navigation and control. First, the basic formulations of GPR and practices of model training are presented. Next, the existing applications of GPR in hypersonic problems are reviewed. Subsequently, a pedagogical example and an applied example of GPR are presented. Finally, the pros and cons of the GPR modeling approach for general and hypersonic-specific applications are summarized.

Keywords: Nonlinear systems identification, Identification for control, Machine learning
Abstract: Many dynamical systems of interest are nonlinear, with examples in turbulence, epidemiology, neuroscience, and finance, making them difficult to control using linear approaches. Model predictive control (MPC) is a powerful model-based optimization technique that enables the control of such nonlinear systems with constraints. However, modern systems often lack computationally tractable models, motivating the use of system identification techniques to learn accurate and efficient models for real-time control. In this tutorial article, we review emerging data-driven methods for model discovery and how they are used for nonlinear MPC. In particular, we focus on the sparse identification of nonlinear dynamics (SINDy) algorithm and show how it may be used with MPC on an infectious disease control example. We compare the performance against MPC based on a linear dynamic mode decomposition (DMD) model. Code is provided to run the tutorial examples and may be modified to extend this data-driven control framework to arbitrary nonlinear systems.

Keywords: Identification, Linear parameter-varying systems, Time-varying systems
Abstract: This paper describes the main features and the most recent developments of system identification in the sense of data-driven modeling of dynamical systems. A brief summary of discrete time-invariant system identification techniques is provided, from the modern work of Gilbert, Kalman and Ho and the introduction of state-space realization, to the most recent developments of the identification of discrete time-varying and nonlinear systems. Important concepts of state-space realization, controllability and observability for linear systems are introduced along with more advanced methods to identify nonlinear dynamics. Numerical examples of varying complexity are considered to demonstrate the capability of the different approaches presented in this paper.

Keywords: Identification, Identification for control, Machine learning
Abstract: Parameter estimation methods to provide data-based models to control complex dynamical systems are reviewed. Starting from least square minimization of the equation error, the tutorial provides an overview of how different perspectives of parameter estimation lead to various algorithms that are used in diverse contexts. Both statistical and deterministic approaches are discussed, and the utility of model inferences are explained. The discussions provide a context and review relevant background with respect to three application papers involving recent advances in Gaussian Process Regression (GPR), state estimation approaches and data-driven modeling.

Keywords: Machine learning, Communication networks, Network analysis and control
Abstract: We propose an online deterministic annealing algorithm for progressive graph partitioning based on the spectral information of the underlying graph Laplacian matrix. Online deterministic annealing is a prototype-based unsupervised learning algorithm that progressively adjusts the number of prototypes used with respect to a performance-complexity trade-off. Due to the online nature of the proposed learning algorithm, the structure of the graph need not be known a priori. In this regard, we construct a distributed approximation algorithm to estimate the spectral information of the graph Laplacian, bypassing the exact computation of its eigenvectors. By propagating an impulse through the graph via a diffusion equation, we show that each node can construct a local learning representation which can be used for spectral clustering. As a result, the proposed approach is suitable for large graphs, requires minimal hyper-parameter tuning, and provides online control over the complexity-accuracy trade-off. We illustrate the properties and evaluate the performance of the proposed methodology in graph partition and image segmentation applications.

Keywords: Machine learning, Game theory
Abstract: Multi-agent imitation learning (MA-IL) aims to inversely learn policies for all agents using demonstrations collected from an expert group. However, this problem has only been studied in the setting of Markov games (MGs) allowing participants for concurrent actions, and do not work for general MGs, with agents inconcurrently making decisions in different turns. In this work, we propose iMA-IL, a novel multi-agent imitation learning framework for general (inconcurrent) Markov games. The learned policies are proven to guarantee subgame perfect equilibrium (SPE), a stronger equilibrium than Nash equilibrium (NE). The experiment results demon- strate that compared to state-of-the-art baselines, our iMA-IL model can better infer the policy of each expert agent using their demonstration data collected from inconcurrent decision- making scenarios.

Keywords: Machine learning, Markov processes, Statistical learning
Abstract: Adversarial attacks on Markov decision processes (MDPs) and reinforcement learning (RL) have been studied in the literature in the context of robust learning and adversarial game theory. In this paper, we introduce a new notion of adversarial attacks on MDP and RL computation that is motivated by the emergence of edge computing. The large-scale computation of MDP and RL models in the form of value/policy iteration and Q-learning is being offloaded from agents to distributed servers, giving rise to edge reinforcement learning. By the inherently distributed nature of edge RL, the MDP/RL computation can be prone to adversarial attacks in different forms. We analyze a probabilistic model of adversarial attacks on the computation of the modified policy iteration method in which the principal contraction property of the Bellman operator is undermined with a certain probability in iterations of the policy evaluation step of the aforementioned method. This can result in luring the agent to search among sub-optimal policies without improving the true values of policies. We prove that under certain conditions, the attacked modified policy iteration method can still converge to the vicinity of the optimal policy with high probability if the number of policy evaluation iterations is larger than a threshold that is logarithmic in the inverse of a desired precision. We also provide an upper bound on the number of iterations needed for the attacked modified policy iteration method to terminate, which holds with an associated confidence level.

Keywords: Machine learning, Optimal control, Neural networks
Abstract: In this paper, a human-behavior learning approach for optimal tracking control of robot manipulators is proposed. The approach is a generalization of the reinforcement learning control problem which merges the capabilities of different intelligent and control techniques in order to solve the tracking task. Three cognitive models are used: robot and reference dynamics and neural networks. The convergence of the algorithm is achieved under a persistent exciting and experience replay fulfillment. The algorithm learns online the optimal decision making controller according to the proposed cognitive models. Simulations were carry out to verify the approach using a 2-DOF planar robot.

Keywords: Machine learning, Optimization, Communication networks
Abstract: Federated Learning (FL) has recently been proposed as a distributed optimization framework under resource constraints and privacy concerns. We study the problem of client scheduling for FL, where the goal is to optimize the performance of FL under certain resource constraints on the FL setup. We show that this problem is a special instance of the general problem of maximizing a submodular function subject to a submodular upper bound constraint. We propose a greedy algorithm to solve this general problem, and provide theoretical approximation guarantees to characterize its performance. The greedy algorithm proposed for the general problem is then applied to solve the FL client scheduling problem with the approximation guarantee. We evaluate the performance of the algorithm using experiments.

Keywords: Machine learning, Agents-based systems, Game theory
Abstract: We consider a pursuit-evasion game played between two agents, `Blue' (the pursuer) and `Red' (the evader), over T time steps. Red aims to attack Blue's territory. Blue's objective is to intercept Red at time T and thereby limit the success of Red's attack. Blue must plan its pursuit trajectory by choosing parameters that determine its course of movement (speed and angle in our setup) such that it intercepts Red at time T. We show that Blue's path-planning problem in pursuing Red, can be posed as a sequential decision making problem under uncertainty. Blue's unawareness of Red's action policy renders the analytic dynamic programming approach intractable for finding the optimal action policy for Blue. In this work, we are interested in exploring data-driven approaches to the policy optimization problem that Blue faces. We apply generative machine learning (ML) approaches to learn optimal action policies for Blue. This highlights the ability of generative ML models to learn the relevant implicit representations for the dynamics of simulated pursuit-evasion games. We demonstrate the effectiveness of our modeling approach via extensive statistical assessments. This work represents a preliminary step towards the further development of generative modeling approaches for policy optimization problems that arise in the context of multi-agent learning and planning.

Keywords: Traffic control, Optimal control, Learning
Abstract: This paper studies the issue of data-driven optimal control design for connected and autonomous vehicles (CAVs) in a mixed-traffic environment. More specifically, we investigate the controllability of a string of vehicles composed of multiple CAVs and heterogeneous human-driven vehicles in a ring road. We use the classical Popov-Belevitch-Hautus (PBH) test to single out the uncontrollable mode, and identify the controllable subspace, based on which we obtain an explicit transformation matrix for Kalman controllable decomposition. Combining with the decomposition result, we formulate a linear quadratic regulator problem with constrained initial states and employ the adaptive dynamic programming method to solve it without relying on the exact knowledge of system parameters. The convergence of the data-driven algorithm has been proved rigorously. The simulation result shows that our theoretical analysis is effective and the proposed data-driven controller yields desirable performance for regulating the mixed-traffic flow.

Keywords: Uncertain systems, Learning, Robust control
Abstract: We develop a control algorithm for the safety of a control-affine system with unknown nonlinear dynamics in the sense of confinement in a given safe set. The algorithm leverages robust nonlinear feedback control laws integrated with on- the-fly, data-driven approximations to output a control signal that guarantees the boundedness of the closed-loop system in the given set. More specifically, it first computes estimates of the dynamics based on differential inclusions constructed from data obtained online from a single finite-horizon trajectory. It then computes a novel feedback safety control law that renders the system forward invariant with respect to the safe set, given an accurate enough estimate, using reciprocal barriers. An extension of the algorithm is capable of coping with the controllability loss incurred by the control matrix along the safe set. The algorithm removes a series of common and limiting assumptions considered in the related literature since it does not require global boundedness, growth conditions, or a priori approximations of the unknown dynamics’ terms.

Keywords: Learning, Uncertain systems, Identification
Abstract: In this paper, we develop an online, data-based framework for optimal actuator placement of uncertain systems. In particular, a set of actuators is chosen to maximize a metric of controllability of the system, but without full knowledge of the system's dynamics, and by only measuring the system's trajectories in an online manner. The metric of controllability is associated with the trace of a discounted Gramian, which satisfies a static Lyapunov equation even if the system's plant matrix is not Hurwitz. Subsequently, an estimator is designed to learn the Gramian's trace exponentially fast in real-time. Finally, we show that the trace estimator can be used to place actuators online, and that the optimal set of actuators is found and scheduled permanently in finite time. The efficacy of the proposed framework is shown through simulations.

Keywords: Optimal control, Variational methods, Constrained control
Abstract: We generalize a Maximum Principle, previously obtained, to cover optimal control problems involving sweeping processes and end point constraints. The work is based on an ingenious smooth approximating family of problems.

Keywords: Learning, Optimal control, Stochastic optimal control
Abstract: This paper presents a model-free Q-learning algorithm for solving the risk-averse optimal control (RAOC) problem. The entropic risk measure is used in the RAOC to account for the variance of the objective function. A one-shot Q-based convex optimization problem is then formed for which the decision variables are the Q-function parameters and the constraints are formed by sampling from an exponential utility-based entropic Bellman inequality. Samples are constructed using only a batch of data collected from a variety of control policies in a fully off-policy manner, which turns a dataset into a Q-learning based risk-averse optimal policy engine. Convergence of the exact optimization problem, which is infinite- dimensional in decision variables and constraints, to the optimal risk-averse Q-function is shown. For the standard convex optimization problem for which function approximation for Q-value estimations as well as constraint sampling are leveraged, the performance of the approximated solutions is verified through a weighted-norm bound and the Lyapunov bound. A simulation example is provided to verify the effectiveness of the presented approach.

Keywords: Adaptive control, Optimization algorithms, Adaptive systems
Abstract: Motivated by recent (semi-global practical) fixed-time convergence results in time-invariant model-free optimization problems, in this paper we introduce new tracking bounds and guidelines for the design of extremum seeking controllers in model-free optimization problems with dynamic cost functions. Using semi-global practical input-to-state stability characterizations, we show that the proposed non-smooth ES dynamics are able to significantly reduce the tracking error compared to the traditional smooth algorithms studied in the literature. Moreover, under a suitable tuning of the gains of the algorithm, the nominal average dynamics of the controller are able to achieve global fixed-time tracking for a general class of dynamic cost functions. For tuning parameters that do not completely eliminate the tracking error in the nominal average dynamics, but which preserve the continuity of the vector field, we show that “almost complete” error rejection is achieved whenever the gain of the algorithm exceeds a particular threshold. Numerical results are presented to illustrate the performance of the algorithms.

Keywords: Identification, Estimation, Linear systems
Abstract: In continuous-time system identification, the intersample behavior of the input signal is known to play a crucial role in the performance of estimation methods. One common input behavior assumption is that the spectrum of the input is band-limited. The sinc interpolation property of these input signals yields equivalent discrete-time representations that are non-causal. This observation, often overlooked in the literature, is exploited in this work to study non-parametric frequency response estimators of linear continuous-time systems. We study the properties of non-causal least-square estimators for continuous-time system identification, and propose a kernel-based non-causal regularized least-squares approach for estimating the band-limited equivalent impulse response. The proposed methods are tested via extensive numerical simulations.

Keywords: Identification, Estimation, Stochastic systems
Abstract: Point process data is increasingly occurring in a wide range of applications including social media. But basic problems such as classifying point processes have not been addressed. Here we study the misclassification error of a point process Bayes rule/likelihood ratio classification rule in a binary classification problem. We first develop the Bhattacharya bound for the error rate for the time-varying Poisson. We then derive a tight computable bound on the Bhattacharya bound for the renewal process. We then develop fast methods for computing these bounds. We study the accuracy of the bounds with some comparative simulations.

Keywords: Identification, Estimation, Stochastic systems
Abstract: Persistent homology is emerging as a powerful approach to comparing networks whose nodes carry signals; but requires a measure of distance between nodes. Almost all existing work applies to scalar networks i.e. where each node carries a scalar signal; and further, these signals are assumed to be white noises though they may be instantaneously cross correlated. We have previously developed frequency domain based distance measures to deal with scalar networks whose nodal signals are cross-autocorrelated. But in most applications networks are vector networks i.e. each node carries a vector of signals. Here we extend our scalar work to provide a frequency domain distance measure for vector networks. The new distance measure is illustrated with comparative simulations They show that persistent homology based on static or white noise vector distance measures fails catastrophically; but when based on dynamic vector distance measures, performs very well.

Keywords: Identification, Filtering, Markov processes
Abstract: This paper tackles the problem of identifying the parameters of a class of stochastic switched systems, where the active subsystem is determined by a Markov chain. This class includes autoregressive models with exogenous inputs (ARX) for which the parameters switch according to a Markov chain and general Markov Jump Linear Systems (MJLSs) with full-state information. The transition probabilities of the Markov chain are assumed to be known, but the active subsystem is unknown. A recursive identification method for the joint maximum a posteriori probability estimate of these parameters and of the unknown mode is proposed relying on relaxed dynamic programming. The method is guaranteed to provide an estimate whose joint posteriori probability is within a constant factor of that of the optimal estimate while reducing the computational complexity. The method is illustrated through a numerical example.

Keywords: Identification, Linear systems, Behavioural systems
Abstract: The paper deals with the problem of designing informative input trajectories for data-driven simulation. First, the excitation requirements in the case of noise-free data are discussed and new weaker conditions, which assume the simulated input to be known in advance, are provided. Then, the case of noisy data trajectories is considered and an input design problem based on a recently proposed maximum likelihood estimator is formulated. A Bayesian interpretation is provided, and the implications of using Hankel and Page matrix representations are demonstrated. Numerical examples show the impact of the designed input on the predictive accuracy.

Keywords: Identification, Linear systems
Abstract: This paper presents a new experiment design method for data-driven modeling and control. The idea is to select inputs online (using past input/output data), leading to desirable rank properties of data Hankel matrices. In comparison to the classical persistency of excitation condition, this online approach requires less data samples and is even shown to be completely sample efficient.

Keywords: Game theory, Information theory and control, Stochastic systems
Abstract: We study continuity properties of value functions in information structures for stochastic team, zero-sum, and general game problems. We establish continuity properties of the value function under total variation, setwise, and weak convergence of information structures. Our analysis reveals that the value function for a bounded game is continuous under total variation convergence of information structures in both zero-sum games and team problems. However, continuity fails under setwise or weak convergence of information structures. The value function exhibits upper semicontinuity properties under weak and setwise convergence of information structures for team problems, and upper or lower semicontinuity properties hold for zero-sum games when such convergence is through a Blackwell-garbled sequence of information structures. Finally, a counterexample reveals that value functions for players may not be continuous even under total variation convergence of information structures in general non-zero-sum games.

Keywords: Stochastic optimal control, Mean field games, Smart grid
Abstract: This paper presents advances in Kullback-Leibler-Quadratic (KLQ) optimal control: a stochastic control framework for Markovian models. The motivation is distributed control of large networked systems. The objective function is composed of a control cost in the form of Kullback-Leibler divergence plus a quadratic cost on the sequence of marginal distributions. With this choice of objective function, the optimal probability distribution of a population of agents over a finite time horizon is shown to be an exponential tilting of the nominal probability distribution. The same is true for the controlled transition matrices that induce the optimal probability distribution.

However, one limitation of the previous work is that randomness can only be introduced via the control policy; all uncontrolled processes must be modeled as deterministic to render them immutable under an exponential tilting. In this work, only the controlled dynamics are subject to tilting, allowing for more general probabilistic models.

Numerical experiments are conducted in the context of power networks. The distributed control techniques described in this paper can transform a large collection of flexible loads into a `virtual battery' capable of delivering the same grid services as traditional batteries. Additionally, quality of service to the load owner is guaranteed, privacy is preserved, and computation and communication requirements are reduced, relative to alternative centralized control techniques.

Keywords: Mean field games, Decentralized control, Networked control systems
Abstract: We propose to study what shall be termed critical nodes for a family of Linear Quadratic Gaussian Graphon Mean Field Games (LQG-GMFG). The critical nodes are defined to be those nodes at which the local mean field is stationary with respect to its index. We present examples of such nodes for the graphons that are the limits of (i) uniform attachment graph and (ii) the Erdos-Renyi graphs. We also investigate the properties of these critical nodes on the value functions and the equilibrium startegies for the LQG-GMFG.

Keywords: Mean field games, Stochastic optimal control, Game theory
Abstract: We consider linear quadratic (LQ) Stackelberg games with a major player (leader) and N minor players (followers) and derive two master equations in a mean field limit model. We show the resulting decentralized strategies are time consistent by adapting a procedure introduced by Ekeland and Lazrak (2006).

Keywords: Game theory, Decentralized control, Stochastic optimal control
Abstract: We investigate finite-horizon stochastic dynamic games among teams. Each team has its own dynamic system, whose evolution is affected by the actions of all players in all teams. Within each team, members share their local states with each other with a delay of d>0. Actions are observed by all agents along with noisy observations of the systems. Such games feature the difficulties of the increasing domain of strategies and interdependence of actions and information over time. In these games, we identify a subclass of Nash Equilibria where the agents use Sufficient Private Information Based (SPIB) strategies, i.e. agents make decisions based on compressed versions of their private information along with the common information. We establish the existence of such equilibria; the proof of existence is not based on standard techniques since SPIB strategies do not feature perfect recall. Finally, we investigate a special case of our model where each agent has their own dynamic system. We show that agents can compress their private information further in this case. Our results provide a foundational step in addressing the difficulties of dynamic games among teams.

Keywords: Stochastic optimal control, Mean field games, Distributed parameter systems
Abstract: We consider a mean-field control problem with linear dynamics and quadratic control. We prove the existence of an optimal control by applying the vanishing viscosity method: we add a (regularizing) heat diffusion with a small viscosity coefficient and let such coefficient go to zero. The main result is that, in this case, the limit optimal control is exactly the optimal control of the original problem.

Keywords: Game theory, Agents-based systems, Network analysis and control
Abstract: Most research in distributed Nash equilibrium seeking assumes that agents communicate truthfully. However, in general noncooperative games agents may have the incentive to send misinformation to neighbouring agents with the goal of minimizing their own costs. Furthermore, such settings can also be susceptible to communication failures and attacks from agents outside the game. In this paper, we design a NE seeking algorithm that is resilient against malicious agents and communication tampering/failures. The problem is challenging because adversarial agents may be indistinguishable from a truthful agent with a modified (and valid) cost function. The core issue is that agents lack any means of verifying if the information they receive is truthful, i.e. there is no ``ground truth". To address this problem, we make use of an observation graph in addition to a communication graph, as well as pruning of extreme messages. Under the assumption that the number of adversaries/malicious agents does not exceed the number of truthful ones, we show that our algorithm is resilient against adversarial agents and converges to the Nash equilibrium.

Keywords: Game theory, Optimization, Stochastic systems
Abstract: We propose a new operator splitting algorithm for distributed Nash equilibrium seeking under stochastic uncertainty, featuring relaxation and inertial effects. The proposed algorithm is derived from a forward-backward-forward scheme for solving structured monotone inclusion problems with Lipschitz continuous and monotone pseudogradient operator. To the best of our knowledge, this is the first distributed generalized Nash equilibrium seeking algorithm featuring acceleration techniques in stochastic Nash equilibrium problems without assuming cocoercivity. Numerical examples illustrate the effect of inertia and relaxation on the performance of our proposed algorithm.

Keywords: Game theory, Distributed control, Control over communications
Abstract: Distributed Nash Equilibrium (NE) seeking for networked games has been widely investigated in recent years. In particular, convergence speed and communication cost are two of the critical concerns for the design of the seeking algorithms. To achieve fast convergence while saving communication resources, this paper investigates the appointed-time distributed NE seeking problem for networked games under a discrete-time communication scenario. By utilizing the idea of motion planning, a new continuous-time distributed NE seeking algorithm, which ensures convergence to NE at the prescribed time, is proposed based on sampled-data information exchange. Compared with the existing related works, the established algorithm has advantages on both the communication cost and convergence speed (i.e., appointed-time convergence). Finally, the effectiveness of the proposed method is verified via numerical simulations.

Keywords: Mean field games, Game theory, Networked control systems
Abstract: In this paper, we propose a game between an exogenous adversary and a network of agents connected via a multigraph. The multigraph is composed of (1) a global graph structure, capturing the virtual interactions among the agents, and (2) a local graph structure, capturing physical/local interactions among the agents. The aim of each agent is to achieve consensus with the other agents in a decentralized manner by minimizing a local cost associated with its local graph and a global cost associated with the global graph. The exogenous adversary, on the other hand, aims to maximize the average cost incurred by all agents in the multigraph. We derive Nash equilibrium policies for the agents and the adversary in the Mean-Field Game setting, when the agent population in the global graph is arbitrarily large and the ``homogeneous mixing" hypothesis holds on local graphs. This equilibrium is shown to be unique and the equilibrium Markov policies for each agent depend on the local state of the agent, as well as the influences on the agent by the local and global mean fields.

Keywords: Game theory, Distributed control, Agents-based systems
Abstract: In this paper, we study the generalized Nash equilibrium (GNE) seeking problem for an N-coalition game with inequality constraints. First, using full decision information, a finite-time average consensus-based approach is proposed. It is demonstrated that the algorithm converges to a GNE (specifically, a variational equilibrium) of the game. Then, using the finite-time consensus tracking technique, we propose a distributed algorithm that only requires neighboring action information for each agent. The solution by using partial decision information covers the distributed GNE seeking solutions of generalized non-cooperative games as a special case.

Keywords: Game theory, Stochastic systems, Variational methods
Abstract: We study stochastic Nash equilibrium problems with expected valued cost functions whose pseudogradient satisfies restricted monotonicity properties which hold only with respect to the solution. We propose a forward-backward algorithm and prove its convergence under restricted strong monotonicity, restricted strict monotonicity and restricted cocoercivity of the pseudogradient mapping. To approximate the expected value, we use either a finite number of samples and a vanishing step size or an increasing number of samples with a constant step. Numerical simulations show that our proposed algorithm might be faster than the available algorithms.

Keywords: Numerical algorithms, Optimization algorithms
Abstract: This paper introduces tvopt, a Python framework for prototyping and benchmarking time-varying (or online) optimization algorithms. The paper first describes the theoretical approach that informed the development of tvopt. Then it discusses the different components of the framework and their use for modeling and solving time-varying optimization problems. In particular, tvopt provides functionalities for defining both centralized and distributed online problems, and a collection of built-in algorithms to solve them, for example gradient-based methods, ADMM and other splitting methods. Moreover, the framework implements prediction strategies to improve the accuracy of the online solvers. The paper then proposes some numerical results on a benchmark problem and discusses their implementation using tvopt. The code for tvopt is available at https://github.com/nicola-bastianello/tvopt.

Keywords: Optimization, Algebraic/geometric methods, Machine learning
Abstract: There has been significant interest in generalizations of the Nesterov accelerated gradient descent algorithm due to its improved performance guarantee compared to the standard gradient descent algorithm, and its applicability to large scale optimization problems arising in deep learning. A particularly fruitful approach is based on numerical discretizations of differential equations that describe the continuous time limit of the Nesterov algorithm, and a generalization involving time-dependent Bregman Lagrangian and Hamiltonian dynamics that converges at an arbitrarily fast rate to the minimum. We develop a Lie group variational discretization based on an extended path space formulation of the Bregman Lagrangian on Lie groups, and analyze its computational properties with two examples in attitude determination and vision-based localization.

Keywords: Optimization, Cyber-Physical Security, Attack Detection
Abstract: In this paper, we focus on using path planning and inter-agent measurements to improve the security of multi-robot systems against possible takeovers from cyber-attackers. We build upon recent trajectory optimization approaches where introspective measurement capabilities of the robots are used in an observation schedule to detect deviations from the preordained routes. This paper proposes additional constraints that can be incorporated in the previous trajectory optimization algorithm based on Alternating Direction Method of Multipliers (ADMM). The new constraints provide guarantees that a compromised robot cannot reach a designed safety zone between observations despite adversarial movement by the attacker. We provide a simulation showcasing the new components of the formulation in a multi-agent map exploration task with several safety zones.

Keywords: Optimization, Distributed control, Network analysis and control
Abstract: Data compression is essential to reduce communication cost in distributed optimization over peer-to-peer networks. In this work, we propose a novel communication-efficient linearly convergent distributed (COLD) algorithm with compressed innovation---the difference between a model and its estimate. COLD supports a class of quantizers with delta-contraction property. For strongly convex distributed problems, we explicitly quantify in theory how the compression affects the linear convergence rate. To the best of our knowledge, we are the first to achieve linear convergence in distributed optimization allowing biased and delta-contracted compressors, which is a typical case in practice. Numerical experiments validate our theoretical results.

Keywords: Optimization, Machine learning, Optimization algorithms
Abstract: Communication delays and synchronization are major bottlenecks for parallel computing, and tolerating asynchrony is therefore crucial for accelerating distributed computation. Motivated by modern interest in optimization problems that do not satisfy convexity assumptions, we present an asynchronous block coordinate descent algorithm for nonconvex optimization problems whose objective functions satisfy the Polyak-Lojasiewicz condition. This condition is a generalization of strong convexity to nonconvex problems and requires neither convexity nor uniqueness of minimizers. Under only assumptions of mild smoothness of objective functions and bounded delays, we prove that a linear convergence rate is obtained. Numerical experiments for logistic regression problems are presented to illustrate the impact of asynchrony upon convergence in this setting.

Keywords: Optimization, Optimization algorithms, Agents-based systems
Abstract: We consider the setting where the nodes of an undirected, connected network collaborate to solve a shared objective modeled as the sum of smooth functions. We assume that each summand is privately known by a unique node. NEAR-DGD is a distributed first order method which permits adjusting the amount of communication between nodes relative to the amount of computation performed locally in order to balance convergence accuracy and total application cost. In this work, we generalize the convergence properties of a variant of NEAR-DGD from the strongly convex to the nonconvex case. Under mild assumptions, we show convergence to minimizers of a custom Lyapunov function. Moreover, we demonstrate that the gap between those minimizers and the second order stationary solutions of the original problem can become arbitrarily small depending on the choice of algorithm parameters. Finally, we accompany our theoretical analysis with a numerical experiment to evaluate the empirical performance of NEAR-DGD in the nonconvex setting.

Keywords: Predictive control for nonlinear systems, Intelligent systems, Nonholonomic systems
Abstract: This paper explores the use of model predictive control (MPC) in dealing with the pursuit-evasion game (PEG) problem where players have incomplete information on their opponents. This is different from most cases in the literature where each player knows all the information (states information and dynamics) on the opponent. The burden caused by such demand for the opponent’s full information induces the need for more sensors during physical implementation as well as high computation time. However, we found that only the current positions, i.e. x-y coordinate of the opponent, are indispensable. Thus, knowing the orientation and the dynamics of the opponent are insignificant to the performance of the game. We propose a new method to exploit a two-player PEG in the presence and absence of obstacles, where each player can only rely on the current position information of its opponent. Several simulation results show that the PEG problem can be handled and obstacles can be avoided using the proposed control protocol. We also show that our approach is robust to measurement noise and can perform better, in terms of the computation than the approach with full information.

Keywords: Automotive control, Optimal control, Predictive control for nonlinear systems
Abstract: This paper studies the active cell balancing problem by using model predictive control (MPC) for real time range extension. Specifically, three MPC formulations are proposed and compared: the first one being a tracking controller to force all cells to follow the same trajectory generated by a nominal cell model, the second one trying to maximize the lowest cell SOC/voltage and the last one minimizing the difference between the highest and lowest cell SOC/voltages. Both steady state and transient conditions are simulated to assess the effectiveness of the proposed controllers, and a range extension of 4% is found for dynamic driving cycle and 7% for steady state condition. Comparing to the literature, our approaches achieve similar range extension, without making the restrictive assumption that the final battery state-of-charge is known in advance, making our approaches more applicable. Real time implementability is demonstrated via throughput analysis.

Keywords: Predictive control for nonlinear systems, Robust control, Stability of nonlinear systems
Abstract: We propose a robust Nonlinear Model Predictive Control (NMPC) scheme that provides an improved trade-off between optimality and complexity when compared to other available strategies. Two controllers are employed in the proposed framework: A multi-stage primary controller that optimizes a given objective in the presence of uncertainties with tightened constraints and a multi-stage ancillary controller that tracks the predicted tree of state and input trajectories of the primary controller. The primary controller optimizes the original objective by considering different realizations of the most significant uncertainties in the predictions. The ancillary controller provides robustness against other uncertainties by tracking the predicted tree of state and input trajectories of the primary controller. We establish sufficient conditions for closed-loop stability. The advantages of the scheme are demonstrated for a continuous stirred tank reactor (CSTR) example.

Keywords: Predictive control for nonlinear systems, Stability of nonlinear systems, Lyapunov methods
Abstract: Model predictive control (MPC) is a valuable tool to deal with systems that require optimal solutions and constraint satisfaction. In the case of systems with uncertainty, the formulation of predictive controllers requires models which are capable to capture system dynamics, constraints and also system uncertainty. In this work we present a formulation for a set-valued model predictive control (SVMPC) where uncertainty is represented in terms of sets. The approach presented here considers a model where the state is set valued and dynamics are defined by a set valued map. The cost function associated to the proposed MPC associates a real-valued cost to each set valued (or tube-based) trajectory. For this formulation, we study conditions that can yield the constrained optimal control problem associated to the set-valued MPC formulation feasible and stable, thus extending existing stability results from classic MPC to a set based approach. Examples illustrate the results along the paper.

Keywords: Predictive control for nonlinear systems, Autonomous vehicles, Optimal control
Abstract: This article proposes a new safety concept: dynamically formulated backup plan safety. The backup plan safety is defined as the ability to complete one of the alternative missions formulated in real-time in the case of primary mission abortion. To incorporate this new safety concept in control problems, we formulate a feasibility maximization problem that adopts additional (virtual) input horizons toward the alternative missions on top of the input horizon toward the primary mission. Cost functions for the primary and alternative missions construct multiple objectives, and multi-horizon inputs evaluate them. To address the feasibility maximization problem, we develop a multi-horizon multi-objective model predictive path integral control (3M) algorithm. Model predictive path integral control (MPPI) is a sampling-based scheme that can help the proposed algorithm deal with nonlinear dynamic systems and achieve computational efficiency by parallel computation. Simulations of the aerial vehicle control problems demonstrate the new concept of backup plan safety and the performance of the proposed algorithm.

Keywords: Predictive control for nonlinear systems, Machine learning, Optimal control
Abstract: Solving Non-Linear Model Predictive Control (NMPC) online is often challenging due to the computational complexities involved. This issue can be avoided by approximating the optimization problem using supervised learning methods which comes with a trade-off on the optimality and/or constraint satisfaction. In this paper, a novel supervised learning framework for approximating NMPC is proposed, where we explicitly impart constraint knowledge within the neural networks. This knowledge is inherited by augmenting the loss function of the neural networks during the training phase with insights from KKT conditions. Logarithmic barrier functions are utilized to augment the loss function including conditions of primal and dual feasibility. The proposed framework can be applied to other machine learning based parametric approximators. This approach is easy to implement and its efficacy is demonstrated on a benchmark NMPC problem for continuous stirred tank reactor (CSTR).

Keywords: Automata, Manufacturing systems and automation
Abstract: Given a number of synchronised automata, compositional verification seeks to verify non-conflictingness without the explicit computation of an overall model. Technically, the approach alternates conflict-preserving abstractions with the composition of a small number of strategically chosen automata and the literature reports substantial computational benefits for examples of practical relevance. In this paper, we re-visit this approach in order to address the situation of preemptive events, i.e., events that are known beforehand to be scheduled at highest priority. Our study is motivated by high-level programming languages commonly used in industrial automation, where events associated with actuators preempt events associated with sensors.

Keywords: Discrete event systems, Automata, Supervisory control
Abstract: In this paper, we investigate the problem of distributed sensing and information transmission in partially observed discrete-event systems, where the sensing and information transmission are complicated by a set of edge sensors. Each edge sensor selectively transmits its observable events, according to an information transmission policies, to a central site for the purpose of decision making. In this paper, we consider a general class of decision making requirement at the central site called the distinguishability. Then we investigate both the verification and synthesis problems. For the verification problem, two different approaches, one based on the observer and the other based on the verifier, are proposed to check whether or not a given set of sensor transmission policies fulfills the distinguishability requirement at the central site. For the synthesis problem, we also develop an effective algorithm to design an observer-based optimal information transmission policy for each edge sensor such that they are verified to be distinguishable.

Keywords: Discrete event systems, Automata, Supervisory control
Abstract: In this paper, we propose a novel framework for modeling and analysis of networked discrete-event systems (DES). We assume that the plant is controlled by a feedback supervisor whose control decisions are subject to communication delays and losses. Furthermore, we consider a general setting where the supervisor sends control decisions to different actuators via different communication channels whose dynamics are independent.We provide a system theoretic approach by identifying the state-space of overall networked system and investigating the dynamic of the entire state-space. Our approach precisely specifies the roles of the supervisor, the communication channels and the actuators. Also, we compare the proposed networked DES model with the existing one and show that the proposed networked model captures physical situations of networked systems more precisely.

Keywords: Discrete event systems, Automata, Supervisory control
Abstract: In this paper, we investigate the problem of optimal supervisory control for cyclic tasks in the context of discrete-event systems (DES). We consider the completion of each single task as the visit of a marked state, and overall control objective is to complete tasks cyclically in the sense that marked states are visited infinitely often. Following the standard optimal supervisory control framework, two types of costs, disable cost and occurrence cost, are considered. However, instead of considering the standard accumulated total cost or the average cost per event, we propose a new measure for the control performance using the average cost per task. We show that such an optimality measure is more suitable for tasks that need to be completed cyclically. Our goal is to design a live and non-blocking supervisor such that the average cost per task in the worst-case is minimized. To solve the problem, we propose a game-theoretical approach by converting the optimal control problem as a two-player graph game. The constructed game is then solved in two stages: one focuses on the optimal execution within each single task cycle and the other focuses on the scheduling strategy among different tasks. Illustrative examples are provided to demonstrate the proposed algorithm.

Keywords: Discrete event systems, Computer/Network Security, Automata
Abstract: Opacity is an information flow property for dynamic systems describing plausible deniability, that is whether an eavesdropper can deduce that ``secret'' behavior has occurred. In particular, K-step opacity considers secret actions that have occurred within the last K-steps in the past. We consider the problem of K-step opacity enforcement over automata using obfuscation. We present a general framework for K-step opacity enforcement and transform the problem of enforcing K-step opacity to enforcing current-state opacity. We can then apply existing obfuscation synthesis methods for current-state opacity to K-step opacity. We demonstrate this approach by enforcing privacy in the context of a novel contact tracing model.

Keywords: Discrete event systems, Supervisory control, Automata
Abstract: In this paper, we propose a new automaton property of N-step nonblockingness for a given positive integer N. This property quantifies the standard nonblocking property by capturing the practical requirement that all tasks be completed within a bounded number of steps. Accordingly, we formulate a new N-step nonblocking supervisory control problem, and characterize its solvability in terms of a new concept of N-step language completability. It is proved that there exists a unique supremal N-step completable sublanguage of a given language, and we develop a generator-based algorithm to compute the supremal sublanguage. Finally, together with the supremal controllable sublanguage, we design an algorithm to compute a maximally permissive supervisory control solution to the new N-step nonblocking supervisory control problem.

Keywords: Adaptive control, Algebraic/geometric methods, Nonlinear systems
Abstract: In this paper, we propose an adaptive control law for completely unknown scalar linear systems based on Lie-bracket approximation methods. We investigate stability and convergence properties for the resulting Lie-bracket system, compare our proposal with existing Nussbaum-type solutions and demonstrate our results with an example. Even though we prove global stability properties of the Lie-bracket system, the stability properties of the proposed dynamics remain open, making the proposed control law an object of further studies. We elaborate the difficulties of establishing stability results by investigating connections to partial stability as well as studying the corresponding Chen-Fliess expansion.

Keywords: Adaptive control, Delay systems, LMIs
Abstract: In this paper, we present a constructive approach to continuous-time extremum seeking (ES) by using a time delay approach to averaging. We consider gradient-based ES of static maps in the case of single-input, and we study two ES methods: the classical one and a more recent bounded ES method. By transforming the ES dynamics into a time delay system where the delay is the period of dither, we derive the practical stability conditions for the resulting time-delay system. The time-delay system stability guarantees the stability of the original ES plant. Under the assumption of some known bounds on the extremum value and the Hessian, the time delay approach provides a quantitative calculation on the lower bound of the frequency and on the resulting ultimate bound. We also give a bound on the neighborhood of the extremum point starting from which the solution is ultimately bounded. When the Hessian and the extremum value bounds are unknown, we provide, for the first time, the asymptotic ultimate bound in terms of the frequency in the case of bounded ES. Moreover, our explicit bound on the seeking error of ES control systems allows to select appropriate tuning parameters (such as dither frequency, magnitude, and control gain).

Keywords: Adaptive control, Lyapunov methods, Neural networks
Abstract: Functional electrical stimulation (FES) can be used as rehabilitative treatment for lost motor neuron function in people with neurological disorders. This paper considers a leg extension machine coupled to a participant for FES-induced closed-loop lower-limb tracking of a desired trajectory. FES-induced control faces challenges as the muscle dynamics exhibit nonlinear behaviors and have unstructured uncertainty. A closed-loop data-driven deep neural network (DNN)-based adaptive control method for FES-induced lower-limb position trajectory tracking is developed. A Lyapunov-based stability analysis is used to develop a closed-loop state-feedback adaptation law for the outer-layer weights of the DNN, which is combined with a feedback controller to yield semi-global asymptotic tracking.

Keywords: Adaptive control, Lyapunov methods, Stochastic systems
Abstract: This paper is motivated by the problem of quantitatively bounding the convergence of adaptive control methods for stochastic systems to a stationary distribution. Such bounds are useful for analyzing statistics of trajectories and determining appropriate step sizes for simulations. To this end, we extend a methodology from (unconstrained) stochastic differential equations (SDEs) which provides contractions in a specially chosen Wasserstein distance. This theory focuses on unconstrained SDEs with fairly restrictive assumptions on the drift terms. Typical adaptive control schemes place constraints on the learned parameters and their update rules violate the drift conditions. To this end, we extend the contraction theory to the case of constrained systems represented by reflected stochastic differential equations and generalize the allowable drifts. We show how the general theory can be used to derive quantitative contraction bounds on a nonlinear stochastic adaptive regulation problem.

Keywords: Adaptive control, Quantized systems, Nonlinear systems
Abstract: In this paper, the attitude tracking control problem of a rigid body is investigated where the states are quantized. An adaptive backstepping based control scheme is developed and a new approach to stability analysis is developed by constructing a new compensation scheme for the effects of the vector state quantization. It is shown that all closed-loop signals are ensured uniformly bounded and the tracking errors converge to a compact set containing the origin. Experiments on a 2 degrees-of-freedom helicopter system illustrate the proposed control scheme.

Keywords: Adaptive control, Direct adaptive control, Robust adaptive control
Abstract: It is well recognized that Almost Strictly Positive Real~(ASPR)-based output feedback control has robustness with respect to disturbances and system's uncertainty. However, most practical systems do not have ASPR property. To solve this problem, the introduction of a parallel feedforward compensator~(PFC) bas been proposed in order to render the resulting augmented system ASPR and adaptive type PFC design methods have been also provided already. The adaptive method can design an appropriate PFC by only utilizing input and output online data of the system without the strict information of the system, and thus it is effective for the uncertain systems. However, most of the schemes were only for stable systems. In this paper, we proposed the adaptive PFC design method for unstable systems. Moreover, the adaptive feedforward~(FF) input based on Radial Basis Function~(RBF) Neural Network~(NN) is introduced in order to achieve the adequate output tracking. The stability of the obtained adaptive control system is also analyzed and the boundedness of all the signals in the control system will be shown.

Keywords: Linear systems, Fault tolerant systems, Emerging control applications
Abstract: Software rejuvenation was born to fix operating system faults by periodically refreshing the run-time code and data. This mechanism has been extended to protect control systems from cyber-attacks. This work proposes a software rejuvenation design method in discrete-time where invariant sets for the safety and mission controllers are designed to schedule the timing of software refreshes. To compute a minimal robust positively invariant (min-RPI) set and the bounded time between software refreshes to ensure system safety, an LP based approach is proposed for stable and unstable systems. Finally, the designed approach is illustrated by the case study of a simulated lab-scale microgrid.

Keywords: Linear systems, Optimal control, Stochastic optimal control
Abstract: The behaviour of a stochastic dynamical system may be largely influenced by those low-probability, yet extreme events. To address it, this paper proposes an infinite-horizon risk-constrained Linear Quadratic Regulator (LQR) framework with average cost. Besides the standard LQR objective, the average one-stage predictive variance of the state penalty is constrained within a user-specified level. By leveraging the dual theory, its optimal solution is first shown to be stationary and affine in the state, i.e., u(lambda^*) = -K(lambda^*)x + l(lambda^*), where lambda^* is an optimal multiplier to address the risk constraint. Then, we establish the stability of the resulting closed-loop system. Furthermore, we propose a primal-dual method with sublinear convergence rate to find an optimal policy-multiplier pair (u(lambda^*), lambda^*). Finally, a numerical example is given to demonstrate the effectiveness of the proposed framework and the primal-dual method.

Keywords: Linear systems, Identification, Subspace methods
Abstract: In this paper, we propose variations of Willems’ fundamental lemma that utilize second-order moments such as correlation functions in the time domain and power spectra in the frequency domain. We believe that using a formulation with estimated correlation coefficients is suitable for data compression, and possibly can reduce noise. Also, the formulations in the frequency domain can enable modeling of a system in a frequency region of interest.

Keywords: Optimal control, Differential-algebraic systems, Linear systems
Abstract: By employing the properties of centered realizations, we devise a modified version of the dual Riccati equation approach to optimal H-2 control for differential-algebraic systems that is guaranteed to be more computationally efficient and numerically accurate than other methods from literature. Moreover, we show that if the optimal controller has an improper transfer function matrix, then all controllers which ensure a finite H-2 norm will share this property.

Keywords: Linear systems, Time-varying systems
Abstract: The purpose of this paper is to provide some contributions to one notion of Output controllability for Linear Time Varying (LTV) systems which is the Complete State to Output Controllability (CSOC), notion introduced in the 60s by P.E.~Sarachik and G.M.~Kranc. More precisely, we consider LTV systems with direct transmission of the input to the output and establish criteria to ensure the CSOC in finite time of these systems. We also give, under the assumption of CSOC in finite time, an explicit expression of a continuous control built by means of a Gramian matrix.

Keywords: Linear systems
Abstract: We revisit the classical pole-placement controller design problem with the objective of ensuring non-overshooting and error-free reference tracking. To this end, we present a number of novel conditions for external positivity of discrete-time transfer functions which characterise this non-overshooting property. These conditions are both necessary and sufficient for first- and second-order systems, but only sufficient for higher-order systems. In addition, they have simple geometric interpretations in terms of allowed locations of the closed-loop poles. Based on these conditions, we propose a control design procedure that guarantees a stable closed-loop system with non-overshooting, error-free tracking of step changes in the reference.

Keywords: Computational methods, Delay systems, Nonlinear output feedback
Abstract: In this paper we present an approximate feedback controller design methodology for tracking/disturbance-rejection regulation problems governed by nonlinear delay differential control systems. The method considered here is a version of the practical regulation approach developed by the authors in a series of articles. It is important to note that this approach to regulation does not require the existence of an exo-system to define disturbances and signals to be tracked. Therefore, this control law enables tracking and disturbance rejection for general reference and disturbance signals. The idea is similar to the inclusion of a cascade controller design providing a sequence of increasingly more accurate and better preforming controllers. The underlying principle derives from well known geometric methods which rely on the existence of an attractive invariant manifold in the case when the reference and disturbance signals are outputs of an autonomous, linear, neutrally stable exo-system. However, we are able accomplish high performance tracking without this assumption. References are provided for the history of the methodology and proofs of the error estimates for general systems. We show, in our included example, that the tracking error can be significantly enhanced using one extra step in the sequence of controllers. In particular, at each step in the cascade controller the error from the previous step provides the reference signal for the next step. In this way, at each step the errors are reduced geometrically.

Keywords: Distributed parameter systems
Abstract: We design a practical mobile actuator guidance policy for linear parabolic equations in 2D: the guidance is chosen so that H-2-measure of uncertainty is minimized provided the system is subject to a distributed disturbance. We first present a guidance policy where the mobile actuator location to be selected will be fixed over a certain time interval of interest. Further we add extra complexity by taking into account the dynamics of the mobile actuator over the 2D domain of interest under reachability constraints. The proposed approach is illustrated through numerical studies.

Keywords: Distributed parameter systems, Sampled-data control, Nonlinear output feedback
Abstract: This paper provides an observer-based sampled-data boundary control strategy for a class of reaction-diffusion PDEs with collocated sensing and Robin actuation. Infinite-dimensional backstepping design is used as the underlying control approach. For the first time, it is shown that the continuous-time output feedback boundary control applied in a sample-and-hold fashion ensures global closed-loop exponential stability, provided that the sampling period is sufficiently small. Further, robustness to perturbations of the sampling schedule is guaranteed. A simulation example demonstrates that the convergence can still be achieved with sampling periods significantly larger than the theoretical maximum of the sampling schedule diameter.

Keywords: Distributed parameter systems, Adaptive control, Identification
Abstract: We present an event-triggered state-feedback boundary control scheme with a, likewise, event-triggered batch least-squares parameter identification for a 2times 2 hyperbolic PDE-ODE system, where two coefficients of the in-domain couplings between two transport PDEs, and the system parameter of a scalar ODE are unknown. The triggering condition is designed based on evaluating both the actuation deviation caused by the difference between the plant states and their sampled values, and the growth of the plant norm. When either condition is met, the piecewise-constant control input and parameter estimates are updated simultaneously. In the closed-loop system, the following results are obtained: 1) the absence of a Zeno phenomenon; 2) finite-time exact identification of the unknown parameters from all but a measure zero set of initial conditions; 3) exponential regulation of the plant states to zero. In the numerical simulation, the design is verified in an application of axial vibration control of a mining cable elevator, where the damping coefficients of the cable and the cage are unknown.

Keywords: Distributed parameter systems, Sampled-data control, Lyapunov methods
Abstract: This paper discusses sampled-data stabilization by switching for 1-D nonlinear reaction-diffusion equation with spatially scheduled actuators. We suggest that the interval [0,1] is divided into N subdomains. We assume that N sensors are placed in each subdomain and measure the average value of the state in the discrete time. We stabilize the system by switching sampled-data static output-feedback. This switching control law can be implemented either by using one moving actuator that can move to the active subdomain in the negligible time or by N actuators placed in each subdomain. In the latter case switching control may reduce the energy that the system spends. Constructive conditions are derived to ensure that the resulting closed-loop system is exponentially stable by means of the Lyapunov approach. A numerical example illustrates the efficiency of the method.

Keywords: Distributed parameter systems, Feedback linearization, Control applications
Abstract: Tokamaks are torus-shaped devices designed to confine a plasma (ionized gas at around 100 million degrees where fusion reactions can take place) using helical magnetic fields. Such magnetic confinement enables light ions, such as isotopes of hydrogen, to stay confined long enough to undergo a fusion reaction. The pitch of the helical magnetic field in a tokamak is characterized by the safety factor q. The safety factor is closely related to the magnetohydrodynamic stability of the plasma. For instance, instabilities that can degrade or even terminate plasma confinement can occur at spatial locations with rational values of the safety factor q. Thus, actively increasing the minimum magnitude of the safety factor can reduce the occurrence of low-order (low rational q values) instabilities. Non-inductive sources of current like neutral beam injection (NBI) and electron cyclotron current drive (ECCD) are used to control the q-profile. ECCD generates electromagnetic waves to drive current and/or heat the plasma. Mirrors are used to control the spatial region of incidence of the generated electromagnetic waves. In this work, the ECCD mirror’s position is treated as a controllable input, and its effects are included in the response model used for control design. A controller based on feedback linearization is proposed to simultaneously allocate the NBI and ECCD powers and the ECCD position to track a target minimum safety factor. The effectiveness of the controller is assessed for a DIII-D tokamak scenario in nonlinear one-dimensional simulations using COTSIM (Control-Oriented Transport SIMulator).

Keywords: Predictive control for linear systems, Distributed control, Adaptive control
Abstract: Adaptive model predictive control (MPC) robustly ensures safety while reducing uncertainty during operation. In this paper, a distributed version is proposed to deal with network systems featuring multiple agents and limited communication. To solve the problem in a distributed manner, structure is imposed on the control design ingredients without sacrificing performance. Decentralized and distributed adaptation schemes that allow for a reduction of the uncertainty online compatibly with the network topology are also proposed. The algorithm ensures robust constraint satisfaction, recursive feasibility and finite gain l2 stability, and yields lower closed-loop cost compared to robust distributed MPC in simulations.

Keywords: Distributed control, Control of networks, Linear systems
Abstract: This paper deals with data-driven output synchronization for heterogeneous leader-follower linear multi-agent systems. Given a multi-agent system that consists of one autonomous leader and a number of heterogeneous followers with external disturbances, we provide necessary and sufficient data-based conditions for output synchronization. We also provide a design method for obtaining such output synchronizing protocols directly from data. The results are then extended to the special case that the followers are disturbance-free. Finally, a simulation example is provided to illustrate our results.

Keywords: Control of networks, Cooperative control
Abstract: This paper focuses on time-critical multiagent systems over the finite-time interval [0,T). Specifically, we propose a distributed control algorithm that guarantees agents to approach a time-varying command at time T with reduced information exchange. The key feature of the proposed algorithm allows T to be user-defined, where it neither depends on the initial conditions of agents nor unknown but bounded time rate of change of the command. Furthermore, an event- triggered approach is utilized to reduce agent-to-agent information exchange. Predicated on a time transformation approach, the proposed algorithm defined over the finite-time interval [0,T) is first transformed to an equivalent form over the infinite-time interval [0,infinity) through a strictly increasing and continuously differentiable function that guarantees solutions in both time intervals being equivalent. Over the infinite-time interval [0,infinity), we then use a quadratic energy function, the Bernoulli equation, and the comparison principle to establish stability and convergence properties of our algorithm.

Keywords: Distributed control, Agents-based systems, Networked control systems
Abstract: In this paper we address the finite-time distributed link prediction problem in networked multi-agent systems which consists in estimating the likelihood of existence of neighboring links in the network, exploiting only local information. Notably, this setting extends the framework introduced by Pech et al., which considered a complete network topology and was based on a centralized architecture, to the case of sparse graphs and distributed computation. Briefly, first we derive an optimality condition for the problem and then we develop a distributed protocol which drives the agents to satisfy in finite-time the optimality condition by relying on a state-of-the-art distributed k-hop observer. Numerical simulations demonstrate the effectiveness of the proposed distributed protocol.

Keywords: Distributed control, Cooperative control
Abstract: A graph theoretic framework recently has been proposed for the robust distributed stabilization of interconnected multiagent systems, while systematically capturing the architectural aspect of cyber-physical systems with separate agent or physical layer and control or cyber layer. We consider a scenario where the control layer faces a series of centralized denial of service attacks, and the agent layer is subject to the modeling uncertainties. We propose a step-by-step procedure to design a control layer that, in the presence of the aforementioned abnormalities, guarantees a level of robustness and resiliency for the final two-layer interconnected multiagent system. The incorporation of an event-triggered strategy further ensures an effective use of the limited energy and communication resources over the control layer. We theoretically prove the resilient, robust, and Zeno-free convergence of all state trajectories to the origin and, via a simulation study, discuss the feasibility of the proposed ideas.

Keywords: Distributed control, Cooperative control
Abstract: Collaborative load manipulation and transportation is an emerging application of multi-unmanned aerial vehicle (UAV) systems. We address a problem where a team of UAVs transport a semi-flexible payload in an environment with multiple obstacles. By considering a semi-flexible payload, we restrict its flexibility to allow for safe-shape manipulation, which helps to navigate between obstacles while avoiding inter-UAV and payload-obstacle collisions. Control barrier functions (CBFs) are used to construct the obstacle avoidance and shape constraints, which are applied along with actuator constraints to an optimization-based control problem. The analysis is supported with simulation results for a team of four UAVs manipulating a payload in the presence of obstacles and settling on a standoff circle about a target.

Keywords: Sensor networks, Autonomous systems, Estimation
Abstract: We present a breadth-first sensor configuration strategy to find near-optimal placement and sensor field of view (FoV). The strategy couples the sensor configuration procedure directly with the decision making task of planning a path for an agent in an unknown static environment comprised of threats. This coupled sensor configuration and path-planning (CSCP) strategy iteratively uses Gaussian Process Regression to construct a threat field estimate and find a candidate optimal path with minimum threat exposure. The strategy utilizes a unique task-driven information gain (TDIG) metric, which yields the sensor configurations when maximized. Due to the non-convex and non-submodular nature of the problem, we present an approximation for the optimization of the TDIG metric. Finally, we discuss the performance of the breadth-first strategy in contrast to a standard and depth-first strategy as well as traditional information-maximization.

Keywords: Sensor networks, Kalman filtering
Abstract: This paper proposes an improved design approach for distributed consensus Kalman filtering (DCKF). We provide an improved consensus gain factor compared to the sub-optimal design proposed in [1]. This factor is derived from an LMI appearing in the stability analysis of the DCKF and can be computed using semi-definite programming. We also propose a decentralized consensus gain that can be computed by each agent in the sensor network, and depends only on local properties of the network, i.e., the number of neighbors of each sensor. We show in simulation that this approach holds even for networks with time varying communication regime. Our results are compared to other existing solutions in the literature with a numerical example.

Keywords: Estimation, Sensor networks, Linear systems
Abstract: This paper focuses on Distributed State Estimation over a peer-to-peer sensor network composed by possible low-computational sensors. We propose a new l-step Neighbourhood Distributed Moving Horizon Estimation technique with fused arrival cost and pre-estimation, improving the accuracy of the estimation, while reducing the computation time compared to other approaches from the literature. Simultaneously, convergence of the estimation error is improved by means of spreading the information amongst neighbourhoods, which comes natural in the sliding window data present in the Moving Horizon Estimation paradigm.

Keywords: Optimization, Optimization algorithms, Sensor networks
Abstract: In networks of autonomous agents (e.g., fleets of vehicles, scattered sensors), the problem of minimizing the sum of the agents’ local functions has received a lot of interest. We tackle here this distributed optimization problem in the case of open networks when agents can join and leave the network at any time. Leveraging recent online optimization techniques, we propose and analyze the convergence of a decentralized asynchronous optimization method for open networks.

Keywords: Stochastic optimal control, Sensor networks, Energy systems
Abstract: Numerous mobile robotic applications require agents to persistently explore and exploit spatiotemporally varying, partially observable environments. Ultimately, the mathematical notion of regret, which quite simply represents the instantaneous or time-averaged difference between the optimal reward and realized reward, serves as a meaningful measure of how well the agents have exploited the environment. However, while numerous theoretical regret bounds have been derived within the machine learning community, restrictions on the manner in which the environment evolves preclude their application to persistent missions. On the other hand, meaningful theoretical properties can be derived for the related concept of dynamic coverage, which serves as an exploration measurement but does not have an immediately intuitive connection with regret. In this paper, we demonstrate a clear correlation between an appropriately defined measure of dynamic coverage and regret, then go on to derive performance bounds on dynamic coverage as a function of the environmental parameters. We evaluate the correlation for several variants of an airborne wind energy system, for which the objective is to adjust the operating altitude in order to maximize power output in a spatiotemporally evolving wind field.

Keywords: Flight control, Cooperative control, Adaptive control
Abstract: This paper studies the leader-following practical consensus with disturbance rejection problem of a group of rigid spacecraft systems with uncertain inertia matrices, which extends our previous work by further considering disturbance rejection. Like our previous results, we use the adaptive distributed observer technique to handle switching communication constraints. To handle the disturbance, we further adopt the internal model approach to convert our problem into a practical stabilization problem of an augmented system. Specific effort is made on designing an event-triggered control law with an event-triggered mechanism that avoids the Zeno phenomenon.

Keywords: Control over communications, Sampled-data control, Networked control systems
Abstract: Self-triggered control (STC) is a well-established technique to reduce the amount of samples for sampled-data systems, and is hence particularly useful for Networked Control Systems. At each sampling instant, an STC mechanism determines not only an updated control input but also when the next sample should be taken. In this paper, a dynamic STC mechanism for nonlinear systems is proposed. The mechanism incorporates a dynamic variable for determining the next sampling instant. Such a dynamic variable for the trigger decision has been proven to be a powerful tool for increasing sampling intervals in the closely related concept of ETC, but was so far not exploited for STC. This gap is closed in this paper. For the proposed mechanism, the dynamic variable is chosen to be the filtered values of the Lyapunov function at past sampling instants. The next sampling instant is, based on the dynamic variable and on hybrid Lyapunov function techniques, chosen such that an average decrease of the Lyapunov function is ensured. The proposed mechanism is illustrated with a numerical example from the literature. For this example, the obtained sampling intervals are significantly larger than for existing static STC mechanisms.

Keywords: Lyapunov methods, Constrained control, Uncertain systems
Abstract: This paper addresses the problem of safety-critical control for systems with unknown dynamics. It has been shown that stabilizing affine control systems to desired (sets of) states while optimizing quadratic costs subject to state and control constraints can be reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). Our recently proposed High Order CBFs (HOCBFs) can accommodate constraints of arbitrary relative degree. One of the main challenges in this approach is obtaining accurate system dynamics, which is especially difficult for systems that require online model identification given limited computational resources and system data. In order to approximate the real unmodeled system dynamics, we define adaptive affine control dynamics which are updated based on the error states obtained by real-time sensor measurements. We define an HOCBF for a safety requirement on the unmodeled system based on the adaptive dynamics and error states, and reformulate the safety-critical control problem as the above mentioned QP. Then, we determine the events required to solve the QP in order to guarantee safety, and derive a condition that guarantees the satisfaction of the HOCBF constraint between events. We illustrate the effectiveness of the proposed framework on adaptive cruise control and compare it with the classical time-driven approach.

Keywords: Hybrid systems, Observers for Linear systems, Networked control systems
Abstract: We present an event-triggered observer design for linear time-invariant systems, where the measured output is sent to the observer only when a triggering condition is satisfied. We proceed by emulation and we first construct a continuous-time Luenberger observer. We then propose a dynamic rule to trigger transmissions, which only depends on the plant output and an auxiliary scalar state variable. The overall system is modeled as a hybrid system, for which a jump corresponds to an output transmission. We show that the proposed event-triggered observer guarantees global practical asymptotic stability for the estimation error dynamics. Moreover, under mild boundedness conditions on the plant state and its input, we prove that there exists a uniform strictly positive minimum inter-event time between any two consecutive transmissions, guaranteeing that the system does not exhibit Zeno solutions. Finally, the proposed approach is applied to a numerical case study of a lithium-ion battery.

Keywords: Delay systems
Abstract: We provide a new positive system approach to event-triggered feedback control of linear systems with state delays. We use an interval observer method and linear Lyapunov functions to prove global exponential stability of the closed loop systems. Our two examples illustrate the usefulness of our method for counteracting the effects of potentially destabilizing state delays.

Keywords: Estimation, Sensor networks, Hybrid systems
Abstract: General nonlinear continuous-time systems are considered for which its state is estimated via a packet-based communication network. We assume that the system has multiple sensor nodes, affected by measurement noise, which can transmit at discrete (non-equidistant) points in time. Moreover, each node can transmit asynchronously. For this setup, we develop a state estimation framework, where the transmission instances of the individual sensor nodes can be generated in either time-triggered or event-triggered fashions. In the latter case, we guarantee the absence of Zeno behavior by construction. It is shown that, under the provided design conditions, an input-to-state stability property is obtained for the estimation error with respect to the measurement noise and process disturbances and that the state is thus reconstructed asymptotically in the absence of noise.

Keywords: Formal Verification/Synthesis, Autonomous systems, Machine learning
Abstract: In many autonomy applications, performance of perception algorithms is important for effective planning and control. In this paper, we introduce a framework for computing the probability of satisfaction of formal system specifications given a confusion matrix, a statistical average performance measure for multi-class classification. We define the probability of satisfaction of a linear temporal logic formula given a specific initial state of the agent and true state of the environment. Then, we present an algorithm to construct a Markov chain that represents the system behavior under the composition of the perception and control components such that the probability of the temporal logic formula computed over the Markov chain is consistent with the probability that the temporal logic formula is satisfied by our system. We illustrate this approach on a simple example of a car with pedestrian on the sidewalk environment, and compute the probability of satisfaction of safety requirements for varying parameters of the vehicle. We also illustrate how satisfaction probability changes with varied precision and recall derived from the confusion matrix. Based on our results, we identify several opportunities for future work in developing quantitative system-level analysis that incorporates perception models.

Keywords: Formal Verification/Synthesis, Autonomous systems, Robust control
Abstract: We present a robust control framework for time-critical systems in which satisfying real-time constraints robustly is of utmost importance for the safety of the system. Signal Temporal Logic (STL) provides a formal means to express a large variety of real-time constraints over signals and is suited for planning and control purposes as it allows us to reason about the time robustness of such constraints. The time robustness of STL particularly quantifies the extent to which timing uncertainties can be tolerated without violating real-time specifications. In this paper, we first pose a control problem in which we aim to find an optimal input sequence to a control system that maximizes the time robustness of an STL constraint. We then propose a Mixed Integer Linear Program (MILP) encoding and provide correctness guarantees along with a complexity analysis of the encoding. We also show in two case studies that maximizing STL time robustness allows to account for timing uncertainties of the underlying control system.

Keywords: Autonomous systems, Formal Verification/Synthesis, Hierarchical control
Abstract: This paper studies the controller synthesis problem for Linear Temporal Logic (LTL) specifications using (constrained) zonotope techniques. To begin with, we implement (constrained) zonotope techniques to partition the state space and further to verify whether the LTL specification can be satisfied. Once the LTL specification can be satisfied, the next step is to design a controller to guarantee the satisfaction of the LTL specification for dynamic systems. Based on the verification of the LTL specification, an abstraction-based control design approach is proposed in this paper: a novel abstraction construction is developed first, then finite local abstract controllers are designed to achieve the LTL specification, and finally the designed abstract controllers are combined and refined as the controller for the original system. The proposed control design strategy is illustrated via a numerical example from autonomous robots.

Keywords: Formal Verification/Synthesis, Discrete event systems, Cyber-Physical Security
Abstract: We consider the probabilistic planning problem where a robot (called Player 1, or P1) can jointly plan the control actions and sensor queries in a sensor network and an attacker (called player 2, or P2) can carry out attacks on the sensors. We model such an adversarial interaction using a formal model--a reachability game with partially controllable observation functions. The main contribution of this paper is to assess the cost of P1's unawareness: Suppose P1 misinterprets the sensor failures as probabilistic node failures due to unreliable network communication, and P2 is aware of P1's misinterpretation and P1's partial observability. Then, from which states can P2 carry out sensor attacks to ensure, with probability one, that P1 will not be able to complete her reachability task even though, due to misinterpretation, P1 believes that she can almost-surely achieve her task. We develop an algorithm to solve the almost-sure winning sensor-attack strategy given P1's observation-based strategy. Our attack analysis could be used for attack detection in wireless communication networks and the design of provably secured attack-aware sensor allocation in decision-theoretic models for cyber-physical systems.

Keywords: Formal Verification/Synthesis, Hybrid systems, Optimal control
Abstract: Abstraction techniques provide formal guarantees for generic optimal control problems on nonlinear and hybrid systems. Computing an abstraction solving the problem over the whole state-space is computationally demanding in high dimensional spaces. We circumvent this curse of dimensionality by introducing a hierarchical abstraction approach for solving an optimal control problem for nonlinear systems with three nested partitions. These nested partitions allow the construction of auxiliary systems that characterize simulation relations, which are suitably exploited to provide upper and lower bounds for a branch and bound algorithm to yield an optimal solution for the control problem. An example illustrates the proposed method.

Keywords: Formal Verification/Synthesis, Hybrid systems, Statistical learning
Abstract: Symbolic control techniques aim to satisfy complex logic specifications. A critical step in these techniques is the construction of a symbolic (discrete) abstraction, a finite-state system whose behaviour mimics that of a given continuous-state system. The methods used to compute symbolic abstractions, however, require knowledge of an accurate closed-form model. To generalize them to systems with unknown dynamics, we present a new data-driven approach that does not require closed-form dynamics, instead relying only the ability to evaluate successors of each state under given inputs. To provide guarantees for the learned abstraction, we use the Probably Approximately Correct (PAC) statistical framework. We first introduce a PAC-style behavioural relationship and an appropriate refinement procedure. We then show how the symbolic abstraction can be constructed to satisfy this new behavioural relationship. Moreover, we provide PAC bounds that dictate the number of data required to guarantee a prescribed level of accuracy and confidence. Finally, we present an illustrative example.

Keywords: Time-varying systems, Control applications, Quantum information and control
Abstract: Rotating wave approximation (RWA) is a well-known technique that has been used to obtain analytical solutions in complex quantum optical models. In this article, the problem of approximating the solution to a single input bilinear system by means of RWA is addressed. This is motivated by the calculation of the response of the nuclei in nuclear magnetic resonance (NMR) spectroscopy to an open-loop controller. The paper presents an analysis on the error resulting from RWA in controlling spin states of the nuclei. We prove the applicability of the rotating wave approximation in the control of spin states by providing a bound on the error of approximation. A simulation of the same has been presented to validate the claim.

Keywords: Control applications, Sampled-data control
Abstract: High magnetic fields significantly improve the resolution and sensitivity of nuclear magnetic resonance (NMR) spectroscopy measurements, which presents exciting research opportunities in areas of chemistry, biology, and material science. Powered magnets can provide much higher magnetic fields than persistent mode superconducting magnets but suffer from temporal magnetic field fluctuations due to power supply ripple and variations in cooling water temperature and flow rate which make powered magnets non-viable for high resolution NMR experiments. Previous work has demonstrated that a multi-rate sampled data cascade control system may be used to improve the resolution of NMR experiments in powered magnets. Despite these advances in reducing temporal magnetic field fluctuations, the field regulation design does not accommodate the use of pulsed field gradients, which are necessary in many NMR experiments. This work presents a control topology which accommodates the use of pulsed field gradient signals with the field regulation system. This control approach is verified using NMR measurements.

Keywords: Autonomous vehicles, Control applications, Variable-structure/sliding-mode control
Abstract: In this paper, a practical fixed-time sliding mode controller is designed for quadrotors. A novel fixed-time stable system is derived, which has a faster convergence rate than existing methods. The proof of the fixed-time convergence is presented. Meanwhile, the faster convergence rate compared with the other two methods is detailed using simulations. Based on this derivation, a fixed-time sliding mode controller with bounded convergence time independent of initial conditions is developed. The comparative simulation and flight experiment results are presented, showing that the proposed control scheme is practical to an actual quadrotor and can achieve good control performance.

Keywords: MEMs and Nano systems, Modeling, Optimal control
Abstract: We address the problem of simultaneous control of micro-objects (particles) immersed in dielectric fluid. An electric field, shaped by an array of thousands of electrodes is used to transport and position particles using dielectrophoretic forces. We use a lumped, 2D, capacitive based (nonlinear) model of motion for the particles behavior that include particle to particle interactions. The particle positions are tracked using a high speed camera and image processing algorithms. We use a model predictive control (MPC) approach to derive control inputs (i.e., electrode potentials) that shape the particles into a desired pattern. To scale the problem with the number of inputs, the control inputs are parameterized as smooth time and space dependent functions avoiding the need to consider each electrode potential as a separate control input. We use automatic differentiation to compute gradients of the particle potential energy, and of the MPC loss and constrain functions. We demonstrate our approach on a scenario where nine particles are transported to a set a final positions while a tenth one is kept stationary.

Keywords: Control applications, Robotics, Evolutionary computing
Abstract: Robust position control is the staple of navigation in mobile robotics. Tricycle robots are nonholonomic ground vehicles with three wheels. While actuated separately, traction and steering are processes coupled through nonlinearities generated by the kinematic model. During operation, environmental changes can cause the model parameters to vary through three unstable behaviors. This study presents a robust control system with modified P-D controllers tuned with real-coded genetic algorithms within a response matching scheme. Results show robustness for 2D trajectories under significant parameter variation, with respect to nonholonomic robot body limitations.

Keywords: Automotive control, Automotive systems
Abstract: This paper describes an approach to the vehicle rollover prevention problem that includes estimation of parameters affecting the roll dynamics and a controller accounting for uncertainties in such parameter estimation. We develop a parameter-adaptive reference governor (PARG) that modifies the driver steering input to enforce a rollover avoidance constraint, and state and input constraints. We design a recursive Bayesian estimator that produces confidence estimates of the parameters, including the center-of-gravity height. The confidence estimates inform a supervised learning algorithm, which constructs online constraint admissible sets that are leveraged by the PARG to ensure rollover prevention. Simulation results on a Fishhook maneuver show that the method robustly prvents rollover, and that the resulting parameter estimates are contained in the confidence sets produced by the Bayesian estimator.

Keywords: Biological systems, Output regulation, Adaptive control
Abstract: We present a control-theoretic model of the optokinetic system, an eye movement system for tracking a moving visual surround. The model adheres to the neural circuit in the brain and is based on the application of adaptive internal models to capture the contribution of the cerebellum. The model is validated through simulations, recovering the basic behaviors of the optokinetic system known from experimental studies.

Keywords: Cellular dynamics, Distributed parameter systems, Stability of nonlinear systems
Abstract: In this work, stabilization of an axonal growth in a neuron associated with the dynamics of tubulin concentration is proposed by designing a boundary control. The dynamics are given by a parabolic Partial Differential Equation (PDE) of the tubulin concentration, with a spatial domain of the axon's length governed by an Ordinary Differential Equation (ODE) coupled with the tubulin concentration in the growth cone. We propose a novel backstepping method for the coupled PDE-ODE dynamics with a moving boundary, and design a control law for the tubulin concentration flux in the soma. Through employing the Lyapunov analysis to a nonlinear target system, we prove a local exponential stability of the closed-loop system under the proposed control law in the spatial H_1-norm.

Keywords: Biological systems, Genetic regulatory systems, Cellular dynamics
Abstract: Inside mammalian cells, single genes are known to be transcribed in stochastic bursts leading to the synthesis of nuclear RNAs that are subsequently exported to the cytoplasm to create mRNAs. We systematically characterize the role of export processes in shaping the extent of random fluctuations (i.e. noise) in the mRNA level of a given gene. Using the method of Partitioning of Poisson arrivals, we derive an exact analytical expression for the noise in mRNA level assuming that the nuclear retention time of each RNA is an independent and identically distributed random variable following an arbitrary distribution. These results confirm recent experimental/theoretical findings that decreasing the nuclear export rate buffers the noise in mRNA level, and counterintuitively, decreasing the noise in the nuclear retention time enhances the noise in the mRNA level. Next, we further generalize the model to consider a dynamic extrinsic disturbance that affects the nuclear-to-cytoplasm export. Our results show that noise in the mRNA level varies non-monotonically with the disturbance timescale. More specifically, high- and low-frequency external disturbances have little impact on the mRNA noise level, while noise is amplified at intermediate frequencies. In summary, our results systematically uncover how the coupling of bursty transcription with nuclear export can both attenuate or amplify noise in mRNA levels depending on the nuclear retention time distribution and the presence of extrinsic fluctuations.

Keywords: Biological systems, Metabolic systems, Adaptive control
Abstract: This paper deals with the problem of robust blood glucose regulation in critically ill patients affected by type 1 diabetes mellitus. The blood glucose measurement and insulin infusion are intravenous. The proposed algorithm regulates blood glucose and keeps it in the normoglycemia range, i.e., 70–180 mg/dl. To this aim, a control law is proposed based on a Nonlinear Model Reference Adaptive Control approach. The algorithm is composed of nonlinear adaptive gains that ensure convergence to zero of the regulation error. The approach is validated in the UVA/Padova metabolic simulator for ten in silico adult patients with unannounced meals. The results perform well and have minimal risk of hyperglycemic and hypoglycemic events.

Keywords: Biological systems, Predictive control for nonlinear systems, Optimal control
Abstract: Microorganisms have evolved submitted to a continuous optimisation process that has improved their capacity to proliferate in nature, developing highly optimized distribution mechanisms of their resources. Considering the microbial self-replication process as a resource allocation problem is a novel approach that has motivated numerous applications to the artificial production of metabolites of interest. Model-based optimal control studies are essential in understanding these naturally-evolved allocation strategies, but they are usually represented by open-loop control laws. In this context, we introduce a hierarchical shrinking-horizon non-linear MPC scheme that aims to maximise the production of a metabolite of interest. The control loop acts on an external signal that is able to disrupt the natural allocation process. The approach uses an optimal control-based input parametrisation that takes into account the structure of the open-loop natural allocation strategy of the cell, to emulate a closed-loop control law. We provide examples of the open-loop control strategies, and simulations of the hierarchical scheme.

Keywords: Optimal control, Biological systems
Abstract: The dynamics of microorganisms in a bioreactor are described by population systems. The tracking of the biomass of a desired trajectory is ensured by means of a nonlinear control law, as long as the reference trajectory complies with certain naturally given inequality constraints. To generate such a reference trajectory, an optimal control problem based on a double integrator with nonlinear state inequality constraints is solved. The necessary optimality criteria are extracted and the differential equations are solved. Using these solutions and parameter assumptions, which are shown to be non-limiting, it is proven that at most one constraint at a time is active. In addition, it is shown that the intervals in which no constraint is active and those in which one constraint is active alternate.

Keywords: Robotics, Autonomous robots, Uncertain systems
Abstract: One of the applications of Extremum Seeking (ES) is to localize the source of a scalar field by using a mobile agent that can measure this field at its current location. While the scientific literature has presented many approaches to this problem, a formal analysis of the behavior of ES controllers for source seeking in the presence of disturbances is still lacking. This paper aims to fill this gap by analyzing a specific version of an ES control algorithm in the presence of source movement and measurement disturbances. We define an approximate version of this controller that captures the main features but allows for a simplified analysis and formally characterize its convergence properties. Through simulations and physical experiments, we compare the theoretically-predicted regions of attraction of the simplified system with the behavior of the full system and show that the simplified version is a good predictor of the behavior of the initial ES controller.

Keywords: Robotics, Nonlinear systems
Abstract: In this paper, we introduce a novel soft robotic system, which is a soft inverted pendulum with a revolute joint at the base. This is an underactuated system as the revolute joint is not actuated. The soft body is hypothesized to be of constant curvature and it is actuated. Motivated by swing up controllers for classical underactuated systems, a switching based swing up and stabilization control of the proposed soft robot system is studied. We demonstrate that the swing up control guides the soft robot to the desired energy level, which is the upright position. Once the swing up is completed, the control is switched to an LQR controller to achieve the stabilization at the vertically upright pose. The switching occurs when the swing up control brings the robot inside the region of attraction for the LQR controller. The simulation results are depicted to illustrate the effectiveness of the proposed control approach for the soft inverted pendulum system.

Keywords: Robotics, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper studies the trajectory tracking problem for an autonomous bicycle that is a non-minimum phase, strongly nonlinear system. As compared with most existing methods dealing only with approximate trajectory tracking, this paper proposes a constructive design to achieve asymptotic trajectory tracking. More specifically, under the assumption that the desired trajectory is generated by a virtual bicycle, a novel asymptotic trajectory tracking controller design scheme is presented. Firstly, the nonlinear dynamics of the autonomous bicycle is established. Secondly, it is decomposed into two interconnected subsystems: a tracking subsystem and a balancing subsystem. For the tracking subsystem, the popular backstepping approach is applied to determine the propulsive force of the bicycle. For the balancing subsystem, optimal control is applied to determine the steering angular velocity of the handlebar in order to balance the bicycle and align the bicycle with the desired yaw angle. Thirdly, to tackle the strong coupling between the tracking and the balancing systems, the small-gain technique is applied for the first time to prove the asymptotic stability of the closed-loop bicycle system. Finally, the efficacy of the proposed exact trajectory tracking control methodology is validated by numerical simulations.

Keywords: Robotics, Robust control, Model/Controller reduction
Abstract: We present a stepping stabilization control that addresses external push disturbances on bipedal walking robots. The stepping control is synthesized based on the step-to-step (S2S) dynamics of the robot that is controlled to have an approximately constant center of mass (COM) height. We first learn a linear S2S dynamics with bounded model discrepancy from the undisturbed walking behaviors of the robot, where the walking step size is taken as the control input to the S2S dynamics. External pushes are then considered as disturbances to the learned S2S (L-S2S) dynamics. We then apply the system-level-synthesis (SLS) approach on the disturbed L-S2S dynamics to robustly stabilize the robot to the desired walking while satisfying the kinematic constraints of the robot. We successfully realize the proposed approach on the walking of the bipedal robot AMBER and Cassie subject to push disturbances, showing that the approach is general, effective, and computationally-efficient for robust disturbance rejection.

Keywords: Robotics, Stability of hybrid systems, Stability of nonlinear systems
Abstract: Control methods for lower-limb powered prostheses remain mostly model-independent and cannot always guarantee stability. Model-dependent prosthesis control methods yield a wider range of stability properties, but require knowledge of the interaction force between the human and prosthesis. Any error in force estimation compromise the formal guarantees. This paper addresses this uncertainty by formalizing the stability of the human-prosthesis system subject to force estimation error. A novel notion of estimate-to-state stability is introduced and provides a means to guarantee exponential convergence of the prosthesis to a set when the controller's model contains estimation error. Conditions are established to ensure input-to-state stability for the human's hybrid periodic orbits when subject to disturbances from the prosthesis control action deviating from its nominal control law. A class of estimate-to-state stable prosthesis controllers is proposed and implemented in simulation, demonstrating how the human-prosthesis system converges to a tube around the desired trajectory resulting in stable walking.

Keywords: Robotics, Supervisory control, Constrained control
Abstract: We are seeking control design paradigms for legged systems that allow bypassing costly algorithms that depend on heavy on-board computers widely used in these systems and yet being able to match what they can do by using less expensive optimization-free frameworks. In this work, we present our preliminary results in modeling and control design of a quadrupedal robot called Husky Carbon, which under development at Northeastern University (NU) in Boston. In our approach, we utilized a supervisory controller and an Explicit Reference Governor (ERG) to enforce ground reaction force constraints. These constrained are usually enforced using costly optimizations. However, in this work, the ERG manipulates the state references applied to the supervisory controller to enforce the ground contact constraints through an update law based on Lyapunov stability arguments. As a result, the approach is much faster to compute than the widely used optimization-based methods.

Keywords: Learning, Mean field games, Statistical learning
Abstract: We consider optimal control of an unknown multi-agent linear quadratic (LQ) system where the dynamics and the cost are coupled across the agents through the mean-field (i.e., empirical mean) of the states and controls. Directly using single-agent LQ learning algorithms in such models results in regret which increases polynomially with the number of agents. We propose a new Thompson sampling based learning algorithm which exploits the structure of the system model and show that the expected Bayesian regret of our proposed algorithm for a system with agents of |M| different types at time horizon T is tilde{mathcal{O}} big( |M|^{1.5} sqrt{T} big) irrespective of the total number of agents, where the tilde{mathcal{O}} notation hides logarithmic factors in T. We present detailed numerical experiments to illustrate the salient features of the proposed algorithm.

Keywords: Observers for nonlinear systems, Optimal control, Optimization algorithms
Abstract: In this paper, we reduce computational burden of moving horizon estimation (MHE) for discrete-time constrained nonlinear systems by providing an iteration scheme that computes a suboptimal state estimate after a limited number of gradient-based optimization algorithm iterations. The optimization algorithm is warm-started by an a priori estimate constructed based on a locally stable, model-based, and recursive state estimation strategy. Due to this implicit stabilizing regularization approach of the a priori estimate, we establish local exponential stability of the underlying estimation error by using Lyapunov arguments. Furthermore, by assuming convexity of the MHE problem, we characterize the performance of the iteration scheme relative to an estimator that knows for instance the optimal solutions in terms of regret upper bounds.

Keywords: Learning, Identification for control
Abstract: In this work, we compare the direct and indirect approaches to data-driven predictive control of stochastic linear time-invariant systems. The distinction between the two approaches lies in the fact that the indirect approach involves identifying a lower dimensional model from data which is then used in a certainty-equivalent control design, while the direct approach avoids this intermediate step altogether. Working within an optimization-based framework, we find that the suboptimality gap measuring the control performance w.r.t. the optimal model-based control design vanishes with the size of the dataset only with the direct approach, while the indirect approach incurs an asymptotic bias. On the other hand, the indirect approach, by relying on the identification of a lower dimensional model, has lower variance and outperforms the direct approach for smaller datasets. Ultimately, by revealing the existence of two non-asymptotic regimes for the performance of direct and indirect data-driven predictive control designs, our study suggests that neither approach is invariably superior and that the choice of design must, in practice, be informed by the available dataset.

Keywords: Predictive control for nonlinear systems, Optimal control, Feedback linearization
Abstract: The widespread adoption of nonlinear Receding Horizon Control (RHC) strategies by industry has led to more than 30 years of intense research efforts to provide stability guarantees for these methods. However, current theoretical guarantees require that each (generally nonconvex) planning problem can be solved to (approximate) global optimality, which is an unrealistic requirement for the derivative-based local optimization methods generally used in practical implementations of RHC. This paper takes the first step towards understanding stability guarantees for nonlinear RHC when the inner planning problem is solved to first-order stationary points, but not necessarily global optima. Special attention is given to feedback linearizable systems, and a mixture of positive and negative results are provided. We establish that, under certain strong conditions, first-order solutions to RHC exponentially stabilize linearizable systems. Crucially, this guarantee requires that state costs applied to the planning problems are in a certain sense `compatible' with the global geometry of the system, and a simple counter-example demonstrates the necessity of this condition. These results highlight the need to rethink the role of global geometry in the context of optimization-based control.

Keywords: Learning, Formal Verification/Synthesis, Optimal control
Abstract: We develop an algorithm for the optimal control of systems governed by unknown, nonlinear dynamics, to deliver tasks expressed as timed temporal logic constraints. The algorithm first computes a sequence of points in the operating environment, along with associated time stamps, so that the system completes its task if it follows the sequence. For the algorithm's second step, we develop a data-driven, on-the-fly control mechanism that learns how to transition from a point in the sequence to the next within a pre-specified time horizon. This algorithm accounts for the unknown dynamics, any unsafe zones in the environment and additional optimality criteria. We show that, after a finite period of data gathering, the resulting controller guarantees that the system indeed follows the sequence of points, leading to the satisfaction of the task.

Keywords: Iterative learning control, Learning, Machine learning
Abstract: We study the problem of online learning and control in partially observable nonlinear dynamical systems, where the model dynamics are unknown and the controlling agent has only access to the system outputs. We propose Model Learning Predictive Control (MLPC), an efficient online control framework that learns to control the unknown system and minimizes the overall control cost. MLPC employs Random Fourier Features (RFF) to represent the nonlinear system dynamics and learns the underlying system up to a confidence interval. Once a reliable estimate of the dynamics is obtained, MLPC deploys an MPC oracle with the estimated system dynamics for planning. MLPC occasionally updates the underlying model estimates and improves the accuracy and the effectiveness of the MPC policies. We derive a novel finite-time approximation error bound under RFF learning and provide stability guarantees for single trajectory online control. We show that MLPC attains tilde{O}(T^{2/3}) regret after T time steps in online control of stable partially observable nonlinear systems against the controller that uses the same MPC oracle with the true system dynamics. We empirically demonstrate the performance of MLPC on the inverted pendulum task and show the flexibility of the proposed general framework via deploying different planning strategies for the controller design to achieve low-cost control policies.

Keywords: Pattern recognition and classification, Machine learning, Learning
Abstract: In this work, we consider a binary classification problem and cast it into a binary hypothesis testing framework, where the observations can be perturbed by an adversary. To improve the adversarial robustness of a classifier, we include an abstain option, where the classifier abstains from making a decision when it has low confidence about the prediction. We propose metrics to quantify the nominal performance of a classifier with an abstain option and its robustness against adversarial perturbations. We show that there exist a tradeoff between the two metrics regardless of what method is used to choose the abstain region. Our results imply that the robustness of a classifier with an abstain option can only be improved at the expense of its nominal performance. Further, we provide necessary conditions to design the abstain region for a 1- dimensional binary classification problem. We validate our theoretical results on the MNIST dataset, where we numerically show that the tradeoff between performance and robustness also exist for the general multi-class classification problems.

Keywords: Machine learning, Optimization, Stochastic optimal control
Abstract: We systematically develop a learning-based treatment of stochastic optimal control (SOC), relying on direct optimization of parametric control policies. We propose a derivation of adjoint sensitivity results for stochastic differential equations through direct application of variational calculus. Then, given an objective function for a predetermined task specifying the desiderata for the controller, we optimize their parameters via iterative gradient descent methods. In doing so, we extend the range of applicability of classical SOC techniques, often requiring strict assumptions on the functional form of system and control. We verify the performance of the proposed approach on a continuous-time, finite horizon portfolio optimization with proportional transaction costs.

Keywords: Machine learning, Optimization algorithms, Large-scale systems
Abstract: We consider the standard federated learning (FL) framework where a set of clients coordinate with a central server to train a statistical model. In a single-machine centralized setting, it is well-known that for smooth and strongly convex finite-sum optimization problems, one can design algorithms that guarantee exact linear convergence to the global minimum without computing full (batch) gradients at every iteration. Despite its popularity, an analog of the above result does not exist in FL. Motivated by this gap, we consider a setting where the local loss function of each client can be expressed as a finite sum of smooth component functions. For this setting, we propose a novel computationally-efficient FL algorithm called FedTrack that rests on two key ideas: (i) using the most recently communicated versions of the clients' gradients in the local update rule, and (ii) incrementally aggregating gradients of the component functions of each client. While the first idea serves to overcome the effect of heterogeneity across the clients' local loss functions, the second helps to significantly reduce the overall number of gradient computations. For both strongly convex and non-convex local loss functions, we prove that the convergence guarantees of FedTrack match their centralized counterparts (up to constants). In particular, for the strongly convex setting, we show that FedTrack guarantees exact linear convergence to the global minimum deterministically.

Keywords: Machine learning, Robust control, Estimation
Abstract: Recurrent Neural networks (RNN) have shown promising potential for learning dynamic features of sequential data. However, artificial neural networks are known to exhibit poor robustness in presence of noisy input, where the sequential architecture of RNNs exacerbates the problem. In this paper, we will use ideas from control and estimation theories to propose a tractable robustness analysis for RNN models that are subject to noisy inputs. The variance of the output of the noisy system is adopted as a robustness measure to quantify the impact of noise on learning. It is shown that the robustness measure can be estimated efficiently using linearization techniques. Using these results, we proposed a learning method to enhance robustness of a RNN with respect to exogenous Gaussian noise with known statistics. Several theoretical upper bounds are also presented to show how the robustness measure depends on the trainable parameters. Our extensive simulations on benchmark problems reveal that our proposed methodology significantly improves robustness of recurrent neural networks in a systematic manner.

Keywords: Machine learning, Optimization, Feedback linearization
Abstract: Many physical system models exhibit a structural property known as differential flatness. Intuitively, differential flatness allows us to separate the system’s nonlinear dynamics into a linear dynamics component and a nonlinear term. In this work, we exploit this structure and propose using a nonparametric Gaussian Process (GP) to learn the unknown nonlinear term. We use this GP in an optimization problem to optimize for an input that is most likely to feedback linearize the system (i.e., cancel this nonlinear term). This optimization is subject to input constraints and a stability filter, described by an uncertain Control Lyapunov Function (CLF), which probabilistically guarantees exponential trajectory tracking when possible. Furthermore, for systems that are control-affine, we choose to express this structure in the selection of the kernel for the GP. By exploiting this selection, we show that the optimization problem is not only convex but can be efficiently solved as a second-order cone program. We compare our approach to related works in simulation and show that we can achieve similar performance at much lower computational cost.

Keywords: Machine learning, Game theory, Variational methods
Abstract: Bayesian regression games are a special class of two-player general-sum Bayesian games in which the learner is partially informed about the adversary's objective through a Bayesian prior. This formulation captures the uncertainty in regard to the adversary, and is useful in problems where the learner and adversary may have conflicting, but not necessarily perfectly antagonistic objectives. Although the Bayesian approach is a more general alternative to the standard minimax formulation, the applications of Bayesian regression games have been limited due to computational difficulties, and the existence and uniqueness of a Bayesian equilibrium are only known for quadratic cost functions. First, we prove the existence and uniqueness of a Bayesian equilibrium for a class of convex and smooth Bayesian games by regarding it as a solution of an infinite-dimensional variational inequality (VI) in Hilbert space. We consider two special cases in which the infinite-dimensional VI reduces to a high-dimensional VI or a nonconvex stochastic optimization, and provide two simple algorithms of solving them with strong convergence guarantees. Numerical results on real datasets demonstrate the promise of this approach.

Keywords: Adaptive control, Optimal control, Nonlinear systems
Abstract: We extend the inverse reinforcement learning (inverse RL) algorithms to multi-player apprentice games described by nonlinear differential equations. In these games, both the expert and the learner have N-player control inputs. Inverse RL algorithms solve the games by learner reconstructing the unknown cost function of each expert player using the demonstration of expert's behavior (states and control inputs of each player), thereby mimicking the given behaviors. We first develop a model-based inverse RL algorithm with two learning stages: an optimal control learning stage and an inverse optimal control learning stage. Then, a model-free off-policy integral inverse RL algorithm is developed by using online expert's demonstrations and learner's behavior trajectories without knowing system dynamics of either expert or the learner. Finally, simulations verify the effectiveness of proposed algorithms.

Keywords: Predictive control for linear systems, Learning
Abstract: We propose an adaptive Model Predictive Safety Certification (MPSC) scheme for learning-based control of linear systems with bounded disturbances and uncertain parameters with known bounds. An MPSC is a modular framework, which can be used in combination with any learning-based controller to ensure state and input constraint satisfaction of a dynamical system by solving an online optimisation problem. By continuously connecting the current system state with a safe terminal set using a robust tube, safety can be ensured. Thereby, the main sources of conservative safety interventions are model uncertainties and short planning horizons. We develop an adaptive mechanism to improve the system model, which leverages set-membership estimation to guarantee recursively feasible and non-decreasing safety performance improvements. In order to accommodate short prediction horizons, iterative safe set enlargements using previously computed robust backup plans are proposed. Finally, we illustrate the increase of the safety performance through the parameter and safe set adaptation for numerical examples with up to 16 state dimensions.

Keywords: Optimal control, Control applications, Variational methods
Abstract: This paper mainly focuses on solving the dynamic optimization of the planar controlled crowd motion models with obstacles which is an application of a class of optimal control problems governed by a general perturbed nonconvex sweeping process. This can be considered as a significant extension of the previous work regarding the controlled crowd motion models, where the obstacles have not been considered. The necessary optimality conditions for the problem under consideration are established and illustrated by a nontrivial example of practical importance.

Keywords: Optimal control, Variational methods, Control applications
Abstract: The paper is devoted to a free-time optimal control problem for sweeping processes. We develop a constructive finite-difference approximation procedure that allows us to establish necessary optimality conditions for discrete optimal solutions and then show how these optimality conditions are applied to solving a controlled marine surface vehicle model.

Keywords: Neural networks, Adaptive control, Lyapunov methods
Abstract: A real-time deep neural network (DNN) adaptive control architecture is developed for uncertain control-affine nonlinear systems to track a time-varying desired trajectory. A Lyapunov-based analysis is used to develop adaptation laws for the output-layer weights and develop constraints for inner-layer weight adaptation laws. Unlike existing works in neural network and DNN-based control, the developed method establishes a framework to simultaneously update the weights of multiple layers for a DNN of arbitrary depth in real-time. The real-time controller and weight update laws enable the system to track a time-varying trajectory while compensating for unknown drift dynamics and parametric DNN uncertainties. A nonsmooth Lyapunov-based analysis is used to guarantee semiglobal asymptotic tracking. Comparative numerical simulation results are included to demonstrate the efficacy of the developed method.

Keywords: Optimal control, Numerical algorithms, Optimization
Abstract: A numerical method and the theory leading to its success are developed in this paper to solve nonstandard optimal control problems involving sweeping processes, in which the sweeping set C is non-convex and coincides with the zerosublevel set of a smooth function having a Lipschitz gradient, and the fixed initial state is allowed to be any point of C. This numerical method was introduced in [7] for a special form of our problem in which the function whose zero-sublevel set defines C is restricted to be twice differentiable and convex, and the initial state is confined in the interior of their convex set C. The remarkable feature of this method is manifested in approximating the sweeping process by a sequence of standard control systems invoking an innovative exponential penalty term in lieu of the normal cone, whose presence in the sweeping process renders most standard methods inapplicable. For a general setting, we prove that the optimal solution of the approximating standard optimal control problem converges uniformly to an optimal solution of the original problem (see Remark 4.1). This numerical method is shown to be efficient through an example in which C is not convex and the initial state is on its boundary.

Keywords: Identification, Network analysis and control, Learning
Abstract: Recently different identification methods have been developed for identifying a single module in a dynamic network. In order to select an appropriate predictor model one typically needs prior knowledge on the topology (interconnection structure) of the dynamic network, as well as on the correlation structure of the process disturbances. In this paper we present a new approach that incorporates the estimation of this prior information into the identification, leading to a fully data-driven approach for estimating the dynamics of a local module. The developed algorithm uses non-causal Wiener filters and a series of convex optimizations with parallel computation capabilities to estimate the topology, which subsequently is used to build the appropriate input/output setting for a predictor model in the local direct method under correlated process noise. A regularized kernel-based method is then employed to estimate the dynamic of the target module. This leads to an identification algorithm with attractive statistical properties that is scalable to handle larger-scale networks too. Numerical simulations illustrate the potential of the developed algorithm.

Keywords: Identification, Optimization
Abstract: Quantization plays an important role as an interface between analog and digital environments. Since quantization is a many to few mapping, it is a non-linear irreversible process. This made, in addition of the quantization noise signal dependency, the traditional methods of system identification no longer applicable. In this work, we propose a method for parsimonious system identification when only quantized measurements of the output are observable. More precisely, we develop an algorithm that aims at identifying a low order system that is compatible with a priori information on the system and the collected quantized output information. Moreover, the proposed approach can be used even if only fragmented information on the quantized output is available. The proposed algorithm relies on an ADMM approach to l_p quasi-norm optimization. Numerical results highlight the performance of the proposed approach when compared to the l_1 minimization in terms of the sparsity of the induced solution.

Keywords: Identification, Stochastic systems, Optimization
Abstract: The paper presents a systematic approach for finding the optimal set of predictors for consistent identification of a single transfer function in an acyclic dynamic network. It is assumed that the topology of the network is known, the forcing inputs are not measured, and the observations have positive additive costs. For a class of networks where the target node is not involved in a feedback loop, sufficient and necessary conditions are derived to consistently identify a certain transfer function via a multi-input single-output prediction error method. This enables designing a systematic graphical approach based on the notion of d-separation to look for an optimal set of predictors that minimizes an appropriate additive cost function. It is shown that the required conditions for consistency and optimality are equivalent to the notion of separation in an undirected graph resulted from systematically manipulating the graphical representation of the network. Then, some well-known algorithms from computer science can be used to find the optimal set of predictors.

Keywords: Identification, Switched systems, Randomized algorithms
Abstract: The identification of switched nonlinear systems involves solving a combinatorial problem that simultaneously addresses the sample-mode assignment and nonlinear model structure selection tasks. The complexity of this problem is often prohibitive, since mode switchings can take place at arbitrary times. To reduce it to an affordable level, one can constrain the mode switchings to occur only at few specific instants. This approach is effective if combined with a refinement strategy, aiming at correcting the number and location of switchings. In this paper, one such strategy is discussed, which employs a local optimization process to correct the position of switchings, and is also capable of detecting and removing redundant modes. An iterative method, applying both an identification step and a refinement step at all iterations, is tested on several numerical benchmarks to illustrate the effectiveness of the refinement strategy. The method does not require prior assumptions on the number of modes.

Keywords: Identification, Time-varying systems, Estimation
Abstract: The problem of identification of a time-varying FIR system is considered and solved using the local basis function approach. It is shown that the estimation (tracking) results can be improved by means of regularization. Two variants of regularization are proposed and compared: the classical L2 (ridge) regularization and a new, reweighted L2 one. It is shown that the new approach outperforms the classical one and is computationally attractive.

Keywords: Identification, Uncertain systems, Optimization
Abstract: This paper proposes a framework for the quantification of structured uncertainty in a plant model according to multivariable input-output data. The only restriction imposed upon such a model is for its outputs to depend continuously on the parameters. An Interval Predictor Model (IPM) prescribes the parameters of a computational model as a bounded, path-connected set thereby making each predicted output an interval-valued function of the inputs. The formulation proposed seeks the parameter set leading to the tightest enclosure of the data. This set, which is modeled as a semi-algebraic set having a tunable complexity level, enables the characterization of parameter dependencies commonly found in practice. This representation of the uncertain parameters makes the resulting plant model amenable to robustness analysis and robust control techniques based on polynomial optimization. Furthermore, scenario theory is used to evaluate the generalization properties of the identified model. This evaluation yields a formally-verifiable, distribution-free upper bound on the probability of future data falling outside the predicted output intervals.

Keywords: Stochastic optimal control, Stability of nonlinear systems, Stochastic systems
Abstract: In this work, we define and establish a selection of significant and new properties for nonlinear stochastic model predictive control (SMPC). First, we ensure that the underlying stochastic properties of the closed-loop stochastic system are indeed well-defined. We then define robust exponential stability in expectation (RESiE) and establish that nonlinear SMPC, under suitable assumptions, renders the origin of the closed-loop system RESiE. We conclude with a numerical example to demonstrate the implications of this analysis.

Keywords: Stochastic optimal control, Autonomous vehicles
Abstract: We present the design of a motion planning algorithm that ensures safety for an autonomous vehicle. In particular, we consider a multimodal distribution over uncertainties; for example, the uncertain predictions of future trajectories of surrounding vehicles reflect discrete decisions, such as turning or going straight at intersections. We develop a computationally efficient, scenario-based approach that solves the motion planning problem with high confidence given a quantifiable number of samples from the multimodal distribution. Our approach is based on two preprocessing steps, which 1) separate the samples into distinct clusters and 2) compute a bounding polytope for each cluster. Then, we rewrite the motion planning problem approximately as a mixed-integer problem using the polytopes. We demonstrate via simulation on the nuScenes dataset that our approach ensures safety with high probability in the presence of multimodal uncertainties, and is computationally more efficient and less conservative than a conventional scenario approach.

Keywords: Stochastic optimal control, Computational methods, Nonlinear systems
Abstract: Novel numerical estimators are proposed for the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function. In contrast to the current numerical method approaches based on discretization of the continuous-time FBSDE results, we propose a converse approach, by first obtaining a discrete-time approximation of the on-policy value function, and then developing a discrete-time result which resembles the continuous-time counterpart. This approach yields improved numerical estimators in the function approximation phase, and demonstrates enhanced error analysis for those value function estimators. Numerical results and error analysis are demonstrated on a scalar nonlinear stochastic optimal control problem, and they show improvements in the performance of the proposed estimators in comparison with the state-of-the-art methodologies.

Keywords: Stochastic optimal control, Constrained control, Optimization algorithms
Abstract: In this paper, we propose a convex optimization approach to chance-constrained drift counteraction optimal control (DCOC) problems for linear systems with additive stochastic disturbances. Chance-constrained DCOC aims to compute an optimal control law to maximize the time duration before the probability of violating a prescribed set of constraints can no longer be maintained to be below a specified risk level. While conventional approaches to this problem involve solving a mixed-integer programming problem, we show that an optimal solution to the problem can also be found by solving a convex second-order cone programming problem without integer variables. We illustrate the application of chance-constrained DCOC to an automotive adaptive cruise control example.

Keywords: Stochastic optimal control, Machine learning, Autonomous systems
Abstract: Kernel embeddings of distributions have recently gained significant attention in the machine learning community as a data-driven technique for representing probability distributions. Broadly, these techniques enable efficient computation of expectations by representing integral operators as elements in a reproducing kernel Hilbert space. We apply these techniques to the area of stochastic optimal control theory and present a method to compute approximately optimal policies for stochastic systems with arbitrary disturbances. Our approach reduces the optimization problem to a linear program, which can easily be solved via the Lagrangian dual, without resorting to gradient-based optimization algorithms. We focus on discrete-time dynamic programming, and demonstrate our proposed approach on a linear regulation problem, and on a nonlinear target tracking problem. This approach is broadly applicable to a wide variety of optimal control problems, and provides a means of working with stochastic systems in a data-driven setting.

Keywords: Stochastic optimal control, Nonlinear systems, Learning
Abstract: We propose a numerical method for the computation of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. By the use of the Girsanov change of probability measures, it is demonstrated how a rapidly-exploring random tree (RRT) can be utilized for the forward integration pass, as long as the controlled drift term is appropriately compensated in the backward integration pass. A numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of the constructed RRT. Moreover, a local entropy-weighted least squares Monte Carlo (LSMC) method is developed to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories.

Keywords: Mean field games, Optimal control, Hierarchical control
Abstract: We present a Stackelberg game with a large number of followers where every player--leader and followers--has its own state and control. We derive the mean field limit of infinitely many followers and address aspects of consistent control. Finally, we propose a numerical method based on the derived model and present numerical results.

Keywords: Game theory, Learning, Distributed parameter systems
Abstract: We consider a distributed learning problem in a two-network zero-sum game with finite action sets, where the agents within each network is connected through time-varying directed graphs and the agents from distinct networks are connected by time-varying bipartite graphs. Each agent in a network has its own cost function and can receive information from its neighbors, while the networks have no global decision-making capability. We propose a distributed multiplicative weights algorithm to compute a mixed-strategy Nash equilibrium. We first establish a sublinear regret bound on the sequence of iterates for each agent. Additionally, we study the time-averaged sequence of iterates and prove its convergence to the set of mixed-strategy Nash equilibria with suitably selected diminishing step-sizes.

Keywords: Game theory, Learning, Optimization algorithms
Abstract: In this paper, we examine the equilibrium tracking and convergence properties of no-regret learning algorithms in continuous games that evolve over time. Specifically, we focus on learning via "mirror descent", a widely used class of no- regret learning schemes where players take small steps along their individual payoff gradients and then "mirror" the output back to their action sets. In this general context, we show that the induced sequence of play stays asymptotically close to the evolving equilibrium of the sequence of stage games (assuming they are strongly monotone), and converges to it if the game stabilizes to a strictly monotone limit. Our results apply to both gradient- and payoff-based feedback, i.e., the "bandit" case where players only observe the payoffs of their chosen actions.

Keywords: Game theory, Optimization algorithms, Randomized algorithms
Abstract: This paper considers no-regret learning for repeated continuous-kernel games with lossy bandit information. At each round, each player chooses an action perturbed around its intended action, and gets the utility value at the corresponding action profile. However, due to various uncertainties or high inquiring costs, the bandit feedback may be lost at random. Therefore, we focus on studying the asynchronous learning strategy of the players to adaptively adjust next actions for minimizing the long-term regret loss compared with a best-fixed action in the hindsight. The paper provides a novel no-regret learning algorithm, called Reweighted Online Gradient Descent with bandit (ROGD-b). We first give the regret analysis for continuous concave games with differentiable and Lipschitz utilities. Furthermore, we show that the action profile converges to Nash equilibrium with probability 1 when the game is strictly monotone. Numerical experiments are given to illustrate the performance of the algorithm.

Keywords: Game theory, Control of networks, Adaptive control
Abstract: We consider the problem of steering the actions of noncooperative players in quadratic network games to the social optimum. To this end, a central regulator modifies the marginal returns of the players, while the players’ strategies are determined by continuous pseudo-gradient dynamics. Depending on the available information on the players parameters and network quantities, suitable static and dynamic intervention protocols are devised that maximize the social welfare at steady-state. We show that adaptive interventions can compensate for the lack of knowledge on network topology and coupling weights. Numerical examples are provided to demonstrate the effectiveness of the proposed interventions.

Keywords: Game theory, Learning, Agents-based systems
Abstract: We propose a second-order extension of the continuous-time game-theoretic mirror descent (MD) dynamics, which we refer to as MD2, and show that MD2 can converge to merely (not necessarily strictly) variationally stable Nash equilibria while enjoying no-regret. MD2 also overcomes the inexact convergence problem of discounted MD. We then derive the rate of convergence of an augmented MD2 whenever the equilibrium is strongly variationally stable. Selected simulations are provided to illustrate and affirm our results.

Keywords: Optimization, Optimization algorithms, Machine learning
Abstract: We consider structured online convex optimization (OCO) with bandit feedback, where either the loss function is smooth or the constraint set is strongly convex. Projection-free methods are among the most popular and computationally efficient algorithms for solving this problem, mainly due to their ability to handle convex constraints appearing in machine learning for which computing projections is often impractical in high-dimensional settings. Despite the improved regret bound results for the full-information setting where the gradients of the functions are readily available, it remains unclear whether simple projection-free zero-order algorithms become more efficient for structured OCO problems in the case when multiple function values can be sampled at each time instance. In this paper, we develop some simple projection-free algorithms and prove that they indeed achieve the same improved regret bounds as the full-information case under various additional problem structures. This implies that leveraging the structural properties of the problem compensates for the lack of access to the gradients. Experiments on the online matrix completion reveal several attractive advantages of the proposed algorithms, including their simplicity, easy implementation, and effectiveness, as they outperform other competing algorithms.

Keywords: Optimization, Optimization algorithms, Machine learning
Abstract: In this paper, a centralized two-block separable convex optimization with equality constraint is considered. The first fully parallel primal-dual discrete-time algorithm called Parallel Alternating Direction Primal-Dual (PADPD) is proposed. In the algorithm, the primal variables are updated in an alternating fashion like Alternating Direction Method of Multipliers (ADMM). The algorithm can handle non-smoothness of objective functions with strong convergence. Unlike existing discrete-time algorithms such as Method of Multipliers (MM), ADMM, Bi- Alternating Direction Method of Multipliers (BiADMM), and Primal-Dual Fixed Point (PDFP) algorithms, all primal and dual variables in the proposed algorithm are updated independent of each other. The algorithm can be directly extended to any finite multi-block optimization without further assumptions while preserving its convergence. It is shown that the rate of convergence of the algorithm for Quadratic/Linear cost functions is exponential or linear under suitable assumptions. Finally, a numerical example is given to show that PADPD not only can compute more iterations (since it is fully parallel) for the same time-step but also can have faster convergence rate than that of ADMM.

Keywords: Optimization, Optimization algorithms, Predictive control for linear systems
Abstract: Convex Quadratic Programs (QPs) have come to play a central role in the computation of control action for constrained dynamical systems. In this paper, we present a novel Homogeneous QP (HQP) formulation which is obtained by embedding the original QP in a larger space. The key properties of the HQP are: (i) is always feasible, (ii) an optimal solution to QP can be readily obtained from a solution to HQP, and (iii) infeasibility of QP corresponds to a particular solution of HQP. An immediate consequence is that all the existing algorithms for QP are now also capable of robustly detecting infeasibility. In particular, we present an Infeasible Interior Point Method (IIPM) for the HQP and show polynomial iteration complexity when applied to HQP. A key distinction with prior IPM approaches is that we do not need to solve second-order cone programs. Numerical experiments on the formulation are provided using existing codes.

Keywords: Optimization, Optimization algorithms
Abstract: Large-scale optimization problems are usually characterized not only by large amounts of data points but points living in a high-dimensional space. Block coordinate methods allow for efficient implementations where steps can be made (block) coordinate-wise. Many existing algorithms rely on trustworthy gradient information and may fail to converge when such information becomes corrupted by possibly adversarial agents. We study the setting where the partial gradient with respect to each coordinate block is arbitrarily corrupted with some probability. We analyze the robustness properties of the normalized random block coordinate method (NRBCM) for non-convex optimization problems. We prove that NRBCM finds an {cal O}(1/sqrt{T})-stationary point after T iterations if the corruption probabilities of partial gradients with respect to each block are below 1/2. With the additional assumption of gradient domination, faster rates are shown. Numerical evidence on a logistic classification problem supports our results.

Keywords: Optimization, Optimization algorithms
Abstract: We investigate the problem of finding inner ap-proximations of positive semidefinite (PSD) cones. We developa novel decomposition framework of the PSD cone by meansof conical combinations of smaller dimensional sub-cones. Weshow that many inner approximation techniques could besummarized within this framework, including the set of (scaled)diagonally dominant matrices, Factor-widthkmatrices, andChordal Sparse matrices. Furthermore, we provide a moreflexible family of inner approximations of the PSD cone, wherewe aim to arrange the sub-cones so that they are maximallyseparated from each other. In doing so, these approximationstend to occupy large fractions of the volume of the PSD cone.The proposed approach is connected to a classical packingproblem in Riemannian Geometry. Precisely, we show thatthe problem of finding maximally distant sub-cones in anambient PSD cone is equivalent to the problem of packingsub-spaces in a Grassmannian Manifold. We further leverageexisting computational method for constructing packings inGrassmannian manifolds to build tighter approximations ofthe PSD cone. Numerical experiments show how the proposedframework can balance between accuracy and computationalcomplexity, to efficiently solve positive-semidefinite programs.

Keywords: Optimization algorithms, Adaptive control, Lyapunov methods
Abstract: Iterative gradient-based algorithms have been increasingly applied for the training of a broad variety of machine learning models including large neural-nets. In particular, momentum-based methods, with accelerated learning guarantees, have received a lot of attention due to their provable guarantees of fast learning in certain classes of problems and multiple algorithms have been derived. However, properties for these methods hold only for constant regressors. When time-varying regressors occur, which is commonplace in dynamic systems, many of these momentum-based methods cannot guarantee stability. Recently, a new High-order Tuner (HT) was developed and shown to have 1) stability and asymptotic convergence for time-varying regressors and 2) non-asymptotic accelerated learning guarantees for constant regressors. These results were derived for a linear regression framework which leads to a quadratic loss function. In this paper, we extend and discuss the results of this same HT for general convex loss functions. Through the exploitation of convexity and smoothness definitions, we establish similar stability and asymptotic convergence guarantees. Finally, we provide numerical simulations supporting the satisfactory behavior of the HT algorithm as well as an accelerated learning property.

Keywords: Predictive control for nonlinear systems, Optimization algorithms, Hybrid systems
Abstract: Nonlinear model predictive control (NMPC) has grown mature and algorithmic techniques exist, e.g., based on sequential quadratic programming (SQP) methods, to handle relatively complex constrained control systems. In addition, model predictive control for hybrid dynamical systems, including both continuous and discrete decision variables, can be implemented efficiently based on state of the art mixed-integer quadratic programming (MIQP) algorithms. This paper proposes a novel mixed-integer SQP (MISQP) optimization algorithm as a heuristic search technique to find feasible, but possibly suboptimal, solutions for real-time implementations of mixed-integer NMPC (MINMPC). Two variants of the MISQP algorithm are described and motivated. Based on a preliminary software implementation, the real-time MISQP performance is illustrated for closed-loop MINMPC simulations on a nontrivial vehicle control case study, featuring worst-case computation times below 30 milliseconds.

Keywords: Predictive control for nonlinear systems, Optimization algorithms, Numerical algorithms
Abstract: In this paper, we present an iterative Model Predictive Control (MPC) design for piecewise nonlinear systems. We consider finite time control tasks where the goal of the controller is to steer the system from a starting configuration to a goal state while minimizing a cost function. First, we present an algorithm that leverages a feasible trajectory that completes the task to construct a control policy which guarantees that state and input constraints are recursively satisfied and that the closed-loop system reaches the goal state in finite time. Utilizing this construction, we present a policy iteration scheme that iteratively generates safe trajectories which have non-decreasing performance. Finally, we test the proposed strategy on a discretized Spring Loaded Inverted Pendulum (SLIP) model with massless legs. We show that our methodology is robust to changes in initial conditions and disturbances acting on the system. Furthermore, we demonstrate the effectiveness of our policy iteration algorithm in a minimum time control task.

Keywords: Predictive control for nonlinear systems, Optimization, Stochastic optimal control
Abstract: Classical source seeking algorithms aim to make the robot reach the source location eventually. This paper proposes a process-aware source seeking approach which finds an informative trajectory to reach the source location. A multi-objective optimization problem is formulated based on rewards for both the search process and the terminal condition. Due to the unknown source location, solutions are found through Bayesian learning model predictive control (BLMPC). The consistency of the Bayesian estimator, as well as the convergence of the proposed algorithm are proved. The performance of the algorithm is evaluated through simulation results. The process-aware source seeking algorithm demonstrates improvements over other classical source seeking algorithms.

Keywords: Predictive control for nonlinear systems, Robotics, Optimal control
Abstract: The dynamics of many systems are most naturally expressed in terms of a state-dependent mass matrix, the most prominent example being rigid body dynamics. However, model predictive control solvers often do not support this structure natively and instead require a reformulation of the system. This paper presents a concise and self-contained derivation of the optimality conditions for this system class and describes the implementation of an efficient solver based on a tailored gradient method. The approach is evaluated using the dynamical model of a robot arm with seven degrees of freedom, whereby it is shown that computation times below one millisecond can be achieved. Furthermore, by exploiting the structure of the optimality conditions with state-dependent mass matrix, the computation times can be reduced significantly.

Keywords: Robotics, Predictive control for nonlinear systems, Control applications
Abstract: The problem of grasping in robotics requires the realization of highly constrained motion and interaction. In this paper, a method is presented in which grasping-based constraints are explicitly taken into account on control level using model predictive control. The approach is an extension of the recently proposed model predictive interaction control (MPIC) for a single-arm robot. The basic idea is to consider the force behavior within the optimization problem, whereby a linear-elastic model is assumed in this paper. The method is evaluated in simulation as well as on an anthropomorphic hand.

Keywords: Robotics, Predictive control for nonlinear systems, Optimal control
Abstract: In this paper, we propose a novel framework on force-and-moment-based Model Predictive Control (MPC) for dynamic legged robots. Specifically, we present a formulation of MPC designed for 10 degree-of-freedom (DoF) bipedal robots using simplified rigid body dynamics with input forces and moments. This MPC controller will calculate the optimal inputs applied to the robot, including 3-D forces and 2-D moments at each foot. These desired inputs will then be generated by mapping these forces and moments to motor torques of 5 actuators on each leg. We evaluate our proposed control design on physical simulation of a 10 degree-of-freedom (DoF) bipedal robot. The robot can achieve fast walking speed up to 1.6 m/s on rough terrain, with accurate velocity tracking. With the same control framework, our proposed approach can achieve a wide range of dynamic motions including walking, hopping, and running using the same set of control parameters.

Keywords: Discrete event systems, Distributed control, Cooperative control
Abstract: This paper considers event-triggered output synchronization of heterogeneous multi-agent systems over directed switching networks. We first propose an internal model for each follower using the event-triggered scheme to track the state of the leader using intermittent communications while avoiding Zeno behavior. The internal model is then embedded in the distributed controller to achieve output synchronization of the system of heterogeneous agents. We prove that the output synchronization errors converge exponentially to zero, and also give a bound on the convergence rate. A numerical example is presented to illustrate the theoretical results.

Keywords: Discrete event systems, Petri nets
Abstract: P-time event graphs (P-TEGs) are specific timed discrete-event systems, in which the timing of events is constrained by intervals. An important problem is to check, for all natural numbers d, the existence of consistent d-periodic trajectories for a given P-TEG. In graph theory, the Proportional-Inverse-Constant-Non-positive Circuit weight Problem (PIC-NCP) consists in finding all the values of a parameter such that a particular parametric weighted directed graph does not contain circuits with positive weight. In a related paper, we have proposed a strongly polynomial algorithm that solves the PIC-NCP in lower worst-case complexity compared to other algorithms reported in literature. In the present paper, we show that the first problem can be formulated as an instance of the second; consequently, we prove that the same algorithm can be used to find d-periodic trajectories in P-TEGs. Moreover, exploiting the connection between the PIC-NCP and max-plus algebra we prove that, given a P-TEG, the existence of a consistent 1-periodic trajectory of a certain period is a necessary and sufficient condition for the existence of a consistent d-periodic trajectory of the same period, for any value of d.

Keywords: Discrete event systems
Abstract: In this letter control design of nondeterministic finite state systems with reachability specifications is addressed. The class of controllers we use is rather general and combines feedforward and output feedback schemes. The proposed controller allows not only the state of the system to reach the desired target set but also the identification of which state of the target set has been reached. Necessary and sufficient conditions are derived for the control problem to admit a solution and a controller is designed. The solution to the investigated problem has important implications in the context of controlled-observability, recovery control and symbolic control design of nonlinear and hybrid systems, as illustrated also through some examples.

Keywords: Discrete event systems, Automata, Estimation
Abstract: The state estimation problem in discrete-event systems (DESs) consists in determining the set of states consistent with the observations. Due to sensor malfunction, the occurrence of some events may become unobservable. Moreover, such loss of observations may be intermittent or permanent. In this paper, we study the state estimation problem of discrete-event systems subject to both intermittent and permanent loss of observations. We assume that both types of observation loss are not recoverable. For each event subject to a permanent loss of observation, we propose an automaton model to describe this kind of malfunction. We show that the automaton model describing the behavior of the system in the presence of observation loss can be obtained through the parallel composition of the above automata and an automaton that is obtained with appropriate changes to keep into account observation loss. Finally, the state estimation problem can be solved by constructing the observer of the new model.

Keywords: Discrete event systems, Automata, Supervisory control
Abstract: This work investigates stabilization of discrete event systems via supervisory control under a series of quantitative constraints. Every event in the system model is weighted by a vector which represents resource payoffs associated with the event. Multidimensional weight flows are generated when events occur successively. The supervisor aims to drive the generated strings to a reach set of target states to stabilize the system. Meanwhile, the supervisor is also responsible for regulating the weight flows so as to guarantee that at each dimension, the average weight every a certain number of events does not fall below a given threshold. Next, the formulated supervisory control problem is transformed to a two-player game between the supervisor and the environment on a specially defined game structure. Then we specify the objective of the game and synthesize game winning supervisors, which turn out to provably solve the proposed problem.

Keywords: Discrete event systems, Supervisory control, Automata
Abstract: A visualization to aid in the construction of inference-/based decentralized supervisors is presented. In the inference-/based architecture, supervisors have different levels of ambiguity, which reflects to what degree a supervisor is confident in its control decision and to what degree a supervisor infers a control decision based on the supervisor’s knowledge of another supervisor’s control decision.

Keywords: Adaptive control, Robust adaptive control
Abstract: It is typically proven in adaptive control that asymptotic stabilization and tracking holds, and that at best a bounded-noise bounded-state property is proven. Recently, it has been shown in both the pole-placement control and the d-step ahead control settings that if, as part of the adaptive controller, a parameter estimator based on the original projection algorithm is used and the parameter estimates are restricted to a convex set, then the closed-loop system experiences linear-like behavior: exponential stability, a bounded gain on the noise in every p-norm, and a convolution bound on the exogenous inputs; this can be leveraged to provide tolerance to unmodelled dynamics and plant parameter time-variation. In this paper, we extend the approach to the more general Model Reference Adaptive Control (MRAC) problem and demonstrate that we achieve the same desirable linear-like closed-loop properties.

Keywords: Adaptive control, Stochastic systems, Neural networks
Abstract: In this paper, we present a Stochastic Deep Neural Network-based Model Reference Adaptive Control (S-DMRAC). Building on our work ``Deep Model Reference Adaptive Control(DMRAC)", we extend the controller capability by using Bayesian deep neural networks (DNN) to representations for modeling non-linearities. A Bayesian approach to DNN learning helped avoid over-fitting the data and provide confidence intervals over the predictions. A Lyapunov-based method is used to adapt the output-layer weights of the DNN model in real-time, while a data-driven supervised learning algorithm is used to update the inner-layer weights, ensuring boundedness and guaranteed tracking performance with a learning-based real-time feedback controller. We also show that the controller's stochastic nature ensured "Induced Persistency of excitation", leading to convergence of the overall system signal.

Keywords: Adaptive control, Time-varying systems, Nonlinear systems
Abstract: This paper proposes an identification-based adaptive control scheme for nonlinear systems with time-varying parameters designed on the basis of the so-called congelation of variables method. First a scalar example to demonstrate the design methodology, which relies on re-arranging the identifier subsystems from a cascaded topology to a cyclic topology, is discussed. A small-gain-like control synthesis exploiting the cyclic topology is then presented to replace the classical control synthesis based on the swapping lemma, which exploits the cascaded topology. Then a state feedback design for a class of lower triangular nonlinear systems is presented: this combines the same design methodology with the backstepping techniques. Boundedness of all closed-loop signals and convergence of the system state are proved. Finally, simulation results showing that the proposed controller achieves superior performance than the classical design are presented.

Keywords: Adaptive control, Autonomous vehicles, Decentralized control
Abstract: Despite the progress in the field of longitudinal formations of automated vehicles, only recently an interpretation of longitudinal platooning has been given in the framework of disturbance decoupling, i.e. the problem of making a controlled output independent of a disturbance. The appealing feature of this interpretation is that the disturbance decoupling approach naturally yields a decentralized controller that guarantees stability and string stability. In this work, we further exploit the disturbance decoupling framework and we show that convergence to a stable, string stable and disturbance decoupled behavior can be achieved even in the presence of parametric uncertainty of the engine time constant. We refer to this framework as adaptive disturbance decoupling.

Keywords: Adaptive systems, Nonlinear systems, Autonomous systems
Abstract: This paper offers new analytical conditions on the system parameters of a particular class of planar dynamical systems which would allow them to undergo a Hopf bifurcation. These systems are constructed as a means of generating multiple behaviors from the same single continuous dynamical system model, without resorting to switching between distinct component continuous dynamics associated to each behavioral mode. This work builds on recent advances which introduced motivation dynamics as an efficient way to design multi-behavioral systems. The contribution of this paper is that it expands the scope of the motivational dynamics approach, and offers explicit analytic conditions on the system parameters to guarantee the existence of bifurcations, which can then be utilized to better engineer the structure and location of the resulting equilibria. Numerical simulations confirm the theoretical predictions for the onset of the Hopf bifurcations.

Keywords: Adaptive control, Neural networks, Nonlinear output feedback
Abstract: This paper is concerned with the problem of globally stable adaptive neural network tracking control for a class of output feedback systems with unknown functions. Unknown functions are approximated via online radial basis function neural network, continuously differentiable functions are introduced into Lyapunov functions to realize parameter estimation. Barbalat's lemma is used to prove that all closed-loop signals are globally uniformity ultimately bounded and tracking error can reach prior accuracy. A simulation example is given to verify the effectiveness of the control method.

Keywords: Fault tolerant systems, Optimal control, Linear systems
Abstract: Resilient systems are able to recover quickly and easily from disturbed system states that might result from hazardous events or malicious attacks. In this paper a novel resilience metric for linear time invariant systems is proposed: the minimum control energy required to disturb the system is set into relation to the minimum control energy needed to recover. This definition extends known disturbance rejection metrics considering random effects to account for adversarial disturbances. The worst-case disturbance and the related resilience index can be computed efficiently via solving a generalized eigenvalue problem that depends on the controllability Gramians of the control and disturbance inputs. The novel metric allows improving system resilience by optimizing the restorative control structure or by hardening the system against specific attack options. The new approach is demonstrated for a coupled mechanical system.

Keywords: Formal Verification/Synthesis, Linear systems
Abstract: In this paper, we derive closed-form expressions for implicit controlled invariant sets for discrete-time controllable linear systems with measurable disturbances. In particular, a disturbance-reactive (or disturbance feedback) controller in the form of a parametrized finite automaton is considered. We show that, for a class of automata, the robust positively invariant sets of the corresponding closed-loop systems can be expressed by a set of linear inequality constraints in the joint space of system states and controller parameters. This leads to an implicit representation of the invariant set in a lifted space. We further show how the same parameterization can be used to compute invariant sets when the disturbance is not available for measurement.

Keywords: Learning, Linear systems, Optimal control
Abstract: Policy gradient algorithms in model-free reinforcement learning have been shown to achieve global exponential convergence for the Linear Quadratic Regulator problem despite the lack of convexity. However, extending such guarantees beyond the scope of standard LQR and full-state feedback has remained open. A key enabler for existing results on LQR is the so-called gradient dominance property of the underlying optimization problem that can be used as a surrogate for strong convexity. In this paper, we take a step further by studying the convergence of gradient descent for the Linear Quadratic Gaussian problem and demonstrate through examples that LQG does not satisfy the gradient dominance property. Our study shows the non-uniqueness of equilibrium points and thus disproves the global convergence of policy gradient methods for LQG.

Keywords: Linear systems, Stability of linear systems, Modeling
Abstract: Realization theory as developed hitherto, deals with the following question: How can an arbitrary transfer function be constructed using standard building blocks. The answer is well known to control engineers, is useful, and consists of integrators, multipliers and summers, because the latter components are standard building blocks and can be mass produced, as integrated circuits.If the transfer function to be realized is improper, differentiators would be required as additional building blocks. Since differentiators amplify high frequency noise they are generally avoided and control theory avoids building improper transfer functions. In general integrators are also susceptible to low frequency noise and may be unsuitable in environments where low frequency noise is present. This suggests that it may be interesting to consider as a standard building block, a system which does not amplify low or high frequencies. It turns out that a first order filter does precisely that, namely the high and low frequency gains are constant, independent of frequency. In the following note, we ask the question: Can an arbitrary transfer function be constructed using first order filters, summers, and multipliers? We show that the answer is ``yes'' regardless of whether the transfer function is proper or improper, and show how such a realization may be constructed for arbitrary linear systems, proper or improper. Indeed we show that almost any first order filter can be used as a building block. We also show through simulations that the first order implementation of a differentiator has superior noise rejection compared to a pure differentiator.

Keywords: Differential-algebraic systems, Observers for Linear systems
Abstract: This paper studies the design of functional observers for linear time-invariant descriptor systems which are not necessarily square and regular. A novel characterization for the existence of functional observers is presented. The observer is realized by a system of ordinary differential equations (ODEs). The existence of ODE observers is proved by means of a set of simple rank conditions on the system coefficient matrices. The proposed conditions are less restrictive than the existing ones. The derivation is purely algebraic, and the algorithm is illustrated by designing an observer-based controller.

Keywords: PID control, Linear systems, Adaptive control
Abstract: With reference to model-based (auto)tuning, we discuss the somehow overlooked role of the procedure used to parametrise the chosen process model structure. We evidence the detrimental effect of neglecting that procedure, particularly when evaluating/comparing tuning rules, thus when setting up a tuning procedure for a given application. We finally formulate a proposal for choosing the best model parametrisation procedure, in a given set, based on the information available in the tuning phase, and on the selected control quality indicator(s).

Keywords: Distributed parameter systems, Delay systems, Observers for Linear systems
Abstract: An asymptotic observer for a linear system, evolving in a Hilbert space, is designed over linear state measurements with time-varying delays. The proposed predictor-based approach reduces the problem to the standard one with non-delayed information on the state, thereby being invariant to the dimensionality of the underlying system. Capabilities of the resulting observer design are illustrated for the linearized Kuramoto-Sivashinsky PDE with periodic boundary conditions and with delayed finite-dimensional measurements.

Keywords: Distributed parameter systems, Observers for nonlinear systems, Lyapunov methods
Abstract: This paper deals with the observer design for a nonlinear 1D heat equation based on a single in-domain measurement motivated by rapid thermal silicon wafer pro- duction processes. A pointwise measurement injection observer is proposed that takes into account the basic nonlinear heat conduction and radiation mechanisms. Exponential convergence of the estimation error under mild and practically reasonable assumptions is formally proven using Lyapunov techniques. The proposed observer ensures a good trade-off between convergence speed and implementation effort. Its effectivenessis demonstrated using numerical simulations.

Keywords: Distributed parameter systems, Observers for Linear systems
Abstract: This paper combines two control design aspects for a class of infinite dimensional systems, and each of the designs aims at significantly reducing the implementation complexity and computational load. A functional observer, and its extension of an unknown input functional observer, aims to reconstruct a functional of the infinite dimensional state. The resulting compensator only requires the solution to an operator Sylvester equation plus one differential equation for each dimension of the control signal, as opposed to an infinite dimensional filter evolution equation and an associated operator Riccati equation for the filter operator covariance. When the functional to be estimated coincides with the expression of a full state feedback control signal, then the functional observer becomes the minimum order compensator. When the parabolic system admits a decomposition whereby the system is decomposed into a lower finite dimensional subspace comprising the unstable eigenspectrum and an infinite stable subspace, then the functional observer-based compensator design becomes the minimum order compensator for the finite dimensional subsystem. This approach dramatically reduces the computation for solving the ARE needed for the full state controller and the associated Sylvester equation needed for the functional observer. Numerical results for a parabolic PDE in one and two spatial dimensions are included.

Keywords: Adaptive control, Distributed parameter systems, Lyapunov methods
Abstract: Adaptive control architectures often make use of Lyapunov functions to design adaptive laws. We are specifically interested in adaptive control methods, such as the well-known L1 adaptive architecture, which employ a parameter observer for this purpose. In such architectures, the observation error plays a critical role in determining analytical bounds on the tracking error as well as the system state. In this paper, we show how the non-existence of coercive Lyapunov operators can impact the analytical bounds, and with it the performance of such adaptive systems.

Keywords: Adaptive control, Distributed parameter systems, Game theory
Abstract: We propose a non-model based strategy for locally stable convergence to Nash equilibria in a quadratic noncooperative (duopoly) game with player actions subject to heterogeneous PDE dynamics. In this duopoly scenario, where different players use different types of PDEs, one player compensates for a delay (transport PDE) and the other a heat (diffusion) PDE, each player having access only to his own payoff value. The proposed approach employs extremum seeking, with sinusoidal perturbation signals applied to estimate the Gradient (first derivative) and Hessian (second derivative) of unknown quadratic functions. In our previous works, we solved Nash equilibrium seeking problems with homogeneous games, where the PDE dynamics of distinct nature were not allowed. This is the first instance of noncooperative games being tackled in a model-free fashion in the presence of heat PDE dynamics AND delays. In order to compensate distinct PDE-modeled processes in the inputs of the two players, we employ boundary control with averaging-based estimates. We apply a small-gain analysis for the resulting Input-to-State Stable (ISS) coupled hyperbolic-parabolic PDE system as well as averaging theory in infinite dimensions, due to the infinite-dimensional state of the heat PDE and the delay, in order to obtain local convergence results to a small neighborhood of the Nash equilibrium. We quantify the size of these residual sets and illustrate the theoretical results numerically on an example combining hyperbolic and parabolic dynamics in a 2-player setting.

Keywords: Distributed parameter systems, Delay systems, Lyapunov methods
Abstract: We study constant input delay compensation by using finite-dimensional observer-based controllers in the case of the 1D heat equation. We consider Neumann actuation with nonlocal measurement and employ modal decomposition with N+1 modes in the observer. We introduce a chain of M sub-predictors that leads to a closed-loop ODE system coupled with infinite-dimensional tail. Given an input delay r, we present LMI stability conditions for finding M and N and the resulting exponential decay rate and prove that the LMIs are always feasible for any r. We also consider a classical observer-based predictor and show that the corresponding LMI stability conditions are feasible for any r provided N is large enough. A numerical example demonstrates that the classical predictor leads to a lower-dimensional observer. However, it is hard for implementation due to the distributed input signal.

Keywords: Distributed control, Decentralized control, Optimization
Abstract: This paper investigates the stochastic distributed nonconvex optimization problem of minimizing a global cost function formed by the summation of n local cost functions. We solve such a problem by involving zeroth-order (ZO) information exchange. In this paper, we propose a ZO distributed primal-dual coordinate method (ZODIAC) to solve the stochastic optimization problem. Agents approximate their own local stochastic ZO oracle along with coordinates with an adaptive smoothing parameter. We show that the proposed algorithm achieves the convergence rate of mathcal{O}(sqrt{p}/sqrt{T}) for general nonconvex cost functions. We demonstrate the efficiency of proposed algorithms through a numerical example in comparison with the existing state-of-the-art centralized and distributed ZO algorithms.

Keywords: Distributed control, Hybrid systems, Stability of hybrid systems
Abstract: We present a hybrid systems framework for multi-agent optimization in which agents execute computations in continuous time and communicate in discrete time. The optimization algorithm is a hybrid version of parallelized coordinate descent. Agents implement a sample-and-hold strategy in which gradients are computed at communication times and held constant during flows between communications. Completeness of maximal solutions under these hybrid dynamics is established. Under assumptions of smoothness and strong convexity, we show that this system exponentially converges to the minimizer of an objective function. Simulation results illustrate this convergence rate.

Keywords: Distributed control, Large-scale systems, Identification for control
Abstract: In this paper we are considering the problem of fixed structure controller synthesis and system identification for large scale systems. The systems considered are referred to as decomposable systems and can be seen as resulting from the interconnection of a large number of identical subsystems. Combining recent results from nonsmooth controller synthesis with the special structure of decomposable systems, we show that the computational complexity of the controller synthesis and system identification procedure scale linearly with the size of the system. Employing results from robust control, the computational complexity of the controller synthesis can be reduced further, however, at the cost of additional conservatism. Using numerical examples we demonstrate that the proposed method can reduce the conservatism compared to existing LMI methods for full order decomposable controllers.

Keywords: Distributed control, Large-scale systems, Transportation networks
Abstract: This paper considers the problem of hub-based platoon coordination for a large-scale transport system, where trucks have individual utility functions to optimize. An event-triggered distributed model predictive control method is proposed to solve the optimal scheduling of waiting times at hubs for individual trucks. In this distributed framework, trucks are allowed to decide their waiting times independently and only limited information is shared between trucks. Both the predicted reward gained from platooning and the predicted cost for waiting at hubs are included in each truck's utility function. The performance of the coordination method is demonstrated in a simulation with one hundred trucks over the Swedish road network.

Keywords: Distributed control, Networked control systems, Nonlinear systems
Abstract: In this paper we analyse the equilibrium configurations for the time-inverted Kuramoto Model with homogeneous agents and a fixed ring topology, where time-inverted means that the coupling between the different states is via a negative factor. This model exhibits a dual behaviour with respect to the classic Kuramoto Model with a positive coupling. In the paper, we show the existence of two possible stable equilibrium configurations: the splay state formation (1-clustered coverage) and the deployment in clusters (κ-clustered coverage). We provide sufficient conditions for the splay state formation and a stability analysis for the networked system. Moreover, we provide some initial results towards the controllability of the final equilibrium configurations. In particular, we lay the foundations to understand the conditions to switch between stable equilibria.

Keywords: Distributed control, Stability of hybrid systems
Abstract: This work proposes a distributed control strategy for the robust global leader-follower phase synchronization of Kuramoto oscillators with inertia. For a convenient design, the phase angles are represented as elements of the unit circle. In particular, we exploit a “half-angle” representation inspired by unit quaternions. The ensuing non-Euclidean state space poses some challenges for robust global stabilization, which can be conveniently overcome with dynamic hybrid feedback. For this reason, we propose a hybrid solution obtained by combining a distributed observer with local hysteresis-based tracking controllers. The overall closed-loop system is analyzed through reduction theorems and Lyapunov-based arguments.

Keywords: Agents-based systems, Cooperative control, Networked control systems
Abstract: Networked dynamical systems (NDSs) have gained considerable attention in recent years, where the networked agents cooperate to accomplish the common task through the interaction topology. In this paper, we focus on the topology inference problem of NDSs. Different from traditional methods, we aim to infer the internal interaction topology from the perspective of node causality and correlation, covering both directed/undirected topology structures and the asymptotic/marginal stabilities of NDSs. Specifically, we propose a causality-based method that takes the noise characteristic into account and asymptotically approaches the real interaction topology structure, with only single round observations over the dynamical process. When the observation number is small, we further design a correlation-based modification to effectively alleviate the influence of noises. We demonstrate the close relation between the proposed method and the traditional Granger estimator and ordinary least square (OLS) estimator in terms of observation rounds and horizon.We further prove the equivalence conditions of the proposed method with the two estimators from the system stability and noise characteristic. The proposed causality-correlation combined method enjoys analogous asymptotic inference performance with Granger estimator of multiple observation rounds, and outperforms OLS estimator in single observation round, verified by extensive simulations.

Keywords: Agents-based systems, Information theory and control, Cooperative control
Abstract: This paper addresses the cooperative search of static ground targets by a group of Unmanned Aerial Vehicles (UAVs) over some region of interest. The search strategy dependents on the availability and accuracy of the information collected. When a target is detected, a probabilistic description of the measurement noise is usually considered, as well as probabilities of false alarm and non-detection, which may prove difficult to characterize a priori.

An alternative modeling is introduced here. The ability to detect and identify a target depends deterministically on the point of view from which the target is observed. Introducing the notion of detectability sets for targets, we propose a robust distributed set-membership estimator to provide set estimates of target locations. The obtained set estimates are guaranteed to contain all target locations when the search is completed. The target search is formulated as a multi-agent cooperative control problem where the control inputs are obtained using a Model Predictive Control (MPC) approach minimizing a measure of the set estimates representing the detection performance. The proposed set estimator and cooperative control scheme are distributed, i.e., accounting only for information from neighbors within communication range. The effectiveness of the proposed algorithm is illustrated by simulation.