Keywords: Nonlinear systems, Estimation
Abstract: The general focus of this talk is on nonlinear control systems in which only limited amounts of data can be exchanged between different parts of the system. In such systems, a control designer is faced not just with the task of developing a control algorithm, but also with deciding whether a given objective is achievable with the available data, or with characterizing the minimal data needed to meet the objective.

In this context, we consider two specific problem settings. We first address state estimation and model detection with finite data rate. Our unifying approach to these two tasks relies on a notion of entropy for dynamical systems. We then discuss observer design with guaranteed robustness to measurement errors. This problem is motivated in part by applications involving synchronization with limited data.

Keywords: Networked control systems, Boolean control networks and logic networks, Manufacturing systems and automation
Abstract: The heterogeneous system consisting of the wireless control system (WCS) and mobile agent system (MAS) is ubiquitous in Industrial Internet of Things (IIoT) systems. Within this system, the positions of mobile agents may lead to shadow fading on the wireless channel that the WCS is controlled over and can significantly compromise the WCS's performance. This paper focuses on the controller design for the MAS to ensure the performance of WCS in the presence of WCS and MAS coupling. Firstly, the constrained finite field network (FFN) with profile-dependent switching topology is adopted to proceed the operational control for the MAS. By virtue of the algebraic state space representation (ASSR) method, an equivalent form is obtained for the WCS and MAS coupling. A necessary and sufficient condition in terms of constrained set stabilization is then established to ensure the Lyapunov-like performance with expected decay rate. Finally, a graphical method together with the breath-first searching is provided to design state feedback controllers for the MAS. With this method, it is easy to check the constrained set stabilization of MAS and to ensure the performance requirements of WCS in the presence of WCS and MAS coupling. The study of an illustrative example shows the effectiveness of the proposed method.

Keywords: Large-scale systems, Lyapunov methods, Neural networks
Abstract: In this paper, we present a risk sensitive control for chaotic synchronization of a class of nonlinear complex dynamic systems. With an eye on a predefined risk sensitive parameter, the goal of this new design approach is to achieve the synchronization of neuronet-type stochastic complex networks toward a chaotic target node. The proposed method is rigorously formulated via stochastic Lyapunov technique, robust inverse optimality, and the Hamilton-Jacobi-Bellman equation, which ensures to accomplish the design objective that includes an achievable meaningful performance index. Furthermore, a numerical example is given to demonstrate the effectiveness of the introduced technique.

Keywords: Cooperative control, Control system architecture
Abstract: In this paper, we study multiagent networks subject to misbehaving nodes (i.e., nodes subject to exogenous disturbances) over fixed, connected, and directed graphs. In contrast to previous studies, which apply feedback control signals to every node in the network to suppress the adverse effect of misbehaving nodes, we propose to apply feedback control signals to a subset of nodes (i.e., driver nodes) in the network due to constraints. In particular, proportional-integral feedback controllers are proposed to be executed by the driver nodes, which guarantee that the overall multiagent network is stable in the sense of input-to-state stability (i.e., they make the resulting closed-loop system matrix Hurwitz). Following that we present a system-theoretical approach to find the steady-state value of each node in the network. In addition, a graph-theoretical approach is utilized to allow users to find steady-state values of critical nodes without requiring the knowledge of the Laplacian matrix of the overall multiagent network. Based on the results presented in this paper, one can gain the understanding how to select the driver nodes to negate the effect of misbehaving nodes on the neighborhood of the critical nodes. Finally, illustrative numerical examples are also presented to demonstrate the efficacy of the proposed approaches.

Keywords: Networked control systems, Machine learning, Kalman filtering
Abstract: In this paper, we consider a wireless network of smart sensors (agents) that monitor a dynamical process and send measurements to a base station that performs global monitoring and decision-making. Smart sensors are equipped with both sensing and computation, and can either send raw measurements or process them prior to transmission. Constrained agent resources raise a fundamental latency-accuracy trade-off. On the one hand, raw measurements are inaccurate but fast to produce. On the other hand, data processing on resource-constrained platforms generates accurate measurements at the cost of non-negligible computation latency. Further, if processed data are also compressed, latency caused by wireless communication might be higher for raw measurements. Hence, it is challenging to decide when and where sensors in the network should transmit raw measurements or leverage time-consuming local processing. To tackle this design problem, we propose a Reinforcement Learning approach to learn an efficient policy that dynamically decides when measurements are to be processed at each sensor. Effectiveness of our proposed approach is validated through a numerical simulation with case study on smart sensing motivated by the Internet of Drones.

Keywords: Sensor fusion, Estimation, Sensor networks
Abstract: In this paper, we study the problem of distributed estimation with an emphasis on communication-efficiency. The proposed algorithm is based on a windowed maximum a posteriori (MAP) estimation problem, wherein each agent in the network locally computes a Kalman-like filter estimate that approximates the centralized MAP solution. Information sharing among agents is restricted to their neighbors only, with guarantees on overall estimate consistency provided via logarithmic opinion pooling. The problem is efficiently distributed using the alternating direction method of multipliers (ADMM), whose overall communication usage is further reduced by a value of information (VoI) censoring mechanism, wherein agents only transmit their primal-dual iterates when deemed valuable to do so. The proposed censoring mechanism is mission-aware, enabling a globally efficient use of communication resources while guaranteeing possibly different local estimation requirements. To illustrate the validity of the approach we perform simulations in a target tracking scenario.

Keywords: Stochastic systems, Networked control systems, Subspace methods
Abstract: The modelling and control of systems on large complex networks is intractable in general. One approach is to use graphon theory which provides limit objects for infinite sequences of graphs permitting one to approximate arbitrarily large networks by infinite dimensional operators. Such a formulation was initiated in the work of Gao and Caines (2020, 2021) extending classical linear system control theory to the control of systems on large networks. This paper introduces infinite dimensional stochastic processes called Q-space noise into this framework. First, Brownian motions in Hilbert spaces are defined. Second, stochastic dynamical systems on large graphs using Q-space noise processes are shown to converge in the graph limit in expectation. Third, state-to-state and linear quadratic control of these systems is formulated and the limit approximations are established. Finally, the behavior of these approximations is illustrated numerically.

Keywords: Distributed control, Predictive control for linear systems, Autonomous systems
Abstract: Connectivity maintenance and safe navigation around obstacles are fundamental challenges for vehicle formations. This paper introduces a novel multi-level supervision distributed architecture aimed at supervising a set of dynamically decoupled agents operating in a 2D space and subject to obstacle avoidance and connectivity- keeping constraints. An existing supervision scheme based on Distributed Command Governor ideas is here extended in order to adequately tackle non-convex obstacle avoidance constraints while maintaining the communication connectivity of the formation, here modeled as a dynamic graph. To this end, the proposed method enforces the existence of a specific minimum spanning tree at each time instant while fulfilling obstacle avoidance constraints by using a separation hyperplane. Conditions that formally prove the feasibility of the proposed strategy are included along with simulations that show the effectiveness of the proposed method.

Keywords: Distributed control, Predictive control for linear systems, Optimization algorithms
Abstract: In this paper, we propose an approach to distributed model predictive control based on distributed optimization and suboptimal model predictive control theory. In the proposed method, the individual subsystems are coupled through the inputs and do not need to exchange information about their system dynamics or objective function. Coordination is achieved by iteratively exchanging an estimation of the subsystem's optimal control sequence only with neighboring subsystems. The subsystems compute iteratively an admissible control sequence, which is asymptotically stabilizing the system. With mild assumptions, a minimum number of iterations for obtaining an admissible control sequence and hence guaranteeing stability of the closed-loop system can be computed independently of the initial overall system state. Furthermore, it is shown that the centrally optimal control sequence is achieved at convergence. The subsystem's practitioner can then choose to apply a suboptimal but stabilizing control sequence or wait for the optimal control sequence when convergence is attained. The approach is illustrated through an academic example.

Keywords: Distributed control, LMIs, Adaptive control
Abstract: This paper provides a distributed adaptive control architecture for uncertain multiagent systems with nonidentical actuation capacities and unknown control effectiveness to achieve cooperative behaviors for real-world applications. In detail, our approach includes a user-assigned Laplacian matrix for creating cooperative behaviors with multiple agents, a hedging-based reference model to provide correct adaptation that is not affected by the presence of heterogeneous actuator dynamics in the networked system, and a distributed adaptive control architecture to deal with the system anomalies. The stability of the overall multiagent system is showed by utilizing the Lyapunov stability theorem, and allowance actuator bandwidth limits are calculated by using linear matrix inequalities. Results of numerical examples are illustrated to indicate the performance of the proposed control algorithm on a multiagent system that is a fully connected circle graph.

Keywords: Distributed control, Control of networks, Stochastic optimal control
Abstract: In this paper we consider the distributed infinite-horizon Linear-Quadratic-Gaussian optimal control problem for discrete-time systems over networks. In particular, the feedback controller is composed of local control stations, which receives some measurement data from the plant process and regulates a portion of the input signal. We provide a solution when the nodes have information on the structural data of the whole network but takes local actions, and also when only local information on the network are available to the nodes. The proposed solution is arbitrarily close to the optimal centralized one (in terms of cost index) when the intermediate consensus steps are sufficiently large

Keywords: Distributed control, Constrained control, Predictive control for nonlinear systems
Abstract: In this paper, we present a distributed model predictive control (DMPC) scheme for dynamically decoupled systems which are subject to state constraints, coupling state constraints and input constraints. In the proposed control scheme, neighbor-to-neighbor communication suffices and all subsystems solve their local optimization problem in parallel. The approach relies on consistency constraints which define a neighborhood around each subsystem’s reference trajectory where the state of the respective subsystem is guaranteed to stay in. Reference trajectories and consistency constraints are known to neighboring subsystems. Contrary to other relevant approaches, the reference trajectories are improved iteratively. Besides, the presented approach allows the formulation of convex optimization problems even in the presence of non-convex state constraints. The algorithm’s effectiveness is demonstrated with a simulation.

Keywords: Distributed control, Optimization algorithms, Sensor networks
Abstract: This paper considers privacy-concerned distributed constraint-coupled resource allocation problems over an undirected network, where each agent holds a private cost function and obtains the solution via only local communication. With privacy concerns, we mask the exchanged information with independent Laplace noise against a potential attacker with potential access to all network communications. We propose a differentially private distributed mismatch tracking algorithm (diff-DMAC) to achieve cost-optimal distribution of resources while preserving privacy. Adopting constant stepsizes, the linear convergence property of diff-DMAC in mean square is established under the standard assumptions of Lipschitz gradients and strong convexity. Moreover, it is theoretically proven that the proposed algorithm is -differentially private. And we also show the trade-off between convergence accuracy and privacy level. Finally, a numerical example is provided for verification.

Keywords: Autonomous systems, Robotics, Optimization
Abstract: Signal Temporal Logic (STL) provides a convenient way of encoding complex control objectives for robotic and cyber-physical systems. The state-of-the-art in trajectory synthesis for STL is based on Mixed-Integer Convex Programming (MICP). The MICP approach is sound and complete, but has limited scalability due to exponential complexity in the number of binary variables. In this letter, we propose a more efficient MICP encoding for STL. Our new encoding is based on the insight that disjunction can be encoded using a logarithmic number of binary variables and conjunction can be encoded without binary variables. We demonstrate in simulation examples that our proposed approach significantly outperforms the state-of-the-art for long and complex specifications.

Keywords: Autonomous systems, Network analysis and control
Abstract: Most of the classic collective behavior occurring in multiagent systems (MASs) are built upon the network topology with sufficient connectivity. The MASs that impose no restrictions on the network topology deserves due attention to reveal more general collective behavior, which means unifying consensus and containment control. This paper examines the collective behavior in the network of homogeneous generic linear agents over the dynamic topology without structural constraints. By virtue of the product of nonnegative matrices and the properties of algebraic digraphs, a systematic coordination analysis with proper parameter selection is put forward. It is shown that local consensus is reached by the agents of the same closed strongly connected component, whereas the agents outside the closed strongly connected components are ultimately confined in the convex hull defined by the agents of the closed strongly connected components. The results of the theoretical analysis are supported by computer simulations.

Keywords: Autonomous systems, Cooperative control, Decentralized control
Abstract: This paper studies leaderless affine formation maneuver control over a directed sensing topology, in the presence of constant input disturbances. Directed sensing topology is important when limitations in the sensing capabilities of an agent are considered. Unlike traditional leader-follower approaches, in case of leaderless formation maneuvers, there are no leaders. First, control laws for leaderless affine formation maneuvering for single integrator agents are presented, considering a directed sensing graph. Global Asymptotic Stability (GAS) is proved for formation tracking error in presence of constant disturbances. Subsequently, for higher-order systems, control laws are designed using an adaptive back-stepping based approach and GAS is proved for such control laws. Simulations are provided to validate the results.

Keywords: Autonomous systems, Stability of hybrid systems, Switched systems
Abstract: In this paper, we show how Behavior Trees that have performance guarantees, in terms of safety and goal convergence, can be extended with components that were designed using machine learning, without destroying those performance guarantees.

Machine learning approaches such as reinforcement learning or learning from demonstration can be very appealing to AI designers that want efficient and realistic behaviors in their agents. However, those algorithms seldom provide guarantees for solving the given task in all different situations while keeping the agent safe. Instead, such guarantees are often easier to find for manually designed model-based approaches. In this paper we exploit the modularity of behavior trees to extend a given design with an efficient, but possibly unreliable, machine learning component in a way that preserves the guarantees. The approach is illustrated with an inverted pendulum example.

Keywords: Autonomous systems, Human-in-the-loop control, Optimization algorithms
Abstract: In this paper, we consider the problem of allo- cating human operator assistance in a system with multiple autonomous robots. Each robot is required to complete inde- pendent missions, each defined as a sequence of tasks. While executing a task, a robot can either operate autonomously or be teleoperated by the human operator to complete the task at a faster rate. We formulate our problem as a Mixed Integer Linear Program, which can be used to optimally solve small to moderate sized problem instances. We also develop an anytime algorithm that makes use of the problem structure to provide a fast and high-quality solution of the operator scheduling problem, even for larger problem instances. Our key insight is to identify blocking tasks in greedily-created schedules and iteratively remove those blocks to improve the quality of the solution. Through numerical simulations, we demonstrate the benefits of the proposed algorithm as an efficient and scalable approach that outperforms other greedy methods.

Keywords: Autonomous systems, Neural networks, Automata
Abstract: This paper presents a new approach to design verified compositions of Neural Network (NN) controllers for autonomous systems with tasks captured by Linear Temporal Logic (LTL) formulas. Particularly, the LTL formula requires the system to reach and avoid certain regions in a temporal/logical order. We assume that the system is equipped with a finite set of trained NN controllers. Each controller has been trained so that it can drive the system towards a specific region of interest while avoiding others. Our goal is to check if there exists a temporal composition of the trained NN controllers - and if so, to compute it - that will yield composite system behaviors that satisfy a user-specified LTL task for any initial system state belonging to a given set. To address this problem, we propose a new approach that relies on a novel integration of automata theory and recently proposed reachability analysis tools for NN-controlled systems. We note that the proposed method can be applied to other controllers, not necessarily modeled by NNs, by appropriate selection of the reachability analysis tool. We focus on NN controllers due to their lack of robustness. The proposed method is demonstrated on navigation tasks for aerial vehicles.

Keywords: Machine learning, Optimal control, Robust control
Abstract: This paper proposes a data-driven methodology to place the closed-loop poles in desired convex regions in the complex plane with sufficient robustness constraints. We considered the system state and input matrices unknown and only used the measurements of system trajectory. The closed-loop pole placement problem in the linear matrix inequality (LMI) regions is considered a classic robust control problem; however, that requires knowledge about the state and input matrices of the linear system. We bring in ideas from the behavioral system theory and persistency of excitation condition-based fundamental lemma to develop a data-driven counterpart that satisfies multiple closed-loop robustness specifications, such as mathcal{D}-stability and mixed H_2/H_{infty} performance specifications. We provide solutions based on the availability of the disturbance input, both in the controlled and fully uncertain environment, leading to data-driven semi-definite programs (SDPs) with sufficient guarantees. We validate the theoretical results with numerical simulations on a third-order dynamic system.

Keywords: Learning, Optimal control, Linear systems
Abstract: When dealing with unknown systems, data can be used to directly learn controllers with desirable features, thus bypassing system identification. In this paper we present strategies to design optimal controls for unknown linear systems directly trough closed-form functions of the data. In particular, when data is sufficiently informative, (i) we find the control input that minimizes a finite-horizon quadratic function of the states and inputs and (ii) we show how these inputs enable the estimate of the static feedback controller that minimizes an infinite-horizon quadratic function, i.e., the Linear Quadratic Regulator. Our formulas are closed-form, making them computationally efficient and of straightforward interpretation.

Keywords: Learning, Optimal control, Stability of nonlinear systems
Abstract: In this paper, we develop a safe pursuit-evasion game for enabling finite-time capture and optimal performance. The pursuit-evasion game is formulated as a zero-sum differential game wherein the pursuer seeks to minimize its relative distance to the target while the evader attempts to maximize it. A critic-only reinforcement learning-based algorithm is then proposed for learning online and in finite time the pursuit-evasion policies, thus enabling finite-time capture of the evader. Safety is ensured by means of barrier functions associated with the obstacles, which are integrated into the running cost. Simulation results illustrate the efficacy of the proposed approach.

Keywords: Machine learning, Optimization, Markov processes
Abstract: We consider a discounted cost constrained Markov decision process (CMDP) policy optimization problem, in which an agent seeks to maximize a discounted cumulative reward subject to a number of constraints on discounted cumulative utilities. To solve this constrained optimization program, we study an online actor-critic variant of a classic primal-dual method where the gradients of both the primal and dual variables are estimated using samples from a single trajectory generated by the underlying time-varying Markov processes. This online primal-dual natural actor-critic algorithm maintains and iteratively updates three variables: a dual variable (or Lagrangian multiplier), a primal variable (or actor), and a critic variable used to estimate the gradients of both primal and dual variables. These variables are updated simultaneously but on different time scales (using different step sizes) and they are all intertwined with each other. Our main contribution is to derive a finite-time analysis for the convergence of this algorithm to the global optimum of a CMDP problem. Specifically, we show that with a proper choice of step sizes the optimality gap and constraint violation converge to zero in expectation at a rate O(1/K^(1/6)), where K is the number of iterations. To our knowledge, this paper is the first to study the finite-time complexity of an online primal-dual actor-critic method for solving a CMDP problem. We also validate the effectiveness of this algorithm through numerical simulations.

Keywords: Optimal control
Abstract: In this letter, we discuss the problem of optimal control for affine systems in the context of data-driven linear programming. We first introduce a unified framework for the fixed point characterization of the value function, Q-function and relaxed Bellman operators. Then, in a model-free setting, we show how to synthesize and estimate Bellman inequalities from a small but sufficiently rich dataset. To guarantee exploration richness, we complete the extension of Willems' fundamental lemma to affine systems.

Keywords: Learning, Optimization, Linear systems
Abstract: The convergence of policy gradient algorithms in reinforcement learning hinges on the optimization landscape of the underlying optimal control problem. Theoretical insights into these algorithms can often be acquired from analyzing those of linear quadratic control. However, most of the existing literature only considers the optimization landscape for static full-state or output feedback policies (controllers). We investigate the more challenging case of dynamic output-feedback policies for linear quadratic regulation (abbreviated as dLQR), which is prevalent in practice but has a rather complicated optimization landscape. We first show how the dLQR cost varies with the coordinate transformation of the dynamic controller and then derive the optimal transformation for a given observable stabilizing controller. At the core of our results is the uniqueness of the stationary point of dLQR when it is observable, which is in a concise form of an observer-based controller with the optimal similarity transformation. These results shed light on designing efficient algorithms for general decision-making problems with partially observed information.

Keywords: Machine learning, Time-varying systems, Adaptive control
Abstract: Changing conditions or environments can cause system dynamics to vary over time. To ensure optimal control performance, controllers should adapt to these changes. As systems become increasingly complex, first-principal modeling and data-based system identification reach their limits. This motivates approaches such as time-varying Bayesian optimization (TVBO) to tune controllers online by directly modeling the control objective. Many online controller-tuning problems can be characterized by two properties. First, they exhibit incremental and lasting changes of the objective due to changes to the system dynamics through, e.g., wear and tear. Second, the optimization problem is convex in the tuning parameters. Current TVBO methods do not explicitly account for these properties, resulting in poor tuning performance and many unstable controllers through over-exploration of the parameter space. We propose a novel TVBO forgetting strategy using Uncertainty-Injection (UI), thereby retaining relevant information about the control objective over time. The control objective is modeled as a spatio-temporal Gaussian process (GP) with UI through a Wiener process in the temporal domain. Further, we explicitly model convex functions through GP models with linear inequality constraints. In numerical experiments, we show that our model outperforms the state-of-the-art method in TVBO, exhibiting reduced regret and less unstable parameter configurations.

Keywords: Nonlinear systems, Neural networks, Learning
Abstract: The energy Casimir method is an effective controller design approach to stabilize port-Hamiltonian systems at a desired equilibrium. However, its application relies on the availability of suitable Casimir and Lyapunov functions, whose computation are generally intractable. In this paper, we propose a neural network-based framework to learn these functions. We show how to achieve equilibrium assignment by adding suitable regularization terms in the training cost. We also propose a parameterization of Casimir functions for reducing the training complexity. Moreover, the distance between the equilibrium of the learned Lyapunov function and the desired equilibrium is analyzed, which indicates that for small suboptimality gaps, the distance decreases linearly with respect to the training loss. Our methods are backed up by simulations on a pendulum system.

Keywords: Distributed control, Optimal control, Machine learning
Abstract: In this paper, we consider a Linear Quadratic optimal control problem with the assumptions that the system dynamics is unknown and that the designed feedback control has to comply with a desired sparsity pattern. An important application where this set-up arises is distributed control of network systems, where the aim is to find an optimal sparse controller matching the communication graph. To tackle the problem, we propose a Reinforcement Learning framework based on a Q-learning scheme preserving a desired policy structure. At each time step the performance of the current candidate feedback is first evaluated through the computation of its Q-function, and then a new sparse feedback matrix, improving on the previous one, is computed. We prove that the scheme produces at each iteration a stabilizing feedback control with the desired sparsity and with non-increasing cost, which in turns indicates that every limit point of the computed feedback matrices is sparse and stabilizing. The algorithm is numerically tested on a distributed control scenario with randomly generated graph and unstable dynamics.

Keywords: Optimal control, Machine learning
Abstract: In recent years there has been a collective research effort to find new formulations of reinforcement learning that are simultaneously more efficient and more amenable to analysis. This paper concerns one approach that builds on the linear programming (LP) formulation of optimal control of Manne. A primal version is called logistic Q-learning, and a dual variant is convex Q-learning. This paper focuses on the latter, while building bridges with the former. The main contributions follow: (i) The dual of convex Q-learning is not precisely Manne's LP or a version of logistic Q-learning, but has similar structure that reveals the need for regularization to avoid over-fitting. (ii) A sufficient condition is obtained for a bounded solution to the Q-learning LP. (iii) Simulation studies reveal numerical challenges when addressing sampled-data systems based on a continuous time model. The challenge is addressed using state-dependent sampling. The theory is illustrated with applications to examples from OpenAI gym. It is shown that convex Q-learning is successful in cases where standard Q-learning diverges, such as the LQR problem.

Keywords: Traffic control, Iterative learning control, Autonomous systems
Abstract: Traffic shock waves are a commonly occurring phenomena caused by the delays in reaction times of Human Driven Vehicles (HDVs) resulting in unnecessary congestion in highway networks. Application of a suitable moving bottleneck control using Connected Autonomous Vehicles (CAVs) can result in shock wave mitigation and smoothing of the traffic flow. This traffic control scheme is dependent on accurately predicting shock wave conditions while choosing the best control to apply for the observation available to the CAV. In this work, we propose the use of a multi-agent shared policy reinforcement learning algorithm which leverages communication between CAVs for improved observability of downstream traffic conditions. A key feature of this method is the ability to perform shock wave dissipation control without the need for global information and the applicability of this method to multi-lane mixed traffic highways of arbitrary structure. We use the shared-parameter Proximal Policy Optimization (PPO) reinforcement learning strategy for obtaining the controls for each CAV in the simulation. We also built a custom SUMO-Gym wrapper for the multi-lane highway simulation with custom designed observation space, action space and rewards for each agent. The shock wave dissipation efficiency is evaluated on a three lane circular highway loop using realistic traffic simulation software and low CAV penetration levels.

Keywords: Identification for control, Statistical learning, Uncertain systems
Abstract: This paper proposes a physically consistent Gaussian Process (GP) enabling the data-driven modelling of uncertain Lagrangian systems. The function space is tailored according to the energy components of the Lagrangian and the differential equation structure, analytically guaranteeing properties such as energy conservation and quadratic form. The novel formulation of Cholesky decomposed matrix kernels allow the probabilistic preservation of positive definiteness. Only differential input-to-output measurements of the function map are required while Gaussian noise is permitted in torques, velocities, and accelerations. We demonstrate the effectiveness of the approach in numerical simulation.

Keywords: Estimation, Linear systems, Optimization
Abstract: This paper is concerned with the distributed robust optimal filtering problem for discrete-time systems subject to both bounded-power disturbances and white Gaussian noises. Relied on the system level synthesis, an upper bound of the estimation performance is characterized in terms of system responses of the error dynamics and parameters of the disturbances and noises. Based on such characterization, a numerically tractable algorithm is presented for distributed multi-objective filter design. Moreover, the relative estimation performance loss incurred by imperfect modeling uncertainty and the finite truncation is explicitly established. A numerical example is also included to show the effectiveness of the current results.

Keywords: Estimation, Linear systems
Abstract: We address the problem of constructing matrix valued interval observers for estimating state transition matrices for time-varying systems. We provide less conservative estimators than those in recent literature. We cover continuous- and discrete-time linear systems, under Metzler or nonnegativity conditions on the coefficient matrices. We show how to satisfy our Metzler conditions after simple changes of coordinates. We illustrate our method using a feedback stabilized underwater marine robotic dynamics with unknown control gains.

Keywords: Estimation, Network analysis and control, Linear systems
Abstract: Optimization of controllability, i.e., to optimize the degree of the controllability using the flexibility of a system such as choosing input nodes and designing network structure, is growing in importance to network systems. For example, if a network system has strong controllability for a certain control input, we know that the input is effective for controlling the system even under a limited energy condition. If a mathematical model of the system is available, the controllability can be optimized by a model-based method. However, we often face the difficulty to incorporate useful prior knowledge of the system. In such a case, a data-driven approach is promising. In this paper, as the initial step toward this direction, we address the problem of determining the sensitivity of a controllability measure with respect to an edge weight of a network system in a data-driven manner. By characterizing the sensitivity by two Lyapunov equations, we clarify that the sensitivity is derived from the so-called data-driven Lyapunov equations. Moreover, this result is extended to the case of high-order sensitivity.

Keywords: Estimation, Statistical learning, Linear systems
Abstract: This paper proposes a primal-dual framework to learn a stable estimator for linear constrained estimation problems leveraging the moving horizon approach. To avoid the online computational burden in most existing methods, we learn a parameterized function offline to approximate the primal estimate. Meanwhile, a dual estimator is trained to check the suboptimality of the primal estimator during execution time. Both the primal and dual estimators are learned from data using supervised learning techniques, and the explicit sample size is provided, which enables us to guarantee the quality of each learned estimator in terms of feasibility and optimality. This in turn allows us to bound the probability of the learned estimator being infeasible or suboptimal. Furthermore, we analyze the stability of the resulting estimator with a bounded error in the minimization of the cost function. Since our algorithm does not require the solution of an optimization problem during runtime, state estimates can be generated online almost instantly. Simulation results are presented to show the accuracy and time efficiency of the proposed framework compared to online optimization of moving horizon estimation and Kalman filter. To the best of our knowledge, this is the first learning-based state estimator with feasibility and near-optimality guarantees for linear constrained systems.

Keywords: Estimation, Computational methods, Filtering
Abstract: Linear minimum mean square error (LMMSE) estimation is often ill-conditioned, suggesting that unconstrained minimization of the mean square error is an inadequate approach to filter design. To address this, we first develop a unifying framework for studying constrained LMMSE estimation problems. Using this framework, we explore an important structural property of constrained LMMSE filters involving a certain prefilter. Optimality is invariant under invertible linear transformations of the prefilter. This parameterizes all optimal filters by equivalence classes of prefilters. We then clarify that merely constraining the rank of the filter does not suitably address the problem of ill-conditioning. Instead, we adopt a constraint that explicitly requires solutions to be well-conditioned in a certain specific sense. We introduce two well-conditioned filters and show that they converge to the unconstrained LMMSE filter as their truncation-power loss goes to zero, at the same rate as the low-rank Wiener filter. We also show extensions to the case of weighted trace and determinant of the error covariance as objective functions. Finally, our quantitative results with historical VIX data demonstrate that our two well-conditioned filters have stable performance while the standard LMMSE filter deteriorates with increasing condition number.

Keywords: Observers for Linear systems, Estimation, Automata
Abstract: This paper proposes a robust output feedback state estimator for uncertain/bounded-error affine systems subject to data losses modeled by an automaton. Specifically, by introducing a novel property known as recurrent recovery, where the estimation errors are required to be recurrent to some minimum recovery levels at each node of the data loss automata, we design a robust estimator design that guarantees that the state estimation errors remain bounded in a recurrent manner despite worst-case realizations of process and sensor noise/uncertainties in addition to missing data. Our design can directly deal with infinite-horizon missing data specifications modeled by automata by recasting the problem into multiple finite-horizon problems of varying lengths, which results in an optimization-based approach with only a finite number of constraints. Moreover, our design is built upon system-level parameterization and for this purpose, we propose a novel affine output feedback strategy that also contributes to the literature of finite-horizon optimal control.

Keywords: Stochastic systems, Optimization
Abstract: We consider a network of agents, each with its own private cost consisting of a sum of two possibly nonsmooth convex functions, one of which is composed with a linear operator. At every iteration each agent performs local calculations and can only communicate with its neighbors. The challenging aspect of our study is that the smooth part of the private cost function is given as an expected value and agents only have access to this part of the problem formulation via a heavy-tailed stochastic oracle. To tackle such sampling-based optimization problems, we propose a stochastic extension of the triangular pre-conditioned primal-dual algorithm. We demonstrate almost sure convergence of the scheme and validate the performance of the method via numerical experiments.

Keywords: Optimization algorithms, Agents-based systems, Stochastic systems
Abstract: We present a multi-agent algorithm for multi-objective optimization problems, which extends the class of consensus-based optimization methods and relies on a scalarization strategy. The optimization is achieved by a set of interacting agents exploring the search space and attempting to solve all scalar sub-problems simultaneously. We show that those dynamics are described by a mean-field model, which is suitable for a theoretical analysis of the algorithm convergence. Numerical results show the validity of the proposed method.

Keywords: Game theory, Optimization algorithms, Agents-based systems
Abstract: We consider potential games with mixed-integer variables, for which we propose two distributed, proximal-like equilibrium seeking algorithms. Specifically, we focus on two scenarios: i) the underlying game is generalized ordinal and the agents update through iterations by choosing an exact optimal strategy; ii) the game admits an exact potential and the agents adopt approximated optimal responses. By exploiting the properties of integer-compatible regularization functions used as penalty terms, we show that both algorithms converge to either an exact or an epsilon-approximate equilibrium. We corroborate our findings on a numerical instance of a Cournot oligopoly model.

Keywords: Optimization algorithms, Decentralized control, Smart grid
Abstract: A major challenge in today’s electricity system is the management of flexibilities offered by new usages, such as smart home appliances or electric vehicles. By incentivizing energy consumption profiles of individuals, demand response seeks to adjust the power demand to the supply, for increased grid stability and better integration of renewable energies. This optimization of flexibility is typically managed by Load Aggregators, independent entities which aggregate and optimize numerous flexibility providers. The consideration of the underlying distribution network constraints, which couple the different actors, leads to a complex multi-agent problem. To address it, we propose a new decentralized algorithm that solves a convex relaxation of the classical Alternative Current Optimal Power Flow (ACOPF) problem, and which relies on local information only. Each computational step is performed in a privacy-preserving manner, and system-wide coordination is achieved via node-specific distribution locational marginal prices (DLMPs). We demonstrate the efficiency of our approach on a 15-bus radial distribution network.

Keywords: Optimization algorithms, Quantum information and control, Machine learning
Abstract: We propose a distributed Quantum State Tomography (QST) protocol, named Local Stochastic Factored Gradient Descent (Local SFGD), to learn the low-rank factor of a density matrix over a set of local machines. QST is the canonical procedure to characterize the state of a quantum system, which we formulate as a stochastic non-convex smooth optimization problem. Physically, the estimation of a low-rank density matrix helps characterizing the amount of noise introduced by quantum computation. Theoretically, we prove the local convergence of Local SFGD for a general class of restricted strongly convex/smooth loss functions. Local SFGD converges locally to a small neighborhood of the global optimum at a linear rate with a constant step size, while it locally converges exactly at a sub-linear rate with diminishing step sizes. With a proper initialization, local convergence results imply global convergence. We validate our theoretical findings with numerical simulations of QST on the Greenberger-Horne-Zeilinger (GHZ) state.

Keywords: Stochastic optimal control, Game theory, Control over communications
Abstract: We consider a multi-agent system in which a decentralized team of agents controls a stochastic system in the presence of an adversary. Instead of committing to a fixed information sharing protocol, the agents can strategically decide at each time whether to share their private information with each other or not. The agents incur a cost whenever they communicate with each other and the adversary may eavesdrop on their communication. Thus, the agents in the team must effectively coordinate with each other while being robust to the adversary's malicious actions. We model this interaction between the team and the adversary as a stochastic zero-sum game where the team aims to minimize a cost while the adversary aims to maximize it. Under some assumptions on the adversary's capabilities, we characterize a min-max control and communication strategy for the team. We supplement this characterization with several structural results that can make the computation of the min-max strategy more tractable.

Keywords: Power systems, Switched systems, Networked control systems
Abstract: Declines in cost and concerns about the environmental impact of traditional generation have boosted the penetration of renewables and non-conventional distributed energy resources into the power grid. The intermittent availability of these resources causes the inertia of the power system to vary over time. As a result, there is a need to go beyond traditional controllers designed to regulate frequency under the assumption of invariant dynamics. This paper presents a learning-based framework for the design of stable controllers based on imitating datasets obtained from linear-quadratic regulator (LQR) formulations for different switching sequences of inertia modes. The proposed controller is linear with a constant feedback-gain, thereby interpretable, does not require the knowledge of the current operating mode, and is guaranteed to stabilize the switching power dynamics. We show that it is always possible to stabilize the switched system using a communication-free local controller whose implementation only requires each node to use its own state. We illustrate our results on a 12-bus 3-region network.

Keywords: Optimization, Distributed control, Optimization algorithms
Abstract: This paper develops a distributed optimization framework for solving the rank-constrained semidefinite programs (RCSPs). Since the rank constraint is non-convex and discontinuous, solving an optimization problem with rank constraints is NP-hard and notoriously time-consuming, especially for large-scale RCSPs. In the proposed approach, by decomposing an unknown matrix into a set of submatrices with much smaller sizes, the rank constraint on the original matrix is equivalently transformed into a set of constraints on the decomposed submatrices. The distributed framework allows parallel computation of subproblems while requiring coordination among them to satisfy the coupled constraints. As the scale of every subproblem solved independently is significantly reduced, the decomposition scheme and the distributed framework can be applied to large-scale RCSPs. Moreover, optimality conditions of the proposed distributed optimization algorithm for RCSPs at the converged point are analyzed. Finally, the efficiency and effectiveness of the proposed method are demonstrated via simulation examples for solving the image denoising problem.

Keywords: Machine learning, Optimization algorithms, Sensor networks
Abstract: We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of n agents seeks to find a common point that enables ``few-shot'' learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push under arbitrary multi-step personalization for smooth strongly convex, and smooth non-convex functions. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.

Keywords: Statistical learning, Large-scale systems, Optimization algorithms
Abstract: Recent years have witnessed a growing interest in the topic of min-max optimization, owing to its relevance in the context of generative adversarial networks (GANs), robust control and optimization, and reinforcement learning. Motivated by this line of work, we consider a multi-agent min-max learning problem, and focus on the emerging challenge of contending with worst-case Byzantine adversarial agents in such a setup. By drawing on recent results from robust statistics, we design a robust distributed variant of the extra-gradient algorithm - a popular algorithmic approach for min-max optimization. Our main contribution is to provide a crisp analysis of the proposed robust extra-gradient algorithm for smooth convex-concave and smooth strongly convex-strongly concave functions. Specifically, we establish statistical rates of convergence to approximate saddle points. Our rates are near-optimal, and reveal both the effect of adversarial corruption and the benefit of collaboration among the non-faulty agents. Notably, this is the first paper to provide formal theoretical guarantees for large-scale distributed min-max learning in the presence of adversarial agents.

Keywords: Resilient Control Systems, Distributed control, Cyber-Physical Security
Abstract: Enhancing resilience in distributed networks in the face of malicious agents is an important problem for which many key theoretical results and applications require further development and characterization. This work develops a new algorithmic and analytical framework for achieving resilience to malicious agents in distributed optimization problems where a legitimate agent's dynamic is influenced by the values it receives from neighboring agents and its own self-serving target function. We show that by utilizing stochastic values of trust between agents it is possible to recover convergence to the system's global optimal point even in the presence of malicious agents. Additionally, we provide expected convergence rate guarantees in the form of an upper bound on the expected squared distance to the optimal value. Finally, we present numerical results that validate the analytical convergence guarantees we present in this paper even when the malicious agents are the majority of agents in the network.

Keywords: Estimation, Statistical learning, Resilient Control Systems
Abstract: We study robust mean estimation in an online and distributed scenario in the presence of adversarial data attacks. At each time step, each agent in a network receives a potentially corrupted data point, where the data points were originally independent and identically distributed samples of a random variable. We propose online and distributed algorithms for all agents to asymptotically estimate the mean. We provide the error-bound and the convergence properties of the estimates to the true mean under our algorithms. Based on the network topology, we further evaluate each agent's trade-off in convergence rate between incorporating data from neighbors and learning with only local observations.

Keywords: Linear systems, Observers for Linear systems
Abstract: In this work, sample-based observability of linear discrete-time systems is studied. That is, we consider the case where the system output measurements are not available at every time instance. It is shown that some discrete-time systems exhibit particular behaviors that lead to pathological sampling. Depending on the characteristics of the system, different sampling schemes are developed that allow the system state to be reconstructed.

Keywords: Linear systems, Sampled-data control, Optimization
Abstract: In this paper the problem of dynamic control allocation is studied in the context of a continuous-time plant regulated by means of a digital controller, comprising sampling and holding blocks. The proposed architecture consists of an annihilator unit designed on the basis of the sampled-data equivalent description of the plant and of a discrete-time steady-state generator/optimizer driven by the exogenous reference signal. By relying on the feed-forward nature of the design, it is shown that the allocator unit yields the optimal solution in finite time, differently from existing solutions purely in the context of continuous-time plants.

Keywords: Linear systems, Compartmental and Positive systems
Abstract: This paper fills the literature gap by considering the LQR design for discrete-time positive linear systems, which remains an open problem due to its nonconvexity. We propose a first-order method to solve this problem where the nonconvexity inherited from positivity and stability constraints was tackled by utilizing the special property of positive linear systems. More specifically, two Lyapunov-based methods were proposed to compute the L-2 and L-infinity projections for single-input and multi-input systems, respectively. Based on the projection theorems, a novel projected gradient descent algorithm was developed for computation and a successively improved sequence of suboptimal solutions is obtained. Finally, two numerical examples are provided to verify the effectiveness of our proposed results. The proposed first-order method is likely to be extended to synthesize more general constrained systems.

Keywords: Linear systems, Optimization, Optimal control
Abstract: Several time-domain constraints for linear systems are characterized using a new representation of the impulse response based on symmetric polynomials. This includes the induced ∞-norm, the peak response and external positivity – performance indices that are complicated to impose in existing LMI frameworks. Each index is described by a convex constraint, explicit in terms of the system poles. Despite their simplicity, numerical examples suggest that these constraints can provide relatively tight performance bounds.

Keywords: Linear systems, Large-scale systems, Network analysis and control
Abstract: The volume of the set of states of a linear system that can be reached using control inputs with bounded energy depends on the determinant of the controllability Gramian. As such, the latter represents a meaningful metric to quantify the degree of controllability of a linear system. In this paper, we conduct a detailed study of this controllability metric for the case of linear network systems controlled by a single node. Our analysis builds on a closed-form expression which uncovers the relation between the determinant of the controllability Gramian and the spectral properties of the underlying system. In particular, we explore the dependence of this metric on the network dimension and provide a characterization of its scaling behavior in terms of exact exponential decay rates. We illustrate and complement our findings with a set of numerical examples.

Keywords: Linear systems, Stability of linear systems, Iterative learning control
Abstract: In this paper, we review different stability definitions that have been used in the literature concerning 2D Roesser models: structural stability, asymptotic stability, and exponential stability. We clarify the relations between all these definitions: when the considered Roesser model is linear, structural stability and exponential stability are equivalent and both of them imply asymptotic stability. However asymptotic stability does not imply exponential or structural stability. The proof is based on previous results obtained for Fornasini-Marchesini models and requires careful switching between both models.

Keywords: Boolean control networks and logic networks, Genetic regulatory systems
Abstract: Boolean network is a popular and well-established modelling framework for gene regulatory networks. The steady- state behaviour of Boolean networks can be described as attractors, which are hypothesised to characterise cellular phenotypes. In this work, we study the target control problem of Boolean networks, which has important applications for cellular reprogramming. More specifically, we want to reduce the total number of attractors of a Boolean network to a single target attractor. Different from existing approaches to solving control problems of Boolean networks with node perturbations, we aim to develop an approach utilising edgetic perturbations. Namely, our objective is to modify the update functions of a Boolean network such that there remains only one attractor. The design of our approach is inspired by Thomas’ first rule, and we primarily focus on the removal of cycles in the interaction graph of a Boolean network. We further use results in the literature to only remove positive cycles which are responsible for the appearance of multiple attractors. We apply our solution to a number of real-life biological networks modelled as Boolean networks, and the experimental results demonstrate its efficacy and efficiency.

Keywords: Discrete event systems, Supervisory control
Abstract: A certain intersection-based architecture for decentralized supervisory control of discrete event systems was proposed in the literature. This existing intersection-based architecture can be regarded as an anti-permissive one. In this paper, we propose a dual architecture, named the permissive intersection-based architecture, and develop a general architecture by combining the anti-permissive and permissive ones. Then, we define a general notion of SEI-coobservability as a part of necessary and sufficient conditions under which the specification is achieved in the developed general architecture and show how to verify it.

Keywords: Automata, Discrete event systems
Abstract: In this paper, we revisit the verification of strong K-step opacity (K-SSO) for partially-observed discrete-event systems modeled as nondeterministic finite-state automata. As a stronger version of the standard K-step opacity, K-SSO requires that an intruder cannot make sure whether or not a secret state has been visited within the last K observable steps. To efficiently verify K-SSO, we propose a new concurrent-composition structure, which is a variant of our previously-proposed one. Based on this new structure, we design an algorithm for deciding K-SSO and prove that the proposed algorithm not only reduces the time complexity of the existing algorithms, but also does not depend on the value of K. Furthermore, a new upper bound on the value of K in K-SSO is derived, which also reduces the existing upper bound on K in the literature. Finally, we illustrate the proposed algorithm by a simple example.

Keywords: Discrete event systems, Automata, Supervisory control
Abstract: In this paper, we revisit the fault diagnosis problem of discrete-event systems (DES) under non-deterministic observations. Non-deterministic observation is a general observation model that includes the case of intermittent loss of observations. In this setting, upon the occurrence of an event, the sensor reading may be non-deterministic such that a set of output symbols are all possible. Existing works on fault diagnosis under non-deterministic observations require to consider all possible observation realizations. However, this approach includes the case where some possible outputs are permanently disabled. In this work, we introduce the concept of output fairness by requiring that, for any output symbols, if it has infinite chances to be generated, then it will indeed be generated infinite number of times. We use an assume-guarantee type of linear temporal logic formulas to formally describe this assumption. A new notion called output-fair diagnosability (OF-diagnosability) is proposed. An effective approach is provided for the verification of OF-diagnosability. We show that the proposed notion of OF-diagnosability is weaker than the standard definition of diagnosability under non-deterministic observations, and it better captures the physical scenario of observation non-determinism or intermittent loss of observations.

Keywords: Supervisory control, Resilient Control Systems
Abstract: One key challenge of cybersecurity of discrete event systems (DES) is how to ensure system resilience against sensor and/or actuator attacks, which may tamper data integrity and service availability. In this paper we discuss decidability issues related to smart sensor attacks. We first present a sufficient and necessary condition that ensures the existence of a smart sensor attack, which reveals a novel demand-supply relationship between an attacker and a controlled plant, represented as a set of risky pairs. Each risky pair consists of a damage string desired by the attacker and an observable sequence feasible in the supervisor such that the latter induces a sequence of control patterns, which allows the damage string to happen. It turns out that each risky pair can induce a smart weak sensor attack. Next, we show that, when the plant, supervisor and damage language are regular, it is possible to remove all such risky pairs from the plant behaviour, via a genuine encoding scheme, upon which we establish our key result that the existence of a nonblocking supervisor resilient against all smart sensor attacks is decidable.

Keywords: Formal Verification/Synthesis, Predictive control for linear systems, Constrained control
Abstract: This paper considers discrete-time linear systems with bounded additive disturbances, and studies the convergence properties of the backward reachable sets of robust controlled invariant sets (RCIS). Under a simple condition, we prove that the backward reachable sets of an RCIS are guaranteed to converge to the maximal RCIS in Hausdorff distance, with an exponential convergence rate. When all sets are represented by polytopes, this condition can be checked numerically via a linear program. We discuss how the developed condition generalizes the existing conditions in the literature for (controlled) invariant sets of systems without disturbances (or without control inputs).

Keywords: Variable-structure/sliding-mode control, Nonlinear output feedback, Sampled-data control
Abstract: A low-chattering bihomogeneous discretization method is proposed for homogeneous sliding mode control, which combines two different homogeneity degrees and coordinate weights distributions. The new method is explicit, always preserves system trajectories and accuracies in simulation and control applications. When properly used, it significantly decreases system vibrations in the absence of sampling noises. Numeric experiments demonstrate the efficacy of the approach.

Keywords: Variable-structure/sliding-mode control, Uncertain systems
Abstract: This paper presents a novel optimal control approach for systems represented by a multi-model, i.e., a finite set of models, each one corresponding to a different operating point. The proposed control scheme is based on the combined use of model predictive control (MPC) and first order integral sliding mode control. The sliding mode control component plays the important role of rejecting matched uncertainty terms possibly affecting the plant, thus making the controlled equivalent system behave as the nominal multi-model. A min-max multi-model MPC problem is solved using the equivalent system without further robustness oriented add-ons. In addition, the MPC design is performed so as to keep the computational complexity limited, thus facilitating the practical applicability of the proposal. Simulation results show the effectiveness of the proposed control approach.

Keywords: Variable-structure/sliding-mode control, Uncertain systems
Abstract: This paper deals with the design of a novel neural network based integral sliding mode (NN-ISM) control for nonlinear systems with uncertain drift term and control effectiveness matrix. Specifically, this paper extends the classical integral sliding mode control law to the case of unknown nominal model. The latter is indeed reconstructed by two deep neural networks capable of approximating the unknown terms, which are instrumental to design the so-called integral sliding manifold. In the paper, the ultimate boundedness of the system state is formally proved by using Lyapunov stability arguments, thus providing the conditions to enforce practical integral sliding modes. The possible generation of ideal integral sliding modes is also discussed. Moreover, the effectiveness of the proposed NN-ISM control law is assessed in simulation relying on the classical Duffing oscillator.

Keywords: Variable-structure/sliding-mode control, Nonlinear systems
Abstract: This article proposes a comparative study of three high order sliding mode control (HOSMCs) algorithms for the 5MW tensioned-leg platform (TLP) based floating offshore wind turbines (FOWT). The control objectives are to regulate the rotor speed to its nominal value and to reduce the fluctuations of the platform pitch angle. A nonlinear control-oriented model of the 5 MW TLP-based FOWT has been selected from the literature and adapted for nonlinear controller design. Then, a continuous-twisting algorithm (CTA), a quasi-continuous homogeneous algorithm (QCA) and a super-twisting algorithm (STA) have been designed to meet the control objectives in turbulent and high wind speed conditions. Finally, these robust nonlinear controllers have been validated in simulations and compared with each other using OpenFAST code in MATLAB/Simulink.

Keywords: Variable-structure/sliding-mode control, Power systems, Power electronics
Abstract: The worldwide transition to climate-friendly energy systems entails the substitution of conventional energy generation based on synchronous generators by renewable energy sources based on power electronics. As a consequence, the overall inertia of the grid decreases, resulting in high volatility of the frequency and posing new challenges for the estimation of the latter quantity. To address this problematic, in the present paper a phase-locked-loop (PLL) based on the super-twisting algorithm is considered for the estimation of the phase angle and time-varying frequency of a symmetric three-phase signal. A rigorous proof of the algorithm's exact convergence in the presence of a fast-varying frequency together with tuning rules for its gains are derived by means of Lyapunov theory. Additionally, an estimate of the region of attraction is provided. The effectiveness of the proposed tuning method is illustrated in numerical simulations, while comparing its performance against a standard synchronous reference frame PLL.

Keywords: Variable-structure/sliding-mode control
Abstract: In this paper, a new framework for the discretization of the super-twisting algorithm is developed. The proposed discretization scheme is not based on the underlying differential equations, but uses the variational formulation of the problem. Discrete-time versions derived in the proposed fashion exhibit a great tracking of the continuous-time energy decay rate and give a good approximation of the continuous-time system. Furthermore, the paper presents a stability proof for an exemplary algorithm, a semi-implicit version of the super-twisting algorithm derived by using variational integrators.

Keywords: Biological systems, Systems biology, Modeling
Abstract: We consider a class of epidemiological models with an arbitrary number of infected compartments. We show that the logarithmic derivatives of the infected states converge to a consensus; this property rigorously explains the feature empirically observed in real epidemic data: the logarithms of the state variables associated with infected categories tend to behave as "parallel lines". We introduce and characterise the class of contagion functions, i.e., linear co-positive functions of the state variables that decrease (resp. increase) when the reproduction number is smaller (resp. larger) than 1. Finally, we analyse the generalised epidemiological model by considering the susceptible state variable along with a variable that aggregates all the infected compartments: this leads to an auxiliary planar system, governed by two differential inclusions, which has the same structure as the two-dimensional SI model and whose coefficients are functions of the original variables. We prove that well known properties of the classical SI model still hold in this generalised case.

Keywords: Biological systems, Observers for nonlinear systems, Lyapunov methods
Abstract: Observation and identification are important issues for the practical use of compartmental models of epidemic dynamics. They are usually evaluated based on the number of infected individuals (the prevalence) or the newly infected cases (the incidence). We are interested in a general question: may the measure of the number of primo-infected individuals and the prevalence improve state estimation? To study this question, we analyze in this paper a simple model of infection with waning immunity and, consequently, the possibility of reinfections. A class of nonlinear observers is built for this model, and tractable sufficient conditions on the gain matrices are established, ensuring asymptotic convergence of the state estimate towards its actual value. Numerical simulations illustrate the method.

Keywords: Biological systems, Observers for nonlinear systems, Nonlinear systems
Abstract: Although an appropriate choice of measured state variables may ensure observability, designing state observers for the state estimation of epidemic models remains a challenging task. Epidemic spread is a nonlinear process, often modeled as the law of mass action, which is of a quadratic form; thus, on a compact domain, its Lipschitz constant turns out to be local and relatively large, which renders the Lipschitz-based design criteria of existing observer architectures infeasible. In this paper, a novel observer architecture is proposed for the state estimation of a class of nonlinear systems that encompasses the deterministic epidemic models. The proposed observer offers extra leverage to reduce the influence of nonlinearity in the estimation error dynamics, which is not possible in other Luenberger-like observers. Algebraic Riccati inequalities are derived as sufficient conditions for the asymptotic convergence of the estimation error to zero under local Lipschitz and generalized Lipschitz assumptions. Equivalent linear matrix inequality formulations of the algebraic Riccati inequalities are also provided. The efficacy of the proposed observer design is illustrated by its application on the celebrated SIDARTHE-V epidemic model.

Keywords: Emerging control applications, Control applications, Optimal control
Abstract: We study the impact of model parameter uncertainty on optimally mitigating the spread of epidemics. We capture the epidemic spreading process using a susceptible infected-removed (SIR) epidemic model and consider testing for isolation as the control strategy. We use a testing strategy to remove (isolate) a portion of the infected population. Our goal is to keep the daily infected population below a certain level, while minimizing the total number of tests. Distinct from existing works on leveraging optimal control strategies in epidemic spreading, we propose a testing strategy by overestimating the seriousness of the epidemic and study the feasibility of the system under the impact of model parameter uncertainty. Compared to the optimal testing strategy, we establish that the proposed strategy under model parameter uncertainty will flatten the curve effectively but require more tests and a longer time period.

Keywords: Estimation, Simulation, Markov processes
Abstract: Diagnostic tests have proven to be a critical tool in controlling the progression of a virus. In this paper, we formulate the testing of a homogeneous population as an optimal control problem. The population state, given by the distribution of agents' viral states in a compartmental model, is assumed to be unknown. Information regarding the population state is provided via noisy tests, which are allocated from a stockpile whose size is updated via a stochastic process. The objective of the control problem is to allocate tests so as to minimize uncertainty of the underlying population state over a finite horizon. As such, the control problem is cast as a POMDP with a negative entropy reward function. We study various heuristic policies and investigate conditions under which each heuristic performs best.

Keywords: Stability of nonlinear systems, Game theory, Emerging control applications
Abstract: A recent article that combines normalized epidemic compartmental models and population games put forth a system theoretic approach to capture the coupling between a population's strategic behavior and the course of an epidemic. It introduced a payoff mechanism that governs the population's strategic choices via incentives, leading to the lowest endemic proportion of infectious individuals subject to cost constraints. Under the assumption that the disease death rate is approximately zero, it uses a Lyapunov function to prove convergence and formulate a quasi-convex program to compute an upper bound for the peak size of the population's infectious fraction. In this article, we generalize these results to the case in which the disease death rate is nonnegligible. This generalization brings on additional coupling terms in the normalized compartmental model, leading to a more intricate Lyapunov function and payoff mechanism. Moreover, the associated upper bound can no longer be determined exactly, but it can be computed with arbitrary accuracy by solving a set of convex programs.

Keywords: Constrained control, Nonlinear systems, Robust control
Abstract: A universal approximation-free adaptive prescribed performance control scheme is designed for unknown SISO nonlinear systems with input saturation. The proposed control method introduces a compromising relaxation of output performance specifications depending on the input limitations. Given the conflicting nature between input and output constraints, the stability properties are inevitably local. In this respect, a sufficient stability condition for the closed-loop system is provided through theoretical analysis. Owing to the adopted prescribed performance control technique, the satisfaction of the aforementioned stability properties guarantees the desired trade-off between input and output constraints. Moreover, no hard calculations are needed, neither for the controller nor for the adaptive law, maintaining the complexity of the control algorithm relatively low. Finally, the proposed approach is clarified and verified by various simulation studies.

Keywords: Optimal control, Aerospace
Abstract: This paper studies a vertical powered descent problem in the context of planetary landing, considering glide-slope and thrust pointing constraints and minimizing any final cost. After stating the Max-Min-Max or Max-Singular-Max form of the optimal control deduced from the Pontryagin Maximum Principle, it theoretically analyzes the optimal trajectory for a more specific problem formulation to show that there can be at most one contact or boundary interval with the state constraint on each Max or Min arc.

Keywords: Constrained control, Stability of nonlinear systems, Lyapunov methods
Abstract: We design static anti-windup gains to mitigate the effect of input saturation in linear output feedback closed loops. The design is conducted with the help of a non-quadratic Lyapunov function involving sign-indefinite quadratic forms, which allows for additional degrees of freedom to be exploited, for the anti-windup gain design. Synthesis conditions, combining the use of sign-indefinite quadratic forms and several sector bound properties are stated in the form of bilinear matrix inequalities ensuring global exponential stability of the closed-loop system. An iterative design algorithm is then devised, based on the resolution of a sequence of semidefinite programs. The superiority of the proposed technique over classical quadratic constructions is illustrated on an example borrowed from the literature, where quadratic positive definite functions are ineffective.

Keywords: Constrained control, Robust control, Stability of nonlinear systems
Abstract: Non-overshooting stabilization is a form of safe control where the setpoint chosen by the user is at the boundary of the safe set. Exponential non-overshooting stabilization, including suitable extensions to systems with deterministic and stochastic disturbances, has been solved by the second author and his coauthors. In this paper we develop homogeneous feedback laws for fixed-time nonovershooting stabilization for nonlinear systems that are input-output linearizable with a full relative degree, i.e., for systems that are diffeomorphically equivalent to the chain of integrators. These homogeneous feedback laws can also assume the secondary role of `fixed-time safety filters' (FxTSf filters) which keep the system within the closed safe set for all time but, in the case where the user's nominal control commands approach to the unsafe set, allow the system to reach the boundary of the safe set no later than a desired time that is independent of nominal control and independent of the value of the state at the time the nominal control begins to be overridden.

Keywords: Optimization, Control applications, Linear systems
Abstract: We consider the problem of sample-based feedback motion planning from measurements affected by systematic errors. Our previous work presented output feedback controllers that use measurements from landmarks in the environment to navigate through a cell-decomposable environment using duality, Control Lyapunov and Barrier Functions (CLF, CBF), and Linear Programming. In this paper, we build on this previous work with a novel strategy that allows the use of measurements affected by systematic errors in perceived depth (similarly to what might be generated by vision-based sensors), as opposed to accurate displacements measurements. As a result, our new method has the advantage of providing more robust performance (with quantitative guarantees) when inaccurate sensors are used. We test the proposed algorithm in the simulation to evaluate the performance limits of our approach predicted by our theoretical derivations.

Keywords: Optimization algorithms, Predictive control for linear systems, Fault tolerant systems
Abstract: When Model Predictive Control (MPC) is used in real-time to control linear systems, quadratic programs (QPs) need to be solved within a limited time frame. Recently, several parametric methods have been proposed that certify the number of computations active-set QP solvers require to solve these QPs. These certification methods, hence, ascertain that the optimization problem can be solved within the limited time frame. A shortcoming in these methods is, however, that they do not account for numerical errors that might occur internally in the solvers, which ultimately might lead to optimistic complexity bounds if, for example, the solvers are implemented in single precision. In this paper we propose a general framework that can be incorporated in any of these certification methods to account for such numerical errors.

Keywords: Energy systems, Power systems, Control of networks
Abstract: A recent formalization of thermodynamics under a dynamical systems approach introduces an axiom restricting the direction of power transmission between two subsystems, reflecting heat transfer from hot to cold bodies. This axiom enables precise results paralleling the statements of classical thermodynamics, including its second law, which places a limit on the amount of work that may be transferred to across system boundaries beyond that imposed by energy conservation. Systems exhibiting non-diffusive power transfer, including those with Hamiltonian dynamics are ruled out. Given that power networks with Hamiltonian dynamics fail the directionality axiom, are they still subject to limitations on their ability to perform work on the surroundings? This paper shows that such systems can satisfy a version of the above axiom involving averages over periodic regimes, revealing limitations on external power transfer and allowing a definition of second-law efficiency and a cyclic interpretation of energy equipartition. Focus is on a class of linear port-Hamiltonian systems, with frequency-domain methods used to describe the pertinent average quantities. The ability to establish an order relationship between the weighted average kinetic and potential energies has a central role. We show that this can be cast as a generalized eigenvalue problem.

Keywords: Energy systems, Power systems, Smart grid
Abstract: This paper deals with a power demand-supply management based on negawatt trading, and the purpose of this research is to keep demand supply balance and to minimize the social cost of negawatt trading, balancing generator, and power flow.

In this research, consumer and Independent System Operator participate in the electricity market, and each consumer acts based on Regret Matching which is one of the learning algorithms.

Furthermore, as the incentive design of negawatt trading, we consider strategy-proofness and individual rationality using VCG mechanism design.

Then, we propose a novel negawatt trading algorithm based on Regret Matching and VCG mechanism design. Finally, simulation results show the effectiveness of the proposed algorithm.

Keywords: Energy systems, Power systems
Abstract: The problem of heat system pricing is considered. A direct extension of locational marginal prices (LMP) in electricity markets to heat systems may lead to revenue inadequate issues. The underlying reason for such a problem is that, unlike electric power, heat has different grades and cannot be considered as homogenized commodity. Accordingly, an energy-grade double pricing rule is proposed in this paper. Heat energy and grade prices are explained as the shadow prices related to the nodal heat balance constraints and temperature requirements constraints at the optimal solution. The resulting merchandise surplus at each dispatch interval can be decomposed into several explainable parts, namely, congestion rent, impact from the last period, and impact from the upcoming period. And the total merchandise surplus over all dispatch intervals can be decomposed into several non-negative interpretable parts, including congestion rent and impact from the initial state, thus guaranteeing the revenue adequacy for the heat system operator. Simulations verify the effectiveness of the proposed mechanism.

Keywords: Energy systems, Power systems
Abstract: Distributed energy resources (DERs) such as solar panels have small supply capacities and cannot be directly integrated into wholesale markets. So, the presence of an intermediary is critical. The intermediary could be a profit-seeking entity (called the aggregator) that buys DER supply from prosumers, and then sells them in the wholesale electricity market. Thus, DER integration has an influence on wholesale market prices, demand, and supply. The purpose of this article is to shed light onto the impact of efficient DER aggregation on the market power of conventional generators. Firstly, under efficient DER aggregation, we quantify the social welfare gap between two cases: when conventional generators are truthful, and when they are strategic. We also do the same when DERs are not present. Secondly, we show that the gap due to market power of generators in the presence of DERs is smaller than the one when there is no DER participation. Finally, we provide explicit expressions of the gaps and conduct numerical experiments to gain deeper insights. The main message of this article is that market power of conventional generators can be mitigated by adopting an efficient DER aggregation model.

Keywords: Energy systems, Smart grid, Machine learning
Abstract: We consider the problem of learning the energy disaggregation signals for residential load data. Such a task is referred as non-intrusive load monitoring (NILM), and in order to find individual devices' power consumption profiles based on aggregated meter measurements, a machine learning model is usually trained based on large amount of training data coming from a number of residential homes. Yet collecting such residential load datasets requires both huge efforts and customers' approval on sharing metering data, while load data coming from different regions or electricity users may exhibit heterogeneous usage patterns. Both practical and privacy concerns make training a single, centralized NILM model challenging. In this paper, we propose a decentralized and task-adaptive learning scheme for NILM tasks, where nested meta-learning and federated learning steps are designed for learning task-specific models collectively. Simulation results on benchmark dataset validate proposed algorithm's performance in efficiently inferring the appliance-level consumption for a variety of homes and appliances.

Keywords: Energy systems, Building and facility automation, Optimization
Abstract: This paper studies the online optimal energy management problem for the building heating system. This system is driven by electric heat pumps, and equipped with the thermal energy storage (TES) that provides heat flexibility. The temporal coupling introduced by building temperature dynamics and TES energy level dynamics, together with the uncertainties of outdoor temperature and real-time electricity price, pose the optimal energy management problem a computationally challenging one. Hence, based on the Lyapunov optimization techniques, we propose an online algorithm to minimize the long-term time average cost of the heating system. Our algorithm can ensure the building temperature constraints and TES energy level constraints are satisfied all the time. The maximum optimality gap of our algorithm is also rigorously derived. Simulation results show that this algorithm can achieve user comfort and reduce energy cost effectiv

Keywords: Game theory
Abstract: Fictitious play has recently emerged as the most accurate scalable algorithm for approximating Nash equilibrium strategies in multiplayer games. We show that the degree of equilibrium approximation error of fictitious play can be significantly reduced by carefully selecting the initial strategies. We present several new procedures for strategy initialization and compare them to the classic approach, which initializes all pure strategies to have equal probability. The best-performing approach, called maximin, solves a nonconvex quadratic program to compute initial strategies and results in a nearly 75% reduction in approximation error compared to the classic approach when 5 initializations are used.

Keywords: Game theory
Abstract: In this paper, we propose a novel method for designing a synchronous strategy updating rule to guarantee the almost sure convergence of the Nash equilibria in evolutionary congestion games. Different from much literature in which the full information of the game is used, in this paper, the information obtained by each player only contains the resource allocation of the last play which is sufficient to guarantee the convergence. The main idea is embedding the player’s time-varying inertia factor in the traditional myopic best response adjustment rule to improve the evolutionary dynamic. The developed theoretical results are simulated by solving a routing problem.

Keywords: Game theory, Transportation networks
Abstract: Whilst overloaded transportation systems bear a significant impact on everyone's welfare, governments strive to improve their performances. Amongst the many solutions proposed, congestion pricing is becoming increasingly popular as it has the potential to reduce congestion by indirectly influencing the drivers' routing choices. Commonly advocated for, the marginal cost mechanism ensures that self-interested decision-making results in optimal system performances. However, such a mechanism suffers from three important drawbacks in that i) it requires levying tolls on every road, ii) it does not allow for upper bounds on the magnitude of the tolls, and iii) it is flow-dependent. In response to these challenges, researchers have introduced the restricted network tolling problem, seeking constant tolls of bounded magnitude that induce equilibria with a small social cost. However, tolls designed through this approach are tailored to a specific traffic demand, resulting in a design that has the potential to exacerbate the very issue it set out to solve, if the demand changes. Our work addresses this issue and aims at infusing robustness guarantees to the restricted network tolling problem. We do so by seeking tolls that have good performance over past demand realizations, and leverage recent results in scenario optimization to equip our design with formal generalization guarantees.

Keywords: Game theory
Abstract: In adversarial interactions, one is often required to make strategic decisions over multiple periods of time, wherein decisions made earlier impact a player's competitive standing as well as how choices are made in later stages. In this paper, we study such scenarios in the context of General Lotto games, which models the competitive allocation of resources over multiple battlefields between two players. We propose a two-stage formulation where one of the players has reserved resources that can be strategically pre-allocated across the battlefields in the first stage. The pre-allocation then becomes binding and is revealed to the other player. In the second stage, the players engage by simultaneously allocating their real-time resources against each other. The main contribution in this paper provides complete characterizations of equilibrium payoffs in the two-stage game, revealing the interplay between performance and the amount of resources expended in each stage of the game. We find that real-time resources are at least twice as effective as pre-allocated resources. We then determine the player's optimal investment when there are linear costs associated with purchasing each type of resource before play begins, and there is a limited monetary budget.

Keywords: Game theory, Transportation networks, Optimization
Abstract: We investigate the problem of platoon matching through the lens of complexity and efficiency. Specifically, we consider a noncooperative game among a number of vehicles that decide to form or not to form a platoon on a single road. To characterize the computational complexity of calculating the Nash equilibria in this game, we obtain a general upper bound for the length of any best response sequence. Then, we completely characterize the Nash equilibrium when the vehicles are interchangeable. Regarding the efficiency, we show that platooning games can be very inefficient in the worst case, as they can always have zero price of anarchy even when the vehicles have the same cost function.

Keywords: Game theory, Uncertain systems, Traffic control
Abstract: The inherent uncertainties in the ride-hailing market complicate the pricing strategies of on-demand platforms that compete each other to offer a mobility service while striving to maximize their profit. Looking at this problem as a stochastic generalized Nash equilibrium problem (SGNEP), we design a distributed, stochastic equilibrium seeking algorithm with Tikhonov regularization to find an optimal pricing strategy. The proposed iterative scheme does not require a potentially infinite number of samples of the random variable to perform the stochastic approximation, thus making it appealing from a practical perspective. Moreover, we show that the algorithm returns a Nash equilibrium under mere monotonicity assumptions and a careful choice of the step size sequence, obtained by exploiting the specific structure of the SGNEP at hand.

Keywords: Nonlinear output feedback, LMIs, Stability of nonlinear systems
Abstract: The redesign of output feedback controllers for linear systems based on adaptive saturation (stubborn) and dead-zone redesign is investigated by showing that input-to-state stability holds in closed loop upon the satisfaction of linear matrix inequalities. Such results are obtained by using sector conditions that are involved in the Lyapunov analysis in order to ensure input-to-state stability. A simulation case study shows the effectiveness of the proposed redesign in denoising and outlier attenuation with increased accuracy and precision.

Keywords: Nonlinear output feedback, Hybrid systems, Observers for nonlinear systems
Abstract: For systems that are not observable at the very equilibrium of interest to be stabilized, output-feedback stabilization is considerably challenging. In this paper we solve this control problem for the case-study of a second-order system that is bilinear and affine, both in the input and the output, but it is unobservable at the target equilibrium. The case-study is representative of a well-studied class of non-uniformly observable systems and stems from automotive control. Our main contribution is a novel certainty-equivalence hybrid controller that achieves asymptotic stabilization semiglobally. The controller relies on a switched observer that estimates the state, provided that the latter is ‘kept away’ from the singular equilibrium. To achieve both competing tasks, stabilization and estimation, the controller also relies on the keen construction of a piece-wise constant, converging, reference. Our main results are illustrated via numerical simulations on a meaningful example.

Keywords: Nonlinear systems, Constrained control, Uncertain systems
Abstract: This paper proposes a funnel control method under time-varying hard and soft output constraints. First, an online funnel planning scheme is designed that generates a constraint consistent funnel, which always respects hard (safety) constraints, and soft (performance) constraints are met only when they are not conflicting with the hard constraints. Next, the prescribed performance control method is employed for designing a robust low-complexity funnel-based controller for uncertain nonlinear Euler-Lagrangian systems such that the outputs always remain within the planned constraint consistent funnels. Finally, the results are verified with a simulation example of a mobile robot tracking a moving object while staying in a box-constrained safe space.

Keywords: Nonlinear systems, Nonlinear output feedback, Control over communications
Abstract: This note studies the problem of master-slave phase synchronization between two oscillators described by Andronov-Hopf equations, where pulse-like signals, which can be heavily corrupted by disturbances, are used for communication. The synchronization conditions are established using the Lyapunov function theory, and the synchronization error bounds are evaluated. The results are motivated and illustrated by designing a synchronization scheme for visible light communication (VLC), where a periodic sequence is usually sent at the beginning of the communication to synchronize the clocks of the transmitter and the receiver.

Keywords: Nonlinear systems, Nonlinear output feedback, Modeling
Abstract: We extend the concept of model approximation via wienerization to systems in nonlinear control canonical normal form. We elaborate on the conditions for, and implications of, analytically separating nonlinear input affine dynamical systems in state space form in a static part plus a dynamic one. In doing so, we discuss under which conditions Wiener models may approximate the resulting models well. More precisely, we report that a specific bijective transformation of the original nonlinear model will separate the system into a multidimensional state space structure for which it is possible to compare nonlinear Wiener control against linear control for underactuated nonlinear systems. We finally assess how the former type of control has better closed-loop performance than the latter by means of quantitative examples.

Keywords: Nonlinear systems, Lyapunov methods, Stability of nonlinear systems
Abstract: In this paper, we consider the L^1-input-to-state stability (L^1-ISS) for nonlinear systems with L^1-inputs. The work is twofold: (i) we provide Lyapunov characterizations of L^1-ISS by using the classical Lyapunov method (CLM); (ii) we introduce a novel method, namely, the approximative Lyapunov method (ALM), for the L^1-ISS analysis of nonlinear systems. In particular, unlike the CLM that mainly focuses on the construction of Lyapunov functions (LFs), the ALM amounts to constructing an approximation of LFs by using a convex function, which provides a new methodology for stability analysis of nonlinear systems. In order to illustrate the difference of the two methods, we provide an example of the L^1-ISS analysis for a class of nonlinear systems with L^1-inputs.

Keywords: Nonholonomic systems, Distributed control, Autonomous robots
Abstract: Low-complexity control (i.e. control avoiding complex function approximation and dynamic adaptive laws) is important in small mobile robots, which cannot be equipped with high-end communication and control hardware. However, the literature has shown that attaining low-complexity control is challenging in the presence of formation constraints such as connectivity maintenance, safety and performance constraints. This work proposes a low-complexity control design (in the framework of funnel control) which is able to handle a large set of formation constraints, while merely relying on static nonlinear feedback, without any function approximation nor dynamic adaptation mechanism. A simulation study further illustrates the method as compared to the state of the art.

Keywords: Control system architecture, Optimal control, Power electronics
Abstract: A model reference control allocation (CA) method is modified to become a control allocation method with integrator (CAI), it significantly improves control performance while readily integrating with existing methods. In general, CA methods take advantage of the redundancy of an overactuated system to achieve control objectives while respecting actuator limits. The method of this paper adds integral compensation to a CA method for multivariable model reference control with the special property that the closed-loop behavior remains the same. The transfer function matrix is preserved, yet zero static error results when experiencing small parametric uncertainties and constant disturbances. The application of the concept to the coordinated control of multiple buck converters feeding a common load is considered. A simulation of the system confirms the benefits of the proposed method.

Keywords: Randomized algorithms, Autonomous robots, Lyapunov methods
Abstract: Sampling-based algorithms, such as Rapidly Exploring Random Trees (RRT) and its variants, have been used extensively for motion planning. Control barrier functions (CBFs) have been recently proposed to synthesize controllers for safety-critical systems. In this paper, we combine the effectiveness of RRT-based algorithms with the safety guarantees provided by CBFs in a method called CBF-RRT^ast. CBFs are used for local trajectory planning for RRT^ast, avoiding explicit collision checking of the extended paths. We prove that CBF-RRT^ast preserves the probabilistic completeness of RRT^ast. Furthermore, in order to improve the sampling efficiency of the algorithm, we equip the algorithm with an adaptive sampling procedure, which is based on the cross-entropy method (CEM) for importance sampling (IS). The procedure exploits the tree of samples to focus the sampling in promising regions of the configuration space. We demonstrate the efficacy of the proposed algorithms through simulation examples.

Keywords: Smart structures, Control applications
Abstract: A control system is termed ``self-powered" if it can operate solely using the energy extracted from the response of the plant in which it is embedded. Due to dissipative losses in the control transducers, associated power electronics, and the energy storage subsystem, self-powered control laws must adhere to feasibility conditions that are more stringent than classical passivity. In this paper, we present a two-stage framework for the synthesis of self-powered feedback controllers, assuming a stochastic disturbance model and a mean-square performance objective. In the first stage, we optimize an LTI, colocated feedback law that adheres to the self-powered feasibility constraint. In the second stage, we design a nonlinear, full-state feedback controller that is guaranteed to improve upon the closed-loop performance obtained by the first stage design and maintain feasibility. We explore a numerical example that demonstrates a novel application of self-powered control involving the virtual ``coupling" of two civil structures excited by an earthquake disturbance.

Keywords: Optimal control, Game theory
Abstract: In this letter, we study dynamic game optimal control with imperfect state observations and introduce an iterative method to find a local Nash equilibrium. The algorithm consists of an iterative procedure combining a backward recursion similar to minimax differential dynamic programming and a forward recursion resembling a risk-sensitive Kalman smoother. A coupling equation renders the resulting control dependent on the estimation. In the end, the algorithm is equivalent to a Newton step but has linear complexity in the time horizon length. Furthermore, a merit function and a line search procedure are introduced to guarantee convergence of the iterative scheme. The resulting controller reasons about uncertainty by planning for the worst case disturbances. Lastly, the low computational cost of the proposed algorithm makes it a promising method to do output-feedback model predictive control on complex systems at high frequency. Numerical simulations on realistic robotic problems illustrate the risk-sensitive behavior of the resulting controller.

Keywords: Optimal control, Constrained control, Numerical algorithms
Abstract: In this article we introduce QuITO - Quasi-Interpolation based Trajectory Optimization, a direct multiple shooting algorithm to solve a class of constrained nonlinear minimum energy optimal control problems. This technique is based on the theory of approximate approximations - a quasi-interpolation scheme. We parameterize the control trajectory using the quasi-interpolation formula, and we discretize the optimal control problem using the collocation points on a uniform cardinal grid, thereby transcribing the optimal control problem (OCP) into a nonlinear program (NLP). Several examples are provided to show the numerical fidelity of the algorithm.

Keywords: Manufacturing systems and automation, Network analysis and control, Stochastic optimal control
Abstract: In this paper, we formulate an optimal ordering policy as a stochastic control problem where each firm decides the amount of input goods to order from their upstream suppliers based on the current inventory level of its output good. For this purpose, we provide a closed-form solution for the optimal request of the raw materials given a fixed production policy. We implement the proposed policy on a 15-firm acyclic network based on a real product supply chain. we first simulate ideal demand situations, and then we implement demand-side shocks (i.e., demand levels outside of those considered in the policy formulation) and supply-side shocks (i.e., halts in production for some suppliers) to evaluate the robustness of the proposed policies.

Keywords: Learning, Optimization, Markov processes
Abstract: In tabular multi-agent reinforcement learning with average-cost criterion, a team of agents sequentially interacts with the environment and observes local incentives. We focus on the case that the global reward is a sum of local rewards, the joint policy factorizes into agents' marginals, and full state observability. To date, few global optimality guarantees exist even for this simple setting, as most results yield convergence to stationarity for parameterized policies in large/possibly continuous spaces. To solidify the foundations of MARL, we build upon linear programming (LP) reformulations, for which stochastic primal-dual methods yield a model-free approach to achieve emph{optimal sample complexity} in the centralized case. We develop multi-agent extensions, whereby agents solve their local saddle point problems and then perform local weighted averaging. We establish that the sample complexity to obtain near-globally optimal solutions matches tight dependencies on the cardinality of the state and action spaces, and exhibits classical scalings with respect to the network in accordance with multi-agent optimization. Experiments corroborate these results in practice.

Keywords: Agents-based systems, Optimal control, Smart grid
Abstract: This paper studies self-sustained dynamic multiagent systems (MAS) for decentralized resource allocation operating at a competitive equilibrium over a finite horizon. The utility of resource consumption, along with the income from resource exchange, forms each agent’s payoff which is aimed to be maximized. Each utility function is parameterized by individual preferences which can be designed by agents independently. By shaping these preferences and proposing a set of utility functions, we can guarantee that the optimal resource price at the competitive equilibrium always remains socially acceptable, i.e., it never violates a given threshold that indicates affordability. First, we show this problem is solvable at the conceptual level under some convexity assumptions. Then, as a benchmark case, we consider quadratic MAS and formulate the associated social shaping problem as a multi-agent LQR problem which enables us to propose explicit utility sets using quadratic programming and dynamic programming. Finally, a numerical algorithm is presented for calculating the range of the preference function parameters which guarantee a socially accepted price. Some illustrative examples are given to examine the effectiveness of the proposed methods.

Keywords: Optimization, Agents-based systems, Large-scale systems
Abstract: Distributed nonconvex optimization problems underlie many applications in learning and autonomy, and such problems commonly face asynchrony in agents’ computations and communications. When delays in these operations are bounded, they are called partially asynchronous. In this paper, we present an uncoordinated stepsize selection rule for partially asynchronous block coordinate descent that only requires local information to implement, and it leads to faster convergence for a class of nonconvex problems than existing stepsize rules, which require global information in some form. The problems we consider satisfy the error bound condition, and the stepsize rule we present only requires each agent to know (i) a certain type of Lipschitz constant of its block of the gradient of the objective and (ii) the communication delays experienced between it and its neighbors. This formulation requires less information to be available to each agent than existing approaches, typically allows for agents to use much larger stepsizes, and alleviates the impact of stragglers while still guaranteeing convergence to a stationary point. Simulation results provide comparisons and validate the faster convergence attained by the stepsize rule we develop.

Keywords: Learning, Networked control systems, Communication networks
Abstract: We consider the problem of distributed non-Bayesian learning (or hypothesis testing) where a group of agents interacts over a peer-to-peer network to identify the true state of the world from a finite set of hypotheses, based on a series of stochastic signals that each agent receives. Prior work on this problem has provided distributed algorithms that guarantee asymptotic learning of the true state, with corresponding efforts to improve the rate of learning. In this paper, we first argue that one can readily modify existing asymptotic learning algorithms to enable learning in finite time, effectively yielding arbitrarily large (asymptotic) rates. Furthermore, we show that such finite-time learning can be achieved via a simple algorithm which only requires the agents to exchange a binary vector (of length equal to the number of possible hypotheses) with their neighbors at each time-step. Finally, we show that if the agents know the diameter of the network, our algorithm can be further modified to allow all agents to learn the true state and stop transmitting to their neighbors after a finite number of time-steps.

Keywords: Optimization algorithms, Distributed control, Networked control systems
Abstract: In this paper, we consider a distributed optimization problem over a network of nodes whose cost functions depend on a decision variable consisting of two parts: global (shared) part and local (node-specific) part. This problem structure arises in several important scenarios where the global part captures the common feature among nodes and the local part represents the personalized feature. To solve this problem, we develop a new personalized distributed stochastic gradient tracking method, where each node locally updates variables in a stochastic way and communicate the shared part with its neighbors for coordination. Leveraging a proper Lyapunov design, we show that the proposed algorithm converges linearly to a neighborhood of the optimum for smooth and strongly convex objective functions. The obtained rate result shows a clear dependence of the convergence performance on the topology and the properties of the objective functions. Numerical examples illustrate the effectiveness of the proposed algorithm towards mitigating data heterogeneity among nodes.

Keywords: Distributed control, Cooperative control, Agents-based systems
Abstract: In its simplest form the well known consensus problem for a networked family of autonomous agents is to devise a set of protocols or update rules, one for each agent, which can enable all of the agents to adjust or tune their ``agreement variable'' to the same value by utilizing real-time information obtained from their ``neighbors'' within the network. The aim of this paper is to study the problem of achieving a consensus in the face of limited information transfer between agents. By this it is meant that instead of each agent receiving an agreement variable or real-valued state vector from each of its neighbors, it receives a linear function of each state instead. The specific problem of interest is formulated and provably correct algorithms are developed for a number of special cases of the problem.

Keywords: Distributed control, Energy systems, Game theory
Abstract: In this paper we propose an original distributed control framework for DC microgrids. We first formulate the (optimal) control objectives as an aggregative game suitable for the energy trading market. Then, based on duality, we analyze the equivalent distributed optimal condition for the proposed aggregative game and design a distributed control scheme to solve it. By interconnecting the DC microgrid and the designed distributed control system in a power preserving way, we steer the DC microgrid's state to the desired optimal equilibrium, satisfying a predefined set of local and coupling constraints. Finally, based on the singular perturbation system theory, we analyze the convergence of the closed-loop system. The simulation results show excellent performance of the proposed control framework.

Keywords: Distributed control, Learning, Agents-based systems
Abstract: In this paper, a new distributed multi-agent off-policy Actor-Critic algorithm for collaborative multitask reinforcement learning is proposed. The Critic stage is based on the distributed gradient temporal difference algorithm GTD(1), while the Actor stage is derived from a predefined global criterion function and consists of a complementary consensus-based exact policy gradient algorithm. A proof that the Feller-Markov properties hold for the derived algorithm at the Actor stage is derived. The weak convergence of the algorithm to the set of stationary points of an attached ODE is proved under mild conditions using the two-time-scale stochastic approximation arguments. An experimental verification of the algorithm properties is given, demonstrating its high efficiency and practical applicability.

Keywords: Distributed control, Network analysis and control, Robotics
Abstract: We present a novel distributed multi-robot coordination strategy to persistently monitor a closed path-like environment. Our monitoring strategy relies on a class of time-inverted Kuramoto dynamics, whose multiple equilibria coincide with different monitoring configurations and allow us to tune the covering time of specific areas based on their priority. We provide a detailed analysis of the equilibria of the considered class of time-inverted Kuramoto dynamics and demonstrate the effectiveness of the proposed monitoring strategy via numerical examples.

Keywords: Distributed control, Networked control systems, Adaptive control
Abstract: In this paper, the problem of distributively tracking the minimum infimum (or maximum supremum) of a set of time-varying signals in finite-time is addressed. More specifically, each agent has access to a local time-varying exogenous signal, and all the agents are required to follow the minimum infimum (or the maximum supremum) of these signals in a distributed fashion. No assumption is made on the network size nor on the bounds of the exogenous signal derivatives. An adaptive protocol is provided which can provably solve the above problem in finite-time for multi-agent systems with undirected connected network topologies. Numerical simulations are provided to corroborate the theoretical findings.

Keywords: Distributed control, Switched systems, Lyapunov methods
Abstract: We address the consensus problem with collision avoidance for multi-agent systems under limited sensing ranges, in the case where new interconnections and agents may be added at any time. The graph topology is represented by a dynamic undirected graph, assumed to be connected only at an initial time, and the open multi-agent system is modeled via a multidimensional impulsive switched representation. We propose a barrier-Lyapunov-function-based consensus control law that guarantees inter-agent collision-avoidance and connectivity maintenance and, relying on the edge-agreement framework, we establish almost-everywhere asymptotic stability of the consensus manifold. The obtained results are also readily applicable to closed multi-agent systems with edge addition. A numerical simulation illustrates the effectiveness of the proposed approach.

Keywords: Game theory, Autonomous systems
Abstract: Oscillations often take place in populations of decision-making individuals that are either a coordinator, who takes action only if enough others do so, or an anticoordinator, who takes action only if few others do so. Populations consisting of exclusively one of these types are known to reach an equilibrium, where every individual is satisfied with her decision. Yet it remains open whether and when oscillations take place in a population consisting of both types, and if they do, what features they share. We take the first step towards answering this question by simulating a well-mixed population of coordinators and anticoordinator, each associated with a possibly unique non-negative threshold and initialized with the strategy A or B. We take the distribution of the actions A over the thresholds as the state of the population dynamics. The dynamics in our example admit two minimally positively invariant sets, where the solution trajectory oscillates, and an equilibrium. We identify the basic properties of the dynamics, based on which, we introduce a class of sets that are positively invariant. Our results highlight the possibility of non-trivial, complex oscillations in the absence of noise and population structure and shed light on the reported oscillations in decision-making populations.

Keywords: Autonomous systems, Cooperative control, Learning
Abstract: In this work, we explore the problem of offline reinforcement learning for a multi-agent system. Offline reinforcement learning differs from classical online and off-policy reinforcement learning settings in that agents must learn from a fixed and finite dataset. We consider a scenario where there exists a large dataset produced by interactions between an agent an its environment. We suppose the dataset is too large to be efficiently processed by an agent with limited resources, and so we consider a multi-agent network that cooperatively learns a control policy. We present a distributed reinforcement learning algorithm based on Q-learning and an approach called offline regularization. The main result of this work shows that the proposed algorithm converges in the sense that the norm squared error is asymptotically bounded by a constant, which is determined by the number of samples in the dataset. In the simulation, we have implemented the proposed algorithm to train agents to control both a linear system and a nonlinear system, namely the well-known cartpole system. We provide simulation results showing the performance of the trained agents.

Keywords: Game theory, Autonomous vehicles, Autonomous robots
Abstract: We consider a mobile robot, modelled as a Dubins' car, following a path. In the vicinity of the path is a human, modelled as an agile point mass, and a static obstacle. The robot's objective is to follow the path unless it is absolutely necessary to deviate so as to avoid collision with the static obstacle or human. We seek to guarantee robot's safety, even if the human is distracted or inattentive, and thus we consider worst-case motions for the human. The resulting problem takes the form of a reversed homicidal chauffeur game, but with the addition of a static obstacle. The static obstacle can interfere with the escape route of the robot, and thus fundamentally changes the form of the game. We propose a navigation algorithm for the robot that provably guarantees safety and that attempts to delay its reaction for as long as possible. We validate the proposed approach in simulation and provide a comparison to existing collision avoidance methods.

Keywords: Game theory, Nonholonomic systems, Optimal control
Abstract: In this paper, we consider the problem of pursuit-evasion between two pursuers against an evader with the agents modelled as Dubins' vehicle and having partial information regarding each other's motion. A novel, two-phase, evasive strategy based on worst-case scenario planning and proximity-based maneuvers is proposed. In the first phase, the evader assumes the pursuers to be holonomic and executes a best response strategy. An analytic and geometric approach is used to obtain the evader's strategy. In the second phase, the evader switches to movement along high-curvature paths to side-step the pursuers when they are in proximity of the evader in an attempt to delay capture. We also propose an approximation to compute the evader strategy in the first phase, which enables it to be applicable against multiple non-holonomic pursuers. Illustrative simulation results are given.

Keywords: Autonomous systems, Optimization
Abstract: This paper proposes using the Koopman operator for reachability analysis of an autonomous dynamical system. In particular, we demonstrate the application of spectral analysis of the Koopman operator involving eigenfunctions and eigenvalues in the approximate computation of forward and backward reachable sets for an autonomous dynamical system. The formal guarantees for the approximate reachable sets are provided using the Hausdorff distance between sets that measure how far the approximate reachable set is from the true reachable set. A computational framework based on convex optimization is provided to compute the Koopman spectrum and the approximate reachable set. Finally, we present simulation results to demonstrate the application of the developed framework.

Keywords: Game theory, Markov processes, Optimal control
Abstract: Inspired by the path coordination problem arising from robo-taxis, warehouse management, and mixed-vehicle routing, we model a group of heterogeneous players responding to stochastic demands as a congestion game under Markov decision process dynamics. Players share a common state-action space but have unique transition dynamics, and each player's unique cost is a function of the joint state-action probability distribution. For a class of player cost functions, we formulate the player-specific optimization problem, prove equivalence between the Nash equilibrium and the solution of a potential minimization problem, and derive dynamic programming approaches to solve the Nash equilibrium. We apply this game to model multi-agent path coordination and introduce congestion-based cost functions that enable players to complete individual tasks while avoiding congestion with their opponents. Finally, we present a learning algorithm for finding the Nash equilibrium that has linear complexity in the number of players. We demonstrate our game model on a multi-robot warehouse path coordination problem, in which robots autonomously retrieve and deliver packages while avoiding congested paths.

Keywords: Machine learning, Stochastic optimal control, Mean field games
Abstract: In this paper, we present a scalable deep learning approach to solve opinion dynamics stochastic optimal control problems with mean field term coupling in the dynamics and cost function. Our approach relies on the probabilistic representation of the solution of the Hamilton-Jacobi-Bellman partial differential equation. Grounded on the nonlinear version of the Feynman-Kac lemma, the solutions of the Hamilton-Jacobi-Bellman partial differential equation are linked to the solution of Forward-Backward Stochastic Differential Equations which can be solved numerically using a novel deep neural network with architecture tailored to the problem in consideration. The resulting algorithm is tested on a polarized opinion consensus experiment. The large-scale (10K) agents experiment validates the scalability and generalizability of our algorithm. The proposed framework opens up the possibility for future applications on extremely large-scale problems.

Keywords: Behavioural systems, Optimal control, Learning
Abstract: In this work, we revisit the Linear Quadratic Gaussian (LQG) optimal control problem from a behavioral perspective. Motivated by the suitability of behavioral models for data-driven control, we begin with a reformulation of the LQG problem in the space of input-output behaviors and obtain a complete characterization of the optimal solutions. In particular, we show that the optimal LQG controller can be expressed as a static behavioral-feedback gain, thereby eliminating the need for dynamic state estimation characteristic of state space methods. The static form of the optimal LQG gain also makes it amenable to its computation by gradient descent, which we investigate via numerical experiments. Furthermore, we highlight the advantage of this approach in the data-driven control setting of learning the optimal LQG controller from expert demonstrations.

Keywords: Mechatronics, Learning, Adaptive control
Abstract: High-performance tracking control of mechanical systems typically requires model-based control as it enables to counteract undesirable dynamics in a timely fashion. The quality of the compensation depends on the accuracy of the underlying model. However, the dynamics are often (partially) infeasible for complex systems in a-priori unknown environments. Due to the favorable properties of model-based control, the goal is to apply feedback control for error compensation only when necessary. In this paper, we present an online learning-based balancing (OLBB) framework to adapt the feedback gains such that the controlled system with state-dependent uncertainties satisfies given performance specifications. For this purpose, an oracle predicts the unknown dynamics and an error model adapts the feedback gains. The framework can be easily applied to existing control approaches to improve the safety and performance of the closed-loop.

Keywords: Learning, Linear systems, Autonomous systems
Abstract: In this work, we consider an actuator redundant system, i.e., a system with more actuators than the number of effective control inputs, and bring together connections between control allocation, actuator selection, and learning. In this kind of systems, the actuator commands can be chosen to meet a given control objective while still having leftover degrees of freedom to use towards minimizing the overall actuation energy. We show that this energy can be further minimized by optimally selecting the actuators themselves, which we perform in two different scenarios; first, in the case where the control objective is not known beforehand; and second, in the case where the control objective is defined to be a stabilizing state feedback controller. To relax the requirement for knowledge of the system's plant matrix, we compose a novel learning mechanism based on policy iteration, which computes the anti-stabilizing solution to an associated algebraic Riccati equation using trajectory data. Simulations are performed that demonstrate our approach.

Keywords: Communication networks, Machine learning
Abstract: A recent emphasis of distributed learning research has been on federated learning (FL), in which model training is conducted by the data-collecting devices. Existing research on FL has mostly focused on a star topology learning architecture with synchronized (time-triggered) model training rounds, where the local models of the devices are periodically aggregated by a centralized coordinating node. However, in many settings, such a coordinating node may not exist, motivating efforts to fully decentralize FL. In this work, we propose a novel methodology for distributed model aggregations via asynchronous, event-triggered consensus iterations over the network graph topology. We consider heterogeneous communication event thresholds at each device that weigh the change in local model parameters against the available local resources in deciding the benefit of aggregations at each iteration. Through theoretical analysis, we demonstrate that our methodology achieves asymptotic convergence to the globally optimal learning model under standard assumptions in distributed learning and graph consensus literature, and without restrictive connectivity requirements on the underlying topology. Subsequent numerical results demonstrate that our methodology obtains substantial improvements in communication requirements compared with FL baselines.

Keywords: Intelligent systems, Machine learning, Control applications
Abstract: Tool face adjustment in oil and gas drilling are vital issues that affect the efficiency and safety of directional drilling. The existing tool face adjustment methods mainly rely on manual real-time intervention for continuous adjustment. Affected by manual experience, the effect is unstable and the labor cost is high. With rising energy consumption, the need for intelligent directional drilling is becoming more pressing. However, establishing an autonomous adjustment approach for the tool face remains challenging because of the variety of complex downhole environments encountered during actual drilling operations. This paper proposes a model-free online learning adaptive decision strategy for cross well intelligent adjustment and stability of tool face. A reward function embedded with expert operating experience is designed to learn the directional policy from the driller's corrective actions. Further, to improve the efficiency of online learning, a priority-based experience playback algorithm is developed. A data-driven directional drilling simulation environment is proposed to realize the accurate simulation of the directional drilling process and pre-training of directional strategy. Simulations are carried out to validate the efficacy of the proposed method. The outcomes of field application suggest that the proposed strategy can achieve decision-making goals in a short period.

Keywords: Stability of nonlinear systems, Optimization algorithms, Neural networks
Abstract: Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method as a general-purpose ROA analysis tool for high-dimensional nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem such that PGD-based iterative methods can be directly applied. In the model-based setting, we show that the PGD updates can be efficiently performed using back-propagation. In the model-free setting, we propose a finite-difference PGD estimate which is general and only requires a black-box simulator for generating the trajectories of the closed-loop system given any initial state. Finally, we demonstrate the scalability and generality of our analysis tool on several numerical examples with large state dimensions or complex image observations.

Keywords: Adaptive control, Iterative learning control, Identification for control
Abstract: We consider the problem of online adaptive control of a linear-quadratic system, where the true system transition parameters (matrices A and B) are unknown. The objective is to design and analyze algorithms that generate control policies with sublinear “regret”, defined as the difference between the cumulative cost of the policies generated by the algorithm and the cumulative cost of the optimal policy. Recent studies show that when the system parameters are fully unknown, for any algorithm that only uses data from the past system trajectory, there is a choice of system parameters such that the algorithm at best achieves a square root regret, providing a hard fundamental limit on the achievable regret in general. However, it is known that (poly)-logarithmic regret is achievable when only matrix A or only matrix B is unknown. We prove a result, encompassing both of these scenarios, showing that (poly)logarithmic regret is achievable when both of these matrices are unknown, but a hint about them is given to the learner over time.

Keywords: Learning, Agents-based systems, Markov processes
Abstract: Cyber and cyber-physical systems equipped with machine learning algorithms such as autonomous cars share environments with humans. In such a setting, it is important to align system (or agent) behaviors with the preferences of one or more human users. We consider the case when an agent has to learn behaviors in an unknown environment. Our goal is to capture two defining characteristics of humans: i) a tendency to assess and quantify risk, and ii) a desire to keep decision making hidden from external parties. We incorporate cumulative prospect theory (CPT) into the objective of a reinforcement learning (RL) problem for the former. For the latter, we use differential privacy. We design an algorithm to enable an RL agent to learn policies to maximize a CPT-based objective in a privacy-preserving manner, and establish guarantees on the privacy of value functions learned by the algorithm when the rewards are sufficiently close. This is accomplished through adding a calibrated noise using a Gaussian process mechanism at each step. Through empirical evaluations, we highlight a privacy-utility tradeoff and demonstrate that the RL agent is able to learn behaviors that are aligned with that of a human user in the same environment in a privacy-preserving manner.

Keywords: Learning, Stability of nonlinear systems, Autonomous systems
Abstract: We consider the problem of learning an inner approximation of the region of attraction (ROA) of an asymptotically stable equilibrium point without an explicit model of the dynamics. Rather than leveraging approximate models with bounded uncertainty to find a (robust) invariant set contained in the ROA, we propose to learn sets that satisfy a more relaxed notion of containment known as recurrence. We define a set to be tau-recurrent (resp. k-recurrent) if every trajectory that starts within the set, returns to it after at most tau seconds (resp. k steps). We show that under mild assumptions a tau-recurrent set containing a stable equilibrium must be a subset of its ROA. We then leverage this property to develop algorithms that compute inner approximations of the ROA using counter-examples of recurrence that are obtained by sampling finite-length trajectories. Our algorithms process samples sequentially, which allows them to continue being executed even after an initial offline training stage. We further provide an upper bound on the number of counter-examples used by the algorithm, and almost sure convergence guarantees.

Keywords: Machine learning, Estimation, Optimization
Abstract: While deep neural networks are capable of achieving state-of-the-art performance in various domains, their training typically requires iterating for many passes over the dataset. However, due to computational and memory constraints and potential privacy concerns, storing and accessing all the data is impractical in many real-world scenarios where the data arrives in a stream. In this paper, we investigate the problem of one-pass learning, in which a model is trained on sequentially arriving data without retraining on previous datapoints. Motivated by the increasing use of overparameterized models, we develop Orthogonal Recursive Fitting (ORFit), an algorithm for one-pass learning which seeks to perfectly fit every new datapoint while changing the parameters in a direction that causes the least change to the predictions on previous datapoints. By doing so, we bridge two seemingly distinct algorithms in adaptive filtering and machine learning, namely the recursive least-squares (RLS) algorithm and orthogonal gradient descent (OGD). Our algorithm uses the memory efficiently by exploiting the structure of the streaming data via an incremental principal component analysis (IPCA). Further, we show that, for overparameterized linear models, the parameter vector obtained by our algorithm is what stochastic gradient descent (SGD) would converge to in the standard multi-pass setting. Finally, we generalize the results to the nonlinear setting for highly overparameterized models, relevant for deep learning. Our experiments show the effectiveness of the proposed method compared to the baselines.

Keywords: Behavioural systems, Robust control, Uncertain systems
Abstract: This paper explores the problem of uncertainty quantification in the behavioral setting for data-driven control. Building on classical ideas from robust control, the problem is regarded as that of selecting a metric which is best suited to a data-based description of uncertainties. Leveraging on Willems’ fundamental lemma, restricted behaviors are viewed as subspaces of fixed dimension, which may be represented by data matrices. Consequently, metrics between restricted behaviors are defined as distances between points on the Grassmannian, i.e., the set of all subspaces of equal dimension in a given vector space. A new metric is defined on the set of restricted behaviors as a direct finite-time counterpart of the classical gap metric. The metric is shown to capture parametric uncertainty for the class of autoregressive (AR) models. Numerical simulations illustrate the value of the new metric with a data-driven mode recognition and control case study.

Keywords: Estimation, Nonlinear systems, Machine learning
Abstract: In this paper we derive rates of convergence of a class of approximations of the Koopman operator that are obtained using the extended domain decomposition (EDMD) method. The paper extends the previous approaches that consider the case in which samples become dense in or approach a smooth Riemannian manifold or a regular subset of RR^d. Here, we extend these results to the case where samples become dense in some compact subset of the state space without requiring the common regularity properties. Rates of convergence of approximations of the Koopman operator are described in terms of approximation spaces, spectral approximation spaces, and in terms of the power function of the kernel of a reproducing kernel Hilbert space sH. A numerical example is presented to demonstrate the results.

Keywords: Nonlinear systems, Estimation, Quantized systems
Abstract: Entropy notions for varepsilon-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which induces the desired stability properties, called an approximating set. We provide conditions on the system which ensures that the approximating set is finite. Lower and upper bounds for the two estimation entropies are computed. The construction of the finite approximating sets induces a robust state estimation algorithm for systems under disturbances using quantized and time-sampled measurements.

Keywords: Estimation, Observers for Linear systems, Stability of nonlinear systems
Abstract: This paper presents a novel observer that estimates the state of a linear system in the observer canonical form in finite time. Due to the specially selected dynamical gain, the proposed observer is easy to tune and, at the same time, it is robust to external perturbations (e.g., state disturbances and measurement noises). The tuning algorithm consists in solving a simple system of linear matrix inequalities with only two adjustable parameters. Theoretical results and a comparison with homogeneous and prescribed-time observers are illustrated by numerical simulation.

Keywords: Observers for nonlinear systems, Estimation, Stability of nonlinear systems
Abstract: We provide a new class of observers for a class of nonlinear systems which are not required to be affine in the unmeasured states. Our observers ensure exponential convergence of the observation errors to zero. The rate of exponential convergence of the observation errors converges to infinity, as the growth rate of the nonlinear state-dependent part of the dynamics converges to zero. Therefore, we call the observers almost finite-time observers. Under global Lipschitz conditions on the state-dependent part of the dynamics, we obtain a global result that ensures convergence of the observers, for all initial states. Then, for cases where the nonlinearity is of order two at the origin, we provide local results ensuring exponential convergence of the observation errors to zero, when the initial state is small enough. We apply our global result to a model of a pendulum with friction, and our local result to aclass of dynamics with Lotka-Volterrra types of nonlinearities

Keywords: Estimation, Robotics, Observers for nonlinear systems
Abstract: In this paper we address smoothing (that is, optimisation-based) estimation techniques for localisation problems in the case where motion sensors are very accurate. Our mathematical analysis focuses on the difficult limit case where motion sensors are infinitely precise, resulting in the absence of process noise. Then the formulation degenerates, as the dynamical model that serves as a soft constraint becomes an equality constraint, and conventional smoothing methods are not able to fully respect it. By contrast, once an appropriate Lie group embedding has been found, we prove theoretically that invariant smoothing gracefully accommodates this limit case in that the estimates tend to be consistent with the induced constraints when the noise tends to zero. Simulations on the important problem of initial alignement in inertial navigation show that, in a low noise setting, invariant smoothing may favorably compare to state-of-the-art smoothers when using precise inertial measurements units (IMU).

Keywords: Observers for nonlinear systems, Estimation, LMIs
Abstract: This paper deals with observer design for a class of nonlinear systems. The paper makes two notable contributions. First, we propose a solution to design an observer for systems without global Lipschitz conditions by extending the nonlinearities to globally Lipschitz functions. Secondly, we provide a novel Linear Matrix Inequality (LMI) condition ensuring asymptotic convergence of the observer. The extension of the nonlinearities is achieved by exploiting old mathematical tools on Kirszbraun-Valentine Lipschitz extensions. Once the structure of the observer, based on the extended functions, is well-posed, we propose a new LMI technique, which is more general and more convenient than those existing in the literature.

Keywords: Optimization, Large-scale systems, Optimization algorithms
Abstract: This paper focuses on solving a data-driven distributionally robust optimization problem over a network of agents. The agents aim to minimize the worst-case expected cost computed over a Wasserstein ambiguity set that is centered at the empirical distribution. The samples of the uncertainty are distributed across the agents. Our approach consists of reformulating the problem as a semi-infinite program and then designing a distributed algorithm that solves a generic semi-infinite problem that has the same information structure as the reformulated problem. In particular, the decision variables consist of both local ones that agents are free to optimize over and global ones where they need to agree on. Our distributed algorithm is an iterative procedure that combines the notions of distributed ADMM and the cutting-surface method. We show that the iterates converge asymptotically to a solution of the distributionally robust problem to any pre-specified accuracy. Simulations illustrate our results.

Keywords: Statistical learning, Optimization, Uncertain systems
Abstract: Ambiguity sets of probability distributions are a prominent tool to hedge against distributional uncertainty in stochastic optimization. The aim of this paper is to build tight Wasserstein ambiguity sets for data-driven optimization problems. The method exploits independence between the distribution components to introduce structure in the ambiguity sets and speed up their shrinkage with the number of collected samples. Tractable reformulations of the stochastic optimization problems are derived for costs that are expressed as sums or products of functions that depend only on the individual distribution components. The statistical benefits of the approach are theoretically analyzed for compactly supported distributions and demonstrated in a numerical example.

Keywords: Statistical learning, Optimization, Uncertain systems
Abstract: This paper introduces a spectral parameterization of ambiguity sets to hedge against distributional uncertainty in stochastic optimization problems. We build an ambiguity set of probability densities around a histogram estimator, which is constructed by independent samples from the unknown distribution. The densities in the ambiguity set are determined by bounding the distance between the coefficients of their Haar wavelet expansion and the expansion of the histogram estimator. This representation facilitates the computation of expectations, leading to tractable minimax problems that are linear in the parameters of the ambiguity set, and enables the inclusion of additional constraints that can capture valuable prior information about the unknown distribution.

Keywords: Predictive control for linear systems, Uncertain systems, Stochastic optimal control
Abstract: We present a novel data-driven distributionally robust Model Predictive Control formulation for unknown discrete-time linear time-invariant systems affected by unknown and possibly unbounded additive uncertainties. We use off-line collected data and an approximate model of the dynamics to formulate a finite-horizon optimization problem. To account for both the uncertainty related to the dynamics and the disturbance acting n the system, we resort to a distributionally robust formulation that optimizes the cost expectation while satisfying Conditional Value-at-Risk constraints with respect to the worst-case probability distributions of the uncertainties within an ambiguity set defined using the Wasserstein metric. Using results from the distributionally robust optimization literature we derive a tractable finite-dimensional convex optimization problem with finite-sample guarantees for the class of convex piecewise affine cost and constraint functions. The performance of the proposed algorithm is demonstrated in closed-loop simulation on a simple numerical example.

Keywords: Markov processes, Stochastic optimal control, Uncertain systems
Abstract: In this paper we consider such Markov decision problems on finite spaces, when the probability distribution of the output conditioned on the sufficient statistic, belongs to a ball, with respect to the total variation distance metric, centered around a nominal conditional distribution.

We formulate the Markov decision problem, as a minimax optimization problem in which the minimization is over the admissible set of control strategies, while the maximization is over the conditional distributions which lie in the total variation distance ball. Then we present a two-step procedure to determine the optimal control strategies. In step 1, we determine the maximizing conditional distribution, as a function of the nominal conditional distribution and the radius of the ball; it is defined on a finite partition of the observation space of aggregating the output states to form new states. In step 2, the minimax problem is transformed into a minimization problem over the control strategies, and a new dynamic programming equation is derived, where the conditional expectation is taken with respect to the maximizing conditional distribution. Surprisingly, although the current paper deals with partially observable Markov decision problems, the mathematical analysis is tractable.

Keywords: Stochastic optimal control, Stochastic systems, Optimal control
Abstract: Wasserstein distributionally robust control (WDRC) is an effective method for addressing inaccurate distribution information about disturbances in stochastic systems. It provides various salient features, such as an out-of-sample performance guarantee, while most of the existing methods use full-state observations. In this paper, we develop a computationally tractable WDRC method for discrete-time partially observable linear-quadratic (LQ) control problems. The key idea is to reformulate the WDRC problem as a novel minimax control problem with an approximate Wasserstein penalty. We derive a closed-form expression of the optimal control policy of the approximate problem using a nontrivial Riccati equation. We further show the guaranteed cost property of the resulting controller and identify a provable bound for the optimality gap. Finally, we evaluate the performance of our method through numerical experiments using both Gaussian and non-Gaussian disturbances.

Keywords: Attack Detection, Cyber-Physical Security, Constrained control
Abstract: This paper studies provable security guarantees for cyber-physical systems (CPS) under actuator attacks. Specifically, we consider safety for CPS and propose a new attack-detection mechanism based on a zeroing control barrier function (ZCBF) condition. To reduce the conservatism in its implementation, we design an adaptive recovery mechanism based on how close the state is to violating safety. We show that the attack-detection mechanism is sound, i.e., there are no false negatives for adversarial attacks. Finally, we use a Quadratic Programming (QP) approach for online recovery (and nominal) control synthesis. We demonstrate the effectiveness of the proposed method in a case study involving a quadrotor with an attack on its motors.

Keywords: Attack Detection, Markov processes, Cyber-Physical Security
Abstract: In this paper, we develop a new change detection algorithm for detecting a change in the Markov kernel over a metric space in which the post-change kernel is unknown. Under the assumption that the pre- and post-change Markov kernel is uniformly ergodic, we derive an upper bound on the mean delay and a lower bound on the mean time between false alarms. A numerical simulation is provided to demonstrate the effectiveness of our method.

Keywords: Attack Detection, Optimization, Cyber-Physical Security
Abstract: This work discusses a novel framework for simultaneous synthesis of optimal watermarking signal and robust controllers in cyber-physical systems to minimize the loss in performance due to added watermarking signal and to maximize the detection rate of the attack. A general dynamic controller is designed to improve system performance with respect to the mathcal H_2 norm, while a watermarking signal is added to improve security performance concerning the detection rate of replay attacks. The attack model considered in the paper is a replay attack, a natural attack mode when the dynamics of the system is unknown to the attacker. The paper first generalizes the existing result on the detection rate of chi^2 detector from a static-LQR controller to a general dynamic controller. The design improvements on both robustness and security fronts are obtained by iteratively solving the convex subsets of the formulated non-convex problem in terms of the controller and watermarking signal. A semi-definite programming optimization is formulated using Linear Matrix Inequality (LMI) results to solve the larger system-level design optimization problem. We highlight the effectiveness of our method over a simplified three-tank chemical system.

Keywords: Computer/Network Security, Cyber-Physical Security, Cooperative control
Abstract: This paper proposes a fragility-aware stealthy attack for multiple mobile robots to secretly learn the accurate model of a multi-hop wireless network (MHWN). The stealthy attack scenario consists of three steps: active detection, node hijacking, and network eavesdropping. Our goal is to design distributed motion control strategies for naive attackers, for obtaining the connectivity information to evaluate the fragility degree of each node in MHWN. In order to achieve this goal, a switching control strategy is employed, which includes (i) move-to-fragile-neighbors-barycenter (MFNB) law and (ii) move-to-undetected-region-centroid (MURC) law. This strategy can drive mobile attackers to detect, hijack and eavesdrop on the MHWN’s nodes as quickly as possible. As a result, the fragility degree map of MHWN can be secretly learned to paralyze the whole network. Finally, the stability of the multi-robot system is proved and the effectiveness of the proposed strategy is verified.

Keywords: Discrete event systems, Game theory, Markov processes
Abstract: We consider the probabilistic planning problem for a defender (P1) who can jointly query the sensors and take control actions to reach a set of goal states while being aware of possible sensor attacks by an adversary (P2) who has perfect observations. To synthesize a provably-correct, attack-aware joint control and active sensing strategy for P1, we construct a stochastic game on graph with augmented states that include the actual game state (known only to the attacker), the belief of the defender about the game state (constructed by the attacker based on his knowledge of the defender's observations). We present an algorithm to compute a belief-based, randomized strategy for P1 to satisfy the reachability objective with probability one, under the worst-case sensor attacks carried out by an informed P2. We prove the correctness of the algorithm and illustrate it using an example.

Keywords: Discrete event systems, Supervisory control, Automata
Abstract: This paper investigates the problem of synthesizing sensor deception attackers against privacy in the context of supervisory control of discrete-event systems (DES). We consider a DES plant controlled by a supervisor, which is subject to sensor deception attacks. Specifically, we consider an active attacker that can tamper with the observations received by the supervisor by, e.g., hacking on the communication channel between the sensors and the supervisor. The privacy requirement of the supervisory control system is to maintain initial-state opacity, i.e., it does not want to reveal the fact that it was initiated from a secret state during its operation. On the other hand, the attacker aims to deceive the supervisor, by tampering with its observations, such that initial-state opacity is violated due to incorrect control actions. In this work, we investigate from the attacker's point of view by presenting an effective approach for synthesizing sensor attack strategies threatening the privacy of the system. To this end, we propose the All Attack Structure (AAS) that records state estimates for both the supervisor and the attacker. This structure serves as a basis for synthesizing a sensor attack strategy. We also discuss how to simplify the synthesis complexity by leveraging the structural property of the initial-state privacy requirement. A running academic example is provided to illustrate the synthesis procedure.

Keywords: Linear systems, Observers for Linear systems, LMIs
Abstract: We study the problems of determining the de- tectability and designing a state observer for linear time- invariant systems from measured data. First, we establish algebraic criteria to verify the detectability of the system from noise-free data. Then, we formulate data-driven linear matrix inequality-based conditions for observer design. Finally, we give conditions to infer the detectability of the system from noisy data.

Keywords: Linear systems, Output regulation, Sampled-data control
Abstract: In this paper, we propose a data-driven control strategy to solve the distributed modal consensus and synchronization problems. The proposed solution relies only on input/output data and does not require any knowledge of the dynamics of each agent. Furthermore, it is shown that the synchronization task requires to address the problem of transferring, in finite time, the state of a system from an initial state to a terminal one in a completely data-driven framework, which is therefore tackled and solved here. The above concepts are then illustrated via a multi-agent system consisting of RC circuits.

Keywords: Linear systems, Optimal control, Stochastic optimal control
Abstract: Motivated by the online policy design approaches in learning theory, new controller design paradigms such as competitive-ratio and regret-optimal control have been recently proposed as alternatives to the classical H₂ and H_∞ controllers. These metrics respectively consider the performances against a clairvoyant controller, which has access to future disturbances, in terms of ratio and difference. Even though the prior works on regret-optimal control provide its exact solution, in the competitive-ratio setting the solution is only provided for the suboptimal problem. In this work, we give the first exact solution to the optimal competitive-ratio control problem and present an explicit construction of the optimal competitive-ratio controller. A key technique that underpins our explicit solution is a reduction of the competitive-ratio control problem to the Nehari extension problem (similar to the regret-optimal control setting). The resulting optimal competitive-ratio controller is given by an explicit state space and the state-feedback law that is inherited from the H₂ controller. Inspired by this explicit solution, we generalize regret-optimal control to have weight functions on the state, input, and noise sequences and show that competitive-ratio control is an instance of this general framework. The utilization of weight functions allow penalization of particular sequences, but still enjoying the explicit and optimal solution for the regret-optimal control problem.

Keywords: Agents-based systems, Linear systems, Discrete event systems
Abstract: Synthesis results are presented for a general discrete-time dynamic output feedback controller and its anti-windup augmentation for operation under bounded actuation. The approach is then adopted for the leader-follower tracking problem for a multi-agent system, relying on the information shared among agents, through the network. For performance, we consider the l_2 gain from the disturbances to a general output, and obtain upper bounds for such gains. The scheme developed includes sufficient conditions, in the form of LMIs, for both controller design and anti-windup synthesis. Numerical examples are presented to show the effectiveness.

Keywords: Numerical algorithms, Linear systems, Identification
Abstract: We present two double recursive block Macaulay matrix algorithms to solve multiparameter eigenvalue problems (MEPs). In earlier work, we have developed a non-recursive approach that finds the solutions of an MEP via a multidimensional realization problem in the null space of the block Macaulay matrix constructed from the coefficient matrices of that MEP. However, this approach requires an iterative increase of the degree of the block Macaulay matrix: in order to determine whether the null space contains all the (affine) solutions of the MEP, we need to compute a basis matrix of the null space for every degree and check its dimension or rank structure. In this letter, we employ a recursive/sparse technique to compute a basis matrix of the null space of the block Macaulay matrix and a recursive technique to perform the necessary rank checks. We provide two system identification examples to show our improvements in computation time and memory usage.

Keywords: LMIs, Linear systems, Computer-aided control design
Abstract: This paper proposes a new Linear Matrix Inequality (LMI) for static output feedback control assuming that a Linear Quadratic Regulator (LQR) has been previously designed for the system. The main idea is to use a quadratic candidate Lyapunov function for the closedloop system parameterized by the unique positive definite matrix that solves the Riccati equation. A converse result will also be proved guaranteeing the existence of matrices satisfying the LMI if the system is static output feedback stabilizable. The static output feedback includes the LQR solution as a special case when the state is available, which is a desirable property. The examples show that the method is successful and works well in practice.

Keywords: Markov processes, Learning, Stochastic optimal control
Abstract: We propose a new framework for sequential decision making, Polya Decision Processes (PDP); it can express the agent's history-dependent transitions by using the Polya urn model. We show that PDP can be converted into a new type of Belief-MDP, whose belief update equation requires only urn model parameters. We introduce its theory, value iteration algorithm, and reinforcement learning algorithm for PDP using the belief state representation. Their effectiveness is confirmed by numerical experiments.

Keywords: Markov processes, Machine learning, Optimization algorithms
Abstract: We study constrained sequential decision-making problems modeled by constrained Markov decision processes with potentially infinite state spaces. We propose a Bregman distance-based direct policy search method -- policy gradient primal-dual mirror descent -- which includes the natural policy primal-dual method and the projected policy primal-dual method as two special cases. When the exact gradient is known, we prove dimension-free global convergence with a sublinear rate in both optimality gap and constraint violation. When the exact gradient is not available, we instantiate our algorithm in the linear function approximation setting and establish sample complexity guarantees. The introduction of the Bregman-distance regularizers enjoys the dimension-free property with applicability to large-scale spaces, the first of its kind in the constrained RL literature.

Keywords: Markov processes, Stochastic systems, Large-scale systems
Abstract: We consider restless bandits with restarts, where the state of the active arms resets according to a known probability distribution while the state of the passive arms evolves in a Markovian manner. We assume that the state of the arm is observed after it is reset but not observed otherwise. We show that the model is indexable and propose an efficient algorithm to compute the Whittle index by exploiting the qualitative properties of the optimal policy. A detailed numerical study of machine repair models shows that Whittle index policy outperforms myopic policy and is close to optimal policy.

Keywords: Markov processes, Stochastic systems, Statistical learning
Abstract: We consider system identification (learning) problems for Gaussian hidden Markov models (GHMMs). We propose an algorithm to tackle the cases where the data is recorded in aggregate (collective) form generated by a large population of individuals following a certain dynamics. Our parameter learning algorithm is built upon the expectation-maximization algorithm with a novel expectation step proposed recently known as the collective Gaussian forward-backward algorithm. The proposed learning algorithm generalizes the traditional Baum-Welch learning algorithm for GHMMs as it naturally reduces to the latter in case of individual observations.

Keywords: Markov processes, Automata, Hybrid systems
Abstract: We study the problem of refining satisfiability bounds for partially-known switched stochastic systems against planning specifications defined using syntactically co-safe Linear Temporal Logic (scLTL). We propose an abstraction-based approach that iteratively generates high-confidence Interval Markov Decision Process (IMDP) abstractions of the system from high-confidence bounds on the unknown component of the dynamics obtained via Gaussian process regression. In particular, we develop a synthesis strategy to sample the unknown dynamics by finding paths which avoid specification-violating states using a product IMDP formulation. We further provide a heuristic to choose among various candidate paths to maximize the information gain. Finally, we propose an iterative algorithm to synthesize a satisfying control policy for the product IMDP system. We demonstrate our work with a case study on mobile robot navigation.

Keywords: Markov processes, Formal Verification/Synthesis, Human-in-the-loop control
Abstract: This paper addresses the online motion planning problem of mobile robots under complex high-level tasks. The robot motion is modeled as an uncertain Markov Decision Process (MDP) due to limited initial knowledge, while the task is specified as Linear Temporal Logic (LTL) formulas. The proposed framework enables the robot to explore and update the system model in a Bayesian way, while simultaneously optimizing the asymptotic costs of satisfying the complex temporal task. Theoretical guarantees are provided for the synthesized outgoing policy and safety policy. More importantly, instead of greedy exploration under the classic ergodicity assumption, a safe-return requirement is enforced such that the robot can always return to home states with a high probability. The overall methods are validated by numerical simulations.

Keywords: Uncertain systems, Robust control, Optimization
Abstract: This paper proposes a method to find super-stabilizing controllers for discrete-time linear systems that are consistent with a set of corrupted observations. The L-infinity bounded measurement noise introduces a bilinearity between the unknown plant parameters and noise terms. A super-stabilizing controller may be found by solving a feasibility problem involving a set of polynomial nonnegativity constraints in terms of the unknown plant parameters and noise terms. A sequence of sum-of-squares (SOS) programs in rising degree will yield a super-stabilizing controller if such a controller exists. Unfortunately, these SOS programs exhibit very poor scaling as the degree increases. A theorem of alternatives is employed to yield equivalent, convergent (under mild conditions), and more computationally tractable SOS programs.

Keywords: Variable-structure/sliding-mode control, Indirect adaptive control
Abstract: In this paper, discrete-time versions of the first-order indirect adaptive sliding mode control (SMC) algorithm are developed and analyzed. In particular, a scalar linear system with parametric uncertainty linear in the state and with bounded input disturbance is considered. The discretization of continuous-time first-order indirect adaptive SMC algorithm may not preserve continuous-time properties like asymptotic stability of the origin. In order to ensure boundedness, knowledge about where the actual parameter may be located is used in this work. Then, for each proposed discretization scheme, conditions on the sampling time are derived that guarantee convergence into a bounded set. The theoretical results are illustrated with simulation examples and the proposed discretization schemes are compared with regard to the condition on sampling time and precision.

Keywords: Uncertain systems, Learning, Optimal control
Abstract: An inherent challenge of learning-based control tasks is posed by uncertainty due to finite training datasets. Even though there are principled tools to obtain confidence bounds for pointwise evaluation of learned dynamics models, it remains a challenging task to quantify the induced uncertainty in downstream quantities of interest due to the intrinsic recursive structure of dynamic systems. In this paper, we view the unknown one-step dynamics as a smooth function in a reproducing kernel Hilbert space and leverage random features for an approximate but highly structured parameterization of pointwise confidence bounds. As a result, we obtain downstream confidence bounds through an optimal control formulation under an uncertainty-aware random feature dynamics model. Our model is effectively a shallow neural network, which enables us to view the corresponding dynamic system as a deep neural network. Exploiting this perspective, we show that a Pontryagin’s minimum principle solution is equivalent to using the Frank-Wolfe algorithm on the induced neural network. Various numerical experiments on dynamics learning showcase the capacity of our methodology.

Keywords: Uncertain systems, Robust control, Markov processes
Abstract: In this paper, we investigate a worst-case-scenario control problem with a partially observed state. We consider a non-stochastic formulation, where noises and disturbances in our dynamics are uncertain variables which take values in finite sets. In such problems, the optimal control strategy can be derived using a dynamic program (DP) with respect to the memory. The computational complexity of this DP can be improved using a conditional range of the state instead of the memory. We present a more general definition of an information state which is sufficient to construct a DP without loss of optimality, and show that the conditional range is an example of an information state. Next, we extend this notion to define an approximate information state and an approximate DP. We also bound the maximum loss of optimality when using an approximate DP to derive the control strategy. Finally, we illustrate our results in a numerical example.

Keywords: Uncertain systems, Variational methods, Computational methods
Abstract: We consider the problem of verifying safety for a pair of identical integrator agents in continuous time with compact set-valued input uncertainties. We encode this verification problem as that of certifying or falsifying the intersection of their reach sets. We transcribe the same into a variational problem, namely that of minimizing the support function of the difference of the two reach sets over the unit sphere. We illustrate the computational tractability of the proposed formulation by developing two cases in detail, viz. when the inputs have time-varying norm-bounded and generic hyperrectangular uncertainties. We show that the latter case allows distributed certification via second order cone programming.

Keywords: Uncertain systems, Autonomous systems, Estimation
Abstract: Guaranteed state estimation for autonomous vehicles in GPS-denied areas that resort to landmarks detection and onboard sensors requires set-membership techniques that are capable of representing heterogeneous bounds using hyper-planes and ellipsoids. Recently, in the literature, the concept of Convex Constrained Generators (CCGs) has been introduced for the case where the dynamical system can be represented by a Linear Parameter-Varying (LPV) model. However, in practical applications, dynamics have uncertain parameters caused by noise-corrupted measurements of quantities of interest such as mass or orientation angles. In this paper, we first explore a closed-form solution for the convex hull of polytopes to showcase the main challenges of guaranteed state estimation for uncertain LPVs. We then propose the use of CCGs to have low conservatism when in the presence of distance measurements and avoid the exponential growth of the generators used in the state representation by performing an approximation using ray-shooting. Simulations illustrate the ability of CCGs to accurately model distance measurements with the corresponding decrease in volume without adding additional constraints.

Keywords: Biological systems, Network analysis and control, Systems biology
Abstract: When facing the global health threat posed by an infectious disease, predictive mathematical models are crucial not only to understand and forecast the epidemic evolution, but also to plan effective control strategies that contrast the disease and its spread in the population. This tutorial aims to give a broad overview of the fundamental developments enabled by systems-and-control methodologies in modelling and controlling epidemiological dynamics across scales, from infection dynamics within hosts to contagion dynamics between hosts. The first part is focused on modelling and control of infectious diseases in the host, capturing the dynamic interplay between pathogens and the immune system, and discussing control strategies to design tailored therapies and treatments to optimally clear the infection. The second part deals with the spread of contagion between hosts: epidemic dynamics are modelled resorting to networked systems where the nodes represent individuals and the links represent interactions that can lead to contagion, and a comparison to compartmental models is carried out. The third part surveys multi-scale models and multi-pronged approaches to contrast the spread of infectious diseases: a holistic perspective is adopted, including behavioural and socio-economic aspects along with public health issues, to discuss optimal epidemic control across scales.

Keywords: Biological systems, Biomolecular systems, Cellular dynamics
Abstract: Control of infectious diseases is a multidisciplinary field linking the application of engineering principles, mathematical modelling, medicine, and biology for healthcare purposes. This talk aims to present interdisciplinary tools to tackle infectious diseases at the host level. Dissecting detailed contributions of key players of the immune system to infectious diseases as well as their respective interactions will be discussed. Parameter fitting procedures to adjust parameters based on experimental data will be developed. Consequently, mathematical models will serve to perform stability analysis and control strategies for personalized therapies based on in-hosts models for several viral infections. Concepts as the critical fraction of susceptible/non-infected cells (under which the infection can no longer increase) can be fully understood and used in more general control objectives if put in terms of the equilibrium sets and their stability. Based on classical Lyapunov methods, a full characterization of the dynamical behaviour of the target-cell models under control actions will be discussed. Furthermore, based on the concept of virus spreadability, antiviral effectiveness thresholds are determined to establish whether a given treatment will be able to clear the infection. Also, it is shown how to simultaneously minimize the total fraction of infected cells while maintaining the virus load under a given level. Several examples for parameter fitting, modelling, and control applications are presented.

Keywords: Biological systems, Network analysis and control, Markov processes
Abstract: The study of epidemic processes has been of interest over a wide range of fields for the past century, including in mathematical systems, biology, physics, computer science, social sciences and economics. There has been renewed interest in the study of epidemic processes focused on the spread of viruses over networks, motivated not only by recent outbreaks of infectious diseases, but also by the rapid spread of opinions over social networks, and the security threats posed by computer viruses. Most of the models considered in recent studies have been focused on network models with static network structures, however many of the systems being considered have inherently dynamic structures. In this talk we will discuss data-informed modeling and equilibria analysis results for epidemic processes over both static and time-varying networks, with the goal being to elucidate the behavior of such spread processes. Multi-strain models, and issues arising from the use of data from ongoing viral outbreaks, will also be discussed as time allows. Simulation results and potential mitigation actions will be reviewed to conclude the talk.

Keywords: Biological systems, Systems biology, Network analysis and control
Abstract: Models are crucial to unveil the mechanisms of epidemic phenomena and forecast their evolution based on the available data. The tutorial will present a holistic viewpoint on epidemics that spans across multiple scales and includes opinion dynamics and socio-economic aspects along with public health issues. Multi-scale models capture both in-host infection (individual-level) and between-host contagion (population-level) dynamics, and consider the interplay between immunological (individual) and epidemiological (population) phenomena. Understanding infection and contagion dynamics through models enables the rigorous design of mathematical control approaches for effective suppression and mitigation. Multi-pronged strategies to control epidemics across scales leverage both pharmaceutical interventions, such as drugs and vaccines, and non-pharmaceutical interventions, including hygiene, use of personal protective equipment, physical distancing, travel bans. Public measures, essential to contain the contagion, need to be carefully planned to maximize their effectiveness and account for opinion-driven adherence to guidelines, treatments and vaccination. The interplay between contagion and opinion-driven behaviours can be captured by coupling epidemic dynamics with opinion dynamics describing how the attitude towards responsible behaviour evolves in the population, thus affecting the spread of the infection.

Keywords: Kalman filtering, Machine learning, Stochastic systems
Abstract: The process and measurement noise covariance matrices significantly impact the Extended Kalman Filter (EKF) performance and are often hand-tuned in practice, which usually entails a tedious task. Q-learning, a well-known method in reinforcement learning, has been applied recently to better adapt the noise covariance matrices for EKF thanks to its simplicity and capability in handling uncertain environments. Typically, some heuristics are involved in designing the Q-learning-based EKF (QLEKF), such as the tuning of grid size and covariance matrices values of each state, which inevitably degrade the estimation performance when the heuristics are not suitable. We propose a dynamic grid-based Q-learning EKF (DG-QLEKF) to overcome that drawback, which brings two novelties, an updated epsilon-greedy algorithm and a dynamic grid strategy. The proposed algorithm and strategy can thoroughly exploit arbitrary search scope and find appropriate values of noise covariance matrices. The effectiveness of DG-QLEKF, applied in navigation for attitude and bias estimation, is validated through the Monte Carlo method and real flight data from an unmanned aerial vehicle. The DG-QLEKF leads to much more improved state estimation than the QLEKF and traditional EKF.

Keywords: Formal Verification/Synthesis, Hybrid systems
Abstract: In this paper, we consider the problem of computing robust controlled invariants for discrete-time monotone dynamical systems. We consider different classes of monotone systems depending on whether the sets of states, control inputs and disturbances respect a given partial order. Then, we present set-based and trajectory-based characterizations of robust controlled invariants for the considered class of systems. Based on these characterizations, we propose an algorithmic approach to the computation of controlled invariants. Finally, we illustrate the proposed approach on an adaptive cruise control problem.

Keywords: Constrained control, Linear systems, Hybrid systems
Abstract: We present a framework on how to dynamically change the shape of invariant polytopes via elementary operations, namely, by adding a vertex or a linear inequality in their description, and at the same time maintain invariance. We work on the face lattice of the polytope, and introduce the cone lattice which embeds in an efficient way the geometric and combinatorial structure of the polytope. Combining the above, we can characterize a priori the complexity of the set resulting from either elementary operation. Importantly, using order-theoretic arguments, we identify the parts of the face and cone lattice that must be recalculated in every operation. Finally, we recall and extend equivalent algebraic conditions for preserving invariance of polytopes for a few families of dynamical systems.

Keywords: Optimization, Uncertain systems, Autonomous systems
Abstract: In the face of an adverse event, autonomous systems may undergo abrupt changes in their dynamics. In such an event, systems should be able to determine their continuing capabilities to then execute a provably completable task. This paper focuses on the scenario of a change in the system dynamics following an adverse event, aiming to determine the system's guaranteed performance capabilities by finding a set of states that are provably reachable by the system. While it is obviously impossible to exactly determine the reachable set without full knowledge of the system dynamics, we present a method of determining its under-approximation while assuming only partial knowledge of the system structure. Our technical approach relies on showing that an intersection of infinitely many ellipsoids --- available velocity sets for each system consistent with the partial knowledge of the dynamics --- is the same as an intersection of some finite collection of ellipsoids. This result enables us to find a maximal ellipsoid lying in such an intersection, yielding a set of velocities that the system is provably able to pursue regardless of its exact dynamics.

Keywords: Optimization, Algebraic/geometric methods, Nonlinear systems
Abstract: This paper presents a method to lower-bound the distance of closest approach between points on an unsafe set and points along system trajectories. Such a minimal distance is a quantifiable and interpretable certificate of safety of trajectories, as compared to prior art in barrier and density methods which offers a binary indication of safety/unsafety. The distance estimation problem is converted into a infinite-dimensional linear program in occupation measures based on existing work in peak estimation and optimal transport. The moment-SOS hierarchy is used to obtain a sequence of lower bounds obtained through solving semidefinite programs in increasing size, and these lower bounds will converge to the true minimal distance as the degree approaches infinity under mild conditions (e.g. Lipschitz dynamics, compact sets).

Keywords: Optimization algorithms, Hierarchical control, Linear systems
Abstract: We study a general class of least-squares problems structured according to a partially ordered set (poset). This is a fundamental optimization problem underlying the design of structured controllers on directed acyclic graphs or posets. We show that the optimality conditions of this problem yield a structured linear system, with sparsity pattern determined by a derived poset known as the poset of intervals. In general, this system could be relatively dense, and thus standard sparse linear algebra techniques may fail to provide significant reduction in computational complexity. Nonetheless, for a broad class of posets called multitrees identified in cite{nayyar_structural_2015} we show that performing elimination according to an order defined by the poset intervals progressively decouples variables, reducing the arithmetic complexity of solving the problem.

Keywords: Energy systems, Estimation, Statistical learning
Abstract: Thermal monitoring plays an essential role in ensuring safe, efficient and long-lasting operation of lithium-ion batteries (LiBs). Existing methods in the literature mostly rely on physics-based thermal models. However, an accurate physical thermal model is practically hard to obtain due to various uncertainties such as uncaptured dynamics, parameter errors, and unknown cooling conditions. Motivated by this problem, this paper considers a data-driven approach named Kriged Kalman filter to estimate the temperature field of LiBs. First, we demonstrate that the evolution of a pouch-type LiB cell's temperature field can be formulated as a spatio-temporal random field in a physically consistent manner. Then, we leverage the Kriged Kalman filter to update and reconstruct the random temperature field sequentially through time using sensor data. Our simulations show that the proposed approach can accurately reconstruct the LiB cell's temperature field with a small number of sensors.

Keywords: Optimization, Automotive control, Mechatronics
Abstract: The electrification trend in automotive industry is having a huge impact also in racing contexts. Indeed, Formula E, the most important competition for electric vehicles (EVs), is currently evolving to its third generation, developing more powerful cars. With respect to fuel-based vehicles, batteries represent the main shortcoming in EVs, due to lower energy and power densities compared to fuel tanks. For this reason, battery sizing in EVs is a real challenge, especially in racing applications, where its size significantly affects vehicle performance. In this letter, the battery sizing for electric racing cars is formulated as a co-design problem, tackled through a double-layer nested optimization approach. The external layer optimizes the battery pack design parameters and the internal one computes its optimal use, mainly the battery power request, to achieve the minimum race time. Finally, the proposed approach is applied to a meaningful case study, that is the Formula E battery pack sizing for the 2021 Rome and Valencia ePrix.

Keywords: Optimization, Smart grid, Automotive systems
Abstract: We consider the problem of deploying a spatial supply infrastructure to serve a distributed demand, motivated by Electrical Vehicle charging facilities. We present a series of optimization problems, which include the global transport cost from demand points to supply stations with bounded capacity, and also model demand elasticity. When supply locations are fixed, linear programs of the class of the Monge-Kantorovich problem apply; here our focus is showing that integer solutions that respect the indivisibility of demand units can be found. If locations are part of the design, the problem is not convex; we study iterative methods that generalize the clustering literature. Also, we investigate the issue of sparsity in the allocation, invoking mixed-integer linear program formulations of the facility location problem. The features and tractability of these methods are demonstrated in illustrative simulations. The paper ends by outlining a methodology through which these tools may be used jointly for the progressive deployment and operation of an EV charging infrastructure.

Keywords: Optimization, Stochastic systems, Energy systems
Abstract: The increasing adoption of electric vehicles has forced power network providers to deal with open issues both in terms of grid stability and electricity market design. On the latter direction, a challenging problem is represented by the development of probabilistic algorithms capable of computing optimal time-varying price profiles for EV charging stations to induce a desired aggregative behavior. Here, the inclusion of demand elasticity represents a key feature to provide usable schemes for real-world cases. In this paper, we propose an ``estimate-then-optimize'' framework for optimal dynamic pricing computation in the presence of price-sensitive customers. It consists of an {estimation} step based on nonparametric kernel estimation to infer about the demand elasticity, followed by an {optimization} step to maximize the expected daily profit. We describe the charging process via a probabilistic framework and the benefits of the proposed formulation are shown through extensive numerical experiments.

Keywords: Smart grid, Power systems, Energy systems
Abstract: This paper studies the problem of scheduling and pricing a large collection of non-preemptive electric loads. For the scheduling problem, we formulate the problem as a mixed integer program and propose an algorithm for solving it based on its convex relaxation. We establish that the admissible schedule produced by our algorithm is near optimal for any finite number of loads, and asymptotically optimal in a per-load cost sense when the number of loads grows to infinite. For the pricing problem, we analyze a natural marginal pricing policy defined for the convex relaxation. We show that the pricing policy is approximately budget adequate, and self-scheduling. Furthermore, we demonstrate that the pricing policy provides proper incentives for each flexible load to report its true flexibility window when the maximum duration of all loads is one. When there are some loads spanning multiple time slots, we show that this flexibility revealing property fails to hold. Our results suggest that while it is possible to achieve most of the aforementioned desirable properties by scheduling and pricing non-preemptive loads via convex relaxation, the marginal pricing rule needs to be adjusted to incentivize flexibility revealing for general non-preemptive loads.

Keywords: Smart grid, Decentralized control, Learning
Abstract: This paper considers the problem of voltage regulation in distribution networks. The primary motivation is to keep voltages within preassigned operating limits by commanding the reactive power output of distributed energy resources (DERs) deployed in the grid. We develop a framework for developing local Volt/Var control that comprises two main steps. In the first, by exploiting historical data and for each DER, we learn a function representing the desirable equilibrium points for the power network. These points approximate solutions of an Optimal Power Flow (OPF) problem. In the second, we propose a control scheme for steering the network towards these favorable configurations. Theoretical conditions are derived to formally guarantee the stability of the developed control scheme, and numerical simulations illustrate the effectiveness of the proposed approach.

Keywords: Hierarchical control, Game theory
Abstract: In this work we present a hierarchical framework for solving discrete stochastic pursuit-evasion games (PEGs) in large grid worlds. Given a partition of the grid world into superstates, the proposed approach creates a two-resolution decision-making process, which consists of a set of local PEGs at the original state level and an aggregated PEG at the superstate level. With a much smaller state space, both the local games and the aggregated game can be easily solved to a Nash equilibrium. Through numerical simulations, we show that the proposed hierarchical framework significantly reduces the computation overhead, while still maintaining a satisfactory performance level when competing against the flat Nash policies.

Keywords: Modeling, Game theory, Machine learning
Abstract: The focus of this paper is designing a game theoretical method using a continuous policy space for modeling human driver interactions on highway traffic. The proposed method is based on Gaussian Processes and developed as an enrichment to the hierarchical decision-making concept called "level-k reasoning". This concept conventionally assigns discrete levels of behaviors to agents. Although shown to be an effective modeling tool, the level-k reasoning approach may pose undesired constraints for predicting human decision making due to a limited number (usually 2 or 3) of driver policies it provides. The proposed approach is put forward to fill this gap in the literature by introducing a continuous domain framework that enables an infinite policy space. By using the approach presented in this paper, more accurate driver models are obtained, which can be employed for creating high fidelity simulation platforms for the validation of autonomous vehicle control algorithms. The proposed method is validated on a traffic dataset and compared with the conventional level-k approach to demonstrate its contributions and implications.

Keywords: Optimization algorithms, Game theory, Agents-based systems
Abstract: This paper focuses on the weighted set cover problem in networking systems and presents a fully distributed algorithm from the perspective of Nash equilibrium learning and selection. By viewing each set as an agent, we recast the problem as a networked ordinal potential game and classify the resulting Nash equilibrium into two categories. We show that each inferior Nash equilibrium (INE) could always be improved via local action exchange and better approximations could be achieved via self-organized selection among superior Nash equilibria (SNEs). By showing the existence of an improvement path that leads any action profile to an SNE, we prove that our algorithm converges in finite time to a conventional Nash equilibrium, where the joint action is a selected SNE. Comparison experiments with typical methods demonstrate the superiority to the state of the art.

Keywords: Optimization algorithms, Game theory, Variational methods
Abstract: We address Nash equilibrium problems in a partial-decision information scenario, where each agent can only exchange information with some neighbors, while its cost function possibly depends on the strategies of all agents. We characterize the relation between several monotonicity and smoothness conditions postulated in the literature. Furthermore, we prove convergence of a preconditioned proximal-point algorithm, under a restricted monotonicity property that allows for a non-Lipschitz, non-continuous game mapping.

Keywords: Constrained control, Mean field games, Lyapunov methods
Abstract: In this article, we prove a general viability theorem for continuity inclusions in Wasserstein spaces, and provide an application thereof to the existence of exponentially stable trajectories obtained via the second method of Lyapunov.

Keywords: Optimization, Agents-based systems, Human-in-the-loop control
Abstract: Pricing mechanisms are commonly advocated as a main tool to shape customers’ demand in societal scale networked infrastructure such as power or transportation systems. Given that the price response function of each user is generally considered private and unknown, most existing algorithms rely on protocols that explicitly or implicitly solicit this information in order to design prices. However, approaches that rely solely on learning the price response through repeated interactions are more practical and gaining traction. In this paper, we model each customer’s price response by an unknown parameter vector and we design a resource pricing mechanism to manage demand in order to maximize total welfare while ensuring that a set of linear constraints on the consumption are satisfied at all time steps with high probability. We propose an algorithm to address this problem that utilizes the well known principle of optimism in the face of uncertainty (OFU), while simultaneously being pessimistic with respect to constraint violation. Our analysis of this algorithm shows that, with high probability, it will not violate the constraints and will achieve O(log(T)sqrt{T}) regret. Numerical experiments validate these results and demonstrate how our algorithm can be applied to demand response management in power distribution systems.