Keywords: Linear systems, Learning
Abstract: We introduce a novel data-driven model-reference control design approach for unknown linear systems with fully measurable state. The proposed control action is composed by a static feedback term and a reference tracking block shaped from data to reproduce the desired behavior in closed-loop. By focusing on the case where the reference model and the plant share the same order, we propose an optimal design procedure with Lyapunov stability guarantees, tailored to handle state measurements with additive noise. Two simulation examples are illustrated to show the potential of the proposed strategy.

Keywords: Optimization algorithms, Machine learning, Computer-aided control design
Abstract: We propose a technique for global optimization considering black-box cost function and constraints, which have to be learned from data during the optimization process, arising for example in plant-control co-design of complex systems or controller tuning based on experiments. Assuming Lipschitz continuity of the cost function and constraints, we build a surrogate model and derive tight bounds on such functions based on a Set Membership framework. An exploitation step is designed to improve on the current best feasible candidate solution, searching in regions where all constraints are estimated as fulfilled, thus preserving safety. On the other hand, an exploration routine aims to discover the shape of the cost and constraint functions by picking points with large uncertainty, prioritizing regions where more constraints are predictably satisfied. The proposed algorithm can intuitively trade-off safety, exploration, and exploitation. The performance is evaluated on the problem of model predictive control tuning for a servomechanism with plant uncertainties and task-level constraints.

Keywords: Randomized algorithms, Uncertain systems, Optimization
Abstract: The scenario approach is a well-established methodology that allows one to generate solutions from a sample of observations (data-driven decision making). In the recent wait-and-judge paradigm to the scenario approach, the risk (i.e., the probability with which a scenario solution does not satisfy new, out-of-sample, constraints) is estimated from an observable called the complexity and this result is used to compute intervals that contain with high confidence the value of the risk. In this paper, we establish a new analytical expression for these confidence intervals and we show that they are centered around the complexity divided by the sample size N while their width uniformly (in the complexity) shrinks to zero for increasing N at the rate O(ln(N)/sqrt(N)) (which is close to the convergence rate of the central limit theorem). This result bears profound implications: (i) it proves the asymptotic consistency of the evaluation of the risk; (ii) as a corollary, it shows that the complexity is an observable that carries the fundamental information on the risk (a quantity that is not directly accessible); (iii) it extends the result that the empirical mean tends to the true probability of an event to the case when the event is chosen based on observations via a scenario decision scheme.

Keywords: Identification, Subspace methods, Kalman filtering
Abstract: The paper demonstrates the claim in the title using missing data estimation as a generic example. The missing data estimation problem includes simulation, Kalman smoothing, and linear quadratic control as special cases. The solution method proposed uses an idea from subspace identification: under a persistency of excitation condition, the image of a Hankel matrix constructed from the data is equal to the behavior of the data-generating system. This fact allows us to construct trajectories of the system directly from observed raw data. The construction of trajectories is the key for solving analysis, signal processing, and control problems without parametric model identification. The resulting methods require solution of systems of linear equations, however, the data is assumed exact and obtained from a linear time-invariant system.

Keywords: Robust control, Sampled-data control, Optimal control
Abstract: Willems' fundamental lemma and system level synthesis both characterize a linear dynamic system by its input/output sequences. In this work, we show that the extension of the fundamental lemma in uncertain LTI systems is equivalent to system level synthesis. Inspired by this observation, a robust closed-loop data-enabled predictive control scheme is proposed, where a novel formulation of a causal feedback control law is further derived. Two numerical experiments, including the temperature control of a single-zone building, are carried out to validate the effectiveness of the proposed data-driven controller.

Keywords: Feedback linearization, Machine learning, Nonlinear output feedback
Abstract: Willems et al. showed that all input-output trajectories of a discrete-time linear time-invariant system can be obtained using linear combinations of time shifts of a single, persistently exciting, input-output trajectory of that system. In this paper, we extend this result to the class of discrete-time single-input single-output flat nonlinear systems. We propose a data-based parametrization of all trajectories using only input-output data. Further, we use this parametrization to solve the data-based simulation and output-matching control problems for the unknown system without explicitly identifying a model. Finally, we illustrate the main results with numerical examples.

Keywords: Machine learning
Abstract: Optimal transport (OT) is a powerful geometric tool for comparing two distributions and has been employed in various machine learning applications. In this work, we propose a novel OT formulation that takes feature correlations into account while learning the transport plan between two distributions. We model the feature-feature relationship via a symmetric positive semi-definite Mahalanobis metric in the OT cost function. For a certain class of regularizers on the metric, we show that the optimization strategy can be considerably simplified by exploiting the problem structure. For high-dimensional data, we additionally propose suitable low-dimensional modeling of the Mahalanobis metric. Overall, we view the resulting optimization problem as a non-linear OT problem, which we solve using the Frank-Wolfe algorithm. Empirical results on the discriminative learning setting, such as tag prediction and multi-class classification, illustrate the good performance of our approach.

Keywords: Machine learning
Abstract: Accelerated gradient-based methods are being extensively used for solving non-convex machine learning problems, especially when the data points are abundant or the available data is distributed across several agents. Two of the prominent accelerated gradient algorithms are AdaGrad and Adam. AdaGrad is the simplest accelerated gradient method, which is particularly effective for sparse data. Adam has been shown to perform favorably in deep learning problems compared to other methods. In this paper, we propose a new fast optimizer, Generalized AdaGrad (G-AdaGrad), for accelerating the solution of potentially non-convex machine learning problems. Specifically, we adopt a state-space perspective for analyzing the convergence of gradient acceleration algorithms, namely G-AdaGrad and Adam, in machine learning. Our proposed state-space models are governed by ordinary differential equations. We present simple convergence proofs of these two algorithms in the deterministic settings with minimal assumptions. Our analysis also provides intuition behind improving upon AdaGrad's convergence rate. We provide empirical results on MNIST dataset to reinforce our claims on the convergence and performance of G-AdaGrad and Adam.

Keywords: Machine learning
Abstract: Reservoir computing is a neuromorphic computing paradigm that employs nonlinear dynamical systems for signal processing. Hardware realization of reservoir computers (RCs), physical RCs, has demonstrated promising ability for applications such as high-speed speech processing. However, physical RCs may be impacted by intrinsic noise in the hardware. In this work, we propose a modified least-mean squares and a modified recursive least squares adaptive algorithms for online estimation using RCs with noisy state measurements. We establish their convergence and demonstrate their efficacy on numerical examples.

Keywords: Machine learning, Stability of nonlinear systems, Power systems
Abstract: We propose a new distributed learning-based framework for stability assessment of a class of networked nonlinear systems, where each subsystem is dissipative. The aim is to estimate, in a distributed manner, a Lyapunov function and associated region of attraction for the networked system. We begin by using a neural network function approximation to learn a storage function for each subsystem such that the subsystem satisfies a local dissipativity property. We next use a satisfiability modulo theories (SMT) solver based falsifier that verifies the local dissipativity of each subsystem by deter- mining an absence of counterexamples that violate the local dissipativity property, as established by the neural network approximation. Finally, we verify network-level stability by using an alternating direction method of multipliers (ADMM) approach to update the storage function of each subsystem in a distributed manner until a global stability condition for the network of dissipative subsystems is satisfied. This step also leads to a network-level Lyapunov function that we then use to estimate a region of attraction. We illustrate the proposed algorithm and its advantages on a microgrid interconnection with power electronics interfaces.

Keywords: Learning, Machine learning, Flight control
Abstract: An important issue in quadcopter control is that an accurate dynamic model of the system is nonlinear, complex, and costly to obtain. This limits achievable control performance in practice. Gaussian process (GP) based estimation is an effective tool to learn unknown dynamics from input/output data. However, conventional GP-based control methods often ignore the computational cost associated with accumulating data during the operation of the system and how to handle forgetting in continuous adaption. In this paper, we present a novel Dual Gaussian Process (DGP) based model predictive control strategy that improves the performance of a quadcopter during trajectory tracking. The bio-inspired DGP structure is a combination of a long-term GP and a short-term GP, where the long-term GP is used to keep the learned knowledge in memory and the short-term GP is employed to rapidly compensate unknown dynamics during online operation. Furthermore, a novel recursive online update strategy for the short-term GP is proposed to successively improve the learnt model during online operation in an efficient way. Numerical simulations are used to demonstrate the effectiveness of the proposed strategy.

Keywords: Learning, Machine learning, Robust control
Abstract: Risk is an inherent component of any decision making process under uncertain conditions, and failure to consider risk may lead to significant performance degradation. We present a policy gradient theorem for the Risk-sensitive Control “exponential of integral” criteria, and propose a risk-sensitive Monte Carlo policy gradient algorithm. Our simulations, together with our theoretical analysis, show that the use of the exponential criteria with an appropriately chosen risk parameter not only results in a risk-sensitive policy, but also reduces variance during learning process and accelerates learning, which in turn results in a policy with higher expected return--- that is to say, risk-sensitiveness leads to sample efficiency and improved performance.

Keywords: Learning, Agents-based systems
Abstract: The inputs and preferences of human users are important considerations in situations where these users interact with autonomous cyber or cyber-physical systems. In these scenarios, one is often interested in aligning behaviors of the system with the preferences of one or more human users. Cumulative prospect theory (CPT) is a paradigm that has been empirically shown to model a tendency of humans to view gains and losses differently. In this paper, we consider a setting where an autonomous agent has to learn behaviors in an unknown environment. In traditional reinforcement learning, these behaviors are learned through repeated interactions with the environment by optimizing an expected utility. In order to endow the agent with the ability to closely mimic the behavior of human users, we optimize a CPT-based cost. We introduce the notion of the CPT-value of an action taken in a state, and establish the convergence of an iterative dynamic programming-based approach to estimate this quantity. We develop two algorithms to enable agents to learn policies to optimize the CPT-value, and evaluate these algorithms in environments where a target state has to be reached while avoiding obstacles. We demonstrate that behaviors of the agent learned using these algorithms are better aligned with that of a human user who might be placed in the same environment, and is significantly improved over a baseline that optimizes an expected utility.

Keywords: Fault tolerant systems, Learning, Uncertain systems
Abstract: A novel fault tolerant control algorithm is proposed in this paper based on model reference reinforcement learning for autonomous surface vehicles subject to sensor faults and model uncertainties. The proposed control scheme is a combination of a model-based control approach and a data-driven method, so it can leverage the advantages of both sides. The proposed design contains a baseline controller that ensures stable tracking performance at healthy conditions, a fault observer that estimates sensor faults, and a reinforcement learning module that learns to accommodate sensor faults using fault estimation and compensate for model uncertainties. The impact of sensor faults and model uncertainties can be effectively mitigated by this composite design. Stable tracking performance can also be ensured even at both the offline training and online implementation stages for the learning-based fault tolerant control. A numerical simulation with gyro sensor faults is presented to demonstrate the efficiency of the proposed algorithm.

Keywords: Machine learning, Optimal control, Predictive control for linear systems
Abstract: Reinforcement learning (RL) is a technique to learn the control policy for an agent that interacts with a stochastic environment. In any given state, the agent takes some action, and the environment determines the probability distribution over the next state as well as gives the agent some reward. Most RL algorithms typically assume that the environment satisfies Markov assumptions (i.e. the probability distribution over the next state depends only on the current state). In this paper, we propose a model-based RL technique for a system that has non-Markovian dynamics. Such environments are common in many real-world applications such as in human physiology, biological systems, material science, and population dynamics. Model-based RL (MBRL) techniques typically try to simultaneously learn a model of the environment from the data, as well as try to identify an optimal policy for the learned model. We propose a technique where the non-Markovianity of the system is modeled through a fractional dynamical system. We show that we can quantify the difference in the performance of an MBRL algorithm that uses bounded horizon model predictive control from the optimal policy. Finally, we demonstrate our proposed framework on a pharmacokinetic model of human blood glucose dynamics and show that our fractional models can capture distant correlations on real-world datasets.

Keywords: Machine learning, Agents-based systems, Neural networks
Abstract: We provide a provably-convergent decentralized actor-critic algorithm for learning deterministic policies on continuous action spaces. Deterministic policies are important in real-world settings. To handle the lack of exploration inherent in deterministic policies, we consider both off-policy and on-policy settings. We give the expression of a local deterministic policy gradient, decentralized deterministic actor-critic algorithms and convergence guarantees for linearly-approximated value functions. This work will help enable decentralized MARL in high-dimensional action spaces and pave the way for more widespread use of MARL.

Keywords: Machine learning, Simulation, Optimal control
Abstract: In this paper, we propose a new type of Actor, named forward-looking Actor or FORK for short, for ActorCritic algorithms. FORK can be easily integrated into a model-free Actor-Critic algorithm. Our experiments on six Box2D and MuJoCo environments with continuous state and action spaces demonstrate significant performance improvement FORK can bring to the state-of-the-art algorithms. A variation of FORK can further solve BipedalWalkerHardcore in as few as four hours using a single GPU.

Keywords: Optimal control, Robotics, Learning
Abstract: The problem of Reinforcement Learning (RL) in an unknown nonlinear dynamical system is equivalent to the search for an optimal feedback law utilizing the simulations/ rollouts of the unknown dynamical system. Most RL techniques search over a complex global nonlinear feedback parametrization making them suffer from high training times as well as variance. Instead, we advocate searching over a local feedback representation consisting of an open-loop sequence, and an associated optimal linear feedback law completely determined by the open-loop. We show that this alternate approach results in highly efficient training, the answers obtained are repeatable and hence reliable, and the resulting closed performance is superior to global state of the art RL techniques. Finally, if we replan, whenever required, which is feasible due to the fast and reliable local solution, allows us to recover global optimality of the resulting feedback law.

Keywords: Identification, Fault diagnosis, Healthcare and medical systems
Abstract: Freeze-drying is a standard procedure in pharmaceutical industry, used to stabilize, store and increase the shelf life of drug products. In this process, the product has to be brought to a very low pressure and the lyophilization chamber has to be perfectly sealed. Even small external leaks can contaminate the entire drug batch. Since a single batch may contain thousands of product vials, freeze-dryer leakages are one of the most critical problems of the entire production chain of lyophilized drugs. We describe a simple mathematical model for lyophilizer leaks and address the problem of identifying and separating internal and external leaks. We propose a leak identification method based on the use of multiple leak detection tests. By using the real data of a pharmaceutical lyophilizer, we show that the proposed method allows to identify internal and external leaks and to estimate their evolution in time.

Keywords: Identification, Modeling, Biomedical
Abstract: One of the main outstanding issues in the neuroscience is estimation of effective connectivity in brain networks, which models the causal interactions among neuronal populations. Estimation of effective connectivity embraces two types of the challenges, such as estimation accuracy and computational complexity. In this paper, we consider resting-state (rs) fMRI data serving as an input for a stochastic linear DCM model. The model parameters are estimated through an EM (expectation maximization) iterative procedure. In this work, we propose the alternative scheme for the hyperparameters estimation aiming in reduction of computational burden of the original EM-algorithm. The simulation results demonstrate the viability of the proposed block-reweighting scheme and represents a promising research direction to be further investigated.

Keywords: Identification, Neural networks, Optimization
Abstract: The identification of states and parameters from noisy measurements of a dynamical system is of great practical significance and has received a lot of attention. Classically, this problem is expressed as optimization over a class of models. This work presents such a method, where we augment the system in such a way that there is no distinction between parameter and state reconstruction. We pose the resulting problem as a batch problem: given the model, reconstruct the state from a finite sequence of output measurements. In the case the model is linear, we derive an analytical expression for the state reconstruction given the model and the output measurements. Importantly, we estimate the state trajectory in its entirety and do not aim to estimate just an initial condition; that is, we use more degrees of freedom than strictly necessary in the optimization step. This particular approach can be reinterpreted as training of a neural network that estimates the state trajectory from available measurements. The technology associated with neural network optimization/training allows an easy extension to nonlinear models. The proposed framework is relatively easy to implement, does not depend on an informed initial guess, and provides an estimate for the state trajectory (which incorporates an estimate for the unknown parameters) over a given finite time horizon.

Keywords: Identification, Variational methods, Nonlinear systems identification
Abstract: The estimation of the Hammerstein system by using a noniterative learning schema is considered, and a novel algorithm based on the Variational Bayes method is presented. To best emulate the original distribution of the system parameters within the set of those with feasible moments, the loss functional is constructed to optimally approximate the true distribution by a product of independent marginals. To guarantee the uniqueness of the model parameterization, the hard equality constraint is imposed on the selected parameter mean value. In our adopted recursive scenario, the transmission of the approximated moments via iterative cycles is avoided by propagating the sufficient statistics associated with the overparameterized model, which is linear in unknown parameters. Moreover, this propagation penalizes the difference of the updated parameters from the previous ones rather than from the initial guess. Due to access to the sufficient statistics and the suitably chosen marginals, the solution we propose is produced in closed form.

Keywords: Identification
Abstract: This work addresses the design of theoretical optimal regularization matrices for the instrumental variable method in the errors-in-variables identification framework. The design is based on the asymptotic statistical properties of the regularized instrumental variable estimator and it provides new bounds for the use of regularization in this scenario as well as new ideas to parametrize and estimate the regularization matrix in practical situations. A numerical example shows the effectiveness of the optimal estimator in comparison with the classic least-squares and instrumental variable methods.

Keywords: Identification
Abstract: Asymptotic theory is one of the core subjects in system identification theory and often used to assess properties of model estimators. In this paper, we focus on the asymptotic theory for the kernel-based regularized system identification and study the convergence in distribution of the generalized cross validation (GCV) based hyper-parameter estimator. It is shown that the difference between the GCV based hyper-parameter estimator and the optimal hyper-parameter estimator that minimizes the mean square error scaled by 1/sqrt{N} converges in distribution to a zero mean Gaussian distribution, where N is the sample size and an expression of covariance matrix is obtained. In particular, for the ridge regression case, a closed-form expression of the variance is obtained and shows the influence of the limit of the regression matrix on the asymptotic distribution. For illustration, Monte Carlo numerical simulations are run to test our theoretical results.

Keywords: Stochastic optimal control, Filtering, Learning
Abstract: Q learning algorithm is a popular reinforcement learning method for finite state/action fully observed Markov decision processes (MDPs). In this paper, we make two contributions: (i) we establish the convergence of a Q learning algorithm for partially observed Markov decision processes (POMDPs) using a finite history of past observations and control actions and show that the limit fixed point equation gives an optimal solution for an approximate belief-MDP. We then provide bounds on the performance of the policy obtained using the limit Q values compared to the performance of the optimal policy for the POMDP, where we also present explicit performance guarantees using recent results on filter stability in controlled POMDPs. (ii) We apply these the results to fully observed MDPs with continuous state spaces and establish the near optimality of learned policies via quantization of the state space, where the quantization is viewed as a measurement channel leading to a POMDP model and a history of unit window size is considered. In particular, we show that Q-learning, with its convergence and near optimality, apply for continuous space MDPs when the state space is quantized.

Keywords: Filtering, Network analysis and control, Stochastic systems
Abstract: We consider the problem of analyzing the performance of distributed filters for continuous-time linear stochastic systems under certain information constraints. We associate an undirected and connected graph with the measurements of the system, where each node gets some reduced dimensional measurements. Each node then runs a locally optimally filter based on the available measurements. In order to reach a consensus, the sensor nodes communicate their estimate to its neighbors at some randomly drawn discrete time instants and the activation times of the edges of the graph are governed by independent Poisson counters. When a node gets some information from its neighbor, it resets its state using the convex combination of the available information. Consequently, each sensor node implements a filtering algorithm in the form of a stochastic hybrid system. We derive bounds on expected value of error covariance for each node, and show that they converge to a common value for each node if the mean sampling rates for communication between sensor units are large enough.

Keywords: Filtering, Stochastic optimal control, Markov processes
Abstract: This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. The MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPIIs we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs.

Keywords: Stochastic systems, Markov processes, Filtering
Abstract: This paper revisits the classical question of the stability of the nonlinear Wonham filter. The novel contributions of this paper are two-fold: (i) definition of the stabilizability for the (control-theoretic) dual to the nonlinear filter; and (ii) the use of this definition to obtain conclusions on the stability of the Wonham filter. Specifically, it is shown that the stabilizability of the dual system is necessary for filter stability and conversely stabilizability implies that the filter asymptotically detects the correct ergodic class. The formulation and the proofs are based upon a recently discovered duality result whereby the nonlinear filtering problem is cast as a stochastic optimal control problem for a backward stochastic differential equation (BSDE). The control-theoretic proof techniques and results may be viewed as a generalization of the classical work on the stability of the Kalman filter.

Keywords: Stochastic systems, Markov processes, Filtering
Abstract: This paper is concerned with the problem of nonlinear filter stability of ergodic Markov processes. The main contribution is the conditional Poincaré inequality (PI) which is shown to yield filter stability. The proof is based upon a recently discovered duality result whereby the nonlinear filtering problem is cast as a stochastic optimal control problem for a backward stochastic differential equation (BSDE). A comparison is made between the stochastic stability of a Markov process and the filter stability. The former is based upon assuming the standard form of PI whereas the latter relies on the conditional PI introduced in this paper.

Keywords: Markov processes, Filtering, Kalman filtering
Abstract: We consider inference (filtering) problems for a class of continuous state collective hidden Markov models, where the data is recorded in aggregate (collective) form generated by a large population of individuals following the same dynamics. We propose an aggregate inference algorithm called collective Gaussian forward-backward algorithm, extending recently proposed Sinkhorn belief propagation algorithm to models characterized by Gaussian densities. Our algorithm enjoys convergence guarantee. In addition, it reduces to the standard Kalman filter when the observations are generated by a single individual. The efficacy of the proposed algorithm is demonstrated through multiple experiments.

Keywords: Game theory, Agents-based systems, Finance
Abstract: A framework for hierarchical noncooperative systems with dynamic agents is proposed. In the characterized framework, agents in each group are incentivized by a corresponding group manager who represents the benefits of group utility via an intra-group incentive mechanism. The coefficients in intra-group incentive functions are characterized as the group manager’s strategy in this paper. The update rules that can be adopted by the group managers are proposed based on local information with continual and intermittent observation on the state of the agents from the other groups. Sufficient conditions under which the trajectory of agents’ state converges towards the group Nash equilibrium are derived for the proposed update rules. On the other hand, to improve the social welfare of the entire system, we propose an inter-group incentive scheme in the group managers level for a system governor to bring agents’ state to a target equilibrium. To deal with the uncertain information on agents' personal payoff functions for the system governor, we present sufficient conditions to guarantee the convergence of agents' state to the target equilibrium. A numerical example is provided to illustrate the efficacy of our results.

Keywords: Game theory, Agents-based systems, Optimization
Abstract: The use of multi-agent systems to solve largescale problems can be an effective method to reduce physical and computational burdens; however, these systems should be robust to sub-system failures. In this work, we consider the problem of designing utility functions, which agents seek to maximize, as a method of distributed optimization in resource allocation problems. Though recent work has shown that optimal utility design can bring system operation into a reasonable approximation of optimal, our results extend the existing literature by investigating how robust the system’s operation is to defective agents and by quantifying the achievable performance guarantees in this setting. Interestingly, we find that there is a trade-off between improving the robustness of the utility design and offering good nominal performance. We characterize this trade-off in the set of resource covering problems and find that there are considerable gains in robustness that can be made by sacrificing some nominal performance.

Keywords: Game theory, Agents-based systems, Robotics
Abstract: This paper considers a one-defender-one-attacker reach-avoid differential game (DG) in the plane which is split by a straight line into a goal region and a play region. The attacker aims at entering the goal region from the play region without being captured, while the defender tries to capture the attacker in the play region. We focus on the defense problem where the defender is a Dubins car with non-zero capture radius and the attacker is a simple-motion model. First, a controller is proposed for the defender to track a heading reference which is derived from an evasion space (ES)-based defense strategy. Then, the upper bounds for the time derivative of the heading reference are computed. Based on it, we show that the defender can succeed to track the heading reference within a finite time. Finally, both simulation and experiment examples are provided, where a vision-based re-localization method is used for the experiment to deal with the coordinate inconsistency problem.

Keywords: Game theory, Agents-based systems
Abstract: Multi-agent systems are designed to concurrently accomplish a diverse set of tasks at unprecedented scale. Here, the central problems faced by a system operator are to decide (i) how to divide available resources amongst the agents assigned to tasks and (ii) how to coordinate the behavior of the agents to optimize the efficiency of the resulting collective behavior. The focus of this paper is on problem (i), where we seek to characterize the impact of the division of resources on the resulting collective behavior in a best-case sense. We focus on a Colonel Blotto game where there are two sub-colonels competing against a common adversary in a two battlefield environment. Here, each sub-colonel is assigned a resource budget and is required to independently allocate its assigned resources. However, the team's success depends on the adversary's response to the allocations of both sub-colonels. The central focus of this manuscript is on how to divide a common pool of resources among the two sub-colonels to optimize the resulting best-case efficiency guarantees. The main result of this paper establishes that performance is not monotonic in the division of resources to the sub-colonels. In particular, a more equitable division can offer better performance than a more centralized division. Hence, this paper demonstrates that the resource division problem is highly non-trivial and worthy of significant future research efforts.

Keywords: Game theory, Statistical learning, Estimation
Abstract: Game theory provides essential analysis in many applications of strategic interactions. However, the question of how to construct a game model and what is its fidelity is seldom addressed. In this work, we consider learning in a class of repeated zero-sum games with unknown, time-varying payoff matrix, and noisy feedbacks, by making use of an ensemble of benchmark game models. These models can be pre-trained and collected dynamically during sequential plays. They serve as prior side information and imperfectly underpin the unknown true game model. We propose OFULinMat, an episodic learning algorithm that integrates the adaptive estimation of game models and the learning of the strategies. The proposed algorithm is shown to achieve a sublinear bound on the saddle-point regret. We show that this algorithm is provably efficient through both theoretical analysis and numerical examples. We use a dynamic honeypot allocation game as a case study to illustrate and corroborate our results. We also discuss the relationship and highlight the difference between our framework and the classical adversarial multi-armed bandit framework.

Keywords: Game theory, Agents-based systems
Abstract: This paper introduces the notion of timed congestion games with agent-specific benefit functions to solve the problem of multi-route multi-fleet platoon matching. The fact that the timed congestion game with agent-specific benefit functions is isomorphic to the congestion game with agent-specific benefit functions is shown. Furthermore, two subclasses of the timed congestion game with agent-specific benefit functions are shown to possess a Nash equilibrium. In addition, in both subclasses, every best response sequence is proven to have finite length. Then, the theoretical results on the timed congestion game with agent-specific benefit functions are applied to the problem of multi-route multi-fleet platoon matching. We illustrate our approach with a numerical example.

Keywords: Optimization algorithms, Decentralized control, Lyapunov methods
Abstract: Distributed optimization often requires finding the minimum of a global objective function written as a sum of local functions. A group of agents work collectively to minimize the global function. We study a continuous-time decentralized mirror descent algorithm that uses purely local gradient information to converge to the global optimal solution. The algorithm enforces consensus among agents using the idea of integral feedback. Recently, the asymptotic convergence of this algorithm was studied for when the global function is strongly convex but local functions are convex. Using control theoretical tools, in this work, we prove (theoretically) that the algorithm indeed achieves local exponential convergence. We also provide a numerical experiment on a real data-set as a validation of the convergence speed of our algorithm.

Keywords: Optimization algorithms, Large-scale systems, Machine learning
Abstract: We consider a class of distributed optimization problem where the objective function consists of a sum of strongly convex and smooth functions and a (possibly nonsmooth) convex regularizer. A multi-agent network is assumed, where each agent holds a private cost function and cooperates with its neighbors to compute the optimum of the aggregate objective. We propose a quasi-Newton Alternating Direction Method of Multipliers (ADMM) where the primal update is solved inexactly with approximated curvature information. By introducing an intermediate consensus variable, we achieve a block diagonal Hessian which eliminates the need for inner communication loops within the network when computing the update direction. We establish global linear convergence to the optimal primal-dual solution without the need for backtracking line search, under the assumption that component cost functions are strongly convex with Lipschitz continuous gradients. Numerical simulations on real datasets demonstrate the advantages of the proposed method over state of the art.

Keywords: Optimization algorithms, Learning, Control Systems Privacy
Abstract: We use gradient sparsification to reduce the adverse effect of differential privacy noise on performance of private machine learning models. To this aim, we employ compressed sensing and additive Laplace noise to evaluate differentially-private gradients. Noisy privacy-preserving gradients are used to perform stochastic gradient descent for training machine learning models. Sparsification, achieved by setting the smallest gradient entries to zero, can reduce the convergence speed of the training algorithm. However, by sparsification and compressed sensing, the dimension of communicated gradient and the magnitude of additive noise can be reduced. The interplay between these effects determines whether gradient sparsification improves the performance of differentially-private machine learning models. We investigate this analytically in the paper. We prove that, for small privacy budgets, compression can improve performance of privacy-preserving machine learning models. However, for large privacy budgets, compression does not necessarily improve the performance. Intuitively, this is because the effect of privacy-preserving noise is minimal in large privacy budget regime and thus improvements from gradient sparsification cannot compensate for its slower convergence.

Keywords: Optimization algorithms, Network analysis and control, Agents-based systems
Abstract: We study the convergence in expectation of the Random Coordinate Descent algorithm (RCD) for solving optimal resource allocations problems in open multi-agent systems, i.e., multi-agent systems that are subject to arrivals and departures of agents. Assuming all local functions are strongly-convex and smooth, and their minimizers lie in a given ball, we analyse the evolution of the distance to the minimizer in expectation when the system is occasionally subject to replacements in addition to the usual iterations of the RCD algorithm. We focus on complete graphs where all agents interact with each other with the same probability, and provide conditions to guarantee convergence in open system. Finally, a discussion around the tightness of our results is provided.

Keywords: Optimization algorithms, LMIs, Optimization
Abstract: Semidefinite programs (SDP) are a staple of today's systems theory, with applications ranging from robust control to systems identification. However, current state-of-the art solution methods have poor scaling properties (both in computational complexity and memory requirements), and thus are limited to relatively moderate size problems. Recently, several approximations have been proposed where the original SDP is relaxed to a sequence of lower complexity problems (such as linear programs (LPs) or second order cone programs (SOCPs)). While successful in many cases, there is no guarantee that these relaxations converge to the global optimum of the original program. Indeed, examples exists where these relaxations "get stuck" at suboptimal solutions. To circumvent this difficulty in this paper we propose an algorithm to solve SDPs based on solving a sequence of LPs or SOCPs, guaranteed to converge in a finite number of steps to an epsilon-suboptimal solution of the original problem. We further provide a bound on the number of steps required, as a function of epsilon and the problem data. The potential of the proposed algorithm is illustrated by solving, in a laptop, benchmark problems from the SDPLib dataset that challenge current SDP solvers running in the same platform.

Keywords: Optimization algorithms, Numerical algorithms, Optimal control
Abstract: Mathematical programs with complementarity constraints are notoriously difficult to solve due to their nonconvexity and lack of constraint qualifications in every feasible point. In this work we focus on the subclass of quadratic programs with linear complementarity constraints. We introduce a novel approach to solving a penalty reformulation using Sequential Convex Programming (SCP) and a homotopy on the penalty parameter. By linearizing the nonconvex penalty function, we obtain convex quadratic subproblems, which have a constant Hessian matrix throughout all. This enables us to obtain solutions with a single KKT matrix factorization. Furthermore, we introduce a globalization scheme in which the underlying merit function is minimized analytically, and we additionally provide a guarantee of descent at each iterate. The algorithmic features and possible computational speedups are illustrated in a numerical experiment.

Keywords: Nonlinear systems, Optimal control, Stability of nonlinear systems
Abstract: Infinite-horizon optimal control problems for nonlinear systems are studied and discussed. First, we thoroughly revisit the formulation of the underlying dynamic optimisation problem together with the classical results providing its solution. Then, we consider two alternative methods to construct solutions (or approximations thereof) of such problems, developed in recent years, that provide theoretical insights as well as computational benefits. While the considered methods are mostly based on tools borrowed from the theories of Dynamic Programming and Pontryagin’s Minimum Principles, or a combination of the two, the proposed control design strategies yield innovative, systematic and constructive methods to provide exact or approximate solutions of nonlinear optimal control problems. Interestingly, similar ideas can be extended also to linear and nonlinear differential games, namely dynamic optimisation problems involving several decision-makers. Due their advantages in terms of computational complexity, the considered methods have found several applications. An example of this is provided, through the consideration of the multi-agent collision avoidance problem, for which both simulations and experimental results are provided.

Keywords: Nonlinear systems, Optimal control, Stability of nonlinear systems

Keywords: Nonlinear systems, Optimal control, Stability of nonlinear systems

Keywords: Petri nets, Discrete event systems
Abstract: The explicit consideration of time is nowadays crucial for the specification and the verification of Discrete Event Systems (DESs). The behaviour of a DES is characterized by the set of event sequences it generates, its language. In many applications, the DES language is compared to a given set of observed sequences to evaluate the DES accuracy. In a timed context, however, the language is infinite, making it difficult to define and evaluate an accuracy indicator. To overcome this difficulty, the accuracy of a Time Petri net model with respect to a set of sequences is formulated in this paper, using a sliding finite-time window mechanism. In addition, an algorithm to evaluate the proposed accuracy measure is provided.

Keywords: Petri nets, Hybrid systems, Modeling
Abstract: Continuous Petri nets aim to avoid the state explosion problem of classical discrete Petri nets by relaxing the integrality constraint of the firing of transitions. Although the resulting formalism can successfully model a number of features of dynamic systems, its use in complex real systems can be hindered by the limited number of possible dynamics and by the difficulty in accommodating uncertain parameters. The modeling formalism of Flexible Nets can overcome these difficulties by significantly extending the modeling and analysis possibilities of continuous Petri nets. The modeling capabilities of Flexible Nets will be presented together with their analysis possibilities in both the transient and steady state.

Keywords: Petri nets, Supervisory control, Optimization
Abstract: Supervisor simplification is of significant importance in the supervisory control of automated manufacturing systems. In many situations, simplification results are not unique. However, there is little study on the optimal simplification. In the paradigm of discrete event systems, solving the optimal simplification is always faced with the combinational explosion problem. Based on the simplification method proposed by the same authors, our work further investigates the optimal simplified supervisor based on genetic algorithm (GA). There are two contributions. First, we present a GA in which our simplification method is embedded to derive the basic optimal simplified supervisor when the parameters of the specifications are fixed. Second, a hierarchical GA is proposed to obtain the advanced optimal simplified supervisor when the parameters of the specifications are changeable. This is a multiple-objective optimization problem where both structure simplification and behavior permissiveness are considered. Examples show the effectiveness of our algorithms in solving the optimal supervisor simplification problem.

Keywords: Control over communications, Discrete event systems, Autonomous vehicles
Abstract: In this paper, we consider the problem of remote observation of a unicycle-type mobile robot through a data rate constrained communication channel, which can only send a limited number of bits per unit of time. The objective is to reconstruct estimates of the state of the robot at the remote location through the messages that are sent. The design of the communication protocol should ensure that the maximum observation error is bounded whilst using as few bits per unit of time as possible. An event-triggered observation scheme is developed specifically for the unicycle-type robot. This observer is tested through experiments on Turtlebots. The experiments show that the event-triggered scheme is very efficient at reducing the average number of required communications.

Keywords: Stability of linear systems, Stochastic systems, Discrete event systems
Abstract: In this paper, we expand the stability theory of dynamical systems on stochastic time scales to the case of stochastic pulse time scales. The class of systems considered here evolve on nonuniform time-domains that consist of a union of disjoint closed intervals with stochastic lengths, followed by random step sizes. Necessary and sufficient conditions for exponential stability almost surely are derived. The approach is based on determining the region of exponential stability almost surely. An illustrative numerical example is presented to validate the results. The class of systems considered in this paper has important applications in, for e.g., control networks subject to communications failures, population dynamics, signal processing with variable sampling, consensus multi-agents systems and wide-area power system controls.

Keywords: Distributed control, Robust adaptive control, Adaptive control
Abstract: We address a distributed adaptive synchronization problem for complex networks composed of nonlinear nodes under state-dependent a priori interconnections, i.e. interconnection terms acting before control design. The interconnection terms are uncertain and the heterogeneous dynamics of the network nodes further contain state-dependent uncertainty and uncertain input matrix gain. Adaptive distributed control laws are proposed to tackle such an unsolved design. The proposed controller is verified in simulation via a multi-area load frequency control for power systems.

Keywords: Adaptive control, Human-in-the-loop control, Modeling
Abstract: This paper introduces an analytical framework for the derivation of distributed-parameter equations of motion of a flexible quadrotor. This approach helps obtain rigid and flexible equations of motion simultaneously, in a decoupled form, which facilitates the controller design. An adaptive controller is implemented using the developed model to prevent excessive oscillations due to flexible dynamics and to compensate uncertainties. Furthermore, a delay-dependent stability condition is obtained for the overall system dynamics, including the human UAV operator with reaction time delay, the adaptive controller and the flexible quadrotor dynamics. It is demonstrated via simulations that the flexible arm tip oscillations are reduced when the closed loop reference model adaptive controller is used, compared to a conventional model reference adaptive controller.

Keywords: Biomedical, Adaptive control, Uncertain systems
Abstract: Multi-microrobot systems are promising to achieve minimally invasive intraarterial therapy or biosensing, but are mainly controlled using a single control input, resulting in underactuated systems. Besides, such systems are sensitive to biological --fatally-- uncertain parameters, notably the pulsatile blood flow, a local information whose in-situ prediction or measurement is hardly possible. Along a reference trajectory, standard backstepping or adaptive control result in practical stabilization, so we propose an adaptive replanning and control to achieve asymptotic stabilization. Simulation results enhance the interest in the proposed approach and its robustness to parametric uncertainties.

Keywords: Robotics, Adaptive control, Control applications
Abstract: In recent years, lower limb exoskeleton has attracted extensive attention in academic and engineering research. In the rehabilitation motion training scenario, it is of concern that the exoskeleton should have the ability to control its own leg movements in order to assist a patient with natural and anthropomorphic gaits. A patient with paralysis in leg, is incapable of controlling and coordinating with the exoskeleton well enough to produce a desirable gait. Thus, a critical problem in this scenario is that the exoskeleton needs to track a desired gait in order for the patient to adopt and produce different walking patterns. To this end, this paper proposes an adaptive tracking controller for the exoskeleton. Although many adaptive control methods exist, a traditional control design utilizes time-triggered method, namely, the system data is periodically sampled and controller parameters are updated at specific time instances. To implement the update mechanism, in this paper, an aperiodically adaptive controller based on policy iteration is developed by integrating event triggering mechanism and reinforcement learning. Further, in order to achieve online learning and adaptation, an actor-critic neural network is employed, and a novel event triggered tuning law is designed to learn the controller signals. We conduct simulations and perform experiments with a lower limb rehabilitation exoskeleton robot. Experimental results demonstrate that the proposed event triggered control strategy can reduce control updates when compared with many traditional time triggered control methods, for a guaranteed control performance.

Keywords: Adaptive control, Cyber-Physical Security, Uncertain systems
Abstract: In this paper, the adaptive secure control problem for a class of second-order nonlinear systems with p-normal form and uncertain time-varying parameters against sensor and actuator deception attacks is investigated. Multiplicative type of attacks are considered. A novel adaptive backstepping based resilient control scheme is constructed by employing the power integrator Lyapunov function technique and a special Nussbaum function. It is shown that all the closed-loop signals remain uniformly bounded despite the occurrence of the attacks. Simulation results on an illustrative example are provided to illustrate the effectiveness of the proposed control scheme.

Keywords: Adaptive control, Machine learning, Autonomous systems
Abstract: In this paper, we propose the Model Reference Adaptive Control & Reinforcement Learning (MRAC-RL) approach to developing online policies for systems in which modeling errors occur in real-time. Although reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems, discrepancies between simulated dynamics and the true target dynamics can cause trained policies to fail to generalize and adapt appropriately when deployed in the real-world. The MRAC-RL framework generates online policies by utilizing an inner-loop adaptive controller together with a simulation-trained outer-loop RL policy. This structure allows MRAC-RL to adapt and operate effectively in a target environment, even when parametric uncertainties exists. We propose a set of novel MRAC algorithms, apply them to a class of nonlinear systems, derive the associated control laws, provide stability guarantees for the resulting closed-loop system, and show that the adaptive tracking objective is achieved. Using a simulation study of an automated quadrotor landing task, we demonstrate that the MRAC-RL approach improves upon state-of-the-art RL algorithms and techniques through the generation of online policies.

Keywords: Stability of linear systems, Estimation
Abstract: We prove a new robust stabilization theorem for systems with time-varying disturbances and sampled measurements, using novel bounds on fundamental matrices for systems with disturbances. Our main tools use properties of positive systems and Metzler matrices. Our numerical example illustrates an advantage of using our method instead of previous methods.

Keywords: Stability of linear systems, PID control
Abstract: In this paper, we study the gain and phase margins achievable by PID controllers in stabilizing linear time-invariant (LTI) systems. The problem under consideration amounts to determining the largest ranges of gain and phase variations such that there exists a single PID controller capable of stabilizing all the plants within the variation ranges. We consider low-order systems, notably the first- and second-order systems. For each class of these systems, we derive explicit expressions of the maximal gain and phase margins achievable. The results demonstrate analytically how a plant's unstable poles and nonminimum phase zeros may confine the maximal gain and phase margins attainable by PID control, which lead to a number of useful observations. First, for minimum phase systems, we show that the maximal gain and phase margins achievable by PID controllers coincide with those by general LTI controllers. Second, for nonminimum phase systems, we show that LTI controllers perform no better than twice than PID controllers, in the sense that the maximal gain and phase margins achievable by general LTI controllers are within a factor of two of those by PID controllers, whereas the former is measured on a logarithmic scale and latter on a linear scale. Finally, we show that PID and PD controllers achieve the same maximal margins, indicating that integral control is immaterial in improving a system's robustness in feedback stabilization. These results thus provide useful insights into PID control, and from a system robustness perspective, offer an interpretation on the effectiveness of PID controllers.

Keywords: Stability of linear systems, Robust control, LMIs
Abstract: This paper introduces the notion of linear Discrete-time Output Negative Imaginary (D-ONI) systems. The D-ONI class is defined in the z-domain and it includes the systems having poles on the unit circle. The proposed definition involves a real parameter delta >= 0, which indicates the strictness properties. delta > 0 specifies the strict subset, Discrete-time Output Strictly Negative Imaginary (D-OSNI), within the stable D-ONI class. Interestingly, the new D-ONI class captures the existing D-NI systems while restricted to discrete-time LTI systems having a real, rational and proper transfer function. However, the D-OSNI systems are not identical to the existing strictly D-NI (D-SNI) subset. Instead, these two subsets intersect each other. An LMI-based state-space characterisation is derived to check the strict/non-strict D-ONI properties of a given system relying on the value of delta. The paper also establishes the connections between the discrete-time Passive and discrete-time NI systems. Finally, a closed-loop stability result is proposed for a positive feedback interconnection of two D-ONI systems without poles at z = -1 and z = +1. Numerical examples are provided throughout the paper to elucidate the usefulness of the D-ONI theory.

Keywords: Linear systems, Computational methods, Stability of linear systems
Abstract: We study the algorithmic complexity of the problem of deciding whether a Linear Time Invariant dynamical system with rational coefficients has bounded trajectories. Despite its ubiquitous and elementary nature in Systems and Control, it turns out that this question is quite intricate, and, to the best of our knowledge, unsolved in the literature. We show that classical tools, such as Gaussian Elimination, the Routh--Hurwitz Criterion, and the Euclidean Algorithm for GCD of polynomials indeed allow for an algorithm that is polynomial in the bit size of the instance. However, all these tools have to be implemented with care, and in a non-standard way, which relies on an advanced analysis.

Keywords: Optimal control, Linear systems, LMIs
Abstract: A reference signal shaping problem formulated as a convex optimization problem is presented for the design of the reference signal in a closed-loop discrete-time linear- time-invariant system, with the purpose that internal control signals and system output are bounded within constraints. A causal solution endures the reference profiles changes only after a system output is required to change. The proposed solution allows us to compute a causal or noncausal reference profile, by adding a time-dependent signal constraint. Feasibility and existence of a reference profile is verified with a linear programming (LP) problem, while an optimal reference profile for the closed-loop system is obtained via a quadratic program (QP) problem. A mass-spring-damper system paired with a PID controller is the illustrative example for closed-loop reference shaping. To evaluate the proposed design, the closed-system response for both causal and noncausal reference profiles are compared.

Keywords: Linear systems, Stability of linear systems, Modeling
Abstract: The control design for juggling a devil stick between two symmetric configurations is proposed. Impulsive forces are applied to the devil-stick at the two configurations; and impulse of the force and its point of application are modeled as the control inputs. The dynamics of the devil-stick is described by a single return Poincaré map and it is shown that the control objective of juggling can be achieved by stabilizing a hybrid orbit. The impulse controlled Poincaré map (ICPM) approach, recently proposed for stabilization of continuous-time orbits of underactuated systems, is extended to achieve asymptotic stabilization of the hybrid orbit. The applicability of the ICPM approach to devil-stick juggling is demonstrated through numerical simulations.

Keywords: Distributed parameter systems, Delay systems, Linear systems
Abstract: This paper deals with prescribed-time stabilization of controllable linear systems with distributed input delay. We model the input delay as a transport PDE and reformulate the original problem as a cascade PDE-ODE system while accounting for the infinite dimensionality of the actuator. We build on reduction-based and backstepping-forwarding transformations to convert the system into a target system having the prescribed-time stability property. Then, we prove the bounded invertibility of the transformations and hence we show that the prescribed-time stability property is preserved into the original problem. To better illustrate the ideas of this approach, we focus first on the scalar case. Then, we give a sketch of the main lines for the general case. To this end, we choose the ODE dynamics of the target system to be a Linear Time-Varying system so that we can rely on recent developments which include a polynomial-based Vandermonde matrix and the generalized Laguerre polynomials that allow a compact formulation for the stability analysis. A simulation example is presented to illustrate the obtained results.

Keywords: Distributed parameter systems, Robust control, Stability of nonlinear systems
Abstract: This paper establishes some input-to-state stability (ISS) properties w.r.t. in-domain process and measurement disturbances for systems described by a normal flow equation governed by an observer-based control scheme without knowledge of the spatial derivatives of the viscosity solution. The approach used to achieve the "a priori" ISS estimates of the solution is not based on Lyapunov arguments and assumes fixed boundary conditions. A one-dimensional case study is addressed by both analytical and numerical treatments, which show the effectiveness of the proposed approach.

Keywords: Observers for nonlinear systems, LMIs, Lyapunov methods
Abstract: We consider Navier-Stokes equations on a rectangle with periodic boundary conditions, and known input. Given continuous measurements as averages of NSE’ solution over a set of squares we design a globally converging observer for NSE by relying upon Lyapunov method: we propose a parametric LMI for determining observer’s gain and size of squares, required for the global convergence. We illustrate the numerical efficacy of our algorithm by applying it to estimate states of NSE with Kolmogorov forcing.

Keywords: Network analysis and control, Distributed parameter systems, Linear systems
Abstract: We have shown in previous work that the synchronization of networks of identical finite-dimensional linear systems can be characterized by the stability of the decoupled systems under suitable complex feedback. In this paper we show that this characterization relies on the equivalence of stability and vanishing-input-vanishing-output properties, which is not guaranteed in general for infinite-dimensional systems. This equivalence is established for second-order parabolic PDEs, thus enabling a synchronization analysis similar to its finite-dimensional counterpart.

Keywords: Control of networks, Distributed parameter systems, LMIs
Abstract: This paper deals with synchronization of a class of infinite-dimensional systems. The considered network is described by a collection of semilinear Lipschitz boundary- actuated infinite-dimensional dynamics. For undirected con- nected graphs, sufficient conditions for asymptotic synchro- nization are established. We show that the proposed conditions when applied to systems of hyperbolic semilinear conservation laws can be recast into a set of matrix inequalities. For this class of systems, sufficient conditions in the form of linear matrix inequalities for the design of synchronizing policies are provided.

Keywords: Distributed parameter systems
Abstract: We consider the output-feedback state consensus problem for a homogeneous multi-agent system consisting of one leader agent and N follower agents. The dynamics of each of these agents is governed by a single-input single-output regular linear system (RLS), with the input to the leader agent being zero at all times. The transfer function of this RLS has all its poles in the closed left-half of the complex-plane, with only a finite number of them lying on the imaginary axis. Each follower agent can access the relative output of all its neighboring agents and some of the follower agents can access the relative output of the leader. The communication graph associated with the exchange of relative outputs is directed. Under this setting, we first establish that a controller solves the above leader-follower consensus problem if and only if it can simultaneously stabilize N regular linear systems. We then adapt a recently developed frequency-domain technique to construct a stable finite-dimensional output-feedback controller which solves the simultaneous stabilization problem, and hence the consensus problem. We demonstrate the efficacy of our controller design technique using a numerical example.

Keywords: Control of networks, Distributed control, Hybrid systems
Abstract: This paper explores the rendezvous problem for a multi-agent system (MAS) with distance-limited, intermittent communication and sensing. Unlike previous works that provide specific event-triggered controllers, we provide a framework that characterize a family of distributed event-triggered controllers leveraging non-singular edge-potentials to achieve approximate rendezvous while maintaining the initial distance-limited graph. The proposed framework excludes the possibility of Zeno behavior and accommodates the development of self-triggered controllers. The combination of continuous and impulsive dynamics results in a hybrid system, where the closed-loop dynamics of the MAS are presented and analyzed using hybrid differential inclusions. The approximate rendezvous problem is recast into a set stabilization problem and sufficient conditions of the rendezvous set are obtained through a Lyapunov-based analysis. Simulation results are provided to validate the development.

Keywords: Game theory, Agents-based systems, Distributed control
Abstract: The theory of learning in games has extensively studied situations where agents respond dynamically to each other by optimizing a fixed utility function. However, in many settings of interest, agent utility functions themselves vary as a result of past agent choices. The ongoing COVID-19 pandemic provides an example: a highly prevalent virus may incentivize individuals to wear masks, but extensive adoption of mask-wearing reduces virus prevalence which in turn reduces individual incentives for mask-wearing. This paper develops a general framework using probabilistic coupling methods that can be used to derive the stochastically stable states of log-linear learning in certain games which feature such game-environment feedback. As a case study, we apply this framework to a simple dynamic game-theoretic model of social precautions in an epidemic and give conditions under which maximally cautious social behavior in this model is stochastically stable.

Keywords: Cooperative control, Distributed control, Optimal control
Abstract: This paper addresses the persistent monitoring problem defined on a network where a set of nodes (targets) needs to be monitored by a team of dynamic energy-aware agents. The objective is to control the agents' motion to jointly optimize the overall agent energy consumption and a measure of overall node state uncertainty, evaluated over a finite period of interest. To achieve these objectives, we extend an established event-driven Receding Horizon Control (RHC) solution by adding an optimal controller to account for agent motion dynamics and associated energy consumption. The resulting RHC solution is computationally efficient, distributed and on-line. Finally, numerical results are provided highlighting improvements compared to an existing RHC solution that uses energy-agnostic first-order agents.

Keywords: Distributed control, Large-scale systems, Linear systems
Abstract: In this paper we consider the problem of designing distributed static state, static output and dynamic output feedback controllers for discrete time systems. The proposed approach is based on iteratively minimizing a finite horizon quadratic cost function, where the descent directions are determined using simulated system trajectories. When the system matrices are sparse, the computational complexity of the proposed algorithm scales linearly with the size of the system.

Keywords: Distributed control, Stochastic systems
Abstract: This paper focuses on the problem of distributed consensus control of multi-agent systems while considering two main practical concerns (i) stochastic noise in the agent dynamics and (ii) predefined performance constraints over evolutions of multi-agent systems. In particular, we consider that each agent is driven by a stochastic differential equation with state-dependent noise which makes the considered problem more challenging compare to non-stochastic agents. The work provides sufficient conditions under which the proposed time-varying distributed control laws ensure consensus in expectation and almost sure consensus of stochastic multi-agent systems while satisfying prescribed performance constraints over evolutions of the systems in the sense of the q th moment. Finally, we demonstrate the effectiveness of the proposed results with a numerical example.

Keywords: Distributed control, Switched systems, Agents-based systems
Abstract: This article presents a particular class of consensus algorithms for multi-agent systems of continuous-time single integrators communicating over a shared broadcast channel. To deal with the problem of interference and packet collision, we consider a time-division multiple access (TDMA) protocol represented by a switching topology where only one agent can transmit at a time. A Lyapunov function is proposed to prove the exponential convergence of this class of consensus algorithms under mild assumptions. We also provide an explicit bound for the consensus errors that depends on the communication protocol parameters.

Keywords: Traffic control, Automotive control
Abstract: We use a monotonicity-based approach to design a safety controller in two realistic driving situations: a vehicle-following scenario and an unprotected left turn scenario. For each scenario we construct a symbolic abstraction of the system and efficiently synthesize a safety controller by exploiting the monotonicity property of the dynamics. We show how monotonicity property makes it possible to deal with complex scenarios, such as the vehicle following scenario with safe impact and left turn scenario, while handling model uncertainty.

Keywords: Automotive systems, Automotive control
Abstract: This work focuses on comparing the ecological potential of variable speed limits (VSL) and signalized access control. A synthetic two-region network composed of an urban and a peri-urban area is considered, and this study aims at improving the energy efficiency in both areas. A microscopic traffic simulator (SUMO) is used to model the dynamics of the system. This latter is controlled by a nonlinear model predictive control (NMPC) framework based on a macroscopic traffic model, which is an adapted version of the cell transmission model (CTM). The controller is coupled with an artificial neural network (ANN) to predict the fuel consumption. Finally, microscopic physical energy and NOx models are used to evaluate the performance of both control actuators. The results reveal that VSL is more promising due to the smoother variation of the densities.

Keywords: Autonomous vehicles, Traffic control, Automotive control
Abstract: Connected and automated vehicles (CAVs) provide the most intriguing opportunity to reduce pollution, energy consumption, and travel delays. In this paper, we address the problem of vehicle platoon formation in a traffic network with partial CAV penetration rates. We investigate the interaction between CAV and human-driven vehicle (HDV) dynamics, and provide a rigorous control framework that enables platoon formation with the HDVs by only controlling the CAVs within the network. We present a complete analytical solution of the CAV control policy and the conditions under which a platoon formation is feasible. We evaluate the solution and demonstrate the efficacy of the proposed framework using numerical simulation.

Keywords: Hybrid systems, Optimization, Traffic control
Abstract: Motivated by the shallow concavity properties that emerge in certain response maps in the context of optimization problems in transportation systems, we study the stability properties of a class of hybrid accelerated extremum seeking (HAES) dynamics interconnected with dynamic plants in the loop. In particular, we establish suitable semi-global practical asymptotic stability properties for different classes of cost functions, as well as tuning conditions for the hybrid extremum seeking algorithm. Additionally, we implement the HAES to optimize the performance of a self-organizing traffic light system (SOTL) in a class of smart transportation systems. We show that the dynamic momentum mechanism incorporate by the HAES can significantly reduce the convergence time in the optimization process compared to the traditional extremum seeking algorithms based on gradient descent flows.

Keywords: Traffic control, Predictive control for linear systems, Transportation networks
Abstract: Adequate traffic signal control strategies are essential to achieve a significant reduction of traffic congestion in urban environment. This work presents a receding-horizon approach for the optimal management of a singular signalized intersection via a computationally efficient Model Predictive Control (MPC) formulation. The control strategy aims at minimizing the overall number of vehicles in queue at the traffic lights in each road, while satisfying additional safety constraints connected to the intersection’s layout. Pedestrian requests are explicitly handled and potential deadlock situations in low traffic scenarios are avoided. The presented approach is validated through a realistic microscopic traffic simulator based on SUMO, in which a real intersection layout from the Italian city of Monza has been accurately reproduced and real-world traffic profiles have been provided as input.

Keywords: Traffic control, Transportation networks, Smart cities/houses
Abstract: We consider an optimal traffic-light control framework for urban traffic intersections to alleviate congestion phenomena. We analyze a scenario in which we provide drivers with information about the waiting time at the intersections. We model the drivers’ lane-changing information-based behavior as the solution of a convex optimization problem. We compute the optimal traffic-light control mechanism as the solution to a bi-level optimization problem. We provide a complete analysis in terms of (i) the existence of a solution; (ii) an iterative algorithm to compute it; (iii) sufficient conditions for the solution’s uniqueness and the algorithm’s convergence. Early simulation results show the proposed control scheme’s effectiveness compared with an optimal control algorithm in the absence of waiting-time information.

Keywords: Control of networks, Robust control, Control over communications
Abstract: This paper addresses the problem of robust control of a linear discrete-time system subject to bounded disturbances and to measurement and control budget constraints.

Using Q-parameterization and a polytope containment method, we prove that the co-design of an affine feedback controller, a measurement schedule and a control schedule can be exactly formulated as a mixed integer linear program with 2 binary variables per time step. As a consequence, this problem can be solved efficiently, even when an exhaustive search for measurement and control times would have been impossible in a reasonable amount of time.

Keywords: Control over communications, Networked control systems, Stability of linear systems
Abstract: We propose (k,k') stabilizing codes, which is a type of delayless error correction codes that are useful for control over networks with erasures. For each input symbol, k output symbols are generated by the stabilizing code. Receiving any k' of these outputs guarantees stability. Thus, the system to be stabilized is taken into account in the design of the erasure codes. Our focus is on LTI systems, and we construct codes based on independent encodings and multiple descriptions. The theoretical efficiency and performance of the codes are assessed, and their practical performances are demonstrated in a simulation study. There is a significant gain over other delayless codes such as repetition codes.

Keywords: Control over communications, Optimization, Randomized algorithms
Abstract: We consider here a scenario where a team of agents want to switch from a fully connected communication network to a network with limited bandwidth because of a cyber attack. The goal is to identify which communication link should still be active in the new network for near-optimal overall performance. This is formulated as a cardinality constrained quadratic minimization problem, which is NP-hard in general. To obtain an approximately optimal solution efficiently, we used random projection (RP) for dimensionality reduction. We show that the value of the sketched team is bounded above by a constant factor times the optimal value of the team with high probability.

Keywords: Control over communications, Sampled-data control
Abstract: In this paper, we focus on reducing the wireless network utilization between an operator and a controlled dynamical system in a discrete-time setting, where the objective of the operator (e.g., ground station) is to provide guidance commands to the controlled dynamical system for command following purposes. To this end, we propose a discrete-time event-triggering approach in order to schedule guidance data transmissions between the operator and the controlled dynamical system. When an event occurs a new guidance command information is sent to the controlled dynamical system through three rules: The first rule is the common one in the literature and it is based on sending sampled points of of the guidance commands. In contrast to the existing results in the literature, the second rule is based on exchanging approximated curve-fitted functions , and the third rule is based on exchanging exact guidance command functions that are valid over a specified time-interval. We provide a rigorous stability analysis of the proposed approach and show that the second and the third rules achieve less guidance data transmissions (i.e., events) between the operator and the controlled dynamical system through an numerical example.

Keywords: Control over communications, Cooperative control, Estimation
Abstract: This paper extends an event-based method for the cooperative control of two mobile objects to cope with unknown signal delays induced by the communication network and the computation times of the controllers. The objects are locally controlled and move autonomously along their planned trajectories. The first object called the stand-on object can freely change its trajectory at any time. The second object named the give-way object has to adapt its trajectory to avoid collisions by invoking communication at event time instants. It is provided with an event-based control unit introduced in a previous paper. Whereas the signal delays were neglected in earlier publications, in this paper these delays are considered and the event generation is improved accordingly. An event-based delay estimator generates an estimate of the signal delays, which is utilised for the event generation by using a two-state Markov model that represents the properties of the network. The event-based approach is derived by combining methods from communication technology and control theory. A simulation study with two quadrotors shows the benefits of the method.

Keywords: Machine learning, Control over communications, Optimization algorithms
Abstract: We propose an adaptive quantization method for two important distributed computation tasks: federated learning and distributed optimization. In both settings, we propose adaptive bit allocation schemes that allow nodes to trade their bandwidth with a minimal communication overhead. We show that the proposed schemes lead to improvement in the speed of convergence of these methods compared to a uniform bit allocation method, especially when the data distribution among the nodes is skewed. Our theoretical results are corroborated by extensive simulations on various datasets.

Keywords: Linear systems, Behavioural systems, Formal Verification/Synthesis
Abstract: Motivated by the growing requirements on the operation of complex engineering systems, we present contracts as specifications for continuous-time linear dynamical systems with inputs and outputs. A contract is defined as a pair of assumptions and guarantees, both characterized in a behavioural framework. The assumptions encapsulate the available information about the dynamic behaviour of the environment in which the system is supposed to operate, while the guarantees express the desired dynamic behaviour of the system when interconnected with relevant environments. In addition to defining contracts, we characterize contract implementation, and we find necessary conditions for the existence of an implementation. We also characterize contract refinement, which is used to characterize contract conjunction in two special cases. These concepts are then illustrated by an example of a vehicle following system.

Keywords: Formal Verification/Synthesis, Decentralized control, Stability of hybrid systems
Abstract: Attractivity specifications consist in driving the state of a system to a target region and to keep it in that region afterwards. In this paper, we develop a compositional approach to symbolic controller synthesis for attractivity specifications. The approach consists in computing iteratively for each subsystem, refinements of the least-violating attractivity controller, and of the associated attractor. The controllers and attractors computed at a given iteration are used at the next iteration as control and external input constraints. The resulting fixed-point algorithm allows us to compute a decentralized attractivity controller for the interconnected system which minimizes the size of the attractor. To illustrate the effectiveness of our approach, we show an application for the temperature regulation of adjacent rooms of a building.

Keywords: Large-scale systems, Computational methods, Optimization
Abstract: This paper addresses the challenge of solving large-scale Mixed Integer Linear Programs (MILPs). A resolution scheme is proposed for the class of MILPs with a hidden constraint-coupled multi-agent structure. In particular, we focus on the problem of disclosing such a structure to then apply a computationally efficient decentralized optimization algorithm recently proposed in the literature. The multi-agent reformulation problem consists in manipulating the matrix defining the linear constraints of the MILP so as to put it in a singly-bordered block-angular form, where the blocks define local constraints and decision variables of the agents, whereas the border defines the coupling constraints. We translate the matrix reformulation problem into a hyper-graph partitioning problem and introduce a novel algorithm which accounts for the specific requirements on the singly-bordered block-angular form to best take advantage of the decentralized optimization approach. Numerical results show the effectiveness of the proposed hyper-graph partitioning algorithm.

Keywords: Large-scale systems, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper deals with general large-scale interconnected systems with general network topology, affected by external disturbances, and with the possibility to ensure the overall stability when some sufficient conditions are met by each agent with respect to its neighbors. We introduce the notion of scalable Mesh Stability (sMS), that requires the existence of trajectory bounds that do not depend on the number of subsystems. The immediate consequence is that perturbations originating in a point of the interconnected system do not amplify through it. A numerical example on interconnection of microgrids shows the interest and the effectiveness of the theoretical result.

Keywords: Large-scale systems, Hybrid systems, Network analysis and control
Abstract: In this paper, we consider the problem of verifying initial-state opacity for networks of discrete-time control systems. We formulate the opacity property as a safety one over an appropriately constructed augmented system, and aim to verify this latter property by finding suitable barrier certificates. To reduce the computational complexity associated with computing barrier certificates for large networks, we propose a compositional approach to construct such barrier certificates for large-scale interconnected systems. This is achieved by introducing local barrier certificates for subsystems in the network and imposing some small-gain type conditions on the gains of those local barrier certificates. We also provide sufficient conditions for verifying the lack of opacity in large-scale networks by constructing barrier certificates ensuring some reachability properties over the augmented systems. To illustrate the effectiveness of our results, we consider the problem of tracking a target using a team of vehicles and verify if its initial position is secret from possible outside intruders.

Keywords: Large-scale systems, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper extends the notion of delta dissipativity, originally introduced in a game theoretic context, to general interconnections of dynamical systems. The main contribution of this paper is a compositionality result that presents conditions under which a large-scale interconnection of delta dissipative systems is delta dissipative. We adapt this result to also analyze stability and asymptotic stability of equilibrium points for the interconnection. Additionally, we formulate a sum-of-squares program for verifying delta dissipativity of a (polynomial) system. The results are illustrated with examples.

Keywords: Control applications, Linear parameter-varying systems, Energy systems
Abstract: Wind turbine testing is of great relevance for the wind industry to reduce the levelized cost of energy. For this purpose, hardware-in-the-loop simulators have been developed within the last years and the concept has been validated using multi-megawatt state-of-the-art nacelles. From the experimental experience of several years, different physical phenomena have been identified, which are insufficiently and unsystematically handled by state-of-the-art control algorithms. These effects include multi-physical couplings, nonlinear friction and periodic disturbances due to unbalanced masses. We analyze the test bench control problem by means of linear parametervarying (LPV) control theory and arrive at a formulation with the scheduling parameters rotation speed and torque. Based on experimental data, we derive a quasi-LPV plant model for which controller synthesis is conducted. In particular, the superior performance of different self-scheduled dynamic output feedback LPV controllers compared to a standard robust controller is shown. By using parameter-dependent Lyapunov functions, we guarantee both stability and performance with respect to scheduling parameters with a bounded rate of variation. Furthermore, we propose an approach for systematically handling highly dynamic fault cases on a test bench by reconfiguring controller properties through the introduction of fault specific dynamic weighting functions. A converter failure test case is analyzed in simulation to show the benefits of the here proposed controller design.

Keywords: Automotive control, Automotive systems, Optimal control
Abstract: While it is widely known that electric vehicles will contribute very substantially to the reduction of exhaust emissions, it is less known that the impact of particulates, in particular from tires, will become larger, due to both the weight of these vehicles and the torque profile of electrical machines. In this paper, we use a recently developed tire emissions estimation method to develop a control approach that reduces these emissions. We show in a comprehensive simulation study using real driving data from German highways that the tire particle controller not only reduces tire emissions but at the same time allows to ensure ride comfort. These results illustrate that there is potentially a substantial environmental benefit to be expected especially depending on the presence, type and prediction of a preceding vehicle.

Keywords: Biomolecular systems
Abstract: Engineering reliable synthetic circuits in living organisms is very challenging because of molecular fluctuations, cell-to-cell variability and metabolic burden, for instance. Recently, the antithetic integral controller (AIC) has been proposed as an effective strategy to design robust synthetic circuits in living cells. In its canonical form, the AIC acts at the single-cell level to regulate the abundance of a certain intracellular component to a prescribed set-point. In this work, we propose a variant of the AIC that allows the control of collective properties of a dynamic cell population, such as the cell number or the total amount of protein expressed across the population. The resulting controller -- which we term shared AIC (sAIC) -- uses a single controller network that acts on all cells simultaneously through a shared environment. We describe the sAIC mathematically using a stochastic multiscale formalism, which accounts for noisy cell-internal dynamics as well as cell division and death events. We demonstrate the effectiveness of the sAIC approach using two simulation-based case studies.

Keywords: Control applications, Robust control, PID control
Abstract: The goal of this paper is to investigate if it is possible to upgrade a given linear controller to a sliding mode one with an improvement of the control performance. Starting from a given linear controller, a design procedure for a sliding mode control having better performance than the linear one, is proposed. If the system has disturbances, which is always the case in experiments, the upgraded sliding mode control brings also a better robustness with respect to the given linear robust controller. The main idea is to divide the state-space into two areas, introducing a design parameter which separates the area of the linear control from the area of the sliding mode control. Some issues related to the chattering reduction are discussed. The control scheme’s efficiency is demonstrated experimentally on a rotary inverted pendulum. The experimental results demonstrate the effectiveness of the obtained controls, and show an improvement with respect to the given linear proportional control.

Keywords: Optimal control, Optimization
Abstract: This paper examines the problem of simultane- ously optimizing the spooling and cross-current flight tra- jectories of a tethered underwater energy harvesting kite. This work is motivated by the potential of tethered kites to provide attractive levelized costs of electricity, especially when cross-current motion is exploited in order to maximize energy harvesting. The literature explores trajectory optimization for tethered energy harvesting, for both airborne and marine hydrokinetic energy systems. However, the simultaneous co-optimization of both the spooling and cross-current trajectories of a marine hydrokinetic kite remains relatively unexplored. The paper formulates this co-optimization problem using a 3 degree-of-freedom kite model, coupled with an inelastic tether. A Fourier series expansion is then used for solving this co-optimization problem, and the key features of the resulting optimal trajectory are analyzed. Significant energy harvesting is achieved through co-optimization, approaching theoretical maximum power for cross-current systems.

Keywords: Transportation networks
Abstract: As electric vehicles (EV) gradually replace traditional fuel vehicles and provide transportation services in cities, eg electric taxi/bus fleets, solar-powered charging stations with energy storage systems have been deployed in urban areas to provide charging services for EV fleets~cite{solarcs}. The mixture of solar-powered and traditional charging stations brings efficiency challenges to charging stations and reliability challenges to power systems. In this paper, we explore e-taxis' mobility and charging demand flexibility to co-optimize service quality of e-taxi fleets and system cost of charging infrastructures. Specifically, we propose SAC, an e-taxi coordination framework to dispatch e-taxis for charging or serving passengers under spatio-temporal dynamics of renewable energy and passenger mobility. We formulate the e-taxi fleet coordination problem as a multi-criterion mixed-integer linear programming problem. We evaluate our solution with a comprehensive dataset for e-taxi systems and charging infrastructures including 726 e-taxis, 7,228 regular fuel taxis, 37 working charging stations, and 62,100 collected taxi trips per day. Our data-driven evaluation shows that SAC significantly outperforms existing solutions, reducing the total reverse power flow per day by up to 95.3%, while maintaining e-taxi service quality with very small overhead.

Keywords: Biological systems, Stochastic systems, Identification
Abstract: The outbreak of the COVID-19 pandemic in 2020 has renewed the interest in epidemic models, striving to infer fruitful information from the available data. The whole world has faced the urge for a sudden comprehension of the spread of the virus and different approaches are nowadays available to cope with the inherent stochasticity of the phenomenon, the fragmentary fashion of usable data and the identifiability problems related to them. This work proposes a novel approach to identify a basic SIR epidemic model with time-varying parameters, where Susceptibles, Infected and Removed (i.e. recovered and deceased) people are accounted for. The standard deterministic approach trivially exploits the average evolution only, disregarding any other information carried out by the epidemiological data. Instead, by suitably formulating a discrete stochastic framework for the mathematical model, the identification task is carried out by exploiting raw data to compute the higher-order moments evolution and involve them in the identification task. The methodology is applied to the Italian COVID-19 case study and shows promising results obtained according to rough epidemic data, essentially provided by the overall amount of contaminated individuals.

Keywords: Control of networks, Game theory, Stochastic systems
Abstract: In this paper, we show how a dynamic population game can model the strategic interaction and migration decisions made by a large population of agents in response to epidemic prevalence. Specifically, we consider a modified susceptible-asymptomatic-infected-recovered (SAIR) epidemic model over multiple zones. Agents choose whether to activate (i.e., interact with others), how many other agents to interact with, and which zone to move to in a time-scale which is comparable with the epidemic evolution. We define and analyze the notion of equilibrium in this game, and investigate the transient behavior of the epidemic spread in a range of numerical case studies, providing insights on the effects of the agents' degree of future awareness, strategic migration decisions, as well as different levels of lockdown and other interventions. One of our key findings is that the strategic behavior of agents plays an important role in the progression of the epidemic and can be exploited in order to design suitable epidemic control measures.

Keywords: Healthcare and medical systems, Biological systems, Optimization
Abstract: In this paper, we focus on the effect that testing centers (which detect and quarantine infected individuals) have on mitigating the evolution of an epidemic. We incorporate diffusion-style mobility of infected but undetected individuals, as opposed to detected and quarantined ones. We compute the total and maximum (over time) spatially averaged density of infected individuals (detected or not), which are useful metrics of the epidemic's impact on a population, as functions of the testing center spatial density. Even under conditions where the epidemic has the natural potential to spread, we find that a `phase transition' occurs as the testing center spatial density increases. For any testing density above a certain threshold the epidemic is suppressed and dies out, while below it propagates and evolves naturally albeit still strongly depending on the testing center density. This analysis further allows to optimize the testing certain density so that the epidemic's evolution does not inundate or exhaust critical health care resources, like ICU bed capacity.

Keywords: Agents-based systems, Network analysis and control
Abstract: This paper proposes a discrete-time network susceptible-infected-susceptible (SIS) model coupled with opinion dynamics, where the opinion dynamics models each individual's perceived severity of illness or perceived susceptibility. The effects of the opinion dynamics on the network SIS model are studied by analyzing the limiting behaviors of the system and providing illustrative simulations.

Keywords: Network analysis and control, Networked control systems
Abstract: We propose an SIR epidemic model coupled with opinion dynamics to study an epidemic and opinions spreading in a network of communities. Our model couples networked SIR epidemic dynamics and opinions towards the severity of the epidemic. We develop an epidemic-opinion based threshold condition to capture the moment when a weighted average of the epidemic states starts to decrease exponentially fast over the network, namely the peak infection time. We define an effective reproduction number to characterize the behavior of the model through the peak infection time. We use both analytical and simulation-based results to illustrate that the opinions reflect the recovered levels within the communities after the epidemic dies out.

Keywords: Biomedical, Biological systems, Control applications
Abstract: Feedback control is considered for a family of nonlinear, deterministic, lumped-parameter models of directly transmitted infectious diseases. Although the available input in the model is not a control signal in the traditional sense, the application of feedback control design can offer guidance to public health and government organizations in implementing actions that curb the spread, minimize the infectious peak, avoid multiple peaks, reduce the number of fatalities, or address other performance criteria. A linearized model is useful in gaining valuable insight into the infectious dynamic spread behavior. Discrete-time models are valuable to explore heuristic policies as may be implemented by governments. The Data Visualization tool from the John's Hopkins CORONAVIRUS Resource Center is used to explore the behavior of the epidemic in Texas.

Keywords: Learning, Manufacturing systems and automation, Robotics
Abstract: Accurate positioning and fast traversal times determine the productivity in machining applications. This paper demonstrates a hierarchical contour control implementation for the increase of productivity in positioning systems. The high-level controller pre-optimizes the input to a low-level cascade controller, using a contouring predictive control approach. This control structure requires tuning of multiple parameters. We propose a sample-efficient joint tuning algorithm, where the performance metrics associated with the full geometry traversal are modeled as Gaussian processes and used to form the global cost and the constraints in a constrained Bayesian optimization algorithm. This approach enables the trade-off between fast traversal, high tracking accuracy, and suppression of vibrations in the system. The performance improvement is evaluated numerically when tuning different combinations of parameters. We demonstrate that jointly tuning the parameters of the contour- and the low-level controller achieves the best performance in terms of time, tracking accuracy, and minimization of the vibrations in the system.

Keywords: Robotics, Constrained control, Uncertain systems
Abstract: Control Barrier Functions (CBF) are widely used to enforce the safety-critical constraints on nonlinear systems. Recently, these functions are being incorporated into a path planning framework to design safety-critical path planners. However, these methods fall short of providing a realistic path considering both the algorithm's run-time complexity and enforcement of the safety-critical constraints. This paper proposes a novel motion planning approach using the well-known Rapidly Exploring Random Trees (RRT) algorithm that enforces both CBF and the robot Kinodynamic constraints to generate a safety-critical path. The proposed algorithm also outputs the corresponding control signals that resulted in the obstacle-free path. The approach also allows considering model uncertainties by incorporating the robust CBF constraints into the proposed framework. Thus, the resulting path is free of any obstacles and accounts for the model uncertainty from robot dynamics and perception. Result analysis indicates that the proposed method outperforms various conventional RRT-based path planners, guaranteeing a safety-critical path with minimal computational overhead. We present numerical validation of the algorithm on the Hamster V7 robot car, a micro autonomous Unmanned Ground Vehicle that performs dynamic navigation on an obstacle-ridden path with various uncertainties in perception noises and robot dynamics.

Keywords: Robotics, Variational methods, Agents-based systems
Abstract: Guaranteed safe online trajectory planning is becoming an increasingly important topic of robotic research, due to the need to react quickly in unknown environments. However,as a result of modelling mismatch, some error during trajectory tracking is inevitable. In this paper, we present Planner-Aware FaSTrack, or PA-FaSTrack, which provides guaranteed Tracking Error Bounds (TEBs) by solving a Hamilton-Jacobi(HJ) variational inequality in the tracking error space. PA-FaSTrack improves upon the state-of-the-art method, FaSTrack, by accounting for motion primitives implied by the planning algorithm in the problem formulation. Our method provides a sequence of TEBs, with each TEB corresponding to a segment of the planned path. We also propose necessary modifications to real time tree based planning algorithms in order to make them compatible with the provided TEB sequence. By integrating planning and tracking more closely together, we greatly decrease the degree of conservatism compared to the original FaSTrack, allowing the autonomous system to navigate safely through much narrower spaces. We demonstrate our method using two representative dynamical systems.

Keywords: Robotics, Nonlinear systems, Manufacturing systems and automation
Abstract: We describe a novel path following pose control framework for articulated robots which is needed for many material handling and surface finishing where constant speed travel is desirable. Using a rotation minimizing frame (RMF) associated with the geometric path, we develop a path following position control law by projecting the robot translation states onto the RMF and based on an analytical description of the reference orientation dynamics of the RMF, we derive a stabilizing controller for orientation control along the path with the Modified Rodrigues parameters to avoid the unwinding problem encountered when rotations of more than 180 degrees are encountered. The effectiveness of the proposed path following framework is verified via simulations on a torque actuated KUKA iiwa robot using a physics-based simulation engine called Bullet.

Keywords: Robotics, Manufacturing systems and automation, Optimization algorithms
Abstract: Workpiece localization is the process of obtaining the location of a workpiece in a robot workspace. The location (position and orientation) is represented by the transformation between the workpiece (local) coordinate frame and the reference (world) frame. In this work, we propose a workpiece localization strategy to automate the localization process by collecting data sequentially and efficiently without the two common restrictive assumptions: the data used to calculate the transformation is readily available and the correspondence between the features used for calculation is known. Correspondingly, two subproblems are involved: (1) determining the correspondence between the measured data and the CAD model data, and (2) determining the next-best-views (NBVs) in case of limited measurement data. We assume the workpiece is convex and has at least three flat surfaces. We use the extended Gaussian images (EGIs) from the Gauss map of both the CAD model point clouds and measured point clouds to find the flat surfaces on the workpiece. A mixed integer convex optimization problem is solved to estimate the correspondence and the rotation between the flat surfaces in the CAD model and the measured point clouds. The translation part of the homogeneous transformation is obtained by solving a least-squares problem using the estimated correspondence. Potential views for further measuring the workpiece are generated by evaluating a defined search region to find the NBVs based on a specified criterion. The workpiece is considered to be fully localized when the distances in the estimated homogeneous transformation matrices are within a predefined threshold. Simulation results are provided to show the effectiveness of the proposed localization method.

Keywords: Autonomous robots, Robotics, Nonlinear systems
Abstract: The method presented in this paper exploits the concept of Control-Coding to enable the implicit transfer of binary information between agents in multi-robot systems. Binary sequences are encoded into the robot’s control action by overlaying secondary control signals on top of the primary control signal, thereby forming a controller-encoder pair which simultaneously (i) stabilizes the robot with respect to its control objective, and (ii) transmits information through the motion of the robot. This motion can be detected and decoded by other robots, so as to extract the transmitted information. The effectiveness of the method is demonstrated by simulating a simple coordination task between two unicycles with limited sensing and non-existent communication capabilities.

Keywords: Mechatronics
Abstract: A haptic interface is a mechatronic system that modulates the physical interaction between a human and their tangible surroundings. Such systems typically take the form of grounded kinesthetic devices, ungrounded wearable devices, or surface devices, and they enable the user to act on and feel a remote or virtual environment. I will elucidate key approaches to creating outstanding haptic interfaces by showcasing examples of my team’s research on both kinesthetic and wearable devices. I will then transition to talking about physical human-robot interaction (pHRI), where the engineered system acts as a social agent rather than a tool. In addition to inventing tactile sensors, we have developed and evaluated methods that allow a robot to learn dynamic physical interactions from demonstrations. We also created HuggieBot, a custom robot that uses visual and haptic sensing to give good hugs. Being held in the arms of a robot highlights the potential social and physical consequences of well-designed control systems and decision-making algorithms. Along the way, I will also share suggestions on recruiting and supporting a diverse team of researchers.

Keywords: Information theory and control
Abstract: Information theory provides a powerful framework for finding fundamental bounds in many areas of science and engineering. Using the common currency of bits, it enables the complexity of a broad range of different systems to be evaluated and compared. This talk is motivated by real-time autonomous navigation problems, in which high-dimensional state variables and measurements (e.g. vision) must be processed with low latency and high reliability. This can challenge the communication and computational capabilities available onboard, with adverse impacts on closed-loop performance. The aim of this talk is to explain how such impacts can be understood in terms of bits.

The first part considers partially observed Markov decision processes, and investigates measures of uncertainty based on the entropy of the state trajectory. Such measures are relevant for persistent navigation and localization problems, since they capture the minimum internal cost, in bits, of storing the belief of an agent about where it is and was, given what it has seen and done. In robotics, (conditional) trajectory entropy has previously been dismissed as intractable, due to the appearance of the entire trajectory in the nonlinear entropy functional. It is shown that, surprisingly, trajectory entropy can be put into convenient stage-additive forms that allow solution by standard techniques. In simulations, the optimal policies that arise are qualitatively and quantitatively different from those produced by previous entropy or information-based approaches, and produce trajectories reminiscent of the motion of some animals.

The second part of this talk focuses on linear models of motion. By adapting universal performance bounds from bit-rate-limited control theory, unexpected fundamental trade-offs are obtained between processor speed, measurement resolution, and sampling rate, in the presence of disturbances. The implications for vision-based autonomous navigation with integrator dynamics are discussed.

Keywords: Stochastic optimal control, Nonlinear systems identification, Predictive control for nonlinear systems
Abstract: This paper considers the problem of determining an optimal control action based on observed data. We formulate the problem assuming that the system can be modelled by a nonlinear state-space model, but where the model parameters, state and future disturbances are not known and are treated as random variables. Central to our formulation is that the joint distribution of these unknown objects is conditioned on the observed data. Crucially, as new measurements become available, this joint distribution continues to evolve so that control decisions are made accounting for uncertainty as evidenced in the data. The resulting problem is intractable which we obviate by providing approximations that result in finite dimensional deterministic optimisation problems. The proposed approach is demonstrated in simulation on a nonlinear system.

Keywords: Predictive control for linear systems, Machine learning, Learning
Abstract: Despite recent advances in computing hardware and optimization algorithms, solving model predictive control (MPC) problems in real time still poses some technical challenges when long prediction and control horizons are used, due to the presence of several optimization variables and constraints. In this paper we propose to reduce the computational burden by shortening the prediction and control horizon to a single step while preserving good closed-loop performance. This is achieved by using machine learning techniques to construct a tailored quadratic and convex terminal cost that approximates the cost-to-go function of constrained linear (possibly parameter-dependent) MPC formulations. The potentials of the proposed MPC with Learned Terminal Cost (LTC-MPC) approach is demonstrated in two numerical examples.

Keywords: Optimal control, Constrained control, Predictive control for nonlinear systems
Abstract: We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.

Keywords: Constrained control, Predictive control for nonlinear systems, Uncertain systems
Abstract: Perception-aware control systematically considers the interdependence between perception and control to optimize the overall performance of the closed-loop system subject to state and input-dependent uncertainty. That is, it accounts for the impact of control on sensing and of sensing on control. Recently, we proposed a perception-aware chance-constrained MPC (PAC-MPC) that considers the impact of control on the evolution of the environment uncertainty. In this paper, we obtain a stabilizing design for the PAC-MPC by first determining stability conditions in a general nonlinear setting and then deriving specific design rules for the linear-Gaussian case. The latter case results in a specific choice of the MPC cost function parameters, and in design conditions for that the estimation algorithm, that determine uncertainty propagation in the MPC prediction model, must satisfy.

Keywords: Predictive control for linear systems, Uncertain systems, Learning
Abstract: In this work we address the problem of performing a repetitive task when we have uncertain observations and dynamics. We formulate this problem as an iterative infinite horizon optimal control problem with output feedback. Previously, this problem was solved for linear time-invariant (LTI) system for the case when noisy full-state measurements are available using a robust iterative learning control framework, which we refer to as robust learning-based model predictive control (RL-MPC). However, this work does not apply to the case when only noisy observations of part of the state are available. This limits the applicability of current approaches in practice: First, in practical applications we typically do not have access to the full state. Second, uncertainties in the observations, when not accounted for, can lead to instability and constraint violations. To overcome these limitations, we propose a combination of RL-MPC with robust output feedback model predictive control, named robust learning-based output feedback model predictive control (RLO-MPC). We show recursive feasibility and stability, and prove theoretical guarantees on the performance over iterations. We validate the proposed approach with a numerical example in simulation and a quadrotor stabilization task in experiments.

Keywords: Predictive control for linear systems, Constrained control, Iterative learning control
Abstract: In practical model predictive control (MPC) implementations, constraints on the states are typically softened to ensure feasibility despite unmodeled disturbances. In this work, we propose a soft constrained MPC formulation supporting polytopic terminal sets in half-space and vertex representation, which significantly increases the feasible set while maintaining asymptotic stability in case of constraint violations. The proposed formulation allows for leveraging system trajectories that violate state constraints to iteratively improve the MPC controller's performance. To this end, we apply convex optimization techniques to obtain a data-driven terminal cost and set, which result in a quadratic MPC problem.

Keywords: Learning, Agents-based systems, Uncertain systems
Abstract: Formation control algorithms for multi-agent systems have gained much attention in the recent years due to the increasing amount of mobile and aerial robotic swarms. The design of safe controllers for these vehicles is a substantial aspect for an increasing range of application domains. However, parts of the vehicle's dynamics and external disturbances are often unknown or very time-consuming to model. To overcome this issue, we present a formation control law for multi-agent systems based on double integrator dynamics by using Gaussian Processes for online learning of the unknown dynamics. The presented approach guarantees a bounded error to the desired formation with high probability, where the bound is explicitly given. A numerical example highlights the effectiveness of the learning-based formation control law.

Keywords: Learning, Biomedical, Predictive control for nonlinear systems
Abstract: Model predictive control (MPC) is becoming one of the leading modern control approaches applied to physiological control systems. However, intra- and interpatient variability usually requires an adaptation of the model to each individual patient or otherwise deeming the controller too conservative. The incorporation of learning in model predictive control is subject to ongoing intensive research to provide tractable and safe implementation in practice. Gaussian processes (GPs) among other learning approaches have been proposed for learning uncertain or unknown system dynamics as well as time varying disturbances. However, the naive incorporation of GPs into MPC, commonly results in complex and nonlinear optimization problems. In this paper, we propose a practical stochastic MPC implementation, that utilizes estimates of the parameter uncertainties and nonlinearities of the system as well as external additive disturbances. By using a linear nominal model augmented with two separate GPs, nonlinearities depending on the state and input as well as temporal disturbances can be considered efficiently in the MPC framework. An iterative optimization scheme is introduced using quadratic programming to circumvent solving a stochastic nonlinear program. The applicability of the proposed approach is demonstrated on a pressure controlled mechanical ventilation problem.

Keywords: Vision-based control, Machine learning, Cooperative control
Abstract: This paper considers a pursuit control based on cooperative estimation of target motion by robotic networks equipped with visual sensors. First, we propose a cooperative pursuit control law with a vision-based observer using visual sensor networks, called networked visual motion observer. Then, we learn position dependent target motion by a Gaussian process and integrate it within the proposed control law. Second, we show that all rigid bodies converge to desired relative poses when at least one robot can obtain visual information of the target. Furthermore, we prove that the total estimation and control error are ultimately bounded with high probability when integrating a GP model. Finally, we demonstrate the effectiveness of the proposed control law through simulations.

Keywords: Learning, Identification, Statistical learning
Abstract: Simultaneous estimation of related tasks has been widely studied in the statistics and machine learning literature, and its effectiveness has been proven in many contexts such as econometrics and bioinformatics. However, state-of-the-art approaches leveraging Gaussian processes are encumbered by high computational costs that hinder their applicability to model-based and adaptive control design. In this paper, we address this issue by approximating non-parametric multi-task models by means of trigonometric basis functions. We estimate the involved parameters in a Bayesian framework using several deterministic and stochastic approaches, and highlight their advantages within an extensive comparative study. Overall, the proposed setup is able to suitably leverage task relatedness to outperform single-task methods, especially when single datasets are small.

Keywords: Predictive control for nonlinear systems, Distributed control, Identification for control
Abstract: This paper focuses on distributed learning-based control of decentralized multi-agent systems where the agents’ dynamics are modeled by Gaussian Processes (GPs). Two fundamental problems are considered: the optimal design of experiment for learning of the agents’ GP models concurrently, and the distributed coordination given the learned models. Using a Distributed Model Predictive Control (DMPC) approach, the two problems are formulated as distributed optimization problems, where each agent’s sub-problem includes both local and shared objectives and constraints. To solve the resulting complex and non-convex DMPC problems efficiently, we develop an algorithm called Alternating Direction Method of Multipliers with Convexification (ADMM-C) that combines a distributed ADMM algorithm and a Sequential Convex Programming method. We prove that, under some technical assumptions, the ADMM-C algorithm converges to a stationary point of the penalized optimization problem. The effectiveness of our approach is demonstrated in numerical simulations of a multi-vehicle formation control example.

Keywords: Formal Verification/Synthesis, Uncertain systems, Machine learning
Abstract: Autonomous systems often have complex and possibly unknown dynamics due to, e.g., black-box components. This leads to unpredictable behaviors and makes control design with performance guarantees a major challenge. This paper presents a data-driven control synthesis framework for such systems subject to linear temporal logic on finite traces (LTLf) specifications. The framework combines a baseline (offline) controller with a novel online controller and refinement procedure that improves the baseline guarantees as new data is collected. The baseline controller is computed offline on an uncertain abstraction constructed using Gaussian process (GP) regression on a given dataset. The offline controller provides a lower bound on the probability of satisfying the LTLf specification, which may be far from optimal due to both discretization and regression errors. The synergy arises from the online controller using the offline abstraction along with the current state and new data to choose the next best action. The online controller may improve the baseline guarantees since it avoids the discretization error and reduces regression error as new data is collected. The new data are also used to refine the abstraction and offline controller using local GP regression, which significantly reduces the computation overhead. Evaluations show the efficacy of the proposed offline-online framework, especially when compared against the offline controller.

Keywords: Machine learning, Learning, Stochastic optimal control
Abstract: We consider the problem of learning the optimal robust value function and the optimal robust policy in discounted-reward Robust Markov decision processes (RMDPs). The goal of the RMDP framework is to find a policy that is robust to the parameter uncertainties due to the mismatch between the simulator model and real-world settings. While the optimal robust value function and policy can be computed using robust dynamic programming, it requires the exact knowledge of the nominal simulator model and the uncertainty set around it. This paper proposes a model-based robust reinforcement learning algorithm that learns an epsilon-optimal robust value function and policy in a finite state and action space setting when the exact knowledge of the nominal simulator model is not known. We assume access to a standard generative sampling model, which can generate next-state samples for all state-action pairs of the nominal simulator model. We give a precise characterization of the sample complexity of obtaining an epsilon-optimal robust value function and policy using our algorithm. Finally, we demonstrate the performance of our algorithm on a gridworld environment.

Keywords: Stochastic optimal control, Machine learning, Markov processes
Abstract: We consider Markov Decision Processes (MDPs) with deterministic transitions and study the problem of regret minimization, which is central to the analysis and design of optimal learning algorithms. We present logarithmic problem-specific regret lower bounds that explicitly depend on the system parameter (in contrast to previous minimax approaches) and thus, truly quantify the fundamental limit of performance achievable by any learning algorithm. Deterministic MDPs can be interpreted as graphs and analyzed in terms of their cycles, a fact which we leverage in order to identify a class of deterministic MDPs whose regret lower bound can be determined numerically. We further exemplify this result on a deterministic line search problem, and a deterministic MDP with state-dependent rewards, whose regret lower bounds we can state explicitly. These bounds share similarities with the known problem-specific bound of the multi-armed bandit problem and suggest that navigation on a deterministic MDP need not have an effect on the performance of a learning algorithm.

Keywords: Machine learning, Stochastic optimal control, Formal Verification/Synthesis
Abstract: We present a model-free reinforcement learning (RL) algorithm to find an optimal policy for a finite-horizon Markov decision process (MDP) while guaranteeing a desired lower bound on the probability of satisfying a signal temporal logic (STL) specification. We propose a method to effectively augment the MDP state space to capture the required state history and express the STL objective as a reachability objective. The planning problem can then be formulated as a finite-horizon constrained Markov decision process (CMDP). For a general finite-horizon CMDP problem with unknown transition probability, we develop a reinforcement learning scheme that can leverage any model-free RL algorithm to provide an approximately optimal policy out of the general space of non-stationary randomized policies. We illustrate our approach in the context of robotic motion planning for complex missions under uncertainty and performance objectives.

Keywords: Machine learning, Optimal control, Nonlinear systems
Abstract: Optimal control of nonlinear systems is a difficult problem which has been addressed by both the Control Theory (CT) and Reinforcement Learning (RL) communities. Frequently, the former relies on the linearization of the system thus obtaining only local guarantees. The latter relies on data to build model-free controllers, focused solely on performances. In this paper we propose a methodology to combine the advantages of both approaches, casting the formulation of an optimal local Linear Quadratic Regulator (LQR) into a Deep RL problem. Our solution builds on the linear framework to derive a learnt nonlinear controller showing local stability properties and global performances.

Keywords: Machine learning, Autonomous systems
Abstract: In this paper, a data-driven output feedback approach is developed for solving H-infinity control problem of linear discrete-time systems based on off-policy reinforcement learning (RL) algorithm. To alleviate the requirement to measure or estimate the system's states, past input-output measurements are leveraged to implicitly reconstruct the system's states. Then, an off-policy input-output Bellman equation is derived based on this implicit reconstruction to evaluate control policies using only input-output measurements. An improved control policy is then learned using the solution to the Bellman equation without requirement of knowing the system's dynamics. In the proposed approach, unlike the on-policy methods, the disturbance does not need to be updated in a predefined manner at each iteration, which makes it more practical. While the state-feedback off-policy RL method is shown to be a bias-free approach for deterministic systems, it is shown that once the system's states have been reconstructed from the input-output measurements, the input-output off-policy method cannot be considered as an immune approach against the probing noises. To cope with this, a discount factor is utilized in the performance function for decaying the deleterious effect of probing noises. Finally, to illustrate the sensitivity of the problem to the probing noises as well as the efficacy of the proposed approach, the flight control system is tested in the simulation.

Keywords: Machine learning, Optimization algorithms, Neural networks
Abstract: One of the main challenges in real-world reinforcement learning is to learn successfully from limited training samples. We show that in certain settings, the available data can be dramatically increased through a form of multi-task learning, by exploiting an invariance property in the tasks. We provide a theoretical performance bound for the gain in sample efficiency under this setting. This motivates a new approach to multi-task learning, which involves the design of an appropriate neural network architecture and a prioritized task-sampling strategy. We demonstrate empirically the effectiveness of the proposed approach on two real-world sequential resource allocation tasks where this invariance property occurs: financial portfolio optimization and meta federated learning.

Keywords: Nonlinear systems identification, Identification, Uncertain systems
Abstract: We revisit the setting and the assumptions that underlie the methodology of Dynamic Mode Decomposition(DMD) in order to highlight caveats as well as potential measures of when the applicability is warranted.

Keywords: Nonlinear systems identification, Neural networks, Learning
Abstract: This paper introduces recurrent equilibrium networks (RENs), a new class of nonlinear dynamical models for applications in machine learning and system identification. The new model class has “built-in” guarantees of stability and robustness: all models in the class are contracting – a strong form of nonlinear stability – and models can have prescribed Lipschitz bounds. RENs are otherwise very flexible: it contains, for example, all stable linear systems, all previously-known sets of contracting recurrent neural networks, all deep feedforward neural networks, and all stable Wiener/Hammerstein models. RENs are parameterized directly by a vector in R^N i.e. stability and robustness are ensured without parameter constraints, which simplifies learning since generic methods for unconstrained optimization can be used. The performance of the robustness of the new model set is evaluated on benchmark nonlinear system identification problems.

Keywords: Nonlinear systems identification, Nonlinear systems, Neural networks
Abstract: The present paper treats the identification of nonlinear dynamical systems using Koopman-based deep state-space encoders. Through this method, the usual drawback of needing to choose a dictionary of lifting functions a priori is circumvented. The encoder represents the lifting function to the space where the dynamics are linearly propagated using the Koopman operator. An input-affine formulation is considered for the lifted model structure and we address both full and partial state availability. The approach is implemented using the the deepSI toolbox in Python. To lower the computational need of the simulation error-based training, the data is split into subsections where multi-step prediction errors are calculated independently. This formulation allows for efficient batch optimization of the network parameters and, at the same time, excellent long term prediction capabilities of the obtained models. The performance of the approach is illustrated by nonlinear benchmark examples.

Keywords: Nonlinear systems identification, Subspace methods, Identification
Abstract: This paper introduces the concept of time-varying Koopman operator to predict the flow of a nonlinear dynamical system. The Koopman operator provides a linear prediction model for nonlinear systems in a lifted space of infinite dimension. An extension of time-invariant subspace realization methods known as the time-varying Eigensystem Realization Algorithm (TVERA) in conjunction with the time- varying Observer Kalman Identification Algorithm (TVOKID) are used to derive a finite dimensional approximation of the infinite dimensional Koopman operator at each time step. An isomorphic coordinate transformations is defined to convert different system realizations from different sets of experiments into a common frame for state propagation and to extract dynamical features in the lifted space defined by the eigenvalues of the Koopman operator. Two benchmark numerical examples are considered to demonstrate the capability of the proposed approach.

Keywords: Identification, Nonlinear systems identification, Differential-algebraic systems
Abstract: Differential-algebraic equations (DAEs) arise naturally as a result of equation-based object-oriented modeling. In many cases, these models contain unknown parameters that have to be estimated using experimental data. However, often the system is subject to unknown disturbances which, if not taken into account in the estimation, can severely affect the model's accuracy. For non-linear state-space models, particle filter methods have been developed to tackle this issue. Unfortunately, applying such methods to non-linear DAEs requires a transformation into a state-space form, which is particularly difficult to obtain for models with process disturbances. In this paper, we propose a simulation-based prediction error method that can be used for non-linear DAEs where disturbances are modeled as continuous-time stochastic processes. To the authors' best knowledge, there are no general methods successfully dealing with parameter estimation for this type of model. One of the challenges in particle filtering methods are random variations in the minimized cost function due to the nature of the algorithm. In our approach, a similar phenomenon occurs and we explicitly consider how to sample the underlying continuous process to mitigate this problem. The method is illustrated numerically on a pendulum example; the simulation results suggest that the method is able to deliver consistent estimates.

Keywords: Learning, Nonlinear systems identification, Uncertain systems
Abstract: In this paper, finite-time identification of discrete-time nonlinear system is studied without the persistence of excitation requirement. A finite-time learning method is introduced to learn the uncertainties of the discrete-time nonlinear systems’ dynamics that employs a memory stack of experienced data fulfilling an easy-to-check rank condition. The proposed method assures the convergence of the estimated parameters in finite time based on a Lyapunov analysis. Finally, simulation results demonstrate the effectiveness of the proposed method in comparison with the existing methods.

Keywords: Stochastic optimal control, Stochastic systems, Randomized algorithms
Abstract: This paper is concerned with solving chance-constrained finite-horizon optimal control problems, with a particular focus on the recursive feasibility issue of stochastic model predictive control (SMPC) in terms of mission-wide probability of safety (MWPS). MWPS assesses the probability that the entire state trajectory lies within the constraint set, and the objective of the SMPC controller is to ensure that it is no less than a threshold value. This differs from classic SMPC where the probability that the state lies in the constraint set is enforced independently at each time instant. Unlike robust MPC, where strict recursive feasibility is satisfied by assuming that the uncertainty is supported by a compact set, the proposed concept of recursive feasibility for MWPS is based on the notion of remaining MWPSs, which is conserved in the expected value sense. We demonstrate the idea of mission-wide SMPC in the linear SMPC case by deploying a scenario-based algorithm.

Keywords: Stochastic optimal control, Stochastic systems, Uncertain systems
Abstract: We revisit the covariance steering problem for discrete-time Gaussian linear systems with a squared Wasserstein distance terminal cost and analyze the properties of its solution in terms of existence and uniqueness. Specifically, we derive the first and second order conditions for optimality and provide analytic expressions for the gradient and the Hessian of the performance index by utilizing specialized tools from matrix calculus. Subsequently, we prove that the optimization problem always admits a global minimizer, and finally, we provide a sufficient condition for the performance index to be a strictly convex function. In particular, we show that when the terminal state covariance is lower bounded, with respect to the Lowner partial order, by the covariance matrix of the desired terminal normal distribution, then the objective function is strictly convex.

Keywords: Stochastic optimal control, Stochastic systems, Uncertain systems
Abstract: In this paper, we present a new control policy parametrization for the finite-horizon covariance steering problem for discrete-time Gaussian linear systems (DTGLS) via which we can reduce the latter stochastic optimal control problem to a tractable optimization problem. We consider two different formulations of the covariance steering problem, one with hard terminal LMI constraints and another one with soft terminal constraints in the form of a terminal cost which corresponds to the squared Wasserstein distance between the actual terminal state (Gaussian) distribution and the desired one. We propose a solution approach that relies on the affine disturbance feedback parametrization for both problem formulations. We show that this particular parametrization allows us to reduce the hard-constrained covariance steering problem into a semi-definite program (SDP) and the soft-constrained covariance steering problem into a difference of convex functions program (DCP). Finally, we show the advantages of our approach over other covariance steering algorithms in terms of computational complexity and computation time by means of theoretical analysis and numerical simulations.

Keywords: Stochastic optimal control, Uncertain systems, Network analysis and control
Abstract: In light of a reliable and resilient power system under extreme weather conditions and natural disasters, micro-grids integrating local renewable resources in a network have been adopted extensively to supply demands when the main utility experiences blackouts. However, the stochastic nature of renewables and unpredictable contingencies are difficult to be addressed by a conventional, deterministic microgrid energy management framework. The paper proposes a distributionally robust chance-constrained (DR-CC) framework for the power dispatch of networked microgrids. The model considers the power flow, dual-mode operation (i.e. on-/off-grid operation), and voltage droop control simultaneously. By analyzing the empirical solar generation forecast and uncertain forecast error distribution, we construct two kinds of ambiguity sets for the DR-CC model, so that the solution is robust for any unimodal forecast error distribution with specific shape and shared moments. We find a special ambiguity set, unimodal & symmetric set, and prove its tractable second-order conic formulation. It meets the reliability requirements verified by the robustness tests and shows less conservativeness in solution, compared with the unimodal ambiguity set in literature.

Keywords: Stochastic optimal control, Uncertain systems, Switched systems
Abstract: The linear quadratic tracking control problem is studied for a class of discrete-time uncertain Markov jump linear systems with time-varying conditional distributions. The controller is designed under the assumption that it has no access to the true states of the Markov chain, but rather it depends on the Markov chain state estimates. To deal with uncertainty, the transition probabilities of Markov state estimates between the different operating modes of the system are considered to belong in an ambiguity set of some nominal transition probabilities. The estimation problem is solved via the one-step forward Viterbi algorithm, while the stochastic control problem is solved via minimax optimization theory. An optimal control policy with some desired robustness properties is designed, and a maximizing time-varying transition probability distribution is obtained. A numerical example is given to illustrate the applicability and effectiveness of the proposed approach.

Keywords: Stochastic optimal control, Optimization, Stochastic systems
Abstract: Restless multi-armed bandits with partially observable states have applications in communication systems, age of information problems and recommendation systems. In this paper, we study two-action multi-state partially observable restless bandit models. We consider three different models based on state observability---1) no information: exact state is not observable for both the actions, 2) partial information: exact state is observable for one action, %on bandit, there is a fixed restart state, i.e., transition occurs from all other states to that state 3) full information: perfect state information is available to the decision maker. %for both actions on a bandit and there are two restart state for two actions. We derive some structural properties and show that the optimal policy is threshold type. Then we claim indexability for the full and partial information models. We present the Monte Carlo rollout policy as an alternative where the indexability result or an index formula are not available. For partial information model, we present a Whittle index computation scheme based on rollout policy. A concentration bound on the value function is derived in terms of horizon length and number of trajectories of the rollout policy. An explicit index formula is obtained for the full information model. Finally we describe rollout policy for the no-information when it is difficult to show indexability. We present numerical examples to evaluate myopic policy, Monte Carlo rollout policy and Whittle index policy. It is observed that the rollout policy does better than the myopic policy.

Keywords: Game theory, Agents-based systems
Abstract: Are subsidies or taxes more effective at influencing user behavior? To answer this question, we focus on the well-studied framework of atomic congestion games which model resource allocation problems in noncooperative environments. Examples of such resource allocation problems include transportation networks, task assignment, content distribution systems, among others. Monetary incentives, in the form of taxes or subsidies, are commonly employed in such systems to influence self-interested behavior and improve system efficiency. Our first result demonstrates that subsidies can provide strong improvement guarantees when compared to taxes of a similar magnitude. While interesting, our second result demonstrates that this improvements come at the expense of robustness. In particular, taxes provide greater robustness guarantees to mischaracterizations in the societal response when compared to subsidies. Hence, whether a system operator should employ taxes or subsidies depends intimately on the knowledge of the user population.

Keywords: Game theory, Autonomous systems
Abstract: Linear threshold models have long served to intuitively capture binary decision-makings in contexts such as vaccination, trading, and innovation, imposing ``to take an action if enough fellows do so''. Similarly, anti-threshold models have been used in contexts such as following fashion, volunteering, and routing, complying ``to take an action if not too many fellows do so''. Despite the achieved useful insights, these models are often against the common-sense intuition that human decision-making is a mixture of both: ``to take an action if enough but not too many fellows do so''. We capture this missing aspect of human perception and introduce the emph{bi-threshold models}, where each individual rather than one, has a pair of possibly unique thresholds and takes an action if and only if the number of others doing so is between the two thresholds. We find the equilibria of the resulting population dynamics and perform convergence analysis for homogeneous populations. Our analysis highlights the difference between bi-threshold and single-threshold models. In particular, we show that cooperation may be a rare outcome compared to single-threshold models. This highlights the dramatic difference in estimations of the cooperators using threshold models in critical situations such as a pandemic or seasonal vaccinations.

Keywords: Game theory, Cooperative control, Optimization algorithms
Abstract: We study the convergence properties of decentralized fictitious play (DFP) for the class of near-potential games where the incentives of agents are nearly aligned with a potential function. In DFP, agents share information only with their current neighbors in a sequence of time-varying networks, keep estimates of other agents' empirical frequencies, and take actions to maximize their expected utility functions computed with respect to the estimated empirical frequencies. We show that empirical frequencies of actions converge to a set of strategies with potential function values that are larger than the potential function values obtained by approximate Nash equilibria of the potential game. This result establishes that DFP has identical convergence guarantees in near-potential games as the standard fictitious play in which agents observe the past actions of all the other agents.

Keywords: Game theory, Optimal control, Linear systems
Abstract: This paper considers a linear-quadratic-Gaussian asset defending differential game (DADG) where the attacker and the defender do not know each other's state information. However, they both know the trajectory of a moving asset. Both players can choose to observe the other player's state information by paying a cost. The defender and the attacker have to craft both control strategies and observation strategies. We obtain a closed-form feedback solution that characterizes the Nash control strategies. We show that the trajectory of the asset does not affect both players' observation choices. Moreover, we show that we can decouple the observation choices of the defender and the attacker. One can obtain the Nash observation strategies by solving two independent optimization problems. A set of necessary conditions is developed to characterize the optimal observation instances. Based on the necessary conditions, we propose an effective algorithm to compute the optimal observation instances numerically. We also present a case study to demonstrate the effectiveness of the optimal observation instances.

Keywords: Game theory, Network analysis and control
Abstract: Network games study the strategic interaction of agents connected through a network. Interventions in such a game -- actions a coordinator or planner may take that change the utility of the agents and thus shift the equilibrium action profile -- are introduced to improve the planner's objective. We study the problem of intervention in network games where the network has a group structure with local planners, each associated with a group. The agents play a non-cooperative game while the planners may or may not have the same optimization objective. We model this problem using a sequential move game where planners make interventions followed by agents playing the intervened game. We provide equilibrium analysis and algorithms that find the subgame perfect equilibrium. We also propose a two-level efficiency definition to study the efficiency loss of equilibrium actions in this type of game.

Keywords: Optimization algorithms, Optimal control, Machine learning
Abstract: In this paper, we consider the problem of constructing constrained smoothing splines, which are of great importance in data science. The novelty of this work is to formulate the problem as an optimal control problem, and we mathematically analyze the optimal smoothing spline with intermediate constraints using first-order optimality condition from a nonstandard version of the Pontryagin maximum principle. We also propose a novel algorithm to compute the optimal constrained splines based on Newton-Raphson iterative scheme combined with stochastic approximation. A numerical example is shown to illustrate the advantages of the proposed method.

Keywords: Optimization algorithms, Optimization, Machine learning
Abstract: This work proposes a distributed algorithm for solving empirical risk minimization problems, called L-DQN, under the master/worker communication model. L-DQN is a distributed limited-memory quasi-Newton method that supports asynchronous computations among the worker nodes. Our method is efficient both in terms of storage and communication costs, i.e., in every iteration, the master node and workers communicate vectors of size O(d), where d is the dimension of the decision variable, and the amount of memory required on each node is O(md), where m is an adjustable parameter. To our knowledge, this is the first distributed quasi-Newton method with provable global linear convergence guarantees in the asynchronous setting where delays between nodes are present. Numerical experiments are provided to illustrate the theory and the practical performance of our method.

Keywords: Optimization algorithms, Optimization
Abstract: We develop a fast smoothing procedure for solving linear two-stage stochastic programs, which outperforms the well-known L-shaped algorithm on large-scale benchmarks. We derive problem-dependent bounds for the effect of smoothing and characterize the convergence rate of the proposed algorithm. The theory suggests that the smoothing scheme can be sped up by sacrificing accuracy in the final solution. To obtain an efficient and effective method, we suggest a hybrid solution that combines the speed of the smoothing scheme with the accuracy of the L-shaped algorithm. We benchmark a parallel implementation of the smoothing scheme against an efficient parallelized L-shaped algorithm on three large-scale stochastic programs, in a distributed environment with 32 worker cores. The smoothing scheme reduces the solution time by up to an order of magnitude compared to L-shaped.

Keywords: Optimization algorithms, Optimization
Abstract: In this paper, we consider the convex, finite sum minimization problem with explicit convex constraints over strongly connected directed graphs. The constraint is an intersection of several convex sets each being known to only one node. To solve this problem, we propose a novel decentralized projected gradient scheme based on local averaging and prove its convergence using only local functions’ smoothness. Experimental studies demonstrate the effectiveness of the proposed method in both constrained and unconstrained problems.

Keywords: Optimization algorithms, Optimization
Abstract: We consider centralized distributed algorithms for general stochastic convex optimization problems which we approximate by a finite-sum minimization problem with summands distributed among computational nodes. We exploit statistical similarity between the summands and the whole sum to construct a distributed accelerated cubic-regularized Newton's method that achieves lower communication complexity bound for this setting and improves upon existing upper bound. Further, we use this algorithm to obtain convergence rate bounds for the original stochastic optimization problem and compare our bounds with the existing ones in several regimes when the goal is to minimize the number of communication rounds and improve the parallelization when increasing the number of workers.

Keywords: Optimization algorithms, Power systems
Abstract: Distributed and decentralized optimization are key for the control of networked systems. Application examples include distributed model predictive control and distributed sensing or estimation. Non-linear systems, however, lead to problems with non-convex constraints for which classical decentralized optimization algorithms lack convergence guarantees. Moreover, classical decentralized algorithms usually exhibit only linear convergence. This paper presents an essentially decentralized primal-dual interior point method with convergence guarantees for non-convex problems at a superlinear rate. We show that the proposed method works reliably on a numerical example from power systems. Our results indicate that the proposed method outperforms ADMM in terms of computation time and computational complexity of the subproblems.

Keywords: Energy systems, Modeling, Mechatronics
Abstract: Port-Hamiltonian systems with two external ports are studied, together with the strategies and obstructions for conversion of energy from one port to the other. Apart from the cyclo-passivity properties, this turns out to be intimately related to the interconnection topology of the system. A prime source of motivation for energy conversion is thermodynamics, in particular the Carnot-Clausius heat engine theory about conversion of thermal into mechanical energy. This classical theory is extended to general port-Hamiltonian systems satisfying structural conditions on their topology. In particular, the operation of Carnot cycles is generalized. This is illustrated by the examples of a precursor to the Stirling engine and an electro-mechanical actuator. Finally, alternative energy conversion schemes for general port-Hamiltonian systems, such as energy-routers, are discussed from the same vantage point.

Keywords: Optimal control, Nonlinear systems, Electrical machine control
Abstract: We consider the problem of controlling a ballast-stabilized offshore wind turbine. We formulate an optimal control problem with the objective of maximising the power generation while minimising structural fatigue of the wind turbine. Due to the nonlinear nature of the model, obtaining a solution to the above control task poses a severe challenge. Recalling that solutions of the optimal control problem are characterised by a certain (unstable) invariant manifold of the underlying Hamiltonian system, we demonstrate that nonlinear control strategies which approximate the solution of the optimal control problem can be constructed through the introduction of an externally stabilised Hamiltonian system. This observation enables the construction of an algorithm to compute (with relatively low computational complexity) an approximate solution of the optimal control problem, without ignoring nonlinearities in the control design. This approach has several benefits, as demonstrated via simulations on a ballast-stabilized offshore wind turbine.

Keywords: Nonlinear systems, Robotics, Control applications
Abstract: This paper proposes a passivity-based control approach that addresses the trajectory tracking problem for a class of mechanical systems that comprises a broad range of robotic arms. The resulting controllers can be naturally saturated and do not require velocity measurements. Moreover, the proposed methodology does not require the implementation of observers, and the structure of the closed-loop system permits the construction of a Lyapunov function, which eases the convergence analysis. To corroborate the effectiveness of the methodology, we perform experiments with the Philips Experimental Robot Arm.

Keywords: Smart structures, LMIs, Mechatronics
Abstract: Self-powered systems are vibration control technologies that fully power their operation via the harvesting, storage, and reuse of energy from exogenous plant disturbances. This paper explores the feasibility and synthesis of colocated feedback control laws for self-powered systems. These control laws are realized through the use of switch-mode power electronics to simulate shunt admittances. The feasible domain of the resulting ``self-powered synthetic admittance'' (SPSA) controllers is more restrictive than classical feedback passivity, due to parasitic transmission and storage losses in the system. In this paper, we derive sufficient conditions for the feasibility of finite-dimensional, linear time-invariant SPSA controllers and present a sub-optimal synthesis procedure for their design. The proposed methodology is employed to design a controller to reduce the seismic response of a base-isolated civil structure.

Keywords: Robotics, Nonlinear systems, Hierarchical control
Abstract: Eigenmanifolds extend eigenspaces to nonlinear mechanical systems with possibly non-Euclidean metrics. Recent work has shown that simple controllers can excite hyper-efficient oscillations by simultaneously stabilizing an Eigenmanifold and regulate the total energy. Yet, existing techniques require imposing assumptions on the system dynamics that the controlled system may not fulfill. This paper overcomes these limitations by allowing for partial dynamic compensation, which produces a good decoupling of the system's dynamics. This decoupling happens in a convenient set of coordinates induced by the modal characterization of the mechanical system. Two control algorithms taking advantage of this property are proposed and validated in simulation.

Keywords: Estimation, Energy systems, Machine learning
Abstract: This paper presents regression and classification methods to estimate wind direction in a wind farm from operational data. Two neural network models are trained using supervised learning. The data are generated using high-fidelity large eddy simulations (LES) of a virtual wind farm with 16 turbines. These simulations include the high-fidelity flow physics and turbine dynamics. The LES data used for training and testing the neural network models are the rotor angular speeds of each turbine. Our neural network models use sixteen angular speeds as inputs to produce an estimate of the wind direction at each point in time. Training and testing of the neural network models are done for seven discrete wind directions, which span the most interesting cases due to symmetry of the wind farm layout. The results of this paper are indicative of the potential that readily available neural network models have to obtain estimates of wind direction in real time.

Keywords: Switched systems, Fault diagnosis
Abstract: Cyber-physical systems (CPSs) are a class of systems integrating cyber and physical components, and security issues of CPSs have gained a lot of attention in recent years. CPSs are modeled as hybrid systems in this paper since the logical and physical behaviors of CPS can be mapped to the discrete-state and continuous-state dynamics of the hybrid system, respectively. Motivated by the importance of situation awareness in an adversarial environment, we consider the mode distinguishability problem for a class of hybrid systems that can describe compromised CPSs. It is found that even though some modes of the hybrid system may not be distinguishable without knowing the attack inputs, the modes could be controlled distinguishable, which means their behaviors can be differentiated under certain control inputs. In this paper, the characterization of controlled distinguishability is studied, and the problem of finding control input for mode identification is proposed.

Keywords: Switched systems, Identification for control, Machine learning
Abstract: The knowledge of restrictions on the set of admissible switching signals is important for the design of control strategies for switched systems. We propose an algorithm that learns these restrictions by collecting data from a gray-box simulation model of the switched system. Our learning technique is a modified version of the well-known (L^{*})-algorithm from machine learning literature. Examples are presented to demonstrate our learning algorithm.

Keywords: Switched systems, Observers for Linear systems
Abstract: In this article, we study the observability and determinability for discrete-time linear switched systems. Studies for the observability for this system class are already available in literature, however, we use assume here that the switching signal is known. This leads to less conservative observability conditions (e.g. observability of each individual mode is not necessary for the overall switched system to be observable); in particular, the dependencies of observability on the switching times and the mode sequences are derived; these results are currently not available in the literature on discrete-time switched systems. In addition to observability (which is concerned with recovering the state from the initial time onwards), we also investigate the determinability which is concerned with the ability to reconstruct the state value at the end of the observation interval. We provide several simple examples to illustrate novel features not seen in the continuous time case or for unswitched systems.

Keywords: Switched systems, Optimal control, Predictive control for linear systems
Abstract: This paper studies the optimal control problem for discrete-time switched linear systems with quadratic cost. A model predictive control (MPC) scheme is proposed which results in closed-loop strategies for both switching and control inputs. To reduce the online computation and ensure stability of the MPC scheme, a two-stage pruning algorithm is constructed which is performed offline. The resulting MPC scheme ensures exponential stability, and the cost function is suboptimal. Stability, feasibility, and suboptimality of the MPC scheme are studied. Simulation results are given for the MPC scheme which shows the proposed approach results in reduced computation and acceptable performance.

Keywords: Switched systems, Optimal control, Predictive control for nonlinear systems
Abstract: We propose an optimistic planning, branch-and-bound algorithm for nonlinear optimal control problems in which there is a continuous and a discrete action (input). The dynamics and rewards (negative costs) must be Lipschitz but can otherwise be general, as long as certain boundedness conditions are satisfied by the continuous action, reward, and Lipschitz constant of the dynamics. We investigate the structure of the space of hybrid-input sequences, and based on this structure we propose an optimistic selection rule for the subset with the largest upper bound on the value, and a way to select the largest-impact action for further refinement. Together, these fully define the algorithm, which we call OPHIS: optimistic planning for hybrid-input systems. A near-optimality bound is provided together with empirical results in two nonlinear problems where the algorithm is applied in receding horizon.

Keywords: Constrained control, Autonomous vehicles, Switched systems
Abstract: In this paper, we introduce an extension of the Action Governor (AG), which is an add-on, supervisory scheme to a nominal control loop to enforce safety-related requirements. This extension enables AG design based on discrete-time piecewise affine (PWA) system models with additive set-bounded disturbance inputs and non-convex exclusion-zone avoidance constraints. We establish theoretical properties and computational approaches for this robust version of AG, and illustrate its application to an autonomous vehicle control problem.

Keywords: Robust adaptive control, Adaptive control, Uncertain systems
Abstract: In this paper, a one-step nonsmooth adaptive controller design framework is proposed the first time for a class of nonlinear systems with general non-parametric uncertainties. By virtue of a novel non-recursive synthesis philosophy, a nonsmooth adaptive stabilizer can be constructed straightforwardly from the system in a possible simplest form, which facilitates practical implementations. In reference to well-acknowledged recursive design based approaches, direct improvements with this new non-recursive methodology include largely reduced synthesis complexity along with an essential detachment of control law design and stability analysis. A numerical example is provided to demonstrate both the simplicity and effectiveness of the proposed algorithm.

Keywords: Robust adaptive control, Uncertain systems, Cooperative control
Abstract: This letter presents an adaptive coverage control strategy for multi-agent systems with obstacle avoidance in the presence of actuator faults and time-varying uncertainties. The strategy is based on a leader-follower approach. Assuming that the motion of the leader is given, one distributes the followers within the leader's obstacle-free sensing range so that collisions with obstacles can be avoided. An optimized distribution is achieved through the Centroidal Voronoi Tessellation (CVT) and a function approximation technique based immersion and invariance (FATII) coverage controller is constructed to realize the CVT. The stability of the FATII coverage controller is established and its validity is tested by simulations.

Keywords: Robust adaptive control, Variable-structure/sliding-mode control
Abstract: In this paper, a novel data-driven control algorithm is presented coupling Model-Free Adaptive Control and Sliding Mode Control, which addresses general discrete-time Single-Input Single-Output nonaffine nonlinear systems and is aimed at strengthening standard techniques in the presence of a class of output-dependent perturbations. Use is made of an equivalent dynamic linearization model obtained adopting a dynamic linearization technique based on pseudo-partial derivatives. A stability proof of convergence of the closed loop system is provided, showing that the closed-loop tracking error is an asymptotically vanishing sequence and ensuring boundedness of the I/O sequences. Validation of the technique has been performed using a discrete-time test plant taken from the literature in the presence of perturbations. Simulation results show a remarkable improvement in terms of control authority and of tracking accuracy with respect to recently published analogous approaches.

Keywords: Stochastic systems, Quantum information and control, Robust adaptive control
Abstract: Stabilizing control synthesis is one of the central subjects in control theory and engineering, and it always has to deal with unavoidable uncertainties in practice. In this study, we propose an adaptive parameter tuning algorithm for robust stabilizing quantum feedback control of N-level quantum angular momentum systems with a robust stabilizing controller proposed by [Liang, Amini, and Mason, {em SIAM J. Control Optim.}, 59 (2021), pp. 669-692]. The proposed method ensures local convergence to the target state. Besides, numerical experiments indicate its global convergence if the learning parameters are adequately determined.

Keywords: Robust adaptive control, Machine learning, Healthcare and medical systems
Abstract: Although Model Predictive Control (MPC) is one of the main algorithms that has been proposed for insulin control in the context of artificial pancreas (AP), it typically requires complex online optimization, which is infeasible for resource-constrained medical devices. MPC also usually relies on state estimation, an error-prone process. In this paper, we introduce a novel approach to insulin control for the AP that uses Imitation Learning to synthesize neural-network policies from MPC-computed demonstrations. Such policies are computationally efficient and, by instrumenting MPC at training time with full state information, they can directly map measurements into optimal therapy decisions, thus bypassing state estimation. We apply Bayesian inference via Monte Carlo Dropout to learn policies, which allows us to quantify prediction uncertainty and thereby derive safer therapy decisions. We show that our control policies trained under specific patient models readily generalize (in terms of model parameters and disturbance distributions) to patient cohorts, consistently outperforming traditional MPC with state estimation.

Keywords: Machine learning, Robust adaptive control, Optimal control
Abstract: Adaptive control is subject to stability and performance issues when a learned model is used to enhance its performance. This paper thus presents a deep learning-based adaptive control framework for nonlinear systems with multiplicatively-separable parametrization, called adaptive Neural Contraction Metric (aNCM). The aNCM approximates real-time optimization for computing a differential Lyapunov function and a corresponding stabilizing adaptive control law by using a Deep Neural Network (DNN). The use of DNNs permits real-time implementation of the control law and broad applicability to a variety of nonlinear systems with parametric and nonparametric uncertainties. We show using contraction theory that the aNCM ensures exponential boundedness of the distance between the target and controlled trajectories in the presence of parametric uncertainties of the model, learning errors caused by aNCM approximation, and external disturbances. Its superiority to the existing robust and adaptive control methods is demonstrated using a cart-pole balancing model.

Keywords: Behavioural systems, Predictive control for linear systems, Robust control
Abstract: Recent work in data-driven control has revived behavioral theory to perform a variety of complex control tasks, by directly plugging libraries of past input-output trajectories into optimal control problems. Despite recent advances, a key aspect remains unclear: how and to what extent do noise-corrupted data impact control performance? In this work, we provide a quantitative answer to this question based on the model-mismatch level incurred during a preliminary system identification phase. We formulate a Behavioral version of the Input-Output Parametrization (BIOP) for the optimal predictive control of unknown systems using output-feedback dynamic control policies. The main advantages of the proposed framework are that 1) the state-space parameters and the initial state need not be specified for controller synthesis, 2) it can be used in combination with state-of-the-art impulse response estimators, and 3) it allows to recover suboptimality results on learning the Linear Quadratic Gaussian (LQG) controller, therefore revealing how the model-mismatch level may affect the performance. Specifically, it is shown that the performance degrades linearly with the model-mismatch incurred by either classical or behavioral-based system identification.

Keywords: Predictive control for linear systems, Learning
Abstract: In this paper, the lattice piecewise affine (PWA) approximation of explicit linear model predictive control (MPC) is proposed. The training data consists of the state samples and corresponding affine control laws, based on which the lattice PWA approximation is constructed. The proposed approximation is identical to the explicit MPC control law in unique-order (UO) regions containing the sample points as interior points, thus resemble the explicit MPC control law in a large number of regions. Through simplifying the terms and literals in the lattice PWA approximation, both the storage requirement and online computation complexity are largely decreased. The performance of the proposed approximation strategy is tested through a simulation example, and the result shows that with a moderate number of sample points, the lattice PWA approximation is very close to the explicit MPC control law.

Keywords: Predictive control for linear systems, Numerical algorithms
Abstract: This paper analyzes the structure of the constrained optimization problem induced by a typical Model Predictive Control (MPC) problem. The main idea is to exploit the particularities of the feasible domain (namely, that input/state/output constraints describe in fact zonotopic sets) to: i) efficiently describe the solution as a piecewise affine function with polyhedral support; ii) exploit the combinatorial properties of zonotopes to reduce the number of candidate active sets. The results are tested over a challenging numerical example.

Keywords: Predictive control for linear systems, Optimization, Optimization algorithms
Abstract: We propose a semi-explicit approach for linear MPC in which a dual active-set quadratic programming algorithm is initialized through a pre-computed warm start. By using a recently developed complexity certification method for active-set algorithms for quadratic programming, we show how the computational complexity of the dual active-set algorithm can be determined offline for a given warm start. We also show how these complexity certificates can be used as quality measures when constructing warm starts, enabling the online complexity to be reduced further by iteratively refining the warm start. In addition to showing how the computational complexity of any pre-computed warm start can be determined, we also propose a novel technique for generating warm starts with low overhead, both in terms of computations and memory.

Keywords: Predictive control for linear systems, Robust control, Sampled-data control
Abstract: Cyber-physical control systems typically consist of two components: a clocked digital controller and a physical plant evolving in continuous time. Clearly, the state and input constraints must be satisfied not only at, but also between sampling times of the controller. We address this issue by proposing a robust output feedback model predictive control approach for sampled-data systems, which are affected by additive disturbances and measurement noise. To guarantee robust constraint satisfaction for an infinite time horizon, we present a scalable approach to compute safe terminal sets. Based on these sets and using scalable reachability analysis and convex optimization algorithms, we construct real-time controllers that explicitly consider all online computation times. We demonstrate the usefulness of our robust control approach using a vehicle platooning benchmark from the literature.

Keywords: Predictive control for linear systems
Abstract: We consider the problem of output feedback control of constrained, linear systems affected by disturbances. An output feedback strategy is proposed combining an observer with a model predictive controller. The approach utilizes a tube starting from the current state estimate evolving over the prediction horizon. It takes both estimation and prediction errors directly into account to reduce conservatism. We present conditions to guarantee closed loop properties such as robust stability and constraint satisfaction. The approach is illustrated via simulation examples.

Keywords: Distributed parameter systems, Optimal control, Stability of nonlinear systems
Abstract: This paper investigates an interval turnpike result for the adjoints/costates of finite- and infinite-dimensional nonlinear optimal control problems under the assumption of an interval turnpike on states and controls. We consider stabilizable dynamics governed by a generator of a semigroup with finite-dimensional unstable part satisfying a spectral decomposition condition and show the desired turnpike property under continuity assumptions on the first-order optimality conditions. We further provide a numerical example with a semilinear heat equation to illustrate the results.

Keywords: Distributed parameter systems
Abstract: This paper considers a class of hyperbolic-parabolic PDE system with mixed-coupling terms, a rather unexplored family of systems. Compared with the previous literature, the coupled system we explore contains more interior-coupling terms, which makes controller design more challenging. Our goal is to design a boundary controller to stabilise the coupled system exponentially. For that, we propose two alternative controllers whose design is based on the backstepping method; the difference between them is in the target system. The first one keeps one of the coupling terms in the hyperbolic PDE which is compensated by adding an stabilising term in the parabolic PDE; however, the price to pay is that the coefficient of the term might become rather large. The second alternative is more involved but results in a target system which is directly stable, with arbitrary convergence rate. After that, we analyse the stability of the two target systems in the L2 X H1 sense. The (highly coupled) backstepping kernels are derived, and their well-posedness is shown in the appropriate spaces by an infinite induction energy series. Moreover, we show the invertibility of transformations by displaying the inverse transformations; this guarantees closed-loop stability. Finally, numerical simulations are implemented, and the results illustrate that the control laws derived from the two transformations can both stabilise the mixed PDE system exponentially, showing the difference between them; namely, that the first transformation imposes larger terms and therefore larger kernels than the second one to stabilise the controlled system.

Keywords: Distributed parameter systems, Stability of nonlinear systems, Healthcare and medical systems
Abstract: We are interested in a system nonlinearly coupling a transport equation and an ODE. This system, inspired by the classical Kermack-Mckendrick epidemic model with age of infection, describes the evolution of an epidemic in the presence of vaccination and of a flux of susceptible population. Our main result provides effective input to state stability (ISS) estimates, assuming the application of an appropriate vaccination policy. The nonlinear character of the system and the presence of state constraints require a detailed preliminary analysis of the system well-posedness.

Keywords: Delay systems, Distributed parameter systems, Networked control systems
Abstract: In this paper, we design a stabilizing state-feedback control law for a system represented by a general class of integral delay equations. This class of systems encompasses some delay reaction-diffusion equations. Interestingly, such a case study also appears when designing stabilizing control laws for underactuated hyperbolic partial differential equations systems or when stabilizing two interconnected hyperbolic systems for which the actuator is only available at the in-between boundary. Under appropriate spectral controllability assumptions, an implementable control law is proposed. The approach is constructive and makes use of the well-known backstepping methodology. Due to the integral terms present in the original system, the proposed problem requires a Fredholm transform, which is not always invertible. The invertibility of this transformation is proved using an operator formulation. In particular, we show that this invertibility property is a consequence of spectral controllability. The existence of the kernels defining the Fredholm transform is proved similarly by showing that they satisfy an invertible integral equation. Some test case simulations complete the paper.

Keywords: Distributed parameter systems, Optimization algorithms, Identification
Abstract: Optimal sensor location for parameter estimation of spatiotemporal systems is usually focused on maximizing an optimality criterion defined on the Fisher information matrix (FIM) associated with the estimated parameters. But different optimality criteria may yield different optimal locations. Therefore, strong interest is generated by compromise locations which would produce decent values for a broadest possible class of design criteria. Here a method is proposed to compute sensor locations which maximize the minimal efficiency with respect to the class of orthogonally invariant information criteria. This class is broad enough to include all optimum design criteria encountered in practice. It turns out that the minimal efficiency with respect to this class equals that with respect to a finite set of criteria generalizing the well-known E-optimum design criterion. The proposed method heavily exploits this property and the attendant semi-infinite programming problem formulation. It alternates between finding the eigenvalues and eigenvectors of the current FIM and solving a linear-programming problem. Thus the nondifferentiability inherent in the original problem is circumvented and the resulting fast algorithm is guaranteed to converge in a finite number of steps.

Keywords: Distributed parameter systems, Optimal control
Abstract: This paper considers the optimal navigation problem wherein the agent has to traverse a spatiotemporally varying hazardous field that has severely lethal effects on the agent. The accumulated effects of the value of the spatiotemporally varying field along the path, represented by the line integral of the spatial field along the chosen trajectory, constitutes the destructive effects of the hazardous field. When this accumulated amount exceeds a prescribed threshold, the agent is incapacitated and can no longer move. We formulate the navigation problem as an optimal control problem in which the agent has to traverse a 2D enclosed domain and reach its goal destination at a free final time while minimizing the cumulative exposure to the hazardous field. Due to practical considerations, the solution to the optimal control problem is admissible only if the accumulated exposure of the agent when it reaches the final destination is well below the deadly threshold. The evolution of the spatiotemporally varying field is described by a 2D advection-diffusion PDE with a moving source that realistically captures rapidly evolving hazardous fields. Extensive numerical studies are included to highlight the various aspects of the proposed constrained navigation problem.

Keywords: Machine learning, Learning, Decentralized control
Abstract: Broader decision-making goals such as risk-sensitivity, exploration, and incorporating prior experience have been critical to successful implementations of cooperative multi-agent reinforcement learning (MARL) in recent years. This trend motivates the study of MARL problems where the objective is any nonlinear function of the team's long-term state-action occupancy measure, i.e., a emph{general utility}, which subsumes the aforementioned goals.

Existing decentralized actor-critic algorithms to solve this problem require extensive message passing per policy update, which may be impractical. Thus, we put forth {bf C}ommunication-{bf E}fficient {bf D}ecentralized {bf S}hadow Reward {bf A}ctor-{bf C}ritic (CE-DSAC) that may operate with time-varying or event-triggered network connectivities. This scheme operates by having agents to alternate between policy evaluation (critic), weighted averaging with neighbors (information mixing), and local gradient updates for their policy parameters (actor). CE-DSAC is different from the usual critic update in its local occupancy measure estimation step which is needed to estimate the derivative of the local utility with respect to their occupancy measure, i.e., the ``shadow reward," and the amount of local weighted averaging steps executed by agents. This scheme improves existing tradeoffs between communications and convergence: to obtain epsilon-stationarity, we require in mathcal{O}(1/epsilon^{2.5}) (Theorem ref{theorem:final}) or faster mathcal{O}(1/epsilon^{2}) (Corollary ref{corollary:communication}) steps with high probability. Experiments demonstrate the merits of this approach for multiple RL agents solving cooperative navigation tasks with intermittent communications.

Keywords: Networked control systems, Agents-based systems, Optimization
Abstract: In this paper, we study entirely self-sustained multi-agent systems with decentralized resource allocation. Agents make local resource decisions, and sometimes, trading decisions to maximize their individual payoffs accruing from the utility of consumption and the income or expenditure from trading. A competitive equilibrium is achieved if all agents maximize their individual payoffs; a social welfare equilibrium is achieved if the total agent utilities are maximized. First, we consider multi-agent systems with static local allocation, and prove from duality theory that under general convexity assumptions, the competitive equilibrium and the social welfare equilibrium exist and agree. Next, we define a social shaping problem for a competitive equilibrium under which the optimal resource price is socially acceptable, and show that agent utility functions can be prescribed in a family of socially admissible quadratic functions, under which the pricing at the competitive equilibrium is always below a threshold. Finally, we extend the study to dynamical multi-agent systems where agents are associated with dynamical states from linear processes, and prove that the dynamic competitive equilibrium and social welfare equilibrium continue to exist and coincide with each other.

Keywords: Optimization, Decentralized control, Agents-based systems
Abstract: We develop a methodology to automatically compute worst-case performance bounds for a class of decentralized algorithms that optimize the average of local functions distributed across a network. We extend the recently proposed PEP approach to decentralized optimization. This approach allows computing the exact worst-case performance and worst-case instance of centralized algorithms by solving an SDP. We obtain an exact formulation when the network matrix is given, and a relaxation when considering entire classes of network matrices characterized by their spectral range. We apply our methodology to the decentralized (sub)gradient method, obtain a nearly tight worst-case performance bound that significantly improves over the literature, and gain insights into the worst communication networks for a given spectral range.

Keywords: Optimization algorithms, Optimization, Neural networks
Abstract: In this paper, we investigate the decentralized statistical inference problem, where a network of agents cooperatively recover a (structured) vector from private noisy samples without centralized coordination. Existing optimization-based algorithms suffer from issues of model mismatches and poor convergence speed, and thus their performance would be degraded provided that the number of communication rounds is limited. This motivates us to propose a learning-based framework, which unrolls well-noted decentralized optimization algorithms (e.g., Prox-DGD and PG-EXTRA) into graph neural networks (GNNs). By minimizing the recovery error via end-to-end training, this learning-based framework resolves the model mismatch issue. Our convergence analysis (with PG-EXTRA as the base algorithm) reveals that the learned model parameters may accelerate the convergence and reduce the recovery error to a large extent. The simulation results demonstrate that the proposed GNN-based learning methods prominently outperform several state-of-the-art optimization-based algorithms in convergence speed and recovery error.

Keywords: Optimization, Distributed control, Machine learning
Abstract: Stochastic approximation, a data-driven approach for finding the root of an unknown operator, provides a unified framework for solving many problems in stochastic optimization and reinforcement learning. Motivated by a growing interest in multi-agent and multi-task learning, we study a decentralized variant of stochastic approximation over a network of agents, where the goal is to find the root of the aggregate of the local operators at the agents. In this method, each agent implements a local stochastic approximation using noisy samples from its operator while averaging its iterates with the ones received from its neighbors. Our main contribution is to provide a finite-time analysis of the decentralized stochastic approximation method and to characterize the impacts of the underlying communication topology between agents. Our model for the data observed at each agent is that it is sampled from a Markov process; this lack of independence makes the iterates biased and (potentially) unbounded. Under mild assumptions we show that the convergence rate of the proposed method is essentially the same as if the samples were independent, differing only by a log factor that represents the mixing time of the Markov process. Finally, we present applications of the proposed method on a number of interesting learning problems in multi-agent systems, including distributed robust system identification and decentralized Q-learning for solving multi-task reinforcement learning.

Keywords: Optimization algorithms, Networked control systems, Variational methods
Abstract: The distributed dual ascent is an established algorithm to solve strongly convex multi-agent optimization problems with separable cost functions, in the presence of coupling constraints. In this paper, we study its asynchronous counterpart. Specifically, we assume that each agent only relies on the outdated information received from some neighbors. Differently from the existing randomized and dual block-coordinate schemes, we show convergence under heterogeneous delays, communication and update frequencies. As a consequence, our asynchronous dual ascent algorithm can be implemented without requiring any coordination between the agents.

Keywords: Machine learning, Traffic control, Identification
Abstract: We consider the problem of traffic density reconstruction using measurements from probe vehicles (PVs) with a low penetration rate. In other words, the number of sensors is small compared to the number of vehicles on the road. The model used assumes noisy measurements and a partially unknown first-order model. All these considerations make the use of machine learning to reconstruct the state the only applicable solution. We first investigate how the identification and reconstruction processes can be merged and how a sparse dataset can still enable a good identification. Secondly, we propose a pre-training procedure that aids the hyperparameter tuning, preventing the gradient descent algorithm from getting stuck at saddle points. Examples using numerical simulations and the SUMO traffic simulator show that the reconstructions are close to the real density in all cases.

Keywords: Automotive control, Autonomous vehicles, Traffic control
Abstract: The development of connected and automated vehicles (CAVs) enables improvements in the safety, smoothness, and energy efficiency of the road transportation systems. This paper addresses the problem of optimally controlling battery-electric CAVs crossing an unsignalized intersection subject to a first-in-first-out crossing policy. The optimal velocity trajectory of each vehicle that minimizes the average energy consumption and travel time, is found by a decentralized model predictive control (DMPC) method via a convex modeling framework so as to ensure computational efficiency and the optimality of the solution. Numerical examples and comparisons with a centralized control counterpart demonstrate the effectiveness of the proposed decentralized coordination scheme and the trade-off between energy consumption and travel time. Further investigation into the size of the sampling interval is also provided in order to show the validity of the method in practice.

Keywords: Traffic control, Hierarchical control, Decentralized control
Abstract: This paper proposes two control architectures to regulate traffic in freeway networks by exploiting the presence of platoons of autonomous and connected vehicles. The idea is based on the assumption that each platoon can be considered as a moving bottleneck that affects the surrounding traffic conditions and then can be adopted as an actuator of traffic control strategies. In this work, the speed of the platoon is considered as a control variable whose profile is defined through two control architectures. In the first scheme, which is a centralized one, the control action is defined by a network supervisor which has complete knowledge of the overall system and also makes predictions about the systems state. In the second scheme which is decentralized, a link supervisor acting in each link of the network is defined. The two schemes are discussed and their effectiveness is analyzed by means of simulations.

Keywords: Traffic control, Control of networks, Modeling
Abstract: This paper is devoted to boundary control design for urban traffic described on a macroscopic scale. The state corresponds to vehicle density that evolves on a continuum two-dimensional domain that represents a continuous approximation of a urban network. Its parameters are interpolated as a function of distance to physical roads. The dynamics are governed by a new macroscopic multi-directional traffic model that encompasses a system of four coupled partial differential equations (PDE) each describing density evolution in one direction layer: North, East, West and South (NEWS). We analyse the class of desired states that the density governed by NEWS model can achieve. Then a boundary control is designed to drive congested traffic to an equilibrium with the minimal congestion level. The result is validated numerically using the real structure of Grenoble downtown (a city in France).