Keywords: Nonlinear systems, Sampled-data control, Delay systems
Abstract: In this paper, we propose a data-driven control framework for locally stabilizing unstable periodic orbits of discrete-time nonlinear systems. Specifically, we explore the scenarios where the locations of the orbits are not precisely known. In our framework, we use a Pyragas-type delayed feedback controller. This controller uses the difference between the current state and a delayed version of the state as feedback to the system. We show that the system under our controller can be described by another nonlinear system with a particular structure. The periodic orbit stabilization problem for the original system is then characterized as an equilibrium stabilization problem for the new system. For this new system, we investigate local exponential stabilization while paying special attention to situations where neither the location of the equilibrium nor the linearized dynamics around that equilibrium are precisely known. To handle such cases, we develop a data-driven framework that accounts for the scenarios where the difference between the state and the equilibrium is not observable. In our framework, we design the gain of a stabilizing controller by using the data generated through a nonlinear projection of the state.

Keywords: Estimation, Switched systems, Statistical learning
Abstract: We study data-driven control of (Markov) jump linear systems with unknown transition probabilities, where both the discrete mode and the continuous state are to be inferred from output measurements. To this end, we develop a receding horizon estimator which uniquely identifies a sub-sequence of past mode transitions and the corresponding continuous state, allowing for arbitrary switching behavior. Unlike traditional approaches to mode estimation, we do not require an offline exhaustive search over mode sequences to determine the size of the observation window, but rather select it online. If the system is weakly mode observable, the window size will be upper bounded, leading to a finite-memory observer. We integrate the estimation procedure with a simple distributionally robust controller, which hedges against misestimations of the transition probabilities due to finite sample sizes. As additional mode transitions are observed, the used ambiguity sets are updated, resulting in continual improvements of the control performance. The practical applicability of the approach is illustrated on small numerical examples.

Keywords: Predictive control for linear systems, Constrained control, Statistical learning
Abstract: In this paper we introduce a novel approach to distributionally robust optimal control that supports online learning of the ambiguity set, while guaranteeing recursive feasibility. We introduce conic representable risk, which is useful to derive tractable reformulations of distributionally robust optimization problems. Specifically, to illustrate the techniques introduced, we utilize risk measures constructed based on data-driven ambiguity sets, constraining the second moment of the random disturbance. In the optimal control setting, such moment-based risk measures lead to tractable optimal controllers when combined with affine disturbance feedback. Assumptions on the constraints are given that guarantee recursive feasibility. The resulting control scheme acts as a robust controller when little data is available and converges to the certainty equivalent controller when a large sample count implies high confidence in the estimated second moment. This is illustrated in a numerical experiment.

Keywords: Learning, Robust control, Nonlinear systems
Abstract: In this paper, we present a data-driven controller design method for continuous-time nonlinear systems, using no model knowledge but only measured data affected by noise. While most existing approaches focus on systems with polynomial dynamics, our approach allows to design controllers for unknown systems with rational or general non-polynomial dynamics. We first derive a data-driven parametrization of unknown nonlinear systems with rational dynamics. By applying robust control techniques to this parametrization, we obtain sum-of-squares based criteria for designing controllers with closed-loop robust stability and performance guarantees for all systems which are consistent with the measured data and the assumed noise bound. We then apply this approach to control systems whose dynamics are linear in general non-polynomial basis functions by transforming them into polynomial systems. Finally, we apply the developed approaches to numerical examples.

Keywords: Optimal control
Abstract: In the context of the linear programming (LP) approach to data-driven control, one assumes that the dynamical system is unknown but can be observed indirectly through data on its evolution. Both theoretical and empirical evidence suggest that a desired suboptimality gap is often only achieved with massive exploration of the state-space. In case of linear systems, we discuss how a relatively small but sufficiently rich dataset can be exploited to generate new constraints offline and without observing the corresponding transitions. Moreover, we show how to reconstruct the associated unknown stage-costs and, when the system is stochastic, we offer insights on the related problem of estimating the expected value in the Bellman operator without re-initializing the dynamics in the same state-input pairs.

Keywords: Power electronics, Machine learning, Predictive control for linear systems
Abstract: Model predictive control is a very appealing methodology to control multi-level power converters, but presents an important drawback in the fact that the control input is held constant during each sampling period. Multirate model predictive control has been proposed to alleviate this issue, but it is significantly more complex to compute in systems operating at high frequencies. In this work, we propose the data-driven implementation of multirate model predictive control using the machine-learning method known as Kinky Inference, in an effort to make this control technique possible to be implemented in the future in specialized hardware. The method is tested in simulation on a five-level diode-clamped converter operating in inverter mode. Results show a significant reduction in THD due to the multirate technique, and Kinky Inference succeeds in learning the control law.

Keywords: Neural networks, Stability of nonlinear systems, Machine learning
Abstract: Deep learning is currently used in the perception pipeline of autonomous systems, such as when estimating the system state from camera and LiDAR measurements. While this practice is typical, hard guarantees on the worst-case behavior of the closed-loop system are rare. In this paper, however, we leverage recent results on neural network approximation, combined with classical input-to-state stability (ISS) properties, and show how to design deep neural networks for state estimation that guarantee the safety and stability of the resulting closed-loop system.

Keywords: Machine learning, Numerical algorithms, Observers for Linear systems
Abstract: In this paper we address the problem of observer design for nonlinear discrete-time systems. Combining the theory of so-called Kazantzis–Kravaris-Luenberger (KKL) observers and Deep Learning, we aim to identify the mapping which transforms a nonlinear dynamics to a stable linear system modulo an output injection and design an asymptotic discrete-time observer. The proposed approach leverages the power of Machine Learning to provide an algorithm based on an unsupervised learning of the mapping, which allows to properly explore the state space.

The approach is illustrated on two examples of the autonomous case and two of the non-autonomous one. These examples have been taken from the literature and judiciously chosen to compare the proposed approach with existing results.

Keywords: Machine learning, Observers for nonlinear systems, Identification
Abstract: We address the problem of output prediction, ie. designing a model for autonomous nonlinear systems capable of forecasting their future observations. We first define a general framework bringing together the necessary properties for the development of such an output predictor. In particular, we look at this problem from two different viewpoints, control theory and data-driven techniques (machine learning), and try to formulate it in a consistent way, reducing the gap between the two fields. Building on this formulation and problem definition, we propose a predictor structure based on the Kazantzis-Kravaris/Luenberger (KKL) observer and we show that KKL fits well into our general framework. Finally, we propose a constructive solution for this predictor that solely relies on a small set of trajectories measured from the system. Our simulations show that our solution allows to obtain an efficient predictor over a subset of the observation space.

Keywords: Neural networks, Identification for control, Predictive control for nonlinear systems
Abstract: Deep learning architectures can accurately model complex dynamics such as Inverse models of nonlinear systems. Their uses in the control context have been well-developed in the past forming a strong basis. However, the black-box nature of such techniques have limited their usability in real-world applications. This work aims to provide an auxiliary framework that combines intelligent data-based information with the conventional Predictive Control approach. By adding the inverse model signals to the closed-loop optimization search, the performance and accuracy are improved in our test cases. We provide a set of benchmark experiments as well as different optimization algorithms that benefit from Inverse models. Our results show a clear addition of Intelligent mechanisms to the standard methodology, without the uncertainties of black-box models and with all the advantages.

Keywords: Machine learning, Robotics
Abstract: This work proposes a scenario-based Deep Reinforcement Learning (DRL) approach enabling robot manipulators to efficiently execute industrial tasks while avoiding the collision with obstacles. The proposal exploits a DRL-based decision maker trained ad hoc so as to be able to automatically select at any time instant the most appropriate control methodology, in a given set, to execute the required industrial task. The capability of performing the selection automatically is "learnt" by training the system relying on a suitably designed reward function. It takes into account the robot relative distances from the target and the obstacles, the computational cost associated with each methodology, as well as the percentage of task completion obtained by applying the selected methodology. The learning skill is enforced by a properly sized Deep Q-Network (DQN). The proposal is assessed relying on realistic robotic manipulator scenarios reproduced in the CoppeliaSim environment.

Keywords: Traffic control, Cyber-Physical Security, Autonomous vehicles
Abstract: Deep learning methods are in the forefront of techniques used to perform complex controls in autonomous vehicles (AVs). Such methods are vulnerable to nuanced types of adversarial attacks, and can have sever safety implications. Specifically, backdoors are an emerging kind of adversarial attacks on deep neural networks (DNNs), where a secret backdoor is injected into the DNNs by an attacker and activated in the presence of well-designed triggers, which necessitate a systematic exploration to enable the study of effective defenses. In this paper, we learn an adversarial distribution for trigger samples by reinforcement learning with the objective that the difference between the adversarial and genuine distributions are minimized. This bypasses many detection algorithms that are designed based on the difference between the adversarial and genuine input samples. Specifically, the difference between two distributions are evaluated by the Jensen-Shannon (JS)-divergence. The adversarial samples generated by the learned adversarial distribution are used for manipulating benign models in two complex traffic control systems. Our results show that our method renders the backdoor attack stealthy overriding the benign control objectives and potentially causing vehicle collisions.

Keywords: Switched systems, Stability of hybrid systems, Randomized algorithms
Abstract: This paper tackles the feedback stabilization of switched linear systems under arbitrary switching. We propose a data-driven approach which allows to compute a stabilizing static feedback using only a finite set of observations of trajectories without any knowledge of the dynamics. We assume that the switching signal is not observed, and as a consequence, we aim at solving a emph{uniform} stabilization problem in which the feedback is stabilizing for all possible switching sequences. In order to generalize the solution obtained from trajectories to the actual system, probabilistic guarantees are derived via geometric analysis in the spirit of scenario optimization. The performance of this approach is demonstrated on a few numerical examples.

Keywords: Networked control systems, Learning, Distributed control
Abstract: In this paper, a distributed learning leader-follower consensus protocol based on Gaussian process regression for a class of nonlinear multi-agent systems with unknown dynamics is designed. We propose a distributed learning approach to predict the residual dynamics for each agent. The stability of the consensus protocol using the data-driven model of the dynamics is shown via Lyapunov analysis. The followers ultimately synchronize to the leader with guaranteed error bounds by applying the proposed control law with a high probability. The effectiveness and the applicability of the developed protocol are demonstrated by simulation examples.

Keywords: Optimal control, Learning, Optimization algorithms
Abstract: In this paper we propose a novel numerical strategy to solve nonlinear optimal control problems for dynamical systems with partially unknown dynamics. The goal is to explore feasible system trajectories minimizing a given finite-time performance criterion. We suppose to be able to actuate an input sequence on the real system, but only an inaccurate description of the dynamics is available for the control design. The proposed learning-driven optimal control strategy combines a trajectory optimization procedure with a Gaussian process regression to iteratively enrich the model and perform the optimization steps. Thanks to this combined scheme, the strategy is able to explore the trajectory manifold while minimizing the cost function. To corroborate the theoretical results, numerical simulations on the optimal control of a pendulum are shown.

Keywords: Stability of hybrid systems, Power systems, Machine learning
Abstract: This paper proposes a tractable framework to determine key characteristics of non-linear dynamic systems by converting physics-informed neural networks to a mixed integer linear program. Our focus is on power system applications. Traditional methods in power systems require the use of a large number of simulations and other heuristics to determine parameters such as the critical clearing time, i.e., the maximum allowable time within which a disturbance must be cleared before the system moves to instability. The work proposed in this paper uses physics-informed neural networks to capture the power system dynamic behavior and, through an exact transformation, converts them to a tractable optimization problem which can be used to determine critical system indices. By converting neural networks to mixed integer linear programs, our framework also allows to adjust the conservativeness of the neural network output with respect to the existing stability boundaries. We demonstrate the performance of our methods on the non-linear dynamics of converter-based generation in response to voltage disturbances.

Keywords: Randomized algorithms, Statistical learning, Linear systems
Abstract: We provide out-of-sample certificates on the controlled invariance property of a given set with respect to a class of black-box linear systems generated by a possibly inexact quantification of some parameters in the state-space matrices. By exploiting a set of realizations of those undetermined parameters, verifying the controlled invariance property of the given set amounts to a linear program, whose feasibility allows us to establish an a-posteriori probabilistic certificate on the controlled invariance property of such a set with respect to the unknown linear time-invariant dynamics. We apply this framework to the control of a networked system with unknown weighted graph.

Keywords: Identification, Information theory and control, Statistical learning
Abstract: Two dynamical systems are topologically equivalent when their phase-portraits can be morphed into each other by a homeomorphic coordinate transformation on the state space. The induced equivalence classes capture qualitative properties such as stability or the oscillatory nature of the state trajectories, for example. In this paper we develop a method to learn the topological class of an unknown stable system from a single trajectory of finitely many state observations. Using a moderate deviations principle for the least squares estimator of the unknown system matrix theta, we prove that the probability of misclassification decays exponentially with the number of observations at a rate that is proportional to the square of the smallest singular value of theta.

Keywords: Estimation, Distributed parameter systems, Process Control
Abstract: In this paper, we are interested in the estimation of Particle Size Distributions (PSDs) during a batch crystallization process in which particles of two different shapes coexist and evolve simultaneously. The PSDs are estimated thanks to a measurement of an apparent Chord Length Distribution (CLD), a measure that we model for crystals of spheroidal shape. Our main result is to prove the approximate observability of the infinite-dimensional system in any positive time. Under this observability condition, we are able to apply a Back and Forth Nudging (BFN) algorithm to reconstruct the PSD.

Keywords: Estimation, Identification, Optimization
Abstract: To avoid non-convexity of the criterion, various relaxations are typically used in input design. For example, the input may be assumed to be stationary and the design problem may be formulated in terms of the correlation coefficients. In this contribution, we instead propose a method to directly design the input sequence. This allows to maximize the information obtained from short-time (transient) experiments using non-stationary inputs. We do this by fitting the achieved Fisher matrix to a desired target matrix in a matrix sense, using the infinity norm. The target matrix can either be the desired Fisher matrix, obtained from quality considerations of the intended use of the model, or a matrix directly representing the performance of the application. An often used quantity is the Hessian of the so called the application cost. Thus, the method is formulated as a time domain optimization problem that is non-convex. This optimization problem is solved by alternative minimization and proximal mapping, where we split the problem in three steps where in each step, the cost function is minimized in respect to one of the variables and other variables are kept fix. We repeat these three steps until convergence. The procedure of the algorithm is summarized in a pseudo code. Finally, we illustrate our method in two numerical examples.

Keywords: Estimation, Identification, Stochastic systems
Abstract: Network system identification of node to node dynamics requires a study of information flows between nodes. Such flows can be measured by trajectory mutual information. But since the interaction between any pair of nodes is affected by other nodes one needs to adjust for those other nodes. Here, for the first time, we develop such a partial measure of trajectory mutual information between point processes and illustrate its use in simulation and on real data.

Keywords: Estimation, Linear systems, Control over communications
Abstract: This paper tackles the problem of estimating the state of a plant when communications are corrupted by cyber-attacks or sensor faults occur. An observer structure based on multi-hop subspace decomposition is proposed, which allows each agent to identify its observable and unobservable subspaces and to reconstruct them based on its own mea- surements and on the information exchanged with the rest of agents. To deal with cyber-attacks, the paper proposes a method based on weighting the information provided by agents according to an assessment of its trustworthiness. An algorithm to dynamically adjust the weighting parameters is provided and the performance of the proposed technique is assessed using a hardware-in-the-loop platform..

Keywords: Estimation, Observers for nonlinear systems, Kalman filtering
Abstract: This paper introduces an algorithm for multiple target tracking where the tracks are evolving on the Special Euclidean group SE(2). Applications include tracking ground targets using cameras on-board a hovering multirotor UAV. Our approach extends the Recursive RANSAC (R-RANSAC) algorithm to nonlinear motion models. In particular, we show how tracks can be initialized in the presence of significant measurement clutter in a way that leads to computationally efficient implementation. Simulation and flight results show significant tracking improvement over using linear constant velocity models.

Keywords: Manufacturing systems and automation, Sensor fusion, Estimation
Abstract: Soft sensors enable the calculation of difficult to measure, delayed or critical variables in different processes. In this work, a data-driven approach for soft sensors based on factor graphs and probabilistic inference is presented. Factor graphs describe the factorization of the system dynamics, which follows from expert knowledge. The probability density functions implied by the factor graph are parameterized with a Gaussian mixture model (GMM) to consider nonlinearities and multimodal process behavior. The approach is used for calculating and forecasting unknown quantities using probabilistic inference. Furthermore, the factorization allows a distributed implementation, the detection of sensor faults as well as their compensation to achieve robustness. An analytical formulation of the uncertainty propagation is also presented for the GMM. The approach is experimentally validated by determining temperature profiles in an industrial oven yielding to an average deviation of less than 2K.

Keywords: Markov processes, Machine learning, Neural networks
Abstract: In this paper, we establish the convergence of fitted value iteration and fitted Q-value iteration for continuous-state continuous-action Markov decision problems (MDPs) with state-dependent action sets. We further extend the algorithm and the convergence result to the case of monotone MDPs.

Keywords: Markov processes, Stochastic optimal control, Computational methods
Abstract: This paper proposes an accelerated method for approximately solving partially observable Markov decision process (POMDP) problems offline. Our method carefully combines two existing tools: Anderson acceleration (AA) and the fast informed bound (FIB) method. Adopting AA, our method rapidly solves an approximate Bellman equation with an efficient combination of previous solution estimates. Furthermore, the use of FIB alleviates the scalability issue inherent in POMDPs. We show the convergence of the overall algorithm to the suboptimal solution obtained by FIB. We further consider a simulation-based method and prove that the approximation error is bounded explicitly. The performance of our algorithm is evaluated on several benchmark problems. The results of our experiments demonstrate that the proposed algorithm converges significantly faster without degrading the quality of the solution compared to its standard counterpart.

Keywords: Markov processes, Uncertain systems, Stochastic optimal control
Abstract: Autonomous vehicles face the problem of optimizing the expected performance of subsequent maneuvers while bounding the risk of collision with surrounding dynamic obstacles. These obstacles, such as agent vehicles, often exhibit stochastic transitions that should be accounted for in a timely and safe manner. The Constrained Stochastic Shortest Path problem (C-SSP) is a formalism for planning in stochastic environments under certain types of operating constraints. While C-SSP allows specifying constraints in the planning problem, it does not allow for bounding the probability of constraint violation, which is desired in safety-critical applications. This work's first contribution is an exact integer linear programming formulation for Chance-constrained SSP (CC-SSP) that attains deterministic policies. Second, a randomized rounding procedure is presented for stochastic policies. Third, we show that the CC-SSP formalism can be generalized to account for constraints that span through multiple time steps. Evaluation results show the usefulness of our approach in benchmark problems compared to existing approaches.

Keywords: Stochastic systems, Markov processes, Machine learning
Abstract: This paper introduces Voronoi Progressive Widening (VPW), a generalization of Voronoi optimistic optimization (VOO) and action progressive widening to partially observable Markov decision processes (POMDPs). Tree search algorithms can use VPW to effectively handle continuous or hybrid action spaces by efficiently balancing local and global action searching. This paper proposes two VPW-based algorithms and analyzes them from theoretical and simulation perspectives. Voronoi Optimistic Weighted Sparse Sampling (VOWSS) is a theoretical tool that justifies VPW-based online solvers, and it is the first algorithm with global convergence guarantees for continuous state, action, and observation POMDPs. Voronoi Optimistic Monte Carlo Planning with Observation Weighting (VOMCPOW) is a versatile and efficient algorithm that consistently outperforms state-of-the-art POMDP algorithms in several simulation experiments.

Keywords: Machine learning, Markov processes
Abstract: We consider the problem of policy search in sequential decision making problems with imperfect information as encapsulated by a partially observed Markov Decision Process (POMDP) over possibly continuous state-spaces. In general, the optimal policy is history-dependent and the objective is non-convex in the policy parameters, making even stationary point policies challenging to ascertain. To address this problem class, we develop a constructive way to succinctly represent the history as an approximate emph{information state}, using Semiparametric Information State Embedding (SISE). SISE alternates between conditional kernel density estimation and fitting the parameters of an Echo State Network (ESN), a one-layer recurrent neural model. Based upon constructing SISE, we develop an actor-critic scheme for policy search over the approximate information states. Our main technical contributions are to (i) establish the convergence and generalization performance of SISE, and (ii) derive the convergence to stationary points of our policy search scheme. Experimentally, our fusion of SISE and actor-critic yields favorable performance in practice on the canonical POMDPs of Tiger, LightDark, and a partially observed variant of CartPole.

Keywords: Control over communications, Optimal control, Markov processes
Abstract: In this paper, we study Markov Decision Processes (MDPs) with self-triggered strategies, where the idea of self-triggered control is extended to more generic MDP models. This extension broadens the application of self-triggering policies to a broader range of systems. We study the co-design problems of the control policy and the triggering policy to optimize two pre-specified cost criteria. The first cost criterion is introduced by incorporating a pre-specified update penalty into the traditional MDP cost criteria to reduce the use of communication resources. A novel dynamic programming (DP) equation called DP equation with optimized lookahead is proposed to solve for the optimal self-triggering policy under this criteria. The second self-triggering policy is to maximize the triggering time while still guaranteeing a pre-specified level of sub-optimality. Theoretical underpinnings are established for the computation and implementation of both policies. Through a gridworld numerical example, we illustrate the two policies' effectiveness in reducing resources consumption and demonstrate the trade-offs between resource consumption and system performance.

Keywords: Game theory, Transportation networks, Behavioural systems
Abstract: Incentives are an effective tool to alter user preferences to promote more efficient group behavior. It is often assumed that these incentives can be levied on any and every user in the system; in many settings, this is not the case. Accordingly, how should a system operator design incentives that only affect a fraction of the users? The network routing literature contains many results showing the effectiveness of monetary taxes to influence self-interested users’ behavior and improve system efficiency. These results typically assume that all users in the network are influenced by incentives; however, this need not be the case if incentives are opted into or if some users do not experience or are unaffected by monetary fees. In this work, we address the problem of designing incentives for populations of users where a fraction of the population is not influenced by incentives in their decision-making process. By focusing on the setting of parallel-network selfish routing problems, surprisingly we find that the tolls that are optimal when the full population is incentivizable remain optimal when only a fraction of the population is incentivizable, though at reduced effectiveness. To measure the impact that the unincentivizable users have on the efficacy of the optimal tolling scheme, we derive worst-case performance bounds in a simple class of networks when only a fraction of the users can be incentivized.

Keywords: Game theory
Abstract: In recent years, a significant research effort has been devoted to the design of distributed protocols for the control of multi-agent systems, as the scale and limited communication bandwidth characteristic of such systems render centralized control impossible. Given the strict operating conditions, it is unlikely that every agent in a multi-agent system will have local information that is consistent with the true system state. Yet, the majority of works in the literature assume that agents share perfect knowledge of their environment. This paper focuses on understanding the impact that inconsistencies in agents' local information can have on the performance of multi-agent systems. More specifically, we consider the design of multi-agent operations under a game theoretic lens where individual agents are assigned utilities that guide their local decision making. We provide a tractable procedure for designing utilities that optimize the efficiency of the resulting collective behavior (i.e., price of anarchy) for classes of set covering games where the extent of the information inconsistencies is known. In the setting where the extent of the informational inconsistencies is not known, we show -- perhaps surprisingly -- that underestimating the level of uncertainty leads to better price of anarchy than overestimating it.

Keywords: Game theory
Abstract: This paper considers two-player zero-sum stochastic Bayesian games with infinite horizon time frame where each player's dynamic state is unknown to the others. Based on our previous work in finite horizon games [1], we provide a sufficient statistic based suboptimal strategy in the infinite horizon games using the receding horizon idea. In this suboptimal strategy, both the strategy and the sufficient statistic can be updated using linear programming (LP) formulations. The performance analysis of the suboptimal strategy provides an upper bound on the performance difference between the suboptimal strategy and the optimal one. It is shown that the upper bound decreases exponentially with respect to the window size. The performance of the suboptimal strategies is demonstrated in an underwater jamming problem.

Keywords: Game theory
Abstract: This study provides a model of cyber deception with asymmetric recognition represented by private beliefs. Signaling games, which are often used in existing works, are built on the implicit premise that the player's belief is public information. However, this assumption, which leads to symmetric recognition, is unrealistic in adversarial decision making. For precise evaluation of risks arising from cognitive gaps, this paper proposes epistemic signaling games based on the Mertens-Zamir model, which explicitly quantifies players' asymmetric recognition. Equilibria of the games are analytically characterized with an interpretation.

Keywords: Game theory
Abstract: We consider a robot-location assignment problem. The problem is confounded by a location network that restricts robots' motion to neighboring locations. The problem can be optimally solved using a centralized Hungarian algorithm. We propose a distributed game-theoretic algorithm, based on the Metropolis-Hastings mechanism, to eliminate the need for a central coordinator. Agents are activated at random. Agents compare their action, location, against a neighboring one based on the location network. If the new location represents an improvement in agent's utility, they move with probability one, otherwise, they move with a probability proportional to the difference in the two locations’ utilities. Our algorithm converges to the optimal assignment of robots to locations. We provide extensive simulations, compare with previous work, and demonstrate the versatility of the proposed algorithm to various task allocation scenarios.

Keywords: Resilient Control Systems, Game theory, Computer/Network Security
Abstract: This study investigates general model-based incident handler's asymptotic behaviors against cyber attacks to control systems. The attacker's and the defender's dynamic decision making is modeled as an equilibrium of a dynamic signaling game. It is shown that the defender's belief on existence of an attacker converges over time for any attacker's strategy provided that the stochastic dynamics of the control system is known to the defender. This fact implies that the rational behavior of the attacker converges to a harmless action as long as the defender possesses an effective counteraction. The obtained result supports the powerful protection capability achieved by model-based defense mechanisms.

Keywords: Optimal control, Optimization, Control education
Abstract: The Elvis problem was introduced into the undergraduate mathematical literature by Timothy Pennings whose dog (named Elvis) enjoyed fetching an object thrown from the shore of Lake Michigan into the water. Elvis was observed to retrieve the object by going in a path that resembled how light would refract in isotropic mediums according to Snell's Law. We retain the problem's "Elvis" nomenclature but greatly generalize the problem by considering anisotropic mediums and use the tools of Convex Analysis to provide a complete description of optimal movement. The velocity sets are closed, bounded convex sets containing the origin in its interior, whereas the original problem used only centered balls. Further generalizations are considered with faster movement allowed on the interface and with more than two mediums.

Keywords: Optimal control, Variational methods, Mean field games
Abstract: In this paper, we obtain several structural results on the value function associated to a mean-field optimal control problem of Bolza type in the space of measures. After establishing the sensitivity relations bridging between the costates of the maximum principle and metric superdifferentials of the value function, we investigate semiconcavity properties of this latter with respect to both variables. We then characterise optimal trajectories using set-valued feedback mappings defined in terms of suitable directional derivatives of the value function.

Keywords: Optimal control, Constrained control, Nonlinear systems
Abstract: First order necessary conditions for optimal control problems in which the dynamic constraint is modeled as a differential inclusion have been know for many years. A definitive version is provided by Clarke's 2005 Memoirs. They key ingredients are the generalized Euler Lagrange inclusion (replacing the costate equation of classical optimal control), the transversality condition and the Weierstrass (or Hamiltonian maximization property) condition. Clarke and de Pinho's 2010 paper provided an important application of this theory, using it as a starting point to derive necessary conditions satisfied by minimizers for problems, having a `controlled differential equation' formulation and involving mixed state/control constraints. In his 2019 paper, Ioffe added a refinement to the Weierstrass condition, identifying a larger set of controls over which the Hamiltonian must be maximized. In this paper, we explore, through new theory and examples, the significance of this refinement. We derive new necessary conditions for mixed constraint problems involving controlled differential equations, via a reduction to a differential inclusion problem, that, for the first time, incorporate Ioffe's refinement. Two examples, concerning differential inclusion problems and controlled differential equations problems with mixed constraints, are presented that show how the extra tests present in the refined Weierstrass conditions can be used to identify extremals, i.e. processes satisfying earlier necessary conditions, that are not minimizers.

Keywords: Optimal control, Variational methods, Pattern recognition and classification
Abstract: We consider an optimal control problem for a system of local continuity equations on a space of probability measures. Such systems can be viewed as macroscopic models of (infinite) ensembles of non-interacting particles or homotypic individuals, representing several different “populations”. For the stated problem, we propose necessary conditions for optimality which involve feedback controls inherent to the extremal structure designed via the standard Pontryagin’s Maximum Principle. These optimality conditions admit a realization as an iterative algorithm for optimal control. As a motivating case, we discuss an application of the derived optimality conditions, and the consequent numeric method to a problem of supervised machine learning via dynamic systems.

Keywords: Optimal control, Linear systems, Optimization
Abstract: Common reinforcement learning methods seek optimal controllers for unknown dynamical systems by searching in the ``policy'' space directly. A recent line of research, starting with [1], aims to provide theoretical guarantees for such direct policy-update methods by exploring their performance in classical control settings, such as the infinite horizon linear quadratic regulator (LQR) problem. A key property these analyses rely on is that the LQR cost function satisfies the ``gradient dominance'' property with respect to the policy parameters. Gradient dominance helps guarantee that the optimal controller can be found by running gradient-based algorithms on the LQR cost. The gradient dominance property has so far been verified on a case-by-case basis for several control problems including continuous/discrete time LQR, LQR with decentralized controller, cH_2/cH_infty robust control.

In this paper, we make a connection between this line of work and classical convex parameterizations based on linear matrix inequalities (LMIs). Using this, we propose a unified framework for showing that gradient dominance indeed holds for a broad class of control problems, such as continuous- and discrete-time LQR, minimizing the L_2 gain, and problems using system-level parameterization. Our unified framework provides insights into the landscape of the cost function as a function of the policy, and enables extending convergence results for policy gradient descent to a much larger class of problems.

Keywords: Optimal control, Control applications
Abstract: This article concerns an optimal crowd motion control problem in which the crowd features a structure given by its organization into N groups (participants) each one spatially confined in a set. The overall optimal control problem consists in driving the ensemble of sets as close as possible to a given point (the 'exit') while the population in each set minimizes its control effort subject to its sweeping dynamics with a controlled state-dependent velocity drift. In order to capture the conflict between the goal of the overall population and those of the various groups, the problem is cast as a bilevel optimization framework. A key challenge of this problem consists in bringing together two quite different paradigms: bilevel programming and sweeping dynamics with a controlled drift. Necessary conditions of optimality in the form of a Maximum Principle of Pontryagin in the Gamkrelidze framework are derived. These conditions are then used to solve a simple illustrative example with two participants, emphasizing the interaction between them.

Keywords: Nonlinear output feedback, Output regulation, Uncertain systems
Abstract: We develop an output-feedback control design that achieves practical prescribed-time stabilization, i.e., convergence to within a pre-specified neighborhood around the origin within a pre-specified ("prescribed") time interval. The control design is applicable to a class of uncertain nonlinear systems in a strict-feedback-like form with parametric and functional uncertainties throughout the system as well as exogenous disturbance inputs. The designed output-feedback control design achieves prescribed-time convergence of both the observer errors and the system state variables as well as the control input. Additionally, by considering the limiting case as the size of the desired convergence neighborhood around the origin is reduced to zero, it is shown that the prior results on prescribed-time stabilization (to zero in prescribed time) can be obtained without having to explicitly introduce the temporal scale transformation used in the prior works. It is also shown that the control design approach can be applied to tracking problems to achieve practical convergence of tracking errors.

Keywords: Nonlinear systems, Algebraic/geometric methods
Abstract: The paper develops an algebraic formalism that is based on the language of ideals and modules, associated with the analytic control system given by a set of difference equations. Using this language we show how the orbits of the system can be determined by the generators of some ideal of the ring of germs of analytic functions. The ideal is invariant with respect to shift operator that is defined by the system. Since the orbits are related to the accessibility property, the conditions for accessibility of the system will be given using the germs of analytic functions and the germs of one-forms, associated with the modules in the ring of germs of analytic functions.

Keywords: Nonlinear systems, Computational methods, Systems biology
Abstract: We study the problem of controlling oscillations in closed loop by combining positive and negative feedback in a mixed configuration. We develop a complete design procedure to set the relative strength of the two feedback loops to achieve steady oscillations. The proposed design takes advantage of dominance theory and adopts classical harmonic balance and fast/slow analysis to regulate the frequency of oscillations. The design is illustrated on a simple two-mass system, a setting that reveals the potential of the approach for locomotion, mimicking approaches based on central pattern generators.

Keywords: Nonlinear systems, Learning, Stability of nonlinear systems
Abstract: In this paper we prove new connections between two frameworks for analysis and control of nonlinear systems: the Koopman operator framework and contraction analysis. Each method, in different ways, provides exact and global analyses of nonlinear systems by way of linear systems theory. The main results of this paper show equivalence between contraction and Koopman approaches for a wide class of stability analysis and control design problems. In particular: stability or stablizability in the Koopman framework implies the existence of a contraction metric (resp. control contraction metric) for the nonlinear system. Further, in certain cases, the converse holds: contraction implies the existence of a set of observables with which stability can verified via the Koopman framework. Moreover, the converse claims are based on a novel relation between the Koopman method and construction of a Kazantzis-Kravaris-Luenberger observer.

Keywords: Nonlinear systems, Optimal control, Stability of nonlinear systems
Abstract: We consider optimal control synthesis problems for control-affine systems under safety constraints. We are interested in the task of optimally driving a system from an initial set to a desired state and avoiding some predefined unsafe sets at the same time. Building on a weaker notion of stability of dynamic systems relying on the Lyapunov density, we formulate the optimal control synthesis problem into a convex optimization problem in the space of densities. In this formulation, the safety constraints are naturally mapped to convex constraints on the densities. To improve scalability, the densities are parameterized by polynomials or rational functions. The Sum-of-Squares technique is then employed to efficiently solve the optimal control problems and verify the safety constraints. Numerical examples are presented to demonstrate the efficacy of the proposed framework.

Keywords: Feedback linearization, Sampled-data control
Abstract: For discrete-time systems, flatness is usually defined by replacing the time-derivatives of the well-known continuous-time definition by forward-shifts. With this definition, the class of flat systems corresponds exactly to the class of systems which can be linearized by a discrete-time endogenous dynamic feedback as it is proposed in the literature. Recently, checkable necessary and sufficient differential-geometric conditions for this property have been derived. In the present contribution, we make an attempt to take into account also backward-shifts. This extended approach is motivated by the one-to-one correspondence of solutions of flat systems to solutions of a trivial system as it is known from the continuous-time case. If we transfer this idea to the discrete-time case, this leads to an approach which also allows backward-shifts. To distinguish the classical definition with forward-shifts and the approach of the present paper, we refer to the former as forward-flatness. We show that flat systems (in the extended sense with backward-shifts) still share many beneficial properties of forward-flat systems. In particular, they still are reachable/controllable, allow a straightforward planning of trajectories and can be linearized by a dynamic feedback.

Keywords: Hybrid systems, Biomedical, Differential-algebraic systems
Abstract: The aim of the paper is to provide a computational tool for planning a haemodialysis process. It is shown that optimization methods can be used to obtain the most effective treatment focused on removing both urea and phosphorus during the process. Our approach to the planning process is based on a model which takes into account a rebound phenomenon and that results in a hybrid model in which sliding motion is likely to occur. For such a model we construct an optimization problem and a computational method for solving it. The presented approach to optimal control problems with hybrid systems is different from the others in several aspects. First of all, it is assumed that a hybrid system can exhibit sliding modes. Secondly, the system's motion on the switching surface is described by index 2 differential--algebraic equations and that guarantees accurate tracking of the sliding motion surface. Thirdly, the gradients of the problem's functionals are evaluated with the help of adjoint equations. The adjoint equations presented in the paper take into account sliding motion and exhibit jump conditions at transition times. The stated adjoint equations are used in a globally convergent algorithm which generates a sequence of controls whose accumulation points satisfy the weak maximum principle for optimal control problems with hybrid systems. The paper presents numerical results of solving haemodialysis planning problem.

Keywords: Hybrid systems, Decentralized control, Autonomous systems
Abstract: In this paper we focus on the problem of decomposing a global Signal Temporal Logic formula (STL) assigned to a multi-agent system to local STL tasks when the team of agents is a-priori decomposed to disjoint sub-teams. The predicate functions associated to the local tasks are parameterized as hypercubes depending on the states of the agents in a given sub-team. The parameters of the functions are, then, found as part of the solution of a convex program that aims implicitly at maximizing the volume of the zero superlevel set of the corresponding predicate function. Two alternative definitions of the local STL tasks are proposed and the satisfaction of the global STL formula is proven when the conjunction of the local STL tasks is satisfied.

Keywords: Hybrid systems, Decentralized control, Network analysis and control
Abstract: We consider the problem of simultaneous scheduling and resource allocation of an incoming flow of requests to a set of computing units. By representing each computing unit as a node, we model the overall system as a multi-queue scheme. Inspired by congestion control approaches in communication networks, we propose an AIMD-like (additive increase multiplicative decrease) admission control policy that is stable irrespective of the total number of nodes and AIMD parameters. The admission policy allows us to establish an event-driven discrete model, triggered by a locally identifiable enabling condition. Subsequently, we propose a decentralized resource allocation strategy via a simple nonlinear state feedback controller, guaranteeing global convergence to a bounded set in finite time. Last, we reveal the connection of these properties with Quality of Service specifications, by calculating local queuing time via a simple formula consistent with Little's Law.

Keywords: Hybrid systems, Estimation, Linear systems
Abstract: We consider the problem of estimating a vector of unknown constant parameters for a hybrid system whose flow and jump dynamics are affine in the unknown parameter. Using a hybrid systems framework, a hybrid algorithm is proposed and sufficient conditions are established to guarantee exponential stability of the parameter estimate. Examples are provided showing the merits of the proposed approach.

Keywords: Hybrid systems, Estimation, Linear systems
Abstract: We consider the problem of estimating a constant or piecewise constant vector of unknown parameters for a linear dynamical system. Using a hybrid systems framework, a hybrid algorithm that achieves finite-time convergence of the parameter estimate to the true value is proposed. Sufficient conditions that guarantee convergence of the parameter estimate are provided. Robustness of the proposed algorithm with respect to measurements noise is analyzed, and examples are provided showing the merits of the proposed approach.

Keywords: Hybrid systems, Robotics
Abstract: This paper investigates transitions between periodic behaviors for the rimless wheel on flat terrain. Control is achieved by an impulsive kick by the spoke touching the ground. We describe several periodic orbits, describable by either rolling or involving a flight phase where the wheel performs a jump. We consider a simple method to transfer between orbit types.

Keywords: Constrained control, Linear systems, Sampled-data control
Abstract: This paper studies the informativity problem for reachability and null-controllability of constrained systems. To be precise, we will focus on an unknown linear systems with convex conic constraints from which we measure data consisting of exact state trajectories of finite length. We are interested in performing system analysis of such an unknown system on the basis of the measured data. However, from such measurements it is only possible to obtain a unique system explaining the data in very restrictive cases. This means that we can not approach this problem using system identification combined with model based analysis. As such, we will formulate conditions on the data under which any such system consistent with the measurements is guaranteed to be reachable or null-controllable. These conditions are stated in terms of spectral conditions and subspace inclusions, and therefore they are easy to verify.

Keywords: Constrained control, Nonlinear output feedback, Optimal control
Abstract: In many safety-critical control systems, possibly opposing safety restrictions and control performance objectives arise. To confront such a conflict, this letter proposes a novel methodology that embeds safety into stability of control systems. The development enforces safety by means of barrier functions used in optimization through the construction of barrier states (BaS) which are embedded in the control system's model. As a result, as long as the equilibrium point of interest of the closed loop system is asymptotically stable, the generated trajectories are guaranteed to be safe. Consequently, a conflict between control objectives and safety constraints is substantially avoided. To show the efficacy of the proposed technique, we employ barrier states with the simple pole placement method to design safe linear controls. Nonlinear optimal control is subsequently employed to fulfill safety, stability and performance objectives by solving the associated Hamilton-Jacobi-Bellman (HJB) which minimizes a cost functional that can involve the BaS. Following this further, we exploit optimal control with barrier states on an unstable, constrained second dimensional pendulum on a cart model that is desired to avoid low velocities regions where the system may exhibit some controllability loss and on two mobile robots to safely arrive to opposite targets with an obstacle on the way.

Keywords: Constrained control, Output regulation, PID control
Abstract: We consider the problem of zeroing an error output of a nonlinear discrete-time system in the presence of constant exogenous disturbances, subject to hard convex constraints on the input signal. The design specification is formulated as a variational inequality, and we adapt a forward-backward splitting algorithm to act as an integral controller which ensures that the input constraints are met at each time step. We establish a low-gain stability result for the closed-loop system when the plant is exponentially stable, generalizing previously known results for integral control of discrete-time systems. Specifically, it is shown that if the composition of the plant equilibrium input-output map and the integral feedback gain is strongly monotone, then the closed-loop system is exponentially stable for all sufficiently small integral gains. The method is illustrated via application to a four-tank process.

Keywords: Constrained control, Robust control, Lyapunov methods
Abstract: The problem of finite-time stabilization of linear systems under state and control input constraints in the presence of external disturbances is addressed. The proposed control is designed such that the closed-loop system is 1) superstable, when system trajectories risk to violate the state constraints, and 2) homogeneous with a negative degree, otherwise. While the former property ensures the fulfillment of the state and control constraints, the latter implies finite-time stability. The robustness of the control scheme with respect to disturbances is studied. Theoretical results are supported by numerical simulations.

Keywords: Constrained control, Stability of nonlinear systems
Abstract: Starting from a symmetric state-feedback solution ensuring α-exponential convergence in an ellipsoidal sublevel set, with asymmetric saturation and single-input linear plants, we propose a novel asymmetric scheduled extension preserving the original symmetric solution in that sublevel set and extending the guaranteed stability region to the union of all possible contractive ellipsoids centered at a shifted equilibrium. Our design being based on the solution of a parametric optimization problem, we prove Lipschitz properties of the ensuing feedback law and we compute its explicit state-feedback expression.

Keywords: Constrained control, Stochastic systems, Uncertain systems
Abstract: Safety is essential for reinforcement learning (RL) applied in real-world situations. Chance constraints are suitable to represent the safety requirements in stochastic systems. Previous chance constrained RL methods usually learn an either conservative or unsafe policy, and some of them also suffer from a low convergence rate. In this paper, we propose a model-based chance constrained actor-critic (CCAC) algorithm which can efficiently learn a safe and non-conservative policy. Different from existing methods that optimize a conservative lower bound, CCAC directly solves the original chance constrained problems, where the objective function and safe probability are simultaneously optimized with adaptive weights. In order to improve the convergence rate, CCAC utilizes the gradient of dynamic model to accelerate policy optimization. The effectiveness of CCAC is demonstrated by a stochastic car-following task. Experiments indicate that CCAC achieves good performance while guaranteeing safety, with a five times faster convergence rate compared with model-free RL methods. It also has 100 times higher online computation efficiency than traditional safety techniques such as stochastic model predictive control.

Keywords: Observers for nonlinear systems, Control over communications
Abstract: A Luenberger-like observer for nonlinear systems combined with the event-triggering mechanism is proposed. A proof of stability of the proposed scheme is presented. Moreover, it is proved that the Zeno behavior does not happen. The results are illustrated by an example.

Keywords: Observers for nonlinear systems, Estimation, Sensor fusion
Abstract: This paper investigates the problem of continuous attitude estimation on SO(3) using continuous angular velocity and linear acceleration measurements as well as intermittent linear velocity and inertial vector measurements. First, we propose a nonlinear observer for the case where all the measurements are continuous and almost global asymptotic stability (AGAS) is shown using the notion of almost global input-to-state stability (ISS) on manifolds. Thereafter, a hybrid attitude observer, with AGAS guarantees, is proposed in terms of intermittent linear velocity and vector measurements. Numerical simulation results are presented to illustrate the performance of the proposed hybrid observer.

Keywords: Observers for nonlinear systems, Mechatronics, Lyapunov methods
Abstract: The problem of estimating the angular speed of a solid body from attitude measurements is addressed. To solve this problem, we propose an observer whose dynamics are not constrained to evolve on any specific manifold. This drastically simplifies the analysis of the proposed observer. Using Lyapunov analysis, sufficient conditions for global asymptotic stability of a set wherein the estimation error is equal to zero are established. In addition, the proposed methodology is adapted to deal with angular speed estimation for systems evolving on the unit circle. The approach is illustrated through several numerical simulations.

Keywords: Estimation, Observers for nonlinear systems, LMIs
Abstract: A systematic procedure to synthesize interval observers for nonlinear discrete-time systems is proposed. The feedback gains and other matrices are found by solving semidefinite feasibility programs. Two cases are considered: (1) the interval observer is in the same coordinate frame as the given system, and (2) the interval observer uses a coordinate transformation. Numerical examples are provided to showcase the effectiveness of the interval observers and demonstrate their application to sampled-data systems.

Keywords: Observers for nonlinear systems, Nonlinear output feedback, Power electronics
Abstract: This paper addresses the problem of the output feedback control of the DC-DC converters with an unknown output load. It is assumed that only the capacitors’ voltage and the input voltage are measured. A general lossless model of the converters is considered which relaxes the knowledge about parasitic elements, such as the input side parasitic resistance, however it makes the control problem hard to tackle by observer-based design due to the lack of detectability. To overcome the detectability obstacle, a filtered transformation based on immersion and invariance technique is proposed that immerses the system dynamics to a proper adaptive observer form. A reduced-order observer is designed based on the transformation that is used in conjunction with a dynamic controller to render the closed-loop system uniformly globally asymptotically stable.

Keywords: Observers for nonlinear systems, Nonlinear systems, Algebraic/geometric methods
Abstract: Many nonlinear systems in robotics and avionics can be represented as group affine dynamical systems on Lie groups. For such systems, the classical pre-observer error dynamics are state-independent but not autonomous (depending on the input signal), making global constructive observer design difficult, although linearisation based observers such as the IEKF are applicable. In this paper, we show that any group affine system can be represented as the sum of a left-invariant vector field and a vector field in the automorphism Lie algebra of the group. This structure is exploited to define a novel geometry-based observer architecture with an autonomous (independent of both the observer state and the input signal) global pre-observer error. The autonomy of the pre-observer error dynamics enables the design of simple observers for group affine systems without the need for linearisation. An example of estimating the relative SE(3) transformation between two vehicles shows the effectiveness of the proposed observer architecture in aiding constructive observer design.

Keywords: Delay systems, Lyapunov methods, Stability of linear systems
Abstract: This paper deals with the admissibility analysis of continuous-time descriptor systems with a constant delay. A transformation-based model is used to convert the original system into a delay-free system. As a result, new regularity and impulse-free characteristics, depending on a constant delay, have been presented. Based on the generalized Lyapunov function, a necessary and sufficient condition has been presented to ensure that the delay-descriptor system is asymptotically stable. The obtained conditions are formulated as a set of linear matrix inequalities. Finally, an example is presented to test the merit and validity of the proposed method.

Keywords: Delay systems, Nonlinear systems
Abstract: In this contribution, we study an homogeneous class of nonlinear time delay systems with time-varying perturbations. Using the Lyapunov-Krasovskii approach, we introduce a functional that leads to perturbation conditions matching those obtained previously in the Razumikhin framework. The functionals are applied to the estimation of the domain of attraction and of the system solutions. An illustrative example is given.

Keywords: Delay systems, Quantized systems, Sampled-data control
Abstract: In this paper we deal with the stabilization problem of nonlinear systems affected by state delays and known time--varying disturbances via quantized sampled--data dynamic output feedback (QSDOF) controllers. In particular, a QSDOF controller for nonlinear time--delay systems is proposed by means of the novel notion of Dynamic Output Steepest Descent Feedback (DOSDF) induced by a general class of Lyapunov-Krasovskii functionals. Then, the theory of the stabilization in the sample--and--hold sense is used in order to show that: there exist a suitably small inter--sampling time and a suitably accurate quantization of the input/output channels such that the semi--global practical stability of the related quantized sampled--data closed--loop system is ensured with arbitrarily small final target ball of the origin. In the theory here developed, time-varying sampling intervals as well as non--uniform quantization of the input/output channels are taken into account. Furthermore, the stable inter--sampling system behaviour is proved. The case of nonlinear delay-free systems is addressed as a special case. An example is presented which validates the theoretical results.

Keywords: Delay systems, Robust control, LMIs
Abstract: Non-conservative bounded real lemma is derived for interval delay systems, which characterizes the achievable H-infinity performance with delay-free parameter-dependent LMIs. The conditions are extended to the evaluation of delay-independent H-infinity performance, and some interpretations are provided on the relationship between the resulting LMIs and a class of decision variables. For both problems, a branch-and-bound method for evaluating the performance is discussed with numerical examples.

Keywords: Delay systems, Robust control, LMIs
Abstract: A novel way to establish conic bounds for stable linear time-invariant (LTI) systems with time-varying state delays is proposed. Two linear matrix inequality (LMI) con- ditions are provided. Alone, they can be used as analysis tools, determining conic bounds for LTI systems subject to bounded, time-varying state delays. Combined with the Conic Sector Theorem, the conic bounds established by the proposed LMIs can be used directly to design conic controllers, ensuring the closed-loop input-output stability.

Keywords: Delay systems, Stability of nonlinear systems
Abstract: We build delay-compensating feedback controls for a class of nonlinear systems that include bilinear systems with arbitrarily long input delays. Unlike prior sequential predictor work, we cover bilinear systems whose state measurements have uncertainty, and we prove input-to-state stability with respect to this uncertainty. We do not require constructing or estimating distributed terms in the control formulas. We illustrate our result in a power systems example.

Keywords: Agents-based systems, Networked control systems
Abstract: Distributed information spreading algorithms represent an important building block in Aggregate Computing. This paper considers a special case, namely one that determines the most probable path for message delivery from a set of sources to each device in a network. We formulate a Lyapunov function to prove its semi-global stability subject to the requirement that estimated probabilities are initialized to the natural interval (0,1) . We also prove that the algorithm converges in a finite time, and is ultimately bounded under persistent measurement errors. Finally, we provide tight bounds for convergence time, the ultimate bound, and the time for its attainment.

Keywords: Network analysis and control, Algebraic/geometric methods, Feedback linearization
Abstract: Given an additive network of input-output systems where each node of the network is modeled by a locally convergent Chen-Fliess series, two basic properties of the network are established. First, it is shown that every input-output map between a given pair of nodes has a locally convergent Chen-Fliess series representation. Second, sufficient conditions are given under which the input-output map between a pair of nodes has a well defined relative degree as defined by its generating series. This analysis leads to the conclusion that this relative degree property is generic in a certain sense.

Keywords: Network analysis and control, Constrained control, Control of networks
Abstract: This paper considers the connectivity maintenance problem for multi-robot systems and proposes a continuous optimization-based controller with Nonsmooth Control Barrier Functions to achieve it. The design is based on the concept of algebraic connectivity of the interaction graph. When viewed as a function of the network state, the algebraic connectivity is not continuously differentiable, a fact neglected in previous controller designs. To illustrate the importance of this observation, we present an example of a simple multi-robot system that fails to maintain connectivity under such controllers. The insights gained allow us to synthesize an optimization-based controller that prescribes that all the nontrivial eigenvalues of the Laplacian remain positive. Using tools from nonsmooth analysis and set-valued theory, we show that the proposed controller is continuous, thereby guaranteeing the existence of the robot trajectories for the closed-loop system and ensuring network connectivity is maintained along them.

Keywords: Network analysis and control, Large-scale systems, Distributed parameter systems
Abstract: In this work we study a continuation method which transforms spatially distributed ODE systems into PDEs that respect the spatial structure of the original ODE systems. Such PDE description can be used not only for analysis but also for a continuous control design which, being discretized back, results in a nontrivial control law for the original ODE system. In this paper we focus on the continuation for linear systems, including multidimensional inhomogeneous systems and in particular linear networks, showing that such systems can be transformed into general second-order parabolic PDEs. The method is applied to the stabilization of a chain of coupled semiconductor lasers. We obtain a PDE model of this system, design a backstepping-based boundary control to stabilize the obtained PDE and then translate the control policy back to the original laser chain, effectively stabilizing it.

Keywords: Network analysis and control, Distributed control, Decentralized control
Abstract: This paper considers the problem of distributively verifying and ensuring strong connectivity of directed networks. Strong connectivity of a directed graph associated with the communication network topology is crucial in ensuring the convergence of many distributed algorithms. Specifically, inspired by maximum consensus algorithm, we first propose a distributed algorithm that enables nodes in a networked system to verify strong connectivity of a directed graph. Then, given an arbitrary weakly connected directed graph, we develop a distributed algorithm to augment additional links to ensure the directed graph’s strong connectivity. Both algorithms are implemented without requiring information of the overall network topology and are scalable (linearly with the number of nodes) as they only require finite storage and converge in finite number of steps. Finally, the proposed distributed algorithms are demonstrated via several examples.

Keywords: Network analysis and control, Finance, Optimization
Abstract: Modern financial networks are characterized by complex structures of mutual obligations. Such interconnections may propagate and amplify individual defaults, leading in some cases to financial disaster. For this reason, mathematical models for the study and control of systemic risk have attracted considerable research attention in recent years. One important line of research is concerned with mechanisms of clearing, that is, the mechanism by which mutual debts are repaid, in the regular regime, or in a default regime. One of the first models of a clearing mechanism was proposed by Eisenberg and Noe in [1], which introduced the concept of clearing vector of payments. In this paper, we propose a necessary and sufficient condition for the uniqueness of the clearing vector applicable to an arbitrary topology of the financial network. Further, we show that the overall system loss can be reduced if one relaxes the pro-rata rule and replaces the clearing vector by a matrix of clearing payments. This approach shifts the focus from the individual interest to the system, or social, interest, in order to control and contain the adverse effects of cascaded failures.

Keywords: Large-scale systems, Nonlinear systems, Distributed parameter systems
Abstract: We show that an infinite network of input-to-state stable (ISS) systems, admitting ISS Lyapunov functions, itself admits an ISS Lyapunov function, provided that the couplings of the subsystems are sufficiently weak. The strength of the couplings is described in terms of the properties of the so-called gain operator, built from the interconnection gains. If the discrete-time system induced by a slightly scaled gain operator is uniformly globally asymptotically stable, an ISS Lyapunov function for the infinite network can be constructed.

Keywords: Network analysis and control, Lyapunov methods, Large-scale systems
Abstract: We present a Lyapunov-based small-gain theorem for input-to-state stability (ISS) of networks composed of infinitely many finite-dimensional systems. A key assumption in our results is that the internal Lyapunov gains, modeling the influence of the subsystems on each other, are linear functions. Moreover, the gain operator constructed from the internal gains is assumed to be subadditive and homogeneous. This covers both max-type and sum-type formulations for the ISS Lyapunov functions of the subsystems. We formulate the small-gain condition in terms of a generalized spectral radius of the gain operator. The effectiveness of our results are illustrated through an example. Particularly, we show that the small-gain condition can easily be checked if the interconnection topology of the network has some kind of symmetry.

Keywords: Large-scale systems, Networked control systems, Distributed control
Abstract: In many networked dynamic systems, algebraic connectivity plays a key role in important system properties such as stability, string stability and rate of convergence. There has therefore been recent interest in classes of networks (e.g. expander networks, random networks) that have good algebraic connectivity properties while maintaining a low nodal degree. In this paper, we give some initial results on methods to allow addition of agents to an existing network. We are particularly interested in methods with minimal disruption to the existing network and which preserve, to the extent possible, algebraic connectivity properties. To this end upon connection of a node any new connections must preserve an upper bound on the degree of all vertices and no existing connections may be severed. We find conditions on the ability to connect nodes and give some indicative studies on the problem of new link selection.

Keywords: Game theory, Networked control systems, Cyber-Physical Security
Abstract: In this paper we formulate a two-player game-theoretic problem on resilient graphs in a multiagent consensus setting. An attacker is capable to disable some of the edges of the network with the objective to divide the agents into clusters by emitting jamming signals while, in response, the defender recovers some of the edges by increasing the transmission power for the communication signals. We consider repeated games between the attacker and the defender where the optimal strategies for the two players are derived in a rolling horizon fashion based on the agents' states and number of agents in each cluster. The players' actions at each discrete-time steps are constrained by their energy for transmissions of signals, with a less strict constraint for the attacker. Simulation results are provided to demonstrate the effects of players' actions on the cluster formation and to illustrate the performance comparison with a non-rolling horizon approach.

Keywords: Network analysis and control, Nonlinear systems, Stochastic systems
Abstract: When measuring nodes’ importance in a network, the interconnections and dynamics are often supposed to be perfectly known. In this paper, we consider networks of agents with both uncertain couplings and dynamics. The network uncertainty is modeled by structured additive stochastic disturbances on each agent’s update dynamics and coupling weights. We then study how these uncertainties change the network centralities. Disturbances on the couplings between agents result in bilinear dynamics, and classical centrality indices from linear network theory need to be redefined. To do that, we first show that, similarly to its linear counterpart, the squaredH2norm of bilinear systems measures the trace of the steady-state error covariance matrix subject to stochastic disturbances. This makes theH2norm a natural candidate for a performance metric of the system. We propose a centrality index for the agents based on the H2-norm and show how it depends on the network topology and the noise structure. Finally, we simulate a few graphs to illustrate how uncertainties on different couplings affect the agents’ centrality rankings compared to a linearized model of the same system.

Keywords: Networked control systems, Network analysis and control, Control of networks
Abstract: We develop a systemic risk framework to explore cascading failures in networked control systems. A time- delayed version of the vehicle platooning problem is used as a benchmark to study the interplay among network connectivity, system dynamics, physical limitations, and uncertainty onto the possibility of cascading failure phenomena. The measure of value-at-risk is employed to investigate the domino effect of failures among pairs of vehicles within the platoon. The systemic risk framework is suitably extended to quantify the robustness of cascading failures via a novel manipulation of bi-variate distribution. We establish closed-form risk formulas that explain the effect of network parameters (e.g., Laplacian eigen-spectrum, time delay), noise statistics, and systemic event sets onto the cascading failures. Our findings can be applied to the design of robust platoons to lower the cascading risk. We support our theoretical results with extensive simulations

Keywords: Computer-aided control design, Control software, Control applications
Abstract: ControlSystems.jl enables the powerful features of the Julia language to be leveraged for control design and analysis. The package provides types for state-space, transfer-function, and time-delay models, together with algorithms for design and analysis. Julia's mathematically-oriented syntax is convenient for implementing control algorithms, and its just-in-time compilation gives performance on par with C. The multiple-dispatch paradigm makes it easy to combine the algorithms with powerful tools from Julia's ecosystem, such as automatic differentiation, arbitrary-precision arithmetic, GPU arrays, and probability distributions. We demonstrate how these features allow complex problems to be solved with little effort.

Keywords: Optimization, Control software, Nonlinear systems
Abstract: Pyomo and Dakota are openly available software packages developed by Sandia National Labs. In this tutorial, methods for automating the optimization of controller parameters for a nonlinear cart-pole system are presented. Two approaches are described and demonstrated on the cart-pole example problem for tuning a linear quadratic regulator and also a partial feedback linearization controller. First the problem is formulated as a pseudospectral optimization problem under an open box methodology utilizing Pyomo, where the plant model is fully known to the optimizer. In the next approach, a black-box approach utilizing Dakota in concert with a MATLAB or Simulink plant model is discussed, where the plant model is unknown to the optimizer. A comparison of the two approaches provides the end user the advantages and shortcomings of each method in order to pick the right tool for their problem. We find that complex system models and objectives are easily incorporated in the Dakota-based approach with minimal setup time, while the Pyomo-based approach provides rapid solutions once the system model has been developed.

Keywords: Control software, Computer-aided control design, Stochastic systems
Abstract: We present the GPU Accelerated Trajectory Evaluation (GATE) tool. GATE is a controller evaluation tool written in C++ and CUDA which utilizes Nvidia GPUs to accelerate parallelizable computations for Monte Carlo controller evaluation. GATE offers plug-and-play capabilities that enable both fast development and fast execution of Monte Carlo simulations for controller robustness evaluation. We illustrate the use of GATE for evaluating the robustness of different controllers on Dubins, mass-spring-damper, cart-pole, and quadcopter systems and for performing sensitivity analysis. We also demonstrate the ability of GATE's core technology to outperform the simulation speeds of five other parallel computation benchmarks.

Keywords: Power systems, Smart grid, Optimization algorithms
Abstract: Recent grid energy problems in California and Texas highlight the need for optimized grid design and operation. The complexity of the electric grid presents difficult control problems that require powerful solvers and efficient formulations for tractable solutions. The Gekko Optimization Suite is a machine learning and optimization package in Python for optimal control with differential algebraic equations and is capable of solving complex grid design and control problems. A series of non-dimensional benchmark cases are proposed for grid energy production. These include (I) load following, (II) cogeneration, (III) tri-generation, and energy storage with (IV) constant production, (V) load following, and (VI) cogeneration. Individual case studies include ramp rate constraints, power production, and energy storage operation as design variables. The tutorials demonstrate methods to solve control problems with sequential and simultaneous solutions of the objective and dynamic constraints. While these tutorials are specific to grid energy system optimization, the tutorials also demonstrate how to efficiently solve large-scale nonlinear dynamic systems with a trade-off analysis between sequential and simultaneous methods.

Keywords: Control software
Abstract: The Python Control Systems Library (python-control) is an open source set of Python classes and functions that implement common operations for the analysis and design of feedback control systems. In addition to support for standard LTI control systems (including time and frequency response, block diagram algebra, stability and robustness analysis, and control system synthesis), the package provides support for nonlinear input/output systems, including system interconnection, simulation, and describing function analysis. A MATLAB compatibility layer provides an many of the common functions corresponding to commands available in the MATLAB Control Systems Toolbox. The library takes advantage of the Python ``scientific stack'' of Numpy, Matplotlib, and Jupyter Notebooks and offers easy interoperation with other category-leading software systems in data science, machine learning, and robotics that have largely been built on Python.

Keywords: Control Systems Privacy, Control over communications
Abstract: Encrypted control systems allow to evaluate feedback laws on external servers without revealing private information about state and input data, the control law, or the plant. While there are a number of encrypted control schemes available for linear feedback laws, only few results exist for the evaluation of more general control laws. Recently, an approach to encrypted polynomial control was presented, relying on two-party secret sharing and an inter-server communication protocol using homomorphic encryption. As homomorphic encryptions are much more computationally demanding than secret sharing, they make up for a tremendous amount of the overall computational demand of this scheme. For this reason, in this paper, we demonstrate that multi-party computation enables secure polynomial control based solely on secret sharing. We introduce a novel secure three-party control scheme based on three-party computation. Further, we propose a novel n-party control scheme to securely evaluate polynomial feedback laws of arbitrary degree without inter-server communication. The latter property makes it easier to realize the necessary requirement regarding non-collusion of the servers, with which perfect security can be guaranteed. Simulations suggest that the presented control schemes are many times less computationally demanding than the two-party scheme mentioned above.

Keywords: Control Systems Privacy, Estimation, Optimal control
Abstract: Privacy is often the first line of defense against cyber-physical attacks. In this paper, we derive guarantees for the privacy of structured inputs to linear time-invariant systems, where the eavesdropper either does not know the input or only knows parts of it a priori. The input is be parametrized by a mixture of discrete and continuous parameters. Privacy guarantees for these parameters are then derived using a Barankin-style bound. Given an open-loop control objective, a modification to the cost function is proposed to enhance privacy. Privacy-utility trade-off bounds are derived for these private open-loop control signals. Finally, the theoretical results are verified both using the physical Temperature Control Lab and a numerical simulation of it.

Keywords: Control Systems Privacy, Cyber-Physical Security, Computer/Network Security
Abstract: This study concentrates on preserving privacy in a network of agents where each agent desires to evaluate a polynomial function over the private values of its immediate neighbors. We provide an algorithm for the exact evaluation of this function while preserving privacy of the involved agents. The solution is based on two cryptographic primitives: Paillier as a Partially Homomorphic Encryption scheme and multiplicative-additive secret sharing. The provided scheme covers a large class of polynomial functions in distributed systems. Moreover, conditions guaranteeing the privacy preservation of the private value of an agent against a set of colluding agents are derived. The simulation results demonstrate that the proposed scheme can be employed in a network to enhance privacy at the cost of extra communication and computation budgets.

Keywords: Control Systems Privacy, Optimization algorithms, Computer/Network Security
Abstract: The least squares problem with l1-regularized regressors, called Lasso, is a widely used approach in optimization problems where sparsity of the regressors is desired. As motivation, we investigate a sparse data predictive control problem, run at a cloud service to control a system with unknown model, using l1-regularization to limit the behavior complexity. The collected input-output data is privacy-sensitive, hence, we design a privacy-preserving solution using homomorphically encrypted data. The main challenges are the non-smoothness of the l1-norm, which is difficult to evaluate on encrypted data, as well as the iterative nature of the Lasso problem. We use a distributed ADMM formulation that enables us to exchange substantial local computation for little communication between a few servers. We give an encrypted multi-party protocol for solving the distributed Lasso problem, by approximating the non-smooth part with a polynomial, evaluating it on encrypted data, and using a more cost effective distributed bootstrapping operation. For the example of data predictive control, we prefer a heterogeneous splitting of the data for better convergence and give an encrypted protocol for one powerful server and a few less powerful devices, added for security reasons. Finally, we provide numerical results for our proposed solutions.

Keywords: Cyber-Physical Security, Control Systems Privacy, Networked control systems
Abstract: Encrypted control seeks confidential controller evaluation in cloud-based or networked systems. Many existing approaches build on homomorphic encryption (HE) that allow simple mathematical operations to be carried out on encrypted data. Unfortunately, HE is computationally demanding and many control laws (in particular non-polynomial ones) cannot be efficiently implemented with this technology. We show in this paper that secure two-party computation using garbled circuits provides a powerful alternative to HE for encrypted control. More precisely, we present a novel scheme that allows to efficiently implement (non-polynomial) max-out neural networks with one hidden layer in a secure fashion. These networks are of special interest for control since they allow, in principle, to exactly describe piecewise affine control laws resulting from, e.g., linear model predictive control (MPC). However, exact fits require high-dimensional preactivations of the neurons. Fortunately, we illustrate that even low-dimensional learning-based approximations are sufficiently accurate for linear MPC. In addition, these approximations can be securely evaluated using garbled circuit in less than 100 ms for our numerical example. Hence, our approach opens new opportunities for applying encrypted control.

Keywords: Control Systems Privacy, Cyber-Physical Security, Agents-based systems
Abstract: This paper considers the use of homomorphic encryption for the realization of distributed formation control of multi-agent systems via edge computer. In our proposed framework, the distributed control computation in the edge computer uses only the encrypted data without the need for a reset mechanism that is commonly required to avoid error accumulation. Simulation results show that, despite the use of encrypted data on the controller and errors introduced by the quantization process prior to the encryption, the formation is able to converge to the desired shape. The proposed architecture offers insight on the mechanism for realizing distributed control computation in an edge/cloud computer while preserving the privacy of local information coming from each agent.

Keywords: Networked control systems, Power systems
Abstract: We consider the control problem of fulfilling the desired total charging/discharging power while balancing the state-of-charge (SoC) of the networked battery units with unknown parameters in a battery energy storage system. We develop power allocating algorithms for the battery units. These algorithms make use of distributed estimators for the average desired power and the average unit state and the adaptive parameter estimators, and are showed to fulfill the desired total power while achieving battery unit SoC balancing as long as the undirected graph that represents the communication network among the battery units is connected and there is at least one battery unit with the knowledge of the desired total power. Numerical simulation verifies the effectiveness of the proposed power allocating algorithms.

Keywords: Smart grid, Power systems, Decentralized control
Abstract: Thermostatic loads have long been recognized as a useful resource for shaping aggregate load dynamics and for mitigation of intermittency in solar and wind based renewable generation. When considering the residential sector, there can be millions of such loads, each with a very small contribution but can be collectively coordinated. Mean field game (MFG) theory has emerged as a natural tool for mathematically capturing this framework. While most existing works on MFG analysis start from agent cost functions to identify the properties of potential Nash equilibria and the associated agent control laws, this work differs in that an inverse process is followed. We reverse engineer the agent cost functions so that, while remaining comfort sensitive, they are guaranteed to lead via decentralized control laws to a precalculated Nash equilibrium. The latter is desirable from an aggregator's point of view. The approach is illustrated in the case of a power reduction objective for a collection of thermal loads. Furthermore, to effectively manage the impacts of load control actions in a power distribution network, it is shown how the control efforts can be adjusted on a nodal basis using voltage sensitivities.

Keywords: Power systems, Smart grid
Abstract: This paper examines the marginal value of mobile energy storage, i.e., energy storage units that can be efficiently relocated to other locations in the power network. In particular, we formulate and analyze the joint problem for operating the power grid and a fleet of mobile storage units. By explicitly connecting the marginal value of mobile storage to locational marginal prices (LMPs), we propose efficient algorithms that only use LMPs and transportation costs to optimize the relocation trajectories of the mobile storage units. Furthermore, we provide examples and conditions under which the marginal value of mobile storage is strictly higher, equal to, or strictly lower than the sum of marginal value of corresponding stationary storage units and wires.

Keywords: Power systems, Reduced order modeling, Nonlinear systems
Abstract: This paper presents parametric assumptions under which the classical Kuramoto model can be uncovered from full-order models of an interconnected group of grid-forming inverters based on droop-control, virtual synchronous machine control and/or dispatchable virtual oscillator control. The equivalence is established with reduced-order models that are derived by leveraging singular-perturbation analysis, time-domain Kron reduction of the network dynamics, and a particular control configuration that is based on the inductance-to-resistance ratio of interconnecting transmission lines. For the derived Kuramoto model, we establish stability, existence, and uniqueness of frequency-synchronized equilibria, as well as recover a closed-form expression for the so-called synchronization frequency. Numerical results compare the phase and frequency response of the full-order model and the reduced-order Kuramoto model.

Keywords: Distributed control, Energy systems, Nonlinear output feedback
Abstract: In this paper, we formulate for the first time an interactive input-output (I-O) distributed control problem in dynamic energy state space. The primary difficulty in distributed control problem by component i is finding consistent relations between control specifications on the output of interest y_i and shared output variables by the neighboring components z_j. The basic idea in the paper is to align these seemingly inconsistent objectives by imposing generalized Tellegen's theorem conditions in energy state-space at the component interfaces. The shared variable z_j is then shown to take on a physical meaning of power and the rate of change of reactive power. Using this modeling approach, we derive sufficient general conditions to guarantee that interconnected components form a feasible system. We elucidate steps to test whether the interconnected components will converge to an equilibrium. We show that the derived I-O feedback control stabilizes and regulates the outputs of interest by exchanging information about interaction variables between the neighboring components. Finally, we illustrate the first-of-its-kind combined feed-forward and feedback control in energy space capable of following an exogenous time-varying power reference by a controllable voltage source in an RLC circuit. Notably, the system is representative of inverter control of DC microgrids.

Keywords: Energy systems, Optimal control, Power systems
Abstract: The battery energy storage system's integration with renewable energy (RE) micro-grids play an important role in solving power supply problems. To achieve reliable and economic operations of a RE micro-grid, in addition to maximize the integration of of renewable resources, the lifetime characteristics of a battery energy storage system also need to be fully investigated.

This research study develops an optimization model that includes battery life loss cost, states switching costs, and operation and maintenance cost to obtain a set of optimal parameters of operation strategy. Considering the lifetime characteristics of battery storage system, a multi-objective optimization to maximize the power sold values, and to minimize the degradations concerning battery life cycles has been achieved being main control objectives of the research under study. Based on a model adopting mixed-integer constraints and dynamics, the problem of optimal load demand tracking, and electricity market participation is solved through the implementation of an model based predictive control (MPC) scheme. The efficacy of the proposed controller is proved through extensive simulations where the RE-based micro-grid running costs are minimized.

Keywords: Network analysis and control, Agents-based systems, Networked control systems
Abstract: A concatenated Friedkin-Johnsen (FJ) model is a two time-scale opinion dynamics model in which stubborn agents discuss a sequence of issues. For each issue, a FJ model is adopted, and concatenation refers to the fact that the final opinion of the agents at issue s becomes the initial condition at issue s+1. In this paper we deal with the case in which the system is open, i.e., the group of interacting agents changes at each issue, and so does their stubbornness. A concatenated FJ model can in this case be represented as an infinite product of stochastic matrices. For such system, we obtain sufficient conditions under which the opinions of the agents converge to consensus.

Keywords: Networked control systems, Game theory, Network analysis and control
Abstract: We study a model of online news dissemination on a Twitter-like social network. Given a news item and its credibility, agents with heterogeneous priors strategically decide whether to share the news with their followers. An agent shares the news, if the news can persuade her followers to take an action (such as voting) in line with the agent's perspectives. We describe the agent's decision making and the conditions that lead to sharing the news with followers, and characterize the size of news spread at the equilibrium of the news-sharing game. We further investigate the impact of the network connectivity, heterogeneity of prior perspectives, and news credibility on the set of the news that can trigger a sharing cascade. Finally, we identify the conditions under which the news with low credibility can spread wider than highly credible news. In particular, we show that when the network is highly-connected or the news is not a "tail event", a sharing cascade can occur even with news that is not credible.

Keywords: Agents-based systems, Stochastic systems, Network analysis and control
Abstract: We develop and analyze a collective decision-making model concerning the adoption and diffusion of a novel product, convention, or behavior within a population. Motivated by the growing social psychology literature on dynamic norms, under which an individual is influenced by changing trends in the population, we propose a stochastic model for the decision-making process encompassing two behavioral mechanisms. The first is social influence, which drives coordination among individuals. Consistent with the literature on social diffusion modeling, we capture such a mechanism through an evolutionary game-theoretic framework for a network of interacting individuals. The second, which is the main novelty of our model, represents the impact of dynamic norms, capturing the tendency of individuals to be attracted to products or behaviors with growing popularity. We analytically determine sufficient conditions under which a novel alternative spreads to the majority of the population. Our findings provide insights into the unique and nontrivial role of human sensitivity to dynamic norms in facilitating social diffusion.

Keywords: Game theory, Network analysis and control, Learning
Abstract: Whilst network coordination and anticoordination games have received a considerable amount of attention in the literature, network games with coexisting coordinating and anti-coordinating players are known to exhibit more complex behaviors. In fact, depending on the network structure, such games may even fail to have pure-strategy Nash equilibria. An example is represented by the well-known matching pennies game. In this work, we first provide graph-theoretic conditions for the existence of pure-strategy Nash equilibria in mixed network coordination/anti-coordination games of arbitrary size. For the case where such conditions are met, we then study the asymptotic behavior of best-response dynamics and provide sufficient conditions for finite-time convergence to the set of Nash equilibria. Our results build on an extension and refinement of the notion of network cohesiveness and on the formulation of the new concept of network indecomposibility.

Keywords: Network analysis and control, Control of networks, Cooperative control
Abstract: Opinion-evolution and spread processes on networks (e.g., infectious disease spread, opinion formation in social networks) are not only high dimensional but also volatile and multi-scale in nature. In this study, we explore whether snapshot data from these processes can admit terse representations. Specifically, using three case studies, we explore whether the data are compressible in the Laplacian-eigenvector basis, in the sense that each snapshot can be approximated well using a (possibly different) small set of basis vectors. The first case study is concerned with a linear consensus model that is subject to a stochastic input at an unknown location; both empirical and formal analyses are used to characterize compressibility. Second, compressibility of state snapshots for a stochastic voter model is assessed via an empirical study. Finally, compressibility is studied for state-level daily COVID-19 positivity-rate data. The three case studies indicate that state snapshots from opinion-evolution and spread processes allow terse representations, which nevertheless capture their rich propagative dynamics.

Keywords: Control of networks, Agents-based systems, Biological systems
Abstract: For a group of autonomous communicating agents to carry out coordinated objectives, it is paramount that they can distinguish meaningful input from disturbance, and come rapidly and reliably to agreement or disagreement in response to that input. We study how opinion formation cascades through a group of networked decision makers in response to a distributed input signal. Using a nonlinear opinion dynamics model with dynamic feedback modulation of an attention parameter, we prove how the triggering of an opinion cascade and the collective decision itself depend on both the distributed input and node agreement and disagreement centrality indices, determined by the spectral properties of the network graph. Moreover, we show how the attention dynamics introduce an implicit threshold that distinguishes between distributed inputs that trigger cascades and ones that are rejected as disturbance.

Keywords: Autonomous systems, Predictive control for nonlinear systems
Abstract: This paper proposes a novel NMPC for multi-rotor aerial vehicles which is designed without stabilizing terminal costs or constraints in its cost function for stabilization. A growth bound sequence is derived from a tailored running cost to ensure the closed-loop stability and provide a measure of the performance of the proposed NMPC scheme. Furthermore, it facilitates the computation of a stabilizing prediction horizon that guarantees the asymptotic stability of the system. The performance of the proposed scheme is investigated through two sets of numerical simulations and compared against the traditional NMPC scheme for the application as proposed in (Kamel et al., 2017). The results show superior performance of the proposed NMPC scheme in terms of tracking accuracy, convergence rate, and computation time.

Keywords: Autonomous vehicles, Game theory
Abstract: Highway merging is a challenging task for an autonomous vehicle, because the vehicle must interact with other vehicles to identify a gap to merge into before the ending of current lane while maintaining safety (i.e., avoiding collisions). In this paper, we treat the problem of autonomous vehicle planning and control for forced merge scenarios by proposing a novel decision-making algorithm based on a partially observable leader-follower game that models the interaction among merging and highway driving vehicles. In the proposed algorithm, the autonomous ego vehicle applies a receding-horizon optimization-based control strategy that adapts to online estimated driving intents of the other vehicles to simultaneously achieve safety (i.e., avoiding collisions) and liveness (i.e., completing the merging task). The algorithm is validated by multiple simulation-based case studies.

Keywords: Autonomous vehicles, Linear parameter-varying systems
Abstract: In this paper, we propose a linear parameter-varying (LPV) kinematic model for automated parking systems. We also design an LPV H2 state feedback controller with the proposed model. The vehicle kinematic model is intrinsically a nonlinear system, but we show that the kinematic model can be represented as an LPV kinematic model. The state feedback control gain is obtained using convex interpolation in the H2 sense of a linear matrix inequality approach. We will show that the proposed parking system guarantees the stability of the closed-loop system with disturbances. To validate the proposed method, we conduct parking experiments with a test vehicle for two scenarios. Using the proposed LPV H2 state feedback controller, the vehicle tracks paths for all scenarios, reaching the final parking spot and showing smooth steering performance. The results show that the lateral position error and the heading angle error are smaller than 0.05 m and 0.005 rad, respectively.

Keywords: Optimal control, Autonomous vehicles, Optimization algorithms
Abstract: We consider the navigation of surface vessels in an evolving ocean wave field, where the detrimental extreme waves are treated as moving and deforming obstacles. We propose a new time-optimal path planning algorithm with avoidance of such obstacles based on the reachability theory. In our framework, the mathematical optimization problem is boiled down to:(1) forward propagation of reachable set described by a variational inequation,which is numerically solved through a prediction-correction scheme;and (2) a backtracking procedure to find the optimal path. The new path planner is tested in a number of cases including one with realistic ocean waves. We demonstrate that our algorithm is able to provide correct paths avoiding high waves, and achieve multiple optimal (equivalent in time) paths when available.Finally, the framework can be straightforwardly extended to other problems involving collision avoidance with moving and deforming obstacles.

Keywords: Autonomous vehicles, LMIs, Optimization algorithms
Abstract: In this paper, we consider the computation of robust N-step backward reachable sets for state- and input constrained linear time varying systems with additive uncertainty. We propose a method to compute a linear, time-varying control law that maximizes the volume of the robust N-step reachable set for the closed loop system. The proposed method is an extension of recent developments and involves the recursive solution of N semi-definite programs (SDP). We demonstrate the performance of the proposed method on the lateral control problem for emergency maneuvers of autonomous vehicles and compare it to results obtained when backward reachability is applied to the same system and a naively designed controller.

Keywords: Autonomous vehicles, Neural networks, Adaptive control
Abstract: The deployment of future intelligent transportation systems is contingent upon seamless and reliable operation of connected and autonomous vehicles (CAVs). One key challenge in developing CAVs is the design of an autonomous controller that can make use of wireless connectivity and accurately execute control decisions, such as a quick acceleration when merging to a highway and frequent speed changes in a stop-and-go traffic. However, the use of conventional feedback controllers or traditional machine learning based controllers, solely trained by the CAV’s local data, cannot guarantee a robust controller performance over a wide range of road conditions and traffic dynamics. In this paper, a new federated learning (FL) framework enabled by the CAVs’ wireless connectivity is proposed for the autonomous controller design of CAVs. In this framework, the learning models used by the controllers are collaboratively trained among a group of CAVs. To capture the varying CAV participation in FL and the diverse local data quality among CAVs, a novel dynamic federated proximal (DFP) algorithm is proposed that accounts for the mobility of CAVs, the wireless channel dynamics, as well as the unbalanced and non-independent and identically distributed data across CAVs. A rigorous convergence study is performed for the proposed algorithm under realistic wireless environments. Then, the impact of varying CAV participation in FL process and diverse local data quality of CAVs on the convergence is explicitly analyzed. Simulation results that use real vehicular data show that the proposed DFP-based controller can accurately track the target speed over time and under different traffic scenarios, and it yields a distance error two times smaller than controllers designed using traditional machine learning solutions trained with the CAV’s local data. The results also show that the proposed DFP algorithm is well-suited for the autonomous controller design in CAVs when compared to popular FL algorithms, such as federated averaging (FedAvg) and federated proximal (FedProx) algorithms.

Keywords: Linear parameter-varying systems, Behavioural systems, Learning
Abstract: Based on the Fundamental Lemma by Willems et al., the full system behaviour of a Linear Time-Invariant (LTI) system can be characterised by a single sequence of data from the system as long the input is persistently exciting. This is an essential result for data-driven analysis and control. In this work, we aim to generalize this LTI result to Linear Parameter-Varying (LPV) systems. Based on the behavioural framework for LPV systems, we prove that one can obtain a result similar to Willems'. This implies that the result is also applicable to nonlinear system behaviour that can be captured with an LPV representation. We show the applicability of our result by connecting it to earlier works on data-driven analysis and control for LPV systems.

Keywords: Automotive control, Automotive systems, Control applications
Abstract: The paper presents a data-driven control design for a steering system, which is based on variable-geometry suspension. The proposed control system into a hierarchical structure with different roles on each layer is ordered, i.e., low-level and high-level controls are designed. The low-level controller is responsible for the realization of the steering angle, while the high-level controller guarantees the trajectory tracking of the vehicle. The low-level controller based on a polytopic model is designed, which model structure through a data-driven identification approach is carried out. The effectiveness and the operation of the hierarchical control structure through a Hardware-in-the-Loop (HiL) simulation are demonstrated. In the HiL simulation environment, the lateral dynamics of the vehicle by the CarMaker software is modeled, and the motion of the variable-geometry suspension through a testbed is realized.

Keywords: Control of networks, Linear systems, Estimation
Abstract: Controllability Gramians are the most widely used measures of controllability, which quantify the reachable state sets of systems with a control input of unit energy. They are used as a performance index in various design problems, such as actuator placement and model reduction. If a mathematical model is available for the system to be considered, a controllability Gramian can be easily calculated from its definition or equivalent conditions. However, this is not always the case, because of the insufficiency of data available for system modeling. In such a case, it is practical to use a data-driven method to directly estimate the controllability Gramian. This paper establishes a data-driven framework of estimating and maximizing the controllability Gramians of unknown linear systems. We first develop a method to estimate controllability Gramians by using the measurement data of system behaviors. Furthermore, we present a data-driven solution to maximize the degree of controllability, measured by the trace of the controllability Gramian, with respect to the input matrix.

Keywords: Nonlinear systems, Large-scale systems, Agents-based systems
Abstract: Koopman operator theory provides a model-free and purely data-driven technique for studying nonlinear dynamical systems. Since the Koopman operator is infinite-dimensional, researchers have developed several methods that provide a finite-dimensional approximation of the Koopman operator so that it can be applied for practical use cases. One common thing with most of the methods is that their solutions are obtained by solving a centralized minimization problem. In this work, we treat the dynamical system to be a multi-agent system and propose an algorithm to compute the finite-dimensional approximation of the Koopman operator in a distributed manner using the knowledge of the topology of the underlying multi-agent system. The proposed distributed approach results in a sparse Koopman whose block structure mimics the Laplacian of the multi-agent system. Extensive simulation studies illustrate the proposed framework on the network of oscillators and the IEEE 68 bus system.

Keywords: Randomized algorithms, Uncertain systems, Estimation
Abstract: We present an algorithm for data-driven reachability analysis that estimates finite-horizon forward reachable sets for general nonlinear systems using level sets of a certain class of polynomials known as Christoffel functions. The level sets of Christoffel functions are known empirically to provide good approximations to the support of probability distributions: the algorithm uses this property for reachability analysis by solving a probabilistic relaxation of the reachable set computation problem. We also provide a guarantee that the output of the algorithm is an accurate reachable set approximation in a probabilistic sense, provided that a certain sample size is attained. We also investigate three numerical examples to demonstrate the algorithm's capabilities, such as providing non-convex reachable set approximations and detecting holes in the reachable set.

Keywords: Stochastic optimal control, Machine learning, Autonomous systems
Abstract: We present algorithms for performing data-driven stochastic reachability as an addition to SReachTools, an open-source stochastic reachability toolbox. Our method leverages a class of machine learning techniques known as kernel embeddings of distributions to approximate the safety probabilities for a wide variety of stochastic reachability problems. By representing the probability distributions of the system state as elements in a reproducing kernel Hilbert space, we can learn the "best fit" distribution via a simple regularized least-squares problem, and then compute the stochastic reachability safety probabilities as simple linear operations. This technique admits finite sample bounds and has known convergence in probability. We implement these methods as part of SReachTools, and demonstrate their use on a double integrator system, on a million-dimensional repeated planar quadrotor system, and a cart-pole system with a black-box neural network controller.

Keywords: Neural networks, Lyapunov methods, Machine learning
Abstract: A method is presented to learn neural network (NN) controllers with stability and safety guarantees through imitation learning (IL). Convex stability and safety conditions are derived for linear time-invariant systems with NN controllers by merging Lyapunov theory with local quadratic constraints to bound the activation functions in the NN. These conditions are incorporated in the IL process, which minimizes the IL loss, and maximizes the volume of the region of attraction associated with the NN controller simultaneously. An alternating direction method of multipliers based algorithm is proposed to solve the IL problem. The method is illustrated on a vehicle lateral control example.

Keywords: Neural networks, Nonlinear systems
Abstract: We propose a data-driven control method for nonlinear dynamical systems based on the Koopman operator theory. Existing Koopman-based control methods apply linear optimal control methods after system identification by approximating the original cost function in the Koopman space. Therefore, errors in system identification and cost approximation deteriorate the control performance. On the other hand, the proposed method directly maximizes the control performance with reinforcement learning, where a controller is modeled by a neural network that consists of a linear quadratic regulator and an encoder that embeds data into the Koopman space. We experimentally demonstrate the effectiveness of the proposed method over existing Koopman-based and reinforcement learning-based methods with two nonlinear dynamical systems.

Keywords: Neural networks, Robust control, Optimization
Abstract: The desire to provide robust guarantees on neural networks has never been more important, as their prevalence in society is increasing. One popular method that has seen a large amount of success is to use bounds on the activation functions within these networks to provide such guarantees. However, due to the large number of possible ways to bound the activation functions, there is a trade-off between conservativeness and complexity. We approach the problem from a different perspective, using polynomial optimisation and real algebraic geometry (the Positivstellensatz) to assert the emptiness of a semi-algebraic set. We show that by using the Positivstellensatz, bounds on the robustness guarantees can be tightened significantly over other popular methods, at the expense of computational resource. We demonstrate the effectiveness of this approach on networks that use the ReLU, sigmoid and tanh activation functions. This method can be extended to more activation functions, and combined with recent sparsity-exploiting methods can result in a computationally acceptable method for verifying neural networks.

Keywords: Neural networks, Stability of nonlinear systems, LMIs
Abstract: This paper is concerned with the stability analysis of the recurrent neural networks (RNNs) by means of the integral quadratic constraint (IQC) framework. The rectified linear unit (ReLU) is typically employed as the activation function of the RNN, and the ReLU has specific nonnegativity properties regarding its input and output signals. Therefore, it is effective if we can derive IQC-based stability conditions with multipliers taking care of such nonnegativity properties. However, such nonnegativity (linear) properties are hardly captured by the existing multipliers defined on the positive semidefinite cone. To get around this difficulty, we loosen the standard positive semidefinite cone to the copositive cone, and employ copositive multipliers to capture the nonnegativity properties. We show that, within the framework of the IQC, we can employ copositive multipliers (or their inner approximation) together with existing multipliers such as Zames-Falb multipliers and polytopic bounding multipliers, and this directly enables us to ensure that the introduction of the copositive multipliers leads to better (no more conservative) results. We finally illustrate the effectiveness of the IQC-based stability conditions with the copositive multipliers by numerical examples.

Keywords: Computer-aided control design, Neural networks
Abstract: In this paper, we consider the computational complexity of formally verifying the behavior of Rectified Linear Unit (ReLU) Neural Networks (NNs), where verification entails determining whether the NN satisfies convex polytopic specifications. Specifically, we show that for two different NN architectures -- shallow NNs and Two-Level Lattice (TLL) NNs -- the verification problem with (convex) polytopic constraints is polynomial in the number of neurons in the NN to be verified, when all other aspects of the verification problem held fixed. We achieve these complexity results by exhibiting explicit (but similar) verification algorithms for each type of architecture. Both algorithms efficiently translate the NN parameters into a partitioning of the NN's input space by means of hyperplanes; this has the effect of partitioning the original verification problem into polynomially many sub-verification problems derived from the geometry of the neurons. We show that these sub-problems may be chosen so that the NN is purely affine within each, and hence each sub-problem is solvable in polynomial time by means of a Linear Program (LP). Thus, a polynomial-time algorithm for the original verification problem can be obtained using known algorithms for enumerating the regions in a hyperplane arrangement. Finally, we adapt our proposed algorithms to the verification of dynamical systems, specifically when these NN architectures are used as state-feedback controllers for LTI systems. We further evaluate the viability of this approach numerically.

Keywords: Formal Verification/Synthesis, Neural networks, Networked control systems
Abstract: We propose a framework for solving control synthesis problems for multi-agent networked systems required to satisfy spatio-temporal specifications. We use Spatio-Temporal Reach and Escape Logic (STREL) as a specification language. For this logic, we define smooth quantitative semantics, which captures the degree of satisfaction of a formula by a multi-agent team. We use the novel quantitative semantics to map control synthesis problems with STREL specifications to optimization problems and propose a combination of heuristic and gradient-based methods to solve such problems. As this method might not meet the requirements of a real-time implementation, we develop a machine learning technique that uses the results of the off-line optimizations to train a neural network that gives the control inputs at current states. We illustrate the effectiveness of the proposed framework by applying it to a model of a robotic team required to satisfy a spatial-temporal specification under communication constraints.

Keywords: Traffic control, Randomized algorithms, Statistical learning
Abstract: We evaluate the robustness of agents' traffic equilibria in randomized routing games characterized by an uncertain network demand with a possibly unknown probability distribution. Specifically, we extend the so-called hose model by considering a traffic equilibrium model where the uncertain network demand configuration belongs to a polyhedral set, whose shape is itself a-priori unknown. By exploiting available data, we apply the scenario approach theory to establish distribution-free feasibility guarantees for agents' traffic equilibria of the uncertain routing game without the need to know an explicit characterization of such set. A numerical example on a traffic network testbed corroborates the proposed theoretical results.

Keywords: Machine learning, Robust control, Uncertain systems
Abstract: The combination of machine learning with control offers many opportunities, in particular for robust control. However, due to strong safety and reliability requirements in many real-world applications, providing rigorous statistical and control-theoretic guarantees is of utmost importance, yet difficult to achieve for learning-based control schemes. We present a general framework for learning-enhanced robust con- trol that allows for systematic integration of prior engineering knowledge, is fully compatible with modern robust control and still comes with rigorous and practically meaningful guarantees. Building on the established Linear Fractional Representation and Integral Quadratic Constraints framework, we integrate Gaussian Process Regression as a learning component and state- of-the-art robust controller synthesis. In a concrete robust con- trol example, our approach is demonstrated to yield improved performance with more data, while guarantees are maintained throughout.

Keywords: Optimization, Agents-based systems, Randomized algorithms
Abstract: We consider linear resource sharing problems with multiple agents. Agents are heterogeneous, with heterogeneity modelled by a tuple of parameters taking value according to an underlying probability distribution, and share a fixed resource amount. We provide an evaluation of a vital indicator for the correct operation of the agents, namely, the probability that the optimal resource share alters in case of a new agent arrival. We view this problem under a data driven lens, and provide a purely a-posteriori and prior-independent characterization of the above mentioned probability by exploiting recent developments in the so called scenario approach theory. The proposed framework is demonstrated on an economic dispatch example in power systems, where agents can be thought of as generating units participating in the power market.

Keywords: Neural networks, Optimization, Optimal control
Abstract: Training of neural networks amounts to nonconvex optimization problems that are typically solved by using backpropagation and (variants of) stochastic gradient descent. In this work, we propose an alternative approach by viewing the training task as a nonlinear optimal control problem. Under this lens, backpropagation amounts to the sequential approach (single shooting) to optimal control, where the states variables have been eliminated. It is well known that single shooting may lead to ill-conditioning, and for this reason the simultaneous approach (multiple shooting) is typically preferred. Motivated by this hypothesis, an augmented Lagrangian algorithm is developed that only requires an approximate solution to the Lagrangian subproblems up to a user-defined accuracy. By applying this framework to the training of neural networks, it is shown that the inner Lagrangian subproblems are amenable to be solved using Gauss-Newton iterations. To fully exploit the structure of neural networks, the resulting linear least-squares problems are addressed by employing an approach based on forward dynamic programming. Finally, the effectiveness of our method is showcased on regression datasets.

Keywords: Iterative learning control, Stochastic optimal control, Adaptive systems
Abstract: We propose a reinforcement learning algorithm based on an adaptive state aggregation scheme defined by a progressively growing set of codevectors placed in the joint state-action space according to a maximum-entropy vector quantization scheme. The proposed algorithm constitutes a two-timescale stochastic approximation algorithm with: (a) a fast component that executes a temporal-difference learning algorithm, and (b) a slow component, based on an online deterministic annealing algorithm, that adaptively partitions the state-action space according to a dissimilarity measure that belongs to the family of Bregman divergences. The proposed online deterministic annealing algorithm is a competitive learning neural network that shows robustness with respect to the initial conditions, requires minimal hyper-parameter tuning, and provides online control over the performance complexity trade-off. We study the convergence properties of the proposed methodology and quantify its performance in simulated experiments. Finally, we show that the generated codevectors can be used as training samples for sparse and progressively more accurate Gaussian process regression.

Keywords: Randomized algorithms, Estimation
Abstract: In this letter, we address the probabilistic error quantification of a general class of prediction methods. We consider a given prediction model and show how to obtain, through a sample-based approach, a probabilistic upper bound on the absolute value of the prediction error. The proposed scheme is based on a probabilistic scaling methodology in which the number of required randomized samples is independent of the complexity of the prediction model. The methodology is extended to address the case in which the probabilistic uncertain quantification is required to be valid for every member of a finite family of predictors. We illustrate the results of the paper by means of a numerical example.

Keywords: Estimation, Identification, Linear systems
Abstract: The parameter estimation problem of a linear time-invariant fractional-order system is investigated by means of the modulating function method. Based on the assumption of known model structure and derivative orders, the modulating function method can be generalized to the fractional-order case in three different ways. We show that two approaches are identical for linear systems. This facilitates the computation of the fractional-order derivatives of modulating functions. In comparison to integer-order systems we have to include the initialization of the fractional-order system. We show that the spline type modulating function is capable of reducing the effect of the memory on the parameter estimation. However, it is not possible to compensate the memory initialization completely. In contrast to these tuning principles also the robustness against measurement noise must be considered. For this purpose we decouple the memory and noise compensation. The adjusted spline-type modulating functions reduce the initialization effect and the recursive least square estimation provides the possibility to increase the numbers of equations such that the effect of the noise is reduced.

Keywords: Estimation, Observers for nonlinear systems, Uncertain systems
Abstract: This paper presents two schemes to jointly estimate parameters and states of discrete-time nonlinear systems in the presence of bounded disturbances and noise. The parameters are assumed to belong to a known compact set. Both schemes are based on sampling the parameter space and designing a state observer for each sample. A supervisor selects one of these observers at each time instant to produce the parameter and state estimates. In the first scheme, the parameter and state estimates are guaranteed to converge within a certain margin of their true values in finite time, assuming that a sufficiently large number of observers is used and a persistence of excitation condition is satisfied in addition to other observer design conditions. This convergence margin is constituted by a part that can be chosen arbitrarily small by the user and a part that is determined by the noise levels. The second scheme exploits the convergence properties of the parameter estimate to perform subsequent zoom-ins on the parameter subspace to achieve stricter margins for a given number of observers. The strengths of both schemes are demonstrated using a numerical example.

Keywords: Estimation, Time-varying systems
Abstract: Due to the increasing popularity of cloud-based architectures, it is of paramount importance to understand how to benefit from shared information for solving collaborative estimation problems and exploit the additional computational resources available. Meanwhile, it is crucial to devise solutions that allow connected devices to retain private data and to carry out the desired tasks on their own, when disconnected from the cloud. In this paper, we present a cloud-aided iterative solution for multi-class parameter estimation for a set of mass-produced devices. The method exploits the similarity between systems operating under comparable conditions and their connection to the cloud, while allowing devices to retain and process raw data privately. The effectiveness of the strategy is assessed on a numerical example, showing its potential.

Keywords: Optimal control, Estimation
Abstract: Safely navigating a surgical tool to a desired location on the surface of the retina during retinal surgery relies on extreme precision and surgical skills. Damage to the delicate retinal tissue often occurs. Previous work demonstrated an approach for robot-assisted navigation in eye surgery using imitation learning and optimal control. To further enhance safety, we present a framework that combines real-time eye geometry estimation and chance-constrained optimal control to bound the probability for tissue damage during autonomous robotic navigation. A neural network is trained to predict the relative location of 3-D points on the retina with respect to the current tool-tip position through expert demonstrations. During inference, a local geometry of the retina is estimated using weighted least squares formulation based on these learned predictions and then employed as a probabilistic collision chance constraint in an optimal control framework. The proposed approach is experimentally validated using a phantom silicone eye suitable for vein cannulation testing. We statistically demonstrate that the network predictions become more accurate as the surgical tool approaches closer to the retinal surface and that the measured mean errors along the lateral directions for navigating to a given point or for a vessel-following task are below 0.1 mm. These results indicate that the proposed technique could serve as a basis to further develop robot-assisted retinal microsurgery with enhanced safety.

Keywords: Estimation, Optimization, Observers for Linear systems
Abstract: This paper proposes a differentiator for sampled signals with bounded noise and bounded second derivative. It is based on a linear program derived from the available sample information and requires no further tuning beyond the noise and derivative bound. A tight bound on the worst-case accuracy, i.e., the worst-case differentiation error, is derived, which is the best among all causal differentiators and is moreover shown to be obtained after a fixed number of sampling steps. Comparisons with the accuracy of existing high-gain and sliding-mode differentiators illustrate the obtained results.

Keywords: Vision-based control, Visual servo control, Estimation
Abstract: This paper presents a novel image-based visual servoing (IBVS) technique for moving targets. A target velocity estimation law is designed using dynamic surface-like filters and incorporated in the control law to compensate for the uncertainties due to the unknown target motion. Rigorous Lyapunov analysis is performed to show that the errors converge exponentially to an ultimate bound for time-varying target motion and asymptotically to zero if the target velocity is constant. The simulation results considering a realistic environment show the adaptability of the proposed scheme to track an arbitrary moving target in 3-dimension (3-D).

Keywords: Autonomous systems, Stochastic systems, Robotics
Abstract: Enforcing safety in the presence of stochastic uncertainty is a challenging problem. Traditionally, researchers have proposed safety in the statistical mean as a safety measure in this case. However, ensuring safety in the statistical mean is only reasonable if system's safe behavior in the large number of runs is of interest, which precludes the use of mean safety in practical scenarios. In this paper, we propose a risk sensitive notion of safety called conditional-value-at-risk (CVaR) safety, which is concerned with safe performance in the worst case realizations. We introduce CVaR barrier functions as a tool to enforce CVaR-safety and propose conditions for their Boolean compositions. Given a legacy controller, we show that we can design a minimally interfering CVaR-safe controller via solving difference convex programs. We elucidate the proposed method by applying it to a bipedal robot locomotion case study.

Keywords: Stochastic optimal control, Markov processes, Autonomous systems
Abstract: We consider the stochastic shortest path planning problem in MDPs, i.e., the problem of designing policies that ensure reaching a goal state from a given initial state with minimum accrued cost. In order to account for rare but important realizations of the system, we consider a nested dynamic coherent risk total cost functional rather than the conventional risk-neutral total expected cost. Under some assumptions, we show that optimal, stationary, Markovian policies exist and can be found via a special Bellman's equation. We propose a computational technique based on difference convex programs (DCPs) to find the associated value functions and therefore the risk-averse policies. A rover navigation MDP is used to illustrate the proposed methodology with conditional-value-at-risk (CVaR) and entropic-value-at-risk (EVaR) coherent risk measures.

Keywords: Autonomous robots, Autonomous systems, Stochastic systems
Abstract: Motion planning is a fundamental problem and focuses on finding control inputs that enable a robot to reach a goal region while safely avoiding obstacles. However, in many situations, the state of the system may not be known but only estimated using, for instance, a Kalman filter. This results in a novel motion planning problem where safety must be ensured in the presence of state estimation uncertainty. Previous approaches to this problem are either conserva- tive or integrate state estimates optimistically which leads to non-robust solutions. Optimistic solutions require frequent replanning to not endanger the safety of the system. We propose a new formulation to this problem with the aim to be robust to state estimation errors while not being overly conservative. In particular, we formulate a stochastic optimal control problem that contains robustified risk-aware safety constraints by incorporating robustness margins to account for state estimation errors. We propose a novel sampling-based approach that builds trees exploring the reachable space of Gaussian distributions that capture uncertainty both in state estimation and in future measurements. We provide robustness guarantees and show, both in theory and simulations, that the induced robustness margins constitute a trade-off between conservatism and robustness for planning under estimation uncertainty that allows to control the frequency of replanning.

Keywords: Stochastic optimal control, Uncertain systems, Autonomous systems
Abstract: In this paper, we study Stochastic Control Barrier Functions (SCBFs) to enable the design of probabilistic safe real-time controllers in presence of uncertainties and based on noisy measurements. Our goal is to design controllers that bound the probability of a system failure in finite-time to a given desired value. To that end, we first estimate the system states from the noisy measurements using an Extended Kalman filter, and compute confidence intervals on the filtering errors. Then, we account for filtering errors and derive sufficient conditions on the control input based on the estimated states to bound the probability that the real states of the system enter an unsafe region within a finite time interval. We show that these sufficient conditions are linear constraints on the control input, and, hence, they can be used in tractable optimization problems to achieve safety, in addition to other properties like reachability, and stability. Our approach is evaluated using a simulation of a lane-changing scenario on a highway with dense traffic.

Keywords: Autonomous systems, Robotics, Intelligent systems
Abstract: Recent trends envisage robots being deployed in areas deemed dangerous to humans, such as buildings with gas and radiation leaks. In such situations, the model of the underlying hazardous process might be unknown to the agent a priori, giving rise to the problem of planning for safe behaviour in partially known environments. We employ Gaussian process regression to create a probabilistic model of the hazardous process from local noisy samples. The result of this regression is then used by a risk metric, such as the Conditional Value-at-Risk, to reason about the safety at a certain state. The outcome is a risk functional that can be employed in optimal motion planning problems. We demonstrate the use of the proposed functional in two approaches. First is a sampling-based motion planning algorithm with an event-based trigger for online replanning. Second is an adaptation to the incremental Gaussian Process motion planner (iGPMP2), allowing it to quickly react and adapt to the environment. Both algorithms are evaluated in representative simulation scenarios, where they demonstrate the ability of avoiding high-risk areas.

Keywords: Stochastic optimal control, LMIs, Linear systems
Abstract: We study a linear-quadratic, optimal control problem on a discrete, finite time horizon with distributional ambiguity, in which the cost is assessed via Conditional Value-at-Risk (CVaR). We take steps toward deriving a scalable dynamic programming approach to upper-bound the optimal value function for this problem. This dynamic program yields a novel, tunable risk-averse control policy, which we compare to existing state-of-the-art methods.

Keywords: Mean field games, Game theory, Estimation
Abstract: We study a class of Keynesian beauty contest games where a large number of heterogeneous players attempt to estimate a common parameter based on their own observations. The players are rewarded for producing an estimate close to a certain multiplicative factor of the average decision, this factor being specific to each player. This model is motivated by scenarios arising in commodity or financial markets, where investment decisions are sometimes partly based on following a trend. We provide a method to compute Nash equilibrium within the class of affine strategies. We then develop a mean-field approximation, in the limit of an infinite number of players, which has the advantage that computing the best-response strategies only requires the knowledge of the parameter distribution of the players, rather than their actual parameters. We show that the mean-field strategies lead to an epsilon-Nash equilibrium for a system with a finite number of players. We conclude by analyzing the impact on individual behavior of changes in aggregate population behavior.

Keywords: Mean field games, Machine learning, Large-scale systems
Abstract: Multi-agent reinforcement learning methods have shown remarkable potential in solving complex multi-agent problems but mostly lack theoretical guarantees. Recently, mean field control and mean field games have been established as a tractable solution for large-scale multi-agent problems with many agents. In this work, driven by a motivating scheduling problem, we consider a discrete-time mean field control model with common environment states. We rigorously establish approximate optimality as the number of agents grows in the finite agent case and find that a dynamic programming principle holds, resulting in the existence of an optimal stationary policy. As exact solutions are difficult in general due to the resulting continuous action space of the limiting mean field Markov decision process, we apply established deep reinforcement learning methods to solve the associated mean field control problem. The performance of the learned mean field control policy is compared to typical multi-agent reinforcement learning approaches and is found to converge to the mean field performance for sufficiently many agents, verifying the obtained theoretical results and reaching competitive solutions.

Keywords: Mean field games, Modeling, Control applications
Abstract: This paper studies a large population evacuation model within the linear quadratic mean field games framework. The evacuation time horizon is fixed, and space is subdivided into regions. Depending on its initial position with respect to the specified regions, each agent has a limited selection of possible exit choices. Agents' motions are affected by their respective regional cohorts' positions mean. Regions interact through their shared exits' flows which creates an inter-regional network effect. A sufficient upper bound on the time horizon is derived to guarantee that finite escape time behavior is avoided. Besides, existence of an infinite population based Nash equilibrium is established. Finally, we illustrate, through simulations, the model's behavior for given agents' initial distributions and exits arrangement setups.

Keywords: Mean field games, Network analysis and control, Large-scale systems
Abstract: This paper studies approximate solutions to large- scale linear quadratic stochastic games with homogeneous nodal dynamics paramaters and heterogeneous network couplings based on the graphon mean field game framework in [1]–[3]. A graphon time-varying dynamical system model is first formulated to study the limit problem of linear quadratic Gaussian graphon mean field games (LQG-GMFG). The Nash equilibrium of the limit problem is then characterized by two coupled graphon time-varying dynamical systems. Based on this characterization, we establish sufficient conditions for the existence of a unique solution to the limit LQG-GMFG problem, and moreover we provide a new asymptotic error bound for the applications of the approximate solutions to finite-network finite-population games. For the computation of LQG-GMFG solutions, two methods are established and employed, where one is based on fixed point iterations and the other is based on a decoupling operator Riccati equation; furthermore, we establish correspondingly two sets of solutions based on spectral decompositions. Finally, a set of numerical simulations on networks associated with different types of graphons are demonstrated.

Keywords: Mean field games, Optimization algorithms, Optimal control
Abstract: In this work we develop an efficient numerical solution method for solving potential mean field games with multiple species. This is done by using recent developments that connect mean field games and entropy-regularized optimal transport. In particular, we reformulate the original problem as a structured entropy-regularized multi-marginal optimal transport problem, and develop highly efficient methods for solving the latter. Finally, we illustrate the proposed method on a problem with four interacting species, where each of the species has different target objectives.

Keywords: Mean field games, Stochastic systems, Markov processes
Abstract: We consider mean field games in the setting of stochastic growth where each player's capital stock is described by Cobb-Douglas production dynamics subject to stochastic depreciation and common noise. We combine both one's own utility and a relative utility to define the individual utility functions. Due to random mean field dynamics, the analysis of the best response relies on a stochastic Hamilton-Jacobi-Bellman equation, which in turn induces a special nonlinear backward SDE. We analyze this BSDE and use it to determine the solution equation system of the mean field game. We further extend the analysis to an AK model for growth dynamics.

Keywords: Optimal control, Differential-algebraic systems, Linear systems
Abstract: This paper is concerned with the linear quadratic optimal control problem for impulse controllable differential algebraic equations on a bounded half open interval. With respect to the cost functional, a general positive semi-definite weight matrix is considered in the terminal cost. It is shown that for this problem, there generally does not exist an input that minimizes the cost functional. First it is shown that the problem can be reduced to finding an input to an index-1 DAE that minimizes a different quadratic cost functional. Second, necessary and sufficient conditions in terms of matrix equations are given for the existence of an optimal control are stated.

Keywords: Neural networks, Machine learning, Optimal control
Abstract: Learning-based predictive control is a promising alternative to optimization-based MPC. However, efficiently learning the optimal control policy, the optimal value function, or the Q-function requires suitable function approximators. Often, artificial neural networks (ANN) are considered but choosing a suitable topology is also non-trivial. Against this background, it has recently been shown that tailored ANN allow, in principle, to exactly describe the optimal control policy in linear MPC by exploiting its piecewise affine structure. In this paper, we provide a similar result for representing the optimal value function and the Q-function that are both known to be piecewise quadratic for linear MPC.

Keywords: Optimal control, Distributed control, Cooperative control
Abstract: This letter considers the optimal distributed control problem for a linear stochastic multi-agent system (MAS). Due to the distributed nature of MAS network, the information available to an individual agent is limited to its vicinity. From the entire MAS aspect, this imposes the structural constraint on the control law, making the optimal control law computationally intractable. This letter attempts to relax such a structural constraint by expanding the neighboring information for each agent to the entire MAS, enabled by the distributed estimation algorithm embedded in each agent. By exploiting the estimated information, each agent is not limited to interact with its neighborhood but further establishing the ‘virtual interactions’ with the non-neighboring agents. Then the optimal distributed MAS control problem is cast as a synthesized control-estimation problem. An iterative optimization procedure is developed to find the control-estimation law, minimizing the global objective cost of MAS.

Keywords: Optimization, Optimal control
Abstract: Actuator placement is a fundamental problem in control design for large-scale networks. In this paper, we study the problem of finding a set of actuator positions by minimizing a given metric, while satisfying a structural controllability requirement and a constraint on the number of actuators. We first extend the classical forward greedy algorithm for applications to graphs that are not necessarily strongly connected. We then improve this greedy algorithm by extending its horizon. This is done by evaluating the actuator position set expansions at the further steps of the classical greedy algorithm. We prove that this new method attains a better performance, when this evaluation considers the final actuator position set. Moreover, we study the problem of minimal backup placements. The goal is to ensure that the system stays structurally controllable even when any of the selected actuators goes offline, with minimum number of backup actuators. We show that this problem is equivalent to the hitting set problem. Our results are verified by a numerical case study.

Keywords: Optimal control, Distributed parameter systems, Decentralized control
Abstract: In this paper, we consider a mean field game model inspired by crowd motion in which agents moving in a given domain aim to reach a given target set in minimal time. To model interaction between agents, we assume that the maximal speed of an agent is bounded as a function of their position and the distribution of other agents. Moreover, we assume that the state of each agent is subject to the constraint of remaining inside the domain of movement at all times, a natural constraint to model walls, columns, fences, hedges, or other kinds of physical barriers at the boundary of the domain. After recalling results on the existence of Lagrangian equilibria for these mean field games and the main difficulties in their analysis due to the presence of state constraints, we show how recent techniques allow us to characterize optimal controls and deduce that equilibria of the game satisfy a system of partial differential equations, known as the mean field game system.

Keywords: Optimal control, Distributed parameter systems, Machine learning
Abstract: A supervised learning approach for the solution of large-scale nonlinear stabilization problems is presented. A stabilizing feedback law is trained from a dataset generated from State-dependent Riccati Equation solves. The training phase is enriched by the use gradient information in the loss function, and the use of hyperparameters accounting for its importance. High-dimensional nonlinear stabilization tests demonstrate that real-time sequential large-scale Algebraic Riccati Equation solves can be substituted by a suitably trained feedforward neural networks.

Keywords: Lyapunov methods
Abstract: This paper considers the problem of approximating the “maximal” region of attraction (the set that contains all asymptotically stable sets) of any given set of locally exponentially stable nonlinear Ordinary Differential Equations (ODEs) with a sufficiently smooth vector field. Given a locally exponential stable ODE with a differentiable vector field, we show that there exists a globally Lipschitz continuous converse Lyapunov function whose 1-sublevel set is equal to the maximal region of attraction of the ODE. We then propose a sequence of d-degree Sum-of-Squares (SOS) programming problems that yields a sequence of polynomials that converges to our proposed converse Lyapunov function uniformly from above in the L1 norm. We show that each member of the sequence of 1-sublevel sets of the polynomial solutions to our proposed sequence of SOS programming problems are certifiably contained inside the maximal region of attraction of the ODE, and moreover, we show that this sequence of sublevel sets converges to the maximal region of attraction of the ODE with respect to the volume metric. We provide numerical examples of estimations of the maximal region of attraction for the Van der Pol oscillator and a three dimensional servomechanism.

Keywords: Stability of hybrid systems, Lyapunov methods, Nonlinear systems
Abstract: Soft-reset controllers are introduced as a way to approximate hard-reset controllers. The focus is on implementing reset controllers that are (strictly) passive and on analyzing their interconnection with passive plants. A passive hard-reset controller that has a strongly convex energy function can be approximated as a soft-reset controller. A hard-reset controller is a hybrid system whereas a soft reset controller corresponds to a differential inclusion, living entirely in the continuous-time domain. This feature may make soft-reset controllers easier to understand and implement. A soft-reset controller contains a parameter that can be adjusted to better approximate the action of the hard-reset controller. Closed-loop asymptotic stability is established for the interconnection of a passive soft-reset controller with a passive plant, under appropriate detectability assumptions. Several examples are used to illustrate the efficacy of soft-reset controllers.

Keywords: Lyapunov methods, Stability of nonlinear systems
Abstract: This paper addresses stochastic stabilization in case where implementation of control policies is digital, i. e., when the dynamical system is treated continuous, whereas the control actions are held constant in predefined time steps. In such a setup, special attention should be paid to the sample-to-sample behavior of the involved Lyapunov function. This paper extends on the stochastic stability results specifically to address for the sample-and-hold mode. We show that if a Markov policy stabilizes the system in a suitable sense, then it also practically stabilizes it in the sample-and-hold sense. This establishes a bridge from an idealized continuous application of the policy to its digital implementation. The central result applies to dynamical systems described by stochastic differential equations driven by the standard Brownian motion. Generalizations are discussed, including the case of non-smooth Lyapunov functions for systems driven by bounded noise. A brief overview of bounded noise models is given.

Keywords: Hierarchical control, Lyapunov methods, Stability of nonlinear systems
Abstract: In this paper, we introduce the notion of periodic safety, which requires that the system trajectories periodically visit a subset of a forward-invariant safe set, and utilize it in a multi-rate framework where a high-level planner generates a reference trajectory that is tracked by a low-level controller under input constraints. We introduce the notion of fixed-time barrier functions which is leveraged by the proposed low-level controller in a quadratic programming framework. Then, we design a model predictive control policy for high-level planning with a bound on the rate of change for the reference trajectory to guarantee that periodic safety is achieved. We demonstrate the effectiveness of the proposed strategy on a simulation example, where the proposed fixed-time stabilizing low-level controller shows successful satisfaction of control objectives, whereas an exponentially stabilizing low-level controller fails.

Keywords: Lyapunov methods, Nonlinear systems, Robust control
Abstract: This work proposes a general framework to analyze robust safety for continuous-time systems described by differential inclusions. While the existing robust-safety literature studied only what we designate by uniform robust safety, in this paper, we make a clear distinction between the uniform and the non-uniform robust-safety notions. For both cases, we establish sufficient (infinitesimal) conditions on the nominal (unperturbed) system, involving only the barrier function and the system’s righthand side. Our results allow for unbounded safety regions as well as non-smooth barrier functions. Throughout the paper, simple examples are provided to motivate the main results.

Keywords: Lyapunov methods, Nonlinear systems, Robust control
Abstract: The current paper proposes a converse robust safety theorem, in terms of barrier functions, for unconstrained differential inclusions. That is, an appropriate inequality involving a barrier function and the system’s right-hand side is shown to be a necessary-and-sufficient condition for robust safety. Our result holds true under mild regularity conditions on the system’s right-hand side and the initial and unsafe sets. In comparison to the existing converse robust-safety theorems, our result is more general as it allows the safety region to be unbounded, the system to be a differential inclusion, and the solutions to be non-unique.

Keywords: Lyapunov methods, Hybrid systems
Abstract: Our aim is to certify that the solutions starting from a specific set of initial states reach the target set without entering unsafe states. This temporal property is called safe reachability, and it has been studied in the literature for nonlinear systems using barrier functions and barrier densities [1]. In the present paper, the safe reachability result on nonlinear systems has been extended to nonlinear systems with state-dependent switching. To derive sufficient conditions for safe reachability, we consider both Carathéodory and Filippov solutions of nonlinear systems. The sum of squares method, together with Putinar Positivstellensatz, has been used to exemplify the results of the paper.

Keywords: Hybrid systems, Sampled-data control, Networked control systems
Abstract: In this paper, we design a model-based event-triggered controller for networked control of a linear time-invariant (LTI) system using a finite-time observer. Under the framework of hybrid dynamical systems, we show that, if the plant dynamics are detectable and stabilizable, then: 1) the zero error set is globally asymptotically stable and globally finite-time stable for the closed-loop system; 2) the closed-loop system is robust to small state perturbations; 3) the state of the plant converges to a neighborhood of the origin that can be made arbitrarily small; 4) the number of transmissions through the network is finite. We illustrate these results through numerical simulations.

Keywords: Hybrid systems, Stability of hybrid systems, Observers for Linear systems
Abstract: We propose a local observer design for hybrid systems with linear flow, jump and output maps, whose jump times are not known/detected. Assuming the solutions of interest admit a dwell-time and the pair of flow/output maps is observable allows us to use a sufficiently fast linear observer during flow and trigger the observer jumps when its estimate reaches the jump set. However, since, as we show, using the plant's output around the jump times is actually counterproductive, we propose to ``disconnect'' the correction term of the observer around the jump times and let the estimate flow in open-loop with the plant's flow map. Local attractivity of an appropriate zero-error set is then shown for the obtained observer and illustrated in simulations.

Keywords: Hybrid systems
Abstract: We study solution concepts and their properties for hybrid systems that can flow and jump, affected by continuous-time inputs. While the solution concepts, the existence of solutions and (forward) completeness properties are extensively discussed in the absence of (external) inputs, there are surprisingly few results when inputs are present, certainly in the case where the flow and jump sets depend on the inputs. Given the relevance of this class of hybrid systems for many applications such as hybrid or networked control for plants subject to disturbances or measurement noise, we discuss in this paper notions of solutions in the presence of inputs and show through various examples the subtleties that can occur. Moreover, we provide tools to guarantee the existence of solutions and results to establish completeness properties.

Keywords: Robotics, Hybrid systems, Optimization
Abstract: Trajectory optimization is a popular strategy for planning trajectories for robotic systems. However, many robotic tasks require changing contact conditions, which is difficult due to the hybrid nature of the dynamics. The optimal sequence and timing of these modes are typically not known ahead of time. In this work, we extend the Iterative Linear Quadratic Regulator (iLQR) method to a class of piecewise smooth hybrid dynamical systems by allowing for changing hybrid modes in the forward pass, using the saltation matrix to update the gradient information in the backwards pass, and using a reference extension to account for mode mismatch. We demonstrate these changes on a variety of hybrid systems and compare the different strategies for computing the gradients.

Keywords: Hybrid systems, Stability of nonlinear systems, Network analysis and control
Abstract: This article considers the stability analysis for a class of hybrid systems with the focus being on the frequently occurring jump dynamics. The system class is modelled as a singularly perturbed hybrid system where the singular perturbation parameter governs the frequency of jumps. Consequently, this results in a quasi steady-state system modeled by a differential equation without any jumps, and the boundary-layer system described by purely discrete dynamics. By imposing appropriate assumptions on the quasi steady-state system and the boundary-layer system, we derive results showing practical convergence to a compact attractor when the jumps occur frequently often. Our system class is motivated by the control design problem in a network of second-order continuous-time coupled oscillators, where each agent communicates the information about its position to the neighbors at discrete times. As a corollary to our main result, we show that if the information exchange between the agents and their neighbors is frequent enough, then the oscillators achieve practical consensus.

Keywords: Nonlinear systems, Constrained control, Optimization algorithms
Abstract: This paper proposes a duality approach to guarantee set invariance for nonlinear dynamical systems. Building from the so-called mirror descent algorithm from the optimization literature, we develop a new version of the given nonlinear system such that the desired set is forward (pre-)invariant. Such new version of the model is constructed using duality between the given system — called the primal system — and a new system — called the dual system. By appropriately mapping the dual system back to the original space, the resulting system — called the modified primal system — has the desired set forward preinvariant. The power of the approach is illustrated in several applications pertaining to constrained optimization and feedback control under constraints. Academic examples are provided to illustrate the approach and utility of the results.

Keywords: Constrained control, Resilient Control Systems, Networked control systems
Abstract: We present a strategy for failure resilient cloud control using model predictive control (MPC) extended with explicit recovery. Based on an arbitrary and unmodified device controller, the remotely operated MPC can safely manipulate the network controlled plant through temporary adjustment of an error signal generator. We show ways to implement the reliable cloud controller, relate it to two-degrees-of-freedom control and robust MPC, and determine stability. Simulations illustrate the obtained performance and resilience to failure. In the conclusion, we elaborate on these results and the concept of elastic control.

Keywords: Constrained control, Variable-structure/sliding-mode control, Robotics
Abstract: In this paper robust constrained control of nonlinear systems that have relative degree two with respect to the control variable is considered. The first time derivative of the control variable is assumed to be the constrained variable. The developed control approach is based on sliding mode control design. It places sliding manifolds below the bounds of the constraints so that the constrained variable will be forced to stay in the admissible region if it approaches a bound. For the proposed control method it is analytically shown that the constrained control problem can be solved. This includes consideration of time-varying behavior of the bounds. Velocity-constrained control of a two-link robot is considered as a numerical example.

Keywords: Stability of linear systems, Lyapunov methods, Constrained control
Abstract: This paper investigates the problem of global regulation for a class of planar systems with uncertain measurements in the presence of control input saturation. By taking into account the saturation nonlinearity at the outset of the controller design, a saturated integral controller is proposed in a very simple form to regulate the planar system with uncertain measurements. Moreover, the problem of tracking a ramp signal with unknown slope and the extension of the case for measurement function with a generalized unknown form are also presented. A novel Lyapunov based stability proof is provided and the simulation studies demonstrate the effectiveness of the proposed method.

Keywords: Nonlinear systems, Constrained control, Chaotic systems
Abstract: The purpose of this paper is to devise an event-triggered control strategy for the master-slave synchronization of Lur’e-type discrete-time systems with input saturation. From Lyapunov theory, conditions in the form of linear matrix inequalities are derived in order to ensure regional (or global, when possible) asymptotic synchronization of the slave system. Considering an a priori given stabilizing synchonization error feedback controller, the event generator design is addressed in two scenarios for the system nonlinearity: a genenal sector-bounded and a piecewise-linear function. Aiming at reducing the number of events (control updates), convex optimization problems are proposed to determine the triggering function parameters. A numerical example demonstrates the application of the proposed method.

Keywords: Fault tolerant systems, Constrained control, Optimal control
Abstract: This paper proposes a set-theoretic Failure Mode and Effect Management (FMEM) strategy that handles stuck/jammed actuators and enforces pointwise-in-time state and control constraints. The approach exploits nesting between constraint admissible and recoverable sets to ensure the existence of a recovery sequence. A reference governor is applied to track reference commands, while imposing constraint satisfaction using the remaining working actuators. Numerical results of an aircraft longitudinal flight application are reported.

Keywords: Observers for nonlinear systems, Optimization, Optimization algorithms
Abstract: The problem of designing an algorithm which computes the left inverse of an injective immersion is studied. It is first shown how standard Newton and gradient descent algorithms fail to work when no assumption is made on the image set of the map to be inverted. In particular, solutions might converge to the wrong point or diverge in finite/infinite time even when the map is a (global) diffeomorphism. Inspired by recent results concerning the implementation of observers in given coordinates, we investigate possible means to modify the given map in order to guarantee completeness and convergence of those algorithms.

Keywords: Observers for nonlinear systems, Nonlinear systems, Stability of nonlinear systems
Abstract: This paper proposes a modification of the multi high-gain observer (MHGO) that uses multiple low-power observers with gains powered up to the order of 2 instead of n, regardless of the order of the system. It is proved that the same features of MHGO, like the existence of ideal state estimation, stability and peaking reduction, remain valid for low power MHGO. Numerical simulation demonstrates the properties of the MHGO design. It also shows reduced sensitivity to high-frequency measurement noise.

Keywords: Observers for nonlinear systems, Robotics, Estimation
Abstract: In this paper we propose a novel observer to solve the problem of visual simultaneous localization and mapping (SLAM), only using the information from a single monocular camera and an inertial measurement unit (IMU). The system state evolves on the manifold SE(3)*R^3n, on which we design dynamic extensions carefully in order to generate an invariant foliation, such that the problem is reformulated into online constant parameter identification. Then, following the recently introduced parameter estimation-based observer (PEBO) and the dynamic regressor extension and mixing (DREM) procedure, we provide a new simple solution. A notable merit is that the proposed observer guarantees almost global asymptotic stability requiring neither persistency of excitation nor uniform complete observability, which, however, are widely adopted in most existing works with guaranteed stability.

Keywords: Observers for nonlinear systems, Time-varying systems, Uncertain systems
Abstract: A novel hybrid differentiator is proposed for any time-varying signal, whose second derivative is uniformly bounded. The exact real-time differentiation is obtained in prescribed time, and it is based on the robust observer design for the perturbed double integrator. The proposed observer strategy is in successive applications of rescaled and standard supertwisting observers with finite (time-varying and respectively constant) gains. The former observer aims to nullify the observation error dynamics in prescribed time whereas the latter observer is to extend desired robustness features to the infinite horizon. The resulting real-time differentiator uses the current signal measurement only and inherits the observer features of robust convergence to the estimated signal derivative in prescribed time regardless of the initial differentiator state. Tuning rules to achieve the exact signal differentiation in prescribed time are explicitly derived. Theoretical results are supported by an experimental study of the exact prescribed-time velocity estimation of an oscillating pendulum, operating under uniformly bounded disturbances. The developed approach is additionally discussed to admit an extension to successive arbitrary-order differentiation.

Keywords: Observers for nonlinear systems, Uncertain systems
Abstract: For a class of nonlinear continuous-time systems with continuous measurements of the output, we provide new reduced order observers that converge in finite time. The convergence time is independent of the initial state. For cases where the measurements are discrete, we provide asymptotically converging observers, whose rate of convergence is proportional to the negative of the logarithm of the size of the sampling interval. Our observers are based on the observability Gramian.

Keywords: Nonlinear systems, Uncertain systems, Observers for nonlinear systems
Abstract: We propose a framework for designing global observers for nonlinear systems with disturbances under geometric conditions based on orbital symmetries. Under some additional restrictions these orbital symmetry-based conditions boil down to geometric homogeneity (at infinity) conditions. Our observers are the result of the combination of a first filter, a state norm estimator, with a second filter adaptively tuned by the first and when compared with the existing literature have a completely novel structure, inherited by the orbital symmetry-based conditions. The second filter adaptively exploits the properties of orbital symmetries of the system to achieve global convergence properties by steering, first, the state estimate close to the state trajectory and acting locally afterwards.

Keywords: Delay systems, Distributed parameter systems
Abstract: This paper investigates prediction-based stabilization for a class of linear systems subject to input-dependent input delay. The delay under consideration is implicitly defined by an integral relation depending on past values of the input. This situation is frequent in the process industry. We propose here to use a prediction on an horizon equal to the equilibrium value of the delay. By relying on a novel transport Partial Differential Equation (PDE) representation of the delay, encapsulating the input-dependency, we prove local exponential stabilization of the closed-loop system.

Keywords: Constrained control, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper studies the stabilization and safety problems of nonlinear time-delay systems, where time delays exist in system state and affect the controller design. Following the Razumikhin approach, we propose a novel control Lyapunov-Razumikhin function to facilitate the controller design and to achieve the stabilization objective. To ensure the safety objective, we propose a Razumikhin-type control barrier function for time-delay systems for the first time. Furthermore, the proposed Razumikhin-type control Lyapunov and barrier functions are merged such that the stabilization and safety control design can be combined to address the stabilization and safety simultaneously, which further extends the control design from the delay-free case into the time-delay case. Finally, the proposed approach is illustrated via a numerical example from mechanic systems.

Keywords: Linear systems, Delay systems, Process Control
Abstract: A distributed-delay based compensator for the Internal Model Control scheme is introduced as an alternative to the repetitive control structure to attenuate periodic disturbance with multiple harmonics. Besides the periodic signal attenuation, the scheme is derived to compensate an input time-delay of the controlled system. Next, the performance requirements on the spectral, frequency and time domain characteristics are used for an optimization-based design. The task is to minimize the H-infinity norm of the weighted sensitivity, while the spectral and structural properties comply with the constraints. The proposed design is tested on a model of hot rolling process, which motivates the research. Numerical analysis and simulation-based verification complete the presentation.

Keywords: Linear systems, Stability of linear systems, Delay systems
Abstract: This paper proposes a method for the stability analysis of a class of fractional-order systems (FOSs) with distributed delay using the cluster treatment of characteristic roots (CTCR) technique. The proposed method is important since no method analyzes the stability of linear FOSs with a distributed delay. For this purpose, we shall show that the stability of a FOS with a distributed delay is equivalent to a class of FOSs with multiple single delays. Thus, the system stability can be comprehensively declared in the parametric space of the distributed delays. Next, the procedure advanced clustering with the frequency sweeping method is employed as the first step to the CTCR to determine stable switching boundaries. Eventually, the practicality and effectiveness of the proposed approach are shown in an illustrative example.

Keywords: Stability of nonlinear systems, Delay systems
Abstract: We consider in this work a class of generalized Persidskii systems with time delays. The input-to-state stability and stabilization conditions for these nonlinear systems are introduced and checked through linear matrix inequalities. A numerical example demonstrates the efficacy of the proposed results.

Keywords: Network analysis and control, Modeling, Linear systems
Abstract: This paper proposes a network model of opinion dynamics based on both the social network structure and network centralities. The conceptual novelty in this model is that the opinion of each individual is weighted by the associated network centrality in characterizing the opinion spread on social networks. Following a degree-centrality-weighted opinion dynamics model, we provide an algorithm to partition nodes of any graph into two and multiple clusters based on opinion disagreements. Furthermore, the partition algorithm is applied to real-world social networks including the Zachary karate club network [1] and the southern woman network [2] and these application examples indirectly verify the effectiveness of the degree-centrality-weighted opinion dynamics model. Finally, properties of general centrality-weighted opinion dynamics model are established.

Keywords: Network analysis and control, Networked control systems, Distributed control
Abstract: We study the problem of resilient average consensus for multi-agent systems with misbehaving nodes.Different from the widely investigated Mean-Subsequence-Reduced-based and detection-isolation-based approaches which guarantee consensus, in this paper, we address this problem by detecting misbehaviors, mitigating corresponding impact and achieving accurate average consensus.General types of misbehaviors are considered, including deception attacks and accidental faults.We characterize the disturbances of misbehaving nodes in a distributed manner via two-hop communication information and develop a deterministic detection-compensation-based consensus (D-DCC) algorithm with a decaying fault-tolerant error bound. Considering scenarios where such information is intermittently available, a stochastic extension named S-DCC is proposed.We prove that D-DCC and S-DCC allow nodes to asymptotically achieve average consensus exactly and in expectation, respectively. Finally, extensive simulations verify the effectiveness of the proposed algorithms.

Keywords: Network analysis and control, Networked control systems, Stochastic systems
Abstract: We provide a theoretical basis and perform an empirical validation of the age-structured SIR model, a variant of the classical Susceptible-Infected-Recovered (SIR) model of epidemic propagation for the dynamics of the COVID-19 pandemic. We first establish that the differential equations that define the age-structured SIR model are the mean-field limits of a continuous-time Markov process that models epidemic spreading at the individual level. We then show that as the population size grows, the infection rate for any pair of age-groups approaches its mean-field limit at least as fast as the inverse of the population size approaches zero. Finally, we evaluate the performance of the model by using system identification to estimate the model parameters on a California COVID-19 dataset and by generating the trajectories of the agewise numbers of susceptible, infected, and recovered individuals in the state for a period of about 340 days. Our results show an excellent agreement between the generated trajectories and the observed numbers.

Keywords: Network analysis and control, Petri nets
Abstract: A class of Adversary Robust Consensus protocols is proposed and analysed. These are inherently nonlinear, distributed, continuous-time algorithms for multi-agents systems seeking to agree on a common value of a shared variable, in the presence of faulty or malicious Byzantine agents, disregarding protocol rules and communicating arbitrary possibly differing values to neighbouring agents. We adopt monotone joint-agent interactions, a general mechanism for processing locally available information and allowing cross-comparisons between state-values of multiple agents simultaneously. The topological features of the network are abstracted as a Petri Net and convergence criteria for the resulting time evolutions formulated in terms of suitable structural properties of its invariants (so called siphons). Finally, simulation results and examples/counterexamples are discussed.

Keywords: Compartmental and Positive systems, Network analysis and control, Linear systems
Abstract: Classical mathematical models of information sharing and updating in multi-agent networks use linear operators. In the paradigmatic DeGroot model, for example, agents update their states with linear combinations of their neighbors' current states. In prior work, an accelerated averaging model employing the use of memory has been suggested to accelerate convergence to a consensus state for undirected networks. There, the DeGroot update on the current states is followed by a linear combination with the previous states. We propose a modification where the DeGroot update is applied to the current and previous states and is then followed by a linear combination step. We show that this simple modification applied to undirected networks permits convergence even for periodic networks. Further, it allows for faster convergence than DeGroot and accelerated averaging models for suitable networks and model parameters.

Keywords: Network analysis and control, Sensor networks
Abstract: We consider a graph property known as r-robustness of the random K-out graphs. Random K-out graphs, denoted as mathbb{H}(n;K), are constructed as follows. Each of the n nodes select K distinct nodes uniformly at random, and then an edge is formed between these nodes. The orientation of the edges is ignored, resulting in an undirected graph. Random K-out graphs have been used in many applications including random (pairwise) key predistribution in wireless sensor networks, anonymous message routing in crypto-currency networks, and differentially-private federated averaging. R-robustness is an important metric in many applications where robustness of networks to disruptions is of practical interest, and r-robustness is especially useful in analyzing consensus dynamics. It was previously shown that consensus can be reached in an r-robust network for sufficiently large r even in the presence of some adversarial nodes. R-robustness is also useful for resilience against adversarial attacks or node failures since it is a stronger property than r-connectivity and thus can provide guarantees on the connectivity of the graph when up to r-1 nodes in the graph are removed. In this paper, we provide a set of conditions for K_n and n that ensure, with high probability (whp), the r-robustness of the random K-out graph.

Keywords: Network analysis and control, Algebraic/geometric methods
Abstract: In this paper, we study graphical conditions for structural controllability and accessibility of drifted bilinear systems over Lie groups. We consider a bilinear control system with drift and controlled terms that evolves over the special orthogonal group, the general linear group, and the special unitary group. Zero patterns are prescribed for the drift and controlled dynamics with respect to a set of base elements in the corresponding Lie algebra. The drift dynamics must respect a rigid zero-pattern in the sense that the drift takes values as a linear combination of base elements with strictly non-zero coefficients; the controlled dynamics are allowed to follow a free zero pattern with potentially zero coefficients in the configuration of the controlled term by linear combination of the controlled base elements. First of all, for such bilinear systems over the special orthogonal group or the special unitary group, the zero patterns are shown to be associated with two undirected or directed graphs whose connectivity and connected components ensure structural controllability/accessibility. Next, for bilinear systems over the special unitary group, we introduce two edge-colored graphs associated with the drift and controlled zero patterns, and prove structural controllability conditions related to connectivity and the number of edges of a particular color.