Keywords: Human-in-the-loop control
Abstract: In this talk, I will discuss the problem of interactive learning by discussing how we can actively learn objective functions from human feedback capturing their preferences. I will then talk about how the value alignment and reward design problem can have solutions beyond active preference-based learning by tapping into the rich context available from large language models. In the second section of the talk, I will more generally talk about the role of large pretrained models in today’s robotics and control systems. Specifically, I will present two viewpoints: 1) pretraining large models for downstream robotics tasks, and 2) finding creative ways of tapping into the rich context of large models to enable more aligned embodied AI agents. For pretraining, I will introduce Voltron, a language-informed visual representation learning approach that leverages language to ground pretrained visual representations for robotics. For leveraging large models, I will talk about a few vignettes about how we can leverage LLMs and VLMs to learn human preferences, allow for grounded social reasoning, or enable teaching humans using corrective feedback. Finally, I will conclude the talk by discussing some preliminary results on how large models can be effective pattern machines that can identify patterns in a token invariant fashion and enable pattern transformation, extrapolation, and even show some evidence of pattern optimization for solving control problems.

Keywords: Optimization
Abstract: The term dual control was introduced in the 1960s to describe the tradeoff between short term control objectives and actions to promote learning. A closely related term is the exploration-exploitation tradeoff. This lecture will review some settings where dual controllers can be designed with performance guarantees, both for practical purposes and for a more fundamental understanding of the interplay between learning and control.

The starting point will be the standard setting of linear systems optimized with respect to quadratic cost. However much of modern learning theory is developed in a discrete setting. By investigating similarities and differences between the two frameworks, we will shed light on the dual control problem and discover new promising results and directions for research.

Keywords: Optimization algorithms, LMIs, Robust control
Abstract: First-order methods provide robust and efficient solutions to large-scale optimization problems. Recent advances in the analysis and design of first-order methods have been fueled by tools from controls, including integral quadratic constraints and multipliers from robust control. Similar advances have been made in the optimization community through the (related) performance estimation framework. Together, these tools have transformed the way in which we analyze and design optimization methods. This talk will provide an overview of these tools and set the stage for the remainder of the session.

Keywords: Optimization algorithms, Robust control, LMIs
Abstract: We expose in a tutorial fashion the mechanisms which underlie the synthesis of optimization algorithms based on dynamic integral quadratic constraints. We reveal how these tools from robust control allow to design accelerated gradient descent algorithms with optimal guaranteed convergence rates by solving small-sized convex semi-definite programs. It is shown that this extends to the design of extremum controllers, with the goal to regulate the output of a general linear closed-loop system to the minimum of an objective function.

Numerical experiments illustrate that we can not only recover gradient decent and the triple momentum variant of Nesterov's accelerated first order algorithm, but also automatically synthesize optimal algorithms even if the gradient information is passed through non-trivial dynamics, such as time-delays.

Keywords: Optimization algorithms, Robust control, Lyapunov methods
Abstract: Iterative gradient-based optimization algorithms are widely used to solve difficult or large-scale optimization problems. There are many algorithms to choose from, such as gradient descent and its accelerated variants such as Polyak's Heavy Ball method or Nesterov's Fast Gradient method. It has long been observed that iterative algorithms can be viewed as dynamical systems, and more recently, as robust controllers. Here, the "uncertainty" in the dynamics is the gradient of the function being optimized. Therefore, worst-case or average-case performance can be analyzed using tools from robust control theory, such as integral quadratic constraints (IQCs). In this tutorial paper, we show how such an analysis can be carried out using an alternative Lyapunov-based approach. This approach recovers the same performance bounds as with IQCs, but with the added benefit of constructing a Lyapunov function.

Keywords: Optimization algorithms
Abstract: We consider the distributed optimization problem for a multi-agent system. Here, multiple agents cooperatively optimize an objective by sharing information through a communication network and performing computations. In this tutorial, we provide an overview of the problem, describe the structure of its algorithms, and use simulations to illustrate some algorithmic properties based on this structure.

Keywords: Optimization, Optimization algorithms, Computer-aided control design
Abstract: The Performance Estimation Problem (PEP) approach consists in computing worst-case performance bounds on optimization algorithms by solving an optimization problem: one maximizes an error criterion over all initial conditions allowed and all functions in a given class of interest. The maximal value is then a worst-case bound, and the maximizer provides an example reaching that worst case. This approach was introduced for optimization algorithms but could in principle be applied to many other contexts involving worst-case bounds. The key challenge is the representation of infinite-dimensional objects involved in these optimization problems such as functions, and complex or non-convex objects as linear operators and their powers, networks in decentralized optimization etc. This challenge can be resolved by interpolation constraints, which allow representing the effect of these objects on vectors of interest, rather than the whole object, leading to tractable finite dimensional problems. We review several recent interpolation results and their implications in obtaining of worst-case bounds via PEP.

Keywords: Optimization algorithms, Optimization, LMIs
Abstract: First-order optimization methods have attracted a lot of attention due to their practical success in many applications, including in machine learning. Obtaining convergence guarantees and worst-case performance certificates for first-order methods have become crucial for understanding ingredients underlying efficient methods and for developing new ones. However, obtaining, verifying, and proving such guarantees is often a tedious task. Therefore, a few approaches were proposed for rendering this task more systematic, and even partially automated. In addition to helping researchers finding convergence proofs, these tools provide insights on the general structures of such proofs. We aim at presenting those structures, showing how to build convergence guarantees for first-order optimization methods.

Keywords: Sampled-data control, Building and facility automation, Predictive control for linear systems
Abstract: Data-driven control can facilitate the rapid development of controllers, offering an alternative to conventional approaches. In order to maintain consistency between any known underlying physical laws and a data-driven decision-making process, preprocessing of raw data is necessary to account for measurement noise and any inconsistencies it may introduce. In this paper, we present a physics-based filter to achieve this and demonstrate its effectiveness through practical applications, using real-world datasets collected in a building on the École Polytechnique Fédérale de Lausanne (EPFL) campus. Two distinct use cases are explored: indoor temperature control and demand response bidding.

Keywords: Data driven control, Nonlinear systems, Feedback linearization
Abstract: We consider the feedback linearization problem, and contribute with a new method that can learn the linearizing controller from a library (a dictionary) of candidate functions. When the dynamics of the system is known, the method boils down to solving a set of linear equations. Remarkably, the same idea extends to the case in which the dynamics of the system is unknown and a linearizing controller must be found using experimental data. In particular, we derive a simple condition (checkable from data) to assess when the linearization property holds over the entire state space of interest and not just on the dataset used to determine the solution. We also discuss important research directions on this topic.

Keywords: Neural networks, Stability of nonlinear systems, Machine learning
Abstract: In this work, we introduce and study a class of Deep Neural Networks (DNNs) in continuous-time. The proposed architecture stems from the combination of Neural Ordinary Differential Equations (Neural ODEs) with the model structure of recently introduced Recurrent Equilibrium Networks (RENs). We show how to endow our proposed NodeRENs with contractivity and dissipativity --- crucial properties for robust learning and control. Most importantly, as for RENs, we derive parametrizations of contractive and dissipative NodeRENs which are unconstrained, hence enabling their learning for a large number of parameters. We validate the properties of NodeRENs, including the possibility of handling irregularly sampled data, in a case study in nonlinear system identification.

Keywords: Formal Verification/Synthesis, Stochastic systems, Reduced order modeling
Abstract: In this paper, we propose a new model reduction technique for linear stochastic systems that builds upon knowledge filtering and utilizes optimal Kalman filtering techniques. This new technique will reduce the dimension of the noise disturbance and will allow any controller designed for the reduced model to be refined into a controller for the original stochastic system, while preserving any specification on the output. Although initially the reduced model will be time-varying, a method will be provided with which the reduced model can become time-invariant if it satisfies some minor technical conditions. We present our theoretical findings with an example that supports the proposed framework and illustrates how model reduction and controller refinement of stochastic systems can be achieved. We finish the paper by considering specific examples to analyze both completeness with respect to controller synthesis and model order reduction with respect to the state.

Keywords: Data driven control, Stability of nonlinear systems, Learning
Abstract: A technique to design controllers for nonlinear systems from data consists of letting the controllers learn the nonlinearities, cancel them out and stabilize the closed-loop dynamics. When control and nonlinearities are unmatched, the technique leads to an approximate cancellation and local stability results are obtained. In this paper, we show that, if the system has some structure that the designer can exploit, an iterative use of the data leads to a globally stabilizing controller even when control and nonlinearities are unmatched.

Keywords: Data driven control, Predictive control for linear systems, Uncertain systems
Abstract: Model predictive control (MPC) is a control strategy widely used in industrial applications. However, its implementation typically requires a mathematical model of the system being controlled, which can be a time-consuming and expensive task. Data-driven predictive control (DDPC) methods offer an alternative approach that does not require an explicit mathematical model, but instead optimize the control policy directly from data. In this paper, we study the impact of two different regularization penalties on the closed-loop performance of a recently introduced data-driven method called γ-DDPC. Moreover, we discuss the tuning of the related coefficients in different data and noise scenarios, to provide some guidelines for the end user.

Keywords: Stochastic optimal control, Predictive control for linear systems, Optimal control
Abstract: This paper studies optimal control problems of unknown linear systems subject to stochastic disturbances of uncertain distribution. Uncertainty about the stochastic disturbances is usually described via ambiguity sets of probability measures or distributions. Typically, stochastic optimal control requires knowledge of underlying dynamics and is as such challenging. Relying on a stochastic fundamental lemma from data-driven control and on the framework of polynomial chaos expansions, we propose an approach to reformulate distributionally robust optimal control problems with ambiguity sets as uncertain conic programs in a finite-dimensional vector space. We show how to construct these programs from previously recorded data and how to relax the uncertain conic program to numerically tractable convex programs via appropriate sampling of the underlying distributions. The efficacy of our method is illustrated via a numerical example.

Keywords: Formal Verification/Synthesis, Stochastic systems, Lyapunov methods
Abstract: This paper proposes a new framework to compute finite-horizon safety guarantees for discrete-time piece-wise affine systems with stochastic noise of unknown distributions. The approach is based on a novel approach to synthesise a stochastic barrier function (SBF) from noisy data and rely on the scenario optimization theory. In particular, we show that the stochastic program to synthesize a SBF can be relaxed into a chance-constrained optimisation problem on which scenario approach theory applies. We further show that the resulting program can be reduced to a linear programming problem, thus guaranteeing efficiency. In contrast to existing approaches, this method is data efficient as it only requires the number of data to be proportional to the logarithm in the negative inverse of the confidence level and is computationally efficient due to its reduction to linear programming. The efficacy of the method is empirically evaluated on various verification benchmarks. Experiments show a significant improvement with respect to state-of-the-art, obtaining tighter certificates with a confidence that is several orders of magnitude higher.

Keywords: Stochastic systems, Robust control, Optimization
Abstract: We study stochastic dynamical systems in settings where only partial statistical information about the noise is available, e.g., in the form of a limited number of noise realizations. Such systems are particularly challenging to analyze and control, primarily due to an absence of a distributional uncertainty model which: (1) is expressive enough to capture practically relevant scenarios; (2) can be easily propagated through system maps; (3) is invariant under propagation; and (4) allows for computationally tractable control actions. In this paper, we propose to model distributional uncertainty via Optimal Transport ambiguity sets and show that such modeling choice satisfies all of the above requirements. We then specialize our results to stochastic LTI systems, and start by showing that the distributional uncertainty can be efficiently captured, with high probability, within an Optimal Transport ambiguity set on the space of noise trajectories. Then, we show that such ambiguity sets propagate exactly through the system dynamics, giving rise to stochastic tubes that contain, with high probability, all trajectories of the stochastic system. Finally, we show that the control task is very interpretable, unveiling an interesting decomposition between the roles of the feedforward and the feedback control terms. Our results are actionable and successfully applied in stochastic reachability analysis and in trajectory planning under distributional uncertainty.

Keywords: Markov processes, Statistical learning, Estimation
Abstract: Reinforcement Learning aims at identifying and evaluating efficient control policies from data. In many real-world applications, the learner is not allowed to experiment and cannot gather data in an online manner (this is the case when experimenting is expensive, risky or unethical). For such applications, the reward of a given policy (the target policy) must be estimated using historical data gathered under a different policy (the behavior policy). Most methods for this learning task, referred to as Off-Policy Evaluation (OPE), do not come with accuracy and certainty guarantees. We present a novel OPE method based on Conformal Prediction that outputs an interval containing the true reward of the target policy with a prescribed level of certainty. The main challenge in OPE stems from the distribution shift due to the discrepancies between the target and the behavior policies. We propose and empirically evaluate different ways to deal with this shift. Some of these methods yield conformalized intervals with reduced length compared to existing approaches, while maintaining the same certainty level.

Keywords: Game theory, Stochastic optimal control, Autonomous systems
Abstract: This paper addresses a continuous-time risk-minimizing two-player zero-sum stochastic differential game (SDG), in which each player aims to minimize its probability of failure. Failure occurs in the event when the state of the game enters into predefined undesirable domains, and one player's failure is the other's success. We derive a sufficient condition for this game to have a saddle-point equilibrium and show that it can be solved via a Hamilton-Jacobi-Isaacs (HJI) partial differential equation (PDE) with Dirichlet boundary condition. Under certain assumptions on the system dynamics and cost function, we establish the existence and uniqueness of the saddle-point of the game. We provide explicit expressions for the saddle-point policies which can be numerically evaluated using path integral control. This allows us to solve the game online via Monte Carlo sampling of system trajectories. We implement our control synthesis framework on two classes of risk-minimizing zero-sum SDGs: a disturbance attenuation problem and a pursuit-evasion game. Simulation studies are presented to validate the proposed control synthesis framework.

Keywords: Formal Verification/Synthesis, Neural networks, Statistical learning
Abstract: We consider data-driven reachability analysis of discrete-time stochastic dynamical systems using conformal inference. We assume that we are not provided with a symbolic representation of the stochastic dynamics, but instead have access to a dataset of K-step trajectories. The reachability problem is to construct a probabilistic flowpipe such that the probability that a K-step trajectory can violate the bounds of the flowpipe does not exceed a user-specified failure probability threshold. The key ideas in this paper are: (1) to learn a surrogate predictor model from data, (2) to perform reachability analysis using the surrogate model, and (3) to quantify the surrogate model’s incurred error using conformal inference in order to give probabilistic reachability guarantees. We focus on learning-enabled control systems with complex closed-loop dynamics that are difficult to model symbolically, but where state transition pairs can be queried, e.g., using a simulator. We demonstrate the applicability of our method on examples from the domain of learning-enabled cyber-physical systems.

Keywords: Traffic control, Smart grid, Smart cities/houses
Abstract: Electric vehicle charging stations are expected to become key players in the future sustainable power system. We propose a framework for using them to provide balancing services to the grid, by implementing charging price control laws that ensure they are able to deliver their committed balancing capacity. The control laws are based on the Coupled Traffic, Energy, and Charging (CTEC) model, incorporating electric vehicle routing and charging decisions based on the charging price and EV state of charge. Charging stations compete with each other and must ensure a certain number of charging vehicles to maintain their role of frequency containment reserves. The results demonstrate the effectiveness of the proposed pricing control scheme in maximizing charging station profits, without violating their balancing reserve capacity commitments.

Keywords: Transportation networks, Mean field games, Markov processes
Abstract: This paper studies the routing and charging behaviors of electric vehicles in a competitive ride-hailing market. When the vehicles are idle, they can choose whether to continue cruising to search for passengers, or move a charging station to recharge. The behaviors of individual vehicles are then modeled by a Markov decision process (MDP). The state transitions in the MDP model, however, depend on the aggregate vehicle flows both in service zones and at charging stations. Accordingly, the value function of each vehicle is determined by the collective behaviors of all vehicles. With the assumption of the large population, we formulate the collective routing and charging behaviors as a mean-field Markov game. We characterize the equilibrium of such a game, prove its existence, and numerically show that the competition among vehicles leads to “inefficient congestion” both in service zones and at charging stations.

Keywords: Transportation networks, Modeling, Optimization
Abstract: This paper introduces a graph-based dynamic model for electric vehicle (EV) mobility in urban areas. The model tracks EV state-of-charge (SoC) changes over time and space, along with power inputs from public charging stations (PCS). It considers driver behavior when deciding when and where to charge, accounting for factors like current SoC, distance to PCS, and charging cost. The model helps identify optimal PCS locations to enhance convenience for EV users and profitability for PCS owners. Additionally, an averaged version of the model is presented to reduce computational overhead while aiding in optimal PCS placement. Simulation results affirm the effectiveness of our model and optimization approach in identifying ideal charging station locations and enhancing EV charging infrastructure accessibility.

Keywords: Smart grid, Energy systems
Abstract: Demand response is expected to play a fundamental role in providing flexibility for balancing operations to the grid. On the other hand, the fast electrification of the transportation sector calls for new solutions to enforce safe and reliable grid operation. Here we consider an electric vehicle charging station that participates in demand response programs. The demand response program asks for a change of the charging station load profile in exchange for a monetary reward. A stochastic receding horizon scheme that exploits the charging flexibility is then designed to optimally coordinate vehicle charging. Numerical simulations show that the proposed approach ensures substantial cost reduction compared to simpler benchmarks while maintaining the computation time feasible for real-world applications.

Keywords: Transportation networks, Game theory, Learning
Abstract: With the electrification of ride-hailing fleets, there will be a need to incentivize where and when the ride-hailing vehicles should charge. In this work, we assume that a central authority wants to control the distribution of the vehicles and can do so by selecting charging prices. Since there will likely be more than one ride-hailing company in the market, we model the problem as a single-leader multiple-follower Stackelberg game. The followers, i.e., the companies, then compete about the charging resources under given prices provided by the leader. We present a learning algorithm based on the concept of contextual bandits that allows the central authority to find an efficient pricing strategy. We also show how the exploratory phase of the learning can be improved if the leader has some partial knowledge about the companies’ objective functions. The efficiency of the proposed algorithm is demonstrated in a simulated case study for the city of Shenzhen, China.

Keywords: Traffic control, Game theory, Optimization algorithms
Abstract: We study the problem of routing plug-in electric and conventional fuel vehicles on a city scale using incentives. In our model, commuters selfishly aim to minimize a local cost that combines travel time and the financial expenses of using city facilities, i.e., parking and service stations. The traffic authority can influence the commuters' routing choice via personalized discounts on parking tickets and on the energy price at service stations. We formalize the problem of optimally designing these monetary incentives to induce traffic decongestion as a large-scale bilevel game, where constraints arise at both levels due to the finite capacities of city facilities and incentives budget. Then, we develop an efficient scalable solution scheme with convergence guarantees based on BIG Hype, a recently-proposed hypergradient-based algorithm for bilevel games. Finally, we validate our approach via numerical simulations over the Anaheim's traffic network, showcasing its advantages in terms of traffic decongestion and scalability.

Keywords: Distributed control, Observers for Linear systems, Adaptive control
Abstract: This paper studies output sign consensus problem for leaderless heterogeneous linear multi-agent systems (MASs) over switching signed graphs. Established on the assumption that the communication graph is jointly eventually positive, a distributed ‘sign observer’ is proposed to estimate a virtual leader. This virtual leader is not pre-determined, but induced from the topology of the graph, the initial conditions of the agents and the structure of the ‘observer’. Based on the distributed ‘sign observer’ and output regulation method, a state feedback controller is designed to drive the output signals of the MAS to have the same sign.

Keywords: Cooperative control, Adaptive control, Distributed control
Abstract: This article addresses the coverage control problems for heterogeneous mobile sensor networks (MSNs) in an environment with unknown event density functions. In contrast to existing works, unknown heterogeneous sensing abilities of the mobile sensor network (MSN) are considered by leveraging a weighted Voronoi diagram, namely, the Power diagram. To guarantee that the time-varying Power diagram converges to that defined by the true sensing weights, an online weight learning law is designed. Moreover, to handle certain applications such as forest fire investigation or nuclear radiation leakage mapping where the density information for the events of interest is not known to the MSN, an adaptive law is presented so that the event density approximation of each sensor converges to the real one along its trajectory. In addition, a move-to-centroid control law is proposed to drive the MSN to a near-optimal coverage configuration as time goes to infinity. Finally, the effectiveness of the proposed approach is illustrated by an example.

Keywords: Cooperative control, Distributed control, Time-varying systems
Abstract: This paper studies the problem of cooperatively solving a time-varying linear equation of the form textbf{A}(t)textbf{emph{x}}(t)=textbf{emph{b}}(t), which always has a unique solution. Each agent has access to only some rows of the time-varying augmented matrix [begin{matrix} textbf{A}(t) & textbf{emph{b}}(t) end{matrix}]. We propose a distributed algorithm for solving the time-varying linear equation.The proposed distributed algorithm enforces local solutions to track local time-varying manifolds corresponding to local linear sub-equations while simultaneously reaching a consensus. This enables all local solutions to converge to the solution of the original time-varying linear equation. Finally, the effectiveness of the proposed algorithm is demonstrated through its application in a cooperative monitoring task.

Keywords: Cooperative control, Distributed control, Resilient Control Systems
Abstract: For large-scale unmanned aerial vehicle (UAV) swarms, the security of communication networks is critical. When subjected to cyberattacks, the performance of swarm systems will be significantly affected. This paper focuses on the fully distributed time-varying formation (TVF) control problem of UAV swarms under Denial-of-Service (DoS) attacks. First, the theoretical framework of the fully distributed dynamic eventtriggering TVF control protocol is introduced. Then, sufficient conditions and critical proofs are provided to demonstrate that the desired formation configuration can be achieved under the influence of DoS attacks, and Zeno behavior is eliminated. Finally, the framework of a mixed-reality swarm flight platform is presented, which includes virtual nodes and physical nodes and integrates the advantages of both simulation and physical experiments, enabling large-scale swarm experiments with less cost and higher efficiency. The formation experiment using this platform validates the efficacy of the proposed control protocol.

Keywords: Autonomous systems, Decentralized control, Formal Verification/Synthesis
Abstract: Multi-agent systems outperform single agent in complex collaborative tasks. However, in large-scale scenarios, ensuring timely information exchange during decentralized task execution remains a challenge. This work presents an online decentralized coordination scheme for multi-agent systems under complex local tasks and intermittent communication constraints. Unlike existing strategies that enforce all-time or intermittent connectivity, our approach allows agents to join or leave communication networks at aperiodic intervals, as deemed optimal by their online task execution. This scheme concurrently determines local plans and refines the commu- nication strategy, i.e., where and when to communicate as a team. A decentralized potential game is modeled among agents, for which a Nash equilibrium is generated iteratively through online local search. It guarantees local task completion and intermittent communication constraints. Extensive numerical simulations are conducted against several strong baselines.

Keywords: Autonomous systems, Hybrid systems, Optimization
Abstract: Multi-agent systems can be extremely efficient when working concurrently and collaboratively, e.g., for transportation, maintenance, search and rescue. Coordination of such teams often involves two aspects: (i) selecting appropriate sub-teams for different tasks; (ii) designing collaborative control strategies to execute these tasks. The former aspect can be combinatorial w.r.t. the team size, while the latter requires optimization over joint state-spaces under geometric and dynamic constraints. Existing work often tackles one aspect by assuming the other is given, while ignoring their close dependency. This work formulates such problems as combinatorial-hybrid optimizations (CHO), where both the discrete modes of collaboration and the continuous control parameters are optimized simultaneously and iteratively. The proposed framework consists of two interleaved layers: the dynamic formation of task coalitions and the hybrid optimization of collaborative behaviors. Overall feasibility and costs of different coalitions performing various tasks are approximated at different granularities to improve the computational efficiency. At last, a Nash-stable strategy for both task assignment and execution is derived with provable guarantee on the feasibility and quality.

Keywords: Estimation, Time-varying systems, Adaptive control
Abstract: In this paper we present a method for estimating time-varying parameters in a linear regression equation. We combine local polynomial regression with dynamic regressor extension and mixing to independently estimate the parameters. During local polynomial regression, a time-varying parameter is approximated by locally constant polynomial coefficients. We propose to use the Bernstein basis instead of the commonly used monomial basis to improve numerical conditioning. A simulation example shows that our proposed estimator has improved performance compared to a similar method and allows a higher polynomial order.

Keywords: Estimation, Uncertain systems, Linear parameter-varying systems
Abstract: In this paper, a new High-Gain Interval Observer (HGIO) structure and its filtered version, named Filtered High-Gain Interval Observer (FHGIO), are proposed for a class of Linear Parameter Varying (LPV) systems subject to additive disturbances and measurement noise. Those uncertainties are assumed to be unknown but bounded with known values. The HGIO is based on a high-gain observer structure from which an interval formulation is deduced taking into account the uncertainties bounds. Then, the proposed HGIO is extended to incorporate a filter for the output estimation error, leading to the FHGIO design whose goal is to reduce the measurement noise amplification. Usually, the design of such interval observers is based on monotone systems theory which is hard to satisfy in many cases. In this paper, suitable changes of coordinates are used to overcome this limitation. Moreover, a sufficient condition for the non-divergence of the radius dynamics and a procedure to design the observers gains ensuring the stability are given for each observer. The efficiency of the proposed observers is illustrated through a simulation on a numerical example.

Keywords: Estimation, Vision-based control, Automotive systems
Abstract: The authors propose a visual-inertial approach to estimate the body-fixed lateral velocity of motorcycles traveling along extra-urban roads. The approach comprises the following steps: First, a monocular camera takes video of the road ahead. Key features from sequential images of the road surface are ex- tracted using the Harris corner detector and matching features are identified using the Fast retina keypoint descriptor. The locations of these features on the road surface are determined using a mapping based on an intuitive ray-casting approach. Next, the feature locations on the road, the angular velocity measurements and the optical flow of the feature projection locations on the image plane are used to formulate the ego- motion of the motorcycle as a system of linear equations from which a velocity estimate is solved for using the least-squares method. Finally, this estimate is fused with readings from an inertial navigation system using a Kalman filter to produce a filtered estimate and correct integrator drift. The approach is validated against simulation data generated using BikeSim and the results are compared against state observer approaches and previously published visual-inertial approaches from the authors.

Keywords: Estimation, Kalman filtering, Statistical learning
Abstract: Gaussian processes provide a compact representation for modeling and estimating an unknown function, that can be updated as new measurements of the function are obtained. This paper extends this powerful framework to the case where the unknown function dynamically changes over time. Specifically, we assume that the function evolves according to an integro-difference equation and that the measurements are obtained locally in a spatial sense. In this setting, we will provide the expressions for the conditional mean and covariance of the process given the measurements, which results in a generalized estimation framework, for which we coined the term Dynamic Gaussian Process (DGP) estimation. This new framework generalizes both Gaussian process regression and Kalman filtering. For a broad class of kernels, described by a set of basis functions, fast implementations are provided. We illustrate the results on a numerical example, demonstrating that the method can accurately estimate an evolving continuous function, even in the presence of noisy measurements and disturbances.

Keywords: Estimation
Abstract: Dynamic state estimation, as opposed to kinematic state estimation, seeks to estimate not only the orientation of a rigid body but also its angular velocity, through Euler's equations of rotational motion. This paper demonstrates that the dynamic state estimation problem can be reformulated as estimating a probability distribution on a Lie group defined on phase space (the product space of rotation and angular momentum). The propagation equations are derived non-parametrically for the mean and covariance of the distribution. It is also shown that the equations can be approximately solved by ignoring the third and higher moments of the probability distribution. Numerical experiments show that the distribution constructed from the propagated mean and covariance fits the sample data better than an extended Kalman filter.

Keywords: Estimation, Observers for Linear systems, Machine learning
Abstract: In this article, we propose a novel moving horizon estimation method for discrete-time linear systems through transfer learning. Most moving horizon estimation designs require data from the considered systems of interest. However, practical processes might suffer from data sparsity issues, especially in a new or early operating environment. Motivated by the idea of transfer learning, this manuscript proposes a moving horizon estimation design using data from a similar but different system (i.e., source system) instead of the considered system (i.e., target system). Based on the data from the source system, we propose a novel moving horizon state estimation method for the target system and provide convergence and stability analyses. The state estimation error is upper bounded by a time-dependent sequence that is related to three types of similarities/differences between target and source systems, including initial conditions, disturbance levels, and model parameters. The effectiveness of the proposed approach is demonstrated through a numerical example.

Keywords: Game theory, Agents-based systems, Optimization
Abstract: Information design in an incomplete information game includes a designer with the goal of influencing players' actions through signals generated from a designed probability distribution so that its objective function is optimized. We consider a setting in which the designer has partial knowledge on agents' payoffs. We address the uncertainty about players' preferences by formulating a robust information design problem against the worst case payoffs. If the players have quadratic payoffs that depend on the players' actions and an unknown payoff-relevant state, and signals on the state that follow a Gaussian distribution conditional on the state realization, then the information design problem under quadratic design objectives is a semidefinite program (SDP). Specifically, we consider ellipsoid perturbations over payoff coefficients in linear-quadratic-Gaussian (LQG) games. We show that this leads to a SDP formulation. Numerical studies are carried out to identify the relation between the perturbation levels and the optimal information structures.

Keywords: Game theory, Transportation networks, Agents-based systems
Abstract: Vehicle-to-Vehicle (V2V) communication is intended to improve road safety through distributed information sharing; however, it is difficult to predict and optimize how human agents will respond to this information. In a Bayesian game, agents probabilistically adopt various types from a fixed, exogenous distribution. Agents in such models ostensibly perform Bayesian inference, which may not be a reasonable cognitive demand for most humans. To complicate matters, real-world information provided to agents is often implicitly dependent on agent behavior, meaning that the distribution of agent types is a function of the behavior of agents (i.e., the type distribution is endogenous). In this paper, we study an existing model of V2V communication, but relax it along two dimensions: first, we pose a behavior model which does not require human agents to perform Bayesian inference; second, an equilibrium model which avoids the challenging endogenous recursion. Surprisingly, we show that the simplified non-Bayesian behavior model yields the exact same equilibrium behavior as the original Bayesian model, which may lend credibility to Bayesian models. However, we also show that the endogenous type model is necessary to obtain certain informational paradoxes; these paradoxes do not appear in the simpler exogenous model. This suggests that standard Bayesian game models with fixed type distributions are not sufficient to express certain important phenomena.

Keywords: Game theory, Agents-based systems, Cooperative control
Abstract: The emergence of new communication technologies allows us to expand our understanding of distributed control and consider collaborative decision-making paradigms. With collaborative algorithms, certain local decision-making entities (or agents) are enabled to communicate with one another and collaborate on their actions to attain better system behavior. By limiting the amount of communication, these algorithms exist somewhere between centralized and fully distributed approaches. To understand the possible benefits of this inter-agent collaboration, we model a multi-agent system as a common interest game in which groups of agents can collaborate on their actions to jointly increase the system welfare. We specifically consider k-strong Nash equilibria as the emergent behavior of these systems and address how well these states approximate the system optimal, formalized by the k-strong price of anarchy ratio. Our main contributions are in generating tight bounds on the k-strong price of anarchy in finite resource allocation games as the solution to a tractable linear program. By varying k –the maximum size of a collaborative coalition– we observe exactly how much performance is gained from inter-agent collaboration. To investigate further opportunities for improvement, we generate upper bounds on the maximum attainable k-strong price of anarchy when the agents’ utility function can be designed.

Keywords: Game theory, Markov processes, Stochastic systems
Abstract: We study coordination in Markov games with asymmetric information. We consider a model where the state consists of different components, each representing the private type of each player. Players' actions depend on their private types and the public observation of past actions. The state components evolve as independent Markov processes conditioned on actions. We propose a solution concept called perfect correlated equilibrium (PCE), realized by a correlation device that observes only the public information of past actions. At time t, the device generates a prescription profile from a commonly known joint distribution, and sends each player a prescription privately before they act. Players are expected to take actions according to the prescriptions at equilibrium by evaluating the suggested prescription at the private types. We introduce "structured" PCE (sPCE), in which the correlation device generates prescriptions based on the common action history through a common belief on the state. We motivate sPCE by showing that any payoff profile induced by a general device can be induced by a structured one. We show that when the correlation device is using structured strategies, players' rationality constraints can be characterized through appropriate Markov decision processes (MDPs). Based on this characterization, we develop a backward dynamic approach, with which one can verify if a structured device is feasible, or even design a structured PCE in a backward recursive manner. Finally, we consider a specific example demonstrating how coordination can improve social welfare.

Keywords: Game theory, Transportation networks, Intelligent systems
Abstract: In this article, we study the optimal design of High Occupancy Toll (HOT) lanes. In our setup, the traffic authority determines road capacity allocation between HOT lanes and ordinary lanes, as well as the toll price charged for travelers who use the HOT lanes but do not meet the high-occupancy eligibility criteria. We build a game-theoretic model to analyze the decisions made by travelers with heterogeneous values of time and carpool disutilities, who must choose between paying or forming carpools to take the HOT lanes, or taking the ordinary lanes. Travelers' payoffs depend on the congestion cost of the lane that they take, the payment and the carpool disutilities. We provide a complete characterization of travelers' equilibrium strategies and resulting travel times for any capacity allocation and toll price. We also calibrate our model on the California Interstate highway 880 and compute the optimal capacity allocation and toll design.

Keywords: Agents-based systems, Game theory, Emerging control applications
Abstract: We consider situations in which an intermediary facilitates the interactions of one or several players with a downstream market in a game of incomplete information. Our key assumption is the presence of information asymmetries: while the intermediary has in general better (less noisy) information about the observable parameters of the game, the players have better information about their own private parameters and preferences. The intermediary seeks to influence the actions of the agents by offering side information and/or certain guarantees regarding the outcomes. For instance, farmers aim to sell their produce in a downstream market. The intermediary has access to more accurate signals regarding the downstream market (e.g. demand, prices, etc.), while the farmers are aware of their own private cost structure. The central problem is then to understand how to design contracts for exchanging information and mediating the interaction between the players and the downstream market in such a way that generates value to the players and revenue to the intermediary.

Prior work on information design with elicitation has shown that in the presence of competition between the players, the intermediary can generate value by coordinating the players' actions in such a way that reduces the negative externalities they exert on each other. In this work, we focus on the question of risk-aversion. Our first result is negative and shows that the intermediary cannot generate revenue when interacting with a single risk-neutral or risk-seeking player. We then explore how this result changes under various relaxations of the model which include altering the risk preference of the player or changing the timing of the game.

Keywords: Optimal control, Nonlinear systems, Modeling
Abstract: In this paper we propose an optimization-based control scheme, which can be used for trajectory generation or receding horizon control for system with nonlinear, but convex dynamics, and both explicit and implicit discrete time models. The scheme uses both the nonlinear model and its linearization to construct a tube containing all possible future system trajectories, and uses this tube to predict performance and ensure constraint satisfaction. The controls sequence and tube cross-sections are optimized online in a sequence of convex programs without the need of pre-computed error bounds. We prove feasibility, stability and non-conservativeness of the approach, with the series of convex programs converging to a point which is a local optimum for the original nonlinear optimal control problem. We further present how a structure-preserving model can be implemented within the approach and used to reduce the number of constraints and guarantee a structure-preserving discrete trajectory solution.

Keywords: Optimal control, Nonlinear systems
Abstract: This study introduces a method to obtain a neighboring extremal optimal control (NEOC) solution for a broad class of nonlinear systems with nonquadratic performance indices by investigating the variation to a known closed-loop optimal control law caused by small, known variations in the system parameters or in the performance index. The NEOC solution can formally be obtained by solving a linear partial differential equation similar to those arising in an iterative solution procedure for a nonlinear Hamilton-Jacobi equation. Motivated by numerical procedures for solving such an equation, we also propose a numerical algorithm based on the Galerkin algorithm that uses basis functions to solve the underlying Hamilton-Jacobi equation. This approach allows the determination of the minimum performance index as a function of both the system state and parameters and extends to allow the determination of the adjustment to an optimal control law given a small adjustment of parameters in the system or the performance index, effectively by computing the derivative of the law with respect to those parameters. The validity of the claims and theory is supported by numerical simulations.

Keywords: Optimal control, Numerical algorithms, Constrained control
Abstract: This article introduces a numerical algorithm that serves as a preliminary step toward solving continuous-time model predictive control (MPC) problems directly without explicit time-discretization. The chief ingredients of the underlying optimal control problem (OCP) are a linear time-invariant system, quadratic instantaneous and terminal cost functions, and convex path constraints. The thrust of the method involves finitely parameterizing the admissible space of control trajectories and solving the OCP satisfying the given constraints at every time instant in a tractable manner without explicit time-discretization. The ensuing OCP turns out to be a convex semi-infinite program (SIP), and some recently developed results are employed to obtain an optimal solution to this convex SIP. A numerical illustration on a benchmark model is included to show the efficacy of the algorithm.

Keywords: Optimal control, Numerical algorithms, Robotics
Abstract: Direct optimal control techniques, relying on numerical methods for constrained optimization, are typically used in trajectory planning tasks in high-dimensional spaces. However, general-purpose solvers often fail to find a feasible solution when facing cluttered environments. Sampling- or graph-based methods, instead, can explore complex configuration spaces but struggle with dynamic constraints. Here, we propose to combine dynamic programming (DP) and derivative-based methods to reliably solve trajectory planning problems. Specifically, we exploit DP to generate a sequence of waypoints in a low-dimensional space, which are then encoded as pointwise path constraints for a high-dimensional trajectory, whose constraint violations are then represented as a penalty within the Bellman equation to recompute the waypoints. This iterative approach, alternating path and trajectory optimization, avoids both the curse of dimensionality for DP and problematic nonconvexities (such as obstacles) for motion planning. We demonstrate our strategy using numerical experiments on a six-degree-of-freedom robotic manipulator moving in a confined space.

Keywords: Optimal control, Numerical algorithms, Variational methods
Abstract: Retractions maps are used to define a discretization of the tangent bundle of the configuration manifold as two copies of the configuration manifold where the dynamics take place. Such discretization maps can be conveniently lifted to a higher-order tangent bundle to construct geometric integrators for the higher-order Euler-Lagrange equations. Given a cost function, an optimal control problem for fully actuated mechanical systems can be understood as a higher-order variational problem. In this paper we introduce the notion of a higher-order discretization map associated with a retraction map to construct geometric integrators for the optimal control of mechanical systems. In particular, we study applications to path planning for obstacle avoidance of a planar rigid body.

Keywords: Optimal control, Optimization, Aerospace
Abstract: This paper proposes an algorithm that solves non-convex optimal control problems with a theoretical guarantee for global convergence to a feasible local solution of the original problem. The proposed algorithm extends the recently proposed successive convexification (SCvx) algorithm to address its key limitation: lack of feasibility guarantee to the original non-convex problem. The main idea of the proposed algorithm is to incorporate the SCvx iteration into an algorithmic framework based on the augmented Lagrangian method to enable the feasibility guarantee while retaining favorable properties of SCvx. Unlike the original SCvx, our approach iterates on both of the optimization variables and the Lagrange multipliers, which facilitates the feasibility guarantee as well as efficient convergence, in a spirit similar to the alternating direction method of multipliers (ADMM). Convergence analysis shows the proposed algorithm's strong global convergence to a feasible local optimum of the original problem and its convergence rate. These theoretical results are demonstrated via numerical examples with comparison against the original SCvx algorithm.

Keywords: Optimization algorithms, Optimal control
Abstract: This work presents PANTR, an efficient solver for nonconvex constrained optimization problems, that is well-suited as an inner solver for an augmented Lagrangian method.The proposed scheme combines forward-backward iterations with solutions to trust-region subproblems: the former ensures global convergence, whereas the latter enables fast update directions. We discuss how the algorithm is able to exploit exact Hessian information of the smooth objective term through a linear Newton approximation, while benefiting from the structure of box-constraints or L1-regularization. An open-source C++ implementation of PANTR is made available as part of the NLP solver library ALPAQA. Finally, the effectiveness of the proposed method is demonstrated in nonlinear model predictive control applications.

Keywords: Optimization algorithms, Optimization, Machine learning
Abstract: Several recent empirical studies demonstrate that important machine learning tasks, e.g., training deep neural networks, exhibit low-rank structure, where the loss function varies significantly in only a few directions of the input space. In this paper, we leverage such low-rank structure to reduce the high computational cost of canonical gradient-based methods such as gradient descent (GD). Our proposed Low-Rank Gradient Descent (LRGD) algorithm finds an epsilon-minimizer of a p-dimensional function by first identifying r < p significant directions, and then estimating the true p-dimensional gradient at every iteration by computing directional derivatives only along those r directions. We establish that the "directional oracle complexity" of LRGD for strongly convex objective functions is O(r log(1/epsilon) + rp). Therefore, when r << p, LRGD provides significant improvement over the known complexity of O(p log(1/epsilon)) of GD in the strongly convex setting. Furthermore, using real and synthetic data, we empirically find that LRGD provides significant gains over GD when the data has low-rank structure, and in the absence of such structure, LRGD does not degrade performance compared to GD.

Keywords: Optimization algorithms, Optimization, Machine learning
Abstract: Low-rank structures have been observed in several recent empirical studies in many machine and deep learning problems, where the loss function demonstrates significant variation only in a lower dimensional subspace. While traditional gradient-based optimization algorithms are computationally costly for high-dimensional parameter spaces, such low-rank structures provide an opportunity to mitigate this cost. In this paper, we aim to leverage low-rank structures to alleviate the computational cost of first-order methods and study Adaptive Low-Rank Gradient Descent (AdaLRGD). The main idea of this method is to begin the optimization procedure in a very small subspace and gradually and adaptively augment it by including more directions. We show that for smooth and strongly convex objectives and any target accuracy epsilon, AdaLRGD's complexity is O(r ln(r/epsilon)) for some rank r no more than dimension d. This significantly improves upon gradient descent's complexity of O(d ln(1/epsilon)) when r << d. We also propose a practical implementation of AdaLRGD and demonstrate its ability to leverage existing low-rank structures in data.

Keywords: Optimization algorithms, Optimization
Abstract: We consider the setting of agents cooperatively minimizing the sum of local objectives plus a regularizer on a graph. This paper proposes a primal-dual method in consideration of three distinctive attributes of real-life multi-agent systems, namely: (i) expensive communication, (ii) lack of synchronization, and (iii) system heterogeneity. In specific, we propose a distributed asynchronous algorithm with minimal communication cost, in which users commit variable amounts of local work on their respective sub-problems. We illustrate this both theoretically and experimentally in the machine learning setting, where the agents hold private data and use a stochastic Newton method as the local solver. Under standard assumptions on Lipschitz continuous gradients and strong convexity, our analysis establishes linear convergence in expectation and characterizes the dependency of the rate on the number of local iterations. We proceed a step further to propose a simple means for tuning agents’ hyperparameters locally, so as to adjust to heterogeneity and accelerate the overall convergence. Last, we validate our proposed method on a benchmark machine learning dataset to illustrate the merits in terms of computation, communication, and run-time saving as well as adaptability to heterogeneity.

Keywords: Optimization algorithms, Optimization
Abstract: This paper presents an almost sure convergence of the zeroth-order mirror descent algorithm. The algorithm admits non-smooth convex functions and a biased oracle which only provides noisy function value at any desired point. We approximate the subgradient of the objective function using Nesterov's Gaussian Approximation (NGA) with certain alternations suggested by some practical applications. We prove an almost sure convergence of the iterates' function value to the neighbourhood of optimal function value, which can not be made arbitrarily small, a manifestation of a biased oracle. This letter ends with a concentration inequality, which is a finite time analysis that predicts the likelihood that the function value of the iterates is in the neighbourhood of the optimal value at any finite iteration.

Keywords: Optimization algorithms, Optimization
Abstract: The symplectic Stiefel manifold is a Riemannian manifold that is a generalization of the symplectic group. In this study, we propose novel conjugate gradient methods on the symplectic Stiefel manifold and compare them with the steepest descent method proposed in existing studies through numerical experiments. Although the theoretical basis of the Riemannian conjugate gradient methods has already been established, special treatment is required to address specific manifolds since these methods utilize some mappings, such as a retraction and vector transport, on the manifold. Numerical experiments demonstrate that the proposed method outperforms existing methods and is efficient.

Keywords: Machine learning, Optimization, Estimation
Abstract: We formulate the Multiple Kernel Learning (abbreviated as MKL) problem for the support vector machine with the infamous (0, 1)-loss function. Some first-order optimality conditions are given and then exploited to develop a fast ADMM solver for the nonconvex and nonsmooth optimization problem. A simple numerical experiment on synthetic planar data shows that our MKL-L_{0/1}-SVM framework could be promising.

Keywords: Machine learning, Robust control, Uncertain systems
Abstract: This paper proposes a control strategy consisting of a robust controller and an Echo State Network (ESN) based control law for stabilizing a class of uncertain nonlinear discrete-time systems subject to persistent disturbances. Firstly, the robust controller is designed to ensure that the closed-loop system is Input-to-State Stable (ISS) with a guaranteed stability region regardless of the ESN control action and exogenous disturbances. Then, the ESN based controller is trained in order to mitigate the effects of disturbances on the system output. A numerical example demonstrates the potentials of the proposed control design method.

Keywords: Machine learning, Stochastic optimal control, Stochastic systems
Abstract: The infinite horizon setting is widely adopted for problems of reinforcement learning (RL). These invariably result in stationary policies that are optimal. In many situations, finite horizon control problems are of interest and for such problems, the optimal policies are time-varying in general. Another setting that has become popular in recent times is of Constrained Reinforcement Learning, where the agent maximizes its rewards while it also aims to satisfy some given constraint criteria. However, this setting has only been studied in the context of infinite horizon MDPs where stationary policies are optimal. We present an algorithm for constrained RL in the Finite Horizon Setting where the horizon terminates after a fixed (finite) time. We use function approximation in our algorithm which is essential when the state and action spaces are large or continuous and use the policy gradient method to find the optimal policy. The optimal policy that we obtain depends on the stage and so is non-stationary in general. To the best of our knowledge, our paper presents the first policy gradient algorithm for the finite horizon setting with constraints. We show the convergence of our algorithm to an optimal policy. We also compare and analyze the performance of our algorithm through experiments and show that our algorithm performs better than other well known algorithms.

Keywords: Machine learning, Uncertain systems, Stochastic systems
Abstract: Safely controlling unknown dynamical systems is one of the biggest challenges in the field of control. Oftentimes, an approximate model of a system's dynamics exists which provides beneficial information for control design. However, differences between the approximate and true systems present challenges as well as safety concerns. We propose an algorithm called SAFESLOPE to safely evaluate points from a Gaussian process model of a function when its Lipschitz constant is unknown. We establish theoretical guarantees for the performance of SAFESLOPE and quantify how multi-fidelity modeling improves the algorithm's performance. Finally, we present a case where SAFESLOPE achieves lower cumulative regret than a naive sampling method by applying it to find the control gains of a linear time-invariant system.

Keywords: Lyapunov methods, Machine learning, Constrained control
Abstract: This paper addresses the problem of safety-critical control for non-affine control systems. It has been shown that optimizing quadratic costs subject to state and control constraints can be sub-optimally reduced to a sequence of quadratic programs (QPs) by using Control Barrier Functions (CBFs). Our recently proposed High Order CBFs (HOCBFs) can accommodate constraints of arbitrary relative degree. The main challenges in this approach are that it requires affine control dynamics and the solution of the CBF-based QP is sub-optimal since it is solved point-wise. To address these challenges, we incorporate higher-order CBFs into neural ordinary differential equation-based learning models as differentiable CBFs to guarantee safety for non-affine control systems. The differentiable CBFs are trainable in terms of their parameters, and thus, they can address the conservativeness of CBFs such that the system state will not stay unnecessarily far away from safe set boundaries. Moreover, the imitation learning model is capable of learning complex and optimal control policies that are usually intractable online. We illustrate the effectiveness of the proposed framework on LiDAR-based autonomous driving and compare it with existing methods.

Keywords: Lyapunov methods, Machine learning, Stability of nonlinear systems
Abstract: Modern data-driven techniques have rapidly progressed beyond modelling and systems identification, with a growing interest in learning high-level dynamical properties of a system, such as safe-set invariance, reachability, input-to-state stability etc. In this paper, we propose a novel supervised Deep Learning technique for constructing Lyapunov certificates, by leveraging Koopman Operator theory-based numerical tools (Extended Dynamic Mode Decomposition and Generalized Laplace Analysis) to robustly and efficiently generate explicit ground truth data for training. This is in stark contrast to existing Deep Learning methods where the loss functions plainly penalize Lyapunov condition violation in the absence of labelled data for direct regression. Furthermore, our approach leads to a linear parameterization of Lyapunov candidate functions in terms of stable eigenfunctions of the Koopman operator, making them more interpretable compared to standard DNN-based architecture. We demonstrate and validate our approach numerically using 2-dimensional and 10-dimensional examples.

Keywords: Agents-based systems, Network analysis and control, Decentralized control
Abstract: Inspired by the dual attitudes theory that implicit opinions are individuals' inner evaluations affected by experience while explicit opinions are external expressions of these evaluations, we propose an asynchronous co-evolution model of dual opinions, where individuals update explicit opinion at each time step but change their implicit opinion based on their own clock. Furthermore, we introduce the after-effect of observed opinion information in this model, which enables individuals to update implicit opinions not only based on the opinion information observed at the current time but also on the information received from the past period of time. We analyze the dynamics of dual opinions in two discussion scenarios: a group of individuals with similar and opposite initial opinions. In the former scenario, rigorous analysis suggests that dual opinions are polarized to extreme opinions, mathematically verifying the empirical finding that group discussion intensifies individuals' preferences, resulting in group polarization. In the latter scenario, our investigation shows that individuals with low bias show acceptance (inward conformity) while those with high bias exhibit compliance (outward conformity). We further analyze the influence of parameters on the co-evolution of dual opinions.

Keywords: Agents-based systems, Network analysis and control, Modeling
Abstract: In this paper we propose an extended version of the Friedkin-Johnsen (FJ) model that accounts for the effects of homophily mechanisms on the agents’ mutual appraisals. The proposed model consists of two difference equations. The first one describes the opinions’ evolution, namely how agents modify their opinions taking into account both their personal beliefs and the influences of other agents, as in the standard FJ model. Meanwhile, the second equation models how the influence matrix involved in the opinion formation process updates according to a homophily mechanism, by allowing both positive and negative appraisals. We derive necessary and sufficient conditions for the proposed time-varying version of the classical FJ model to asymptotically converge to a constant solution. In the case of a single discussion topic, asymptotic convergence is always ensured and the limit behavior of the system is derived in closed form.

Keywords: Agents-based systems, Network analysis and control
Abstract: This work addresses multi-agent consensus networks where adverse attackers affect the convergence performances of the protocol by manipulating the edge weights. We generalize [1] and provide guarantees on the agents’ agreement in the presence of attacks on multiple links in the network. A stability analysis is conducted to show the robustness to channel tampering in the scenario where part of the codeword, corresponding to the value of the edge weights, is corrupted. Exploiting the built-in objective coding, we show how to compensate the conservatism that may emerge because of multiple threats in exchange for higher encryption capabilities. Numerical examples related to semi-autonomous networks are provided.

Keywords: Agents-based systems, Networked control systems, Robust control
Abstract: An important aspect in jointly analysing networked control systems and their communication is to model the networking in a sufficiently rich but at the same time mathematically tractable way. As such, this paper improves on a recently proposed scalable approach for analysing multi-agent systems with stochastic packet loss by allowing for heterogeneous transmission probabilities and temporal correlation in the communication model. The key idea is to consider the transmission probabilities as uncertain, which facilitates the use of tools from robust control. Due to being formulated in terms of linear matrix inequalities that grow linearly with the number of agents, the result is applicable to very large multi-agent systems, which is demonstrated by numerical simulations with up to 10000 agents.

Keywords: Agents-based systems, Machine learning, Cooperative control
Abstract: A robust multi-agent reinforcement learning (MARL) algorithm using a nature actor has been shown to be effective in finding a robust Nash equilibrium (NE) of a Markov game with model uncertainty. However, since a game-size scaling increases the search space and challenges reaching the NE, the robust property of the algorithm is reduced in environments with many agents. This paper proposes an evolutionary diversity-maintaining population curriculum (EDPC) framework with a robust attention-based multi-agent deep deterministic policy gradient (RA-MADDPG) algorithm, which enables an efficient robust NE search by a structured search space expansion. In the EDPC framework, the MARL divides into several stages, and when moving on to the next stage, a population consisting of larger games is made with two parent games from the previous stage. We introduce reward-proportionate parent selection and reward-guided mutation methods to continue reinforcing superior agents and maintain the diversity of the population. Furthermore, the RA-MADDPG is used to solve the robust Markov game at each stage with nature actors with attention-based architectures. The scalability and robustness of the proposed method are evaluated for different numbers of agents and levels of model uncertainty.

Keywords: Agents-based systems, Optimization, Cooperative control
Abstract: Assignment problems and their variants are ubiquitous across resource allocation applications. While they traditionally focus on minimizing costs or maximizing utility, fairness is also an important consideration, especially for task assignment in multi-agent systems. We propose algorithms for assigning tasks to agents that consider lexicographic min-max fairness, a stronger notion of fairness than min-max fairness, which minimizes the maximum cost to any single agent. We apply our proposed approaches to both one-to-one and one-to-many assignment problems. Due to the computational challenges of one-to-many task assignments, we develop tractable approaches to achieve approximate fairness. Finally, we use the proposed methods to evaluate the trade-offs between efficiency and fairness through numerical experiments

Keywords: Cooperative control, Learning, Agents-based systems
Abstract: Achieving a connected, collision-free and time-efficient coverage in unknown environments is challenging for multi-agent systems. Particularly, agents with second-order dynamics are supposed to efficiently search and reach the optimal deployment positions over targets whose distribution is unknown, while preserving the distributed connectivity and avoiding collision. In this paper, a safe reinforcement learning based shield method is proposed for unknown environment exploration while correcting actions of agents for safety guarantee and avoiding invalid samples into policy updating. The shield is achieved distributively by a control barrier function and its validity is proved in theory. Moreover, policies of the optimal coverage are centrally learned via reward engineering and executed distributively. Numerical results show that the proposed approach not only achieves zero safety violations during training, but also speeds up the convergence of learning.

Keywords: Cooperative control, Optimization, Autonomous vehicles
Abstract: A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant framework for cooperative lane-change maneuvers for an arbitrary number of CAVs on highways that aims to interrupt traffic flow as minimally as possible. Moreover, we explicitly impose feasibility constraints in the optimization formulation by using reachability set theory, leading to a unified design that removes the need for an iterative procedure used in prior work. We quantitatively evaluate the effectiveness of our framework and compare it against previously offered approaches in terms of maneuver time and incurred throughput disruption.

Keywords: Cooperative control, Optimization, Networked control systems
Abstract: This paper considers a cooperative decision-making method for an adversarial bandit problem on open multi-agent systems. In an open multi-agent system, the network configuration changes dynamically as agents freely enter and leave the network. We propose a distributed Exp3 policy in which a group of agents exchanges the estimation of the expected reward of each arm with active neighboring agents. Then, each agent updates the probability distribution of choosing arms by combining the estimated rewards of neighboring agents. We derive a sufficient condition for a sublinear bound of a pseudo regret. The numerical example shows that active agents can cooperatively find the optimal arm by the proposed Exp3 policy algorithm.

Keywords: Cooperative control, Optimization algorithms, Autonomous vehicles
Abstract: In this paper, we present an optimization-based control strategy for coordinating multiple electric automated vehicles (AVs) in confined sites. The approach focuses on obtaining and keeping energy-efficient driving profiles for the AVs while avoiding collisions in cross-intersections, narrow roads, and merge crossings. Specifically, the approach is composed of two optimization-based components. The first component obtains the energy-efficient profiles for each individual AV by solving a Nonlinear Program (NLP) for the vehicle's complete mission route. The conflict resolution, which is performed by the second component, is accomplished by solving a time-scheduling Mixed Integer Linear Programming (MILP) problem that exploits the application characteristics. We demonstrate the performance of the algorithm through a non-trivial comparative simulation example with an alternative optimization-based heuristic.

Keywords: Cooperative control, Optimization algorithms, Networked control systems
Abstract: In this paper, we present cooperative rebalancing control of a one-way car-sharing service, where several service providers operate vehicles independently while sharing the common rental stations. The objective of service providers is to reduce the number of deadhead vehicles considering limited parking slots at stations. To this end, we propose a rebalancing control method by a distributed dual decomposition algorithm. Each provider transmits the estimation of the dual optimizers to the neighboring providers in an event-triggered manner. A numerical example shows that all service providers can find an optimal rebalancing solution while effectively reducing the number of communications.

Keywords: Cooperative control, Smart grid, Distributed control
Abstract: Power sharing and voltage regulation are fundamental but conflict control objectives of DC microgrids. This paper presents a distributed control strategy to achieve adjustment of the control objectives from accurate power sharing to accurate voltage regulation. At the same time, the bus voltage of critical node is regulated to reach the rated value of the DC microgrid. Based on this control strategy, steady state characteristics of the closed-loop system are analyzed. For a given DC microgrid, the proposed control method is verified experimentally.

Keywords: Networked control systems, Agents-based systems, Cooperative control
Abstract: A recent paper has shown that linear oscillators can be synchronised only if their parameters are exactly the same. The main reason for this sensitivity lies in the fact that the oscillator network loses energy whenever the oscillators do not follow precisely the same output trajectory. This paper deals with the question how to extend oscillators to make them synchronisable in a practical sense. The oscillators are equipped with an energy source that replaces the energy lost during the synchronisation. It is shown that a power supply rate exists such that oscillators with different eigenfrequencies can phase lock, which means that they follow the same sinusoidal trajectory with some phase gap. For two coupled oscillators explicit relations for the frequency and the phase shift of the synchronous behaviour are derived.

Keywords: Networked control systems, Communication networks, Optimization
Abstract: Networked Controlled Systems (NCS) are control systems that rely on the performance of the communications to ensure a desired Quality-of-Control (QoC). However, the wireless link is imperfect; the packet has an intrinsic latency, and packets can be lost due to its stochastic nature. Although the newest generation of wireless networks (5G and beyond) can provide Ultra-Reliable Low Latency Communications (URLLC) to attempt to remove the problems caused by the wireless network. This methodology is expensive regarding communications resources, and for NCS, the constant stream of data could be spared since some updates might contain similar information. Although there are multiple methodologies to design a NCS, not many methods attempt to develop the communications system based on reducing the consumption of communications resources. Therefore, this work finds the maximum transmission interval and delay to optimize the maximum peak Age-of-Information (AoI) while getting a model of the Maximum Allowable Packet Loss Probability (MAPLP) for the case that the H∞ norm of the control system must be maintained lower than a specified threshold. Finally, the model is validated in the case of platooning using Cooperative Adaptive Cruise Control(CACC), showing a high accuracy compared to the results of the solution of the optimization problem.

Keywords: Networked control systems, Constrained control, Optimization algorithms
Abstract: Communication scheduling is needed when control loops of several safety-critical systems are closed through a shared communication medium. To enable schedulability, control for each system is designed primarily to minimize its communication demand. In this paper, we study communication demand minimization for a class of perturbed multi-agent networked control systems with a shared communication medium and subject to input and coupled state constraints. First, a framework to design communication schedule and control is recalled such that state and input constraints are satisfied under no coupling assumption. Then, a heuristic method is proposed to decouple state constraints such that the overall communication demand of the systems is minimized. Effectiveness of the proposed results are illustrated through a numerical example.

Keywords: Networked control systems, Control of networks, Resilient Control Systems
Abstract: System performance of distributed control systems and networked computing systems is strongly dependent of the underlying communication topology. This paper considers the rarely studied problem of how the topology can maintain resilience by reconfiguration in case that agents leave or join the network during online operation. Existing optimization-based approaches which reconfigure the entire network can typically not be used in this case, since the computational burden for online application is too high. Thus, this paper proposes a novel combined offline-online scheme which optimizes the topology for high convergence rate (of e.g. consensus problems) while providing guarantees for the robustness against agent failures. In the offline part, an optimization of the entire topology is carried out using novel constraints to prepare resilience of the online procedure. For the latter, the proposed scheme guarantees that robustness is maintained for joining agents and if a specified number of agents leave the network. In simulation, the proposed scheme is compared to existing approaches and the advantages of the online-offline procedure are demonstrated.

Keywords: Networked control systems, Control over communications, Constrained control
Abstract: This paper studies an N–agent cost-coupled game where the agents are connected via an unreliable capacity constrained network. Each agent receives state information over that network which loses packets with probability p. A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit C < N on the number of transmission attempts at each instant. Under a standard information structure, we show that the problem can be decoupled into a scheduling problem for the BS and a game problem for the N agents. Since the scheduling problem is an NP hard combinatorics problem, we propose an approximately optimal solution which approaches the optimal solution as N tends to infinity. In the process, we also provide some insights on the case without channel erasure. Next, to solve the large population game problem, we use the mean-field game framework to compute an approximate decentralized Nash equilibrium. Finally, we validate the theoretical results using a numerical example.

Keywords: Networked control systems, Control Systems Privacy
Abstract: We consider a cooperative Linear Quadratic Gaussian (LQG) control system, in which an individual user owns a local plant whose control inputs are provided by a server. In the cooperation, the user takes the plant states as private information and desires to maximize the privacy preservation while ensuring that the server still provides a certain level of control performance. Moreover, the user requires a privacy scheme that is used locally and is unknown to the server, so that it can create a deviation in the server’s knowledge of the states from the true value. To achieve this, we propose two privacy schemes localized at the user side, which inject perturbations in the innovation data sent to the server. For both schemes, firstly, we analyze the privacy preservation quality provided by the scheme and the performance loss in the LQG control caused by it. Secondly, based on the trade-off between them, we propose an optimization problem. Thirdly, we propose a recovery procedure by which the control performance is recovered to the optimal one, i.e., the privacy preservation is achieved without any performance loss in control. Finally, simulations are provided, and we give discussions on the two schemes based on the simulation results.

Keywords: Identification, Sampled-data control
Abstract: Fast-sampled models are essential for control design, e.g., to address intersample behavior. The aim of this paper is to develop a non-parametric identification technique for fast-sampled models of systems that have relevant dynamics and actuation above the Nyquist frequency of the sensor, such as vision-in-the-loop systems. The developed method assumes smoothness of the frequency response function, which allows to disentangle aliased components through local models over multiple frequency bands. The method identifies fast-sampled models of slowly-sampled systems accurately in a single identification experiment. Finally, an experimental example demonstrates the effectiveness of the technique.

Keywords: Identification, Statistical learning, Machine learning
Abstract: Applying regularization in reproducing kernel Hilbert spaces has been successful in linear system identification using stable kernel designs. From a Gaussian process perspective, it automatically provides probabilistic error bounds for the identified models from the posterior covariance, which are useful in robust and stochastic control. However, the error bounds require knowledge of the true hyperparameters in the kernel design. They can be inaccurate with estimated hyperparameters for lightly damped systems or in the presence of high noise. In this work, we provide reliable quantification of the estimation error when the hyperparameters are unknown. The bounds are obtained by first constructing a high-probability set for the true hyperparameters from the marginal likelihood function. Then the worst-case posterior covariance is found within the set. The proposed bound is proven to contain the true model with a high probability and its validity is demonstrated in numerical simulation.

Keywords: Identification, Stochastic systems
Abstract: We adopt a Bayesian perspective to identify the unknown parameters of linear stochastic systems with possibly non-Gaussian disturbance distributions. The key idea of our algorithm is to alternately execute L randomly selected linear state-feedback controllers and keep track of a maximum a posteriori estimator. The proposed algorithm asymptotically achieves the concentration of posterior distributions around the true system parameters. We also derive probabilistic bounds for the concentration based on the classical results regarding the asymptotic properties of posterior distributions. An empirical demonstration is provided as well.

Keywords: Identification, Systems biology, Nonlinear systems identification
Abstract: This work considers an important problem of identifying the dynamics of chemical reaction networks from time-series data. We propose an approach to identify complex chemical reaction networks (CRN) from concentration data using the concept of sparse model identification. Particularly, we demonstrate challenges associated with the application of the sparse identification of nonlinear dynamics (SINDy) and its variants to data obtained from CRNs. We develop a SINDy-CRN algorithm based on the properties of CRNs for identifying governing equations of a CRN. The proposed algorithm is illustrated using a numerical simulation example.

Keywords: Identification, Time-varying systems, Stochastic systems
Abstract: The problem of noncausal identification of a time-varying linear system subject to both smooth and occasional jump-type changes is considered and solved using the preestimation technique combined with the basis function approach to modeling the variability of system parameters. The proposed estimation algorithms yield very good parameter tracking results and are computationally attractive.

Keywords: Identification, Uncertain systems, Modeling
Abstract: This paper is concerned with identifying linear system dynamics without the knowledge of individual system trajectories, but from the knowledge of the system's reachable sets observed at different times. Motivated by a scenario where the reachable sets are known from partially transparent manufacturer specifications or observations of the collective behavior of adversarial agents, we aim to utilize such sets to determine the unknown system's dynamics. This paper has two contributions. Firstly, we show that the sequence of the system's reachable sets can be used to uniquely determine the system's dynamics for asymmetric input sets under some generic assumptions, regardless of the system's dimensions. We also prove the same property holds up to a sign change for two-dimensional systems where the input set is symmetric around zero. Secondly, we present an algorithm to determine these dynamics. We apply and verify the developed theory and algorithms on an unknown band-pass filter circuit solely provided the unknown system's reachable sets over a finite observation period.

Keywords: Robust adaptive control, Adaptive control, Switched systems
Abstract: Supervisory Control has been shown to be a very effective approach to adaptive control which ensures step-tracking, exponential stability, and a degree of robustness to unmodelled dynamics. Here we apply the technique in the discrete-time setting and prove a new linear-like convolution bound on the effect of the noise/disturbance. This property is then leveraged to prove robustness to slow time-variations.

Keywords: Robust adaptive control, Indirect adaptive control, Adaptive control
Abstract: This work presents a new sufficient condition for synthesizing nonlinear controllers that yield bounded closed-loop tracking error transients despite the presence of unmatched uncertainties that are concurrently being learned online. The approach utilizes contraction theory and addresses fundamental limitations of existing approaches by allowing the contraction metric to depend on the unknown model parameters. This allows the controller to incorporate new model estimates generated online without sacrificing its strong convergence and bounded transients guarantees. The approach is specifically designed for trajectory tracking so the approach is more broadly applicable to adaptive model predictive control as well. Simulation results on a nonlinear system with unmatched uncertainties demonstrates the approach.

Keywords: Robust adaptive control, Output regulation, Adaptive control
Abstract: It is well-known in adaptive control that when regressors are not persistently exciting (PE), then parameter adaptation is not robust. A number of adhoc modifications of parameter adaptation laws were developed to overcome this problem. In this paper we examine the PE subspace, a geometric characterization of a regressor’s excitation, which allows a more intrinsic modification of parameter adaptation laws. Our modular method, the mu-modification, is premised on the Use it or Lose it Principle of neuroplasticity, stating that parameters not excited by a regressor may be forgotten. This paper develops these ideas in the context of adaptive output regulation, with attention to the geometric properties of the PE subspace under linear filtering, such as when using augmented errors.

Keywords: Robust adaptive control, Predictive control for linear systems, Neural networks
Abstract: This paper presents a deep learning based model predictive control (MPC) algorithm for systems with unmatched and bounded state-action dependent uncertainties of unknown structure. We utilize a deep neural network (DNN) as an oracle in the underlying optimization problem of learning based MPC (LBMPC) to estimate unmatched uncertainties. Generally, DNNs as oracle are considered difficult to employ with LBMPC due to the technical difficulties associated with the estimation of their coefficients in real time. We employ a dual-timescale adaptation mechanism, where the weights of the last layer of the neural network are updated in real time while the inner layers are trained on a slower timescale using the training data collected online and selectively stored in a buffer. Our results are validated through a numerical experiment on the compression system model of a jet engine. These results indicate that the proposed approach is implementable in real time and carries the theoretical guarantees of LBMPC.

Keywords: Adaptive control, Neural networks, Flight control
Abstract: In this paper, we propose an adaptive prescribed-time control algorithm for the fixed-wing unmanned aerial vehicle (UAV). How to follow the desired trajectory within a predetermined time is a problem worth investigating in fixed-wing UAV tracking missions. To this end, a novel method based on time-varying state feedback and segmented neural network (SNN) is proposed, using practice prescribed-time input-to-state stable to guarantee the convergence of all signals in the prescribed time. Considering the input saturation and state constraints, we give the basis for selecting the prescribed time with different initial conditions, rather than an arbitrary one. Finally, the simulation shows that the proposed method can realize prescribed-time tracking control with input saturation, despite large initial states, and the magnitude of the control changes moderately.

Keywords: Adaptive control, Automotive control, Learning
Abstract: This paper investigates synchronized optimization with prescribed performance for the strict-feedback system with time-synchronized convergence property, which is the highly essential performance desired in various real-world high-precision control applications. The prescribed performance is considered to keep the state-variables within a predefined region during the control period to meet the required system performance. To consider optimization performance while also concurrently attaining the time-synchronized properties simultaneously of each backstepping subsystem, optimized backstepping is utilized to establish the learning framework; wherein the norm-normalized sign function is appropriately incorporated in each backstepping subsystem, which generates the decomposition of the optimal system control and gradient term of the cost function with appropriate time-synchronized control items and unknown independently learning parts to be approximated with neural networks. With this decomposition design, the learning objective is transformed to adaptively explore the optimal control parameter in the admissible policy region. By additionally employing the adaptive dynamic programming technique, actor-critic method, and gradient-constrained method, the solution of the Hamilton-Jacobi-Bellman equation is iteratively approximated while the learnable parameter stays within the predefined region. The work here has the outcome of time-synchronized convergence which surpasses the usual typical developments in this class of problems considered. The proposed method is verified with the vehicle platoon problem to show its effectiveness in that the system preserves special properties of time-synchronized stability and control while optimizing the overall system control.

Keywords: Power electronics, Stability of hybrid systems, Distributed control
Abstract: We analyze a self-interleaving circuitry for driving signals in multicellular converters based on an interconnection graph with a ring topology that induces desirable fault-tolerant features. Using nonlinear hybrid dynamical tools, we show that the dynamics of this electronic solution can be formulated as a system with a sampled-data feedback law emulating a first-order Kuramoto-like model. For this Kuramoto model, under general conditions on the coupling functions, we provide a Lyapunov-based proof of local asymptotic stability of the splay-state (interleaved) configuration. We then illustrate the relation with the emulation-based sampled-data scenario via simulation results.

Keywords: Power electronics, Switched systems, Observers for nonlinear systems
Abstract: In this paper, an observer-based switched control law is proposed for the three level neutral point clamped (NPC) converter operating as a rectifier. Modeling the converter as a switched affine system, the proposed control is based on the well known argmin control law to track a varying state reference trajectory. A full-order observer is introduced to compute the control law with only the measure of the input and the output voltages. The control aims at tracking a state reference defined from a power analysis and three objectives are addressed: to stabilize the output at a given DC voltage, to ensure a unit power factor by having the input current and voltage on phase and to have balanced capacitor voltages on the output. Based on a unified modeling methodology, the control and the observer are easily derived from LMI conditions. An outer loop is added to regulate the output when constant perturbations are considered. The results are illustrated by simulations on MATLAB/Simulink.

Keywords: Power generation, Predictive control for nonlinear systems, Optimal control
Abstract: In this paper, an offset-free bilinear model predictive control approach for wind turbines is presented. State-of-the-art controllers employ different control loops for pitch angle and generator torque which switch depending on wind conditions. In contrast, the presented controller is based on one unified control law that works for all wind conditions. The inherent nonlinearity of wind turbines is addressed through a piecewise-affine approximation, which is modelled in a mixed-integer fashion. The presented controller is compared to a state-of-the-art baseline controller in a numerical case study using OpenFAST. Simulation results show that the presented controller ensures accurate reference power tracking. Additionally, damage equivalent loads are reduced for higher wind speeds.

Keywords: Power generation
Abstract: The distributed constrained optimization problem over an undirected communication topology is investigated in this study. It focuses on addressing a global coupled equality constraint that applies to all agents. To tackle this problem, a distributed approach with arbitrary initialization is developed by virtue of the aperiodic sampling control idea and the consensus-based multi-agent system(MAS) technology. This approach is developed to address constrained optimization problems within a pre-specified time. In addition, this predefined time is freely defined by users and irrelevant to the initial states, control coefficients, and network structure of systems. The Lyapunov stability theory completes the convergence proof of the developed method. Then, the developed method is extended to handle distributed nonlinear constrained optimization problems. Finally, The availability of two developed methods is demonstrated through two simulation examples.

Keywords: Power systems, Electrical machine control, Power generation
Abstract: This paper investigates the sub-synchronous oscillation occurring in high voltage direct current (HVDC)-connected permanent magnet synchronous generator (PMSG)-based offshore wind farm. To do so, firstly, a comprehensive system model is developed, which incorporates the dynamics of the PMSG-based wind energy conversion system (WECS), the ac collection grid, and the HVDC transmission system. Subsequently, small signal model of the comprehensive system is derived. Based on the small signal model, the critical system mode is obtained using modal analysis. Special attention is paid to the influence of sub-synchronous mode on the onshore grid. Moreover, the influence of controller parameters on the sub-synchronous mode is investigated, which facilitates the design of converter controllers in HVDC-integrated PMSG-based offshore wind farm. Additionally, the modal analysis results are verified through time-domain simulations.

Keywords: Power systems, Game theory, Power generation
Abstract: The proliferation of batteries, photovoltaic cells and Electric Vehicles (EVs) in electric power networks can result in network congestion. A redispatch market that allows the Distribution System Operators (DSOs) to relieve congested networks by asking the energy consumers to adjust their scheduled consumption is an alternative to upgrading network capacity. However, energy consumers can strategically increase their bids on the day-ahead market in anticipation of payouts from the redispatch market. This behaviour, which is called increase-decrease gaming, can aggravate congestion and allow the energy consumers to extract windfall profits from the DSO. In this paper, we model the increase-decrease game for large populations of energy consumers in power networks using a mean field game approach. The agents (energy consumers) maximize their individual welfare on the day-ahead market with anticipation of the redispatch market, coupled via the electricity price. We show that there exists a Nash equilibrium for this game and use an algorithm that converges to the Nash equilibrium for the infinite population case.

Keywords: Adaptive control, Optimal control, Nonlinear systems
Abstract: In this work, we formulate and solve the problem of inverse optimal adaptive prescribed performance control and consider its application to compliant actuator-driven robot manipulators. A definition and sufficient conditions for this problem are introduced and derived based on adaptive control Lyapunov function method. An auxiliary system is constructed and incorporated with prescribed performance bounds so as to design a new class of inverse optimal adaptive controllers for the control system. By exploring the links between inverse optimality and stability, it is proved that the proposed controller ensures both inverse optimality and prescribed transient performance of the control system. Above developments are illustrated via an application to robot manipulators driven by compliant actuators. The inverse optimal adaptive control problem for robot manipulators with guaranteed transient performance has not been addressed in the literature.

Keywords: Markov processes, Learning, Estimation
Abstract: Stochastic gradient Langevin dynamics (SGLD) are a useful methodology for sampling from probability distributions. This paper provides a finite sample analysis of a passive stochastic gradient Langevin dynamics algorithm (PSGLD) designed to achieve inverse reinforcement learning. By "passive", we mean that the noisy gradients available to the PSGLD algorithm (inverse learning process) are evaluated at randomly chosen points by an external stochastic gradient algorithm (forward learner). The PSGLD algorithm thus acts as a randomized sampler which recovers the cost function being optimized by this external process. Previous work has analyzed the asymptotic performance of this passive algorithm using stochastic approximation techniques; in this work we analyze the non-asymptotic performance. Specifically, we provide finite-time bounds on the 2-Wasserstein distance between the passive algorithm and its stationary measure, from which the reconstructed cost function is obtained.

Keywords: Kalman filtering, Identification, Optimal control
Abstract: This paper focuses on two inverse problems of the Kalman filter in which the process and measurement noises are correlated. The unknown covariance matrix in a stochastic system is reconstructed from observations of its posterior beliefs. For the standard inverse Kalman filtering problem, a novel duality-based formulation is proposed, where a well-defined inverse optimal control (IOC) problem is solved instead. Identifiability of the underlying model is proved, and a least squares estimator is designed that is statistically consistent. The time-invariant case using the steady-state Kalman gain is further studied. Since this inverse problem is ill-posed, a canonical class of covariance matrices is constructed, which can be uniquely identified from the dataset with asymptotic convergence. Finally, the performances of the proposed methods are illustrated by numerical examples.

Keywords: Optimal control, Autonomous systems
Abstract: This paper proposes a data-driven, iterative approach for inverse optimal control (IOC), which aims to learn the objective function of a nonlinear optimal control system given its states and inputs. The approach solves the IOC problem in a challenging situation when the system dynamics is unknown. The key idea of the proposed approach comes from the deep Koopman representation of the unknown system, which employs a deep neural network to represent observables for the Koopman operator. By assuming the objective function to be learned is parameterized as a linear combination of features with unknown weights, the proposed approach for IOC is able to achieve a Koopman representation of the unknown dynamics and the unknown weights in objective function together. Simulation is provided to verify the proposed approach.

Keywords: Robotics, Learning, Human-in-the-loop control
Abstract: Robots have been increasingly better at doing tasks for humans by learning from their feedback, but still often suffer from model misalignment due to missing or incorrectly learned features. When the features the robot needs to learn to perform its task are missing or do not generalize well to new settings, the robot will not be able to learn the task the human wants and, even worse, may learn a completely different and undesired behavior. Prior work shows how the robot can detect when its representation is missing some feature and can, thus, ask the human to be taught about the new feature; however, these works do not differentiate between features that are completely missing and those that exist but o not generalize to new environments. In the latter case, the robot would detect misalignment and simply learn a new feature, leading to an arbitrarily growing feature representation that can, in turn, lead to spurious correlations and incorrect learning down the line. In this work, we propose separating the two sources of misalignment: we propose a framework for determining whether a feature the robot needs is incorrectly learned and does not generalize to new environment setups vs. is entirely missing from the robot’s representation. Once we diagnose the source of error, we show how the human can initiate the realignment process for the model: if the feature is missing, we follow prior work for learning new features; however, if the feature exists but does not generalize, we use data augmentation to expand its training and, thus, complete the repair process. We demonstrate the proposed approach in experiments with a simulated 7DoF robot manipulator and physical human corrections.

Keywords: Chemical process control, Control applications, Neural networks
Abstract: Owing to the frequent fluctuations encountered in raw material characteristics and operational conditions in the process industry, traditional data-driven approaches prove inadequate in adapting the adjustment of operational variables. Furthermore, the potential of multi-view data, including images, audio, and sensor data, remains underexploited in industrial processes. This study proposes an operational decision-making method based on feedstock-guided multi-view actor-critic (FMAC-ODM) using multi-view data to address these issues. This method utilizes the idea of reinforcement learning (RL) for enhanced decision-making. First, the problem of optimizing operational variables is reformulated into a continuous RL problem to acquire an improved decision-making policy aligned with the current operational conditions. Subsequently, the inclusion of feedstock properties in the state space is implemented to provide essential guidance for the decision-making process. Finally, in pursuit of a comprehensive understanding and bolstering the precision of the decision-making strategy, multi-view data sourced from the industrial site is harnessed as a surrogate for human observation. The effectiveness of the proposed decision-making method is substantiated through its practical application in the industrial flotation process.

Keywords: Fluid flow systems, Network analysis and control, Nonlinear systems
Abstract: In this paper, using tools from graph theory we provide verifiable necessary and sufficient conditions for the existence of a unique hydraulic equilibrium in district heating systems of meshed topology and containing multiple heat sources. Even though numerous publications have addressed the design of efficient algorithms for numerically finding hydraulic equilibria in the general context of water distribution networks, this is not the case for the analysis of existence and uniqueness. Moreover, most of the existing work dealing with these aspects exploit the equivalence between the nonlinear algebraic equations describing the hydraulic equilibria and the KKT conditions of a suitably defined nonlinear convex optimization problem. Differently, this paper proposes necessary and sufficient graph-theoretic conditions on the actuator placement for the existence and uniqueness of a hydraulic equilibrium, independent of the actuators' control objective. An example based on a representative district heating network is considered to illustrate the key aspects of our contribution, and an explicit formulation of the steady state solution is given for the case in which pressure drops through pipes are linear with respect to the flow rate.

Keywords: Fuzzy systems, Stability of nonlinear systems, LMIs
Abstract: This paper aims to investigate the relaxed stability condition for interval type-2 Takagi--Sugeno fuzzy systems via membership-parameter matrix inequalities. The membership framework of interval type-2 fuzzy sets is structured in convex polytopes with a straightforward method. The stabilization synthesis with a non-parallel distributed compensator controller incorporating lower and upper membership functions is achieved in the sense of a matrix. Moreover, this paper introduces the relaxation strategy for the orthogonal complement, effectively reducing the number of decision variables related to the linear matrix inequalities. In conclusion, examples are presented to demonstrate the effectiveness and applicability of the proposed methods.

Keywords: Extremum seeking, Delay systems, Constrained control
Abstract: In this paper, we present a time-delay approach to gradient-based bounded extremum seeking (ES) with large measurement constant delay, for an unknown single-input static quadratic map. We assume that the extremum point and the Hessian H belong to known intervals, whereas the sign of H is known. We apply a time-delay approach to the bounded ES system and arrive at the neutral type system with a nominal linear delayed system. We present the latter system as a retarded one and employ variation of constants formula for practical stability analysis. Explicit conditions in terms of simple scalar inequalities depending on tuning parameters and delay are established to guarantee the practical stability of the bounded ES control systems. Given any delay and neighborhood of the extremum point and through the solution of the constructed inequalities, we find lower bounds on the dither period that ensures the practical stability.

Keywords: Grey-box modeling, Machine learning, Nonlinear systems identification
Abstract: A framework for dynamic system model identification from scarce and noisy data is proposed. This framework uses symbolic regression via genetic programming with a gradient-based parameter estimation step to identify a differential equation model and its parameters from available system data. The effectiveness of the method is demonstrated by identifying four synthetic systems: an ideal plug flow reactor (PFR) with an irreversible chemical reaction, an ideal continuously stirred tank reactor (CSTR) with an irreversible chemical reaction, a system described by Burgers’ Equation, and an ideal PFR with a reversible chemical reaction. The results show that this framework can identify PDE models of systems from broadly spaced and noisy data. When the data was not sufficiently rich, the framework discovered a surrogate model that described the observations in equal or fewer terms than the true system model. Additionally, the method can select relevant physics terms to describe a system from a list of candidate arguments, providing valuable models for use in controls applications.

Keywords: Observers for nonlinear systems
Abstract: A multi-observer is a bank of observers which is used for state estimation in various applications. However, it has an implementation bottleneck when a large number of observers are required for the desired estimation performance. To overcome this problem, we propose the design method of a state-sharing multi-observer for a class of nonlinear systems. The state-sharing multi-observer is a single observer that integrates a bank of observers, and its state size is independent of the number of observers. We analyze the error of the state obtained from the state-sharing multi-observer, and then show its applicability to multi-observer based algorithms such as supervisory observers and in secure state estimation.

Keywords: Energy systems, Fault tolerant systems, Robust control
Abstract: This paper deals with the design and implementation of a robust control approach for grid-tied PV systems. The main controller’s objectives are to inject the maximum available power to the grid, whilst guaranteeing power quality even under faulty conditions. To this end, a multi-loop regulator is designed for the Cascaded H-Bridge Multilevel Inverters (CHBMIs) by combining a sliding mode control strategy for maximum power point tracking and a Lyapunov approach for Power Factor Correction (PFC). Validation of the proposed approach using Matlab/ SimPowerSimscape environment confirmed the ability of the proposed approach to successfully accomplish its objectives in terms of references tracking and regulation under both matching and mismatching irradiation levels as well as faulty modes stemming from the photovoltaic (PV) panels and/or the dc-dc converters. Additionally, the proposed approach was shown to outperform conventional approaches and enable the continuous operation of the PV system under various failure modes affecting multiple PV panels and/or their associated dc-dc boost converters.

Keywords: Linear parameter-varying systems, Adaptive control
Abstract: Through the use of the Fundamental Lemma for linear systems, a direct data-driven state-feedback control synthesis method is presented for a rather general class of nonlinear (NL) systems. The core idea is to develop a data-driven representation of the so-called velocity-form, i.e., the time-difference dynamics, of the NL system, which is shown to admit a direct linear parameter-varying (LPV) representation. By applying the LPV extension of the Fundamental Lemma in this velocity domain, a state-feedback controller is directly synthesized to provide asymptotic stability and dissipativity of the velocity-form. By using realization theory, the synthesized controller is realized as a NL state-feedback law for the original unknown NL system with guarantees of universal shifted stability and dissipativity, i.e., stability and dissipativity w.r.t. any (forced) equilibrium point, of the closed-loop behavior. This is achieved by the use of a single sequence of data from the system and a predefined basis function set to span the scheduling map. The applicability of the results is demonstrated on a simulation example of an unbalanced disc.

Keywords: Linear parameter-varying systems, Algebraic/geometric methods, Identification
Abstract: The paper makes the first steps towards a behavioral theory of LPV state-space representations with an affine dependency on scheduling, by characterizing minimality of such state-space representations. It is shown that minimality is equivalent to observability, and that minimal realizations of the same behavior are isomorphic. Moreover, this isomorphism does not depend on the scheduling variable. Furthermore, controllability of behaviors is equivalent to span-reachability of their minimal state-space representations. Finally, we establish a formal relationship between minimality of LPV state-space representations with an affine dependence on scheduling and minimality of LPV state-space representations with a dynamic and meromorphic dependence on scheduling.

Keywords: Linear parameter-varying systems, Predictive control for nonlinear systems, Modeling
Abstract: This paper proposes a novel approach for modeling and controlling nonlinear systems with varying parameters. The approach introduces the use of a parameter-varying Koopman operator (PVKO) in a lifted space, which provides an efficient way to understand system behavior and design control algorithms that account for underlying dynamics and changing parameters. The PVKO builds on a conventional Koopman model by incorporating local time-invariant linear systems through interpolation within the lifted space. This paper outlines a procedure for identifying the PVKO and designing a model predictive control using the identified PVKO model. Simulation results demonstrate that the proposed approach improves model accuracy and enables predictions based on future parameter information. The feasibility and stability of the proposed control approach are analyzed, and their effectiveness is demonstrated through simulation.

Keywords: Linear parameter-varying systems, Stochastic systems, LMIs
Abstract: This paper is concerned with robust control of discrete-time linear stochastic systems with coefficient matrices given by polytopic martingales. To the best of our knowledge, this class of stochastic systems have not been dealt with as a target of control due to the absence of required theory. For such systems, we discuss the following two types of approaches for robust stabilization: One is the proposed stochastic control approach using the martingale property of the coefficient matrices, and the other is a deterministic control approach without using the information. Through theoretical and numerical comparisons of the two approaches, we demonstrate the effectiveness of the proposed stochastic control approach in the sense of conservativeness.

Keywords: Linear parameter-varying systems, Uncertain systems, Robust control
Abstract: This paper focuses on the control design and analysis for nonstationary linear parameter-varying systems with affine parameter dependence and uncertain initial conditions. The uncertain initial state and the disturbance input are allowed to reside in two separate norm balls. Convex analysis and synthesis conditions are derived, and a reachability analysis result for systems with pointwise-bounded inputs is developed, enabling the construction of ellipsoids in which the state or some output of interest lies at specified time instants. The usefulness of the proposed approach is demonstrated through an illustrative example involving a two-mass rotational system.

Keywords: Mechatronics, Linear parameter-varying systems, Neural networks
Abstract: The performance of a feedforward controller is primarily determined by the extent to which it can capture the relevant dynamics of a system. The aim of this paper is to develop an input-output linear parameter-varying (LPV) feedforward parameterization and a corresponding data-driven estimation method in which the dependency of the coefficients on the scheduling signal are learned by a neural network. The use of a neural network enables the parameterization to compensate a wide class of constant relative degree LPV systems. Efficient optimization of the neural-network-based controller is achieved through a Levenberg-Marquardt approach with analytic gradients and a pseudolinear approach generalizing Sanathanan-Koerner to the LPV case. The performance of the developed feedforward learning method is validated in a simulation study of an LPV system showing excellent performance.

Keywords: Biological systems, Systems biology, Uncertain systems
Abstract: The interaction of phase-separating systems with chemical reactions is of great interest in various contexts, from biology to material science. In biology, phase separation is thought to be the driving force behind the formation of biomolecular condensates, i.e. organelles without a membrane that are associated with cellular metabolism, stress response, and development. RNA, proteins, and small molecules participating in the formation of condensates are also involved in a variety of biochemical reactions: how do the chemical reaction dynamics influence the process of phase separation? Here we are interested in finding chemical reactions that can arrest the growth of condensates, generating stable spatial patterns of finite size (microphase separation), in contrast with the otherwise spontaneous (unstable) growth of condensates. We consider a classical continuum model for phase separation coupled to a chemical reaction network (CRN), and we seek conditions for the emergence of stable oscillations of the solution in space. Given reaction dynamics with uncertain rate constants, but known structure, we derive easily computable conditions to assess whether microphase separation is impossible, possible for some parameter values, or robustly guaranteed for all parameter values within given bounds. Our results establish a framework to evaluate which classes of CRNs favor the emergence of condensates with finite size, a question that is broadly relevant to understanding and engineering life.

Keywords: Systems biology, Biological systems, Cellular dynamics
Abstract: For over two decades, Flux Balance Analysis (FBA) has been successfully used for predicting growth rates and intracellular reaction rates in microbiological metabolism. An aspect that is often omitted from this analysis, is segregation or heterogeneity between different cells. In this work, we propose an extended FBA method to model cell size distributions in balanced growth conditions. Hereto, a mathematical description of the concept of balanced growth in terms of cell mass distribution is presented. The cell mass distribution, quantified by the Number Density Function (NDF), is affected by cell growth and cell division. An optimization program is formulated in which the NDF, average cell culture growth rate and reaction rates per cell mass are treated as optimization variables. As qualitative proof of concept, the methodology is illustrated on a core carbon model of Escherichia coli under aerobic growth conditions. This illustrates feasibility and applications of this method, while indicating some shortcomings intrinsic to the simplified biomass structuring and the time invariant approach.

Keywords: Biomolecular systems, Genetic regulatory systems, PID control
Abstract: We propose a multicellular implementation of a classical PD feedback controller to regulate gene expression in a microbial consortium. The implementation involves distributing the proportional and derivative control actions between two different cellular populations that can communicate with each other and regulate the output of a third target cellular population. We derive analytical conditions on biological parameters and control gains to adjust the system's static and dynamical properties. We then evaluate the strategy's performance and robustness through extensive in silico experiments in BSim, a realistic simulator of bacterial populations.

Keywords: Systems biology, Hybrid systems, Nonlinear systems
Abstract: Negative feedback regulation is a well-known motif for suppressing deleterious fluctuations in gene product levels. We systematically compare two scenarios where negative feedback is either implemented in the protein production rate (regulated synthesis) or in the protein degradation rate (regulated degradation). Our results show that while in low-noise regimes both schemes are identical, they begin to show remarkable differences in high-noise regimes. Analytically solving for the probability distributions of the protein levels reveals that regulated synthesis is a better strategy to suppress random fluctuations while also minimizing protein levels dipping below a threshold. In contrast, regulated degradation is preferred if the goal is to minimize protein levels going beyond a threshold. Finally, we compare and contrast these distributions not only in a single cell over time but also in an expanding cell population where these effects can be buffered or exacerbated due to the coupling between expression and cell growth.

Keywords: Systems biology, Biomolecular systems, Markov processes
Abstract: Enzyme-driven catalysis of a substrate into a product forms the fundamental backbone of cellular metabolic pathways. In the deterministic formulation of such a reaction scheme, the equilibrium level of the metabolic product is independent of the steady-state enzyme, so that any perturbation in enzyme levels causes a transient change in metabolic product levels that perfectly adapts to the original enzyme-independent steady state. In this work, we consider a stochastic formulation of the problem, where enzyme levels constantly fluctuate due to the inherently noisy gene expression process as well as to the extrinsic noise in substrate availability. Our results show that such out-of-equilibrium fluctuations can result in positive (or negative) enzyme-product and substrate-product correlations, whose behavior qualitatively and quantitatively changes in different scenarios characterized by perturbations of nominal parameters and variable noise levels.

Keywords: Biomolecular systems, Systems biology, Genetic regulatory systems
Abstract: Hill functions are often used in stochastic models of gene regulation to approximate the dependence of gene activity on the concentration of the transcription factor (TF) that regulates the gene. However, it is generally unknown how much error one may incur from this approximation. We investigate this question in the context of transcriptional networks (TNs). Under the assumption of rapid binding and unbinding of TFs with their gene targets, we bound the approximation error (in terms of the total variation distance) between a mass-action stochastic model and a corresponding model with Hill function propensities. To do so, we use a combination of singular perturbation theory and moment analysis for stochastic chemical reaction networks. We assume throughout that TFs regulate genes in a one-to-one fashion, each regulated gene produces a single TF, TFs do not multimerize, and each gene only has a single TF binding site. These results are pertinent for the modeling of TNs and may also carry relevance for more general biological processes.

Keywords: Predictive control for nonlinear systems, Agents-based systems, Autonomous systems
Abstract: In control system networks, reconfiguration of the controller when agents are leaving or joining the network is still an open challenge, in particular when operation constraints that depend on each agent’s behavior must be met. Drawing our motivation from mobile robot swarms, in this paper, we address this problem by optimizing individual agent performance while guaranteeing persistent constraint satisfaction in presence of bounded communication range and time-varying network topology. The approach we propose is a model predictive control (MPC) formulation, building on multi-trajectory MPC (mt-MPC) concepts. To enable plug and play operations when the system is in closed-loop without the need of a request, the proposed MPC scheme predicts two different state trajectories in the same finite horizon optimal control problem. One trajectory drives the system to the desired target, assuming that the network topology will not change in the prediction horizon, while the second one ensures constraint satisfaction assuming a worst-case scenario in terms of new agents joining the network in the planning horizon. Recursive feasibility and stability of the closed-loop system during plug and play operations are shown. The approach effectiveness is illustrated with a numerical simulation.

Keywords: Predictive control for nonlinear systems, Data driven control, Nonlinear systems identification
Abstract: Indirect data-driven predictive control (DPC) algorithms for nonlinear systems typically employ multi-step predictors, which are identified from input-output data using neural networks. In this paper we put forward a unifying multi-step prediction network architecture, i.e., the deep subspace prediction network (DSPN). We then prove that the DSPN architecture specialized to multi-layer-perceptron neural networks recovers the linear predictor corresponding to subspace predictive control for a sufficient number of hidden layer neurons. Hence, we establish a well-posed generalization of subspace predictive control for nonlinear systems. Moreover, we develop a regularized DSPN architecture that embeds a linear subspace predictor to improve extrapolation properties for non-training data. Simulation results on a benchmark inverted pendulum show that nonlinear DPC based on DSPN achieves high control performance for both noiseless and noisy data.

Keywords: Predictive control for nonlinear systems, Automotive systems, Optimal control
Abstract: In this paper, we present conditions under which the terminal ingredients, defined by discrete-time control barrier function (DTCBF) certificates, guarantee recursive feasibility in nonlinear MPC. Further, we introduce the notion of quasi-DTCBF (qDTCBF) certificates. Compared to DTCBFs, qDTCBF conditions can be satisfied with tighter control input bounds, which is highly advantageous if only limited actuation is possible. Both certificates encourage an earlier reaction of the control system and result in a lower cumulative MPC cost. The methodology is applied to a lane merging problem in automated driving, in which DTCBF and qDTCBF certificates subject to input constraints form the terminal ingredients to guarantee recursive feasibility of the nonlinear MPC scheme. A simulation study demonstrates the efficacy of the concept.

Keywords: Predictive control for nonlinear systems, Hybrid systems, Constrained control
Abstract: Model predictive control (MPC) is promising for fueling and core density feedback control in nuclear fusion tokamaks, where the primary actuators, frozen hydrogen fuel pellets fired into the plasma, are discrete. Previous density feedback control approaches have only approximated pellet injection as a continuous input due to the complexity that it introduces. In this letter, we model plasma density and pellet injection as a hybrid system and propose two MPC strategies for density control: mixed-integer (MI) MPC using a conventional mixed-integer programming (MIP) solver and MPC utilizing our novel modification of the penalty term homotopy (PTH) algorithm. By relaxing the integer requirements, the PTH algorithm transforms the MIP problem into a series of continuous optimization problems, reducing computation complexity. By adding a logarithmic barrier term to the PTH algorithm, we prevent the concave penalty term and active path constraints from causing the optimization problem to yield non-integer solutions. Both strategies perform well with regards to reference tracking without violating path constraints and satisfy the computation time limit for real-time control of the pellet injection system. However, the computation time of the PTH-based MPC strategy consistently outpaces the conventional MI-MPC strategy and is especially beneficial for MPC formulations with longer prediction horizons.

Keywords: Predictive control for nonlinear systems, Lyapunov methods, Stability of nonlinear systems
Abstract: We provide extensions to the new flexible-step model predictive control (MPC) scheme, which is based on the idea of generalized discrete-time control Lyapunov functions. These facilitate the implementation of a flexible number of control inputs in each iteration of the MPC scheme. We present relaxed recursive feasibility and stability results and provide a converse Lyapunov result. These results combined simplify the design of the flexible-step MPC scheme. We demonstrate the capabilities of the flexible-step MPC algorithm for a nonholonomic system, where the standard one-step implementation may suffer from lack of asymptotic convergence.

Keywords: Predictive control for nonlinear systems, Machine learning, Linear parameter-varying systems
Abstract: This paper presents a hybrid neural network (NN) approach for adaptive scenario-based model predictive control (SMPC) design of nonlinear systems in the linear parameter-varying (LPV) framework. In particular, a deterministic artificial neural network (ANN)-based LPV model is learned from data as the nominal model. Then, a Bayesian NN (BNN) is used to describe the mismatch between the plant and the LPV-ANN model. Adaptive scenarios are generated online based on the BNN model to reduce the conservativeness of scenario generation. Moreover, a probabilistic safety certificate is incorporated into the scenario generation by ensuring that the trajectories of scenarios contain the trajectory of the system and that all the scenarios satisfy the constraints with a high probability. Furthermore, conditions for the recursive feasibility of the SMPC are given. Experiments on the closed-loop simulations of a two-tank system demonstrate that the proposed approach can better model the behaviors of nonlinear systems than sole ANN/BNN models can, and the SMPC based on the hybrid NN (HyNN) model can improve the control performance compared to the SMPC with a fixed scenario tree.

Keywords: Stochastic systems, Adaptive control, Constrained control
Abstract: We consider the problem of adaptive stabilization for discrete-time, multi-dimensional linear systems with bounded control input constraints and unbounded stochastic disturbances, where the parameters of the system are unknown. To address this challenge, we propose a certainty-equivalent control scheme combining online parameter estimation with saturated linear control. We establish the existence of a high probability stability bound on the closed-loop system, under additional assumptions on the system and noise processes. Numerical examples are presented to illustrate our results.

Keywords: Stochastic systems, Fault detection, Hybrid systems
Abstract: This paper proposes event-driven asynchronous filters for positive Markov jump systems by employing hidden Markov model. Based on the output of sensor measurement, a weighted event-driven threshold is established in the form of 1-norm. The stability of the corresponding augmented system can be guaranteed by transforming the error signal into interval uncertain form. Under the established triggering condition, an event-driven positive l1-gain asynchronous filter is constructed for positive Markov jump systems. Then, the asynchronous filter design for positive Markov jump systems with partial information of hidden Markov model is further addressed. All presented conditions are described in the linear programming form. Finally, one example is given to illustrate the effectiveness of the proposed design.

Keywords: Stochastic systems, Filtering, Computational methods
Abstract: Existing studies have pointed out numerical instability in the Kalman filter of atomic clocks, but the reasons for such instability have not been clarified mathematically. In this paper, we mathematically clarify the reason for the numerical instability by a new approach of spectral decomposition of the error covariance matrix in the Kalman filter. In particular, we reveal the fact that the error covariance matrix for homogeneous undetectable atomic clock ensembles can be decomposed into a diverging part and a converging part. Furthermore, the Kalman gain is solely influenced by the converging part, but not the diverging part, meaning that the Kalman gain converges to a steady-state value if ideal computation is possible without computation error. We present an alternative method to the conventional Kalman filter to avoid numerical instability and reduce computation cost where the covariance of Kalman filter can be computed rigorously only using three n-dimensional Riccati iterations instead of an nN-dimensional Riccati iterations for an n-order clock model with N clocks. A numerical example is provided to illustrate the efficacy of our approach.

Keywords: Stochastic systems, Flight control
Abstract: We estimate the probability that the first achievement of a given level by the component y_1(x) of n-dimensional continuous process y(x)={y_1(x),..., y_n(x)} occurs at some moment x* from a given interval (x',x") and, at this moment x*, the condition (y_2(x*),...,y_n(x*)∈D holds, where D is a given domain of (n−1)-dimensional Euclidean space R^{n−1}. The need to calculate the above-mentioned probability arises in the problems of aircraft control during landing.

Keywords: Stochastic systems, Fluid flow systems, Control of networks
Abstract: Motivated by applications in hydrodynamics and networks of thermostatically-control loads in buildings we study control of linear dynamical systems driven by additive and also multiplicative noise of a general position. Utilizing mathematical theory of stochastic multiplicative processes we present a universal way to estimate fat, algebraic tails of the state vector probability distributions. This prompts us to introduce and analyze mean-q-power stability criterion, generalizing the mean-square stability criterion, and then juxtapose it to other tools in control.

Keywords: Stochastic systems, Game theory, Attack Detection
Abstract: We study a setting where a detector wishes to detect and deter adversarial manipulation in an electronic voting machine. An adversary tries to win the election by tampering the votes while obfuscating its manipulation. We pose this problem as a game between the detector and the adversary and characterize the equilibrium payoffs for the players and the asymptotic nature of these payoffs. We find that if the detector is too cautious, then in equilibrium the adversary wins with a probability higher than its prior probability of winning. We derive an expression for the deterrence threshold, i.e., the minimum level of false-alarm that the detector should endure so that the adversary is not any better off by the manipulation. With this, asymptotically, the detector can ensure that the probability of missed-detection becomes zero by appropriately adjusting the rate of decay of probability of false-alarm. But if this rate of decay is too `fast', then the adversary can get an arbitrarily high probability of winning in spite of having a vanishing prior probability of winning. We then extend the results to a setting where the detector has incomplete information about the adversary.

Keywords: Networked control systems, Information theory and control, Control over communications
Abstract: Encrypted control is a framework for the secure outsourcing of controller computation using homomorphic encryption that allows to perform arithmetic operations on encrypted data without decryption. In a previous study, the security level of encrypted control systems was quantified based on the difficulty and computation time of system identification. This study investigates an optimal design of encrypted control systems when facing an attack attempting to estimate a system parameter by the least squares method from the perspective of the security level. This study proposes an optimal H₂ controller that maximizes the difficulty of estimation and an equation to determine the minimum security parameter that guarantee the security of an encrypted control system as a solution to the design problem. The proposed controller and security parameter are beneficial for reducing the computation costs of an encrypted control system, while achieving the desired security level. Furthermore, the proposed design method enables the systematic design of encrypted control systems.

Keywords: Optimization algorithms, Numerical algorithms, Computer/Network Security
Abstract: In this paper, we evaluate the different fully homomorphic encryption schemes, propose an implementation, and numerically analyze the applicability of gradient descent algorithms to solve quadratic programming in a homomorphic encryption setup. The limit on the multiplication depth of homomorphic encryption circuits is a major challenge for iterative procedures such as gradient descent algorithms. Our analysis not only quantifies these limitations on prototype examples, thus serving as a benchmark for future investigations, but also highlights additional trade-offs like the ones pertaining the choice of gradient descent or accelerated gradient descent methods, opening the road for the use of homomorphic encryption techniques in iterative procedures widely used in optimization based control. In addition, we argue that, among the available homomorphic encryption schemes, the one adopted in this work, namely CKKS, is the only suitable scheme for implementing gradient descent algorithms. The choice of the appropriate step size is crucial to the convergence of the procedure. The paper shows firsthand the feasibility of homomorphically encrypted gradient descent algorithms.

Keywords: Control Systems Privacy, Markov processes, Robotics
Abstract: We examine a novel setting in which two parties have partial knowledge of the elements that make up a Markov Decision Process (MDP) and must cooperate to compute and execute an optimal policy for the problem constructed from those elements. This situation arises when one party wants to give a robot some task, but does not wish to divulge those details to a second party---while the second party possesses sensitive data about the robot's dynamics (information needed for planning). Both parties want the robot to perform the task successfully, but neither is willing to disclose any more information than is absolutely necessary. We utilize techniques from secure multi-party computation, combining primitives and algorithms to construct protocols that can compute an optimal policy while ensuring that the policy remains opaque by being split across both parties. To execute a split policy, we also give a protocol that enables the robot to determine what actions to trigger, while the second party guards against attempts to probe for information inconsistent with the policy's prescribed execution. In order to improve scalability, we find that basis functions and constraint sampling methods are useful in forming effective approximate MDPs. We report simulation results examining performance and precision, and assess the scaling properties of our Python implementation. We also describe a hardware proof-of-feasibility implementation using inexpensive physical robots, which, being a small-scale instance, can be solved directly.

Keywords: Attack Detection, Cyber-Physical Security, Resilient Control Systems
Abstract: Multiplicative watermarking (MWM) is an active diagnosis technique for the detection of highly sophisticated at- tacks, but is vulnerable to malicious agents that use eavesdropped data to identify and then remove or replicate the watermark. In this work, we propose a scheme to protect the parameters of MWM, by proposing a design strategy based on piecewise affine (PWA) hybrid dynamical systems, called hybrid multiplicative watermarking (HMWM). Due to the design decision to make certain states of the HMWM systems unobservable, we show that parameter reconstruction by an eavesdropper is infeasible, from both a computational and a system-theoretic perspective, while not altering the system’s closed-loop performance.

Keywords: Attack Detection, Identification, Cyber-Physical Security
Abstract: The paper discusses delay injection attacks on regulator loops and suggests joint recursive prediction error identification of delay and dynamics for supervision and attack detection. The control system is assumed to be operated either in open- or closed-loop mode. It is shown why delay insertion in the feedback path before the user switches to closed-loop operation is advantageous to disguise the attack. Delay attack monitoring is preferably performed continuously, allowing for early attack detection before closed-loop control is initiated. The detection performance is evaluated numerically for a linearized automotive cruise control feedback loop.

Keywords: Control Systems Privacy, Cyber-Physical Security, Optimization
Abstract: Control related data, such as system states and inputs or controller specifications, is often sensitive. Meanwhile, the increasing connectivity of cloud-based or networked control results in vast amounts of such data, which poses a privacy threat, especially when evaluation on external platforms is considered. In this context, a cipher based on a random affine transformation gained attention, which is supposed to enable privacy-preserving evaluations of quadratic programs (QPs) with little computational overhead compared to other methods.

This paper deals with the security of such randomly transformed QPs in the context of model predictive control (MPC). In particular, we show how to construct attacks against this cipher and thereby underpin concerns regarding its security in a practical setting. To this end, we exploit invariants under the transformations and common specifications of MPC-related QPs. Our numerical examples then illustrate that these two ingredients suffice to extract information from ciphertexts.

Keywords: Networked control systems, Sampled-data control, Control over communications
Abstract: This work studies the performance of an event-based control approach, namely level-triggered sampling, in a standard multidimensional single-integrator setup. We falsify a conjecture from the literature that the deployed p-norm in the triggering condition supposedly has no impact on the performance of the sampling scheme in that setting. In particular, we show for the considered setup that the usage of the maximum norm instead of the Euclidean norm induces a performance deterioration of level-triggered sampling for sufficiently large system dimensions, when compared to periodic control at the same sampling rate. Moreover, we investigate the performance for other p-norms in simulation and observe that it degrades with increasing p. In addition, our findings reveal the previously unknown role of the triggering rule in the cause of a recently discovered phenomenon: Previous work has shown for a single-integrator consensus setup that the commonly observed performance advantage of event-based control over periodic control can be lost in distributed settings with a cooperative control goal. In our work, we obtain similar results for a non-cooperative setting only by adjusting the norm in the level-triggered sampling scheme. We therefore demonstrate that the performance degradation found in the distributed setting originates from the triggering rule and not from the considered cooperative control goal.

Keywords: Distributed control, Control of networks, Optimization algorithms
Abstract: This paper investigates the event-triggered distributed optimization problems (ETDOPs) over strongly connected directed networks. By assigning an additional scalar state variable to each agent and utilizing diminishing time-varying gain/step-size, a class of modified event-triggered distributed optimization algorithms (ETDOAs) is proposed, which can address the ETDOPs well and can avoid the inverse operation of some estimators in the existing literature. Compared with the existing DOAs, this paper gives a new idea to solve the DOPs under weighted-unbalanced digraphs and continuous communication of agent networks is avoided. Finally, numerical simulations are given to illustrate the effectiveness of the proposed ETDOAs.

Keywords: Agents-based systems, Cooperative control, Distributed control
Abstract: This paper studies the dynamic average consensus problem of multi-agent systems under event-triggered communication. In this problem, each agent has access to a time-varying reference signal and aims to track the average of all reference signals. Distributed algorithms with event-triggered communication have been developed to achieve dynamic average consensus. Nevertheless, these existing event-triggered communication mechanisms cannot guarantee the existence of a designable positive minimum inter-event time (MIET), which is important in their practical implementation. Motivated by this observation, we propose a distributed dynamic event-triggered communication mechanism (ETCM) for each agent. It is shown that the proposed ETCM guarantees the existence of a positive MIET that is locally adjustable by tuning design parameters. It is also shown that the dynamic average consensus is achieved with any pre-specified level of accuracy. As an illustrative example, the theoretical results are applied to a networked battery energy storage system for state-of-charge balancing and desired total power tracking.

Keywords: Networked control systems, Network analysis and control, Estimation
Abstract: Emerging cyber-physical systems impel the development of advanced network scheduling schemes to utilize communication and computation resources efficiently. This paper investigates the event-based schedule for remote state estimation in networked control systems (NCSs) subject to delay and packet dropouts. The scheduler decides whether or not to send out a local estimate according to the Value of Information (VoI) metric, which measures the relative importance of an information update. In addition, we model the triggering intervals as a Markov chain and analyze the tradeoff between the estimation performance and communication cost under the proposed VoI-based scheduling for the first-order system.

Keywords: Control over communications, Networked control systems, Sampled-data control
Abstract: This paper proposes a new control method called event-triggered parameterized control (ETPC). We showcase this method by focusing on the specific problem of stabilization of linear systems. In this control method, between two consecutive events, each control input to the plant is a linear combination of a set of linearly independent scalar functions. At each event, the coefficients of the parameterized control input are chosen to minimize the error in approximating a model based control signal and then they are communicated to the actuator. We design two event-triggering rules that guarantee global asymptotic stability of the origin of the closed loop system under some conditions on the model uncertainty. We also show the existence of a uniform positive lower bound on the inter-event times. We illustrate our results through numerical examples. We compare the proposed control method with event-triggered zero-order-hold control and show a significant improvement in terms of the average inter-event times.

Keywords: Agents-based systems, Stochastic optimal control, Networked control systems
Abstract: This paper starts by considering an optimal control formulation of the consensus problem on complete graphs with a cost capturing disagreement and agents modeled by integrators. An optimal control policy for this problem is shown to be the well-known consensus algorithm by which each agent resets its state to the average of its and other agents' state values received at every time step. The framework is extended to the case where agents can only exchange information periodically, with a period larger than one. Then an event-triggered control strategy is proposed that results in a better cost than that of the optimal periodic one with the same average transmission rate, that is, it is consistent. According to this strategy, each agent distributedly transmits its state if the error between its current state and a common consensus estimate based on previously transmitted agents' data exceeds a threshold.

Keywords: Networked control systems, Information theory and control, Control over communications
Abstract: We propose an adaptive coding approach to achieve linear-quadratic-Gaussian (LQG) control with near-minimum bitrate prefix-free feedback. Our approach combines a recent analysis of a quantizer design for minimum rate LQG control with work on universal lossless source coding for sources on countable alphabets. It was recently demonstrated that the aforementioned quantizer's outputs are an asymptotically stationary, ergodic process. To enable LQG control with provably near-minimum bitrate, the quantizer outputs must be encoded into binary codewords efficiently. This is possible given knowledge of the quantizations' probability distributions, or of their limiting distribution. Obtaining such knowledge is challenging; the distributions do not readily admit closed form descriptions. This motivates the application of universal source coding. Our main theoretical contribution in this work is a proof that (after an invertible transformation), the quantizer outputs are random variables that fall within an exponential or power-law envelope class (depending on the plant dimension). Using ideas from universal coding on envelope classes, we develop a practical, zero-delay, fixed precision source code for the quantizer outputs. We evaluate the performance of this approach numerically, and demonstrate competitive results with respect to fundamental tradeoffs between bitrate and LQG control performance.

Keywords: Quantized systems, Switched systems, Information theory and control
Abstract: In this work, we argue that the usual notion of controllability is unfit for systems that operate with finite data-rate constraints. We deal with this issue by defining a new concept of controllability with finite data-rate. Then, we specialize our discussion to the case of switched linear systems. We state a necessary condition and a sufficient condition for our new controllability notion to hold. Next, we take advantage of the switched linear system's structure to present a simple sufficient condition for controllability with finite data-rate that only involves the controllable subspace of the individual modes and some mild assumptions about the switching signal that guarantee that our sufficient condition holds. We also present another sufficient condition for systems that activate some controllable mode often enough. In particular, we illustrate the power of this result by deriving relations between the sampling time and the Average Dwell-Time (ADT) of the switching signal that guarantee that the switched system is controllable with finite data-rate. Finally, we discuss the gap between the necessary and the sufficient conditions and show that the sufficient condition is not necessary.

Keywords: Learning, Data driven control, Identification for control
Abstract: Learning for control in repeated tasks allows for well-designed experiments to gather the most useful data. We consider the setting in which we use a data-driven controller that does not have access to the true system dynamics. Rather, the controller uses inferred dynamics based on the available information. In order to acquire data that is beneficial for this controller, we present an experimental design approach that leverages the current data to improve expected control performance. We focus on the setting in which inference on the unknown dynamics is performed using Gaussian processes. Gaussian processes not only provide uncertainty quantification but also allow us to leverage structures inherit to Gaussian random variables. Through this structure, we design experiments via gradient descent on the expected control performance with respect to the experiment input. In particular, we focus on a chance-constrained minimum expected time control problem. Numerical demonstrations of our approach indicate our experimental design outperforms relevant benchmarks.

Keywords: Information theory and control, Machine learning, Stochastic optimal control
Abstract: In Shannon’s classical information-theoretic lossy coding problem, one is allowed to encode long sequences of source symbols at once in order to achieve a lower distortion, which is optimal in the limit of unbounded block lengths. Such a block-coding approach is undesirable in many delay-sensitive applications, such as networked control, sensor networks and live-streaming, among others. Accordingly, we are interested in a variant of Shannon’s lossy coding problem, where one wishes to send an information source causally at a fixed rate with no delay over a channel with feedback, while minimizing the average distortion at the receiver. Thus, the classical block-coding approach is not viable. This problem has previously been studied using stochastic control techniques, leading to existence, structural, and general approximation results. However, these techniques do not provide closed-form solutions for either optimal performance or code designs, and they lead to algorithmic implementations that are computationally difficult. To address this problem, we propose a reinforcement learning approach by building on recent results on quantized Q-learning. We will consider the case of a finite-alphabet Markov source over a discrete memoryless channel. After developing some supporting technical results on regularity and stability properties of the associated Markov process, we rigorously justify convergence of a quantized Q-learning algorithm to a near-optimal policy for this problem. Finally, we illustrate our theoretical findings via simulations.

Keywords: Transportation networks, Game theory
Abstract: We study optimal information provision in transportation networks when users are strategic and the network state is uncertain. An omniscient planner observes the network state and discloses information to the users with the goal of minimizing the expected travel time at the user equilibrium. Public signal policies, including full-information disclosure, are known to be inefficient in achieving optimality. For this reason, we focus on private signals and restrict without loss of generality the analysis to signals that coincide with path recommendations that satisfy obedience constraints, namely users have no incentive in deviating from the received recommendation according to their posterior belief. We first formulate the general problem and analyze its properties for arbitrary network topologies and delay functions. Then, we consider the case of two parallel links with affine delay functions, and provide sufficient conditions under which optimality can be achieved by information design. Interestingly, we observe that the system benefits from uncertainty, namely it is easier for the planner to achieve optimality when the variance of the uncertain parameters is large. We then provide an example where optimality can be achieved even if the sufficient conditions for optimality are not met.

Keywords: Information theory and control, Uncertain systems, Stability of linear systems
Abstract: We consider the control of a linear system observed over multiplicative-noise. Specifically, the controller must stabilize the system using a control action based on observations of the system state that have been multiplied by i.i.d. random variables. While there is a long history of work on this fundamental problem, much of it has focused on understanding the performance of linear controllers, and the optimal control strategy for such a system remains unknown. In this paper, we consider the case of uniform multiplicative observation noise, and provide a non-linear control strategy based on the maximum a-posteriori (MAP) estimator of the state. We explicitly compute the convergence rates of different moments of the system under this control strategy, and find that the MAP-based strategy outperforms the best memoryless linear strategy when the ``signal-to-noise'' ratio (SNR) of the multiplicative noise, i.e. the ratio of the mean to the standard deviation, is low. In the high SNR regime we see that the MAP strategy is also a linear memoryless strategy, however, it is suboptimal and is outperformed by the optimal linear controller.

Keywords: Modeling, Hybrid systems, Nonlinear systems
Abstract: A major limitation of the classical control theory is the assumption that the state space remains stationary in time. This prevents analyzing and even formalizing the stability and control problems for open multi-agent systems whose agents may enter or leave the network, industrial processes where the sensors or actuators may be exchanged frequently, smart grids, etc. In this work, we propose a framework of live systems that covers a rather general class of systems with a time-varying state space. We argue that input-to-state stability is a proper stability notion for this class of systems, and many of the classic tools and results, such as Lyapunov methods and superposition theorems, can be extended to this setting.

Keywords: Modeling, Model/Controller reduction
Abstract: We propose a method for designing stable limit-cycle oscillators with prescribed periodic trajectories and phase-response properties in general dimensions based on the phase reduction theory. The vector field of the oscillator is approximated by polynomials and their coefficients are optimized to satisfy required conditions. Linear stability of the periodic trajectory is ensured by imposing conditions on the eigenvalues of the monodromy matrix based on Floquet theory. We verify the validity of the proposed method by designing several types of oscillators with given properties. As an application, we design two oscillators with the same periodic trajectory but with different phase-response properties and show their distinct synchronization dynamics under the same periodic input.

Keywords: Modeling, Nonlinear output feedback
Abstract: Interconnection schemes are ubiquitous in physical systems. For instance, in multi-domain systems consisting of interconnected subsystems from different physical domains. Furthermore, the interconnection of two or more systems has also been exploited to analyze and control dynamical systems, especially passive ones. To this end, the most common interconnection structure is the negative feedback interconnection. However, this approach is unsuitable to directly couple the states of the subsystems in the overall system's energy as customarily occurs in physical systems. This letter provides two interconnection approaches that overcome this issue. Notably, it is shown that these interconnection structures are suitable for decomposing passive systems into the interconnection of simpler passive subsystems. Moreover, these interconnections schemes allow the interpretation of some existing nonlinear control approaches as the interconnection of a passive plant with a passive controller. Additionally, the interpretation of the proposed interconnection structures is provided via bond graphs.

Keywords: Modeling, Nonlinear systems, Stability of linear systems
Abstract: In this paper we propose a consensus model using fractional calculus, which is an emerging topic in multi-agent modeling. Fractional models have infinite memory and can be understood as a relatively simple extension of traditional calculus. We propose a model structure motivating it by psychological research. For such model we also provide a stability analysis allowing results on possibilities of consensus arising in the modelled group of agents. To achieve this, we use fractional difference equations, which illustrate our considerations for agent groups of increasing complexity.

Keywords: Modeling, Stability of nonlinear systems, Network analysis and control
Abstract: Motivated by neuromorphic computing applications, this paper considers electrical circuits comprising memristors and grounded capacitors, connected to external sources. By using the flux-charge domain modelling approach, we will derive an initial value problem describing the dynamic behaviour of this circuit. Given an initial value and a fixed input, we will show that the fluxes in this circuit converge to an equilibrium. Furthermore, we show that when the fluxes reach this equilibrium, we achieve voltage synchronisation, i.e. no more currents are flowing through the circuit. These results are emphasised in an illustration.

Keywords: Modeling, Stochastic systems, Uncertain systems
Abstract: In micro-assembly applications, ensemble of chiplets immersed in a dielectric fluid are steered using dielectrophoretic forces induced by an array of electrode population. Generalizing the finite population deterministic models proposed in prior works for individual chiplet position dynamics, we derive a controlled mean field model for a continuum of chiplet population in the form of a nonlocal, nonlinear partial differential equation. The proposed model accounts for the stochastic forces as well as two different types of nonlocal interactions, viz. chiplet-to-chiplet and chiplet-to-electrode interactions. Both of these interactions are nonlinear functions of the electrode voltage input. We prove that the deduced mean field evolution can be expressed as the Wasserstein gradient flow of a Lyapunov-like energy functional. With respect to this functional, the resulting dynamics is a gradient descent on the manifold of joint population density functions with finite second moments that are supported on the position coordinates.

Keywords: Power systems, Game theory, Optimization
Abstract: Online feedback optimization (OFO) is an emerging control methodology for real-time optimal steady-state control of complex dynamical systems. This tutorial focuses on the application of OFO for the autonomous operation of large-scale transmission grids, with a specific goal of minimizing renewable generation curtailment and losses while satisfying voltage and current limits. When this control methodology is applied to multi-area transmission grids, where each area independently manages its congestion while being dynamically interconnected with the rest of the grid, a non-cooperative game arises. In this context, OFO must be interpreted as an online feedback equilibrium seeking (FES) scheme. Our analysis incorporates technical tools from game theory and monotone operator theory to evaluate the stability and robustness of multi-area grid operation. Through numerical simulations, we illustrate the key challenge of this non-cooperative setting: on the one hand, independent multi-area decisions are suboptimal compared to a centralized control scheme; on the other hand, some areas are heavily penalized by the centralized decision, which may discourage participation in the coordination mechanism.

Keywords: Optimization algorithms, Adaptive control, Power systems
Abstract: Online feedback-based optimization has become a promising framework for real-time optimization and control of complex engineering systems. This paper surveys the recent advances in the field as well as provides novel convergence results for primal-dual online algorithms with heterogeneous step sizes for different elements of the gradient. The analysis is performed for quadratic programs and the approach is illustrated on applications for adaptive step-size and model-free online algorithms, in the context of optimal control of modern power systems.

Keywords: Power systems, Smart grid, Optimal control
Abstract: We have all heard that there is growing need to secure resources to obtain supply-demand balance in a power grid facing increasing volatility from renewable sources of energy. There are mandates for utility scale battery systems in regions all over the world, and there is a growing science of "demand dispatch" to obtain virtual energy storage from flexible electric loads such as water heaters, air conditioning, and pumps for irrigation. The question addressed in this tutorial is how to manage a large number of assets for balancing the grid. The focus is on variants of the economic dispatch problem, which may be regarded as the "feed-forward" component in an overall control architecture.

1) The resource allocation problem is identical to a finite horizon optimal control problem with degenerate cost---so called "cheap control". This implies a form of state space collapse, whose form is identified: the marginal cost for each load class evolves in a two-dimensional subspace, spanned by a scalar co-state process and its derivative.

2) The implication to distributed control is remarkable. Once the co-state process is synthesized, this common signal may be broadcast to each asset for optimal control. However, the optimal solution is extremely fragile, in a sense made clear through results from numerical studies.

3) Several remedies are proposed to address fragility. One is described through "robust training" in a particular Q-learning architecture (one approach to reinforcement learning). In numerical studies it is found that specialized training leads to more robust control solutions.

Keywords: Power systems, Power electronics, Power generation
Abstract: Electrical power systems are transitioning from fuel-based generation to renewable generation and transmission interfaced by power electronics. This transition challenges standard power system modeling, analysis, and control paradigms across timescales from milliseconds to seasons. This tutorial focuses on frequency stability on timescales of milliseconds to seconds. We first review basic results for grid-following (GFL) and grid-forming (GFM) control of voltage source converters (VSCs), typical renewable generation, and high voltage direct current (HVdc) transmission. In this context, it becomes apparent that GFL and GFM control functions are needed to operate emerging power systems. However, combining GFL resources, GFM resources, and legacy generation on the same system results in highly complex dynamics that are a significant obstacle to stability analysis. The remainder of the tutorial provides an overview of recent developments in universal GFM controls that bridge the gap between GFL and GFM control and provide a pathway to a coherent control and analysis framework accounting for power generation, power conversion, and power transmission.

Keywords: Power systems, Smart grid
Abstract: In this tutorial we present a simple approach to modeling unbalanced three-phase power flows. We allow general non-ideal models of voltage sources and ZIP loads. The basic idea is to explicitly separate a device/transformer model into an internal model, that depends on the characteristics of the single-phase devices or transformers, and a conversion rule, that depends on their configuration. This allows us to exploit common structures across different device/transformer variants and derive their external models that are general and unified. We illustrate the model by formulating a three-phase optimal power flow problem as a quadratically constrained quadratic program.

Keywords: Power systems, Nonlinear systems
Abstract: Solving power flow is perhaps the most fundamental calculation related to the steady state behavior of alternating-current (AC) power systems. The normally radial (tree) topology of a distribution network induces a spatially recursive structure in power flow equations, which enables a class of efficient solution methods called backward/forward sweep (BFS). In this paper, we revisit BFS from a new perspective, focusing on its convergence. Specifically, we describe a general formulation of BFS, interpret it as a special Gauss-Seidel algorithm, and then illustrate it in a single-phase power flow model. We prove a sufficient condition under which the BFS is a contraction mapping on a closed set of safe voltages and thus converges geometrically to a unique power flow solution. We verify the convergence condition, as well as the accuracy and computational efficiency of BFS, through numerical experiments in IEEE test systems.

Keywords: Identification for control, Output regulation
Abstract: We generalize a recently introduced data-driven approach for model-reference control design with closed-loop stability guarantees to the case of single-input single-output systems with inaccessible state. By considering a dynamic controller with fixed structure and leveraging a data-based description of the closed-loop dynamics, we propose a two-stage strategy for the optimization of the controller's parameters to match the desired closed-loop behavior. By means of a benchmark simulation example, we show the potential of the proposed approach and the impact of a simple strategy to handle noisy measurements.

Keywords: Data driven control, Optimal control, Learning
Abstract: This paper proposes a novel approach for Predictive Control utilizing Reinforcement Learning (RL) and Data-Driven techniques to derive optimal control policies for real systems. Using pure input-output multi-step predictors based on Subspace Identification and RL techniques, the resulting predictive control scheme can approximate the optimal control policy of a system with high accuracy, even if the predictor cannot accurately capture the true system dynamics. One of the key contributions of the proposed approach is the extension of the framework connecting Model Predictive Control (MPC) and RL to one that does not require explicit state-space models, nor to define a notion of state at all. The paper demonstrates the efficacy of the proposed approach through an illustrative example, highlighting the ability of our approach to provide an optimal control policy for a real system without requiring any prior knowledge about its internal dynamics.

Keywords: Identification for control, Statistical learning, Linear systems
Abstract: We present a local minimax lower bound on the excess cost of designing a linear-quadratic controller from offline data. The bound is valid for any offline exploration policy that consists of a stabilizing controller and an energy bounded exploratory input. The derivation leverages a relaxation of the minimax estimation problem to Bayesian estimation, and an application of van Trees inequality. We show that the bound aligns with system-theoretic intuition. In particular, we demonstrate that the lower bound increases when the optimal control objective value increases. We also show that the lower bound increases when the system is poorly excitable, as characterized by the spectrum of the controllability gramian of the system mapping the noise to the state and the H-infinity norm of the system mapping the input to the state. We further show that for some classes of systems, the lower bound may be exponential in the state dimension, demonstrating exponential sample complexity for learning the linear-quadratic regulator.

Keywords: Identification for control
Abstract: In the context of data-driven control, persistence of excitation (PE) of an input sequence is defined in terms of a rank condition on the Hankel matrix of the input data. For nonlinear systems, recent results employed rank conditions involving collected input and state/output data, for which no guidelines are available on how to satisfy them a priori. In this paper, we first show that a set of discrete impulses is guaranteed to be persistently exciting for any controllable LTI system. Based on this result, for certain classes of nonlinear systems, we guarantee persistence of excitation of sequences of basis functions a priori, by design of the physical input only.

Keywords: Healthcare and medical systems, Optimal control, Statistical learning
Abstract: We investigate adaptive protocols for the elimination or reduction of the use of medications or addictive substances. We formalize this problem as online optimization, minimizing the cumulative dose subject to constraints on individual well-being. We adapt a model of addiction from the psychology literature and show how it can be described by a class of linear time-invariant systems. For such systems, the optimal policy amounts to taking the smallest dose that maintains well-being. We derive a simple protocol based on integral control that requires no system identification, only needing approximate knowledge of the instantaneous dose response. This protocol is robust to model misspecification and is able to maintain an individual's well-being during the tapering process. Numerical experiments demonstrate that the adaptive protocol outperforms non-adaptive methods in terms of both maintenance of well-being and rate of dose reduction.

Keywords: Predictive control for nonlinear systems, Machine learning, Robotics
Abstract: Learning-based optimal control algorithms control unknown systems using past trajectory data and a learned model of the system dynamics. These controllers use either a linear approximation of the learned dynamics, trading performance for faster computation, or nonlinear optimization methods, which typically perform better but can limit real-time applicability. In this work, we present a novel nonlinear controller that exploits differential flatness to achieve similar performance to state-of-the-art learning-based controllers but with significantly less computational effort. Differential flatness is a property of dynamical systems whereby nonlinear systems can be exactly linearized through a nonlinear input mapping. Here, the nonlinear transformation is learned as a Gaussian process and is used in a safety filter that guarantees, with high probability, stability as well as input and flat state constraint satisfaction. This safety filter is then used to refine inputs from a Flat Model Predictive Controller to perform constrained nonlinear learning-based optimal control through two successive convex optimizations. We compare our method to state-of-the-art learning-based control strategies and achieve similar performance, but with significantly better computational efficiency, while also respecting flat state and input constraints, and guaranteeing stability.

Keywords: Learning, Pattern recognition and classification
Abstract: Adjacent lane-changing is one of the most dangerous maneuvers which may lead to rear-end crash, uncomfortable braking and sharp steering. If the autonomous driving system can predict the potential latent lane-changing intentions of surrounding vehicles in advance, the driver will have more time to make reasonable response. In this paper, we focus on how to give accurate and reliable prediction for latent lanechangings, especially before the vehicles merge into the target lanes. A prediction model based on dual Gaussian-mixed hidden Markov models is developed to exploit the advantages of different features more effectively. Since there is no comprehensive criteria to evaluate the accuracy and predictability performance simultaneously, we propose two new metrics for quantitative analysis as supplement to the classical indicators. Comparative validation on Next Generation Simulation (NGSIM) database shows that our model has a high recognition accuracy of 93.05% for lane-changing intention with earlier prediction over the existing homologous methods.

Keywords: Observers for nonlinear systems, Learning, Uncertain systems
Abstract: In this paper, we introduce a Gaussian process based moving horizon estimation (MHE) framework. The scheme is based on offline collected data and offline hyperparameter optimization. In particular, compared to standard MHE schemes, we replace the mathematical model of the system by the posterior mean of the Gaussian process. To account for the uncertainty of the learned model, we exploit the posterior variance of the learned Gaussian process in the weighting matrices of the cost function of the proposed MHE scheme. We prove practical robust exponential stability of the resulting estimator using a recently proposed Lyapunov-based proof technique. Finally, the performance of the Gaussian process based MHE scheme is illustrated via a nonlinear system.

Keywords: Predictive control for nonlinear systems, Learning, Optimization
Abstract: Model predictive control effectively handles complex dynamical systems with constraints, but its high computational demand often makes real-time application infeasible. We propose using Gaussian process regression to learn an approximation of the controller offline for online use. Our approach incorporates a robust predictive control scheme and provides bounds on approximation errors to ensure recursive feasibility and input-to-state stability. Exploiting a sampling-based scenario approach, we develop an efficient sampling strategy and guarantee that, with high probability, the approximation error remains within acceptable bounds. Our method demonstrates enhanced efficiency and reduced computational demand in an example application.

Keywords: Data driven control, Uncertain systems, Statistical learning
Abstract: Gaussian process state space models are becoming common tools for the analysis and design of nonlinear systems with uncertain dynamics. When designing control policies for these systems, safety is an important property to consider. In this paper, we provide safety guarantees by computing finite-horizon forward reachable sets for Gaussian process state space models. We use data-driven reachability analysis to provide exact probability measures for state trajectories of arbitrary length, even when no data samples are available. We investigate two numerical examples to demonstrate the power of this approach, such as providing highly non-convex reachable sets and detecting holes in the reachable set.

Keywords: Optimization, Process Control
Abstract: This paper considers the problem of Bayesian optimization (BO) for systems with safety-critical constraints. Recent work has shown that a theoretically consistent way to account for constraints in BO is to relax the constraint functions such that the feasible region has a high probability of containing the global solution. However, by construction, these approaches are unable to ensure safe/feasible operation at every query, which is unacceptable in safety-critical applications. Alternatively, safe BO methods force the query points to remain in the interior of a partially-revealed safety region, which may result in unacceptable (and unquantified) performance losses. This paper presents a new safe BO method that avoids these performance losses by systematically incorporating potential performance gains from enlargement of the safety region. The proposed method avoids getting stuck at suboptimal points based on a potentially small initial safety region due to limited initial exploration of the safety boundary. The performance of the proposed method is demonstrated for safe control of a cold atmospheric plasma jet towards personalized plasma medicine.

Keywords: Machine learning, Optimization
Abstract: This paper studies the problem of online performance optimization of constrained closed-loop control systems, where both the objective and the constraints are unknown black-box functions affected by exogenous time-varying contextual disturbances. A primal-dual contextual Bayesian optimization algorithm is proposed that achieves sublinear cumulative regret with respect to the dynamic optimal solution under certain regularity conditions. Furthermore, the algorithm achieves zero time-average constraint violation, ensuring that the average value of the constraint function satisfies the desired constraint. The method is applied to both sampled instances from Gaussian processes and a continuous stirred tank reactor parameter tuning problem; simulation results show that the method simultaneously provides close-to-optimal performance and maintains constraint feasibility on average. This contrasts current state-of-the-art methods, which either suffer from large cumulative regret or severe constraint violations for the case studies presented.

Keywords: Transportation networks, Traffic control, Optimization
Abstract: This paper presents a time-invariant network flow model capturing two-person ride-pooling that can be integrated within design and planning frameworks for Mobility-on-Demand systems. In these type of models, the arrival process of travel requests is described by a Poisson process, meaning that there is only statistical insight into request times, including the probability that two requests may be pooled together. Taking advantage of this feature, we devise a method to capture ride-pooling from a stochastic mesoscopic perspective. This way, we are able to transform the original set of requests into an equivalent set including pooled ones which can be integrated within standard network flow problems, which in turn can be efficiently solved with off-the-shelf LP solvers for a given ride-pooling request assignment. Thereby, to compute such an assignment, we devise a polynomial-time algorithm that is optimal w.r.t. an approximated version of the problem. Finally, we perform a case study of Sioux Falls, USA, where we quantify the effects that waiting time and experienced delay have on the vehicle-hours traveled. Our results suggest that the higher the demands per unit time, the lower the waiting time and delay experienced by users. In addition, for a sufficiently large number of demands per unit time, with a maximum waiting time and experienced delay of 5 minutes, more than 90% of the requests can be pooled.

Keywords: Transportation networks, Game theory, Optimization
Abstract: Credit-based congestion pricing (CBCP) has emerged as a mechanism to alleviate the social inequity concerns of road congestion pricing - a promising strategy for traffic congestion mitigation - by providing low-income users with travel credits to offset some of their toll payments. While CBCP offers immense potential for addressing inequity issues that hamper the practical viability of congestion pricing, the deployment of CBCP in practice is nascent, and the potential efficacy and optimal design of CBCP schemes have yet to be formalized. In this work, we study the design of CBCP schemes to achieve particular societal objectives and investigate their influence on traffic patterns when routing heterogeneous users with different values of time (VoTs) on a multi-lane highway with an express lane (EL). To this end, we introduce a new non-atomic congestion game model of a mixed-economy, wherein eligible users receive travel credits while the remaining ineligible users pay out-of-pocket to use the EL. In this setting, we investigate the effect of CBCP schemes on traffic patterns by characterizing the properties (i.e., existence, comparative statics) of the corresponding Nash equilibria and, in the setting when eligible users have time-invariant VoTs, develop a convex program to compute these equilibria. We further present a bi-level optimization framework to design optimal CBCP schemes to achieve a central planner’s societal objectives. Finally, we conduct numerical experiments based on a case study of the San Mateo 101 Express Lanes Project, one of the first CBCP pilots. Our results demonstrate the potential of CBCP to enable low-income users to avail of the travel time savings provided by congestion pricing on ELs while having comparatively low impacts on the travel costs of other road users.

Keywords: Autonomous vehicles, Emerging control applications, Optimal control
Abstract: This paper considers electric automated buses traveling in inter-urban roads and following a given route including stops, that must be reached according to a given timetable. Some of these stops are provided with a charging infrastructure allowing to charge the bus batteries. The paper proposes a decentralized control scheme for determining the optimal speed profiles, the dwell and charging times of the buses, by taking into account the traffic conditions along the road through a suitable traffic flow prediction model. Two objectives are considered contemporarily: the minimization of the deviations from the timetable and the minimization of the energy lack at the end of the bus route. To attain both these conflicting objectives, a lexicographic approach is adopted to design the controller which considers that, depending on the system state, the priority of the two objectives can change. Accordingly, the proposed control scheme changes the objective prioritization in real time and switches between two different lexicographic-based optimal control solutions. Some tests are discussed in the paper to show the effectiveness of the proposed control scheme.

Keywords: Transportation networks, Mean field games, Iterative learning control
Abstract: Understanding travelers' day-to-day departure time choice (DDTC) is vital for managing traffic congestion, especially in multi-modal transportation systems. While providing real-time traffic information and alternative trip plans brings convenience to travelers, their collective travel patterns may conversely lead to unstable traffic equilibrium states. We investigate a DDTC problem with mode switching in this paper. A group of heterogeneous agents can adaptively choose their modes and departure times to minimize total travel costs in a dynamic game. Using a customized hierarchical soft actor-critic (HSAC) algorithm with a continuum approximation of other agents, the traffic dynamics will converge to an approximate Markovian Perfect Equilibrium (MPE). Our findings also shed light on changes in long-term travel behavior due to the widespread deployment of emerging mobility and travel information technology. This approach serves as a foundation for promoting intelligent travel plans through adaptive traffic control policies.

Keywords: Game theory, Traffic control
Abstract: Within mobility systems, the presence of self-interested users can lead to aggregate routing patterns that are far from the societal optimum which could be achieved by centrally controlling the users' choices. In this paper, we design a fair incentive mechanism to steer the selfish behavior of the users to align with the societally optimal aggregate routing. The proposed mechanism is based on an artificial currency that cannot be traded or bought, but only spent or received when traveling. Specifically, we consider a parallel-arc network with a single origin and destination node within a repeated game setting whereby each user chooses from one of the available arcs to reach their destination on a daily basis. In this framework, taking faster routes comes at a cost, whereas taking slower routes is incentivized by a reward. The users are thus playing against their future selves when choosing their present actions. To capture this complex behavior, we assume the users to be rational and to minimize an urgency-weighted combination of their immediate and future discomfort. To design the optimal pricing, we first derive a closed-form expression for the best individual response strategy. Second, we formulate the pricing design problem for each arc to achieve the societally optimal aggregate flows, and reformulate it so that it can be solved with gradient-free optimization methods. Our numerical simulations show that it is possible to achieve a near-optimal routing whilst significantly reducing the users' perceived discomfort when compared to a centralized optimal but urgency-unaware policy.

Keywords: Traffic control, Autonomous vehicles, Optimal control
Abstract: In this paper, we present an optimal control framework to address motion coordination of connected automated vehicles (CAVs) in the presence of human-driven vehicles (HDVs) in merging scenarios. Our framework combines an unconstrained trajectory solution of a low-level energy-optimal control problem with an upper-level optimization problem that yields the minimum travel time for CAVs. We predict the future trajectories of the HDVs using Newell's car-following model. To handle potential deviations of HDVs' actual behavior from the predicted one, we design a safety filter for CAVs based on control barrier functions. The effectiveness of the proposed control framework is demonstrated via simulations with heterogeneous human driving behaviors.

Keywords: Communication networks, Optimization algorithms, Quantized systems
Abstract: In the modern paradigm of multi-agent networks, communication has become one of the main bottlenecks for decentralized optimization, where a large number of agents are involved in minimizing the average of the local cost functions. In this paper, we propose a robust compressed push-pull algorithm (RCPP) that combines gradient tracking with communication compression. In particular, RCPP is compatible with a much more general class of compression operators that allow both relative and absolute compression errors. We show that RCPP achieves linear convergence rate for smooth objective functions satisfying the Polyak-Łojasiewicz condition over general directed networks. Numerical examples verify the theoretical findings and demonstrate the efficiency, flexibility, and robustness of the proposed algorithm.

Keywords: Optimization algorithms, Control Systems Privacy, Cooperative control
Abstract: Privacy protection is gaining increased attention in distributed optimization and learning. As differential privacy is becoming a de facto standard for privacy preservation, recently results have emerged integrating differential privacy with distributed optimization. However, to ensure differential privacy (with a finite cumulative privacy budget), all existing approaches have to sacrifice provable convergence to the optimal solution. In this paper, we propose a differentially-private distributed optimization algorithm that can ensure, for the first time, both epsilon-differential privacy and optimality, even on the infinite time horizon. Numerical simulation results confirm the effectiveness of the proposed approach.

Keywords: Optimization algorithms, Distributed control, Machine learning
Abstract: In this paper, we propose a distributed stochastic first-order method for saddle point problems over strongly connected graphs. Existing methods generally suffer from a steady-state error that arises due to the heterogeneous nature of data distribution (captured by the local versus global cost gaps) and the variance of the stochastic gradients. We propose~SGDA, a distributed stochastic gradient descent ascent method that uses network-level textit{gradient tracking} to eliminate the steady-state error component due to the local versus global cost gap. We show that~SGDA converges linearly to an error ball around the unique saddle point for sufficiently small constant step-sizes when the global cost is strongly concave-convex (a necessary condition for the existence of a unique saddle point). Moreover, we show that the size of this error ball depends on the variance of the stochastic gradients. We provide numerical experiments to illustrate the convergence properties of~SGDA for different applications and highlight the significance of gradient tracking. We also show the performance of~SGDA for training modern applications like distributed generative adversarial networks (GANs).

Keywords: Machine learning, Optimization, Markov processes
Abstract: Meta-Reinforcement Learning (MRL) is a promising framework for training agents that can quickly adapt to new environments and tasks. In this work, we study the MRL problem under the policy gradient formulation, where we propose a novel algorithm that uses Moreau envelope surrogate regularizers to jointly learn a meta policy that is adjustable to the environment of each individual task. Our algorithm, called Moreau Envelope Meta-Reinforcement Learning (MEMRL), learns a meta-policy that can adapt to a distribution of tasks by efficiently updating the policy parameters using a combination of gradient-based optimization and Moreau Envelope regularization. Moreau Envelopes provide a smooth approximation of the policy optimization problem, which enables us to apply standard optimization techniques and converge to an appropriate stationary point. We provide a detailed analysis of the MEMRL algorithm, where we show a sublinear convergence rate to a first-order stationary point for non-convex policy gradient optimization. We finally show the effectiveness of MEMRL on a multi-task 2D-navigation problem.

Keywords: Game theory, Optimization algorithms, Distributed control
Abstract: This paper studies a class of non-cooperative games, known as N-cluster game, which subsumes both cooperative and non-cooperative nature among multiple agents in the two problems. Moreover, we consider a partial-decision information game setup, i.e., the agents have no direct access to the decisions of other agents in all clusters, and hence need to communicate with each other. We propose a distributed NE seeking algorithm by a synthesis of consensus and gradient tracking. Unlike other existing discrete-time methods for N-cluster games where a common step-size is either publicly known by all agents or only known by agents from the same cluster, the proposed algorithm can work with fully uncoordinated constant step-sizes, which allows the agents (both within and across the clusters) to choose their own preferred step-sizes. We prove that all agents' decisions converge linearly to their corresponding NE so long as the largest step-size and the heterogeneity of the step-sizes are small. We verify the derived results through a numerical example in a Cournot competition game.

Keywords: Optimization algorithms, Cooperative control, Agents-based systems
Abstract: This paper studies a class of personalized distributed bilevel optimization problems over networks, where nodes aim at jointly optimizing the sum of outer-level objectives that depend on the solution of inner-level optimization problems. The existing algorithms for distributed bilevel optimization problems usually require extra computation loops for estimating hypergradients. To facilitate computational efficiency, we develop a loopless distributed algorithm that employs certain steps to approximate the optimal solution of inner-level optimization problems, and track Hessian-inverse-vector products in a recursive manner. We prove that for stochastic nonconvex-strongly-convex problems, the proposed algorithm achieves the state of the art O(epsilon ^{-2}) communication cost, while improving the computational cost by O(log(frac{1}{epsilon})). Numerical experiments validate our theoretical findings.