Keywords:
Abstract: This activity is aimed at building a community for individuals who identify as under-represented within the Control Systems Society. The event is devoted to informal interactions. The breakfast is open to students, postdocs, researchers, and faculty members who identify as under-represented in the control systems field, STEM, or the academic community. This may include, but is not limited to, those who identify as being Black/African American, Hispanic, Indigenous, LGBTQ+, or as having a disability.

(While women are welcome, this event is aimed at those from other under-represented groups. Women are encouraged to attend the Women in Controls luncheon, which provides a similar venue for community building.)

Keywords: Intelligent systems
Abstract: AI has drastically changed the landscape of several engineering disciplines, such as computer vision and natural language processing, and is now an integral part of several commercial products. In this talk I will argue, through several vignettes, that ideas and techniques from control can help answer fundamental questions about the power, limitations, and societal impact of AI.

Keywords: Adaptive systems, Autonomous robots, Learning
Abstract: The opening lecture of this tutorial will offer a comprehensive introduction to the field of DS-based control. It will lay the theoretical groundwork for learning stable DS from demonstrations and illustrate how these learned policies can be applied to control robotic systems, ensuring both robustness and responsiveness.

Keywords: Adaptive systems, Autonomous robots, Learning 
Abstract: The second lecture of this tutorial will concentrate on modulating DSs in scenarios in- volving obstacle avoidance, presenting data-driven approaches to learning the modulation. Beginning with an overview of DS modulation theory, the lecture will trace the field’s progression towards handling increasingly complex situations without compromising guarantees of non-penetrability.

Keywords: Adaptive systems, Autonomous robots, Learning 
Abstract: The concluding lecture of this tutorial will delve into the emerging area of DSs enhanced by differential geometry, known as Geometrical DSs (GDS), within both the learning and control spheres. It will begin with a broad discussion on the motivations and objectives behind integrating differential geometry tools into DS-based control systems. Following this, the lecture will lay out the theoretical underpinnings necessary for constructing GDSs. The session will then highlight various applications and scenarios where GDSs are employed, outlining their benefits and limitations.

Keywords: Quantum information and control
Abstract: This letter investigates parameter estimation in quantum systems that undergo dynamical evolution. Optimal control problems are formulated to maximize the information, about an unknown parameter, extracted by a given quantum measurement apparatus. This letter introduces the concept of ``admissible controls''---control laws that do not depend on the unknown parameter they elicit. For scalar parameter estimation in unital quantum systems interrogated by binary measurements, this letter derives a necessary and sufficient condition on quantum measurement operators so that an information maximizing control law is admissible. When the admissibility condition is satisfied, it is shown that the resulting optimal control problem may be solved using well-established techniques.

Keywords: Quantum information and control
Abstract: The non-commutative nature of quantum mechanics imposes fundamental constraints on system dynamics, which in the linear realm are manifested by the physical realizability conditions on system matrices. These restrictions endow system matrices with special structure. The purpose of this paper is to study such structure by investigating zeros and poses of linear quantum systems. In particular, we show that -s_0^ast is a transmission zero if and only if s_0 is a pole, and which is further generalized to the relationship between system eigenvalues and invariant zeros. Additionally, we study left-invertibility and fundamental tradeoff for linear quantum systems in terms of their zeros and poles.

Keywords: Quantum information and control
Abstract: Pulse-based quantum machine learning (QML) models possess full expressivity when they are ensemble controllable. However, the accompanied barren plateaus render training intractable for high-dimensional systems. In this paper, we show that the trade-off can be understood via controllability analysis. We first apply the Fliess-series expansion to pulse-based QML models to investigate the effect of control system structure on model expressivity, which leads to a universal criterion for assessing the expressivity of generic QML models. We then show that uncontrollable pulse-based models that evolve on low-dimensional manifolds can achieve better balance between expressivity and trainability. Numerical experiments verify the proposed criterion and our analysis, which further demonstrate that the expressivity can be enhanced by increasing dimensionality, while barren plateaus can be avoided by reducing the controllability. Our approach provides a promising path for designing pulse-based QML models that are both highly expressive and trainable.

Keywords: Quantum information and control
Abstract: The estimation of all the parameters in an unknown quantum state/measurement device, commonly referred to as quantum state/detector tomography (QST/QDT), plays a pivotal role in characterization and control of quantum systems. In this paper, we introduce a novel method to concurrently identify a quantum state and a detector using multiple Hamiltonians. We then develop two solutions, i.e., a closed-form algorithm and the sum of squares (SOS) optimization. The numerical examples demonstrate that the closed-form solution offers computational efficiency, while the SOS optimization method ensures accuracy.

Keywords: Quantum information and control, Predictive control for nonlinear systems, Robust control
Abstract: This paper presents a robust control approach for a two-level open quantum system subject to bounded uncertainties through the application of model predictive control. We demonstrate two common cases: depolarizing decoherence and phase-damping decoherence in open quantum systems. We develop a model predictive control method using quantum measurements and provide a lower bound on the probability of obtaining the predicted state. Numerical results illustrate the effectiveness of this model predictive control approach in achieving stability and robustness in open quantum systems.

Keywords: Identification for control, Estimation, Quantum information and control
Abstract: This paper addresses the problem of parameter estimation for quantum systems undergoing homodyne detection. By generalizing the stability of quantum filters under quantum non-demolition measurements to cases with mismatched parameters, we propose an estimation approach that extends beyond finite parameter sets. Leveraging maximum likelihood estimation, our method offers improved reliability, flexibility, and potential computational savings compared to existing techniques. The asymptotic behavior of the estimator is rigorously analyzed, and simulations on spin-1/2 systems demonstrate the effectiveness and robustness of the proposed approach.

Keywords: Information theory and control, Data driven control, Cooperative control
Abstract: This paper presents two algorithms for multi-agent dynamic coverage in spatiotemporal environments, where the coverage algorithms are informed by the method of data assimilation. In particular, we show that by explicitly modeling the environment using a Gaussian Process (GP) model, and considering the sensing capabilities and the dynamics of a team of robots, we can design an estimation algorithm and multi-agent coverage controller that explores and estimates the state of the spatiotemporal environment. The uncertainty of the estimate is quantified using clarity, an information-theoretic metric, where higher clarity corresponds to lower uncertainty. By exploiting the relationship between GPs and Stochastic Differential Equations (SDEs) we quantify the increase in clarity of the estimated state at any position due to a measurement taken from any other position. We use this relationship to design two new coverage controllers, both of which scale well with the number of agents exploring the domain, assuming the robots can share the map of the clarity over the spatial domain via communication. We demonstrate the algorithms through a realistic simulation of a team of robots collecting wind data over a region in Austria.

Keywords: Cooperative control, Robotics, Agents-based systems
Abstract: In this paper, we propose a sub-optimal approach for the multi-agent navigation problem in simply-connected workspaces. We design a decentralized control law exhibiting the following three properties: (1) navigation of each agent with the optimal policy towards its destination, (2) avoidance of collision with other nearby agents and the workspace boundary, and (3) knowledge about the current position and not the destination of nearby agents. Moreover, we refer to the sub-optimal approach because the computational complexity and time needed to calculate the global optimal solution become unrealistic as the number of agents increases. In our case, each agent has a predetermined optimal policy calculated by a novel off-policy iterative method, to go towards its destination and it deviates from it in order to avoid collisions. The purpose of the simulation study is to examine how much the sub-optimal greedy trajectory of each agent deviates from the optimal one.

Keywords: Cooperative control, Adaptive control, Uncertain systems
Abstract: We consider the consensus problem for 2nd-order MIMO multi-agent systems with unknown nonlinear terms. We propose a novel control algorithm based on the Barrier Integral Control (BRIC) that combines reciprocal barrier functions with integral terms of the multi-agent disagreement errors and guarantees asymptotic consensus despite the unknown dynamics. The control algorithm is distributed, in the sense that each agent calculates its own control signal based on local information from its neighbouring agents, and does not use any a priori information from the agents' dynamics. Furthermore and unlike previous works, it does not rely on boundedness assumptions or approximation of the dynamic terms and constitutes smooth feedback of the multi-agent states. Finally, simulation results verify the theoretical findings.

Keywords: Autonomous robots, Distributed control, Large-scale systems
Abstract: We present a solution for locating the source, or maximum, of an unknown scalar field using a swarm of mobile robots. Unlike relying on the traditional gradient information, the swarm determines an ascending direction to approach the source with arbitrary precision. The ascending direction is calculated from field strength measurements at the robot locations and their relative positions concerning the swarm centroid. Rather than focusing on individual robots, we focus the analysis on the density of robots per unit area to guarantee a more resilient swarm, i.e., the functionality remains even if individuals go missing or are misplaced during the mission. We reinforce the algorithm's robustness by providing sufficient conditions for the swarm shape so that the ascending direction is almost parallel to the gradient. The swarm can respond to an unexpected environment by morphing its shape and exploiting the existence of multiple ascending directions. Finally, we validate our approach numerically with hundreds of robots. The fact that a large number of robots with a generic formation always calculate an ascending direction compensates for the potential loss of individuals.

Keywords: Stability of nonlinear systems, Lyapunov methods, Agents-based systems
Abstract: This paper aims to provide a generalized method of constructing distributed controllers for distance-based multi-agent formations; it also looks into possible degenerate cases when agents' positions are co-located. We first extend the definition of the distance error into a general function of the inter-agent distance, based on which a family of generalized controllers is derived. We then discuss singularities caused by co-located agents and amend some mistakes and gaps in the literature on distance-based formation control. These enable us to propose formation control methods of constructing singularity-free controllers while preserving the stability properties of the distance error system.

Keywords: Cooperative control, Estimation, Agents-based systems
Abstract: This paper investigates the distributed formation control problem solely based on range-only measurement, without the availability of bearing or relative bearing information. Without bearing information, local relative position necessary for maintaining rigid formation control cannot be obtained only based on range measurement. Correspondingly, range-only bearing estimator for a single landmark is proposed that can be directly combined with standard distributed rigid formation control law. Asymptotic stability analysis of the desired formation shape is presented and the proposed distributed approach is validated via numerical simulations.

Keywords: Nonlinear systems, Stability of nonlinear systems, Modeling
Abstract: We consider an isothermal flow through two pipes; at the junction, the flow is modified by some devices, for instance a valve. We first provide a general mathematical framework to model the coupling conditions for the flow at both sides of the junction. A key feature in the modeling is the {em coherence}; it is related to the chattering, i.e., the rapid switch on and off of a valve, which in turn is linked to the stability of the numerical schemes to find the solutions. We discuss the coherence of some models (for instance, for control valves), we provide numerical simulations showing the chattering, and finally give a procedure to eliminate it from a theoretical point of view.

Keywords: Optimization, Uncertain systems, Energy systems
Abstract: Natural gas consumption by users of pipeline networks is subject to increasing uncertainty that originates from the intermittent nature of electric power loads serviced by gas-fired generators. To enable computationally efficient optimization of gas network flows subject to uncertainty, we develop a finite volume representation of stochastic solutions of hyperbolic partial differential equation (PDE) systems on graph-connected domains with nodal coupling and boundary conditions. The representation is used to express the physical constraints in stochastic optimization problems for gas flow allocation subject to uncertain parameters. The method is based on the stochastic finite volume approach that was recently developed for uncertainty quantification in transient flows represented by hyperbolic PDEs on graphs. In this study, we develop optimization formulations for steady-state gas flow over actuated transport networks subject to probabilistic constraints. In addition to the distributions for the physical solutions, we examine the dual variables that are produced by way of the optimization, and interpret them as price distributions that quantify the financial volatility that arises through demand uncertainty modeled in an optimization-driven gas market mechanism. We demonstrate the computation and distributional analysis using a single-pipe example and a small test network.

Keywords: Power systems, Markov processes, Simulation
Abstract: Faced with the complexities of managing natural gas-dependent power system amid the surge of renewable integration and load unpredictability, this study explores strategies for navigating emergency transitions to costlier secondary fuels. Our aim is to develop decision-support tools for operators during such exigencies. We approach the problem through a Markov Decision Process (MDP) framework, accounting for multiple uncertainties. These include the potential for dual-fuel generator failures and operator response during high-pressure situations. Additionally, we consider the finite reserves of primary fuel, governed by gas-flow partial differential equations (PDEs) and constrained by nodal pressure. Other factors include the variability in power forecasts due to renewable generation and the economic impact of compulsory load shedding. For tractability, we address the MDP in a simplified context, replacing it by Markov Processes evaluated against a selection of policies and scenarios for comparison. Our study considers two models for the natural gas system: an over-simplified model tracking linepack and a more nuanced model that accounts for gas flow network heterogeneity. The efficacy of our methods is demonstrated using a realistic model replicating Israel's power-gas infrastructure.

Keywords: Computational methods, Energy systems, Optimization
Abstract: In many power systems, particularly those isolated from larger intercontinental grids, reliance on natural gas is crucial. This dependence becomes particularly critical during periods of volatility or scarcity in renewable energy sources, further complicated by unpredictable consumption trends. To ensure the uninterrupted operation of these isolated gas-grid systems, innovative and efficient management strategies are essential. This paper investigates the complexities of achieving synchronized, dynamic, and stochastic optimization for autonomous transmission-level gas-grid infrastructures. We introduce a novel methodology grounded in differentiable programming, which synergizes symbolic programming, a conservative numerical method for solving gas-flow partial differential equations, and automated sensitivity analysis powered by SciML/Julia. Our methodology refines the simulation and co-optimization landscape for gas-grid systems by grounding gas dynamics in physics-adherent simulation. We demonstrate efficiency and precision of the methodology by solving a stochastic optimal gas flow problem, phrased on an open source model of Israel's gas grid model.

Keywords: Energy systems, Modeling, Optimization
Abstract: To match the growing demand for biomethane production, anaerobic digestors need an optimal management of the input diet, and the said diet must not be constrained to a single substrate — that is, co-digestion is required. Co-digestion is far more complicated to govern than single- substrate digestion, and constitutes a very hard challenge for the instrumentation and control equipment typically installed aboard full-scale plants. We propose a solution based on offline trajectory optimisation with the aid of a complex first-principle digestor model extended to embrace the co-digestion of complex substrates, followed by online control with a purpose-specific scheme to mitigate the risk of inhibition. We show simulation results and some preliminary tests on a pilot plant.

Keywords: Control applications, Energy systems, Robust control
Abstract: In this work, the current control of a proton exchange membrane electrolyzer (PEMEL) associated with a DC/DC converter is studied in a renewable energy sources context. Five realistic scenarii are considered including both supplied energy variations (smooth or abrupt) due to intermittent nature of renewable energy sources (RES) and process parameters uncertainties. The model used for the proton exchange membrane electrolyzer coupled to a DC/DC converter that is proposed in a paper and obtained from measurements made on an experimental test bench. To maximize hydrogen production, a two-degree of freedom H-infinity robust controller with integral action on the current is designed by using the normalized coprime factorization approach with loop-shaping. This robust H-infinity controller is compared with the PID designed in the above-mentioned paper. For the five scenarii above-mentioned, the robust H-infinity controller provides satisfactory results, but this is not the case for the PID controller. Thus, it is interesting to employ a more complex control law than the widely-used PID to take into account the problems generated by the use of renewable energy sources to reduce the greenhouse gas emissions.

Keywords: Machine learning, Predictive control for nonlinear systems, Learning
Abstract: This paper proposes a nonlinear model predictive control (NMPC) approach for incrementally input-to-state stable gated recurrent units (GRU) neural networks affected by state and output disturbances. In particular, a Luenberger-like observer is designed for state and disturbance estimation with guaranteed convergence properties. This paves the way for the design of an NMPC regulator capable of rejecting unknown piecewise-constant disturbances. The method is tested in simulation on a nonlinear benchmark system, i.e., a chemical reaction process, showing promising results.

Keywords: Predictive control for nonlinear systems, Constrained control, Neural networks
Abstract: This paper proposes a stabilizing Model Predictive Control algorithm, specifically designed to handle systems learned by Incrementally Input-to-State Stable Recurrent Neural Networks, in presence of input and incremental input constraints. Closed-loop stability is proven by relying on the Incremental Input-to-State Stability property of the model, and on a terminal equality constraint involving the control sequence only. The Incremental Input-to-State Stability is also used to derive a suitable formulation of the Model Predictive Control terminal cost. The proposed control algorithm can be readily applied to a wide range of Recurrent Neural Networks, including Gated Recurrent Units, Echo State Networks, and Neural Nonlinear Autoregressive eXogenous models. Furthermore, this work specializes the approach to handle the particular case of Long Short-Term Memory Networks, and showcases its effectiveness on a four tanks process benchmark.

Keywords: Machine learning, Predictive control for nonlinear systems, Chemical process control
Abstract: In this work, we develop novel machine learning modeling and predictive control techniques for nonlinear chemical systems with asynchronous and delayed measurements in both offline and online data collection. Specifically, Phased Long Short-Term Memory (PLSTM) network is used to learn the process dynamics amidst the irregularities in the data, during the offline training process. The generalization performance of PLSTM is theoretically studied on the basis of statistical machine learning theory to better understand the capabilities of PLSTM models. The PLSTM model is employed to forecast the evolution of states for a Lyapunov-based Model Predictive Controller (LMPC) that is designed to account for data loss and delays in real-time implementation. Finally, an application to a benchmark chemical process is adopted to show the effectiveness of PLSTM modeling and predictive control methods.

Keywords: Data driven control, Predictive control for nonlinear systems, Stability of nonlinear systems
Abstract: In this paper, we consider the design of data-driven predictive controllers for nonlinear systems from input-output data using linear-in-control input Koopman lifted models. Instead of identifying and simulating a Koopman model to predict future outputs, we design a subspace predictive controller in the Koopman space. This allows us to learn the observables minimizing the multi-step output prediction error, preventing the propagation of prediction errors. We compute a terminal cost and terminal set in the Koopman space, and we obtain recursive feasibility guarantees through an interpolated initial state. As a third contribution, we introduce a novel regularization cost yielding input-to-state stability guarantees with respect to the prediction error for the resulting closed-loop system. The performance is illustrated on a nonlinear benchmark example from the literature.

Keywords: Predictive control for nonlinear systems, Data driven control, Stability of nonlinear systems
Abstract: We investigate nonlinear model predictive control (MPC) with terminal conditions in the Koopman framework using extended dynamic mode decomposition (EDMD) to generate a data-based surrogate model for prediction and optimization. We rigorously show recursive feasibility and prove practical asymptotic stability w.r.t. the approximation accuracy. To this end, finite-data error bounds are employed. The construction of the terminal conditions is based on recently derived proportional error bounds to ensure the required Lyapunov decrease. Finally, we illustrate the effectiveness of the proposed data-driven predictive controller including the design procedure to construct the terminal region and controller.

Keywords: Data driven control, Energy systems, Predictive control for nonlinear systems
Abstract: Energy storage solutions are becoming increasingly important due to their capability to mitigate the variability of renewable power generation. Among these, Pumped Thermal Electricity Storage (PTES) offers a grid-level, long-lasting, and environmentally friendly alternative to batteries. However, the dynamic modeling of PTES is challenging due to its complex system architecture, which includes many interdependent components that interact in nonlinear ways. In this paper, we propose a robust data-driven energy management strategy based on Zonotopic Predictive Control (ZPC) and MixedInteger Programming (MIP) for PTES with unknown nonlinear dynamics connected to a residential Multi-Energy System (MES). Data-driven reachable sets, derived from past input-output trajectory data, are constructed as a substitute for a dynamic model. To account for noisy data, a matrix zonotope recursion that over-approximates the reachable sets is employed to ensure robust constraint satisfaction. Simulation results demonstrate the effectiveness and robustness of the approach, showing costefficient storage of off-peak and excess renewable electricity as heat, which is released during peak hours. An average reduction in the renewable curtailment rate from 6.07% to 2.94% is observed due to the use of PTES.

Keywords: Control of networks, Network analysis and control, Decentralized control
Abstract: We study the problem of controlling an ensemble of nonidentical dynamical units to a desired trajectory set by the pinner in the presence of multi-body interactions between the units. We provide necessary and sufficient conditions for local bounded convergence, and estimate the convergence bound as a function of the parameter mismatch between the units, and of the directed hypergraph describing their interacting topology. Numerical simulations on a network of Rössler oscillators are performed to illustrate the robustness of the approach.

Keywords: Control of networks, Lyapunov methods, Markov processes
Abstract: Bipartite synchronization control for Markovian jumping networks with signed graphs is investigated in this paper. The relationships between some nodes are cooperative, and the relationships between another nodes are competitive. In addition, the coupling connections in the considered network exhibit dynamic behavior. The bipartite synchronization problem is formulated as a stabilization problem for the Markovian jumping system, where the analysis based on the Lyapunov stability theory shows that the bipartite synchronization is achieved in the sense that the bipartite synchronization error system is asymptotically stable in the mean square. The effectiveness of the obtained result is shown in a numerical simulation using a cooperation-competition network of single-link robot arms.

Keywords: Control of networks, Network analysis and control, Networked control systems
Abstract: In recent years, the notion of r-robustness for the communication graph of the network has been introduced to address the challenge of achieving consensus in the presence of misbehaving agents. Higher r-robustness typically implies higher tolerance to malicious information towards achieving resilient consensus, but it also implies more edges for the communication graph. This in turn conflicts with the need to minimize communication due to limited resources in real-world applications (e.g., multi-robot networks). In this paper, our contributions are twofold. (a) We provide the necessary subgraph structures and tight lower bounds on the number of edges required for graphs with a given number of nodes to achieve maximum robustness. (b) We then use the results of (a) to introduce two classes of graphs that maintain maximum robustness with the least number of edges. Our work is validated through a series of simulations.

Keywords: Control of networks, Optimization algorithms
Abstract: In this paper, we solve the optimal H₂ control problem distributed over networks with an arbitrary graph, with the assumption that the controller network has no delays but only sparsity constraints. Our solution extends the previous approach valid for strongly connected graphs. Under the quadratic invariance structure assumption, we follow the Youla parameterization framework for the distributed control synthesis setup. We show that the optimal solution is finite-dimensional and can be computed by solving three algebraic Riccati equations: two are standard for centralized H₂ control, while the third emerges from the sparsity constraints imposed by the network. We present a 4-car platoon example for method validation.

Keywords: Control of networks, Optimization
Abstract: Brain networks typically exhibit specific synchronization patterns where several synchronized clusters coexist. On the other hand, neurological disorders are considered to be related to pathological synchronization such as excessive synchronization of large populations of neurons. Motivated by these facts, this paper presents two approaches to control the cluster synchronization of Kuramoto oscillators. One is based on the use of pacemakers to the clusters, and the other is based on feeding back the mean phases to the clusters. We first show conditions on the pacemaker weights and the feedback gains for the network to achieve cluster synchronization. Then, we propose a method to find optimal feedback gains through convex optimization. A numerical example demonstrates the effectiveness of the proposed methods.

Keywords: Control of networks, Uncertain systems, Randomized algorithms
Abstract: This paper studies the problem of modifying the input matrix of a structured system to make the system strongly structurally controllable. We focus on the generalized structured systems that rely on zero/nonzero/arbitrary structure, i.e., some entries of system matrices are zeros, some are nonzero, and the remaining entries can be zero or nonzero (arbitrary). We derive the feasibility conditions of the problem, and if it is feasible, we reformulate it into another equivalent problem. This new formulation leads to a greedy heuristic algorithm. However, we also show that the greedy algorithm can give arbitrarily poor solutions for some special systems. Our alternative approach is a randomized Markov chain Monte Carlo-based algorithm. Unlike the greedy algorithm, this algorithm is guaranteed to converge to an optimal solution with high probability. Finally, we numerically evaluate the algorithms on random graphs to show that the algorithms perform well.

Keywords: Game theory, Agents-based systems, Algebraic/geometric methods
Abstract: This paper presents a method for herding and capturing a swarm of adversarial agents. An open formation, FlexNet, is formed, which can achieve effective capture by adjusting the size and shape of the formation based on the distance between the pursuers and evaders. A capture strategy is given through rigorous analytical proof under a novel interaction model. We demonstrate that the conditions for successful capture and the proposed strategy allow the construction of a continuous dynamics structure to hold. Also, we introduce the FOV model to each pursuer and extend the single-axis motion to the whole 2D environment. Simulations are provided to demonstrate the efficacy of the proposed approach.

Keywords: Game theory, Agents-based systems, Distributed control
Abstract: We study a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies. Unlike conventional population game formulations, we consider a scenario where agents must estimate the payoffs through local measurements and communication with their neighbors. By employing task allocation games -- dynamic extensions of conventional population games -- we examine how errors in payoff estimation by individual agents affect the convergence of the strategy revision process. Our main contribution is an analysis of how estimation errors impact the convergence of the agents' strategy profile to equilibrium. Based on the analytical results, we propose a design for a time-varying strategy revision rate to guarantee convergence. Simulation studies illustrate how the proposed method for updating the revision rate facilitates convergence to equilibrium.

Keywords: Game theory, Agents-based systems, Learning
Abstract: Traditionally based on convexity, multi-agent decision-making models can hardly handle scenarios where agents' cost functions defy this assumption, which is specifically required to ensure the existence of several equilibrium concepts. More recently, the advent of machine learning (ML), with its inherent non-convexity, has changed the conventional approach of pursuing convexity at all costs. This paper explores and integrates the robustness of game theoretic frameworks in managing conflicts among agents with the capacity of ML approaches, such as deep neural networks (DNNs), to capture complex agent behaviors. Specifically, we employ feed-forward DNNs to characterize agents' best response actions rather than modeling their goals with convex functions. We introduce a technical assumption on the weight of the DNN to establish the existence and uniqueness of Nash equilibria and present two distributed algorithms based on fixed-point iterations for their computation. Finally, we demonstrate the practical application of our framework to a noncooperative community of smart energy users under a dynamic time-of-use energy pricing scheme.

Keywords: Game theory, Agents-based systems, Optimization
Abstract: We study mechanisms incentivizing the contribution of selfish agents addressing crowd tasks in a Bayesian setting with externalities due to network effects. A notable example is crowdsensing, where a large group of individuals, with mobile devices capable of sensing and computing collectively, share data. The central problem we investigate is the relationship between direct and indirect mechanisms. Indeed, while direct mechanisms represent tools to address implementation problems optimally, the requirement that the contribution level of every agent when addressing the task is chosen by the mechanism makes these mechanisms hardly used in practice. On the other hand, while indirect mechanisms allow every agent to be free to choose their contribution level, these mechanisms may be highly inefficient. Our desideratum is to design indirect mechanisms that closely match the performance of optimal direct mechanisms. We design an indirect mechanism without network effects such that the Price of Anarchy over the revenue is one. We also extend our results to the case with network effects, providing an upper bound to the Price of Stability and showing that the inefficiency is low in practice. Our results suggest that indirect revelation mechanisms can be an excellent option in real-world applications.

Keywords: Game theory, Agents-based systems, Optimization algorithms
Abstract: In this paper, we present an efficient algorithm to solve online Stackelberg games, featuring multiple followers, in a follower-agnostic manner. Unlike previous works, our approach works even when leader has no knowledge about the followers' utility functions, strategy space or learning algorithm. Our algorithm introduces a unique gradient estimator, leveraging specially designed strategies to probe followers. In a departure from traditional assumptions of optimal play, we model followers' responses using a convergent adaptation rule, allowing for realistic and dynamic interactions. The leader constructs the gradient estimator solely based on observations of followers' actions. We provide both non-asymptotic convergence rates to stationary points of the leader's objective and demonstrate asymptotic convergence to a emph{local Stackelberg equilibrium}. To validate the effectiveness of our algorithm, we use this algorithm to solve the problem of incentive design on a large-scale transportation network, showcasing its robustness even when the leader lacks access to followers' demand information.

Keywords: Game theory, Air traffic management, Agents-based systems
Abstract: Coordination in multiplayer games enables players to avoid the lose-lose outcome that often arises at Nash equilibria. However, designing a coordination mechanism typically requires the consideration of the joint actions of all players, which becomes intractable in large-scale games. We develop a novel coordination mechanism, termed reduced rank correlated equilibria. The idea is to approximate the set of all joint actions with the actions used in a set of pre-computed Nash equilibria via a convex hull operation. In a game with n players and each player having m actions, the proposed mechanism reduces the number of joint actions considered from O(m^n) to O(mn) and thereby mitigates computational complexity. We demonstrate the application of the proposed mechanism to an air traffic queue management problem. Compared with the correlated equilibrium-a popular benchmark coordination mechanism-the proposed approach is capable of solving a queue management problem involving four thousand times more joint actions while yielding similar or better performance in terms of a fairness indicator and showing a maximum optimality gap of 0.066% in terms of the average delay cost. In the meantime, it yields a solution that shows a 58.5% to 99.5% improvement in a fairness indicator and a 1.8% to 50.4% reduction in average delay cost compared to the Nash solution, which does not involve coordination.

Keywords: Optimal control, Computational methods, Variational methods
Abstract: A modified form of Legendre-Gauss orthogonal direct collocation is developed for solving optimal control problems whose solutions are nonsmooth due to control discontinuities. This new method adds switch time variables, control variables, and collocation conditions at both endpoints of a mesh interval, whereas these new variables and collocation conditions are not included in standard Legendre-Gauss orthogonal collocation. The modified Legendre-Gauss collocation method alters the search space of the resulting nonlinear programming problem and optimizes the switch point of the control solution. The transformed adjoint system of the modified Legendre-Gauss collocation method is then derived and shown to satisfy the necessary conditions for optimality. Finally, an example is provided where the optimal control is bang-bang and contains multiple switches. This method is shown to be capable of solving complex optimal control problems with nonsmooth solutions.

Keywords: Optimal control, Constrained control, Variational methods
Abstract: The paper addresses the problem of optimal control design in presence of singular solutions for single input dynamics. The dynamical extension for systems obtained adding an integrator on the input is addressed and analyzed. The possibility of computing the optimal control for dynamically extended systems from the solution of the initial ones is investigated, as well as the inverse procedure. These relationships are well evidenced for the singular solutions, showing the possibility of simplifying the optimal control computation. An example is introduced to better highlight the presented results.

Keywords: Optimal control, Nonlinear systems, Constrained control
Abstract: The motivation of this paper stems from a family of optimal control problems wherein the control’s active duration is constrained within a predefined limit. The duration constraint can be perceived as an additional variable in the dynamics, and the relaxation of the naturally associated control problem is equivalent to a an L1-constraint. The paper provides a generalization of a preliminary work by the authors to encompass scenarios with non-convex dynamics. The relaxed problems are formulated through Linear Programming techniques and their qualitative properties, alongside the inter-relations between different formulations, are investigated.

Keywords: Optimal control, Nonlinear systems, Stability of nonlinear systems
Abstract: This paper introduces a finite-frequency dynamic output-feedback control synthesis approach for nonlinear Lur'e-type control systems, focusing on finite-frequency disturbances. The method addresses the challenge of handling the infinite number of higher harmonics resulting from nonlinearities. Despite the infinite number of higher harmonics, we present a performance bound for the closed-loop nonlinear control system analogues to the L₂-gain for linear time-invariant systems. The synthesis problem then aims to minimise this performance bound while ensuring closed-loop stability through the notion of global asymptotic convergence. A numerical example on a nonlinear mechanical system demonstrates improved performance compared to traditional full-frequency output-feedback control.

Keywords: Optimal control, Nonlinear systems, Numerical algorithms
Abstract: This paper investigates the central role played by the Hamiltonian in continuous-time nonlinear optimal control problems. We show that the strict convexity of the Hamiltonian in the control variable is a sufficient condition for the existence of a unique optimal trajectory, and the nonlinearity/non-convexity of the dynamics and the cost are immaterial. The analysis is extended to discrete-time problems, revealing that discretization destroys the convex Hamiltonian structure, leading to multiple spurious optima, unless the time discretization is sufficiently small. We present simulated results comparing the ``indirect" Iterative Linear Quadratic Regulator (iLQR) and the ``direct" Sequential Quadratic Programming (SQP) approach for solving the optimal control problem for the cartpole and pendulum models to validate the theoretical analysis. Results show that the ILQR always converges to the ``globally" optimum solution while the SQP approach gets stuck in spurious minima given multiple random initial guesses for a time discretization that is insufficiently small, while both converge to the same unique solution if the discretization is sufficiently small.

Keywords: Optimal control, Nonlinear systems
Abstract: A finite-horizon nonlinear optimal control problem is considered. Stat-quad duality is used to generate an equivalent problem with linear dynamics and a modification term in the running cost and two auxiliary controls processes. This problem form is used to obtain a representation of the value function as a staticization problem over a set of quadratic functions, where the coefficients of the quadratics consists of the solution to a differential Riccati equation, a linear ODE and an integral. This representation allows the value function to be evaluated independently at any time and any point in the state space. A specialized numerical method is proposed for solving the resulting staticization problem, which is able to leverage the low dimensionality of nonlinearity. A numerical example with five-dimensional state space is included.

Keywords: Predictive control for linear systems
Abstract: In this paper, a new variant of the predictor-corrector interior point method (IPM) pipeline is proposed for model predictive control (MPC) problems for linear time-invariant systems, which can be reformulated as quadratic programming (QP) problems. At each iteration in the IPM, finding the search direction via solving a linear system of equations is usually the step with the highest computational cost. A modified Uzawa algorithm is developed to improve the performance in the proposed IPM, which can address the ill-conditioning issue at the late iterations and reduce computational cost. Results of an MPC problem example are presented to show the performance of the proposed pipeline.

Keywords: Predictive control for linear systems, Optimal control, Time-varying systems
Abstract: Periodic dynamical systems, distinguished by their repetitive behavior over time, are prevalent across various engineering disciplines. In numerous applications, particularly within industrial contexts, the implementation of model predictive control (MPC) schemes tailored to optimize specific economic criteria was shown to offer substantial advantages. However, the real-time implementation of these schemes is often infeasible due to limited computational resources. To tackle this problem, we propose a resource-efficient economic model predictive control scheme for periodic systems, leveraging existing single-layer MPC techniques. Our method relies on a single quadratic optimization problem, which ensures high computational efficiency for real-time control in dynamic settings. We prove feasibility, stability and convergence to optimum of the proposed approach, and validate the effectiveness through numerical experiments.

Keywords: Predictive control for linear systems, Optimization, Linear systems
Abstract: Model predictive control (MPC) solves a receding-horizon optimization problem in real-time, which can be computationally demanding when there are thousands of constraints. To accelerate online computation of MPC, we utilize data to adaptively remove the constraints while maintaining the MPC policy unchanged. Specifically, we design the removal rule based on the Lipschitz continuity of the MPC policy. This removal rule can use the information of historical data according to the Lipschitz constant and the distance between the current state and historical states. In particular, we provide the explicit expression for calculating the Lipschitz constant by the model parameters. Finally, simulations are performed to validate the effectiveness of the proposed method.

Keywords: Predictive control for linear systems, Optimization algorithms, Embedded systems
Abstract: Model Predictive Control (MPC) is a popular control approach due to its ability to consider constraints, including input and state restrictions, while minimizing a cost function. However, in practice, these constraints can result in feasibility issues, either because the system model is not accurate or due to the existence of external disturbances. To mitigate this problem, a solution adopted by the MPC community is the use of soft constraints. In this article, we consider a not-so-typical methodology to encode soft constraints in a particular MPC formulation known as MPC for Tracking (MPCT), which has several advantages when compared to standard MPC formulations. The motivation behind the proposed encoding is to maintain the semi-banded structure of the ingredients of a recently proposed solver for the considered MPCT formulation, thus providing an efficient and fast solver when compared to alternative approaches from the literature. We show numerical results highlighting the benefits of the formulation and the computational efficiency of the solver.

Keywords: Predictive control for linear systems, Optimization, Linear systems
Abstract: Multi-Horizon model predictive control (MPC) is a method that uses time coarsening to increase the prediction horizon by using several models, each with a different sampling time that gradually increases later in the horizon. This facilitates having a longer prediction interval without significantly impacting the computational load or compromising the response time. However, the use of models with different granularity make guaranteeing recursive feasibility challenging as conventional approaches cannot be applied directly. This work proposes a constraint tightening strategy to enforce recursive feasibility in time-invariant multi-horizon MPC schemes. The state constraint in the optimization is replaced by adaptive state and input constraint at each time step that depend on the sampling time to ensure that the trajectory remains feasible between two sampling points even as the sampling time increases. An extensive numerical study illustrates the effectiveness and scalability of our approach and compares its performance to standard MPC and multi-horizon MPC controllers without any constraint tightening.

Keywords: Predictive control for linear systems, Optimization algorithms, Optimization
Abstract: We propose a combinatorial method for computing explicit solutions to multi-parametric quadratic programs, which can be used to compute explicit control laws for linear model predictive control. In contrast to classical methods, which are based on geometrical adjacency, the proposed method is based on combinatorial adjacency. After introducing the notion of combinatorial adjacency, we show that the explicit solution forms a connected graph in terms of it. We then leverage this connectedness to propose an algorithm that computes the explicit solution. The purely combinatorial nature of the algorithm leads to computational advantages since it enables demanding geometrical operations (such as computing facets of polytopes) to be avoided. Compared with classical combinatorial methods, the proposed method requires fewer combinations to be considered by exploiting combinatorial connectedness. We show that an implementation of the proposed method can yield a speedup of about two orders of magnitude compared with state-of-the-art software packages such as MPT and POP.

Keywords: Boolean control networks and logic networks, Uncertain systems
Abstract: In this paper, we propose reachability analysis using constrained polynomial logical zonotopes. We perform reachability analysis to compute the set of states that could be reached. To do this, we utilize a recently introduced set representation called polynomial logical zonotopes for performing computationally efficient and exact reachability analysis on logical systems. Notably, polynomial logical zonotopes address the "curse of dimensionality" when analyzing the reachability of logical systems since the set representation can represent 2^h binary vectors using h generators. After finishing the reachability analysis, the formal verification involves verifying whether the intersection of the calculated reachable set and the unsafe set is empty or not. Polynomial logical zonotopes lack closure under intersections, prompting the formulation of constrained polynomial logical zonotopes, which preserve the computational efficiency and exactness of polynomial logical zonotopes for reachability analysis while enabling exact intersections. Additionally, an extensive empirical study is presented to demonstrate and validate the advantages of constrained polynomial logical zonotopes.

Keywords: Formal Verification/Synthesis, Autonomous robots
Abstract: This work addresses maximally robust control synthesis under unknown disturbances. We consider a nonlinear system, subject to a Signal Temporal Logic (STL) specification and jointly synthesize the maximal possible disturbance bounds and the corresponding controllers that ensure the STL specification is satisfied under these bounds. Many works have considered STL satisfaction under given bounded disturbances yet, to the authors' best knowledge, this is the first work that aims to maximize the permissible disturbance set and find corresponding maximally robust controllers. We, therefore, introduce disturbance robustness as a model-based robustness metric for STL planning and control synthesis. We extend the notion of disturbance-robust semantics for STL, which is a property of a specification, dynamical system, and controller, and provide an algorithm for maximally robust controllers satisfying an STL specification using Hamilton-Jacobi reachability. We show its soundness and provide a simulation example with an Autonomous Underwater Vehicle (AUV).

Keywords: Data driven control, Learning, Model/Controller reduction
Abstract: Data-Enabled Predictive Control (DeePC) bypasses the need for system identification by directly leveraging raw data to formulate optimal control policies. However, the size of the optimization problem in DeePC grows linearly with respect to the data size, which prohibits its application to resource-constrained systems due to high computational costs. In this paper, we propose an efficient approximation of DeePC, whose size is invariant with respect to the amount of data collected, via differentiable convex programming. Specifically, the optimization problem in DeePC is decomposed into two parts: a control objective and a scoring function that evaluates the likelihood of a guessed I/O sequence, the latter of which is approximated with a size-invariant learned optimization problem. The proposed method is validated through numerical simulations on a quadruple tank system, illustrating that the learned controller can reduce the computational time of DeePC by a factor of 5 while maintaining its control performance.

Keywords: Stochastic systems, Formal Verification/Synthesis, Nonlinear systems
Abstract: Uncertainty propagation in non-linear dynamical systems has become a key problem in various fields including control theory and machine learning. In this work, we focus on discrete-time non-linear stochastic dynamical systems. We present a novel approach to approximate the distribution of the system over a given finite time horizon with a mixture of distributions. The key novelty of our approach is that it not only provides tractable approximations for the distribution of a non-linear stochastic system but also comes with formal guarantees of correctness. In particular, we consider the Total Variation (TV) distance to quantify the distance between two distributions and derive an upper bound on the TV between the distribution of the original system and the approximating mixture distribution derived from our framework. We show that in various cases of interest, including in the case of Gaussian noise, the resulting bound can be efficiently computed in closed form. This allows us to quantify the correctness of the approximation and to optimize the parameters of the resulting mixture distribution to minimize such distance. The effectiveness of our approach is illustrated on several benchmarks from the control community.

Keywords: Constrained control, Cyber-Physical Security, Uncertain systems
Abstract: Cyber-physical systems can be subject to sensor attacks, e.g., sensor spoofing, leading to unsafe behaviors. This paper addresses this problem in the context of linear systems when an omniscient attacker can spoof several system sensors at will. In this adversarial environment, existing results have derived necessary and sufficient conditions under which the state estimation problem has a unique solution. In this work, we consider a severe attacking scenario when such conditions do not hold. To deal with potential state estimation uncertainty, we derive an exact characterization of the set of all possible state estimates. Using the framework of control barrier functions, we propose design principles for system safety in offline and online phases. For the offline phase, we derive conditions on safe sets for all possible sensor attacks that may be encountered during system deployment. For the online phase, with past system measurements collected, a quadratic program-based safety filter is proposed to enforce system safety. A 2D-vehicle example is used to illustrate the theoretical results.

Keywords: Autonomous systems, Stochastic optimal control, Control applications
Abstract: The paper presents Maximal Covariance Backward Reachable Trees (operatorname{MAXCOVAR} BRT), which is a multi-query algorithm for planning of dynamic systems under stochastic motion uncertainty and constraints on the control input with explicit coverage guarantees. In contrast to existing roadmap-based probabilistic planning methods that sample belief nodes randomly and draw edges between them cite{csbrm_tro2024}, under control constraints, the reachability of belief nodes needs to be explicitly established and is determined by checking the feasibility of a non-convex program. Moreover, there is no explicit consideration of coverage of the roadmap while adding nodes and edges during the construction procedure for the existing methods. Our contribution is a novel optimization formulation to add nodes and construct the corresponding edge controllers such that the generated roadmap results in provably maximal coverage under control constraints as compared to any other method of adding nodes and edges. We characterize formally the notion of coverage of a roadmap in this stochastic domain via introduction of the h-operatorname{BRS} (Backward Reachable Set of Distributions) of a tree of distributions under control constraints, and also support our method with extensive simulations on a 6 DoF model.

Keywords: Data driven control, Linear systems, LMIs
Abstract: In this paper we propose a non-conservative dynamic output-feedback controller synthesis method for a class of discrete-time linear time-invariant systems. The synthesis goal is to render the closed-loop system dissipative with respect to a given generic unstructured quadratic supply rate. The model of the plant is assumed to be unknown, but instead we require knowledge of trajectories in the control channel, i.e., of the controlled input and measured output available for control. It is assumed that the trajectories are corrupted by bounded noise. The controller synthesis method is based on a convexification procedure which relies on the dualization lemma.

Keywords: Data driven control, Linear systems, Sampled-data control
Abstract: We set up a continuous-time data-driven control framework based on sampling linear functionals. Under some recently established sufficient conditions for informativity for system identification, we give data-based solutions to stabilization and optimal quadratic regulation problems.

Keywords: Data driven control, Linear systems, Stability of linear systems
Abstract: We study data-driven analysis and control of 2D Fornasini-Marchesini second models assuming that the input and state variable are directly measurable. We give necessary and sufficient conditions for the data to be informative for identification and a 2D version of the “fundamental lemma”. We show how to obtain a system representation from sufficiently informative data, and we propose a data-driven approach for stability verification and state-feedback stabilization via linear matrix inequalities.

Keywords: Data driven control, Predictive control for linear systems, Computational methods
Abstract: We present some preliminary ideas on a data-driven Model Predictive Control framework for continuous-time systems. We use Chebyshev polynomial orthogonal bases to represent system trajectories and subsequently develop a data-driven continuous-time version of the classical Model Predictive Control algorithm. We investigate the effects of the parameters in our framework with two numerical examples and draw comparison to model-driven MPC schemes.

Keywords: Data driven control, Predictive control for linear systems, Linear parameter-varying systems
Abstract: This paper presents a new robust data-driven predictive control scheme for linear time-varying (LTV) systems with unknown nominal system models. To tackle the challenges arising from the unknown nominal model and the time-varying nature of the system, a data-dependent optimization problem is formulated using input-state-output data. It calculates an upper bound on the objective function and, at the same time, designs a state feedback controller to minimize the bound. Moreover, two significant concerns, namely the feasibility of the optimization problem and the stability of the closed-loop system under the designed controller, are thoroughly investigated. Compared with the existing data-enabled predictive control method for LTV systems, the proposed control scheme does not require the collected data to satisfy the persistently exciting (PE) condition and uniformly exponentially stabilizes the system. Finally, a numerical example is provided to demonstrate the effectiveness of the proposed method.

Keywords: Data driven control, Predictive control for linear systems, Stochastic optimal control
Abstract: We consider the problem of direct data-driven predictive control for unknown stochastic linear time-invariant (LTI) systems with partial state observation. Building upon our previous research on data-driven stochastic control, this paper (i) relaxes the assumption of Gaussian process and measurement noise, and (ii) enables optimization of the gain matrix within the affine feedback policy. Output safety constraints are modelled using conditional value-at-risk, and enforced in a distributionally robust sense. Under idealized assumptions, we prove that our proposed data-driven control method yields control inputs identical to those produced by an equivalent model-based stochastic predictive controller. A simulation study illustrates the enhanced performance of our approach over previous designs.

Keywords: Reinforcement learning, Agents-based systems, Numerical algorithms
Abstract: Federated Reinforcement Learning (FRL) provides a promising way to speedup training in reinforcement learning using multiple edge devices that can operate in parallel. Recently, it has been shown that even when these edge devices have access to different dynamic models, an optimal convergence rate that has a linear speedup proportional to the number of devices is achievable. However, this result requires that the stepsize in the algorithm be chosen in a manner dependent on the unknown model parameters. Also, it applies only to a discounted setting, which has been argued to fit episodic tasks better than continuing control tasks. In this paper, we obtain finite-time bounds for heterogeneous FRL with average rewards. We show that the optimal convergence rate with a linear speedup is possible even with a universal stepsize choice, independent of the underlying dynamics. To achieve our result, we modify the existing one-timescale FRL method to a novel two-timescale variant that additionally incorporates iterate averaging.

Keywords: Reinforcement learning, Cooperative control, Autonomous robots
Abstract: Cooperative multi-agent learning plays a crucial role for developing effective strategies to achieve individual or shared objectives in multi-agent teams. In real-world settings, agents may face unexpected failures, such as a robot's leg malfunctioning or a teammate's battery running out. These malfunctions decrease the team's ability to accomplish assigned task(s), especially if they occur after the learning algorithms have already converged onto a collaborative strategy. Current leading approaches in Multi-Agent Reinforcement Learning (MARL) often recover slowly -- if at all -- from such malfunctions. To overcome this limitation, we present the Collaborative Adaptation (CA) framework, highlighting its unique capability to operate in both continuous and discrete domains. Our framework enhances the adaptability of agents to unexpected failures by integrating inter-agent relationships into their learning processes, thereby accelerating the recovery from malfunctions. We evaluated our framework's performance through experiments in both discrete and continuous environments. Empirical results reveal that in scenarios involving unforeseen malfunction, although state-of-the-art algorithms often converge on sub-optimal solutions, the proposed CA framework mitigates and recovers more effectively.

Keywords: Reinforcement learning, Cooperative control, Transportation networks
Abstract: Optimal control methods provide solutions to safety-critical problems but easily become intractable. Control Barrier Functions (CBFs) have emerged as a popular technique that facilitates their solution by provably guaranteeing safety, through their forward invariance property, at the expense of some performance loss. This approach involves defining a performance objective alongside CBF-based safety constraints that must always be enforced with both performance and solution feasibility significantly impacted by two key factors: (i) the selection of the cost function and associated parameters, and (ii) the calibration of parameters within the CBF-based constraints, which capture the trade-off between performance and conservativeness. %as well as infeasibility. To address these challenges, we propose a Reinforcement Learning (RL)-based Receding Horizon Control (RHC) approach leveraging Model Predictive Control (MPC) with CBFs (MPC-CBF). In particular, we parameterize our controller and use bilevel optimization, where RL is used to learn the optimal parameters while MPC computes the optimal control input. We validate our method by applying it to the challenging automated merging control problem for Connected and Automated Vehicles (CAVs) at conflicting roadways. Results demonstrate improved performance and a significant reduction in the number of infeasible cases compared to traditional heuristic approaches used for tuning CBF-based controllers, showcasing the effectiveness of the proposed method. In order to guarantee reproducibility, our code is provided here: https://github.com/EhsanSabouni/CDC2024_RL_adpative_MPC_CBF/tree/main

Keywords: Reinforcement learning, Data driven control, Neural networks
Abstract: We introduce State Planning Policy Reinforcement Learning (SPP-RL), an online RL approach, where the actor plans for the next state given the current state. To communicate the actor output to the environment, we incorporate an inverse dynamics control model and train it using supervised learning. SPP-RL introduces a novel way of ensuring reachability of planned target states by employing constrained optimization and the Lagrange multiplier method. We demonstrate the versatility of the SPP-RL by implementing variants of 3 standard RL algorithms: DDPG, TD3, and SAC. We conduct a thorough evaluation across 7 benchmarks, including Safety-Gym Level 0, AntPush and MuCoJo. Our results consistently showcase that SPP algorithms outperform their vanilla counterparts in terms of the average return in a systematic and significant manner. Moreover, we present SPP-RL convergence proof within a finite setting. Furthermore, we share our source code and trained models.

Keywords: Reinforcement learning, Linear systems, Uncertain systems
Abstract: We study the problem of policy estimation for the Linear Quadratic Regulator (LQR) in discrete-time linear time-invariant uncertain dynamical systems. We propose a Moreau Envelope-based surrogate LQR cost, built from a finite set of realizations of the uncertain system, to define a meta-policy efficiently adjustable to new realizations. Moreover, we design an algorithm to find an approximate first-order stationary point of the meta-LQR cost function. Numerical results show that the proposed approach outperforms naive averaging of controllers on new realizations of the linear system. We also provide empirical evidence that our method has better sample complexity than Model-Agnostic Meta-Learning (MAML) approaches.

Keywords: Reinforcement learning, Manufacturing systems and automation
Abstract: We investigate Deep Reinforcement Learning (DRL) for a large scale manufacturing problem of pallet loop control in an automotive assembly plant with multiple body types assembled in the same facility. The pallet loop comprises of three body construction lines that merges into a complex shared conveyor system and delivers units to the paint shop with empty pallets returning back to the start of construction lines. The decision problem is to dynamically control pallet selection at the various merge points to meet, often highly variable, daily delivery targets while being robust to unanticipated production downtime, part shortages and failures. We discuss the challenges associated with this problem and present solutions developed to successfully train and deploy a DRL control policy. Specifically, we introduce concepts of transient and steady-state reward functions required to address the problem of delayed rewards owing to spatially distributed and temporally shifted control locations. Simulation and real world deployment results show that the trained DRL policy outperforms human designed decision rules in that i) it discovered a control policy with several benefits over base human designed decisions, ii) it demonstrated robustness against uncertain downtime in body construction lines, and iii) it was resilient to variations in delivery targets. We present results from plant performance before and after the implementation of the proposed DRL control policy.

Keywords: Distributed parameter systems, Adaptive systems
Abstract: This paper proposes a synchronization scheme for identical structurally perturbed second order infinite dimensional systems. The structured perturbations modelled by nonlinear functions of the position and velocity outputs are estimated adaptively using a Reproducing Kernel Hilbert Space and their adaptive estimates are used in the synchronization controllers. The kernel space-based adaptive scheme enables the functional estimation of nonlinear perturbation terms without imposing any a priori assumptions on the parametrization of nonlinear perturbation terms. Adaptation of the consensus protocols used for synchronization enable the synchronization of the networked infinite dimensional systems and a model reference control component of the controller ensure each networked system follows an idealized leader.

Keywords: Distributed parameter systems, Cellular dynamics, Backstepping
Abstract: Exploring novel strategies for regulating axon growth, we introduce a periodic event-triggered control (PETC) to enhance the practical implementation of the associated PDE backstepping control law. Neurological injuries may impair neuronal function, but therapies like Chondroitinase ABC (ChABC) have shown promise in improving axon elongation by influencing the extracellular matrix. This matrix, composed of macromolecules and minerals, regulates tubulin concentration, potentially aiding neuronal recovery. The concentration and spatial distribution of tubulin influence axon elongation dynamics. Recent research explores feedback control strategies for this model, leading to the development of an event-triggering control (CETC). In this approach, the control law updates when the triggering condition is met, reducing actuation resource consumption. Through redesigning the triggering mechanism, we introduce PETC, updating control inputs at intervals but evaluating the event trigger periodically, making it ideal for time-sliced actuators like ChABC. PETC is a step forward in designing feasible feedback laws for neuron growth. This strategy sets an upper bound on event triggers between periodic checks, ensuring convergence and preventing Zeno behavior. Through Lyapunov analysis, we demonstrate the local exponential convergence of the system with PETC in the L^2-norm. Numerical examples confirm the theoretical findings.

Keywords: Delay systems, Linear systems, Stability of linear systems
Abstract: In this paper, we design a stabilizing state-feedback control law for a system represented by a general class of integral delay equations subject to a pointwise and distributed input delay. The proposed controller is defined in terms of integrals of the state and input history over a fixed-length time window. We show that the closed-loop stability is guaranteed, provided the controller integral kernels are solutions to a set of Fredholm equations. The existence of solutions is guaranteed under an appropriate spectral controllability assumption, resulting in an implementable stabilizing control law. The proposed methodology appears simpler and more generic compared to existing results in the literature. In particular, under additional regularity assumptions, the proposed approach can be expanded to address the degenerate case where only a distributed control term is present.

Keywords: Backstepping, Distributed parameter systems, Numerical algorithms
Abstract: In this paper, we extend our previous work on the power series method for computing backstepping kernels. Our first contribution is the development of initial steps towards a MATLAB toolbox dedicated to backstepping kernel computation. This toolbox would exploit MATLAB’s linear algebra and sparse matrix manipulation features for enhanced efficiency; our initial findings show considerable improvements in computational speed with respect to the use of symbolical software without loss of precision at high orders. Additionally, we tackle limitations observed in our earlier work, such as slow convergence (due to oscillatory behaviors) and non-converging series (due to loss of analiticity at some singular points). To over- come these challenges, we introduce a technique that mitigates this behaviour by computing the expansion at different points, denoted as localized power series. This approach effectively navigates around singularities, and can also accelerates con- vergence by using more local approximations. Basic examples are provided to demonstrate these enhancements. Although this research is still ongoing, the significant potential and simplicity of the method already establish the power series approach as a viable and versatile solution for solving backstepping kernel equations, benefiting both novel and experienced practitioners in the field. We anticipate that these developments will be particularly beneficial in training the recently introduced neural operators that approximate backstepping kernels and gains.

Keywords: Distributed parameter systems, Stability of nonlinear systems, Sampled-data control
Abstract: This paper introduces a new framework of nonlinear, discrete-time, boundary control systems (BCSs) in the port-Hamiltonian form. The contribution is twofold. We start with a discrete-time approximation of a nonlinear port-Hamiltonian BCS, i.e. a dynamical system modelled by a nonlinear partial differential equation with boundary actuation and sensing. The most important feature is that the discretisation is performed in time only so that the "distributed nature" of the state is preserved. By approximating the gradient of the Hamiltonian density with its discrete gradient, the obtained sampled dynamics inherit the passivity of the original one. Besides, we prove that, under mild conditions, it is well-posed, i.e. the "next" state always exists. The second contribution deals with control design. More precisely, we have determined sufficient conditions for the plant dynamics and a static output feedback gain to make the closed-loop system asymptotically stable.

Keywords: Distributed parameter systems, Lyapunov methods
Abstract: We consider a 1D semilinear reaction-diffusion system with controlled heat flux at one of the boundaries. We design a finite-dimensional state-feedback controller guaranteeing that a given quadratic cost does not exceed a prescribed value for all nonlinearities with a predefined Lipschitz constant. To this end, we perform modal decomposition and truncate the highly damped (residue) modes. To deal with the nonlinearity that couples the residue and dominating modes, we combine the direct Lyapunov approach with the S-procedure and Parseval's identity.} The truncation may lead to spillover: the ignored modes can deteriorate the overall system performance. {Our main contribution is spillover avoidance via the L2 separation of the residue. Namely, we calculate the L2 input-to-state gains for the residue modes and add them to the control weight in the quadratic cost used to design a controller for the dominating modes. A numerical example demonstrates that the proposed idea drastically improves both the admissible Lipschitz constants and guaranteed cost bound compared to the recently introduced direct Lyapunov method.

Keywords: Cyber-Physical Security, Agents-based systems
Abstract: Cybersecurity has in recent years emerged as a paramount concern in the design and operation of industrial systems and civil infrastructures, due mainly to their susceptibility to malicious cyber attacks which take advantage of the vulnerability of communication networks and IT devices. In this paper, we investigate such an attack and counter attack scenario by considering multiagent systems, a somewhat basic prototype of cyberphysical systems. We study false data injection attacks launched on the agent sensors, and possible defense of such attacks at the agent actuators. The primary issue under consideration is the stealthiness of the attacks, while steering a multiagent system away from its consensual state. We propose a metric to quantify the stealthiness, and formulate the stealthiness problem as one of zero-sum games. We solve the problem explicitly, which gives rise to a fundamental bound on the stealthiness achievable, and as well optimal attack and defense strategies that achieve the optimal stealthiness, both of which can be obtained in terms of certain augmented controllability Gramians associated with the agents. The stealthiness bound is seen to depend on agent dynamics and network characteristics including a measure of connectivity.

Keywords: Cyber-Physical Security, Attack Detection, Agents-based systems
Abstract: Multi-agent, collaborative sensor fusion is a vital component of a multi-national intelligence toolkit. In safety-critical and/or contested environments, adversaries may infiltrate and compromise a number of agents. We analyze state of the art multi-target tracking algorithms under this compromised agent threat model. We show that the track existence probability test ("track score") is significantly vulnerable to even small numbers of adversaries. To add security awareness, we design a trust estimation framework using hierarchical Bayesian updating. Our framework builds beliefs of trust on tracks and agents by mapping sensor measurements to trust pseudomeasurements (PSMs) and incorporating prior trust beliefs in a Bayesian context. In case studies, our trust estimation algorithm accurately estimates the trustworthiness of tracks/agents, subject to observability limitations.

Keywords: Cyber-Physical Security, Automata, Discrete event systems
Abstract: The qualitative opacity of a secret is a security property, which means that a system trajectory satisfying the secret is observation-equivalent to a trajectory violating the secret. In this paper, we study how to synthesize a control policy that maximizes the probability of a secret being made opaque against an eavesdropping attacker/observer, while subject to other task performance constraints. In contrast to the existing belief-based approach for opacity-enforcement, we develop an approach that uses the observation function, the secret, and the model of the dynamical systems to construct a so-called opaque-observations automaton that accepts the exact set of observations that enforce opacity. Leveraging this opaque-observations automaton, we can reduce the optimal planning in Markov decision processes(MDPs) for maximizing probabilistic opacity or its dual notion, transparency, subject to task constraints into a constrained planning problem over an augmented-state MDP. Finally, we illustrate the effectiveness of the developed methods in robot motion planning problems with opacity or transparency requirements.

Keywords: Autonomous vehicles, LMIs, Observers for Linear systems
Abstract: This letter introduces a new solution for cyberattack detection within the context of cooperative adaptive cruise Control (CACC) system. Our approach involves a novel method that consists of shifting the original output to establish a new system that adheres to the observer matching condition (OMC). Additionally, we represent the system in descriptor form and design a delayed unknown input observer (UIO) to achieve an Input-to-State Stable (ISS) bound on both the cyberattack and the state of the CACC system. Leveraging Lyapunov stability theory, we propose sufficient conditions expressed in terms of linear matrix inequality. To demonstrate the effectiveness of our algorithm, we proposed two simulation scenarios utilizing both Matlab software and the Carla simulator.

Keywords: Petri nets, Optimization, Cyber-Physical Security
Abstract: This paper provides a necessary and sufficient condition to assess Current State Opacity (CSO) in discrete event systems modeled as bounded Petri nets. The provided condition requires testing a finite number of sequences, without the need to build any observer, neither complete nor reduced. By exploiting the proposed condition, it is then possible to implement an algorithm that requires the solution of a finite number of optimization problems to check whether a system is CSO or not. Therefore, the proposed condition enables to perform CSO assessment by using off-the-shelf commercial software, not suffering from the curse of dimensionality, contrary to the observed-based approaches proposed in literature to deal with opacity. The effectiveness of the proposed approach is shown by means of an example.

Keywords: Optimization, Optimization algorithms, Statistical learning
Abstract: We study how to construct optimization problems whose outcome are sets of feasible, close-to-optimal decisions for human users to pick from, instead of a single, hardly explainable ``optimal'' directive.

In particular, we explore two complementary ways to render convex optimization problems stemming from cyber-physical applications ``flexible''. In doing so, the optimization outcome is a trade off between engineering best and flexibility for the users to decide to do something slightly different. The first method is based on robust optimization and convex reformulations. The second one is stochastic and inspired from stochastic optimization with decision-dependent distributions.

Keywords: Emerging control applications, Control applications, Optimization
Abstract: This paper proposes a feedback control perspective for Human-Earth Systems (HESs) which essentially are complex systems that capture the interactions between humans and nature. Recent attention in HES research has been directed towards devising strategies for climate change mitigation and adaptation, aimed at achieving environmental and societal objectives. However, existing approaches heavily rely on HES models, which inherently suffer from inaccuracies due to the complexity of the system. Moreover, overly detailed models often prove impractical for optimization tasks. We propose a framework inheriting from feedback control strategies the robustness against model errors, because inaccuracies are mitigated using measurements retrieved from the field. The framework comprises two nested control loops. The outer loop computes the optimal inputs to the HES, which are then implemented by actuators controlled in the inner loop. Potential fields of applications are also identified.

Keywords: Emerging control applications, Control applications
Abstract: Control-based continuation (CBC) is a systematic method for exploring the dynamics of nonlinear systems and tracing bifurcation diagrams directly during experimental tests. To find a possible natural response of an underlying system under test, CBC iterates on a control reference signal (usually approximated using Fourier series) to make the controller noninvasive. The many Fourier modes required to accurately represent possible responses of the system lead to high testing times. The main contribution of this paper is to propose a control strategy that can asymptotically track the possible natural responses of a system in a noninvasive manner whilst using approximate reference signals described by short Fourier series. Our approach relies on a time-varying control strategy where noninvasiveness is guaranteed by design, and the mechanism of the controller compensates for the errors in the reference signals due to partial Fourierization. Accordingly, asymptotic tracking of the full Fourier series (i.e., the possible natural responses of the system) based on short Fourier series as reference signals is guaranteed. The proposed control strategy is validated by rigorous analysis and a simulation example.

Keywords: Emerging control applications, Cooperative control, Simulation
Abstract: This paper introduces a new dynamic consensus control strategy for heterogeneous (or nonidentical) DC/DC power converters (e.g. buck converters) utilizing Negative Imaginary (NI) systems theory. The study has been motivated by the application of DC/DC buck converters in supplying an equal amount of current to the energizing coils used in the ship degaussing process. The main challenge for the buck converters is to supply the desired load current to the coils when load demand varies and in the presence of exogenous disturbances. This intrigues the need for designing a cooperative control scheme for a heterogeneous buck converter system to supply the same load current that maybe time-varying. As the average model (i.e. the large-signal model) of a DC/DC buck converter inherently satisfies the strictly NI (SNI) property for any combination of resistance (R), inductance (L), and capacitance (C), a distributed NI control scheme fits well to this problem. This paper has theoretically proved that a homogeneous NI (with a `mixed' property) cooperative control scheme can drive heterogeneous buck converter agents, connected via undirected graphs, supplying the same load current. The simulation studies demonstrate that a simple type-I second-order NI control scheme achieves the desired objectives subject to load variation and exogenous disturbances.

Keywords: Emerging control applications, Optimal control, Game theory
Abstract: This paper analyzes an open-loop Nash equilibrium of climate change policies and long-term mitigation pathways based on Model Predictive Control (MPC) in noncooperative dynamic games between developed and developing regions. After introducing a globally-used mathematical framework for a well-known integrated assessment model (IAM), the Regional Integrated Climate and Economy (RICE) model, with economic evaluations in multi-regional heterogeneity, this paper presents a dynamic noncooperative game between developed and developing regions based on possible international agreements. In particular, through numerical analysis, we show that an MPC-based iterative decision process, selected from the permissible control range to support a prescribed rising surface temperature since the Industrial Revolution, is effective.

Keywords: Emerging control applications, Robust control, LMIs
Abstract: Motivated by how forecast errors exacerbate order fluctuations in supply chains, we leverage robust feedback controller synthesis to characterize, compute, and minimize the worst-case order fluctuation experienced by an individual supply chain vendor. Assuming bounded forecast errors and demand fluctuations, we model forecast error and demand fluctuations as inputs to linear inventory dynamics and use the ell_infty gain to define a transient Bullwhip measure. In contrast to the existing Bullwhip measure, the transient Bullwhip measure explicitly depends on the forecast error. This enables us to separately quantify the transient Bullwhip measure's sensitivity to forecast error and demand fluctuations. To compute the controller that minimizes the worst-case peak gain, we formulate an optimization problem with bilinear matrix inequalities and show that it is equivalent to minimizing a quasi-convex function on a bounded domain. We simulate our model for vendors with non-zero perishable rates and order backlogging rates, and prove that the transient Bullwhip measure can be bounded by a monotonic quasi-convex function whose dependency on the product backlog rate and perishing rate is verified in simulation.

Keywords: Control applications
Abstract: This work presents a new formulation of the integral interconnection and damping-assignment passivity-based control methodology for underactuated mechanical systems subject to both matched and unmatched disturbances, either constant or position-dependent. The new controller is also applicable to systems with non-constant input matrix. Simulations results on two examples demonstrate its effectiveness.

Keywords: Fault detection, Attack Detection, Fault diagnosis
Abstract: Considering potential threats to cyber-physical systems such as component faults and stealthy cyber-attacks, an adaptive observer-based threat discrimination method is proposed for identifying the occurring threat type. Typically, stealthy attacks have only weak effects easily obscured by disturbances on the system outputs. To solve this problem, a parameter adaptation algorithm based on a newly designed dynamic adaptive gain generator is proposed, aiming at improving the sensitivity of the adaptive threat discrimination scheme to potential threats. Only the strictly positive real condition of the proposed gain generator sufficiently ensures the stability of the adaptive observer error system. A moment-matching method is then developed to determine the proper parameters of the gain generator, allowing for the improvement of the sensitivity of the threat discriminators. A numerical example to demonstrate the effectiveness of the proposed methodology is presented.

Keywords: Fault detection, Estimation, Filtering
Abstract: In this paper, we describe an undesirable weak practical super-martingale hallucination phenomenon that can emerge in the Bayesian quickest detection and identification problem. We establish that when measurements are insufficiently informative, a situation described by a relative entropy condition on measurement densities, the Bayesian quickest detection and identification solution can (undesirably) become increasingly confident that a change has occurred, even when it has not. Finally, we illustrate the phenomenon in simulation studies and the vision-based aircraft detection application which illustrates the optimal rule can be unsuitable in the sense of hallucinating a change that has not occurred.

Keywords: Fault detection, Fault diagnosis, Learning
Abstract: Reliable aero-engine anomaly detection is crucial for ensuring aircraft safety and operational efficiency. This research explores the application of the Fisher autoencoder as an unsupervised deep learning method for detecting anomalies in aero-engine multivariate sensor data, using a Gaussian mixture as the prior distribution of the latent space. The proposed method aims to minimize the Fisher divergence between the true and the modeled data distribution in order to train an autoencoder that can capture the normal patterns of aero-engine behavior. The Fisher divergence is robust to model uncertainty, meaning it can handle noisy or incomplete data. The Fisher autoencoder also has well-defined latent space regions, which makes it more generalizable and regularized for various types of aero-engines as well as facilitates diagnostic purposes. The proposed approach improves the accuracy of anomaly detection and reduces false alarms. Simulations using the CMAPSS dataset demonstrate the model's efficacy in achieving timely anomaly detection, even in the case of an unbalanced dataset.

Keywords: Fault detection, Filtering
Abstract: In this paper we present a novel framework for quickly detecting a change in a general dependent stochastic process. We propose that any general dependent Bayesian quickest change detection (QCD) problem can be converted into a hidden Markov model (HMM) QCD problem, provided that a suitable state process can be constructed. The optimal rule for HMM QCD is then a simple threshold test on the posterior probability of a change. We investigate case studies that can be considered structured generalisations of Bayesian HMM QCD problems including: quickly detecting changes in statistically periodic processes and quickest detection of a moving target in a sensor network. Using our framework we pose and establish the optimal rules for these case studies. We also illustrate the performance of our optimal rule on real air traffic data to verify its simplicity and effectiveness in detecting changes.

Keywords: Fault detection, Nonlinear systems
Abstract: To handle complex non-linearity and latent dynamics in industrial processes,this paper proposes a kernel latent vector auto-regressive (K-LaVAR) algorithm for nonlinear dynamic process modeling and monitoring. By combining kernel mapping and the latent dynamic model, the K-LaVAR algorithm enables nonlinear dimension reduction and circumvents the excessively large dimension issue induced by the kernel mapping. In addition, dual monitoring indices are developed to discern normal variations from dynamic and static aspects with respective statistical control limits. A numerical simulation and the revamped Tennessee Eastman Process (TEP) simulation benchmark are adopted to demonstrate the advantages of the K-LaVAR model in dynamic latent variables extraction, overall monitoring performance improvement, and ensuring prompt detection of process disturbances.

Keywords: Fault detection, LMIs, Optimization algorithms
Abstract: In this paper, an observer design problem for a Linear Time-Invariant (LTI) system is studied. An iterative algorithm is proposed as part of the design process,effectively enabling the system to detect fault signals in a finite frequency range. Both the H−-index and H∞-norm are introduced and combined in an observer-based design to generate a residual signal that is sensitive to the fault signal and insensitive to the disturbances over a specified finite frequency range. In particular, our approach enforces an upper bound on the H∞- norm of the disturbances to residual transfer matrix and a lower bound on the H−-index of the faults to residual transfer matrix to ensure that the system achieves optimal performance in detecting all fault signals while limiting the impact of disturbances on the residual within this frequency range. This approach is achieved through the generalized Kalman-Yakubovich-Popov (gKYP) Lemma and uses the Projection Lemma in a novel way to reformulate the problem as a linear matrix inequality (LMI) optimization problem. To address the challenge of finding the best multiplier from the Projection Lemma, an iterative process is designed to obtain a local optimum. The initial solution of this iterative process can be selected from any existing algorithms, leading to an improved version of the observer since each iteration gives a solution which is at least as effective as the previous solution. A numerical example is provided in the last section to illustrate the effectiveness of our approach.

Keywords: Biological systems, Learning, Systems biology
Abstract: Learning in living organisms is typically associated with networks of neurons. The use of large numbers of adjustable units has also been a crucial factor in the continued success of artificial neural networks. In light of the complexity of both living and artificial neural networks, it is surprising to see that very simple organisms – even unicellular organisms that do not possess a nervous system – are capable of certain forms of learning. Since in these cases learning may be implemented with much simpler structures than neural networks, it is natural to ask how simple the building blocks required for basic forms of learning may be. The purpose of this study is to discuss the simplest dynamical systems that model a fundamental form of non-associative learning, habituation, and to elucidate technical implementations of such systems, which may be used to implement non-associative learning in neuromorphic computing and related applications.

Keywords: Biological systems, Nonlinear systems, Network analysis and control
Abstract: Epilepsy is one of the most common neurological disorders globally, affecting millions of individuals. Despite significant advancements, the precise mechanisms underlying this condition remain largely unknown, making accurately predicting and preventing epileptic seizures challenging. In this paper, we employ a bistable model, where a stable equilibrium and a stable limit cycle coexist, to describe epileptic dynamics. The equilibrium captures normal steady-state neural activity, while the stable limit cycle signifies seizure-like oscillations. The noise-driven switch from the equilibrium to the limit cycle characterizes the onset of seizures. The differences in the regions of attraction of these two stable states distinguish epileptic brain dynamics from healthy ones. We analytically construct the regions of attraction for both states. Further, using the notion of input-to-state stability with respect to small inputs, we analytically show how the regions of attraction influence the robustness of the system subject to external perturbations. Generalizing the bistable system into coupled networks, we also find the role of network parameters in shaping the regions of attraction. Our findings shed light on the intricate interplay between brain networks and epileptic activity, offering mechanistic insights into potential avenues for more predictable treatments.

Keywords: Biological systems, Optimal control, Nonlinear systems
Abstract: We investigate optimization of an algal-bacterial consortium, where an exogenous control input modulates bacterial resource allocation between growth and synthesis of a resource that is limiting for algal growth. Maximization of algal biomass synthesis is pursued in a continuous bioreactor, with dilution rate as an additional control variable. We formulate optimal control in the two variants of static and dynamic control problems, and address them by theoretical and numerical tools. We explore convexity of the static problem and uniqueness of its solution, and show that the dynamic problem displays a solution with bang-bang control actions and singular arcs that result in cyclic control actions. We finally discuss the relation among the two solutions and show the extent to which dynamic control can outperform static optimal solutions.

Keywords: Biological systems, Robust control, Data driven control
Abstract: This paper explores the continuity characteristics of value functions associated with optimal control in circadian rhythm entrainment problems. Our results demonstrate that when the optimal objective is to minimize the time required for entrainment, the corresponding value function is not Lips- chitz continuous, suggesting that the optimal cost and optimal trajectory are not robust under perturbation. As an alternative, we propose a new objective function that is based on the cumulative squared tracking error and show that the resulting value function is Lipschitz continuous. Through numerical simulations, we further establish that data-driven feedback control systems exhibit higher robustness to input perturbation when the data are collected from optimal control solutions that minimize the cumulative squared tracking error, as opposed to those that are time-optimal.

Keywords: Biological systems, Stability of nonlinear systems, Stability of hybrid systems
Abstract: The Impulsive Goodwin's Oscillator (IGO) is a mathematical model of a hybrid closed-loop system. It arises by closing a special kind of continuous linear positive time-invariant system with impulsive feedback, which employs both amplitude and frequency pulse modulation. The structure of IGO precludes the existence of equilibria, and all its solutions are oscillatory. With its origin in mathematical biology, the IGO also presents a control paradigm useful in a wide range of applications, in particular dosing of chemicals and medicines. Since the pulse modulation feedback mechanism introduces significant nonlinearity and non-smoothness in the closed-loop dynamics, conventional controller design methods fail to apply. However, the hybrid dynamics of IGO reduce to a nonlinear, time-invariant discrete-time system, exhibiting a one-to-one correspondence between periodic solutions of the original IGO and those of the discrete-time system. The paper proposes a design approach that leverages the linearization of the equivalent discrete-time dynamics in the vicinity of a fixed point. A simple and efficient local stability condition of the 1-cycle in terms of the characteristics of the amplitude and frequency modulation functions is obtained.

Keywords: Biological systems, Stochastic systems, Biomedical
Abstract: We formulate a mechanistic model capturing the dynamics of neurotransmitter release in a chemical synapse. The proposed modeling framework captures key aspects such as the random arrival of action potentials (AP) in the presynaptic (input) neuron, probabilistic docking and release of neurotransmitter-filled vesicles, and clearance of the released neurotransmitter from the synaptic cleft. Analysis of the model results in analytical expressions for the statistical moments of docked vesicle count and the number of neurotransmitters in the cleft as a function of model parameters, and the frequency of presynaptic APs. We next consider a postsynaptic (output) neuron that fires an AP based on integrating upstream neurotransmitter activity. Our results show the existence of an optimal presynaptic AP frequency that maximizes the probability of a postsynaptic AP firing. We extend these results to consider feedforward regulation where in addition to a direct excitatory synapse, the input neuron also impacts the output indirectly via an inhibitory interneuron, and identify parameter regimes where the feedforward neuronal networks result in band-pass filtering, i.e., the output neuron frequency is maximized at intermediate input neuron frequencies.

Keywords: Linear systems, Autonomous systems
Abstract: We show that applying a generalization of the Cayley-Hamilton Theorem to a state-space representation of a single-output, multidimensional (mD), linear, shift-invariant, causal, autonomous system, with distinct eigentuples, results in an equivalent difference equation representation. The proposed method is also applicable to parameterize a set of mD shift-invariant difference equations in terms of a given set of eigenvalues. Lastly, a closed-form expression in terms of the eigenvalues of the system matrices is derived.

Keywords: Linear systems, Computational methods
Abstract: Model-based feedforward is essential to realize the high-precision tracking in advanced manufacturing. The crucial step is to solve the inverse problem, which has unstable solution for non-minimum phase systems. The existing stable inversion method necessitates fixed initial condition and final condition of the tracking task. However, the initial condition and final condition are assigned differently according to diverse requirements in practice. In this paper, a new stable inversion based feedforward control method is proposed. By reformulating the tracking problem in lifted time form and truncating the infinite dimensional inverse operator, the proposed method addresses the inverse problem for stable solution and permits arbitrary initial condition and final condition with rigorous stability analysis and robustness analysis. The implementation of the developed method is presented. The high precision tracking performance of the proposed method is illustrated by simulation.

Keywords: Linear systems, Constrained control, Predictive control for linear systems
Abstract: The maximal admissible set (MAS) of a dynamical system characterizes the set of all initial conditions and constant inputs for which the ensuing response satisfies the specified state/output constraints for all time. For a discrete-time, linear time-invariant (LTI) system subject to polytopic constraints and unknown bounded disturbances, the MAS is known to be a polytope, which may not be finitely determined (i.e., it may not be defined by a finite number of inequalities). Thus, the steady-state constraint is usually tightened, which results in a finitely-determined inner approximation of the MAS. However, the complexity of this approximation is not known apriori from problem data. This paper presents and compares two computationally efficient methods, based on matrix power series and on quadratic ISS-Lyapunov functions, respectively, to upper bound the complexity of the MAS. The bounds may facilitate the online computation of the MAS and the implementation of robust reference governors and model predictive controllers.

Keywords: Linear systems, Data driven control, Stability of linear systems
Abstract: In this work, we derive a data-based test to assess the stability of discrete-time, linear, time-invariant systems directly from finite datasets of noisy input-output trajectories, without the need to estimate the system matrices or the noise statistics. We characterize the performance of our test and we show that, despite inaccuracies in the data due to the system noise, the test provides accurate results when the available data is sufficiently large yet finite (the required amount of data depends on the properties of the system, which we also characterize). These results complement the body of literature on data-driven control and finite-sample analysis, and they provide new ways to assess the stability of control systems that do not assume, nor require the estimation of, a model of the system and noise and do not rely on solving eigenvalue equations.

Keywords: Linear systems, Delay systems, Switched systems
Abstract: In this paper, we address the positive invariance property in linear discrete-time systems in the presence of time-varying delays in the states. We build the study on a recently proposed approach based on an appropriate model transformation which allows the derivation of delay-dependent invariance conditions. For polyhedral sets, we establish necessary and sufficient invariance conditions with respect to the transformed model and prove that such conditions imply the confinement of the state trajectories of the original system in the set for arbitrary realizations of the varying delay, as long as the initial states belong to a set which is positively invariant with respect to an augmented switching system without delay, and can be computed in a finite number of steps known in advance.

Keywords: Linear systems, Differential-algebraic systems, Observers for Linear systems
Abstract: This paper considers the class of positive singular systems and provides a new approach to achieve stabilization and observer design with positivity constraints. First, the connection between singular systems and its equivalent input derivative systems is established and their positivity with stability properties are analyzed. Then, an algebraic transformation is introduced, which allows to eliminate the derivative inputs and to obtain an equivalent standard system. A careful mathematical derivation was performed to obtain closed-form expressions for the coefficient matrices of the resulting transformed system. Consequently, the design of positive stabilization of positive singular systems was made possible using the equivalent standard state space representation. Finally, the design of positive observer for positive singular system is provided. It is shown that a similar procedure is required to eliminate the input derivatives that appear in the output equation allowing the observer to be designed. Both positive stabilization and positive observer designs are formulated and solved in terms of LMIs.

Keywords: Stochastic optimal control, Autonomous systems, Robotics
Abstract: We introduce a nonlinear stochastic model pre- dictive control path integral (MPPI) method that considers chance constraints on system states. The proposed belief-space stochastic MPPI (BSS-MPPI) applies Monte-Carlo sampling to evaluate state distributions resulting from underlying systematic disturbances, and utilizes a Control Barrier Function (CBF) inspired heuristic in belief space to fulfill the specified chance constraints. Compared to several previous stochastic predictive control methods, our approach applies to general nonlinear dy- namics without requiring the computationally expensive system linearization step. Moreover, the BSS-MPPI controller can solve optimization problems without limiting the form of the objective function and chance constraints and is parallelizable. Results on a realistic race-car simulation study show significant reductions in constraint violation compared to some of the prior MPPI approaches, while being comparable in computation times.

Keywords: Stochastic optimal control, Closed-loop identification, Adaptive control
Abstract: This work examines the optimal covariance steering problem for systems subject to unknown parameters that enter multiplicatively with the state and control, in addition to additive disturbances. In contrast to existing works, the unknown parameters are modeled as random variables and are estimated online. A dual control problem is formulated in which the effect of the planned control policy on the parameter estimates is modeled and optimized for. The parameter estimates are then used to modify the pre-computed control policy online in an adaptive control fashion. Finally, the proposed approach is demonstrated in a vehicle control example with closed-loop parameter identification.

Keywords: Stochastic optimal control, Constrained control, Optimal control
Abstract: This article explores different methods of constraining the terminal state of nonlinear stochastic systems and presents the results of the Distributionally Constrained Convex Duality Optimal Control (DC-CDOC) for each case. Specifically, we consider: (a) almost surely equality constraints on the terminal state, (b) constraints on the expected value of the terminal state, and (c) constraints on the probability distributions of the terminal state, both (i) under the total probability measure and (ii) under all conditional probability measures. For each case, the associated optimal control problem is formulated as a convex linear program on the space of Radon measures, and by exploiting the duality relations between the space of measures and that of continuous functions, we derive the optimality conditions in the form of an optimization problem over the space of differentiable functions constrained to a Hamilton-Jacobi (HJ) inequality and, in some of these cases, a terminal value inequality. An iterative algorithm is also proposed for identifying the value function in the studied cases.

Keywords: Stochastic optimal control, Cooperative control, Linear systems
Abstract: We formulate and solve a decentralized optimal control problem with cooperative, coupled linear-quadratic subsystems and additional risk-aware costs depending on the covariance and skew of the disturbance. In contrast to related work, this problem quantifies the variability of the subsystem state energy rather than merely its expectation. To tackle this challenge, we develop an alternative linear-algebraic approach illuminating a family of matrices with compelling properties and interpretation. Notably, this approach facilitates an efficient analytical treatment of coupled subsystems, suggesting its potential applicability to future problems in cooperative control.

Keywords: Stochastic optimal control, Optimization, Filtering
Abstract: This paper studies the problem of developing computationally efficient solutions for steering the distribution of the state of a stochastic, linear dynamical system between two boundary Gaussian distributions in the presence of chance-constraints on the state and control input. It is assumed that the state is only partially available through a measurement model corrupted with noise. The filtered state is reconstructed with a Kalman filter, the chance constraints are reformulated as difference of convex (DC) constraints, and the resulting covariance control problem is reformulated as a DC program, which is solved using successive convexification. The efficiency of the proposed method is illustrated on a double integrator example with varying time horizons, and is compared to other state-of-the-art chance-constrained covariance control methods.

Keywords: Stochastic optimal control, Optimization algorithms, Linear systems
Abstract: In this study, we address optimal control problems focused on steering the probabilistic distribution of state variables in linear dynamical systems. Specifically, we address the problem of controlling the structural properties of Gaussian state distributions to predefined targets at terminal times. This task is not yet explored in existing works that primarily aim to exactly match state distributions. By employing the Gromov-Wasserstein (GW) distance as the terminal cost, we formulate a problem that seeks to align the structure of the state density with that of a desired distribution. This approach allows us to extend the control objectives to capture the distribution's shape. We demonstrate that this complex problem can be reduced to a Difference of Convex (DC) programming, which is efficiently solvable through the DC algorithm. Through numerical experiments, we confirm that the terminal distribution indeed gets closer to the desired structural properties of the target distribution.

Keywords: Algebraic/geometric methods, Nonlinear systems, Computational methods
Abstract: We show that, for quadratizable dynamic nonlinear systems, all the equilibrium points satisfy an augmented system of nonlinear equations obtained by applying exact quadratization to a suitably modified dynamic system. For autonomous polynomial dynamic systems (i.e. with no input) this method can be viewed as a general method of solving systems of polynomial equations, which has a lower computational complexity with respect to the classical one consisting of the Buchberger’s algorithm (that searches for a Groebner basis) followed by variable elimination. As a matter, we show that the augmented system of polynomials obtained by quadratization is always a Groebner basis for the ideal associated to the originary problem. This allows skipping the computationally heavy Buchberger’s algorithm, and applying directly elimination theory in the solution-searching algorithm.

Keywords: Algebraic/geometric methods, Nonlinear systems, Computational methods
Abstract: In this paper, we efficiently compute overapproximating reachable sets for control systems evolving on Lie groups, building off results from monotone systems theory and geometric integration theory. We consider intervals in the tangent space, which describe real sets on the Lie group through the exponential map. A local equivalence between the original system and a system evolving on the Lie algebra allows existing interval reachability techniques to apply in the tangent space. Using interval bounds of the Baker-Campbell-Hausdorff formula, these reachable set estimates are extended to arbitrary time horizons in an efficient Runge-Kutta-Munthe-Kaas integration algorithm. The algorithm is demonstrated through consensus on a torus and attitude control on SO(3).

Keywords: Algebraic/geometric methods, Nonlinear systems, LMIs
Abstract: This paper presents an algorithmic approach towards bounding the peak time-windowed average value attained by a state function along trajectories of a dynamical system. An example includes the maximum average current flowing across a power line in any 5-minute window. The peak time-windowed mean estimation task may be posed as a finite-dimensional but nonconvex optimization problem in terms of an initial condition and stopping time. This problem can be lifted into an infinite-dimensional linear program in occupation measures, where no conservatism is introduced under compactness and dynamical regularity assumptions. The peak time-windowed mean estimation linear program is in turn truncated into a convergent sequence of semidefinite programs using the moment-Sum-of-Squares hierarchy. Bounds of the time-windowed mean are computed for example systems.

Keywords: Algebraic/geometric methods, Nonlinear systems, Lyapunov methods
Abstract: Trajectory tracking for underactuated systems has been heavily studied for several decades. When the system state-space is a Lie group, the group multiplication can be used to define a global error. In this paper, we consider the spatial, or right-invariant, group error in the design of a tracking controller for left-invariant systems. This choice of error is shown to exhibit synchrony; that is, the error kinematics depend linearly on the control input difference. In particular, if the actual system is driven by the desired control signal, then the error is constant. This property is used to propose a simple nonlinear tracking control scheme that is globally stable and locally exponentially stable for a class of persistently exciting trajectories for left-invariant systems on Lie-groups. We explore the example system of a mobile robot and show that in this particular case, the proposed control scheme is almost-globally asymptotically stable.

Keywords: Algebraic/geometric methods, Constrained control, Computational methods
Abstract: This study proposes a method for designing a feedback controller that makes a prescribed set invariant for a given polynomial system. Although the system is restricted to polynomial systems, the class of invariant sets is not limited to algebraic sets; it is the zero sets of nonlinear functions satisfying a specific type of partial differential equations (PDE) with coefficients in polynomials. Based on the algebraic relations between a nonlinear function and its derivatives derived from the PDE, a constructive sufficient condition for the existence of the desired controllers is provided. Using symbolic computation, the controllers can be computed exactly. Numerical examples are provided to illustrate the effectiveness of the proposed method.

Keywords: Algebraic/geometric methods, Observers for nonlinear systems, Kalman filtering
Abstract: The extended Kalman filter (EKF) relies on noise parameters, notably the covariance matrix of the observation noise. To identify them using real data, the standard approach consists in maximizing the likelihood of the EKF’s estimates. To perform the optimization, we propose in this paper to use Amari’s natural gradient descent, in a way that preserves positive semi-definiteness of the covariance parameter. We derive the corresponding equations, and we bring the method to bear on a real-world experiment, where we identify the covariance matrix of a GNSS for a vehicle localization problem.

Keywords:
Abstract: Developing a successful grant proposal is a time consuming and cumbersome process with many steps, uncertainties, and hidden corners. Yet it is key to the growth and success of an educational institution’s faculty and their teaching and research objectives. Having a partner that understands federal and private funding mechanisms and knows how to navigate the various steps of submitting a successful grant proposal is a valuable asset. Especially if such assistance is provided at no cost! In this session we present Quanser's approach to securing funding for academic faculty and how a partnership with Quanser can help you increase the chances of your proposal's success and accessing the tools you need to conduct the proposed work. We will look at examples of previous awarded applications and some of the best practices that were deployed to make them successful. We will also present a few relevant current and upcoming funding opportunities across North America, UK and Europe for your review and consideration.

*The lunch sessions are scheduled to start at 12:10 and conclude at 13:10. This timing is intended to allow for a smooth transition between the morning and afternoon sessions, giving participants sufficient time to navigate between consecutive events.*

Keywords:
Abstract: The Women in Control Committee (WiC) is dedicated to empowering and promoting gender diversity in the Control Systems Society (CSS) by facilitating the development of mentoring and programs to promote the retention, recruitment, and growth of women CSS members. The WiC luncheon at CDC 2024 in Milan, Italy provides the opportunity to network, discuss women's roles in CSS, inspire the next generation of female leaders, and foster collaborations to advance women's leadership.

This special session will celebrate the 30th Anniversary of the CSS Women in Control committee. As such, it will focus on providing a perspective about the past, present and future directions of WiC.

You are cordially invited to attend the Women in Control luncheon to meet, greet and network with colleagues while listening to inspiring speakers.

Keywords: Hybrid systems, Stability of hybrid systems, Supervisory control
Abstract: This tutorial paper introduces hybrid feedback control through a self-contained examination of hybrid control systems modeled by the combination of differential and difference equations with constraints. Using multiple examples, it illustrates the power of hybrid feedback control, which stems from the integration of continuous and discrete dynamics, where state variables update instantaneously at specific events while flowing continuously otherwise. The paper defines hybrid closed-loop systems as interconnected hybrid plants and controllers with designated inputs and outputs, and formalizes their solutions. It summarizes key properties of hybrid systems and reviews various control strategies, including supervisory control with logic variables to select feedback controllers, event- triggered control to minimize control input updates, and strategies using multiple Lyapunov-like functions for stabilization. Pointers to further reading and other strategies in the literature are provided.

Keywords: Hybrid systems, Stability of hybrid systems, Hierarchical control
Abstract: This module of the tutorial presents analysis tools for stability, attractivity, invariance, and robustness of hybrid systems.

Keywords: Hybrid systems, Stability of hybrid systems, Hierarchical control
Abstract: This module of the tutorial introduces a uniting control scheme to solve feed- back control problems with high performance and robustness. This strategy is then recast into a general supervisory control approach.

Keywords: Hybrid systems, Stability of hybrid systems, Hierarchical control
Abstract: This module of the tutorial presents a hybrid systems approach to control of continuous-time systems with intermittent and event- triggered control and measurement updates.

Keywords: Hybrid systems, Stability of hybrid systems, Hierarchical control
Abstract: This module of the tutorial introduces a multiple Lyapunov function control approach based on synergism.

Keywords: Hybrid systems, Stability of hybrid systems, Hierarchical control
Abstract: This module of the tutorial provides an overview of invariant-based control, CLF-based control, passivity-based control, and temporal logic. It concludes with several open problems.

Keywords: Quantum information and control, Model/Controller reduction, Filtering
Abstract: Leveraging an algebraic approach built on minimal realizations and conditional expectations in quantum probability, we propose a method to reduce the dimension of quantum filters in discrete-time, while maintaining the correct distributions on the measurement outcomes and the expectations of some relevant observable. The method is presented for general quantum systems whose dynamics depend on measurement outcomes, hinges on a system-theoretic observability analysis, and is tested on prototypical examples.

Keywords: Quantum information and control, Model/Controller reduction
Abstract: The so-called "adiabatic elimination" of fast decaying degrees of freedom in open quantum systems can be performed with a series expansion in the timescale separation. The associated computations are significantly more difficult when the remaining degrees of freedom (center manifold) follow fast unitary dynamics instead of just being slow. This paper highlights how a formulation with Sylvester's equation and with adjoint dynamics leads to systematic, explicit expressions at high orders for settings of physical interest.

Keywords: Quantum information and control, Optimal control, Nonlinear systems
Abstract: We address the problem of cooling a Markovian quantum system to a pure state in the shortest amount of time possible. Here, the system drift takes the form of a Lindblad master equation and we assume fast unitary control. This setting allows for a natural reduction of the control system to the eigenvalues of the state density matrix. We give a simple necessary and sufficient characterization of systems which are (asymptotically) coolable and present a powerful maximum principle, based on majorization, which allows to considerably simplify the search for optimal cooling solutions. With these tools at our disposal, we derive explicit, provably time-optimal cooling protocols for rank one qubit systems, inverted lambda-systems on a qutrit, and a certain system consisting of two coupled qubits.

Keywords: Quantum information and control
Abstract: In this article, we discuss how a three-level closed quantum system with dispersed parameters can be steered between eigenstates via a scalar control. The technique exploits a dynamical decoupling of the control based on the rotating wave approximation, which works under suitable conditions on the spectral gaps of the system and on the bounds on the parameter dispersion. We test numerically the sharpness of the conditions on several examples.

Keywords: Quantum information and control
Abstract: We analyze the problem of reconstructing an unknown quantum state of a multipartite system from repeated measurements of local observables. In particular, via a system-theoretic observability analysis, we show that, even when the initial state is not uniquely determined for a static system, this can be reconstructed if we leverage the system’s dynamics. The choice of dynamical generators and the effect of finite samples is discussed, along with an illustrative example.

Keywords: Quantum information and control, Cyber-Physical Security, Control Systems Privacy
Abstract: Gentle quantum leakage is proposed as a measure of information leakage to arbitrary eavesdroppers that aim to avoid detection. Gentle (also sometimes referred to as weak or non-demolition) measurements are used to encode the desire of the eavesdropper to evade detection. The gentle quantum leakage meets important axioms proposed for measures of information leakage including positivity, independence, and unitary invariance. Global depolarizing noise, an important family of physical noise in quantum devices, is shown to reduce gentle quantum leakage (and hence can be used as a mechanism to ensure privacy or security). A lower bound for the gentle quantum leakage based on asymmetric approximate cloning is presented. This lower bound relates information leakage to mutual incompatibility of quantum states. A numerical example, based on the encoding in the celebrated BB84 quantum key distribution algorithm, is used to demonstrate the results.

Keywords: Game theory, Learning, Agents-based systems
Abstract: Matching algorithms have demonstrated great success in several practical applications, but they often require centralized coordination and plentiful information. In many modern online marketplaces, agents must independently seek out and match with another using little to no information. For these kinds of settings, can we design decentralized, limited-information matching algorithms that preserve the desirable properties of standard centralized techniques? In this work, we constructively answer this question in the affirmative. We model a two-sided matching market as a game consisting of two disjoint sets of agents, referred to as proposers and acceptors, each of whom seeks to match with their most preferable partner on the opposite side of the market. However, each proposer has no knowledge of their own preferences, so they must learn their preferences while forming matches in the market. We present a simple online learning rule that guarantees a strong notion of probabilistic convergence to the welfare-maximizing equilibrium of the game, referred to as the proposer-optimal stable match. To the best of our knowledge, this represents the first completely decoupled, communication-free algorithm that guarantees probabilistic convergence to an optimal stable match, irrespective of the structure of the matching market.

Keywords: Neural networks, Distributed control, Networked control systems
Abstract: In this paper, we propose the Liquid-Graph Time-constant (LGTC) network, a continuous graph neural network (GNN) model for multi-agent system control based on the recent Liquid Time Constant (LTC) network. We analyse its stability leveraging contraction analysis and propose a closed-form model that preserves the model contraction rate and does not require solving an ODE at each iteration. Compared to discrete models like Graph Gated Neural Networks (GGNNs), the higher expressivity of the proposed model guarantees remarkable performance while reducing the large amount of communicated variables normally required by GNNs. We evaluate our model on a distributed multi-agent control case study (flocking) taking into account variable communication range and scalability under non-instantaneous communication.

Keywords: Optimal control, Learning, Linear systems
Abstract: We address the problem of designing an LQR controller in a distributed setting, where M similar but not identical systems share their locally computed policy gradient (PG) estimates with a server that aggregates the estimates and computes a controller that, on average, performs well on all systems. Learning in a distributed setting has the potential to offer statistical benefits -- multiple datasets can be leveraged simultaneously to produce more accurate policy gradient estimates. However, the interplay of heterogeneous trajectory data and varying levels of local computational power introduce bias to the aggregated PG descent direction, and prevents us from fully exploiting the parallelism in the distributed computation. The latter stems from synchronous aggregation, where straggler systems negatively impact the runtime. To address this, we propose an asynchronous policy gradient algorithm for LQR control design. By carefully controlling the "staleness" in the asynchronous aggregation, we show that the designed controller converges to each system's epsilon-near optimal controller up to a heterogeneity bias. Furthermore, we prove exact local convergence at a sub-linear rate.

Keywords: Cyber-Physical Security, Switched systems, Identification
Abstract: We investigate the problem of deceiving a malicious agent employing an identification method to estimate the closed-loop dynamics of a cyber-physical system. In particular, we propose a moving target defense mechanism that utilizes stochastic switching between linear closed-loop dynamics to drive a linear system identification process of a potential adversary to sub-optimal solutions with non-vanishing error. We provide a statistical analysis of the induced identification error and show that it is not possible for any linear system identification method to reconstruct the average dynamics of a stochastic switched linear system. Finally, we utilize the theory of Markov jump linear systems to guarantee asymptotic stability of the switching system, and formulate the switching control problem as an optimization problem that guarantees stability while taking into account the trade-off between security and switching effort. Simulation results showcase the efficacy of the proposed approach in inducing identification error for the adversary using minimal switching.

Keywords: Attack Detection, Networked control systems, Robust control
Abstract: The intrinsic, sampling, and enforced zero dynamics attacks (ZDAs) are among the most detrimental stealthy attacks in robotics, aerospace, and cyber-physical systems. They exploit internal dynamics, discretization, redundancy/asynchronous actuation and sensing, to construct disruptive attacks that are completely stealthy in the measurement. They work even when the systems are both controllable and observable. This paper presents a novel framework to robustly and optimally detect and mitigate ZDAs for networked linear control systems. We utilize controllability, observability, robustness, and sensitivity metrics written explicitly in terms of the system topology, thereby proposing a robust and optimal switching topology formulation for resilient ZDA detection and mitigation. Our main contribution is the reformulation of this problem into an equivalent rank-constrained optimization problem (i.e., optimization with a convex objective function subject to convex constraints and rank constraints), which can be solved using convex rank minimization approaches. The effectiveness of our method is demonstrated using networked double integrators subject to ZDAs.

Keywords: Agents-based systems, Cooperative control, Machine learning
Abstract: We consider a cooperative multiplayer bandit learning problem where the players are only allowed to agree on a strategy beforehand, but cannot communicate during the learning process. In this problem, each player simultaneously selects an action. Based on the actions selected by all players, the team of players receives a reward. The actions of all the players are commonly observed. However, each player receives a noisy version of the reward which cannot be shared with other players. Since players receive potentially different rewards, there is an asymmetry in the information used to select their actions. In this paper, we provide an algorithm based on upper and lower confidence bounds that the players can use to select their optimal actions despite the asymmetry in the reward information. We show that this algorithm is asymptotically optimal in T. We also show that it performs empirically better than the current state-of-the-art algorithm for this environment.

Keywords: Energy systems, Power systems, Optimization
Abstract: Energy prices and net power injection limitations regulate the operations in distribution grids and typically ensure that operational constraints are met. Nevertheless, unexpected or prolonged abnormal events could undermine the grid's functioning. During contingencies, customers could contribute effectively to sustaining the network by providing services. This paper proposes an incentive mechanism that promotes users' active participation by essentially altering the energy pricing rule. The incentives are modeled via a linear function whose parameters can be computed by the system operator (SO) by solving an optimization problem. Feedback-based optimization algorithms are then proposed to seek optimal incentives by leveraging measurements from the grid, even in the case when the SO does not have a full grid and customer information. Numerical simulations on a standard testbed validate the proposed approach.

Keywords: Power systems, Data driven control, Switched systems
Abstract: This paper presents a novel data-driven control algorithm that coordinates a large aggregation of heterogeneous thermostatically controlled loads (TCLs) with unknown temperature dynamics and disturbance distributions to provide balancing services to the grid. Each TCL is subject to local constraints on their temperatures due to user comfort and global constraints on the number of TCLs in a mode, called mode-counting constraints, for the safe operation of the distribution network. We first develop a data-driven method to compute sets of modes at each discretized interval of the temperature state space in which the probability of moving outside of the temperature dead-band at the next step is bounded at a certain confidence level. Subsequently, we design a model predictive control (MPC) algorithm that instructs every TCL to switch to the modes within the obtained set while satisfying the mode-counting constraints. A case study compares the proposed control algorithm with a benchmark and verifies its effectiveness in maintaining end-user comfort and the safe operation of the distribution network.

Keywords: Power systems, Optimization, Energy systems
Abstract: There is growing interest in understanding how interactions between system-wide objectives and local community decision-making will impact the clean energy transition. The concept of energysheds has gained traction in the areas of public policy and social science as a way to study these relationships. However, development of technical definitions of energysheds that permit system analysis are still largely missing. In this work, we propose a mathematical definition for energysheds, and introduce an analytical framework for studying energyshed concepts within the context of future electric power system operations. This framework is used to develop insights into the factors that impact a community’s ability to achieve energyshed policy incentives within a larger connected power grid, as well as the tradeoffs associated with different spatial policy requirements. We also propose an optimization-based energyshed policy design problem, and show that it can be solved to global optimality within arbitrary precision by employing concepts from quasi-convex optimization. Finally, we investigate how interconnected energysheds can cooperatively achieve their objectives in bulk power system operations.

Keywords: Power systems, Optimization, Modeling
Abstract: This paper introduces Inverse Distributionally Robust Optimization (I-DRO) as a method to infer the conservativeness level of a decision-maker, represented by the size of a Wasserstein metric-based ambiguity set, from the optimal decisions made using Forward Distributionally Robust Optimization (F-DRO). By leveraging the Karush-Kuhn-Tucker (KKT) conditions of the convex F-DRO model, we formulate I-DRO as a bi-linear program, which can be solved using off-the-shelf optimization solvers. Additionally, this formulation exhibits several advantageous properties. We demonstrate that I-DRO not only guarantees the existence and uniqueness of an optimal solution but also establishes the necessary and sufficient conditions for this optimal solution to accurately match the actual conservativeness level in F-DRO. Furthermore, we identify three extreme scenarios that may impact I-DRO effectiveness. Our case study applies F-DRO for power system scheduling under uncertainty and employs I-DRO to recover the conservativeness level of system operators. Numerical experiments based on an IEEE 5-bus system and a realistic NYISO 11-zone system demonstrate I-DRO performance in both normal and extreme scenarios.

Keywords: Energy systems, Large-scale systems, Decentralized control
Abstract: The proliferation of behind-the-meter (BTM) distributed energy resources (DER) within the electrical distribution network presents significant supply and demand flexibilities, but also introduces operational challenges such as voltage spikes and reverse power flows. In response, this paper proposes a network-aware dynamic pricing framework tailored for energy-sharing coalitions that aggregate small, but ubiquitous, BTM DER downstream of a distribution system operator's (DSO) revenue meter that adopts a generic net energy metering (NEM) tariff. By formulating a Stackelberg game between the energy-sharing market leader and its prosumers, we show that the dynamic pricing policy induces the prosumers toward a network-safe operation and decentrally maximizes the energy-sharing social welfare. The dynamic pricing mechanism involves a combination of a locational ex-ante dynamic price and an ex-post allocation, both of which are functions of the energy sharing's BTM DER. The ex-post allocation is proportionate to the price differential between the DSO NEM price and the energy-sharing locational price. Simulation results using real DER data and the IEEE 13-bus test systems illustrate the dynamic nature of network-aware pricing at each bus, and its impact on voltage.

Keywords: Smart grid, Optimization, Statistical learning
Abstract: Information asymmetry between the Distribution System Operator (DSO) and Distributed Energy Resource Aggregators (DERAs) obstructs designing effective incentives for voltage regulation. To capture this effect, we employ a Stackelberg game-theoretic framework, where the DSO seeks to overcome the information asymmetry and refine its incentive strategies by learning from DERA behavior over multiple iterations. We introduce a model-based online learning algorithm for the DSO, aimed at inferring the relationship between incentives and DERA responses. Given the uncertain nature of these responses, we also propose a distributionally robust incentive design model to control the probability of voltage regulation failure and then reformulate it into a convex problem. This model allows the DSO to periodically revise distribution assumptions on uncertain parameters in the decision model of the DERA. Finally, we present a gradient-based method that permits the DSO to adaptively modify its conservativeness level, measured by the size of a Wasserstein metric-based ambiguity set, according to historical voltage regulation performance. The effectiveness of our proposed method is demonstrated through numerical experiments.

Keywords: Optimal control, Switched systems, Numerical algorithms
Abstract: Abstract— The Finite Elements with Switch Detection (FESD) method is a highly accurate direct transcription method for optimal control of several classes of nonsmooth dynamical systems. This paper extends the FESD method to Projected Dynamical Systems (PDS) and first-order sweeping processes with time-independent sets. FESD discretizes an equivalent dynamic complementarity system and exploits the structure of the discontinuities present in the continuous-time complementarities. This is achieved by allowing integration step sizes to be degrees of freedom, and introducing additional complementarity constraints, enabling the exact detection of nonsmooth events. In contrast to fixed step Runge-Kutta integrators, the guaranteed smoothness of the system on each step allows for the recovery of full-order integration accuracy and the correct computation of numerical sensitivities. Numerical examples illustrate the effectiveness of the proposed method in simulation and optimal control contexts. FESD for PDS and examples are implemented in the open-source software package nosnoc.

Keywords: Optimal control, Hybrid systems, Optimization algorithms
Abstract: In this study, we propose a novel penalty reformulation and numerical solution method for optimal control problems (OCPs) of linear complementarity systems. The proposed reformulation aims to construct a penalty term tailored to the complementarity constraints using the D-gap function. The proposed penalty term is nonconvex but exhibits a convexity structure that can be utilized by convexity-exploiting solution methods. To solve the reformulated OCP efficiently, we propose a solution method using the sequential convex quadratic programming framework. The convexity of subproblems is guaranteed by a pre-computed regularization matrix using the parameter of the D-gap function. The proposed method is globalized using a merit line search strategy. We confirmed the effectiveness of the proposed method using a benchmark test in comparison with several state-of-the-art methods.

Keywords: Optimal control, Optimization, Constrained control
Abstract: We address the problem of optimal control of a nonsmooth dynamical system described by an evolution variational inequality. We consider both the discrete-time and continuous-time versions of the problem and we relax the problem in the space of measures. We show that there is no gap between the original finite-dimensional problem and the relaxed problem. We show the convergence of the relaxed discrete-time optimal control in measures to continuous-time optimal control in measures. This paves the way to a sound implementation of the moment sum-of-squares hierarchy to solve numerically the optimal control of nonsmooth dynamical systems.

Keywords: Neural networks, Optimal control, Switched systems
Abstract: Neural networks (NN) have been successfully applied to approximate various types of complex control laws, resulting in low-complexity NN-based controllers that are fast to evaluate. However, when approximating control laws using NN, performance and stability guarantees of the original controller may not be preserved. Recently, it has been shown that it is possible to provide such guarantees for linear systems with NN-based controllers by analyzing the approximation error with respect to a stabilizing base-line controller or by computing reachable sets of the closed-loop system. The latter has the advantage of not requiring a base-line controller. In this paper, we show that similar ideas can be used to analyze the closed-loop behavior of piecewise affine (PWA) systems with an NN-based controller. Our approach builds on computing over-approximations of reachable sets using mixed-integer linear programming, which allows to certify that the closed-loop system converges to a small set containing the origin while satisfying input and state constraints. We also show how to modify a given NN-based controller to ensure asymptotic stability for the controlled PWA system.

Keywords: Switched systems, Optimal control
Abstract: This paper extends the Finite Elements with Switch Detection and Jumps (FESD-J) [1] method to problems of rigid body dynamics involving patch contacts. The FESD-J method is a high accuracy discretization scheme suitable for use in direct optimal control of nonsmooth mechanical systems. It detects dynamic switches exactly in time and, thereby, maintains the integration order of the underlying Runge- Kutta (RK) method. This is in contrast to commonly used time-stepping methods which only achieve first-order accuracy. Considering rigid bodies with possible patch contacts results in nondifferentiable signed distance functions (SDF), which introduces additional nonsmoothness into the dynamical system. In this work, we utilize so-called equivalent contact points (ECP), which parameterize force and impulse distributions on contact patches by evaluation at single points. We embed a nondifferentiable SDF into a complementarity Lagrangian system (CLS) and show that the determined ECP are well-defined. We then extend the FESD-J discretization to the considered CLS such that its integration accuracy is maintained. The functionality of the method is illustrated for both a simulation and an optimal control example.

Keywords: Optimal control, Networked control systems, Optimization algorithms
Abstract: Distributed optimisation algorithms are used in a wide variety of problems where the goal is for a set of agents in a network to solve a network-wide optimisation problem via decentralised update rules. Here, we use inverse optimal control to show that a broad class of augmented primal-dual distributed optimisation algorithms can be viewed as the optimal solution to a network-wide optimal control problem, thus providing a performance metric for such methods. The result is shown when inequality constraints are also present, which is a setting that requires additional consideration due to the non-smooth nature of the algorithm dynamics.

Keywords: Networked control systems, Agents-based systems, Modeling
Abstract: Frequency synchronization of bounded confidence Kuramoto oscillators is analyzed. The dynamics of each oscillator is defined by the average of the phase differences with its neighbors, where any two oscillators are considered neighbors if their geodesic distance is less than a certain confidence threshold. A phase-dependent graph is defined whose nodes and edges represent the oscillators and their connections, respectively. It is studied how the connectivity of the graph influences steady-state behaviors of the oscillators. It is proved that the oscillators synchronize asymptotically if the subgraph of each partition, possibly not complete, eventually remains constant over time. Simulation results show the application of the theoretical findings also in the presence of oscillators having different natural frequencies.

Keywords: Networked control systems, Biomedical, Control applications
Abstract: In this letter, an epidemic dynamical model driven by perceived disease severity opinion in societies is investigated along with its various dynamical characteristics. More specifically, the epidemic model namely Susceptible-Infected-Recovered-Vaccinated (SIRV) is considered over a transmission network, while the opinion reflecting the perceived disease risk evolves over a social network. In particular, the global and the local stability conditions of the disease-free equilibrium (DFE), i.e., there is no disease in the network, have been investigated, wherein the local stability is revealed to be linked with the basic reproduction rate and the transverse (non-zero) eigenvalues of the Jacobian evaluated at the DFE points. Moreover, the local stability analysis of the endemic equilibrium (EE), i.e., where disease persists in the network, has been investigated. The simulation results verify the theoretical methods.

Keywords: Networked control systems, Control over communications, Autonomous vehicles
Abstract: We study predecessor-following platoons in which each vehicle-to-vehicle (V2V) communication is affected by a different probability of successful transmission. We model the overall platoon as a stochastic hybrid system, and analyse its stochastic L_2 string stability via a small-gain approach. We provide an explicit string stability condition that illustrates the interplay between the channel success probabilities, transmission rate, and time headway constant. We illustrate our findings through numerical simulations.

Keywords: Networked control systems, Decentralized control, Sensor networks
Abstract: This paper presents a coverage control strategy for a team of aerial robots equipped with downward facing cameras. Based on the observation that the resolution of a camera mounted on an aerial robot degrades with the altitude of the robot, we propose a decentralized gradient-based controller that allows each robot to trade off between the size of the area it monitors and the quality of sensing it performs over the area. Moreover, the proposed controller drives a team of robots to a configuration that maximizes the joint probability for detecting targets or events of interest in the coverage domain. To ensure inter-robot collision avoidance during deployment, we utilize control barrier functions to prevent the robots from getting closer to each other than a specified safety distance. The proposed controller is experimentally validated in simulations.

Keywords: Networked control systems, Delay systems, Attack Detection
Abstract: We address the problem of event-triggered networked control of nonlinear systems under simultaneous deception and Denial-of-Service (DoS) attacks. By DoS attacks, we refer to disruptions in the communication channel that prevent sensor measurements from reaching the controller. When the system undergoes a deception attack, the controller receives a modified output, deviating from the sensor's original measurement. We implement the input delay approach and the Lyapunov--Krasovskii technique to obtain sufficient conditions, expressed in terms of linear matrix inequalities (LMIs), that characterize the duration of the DoS interruptions under which input-to-state stability (ISS) of the closed-loop system is preserved.

Furthermore, we explore scenarios involving simultaneous attacks, where the DoS is modeled as a stochastic Bernoulli process. The closed-loop system is then considered as a stochastic impulsive system. In a similar manner, we derive conditions to ensure mean-square ISS for this case. A numerical example illustrates the efficiency of the results.

Keywords: Networked control systems, Delay systems, Sampled-data control
Abstract: The preservation of the global exponential stability property for the class of fully nonlinear retarded globally Lipschitz systems with time-varying delays under suitable fast sampling is proven through this manuscript. Two main classes of time-varying delays are considered: i) piece-wise constant state delays and ii) continuous-time Lipschitz state delays. Halanay's inequality along with the equivalence between the piece-wise constant delay global exponential stability and measurable delay global exponential stability properties are exploited to demonstrate these results.

Keywords: Game theory
Abstract: When users access shared resources in a selfish manner, the resulting societal cost and perceived users' cost is often higher than what would result from a centrally coordinated optimal allocation. While several contributions in mechanism design manage to steer the aggregate users choices to the desired optimum by using monetary tolls, such approaches bear the inherent drawback of discriminating against users with a lower income. More recently, incentive schemes based on artificial currencies have been studied with the goal of achieving a system-optimal resource allocation that is also fair. In this resource-sharing context, this paper focuses on repeated weighted congestion game with two resources, where users contribute to the congestion to different extents that are captured by individual weights. First, we address the broad concept of fairness by providing a rigorous mathematical characterization of the distinct societal metrics of equity and equality, i.e., the concepts of providing equal outcomes and equal opportunities, respectively. Second, we devise weight-dependent and time-invariant optimal pricing policies to maximize equity and equality, and prove convergence of the aggregate user choices to the system-optimum. In our framework it is always possible to achieve system-optimal allocations with perfect equity, while the maximum equality that can be reached may not be perfect, which is also shown via numerical simulations.

Keywords: Game theory
Abstract: Definition of weak Pareto improvement is given for noncooperative systems with vector-valued payoff functions. The region where a system trajectory is Pareto improving is characerized. Some necessary and sufficient conditions are given for when the system trajectory is (weak) Pareto improving in the entire state space. We provide several numerical examples to illustrate our results.

Keywords: Game theory
Abstract: It has been previously shown that a finite well-mixed population of individuals imitating the highest earners in a binary game can undergo perpetual fluctuations. However, it remains unknown whether the fluctuations in the population proportions of the two strategies persist as population size grows. In this paper, we answer this question for an imitative population with diagonal anticoordination matrices. We show that the collection of Markov chains corresponding to the population dynamics is a family of generalized stochastic approximation process for a good upper semicontinuous differential inclusion. We additionally show that the differential inclusion always converges to an equilibrium. This convergence, based on the available results in the stochastic approximation theory, implies that the lengths of the fluctuations in the population proportions of the two strategies in a finite population of imitators with diagonal anticoordination payoff matrices vanish with probability one as population size grows. Furthermore, taking the same steps for a population of imitators with diagonal coordination payoff matrices results in a similar conclusion, which is consistent with the previously reported results for finite populations of imitators with coordination payoff matrices.

Keywords: Game theory, Optimization algorithms, Agents-based systems
Abstract: We develop a best-response algorithm for solving constrained Markov games assuming limited violations for the potential game property. The limited violations of the potential game property mean that changes in value function due to unilateral policy alterations can be measured by the potential function up to an error alpha. We show the existence of stationary epsilon-approximate constrained Nash policy whenever the set of feasible stationary policies is non-empty. Our setting has agents accessing an efficient probably approximately correct solver for a constrained Markov decision process which they use for generating best-response policies against the other agents' former policies. For an accuracy threshold epsilon>4alpha, the best-response dynamics generate provable convergence to epsilon-Nash policy in finite time with probability at least 1-delta at the expense of polynomial bounds on sample complexity that scales with the reciprocal of epsilon and delta.

Keywords: Game theory, Optimization algorithms, Randomized algorithms
Abstract: This paper deals with distributed Nash equilib- rium seeking in n-cluster games with zero-order information. In such games agents have an access only to the values of their own cost functions and can communicate with their neighbors in the same cluster. The agents within each cluster are cooperative and intend to minimize the overall cluster’s cost. This cost depends, however, on actions of agents from other clusters. Thus, there is a game between the clusters. We present a fully distributed gradient play algorithm to solve this game. The algorithm does not require agents to have any knowledge about action sets, actions or cost functions of others and is based on the zero- order information in the system. We prove convergence of the proposed procedure and estimate its convergence rate which turns out to be optimal up to a logarithmic term in the class of problems under consideration.

Keywords: Game theory, Optimization algorithms, Stochastic systems
Abstract: We focus on problems of stochastic generalized Nash equilibrium seeking with joint constraints and expectation-valued operators. In this work, the stochastic variance-reduced gradient (SVRG) technique is modified to contend with infinite sample space and then, a stochastic forward-backward-forward splitting scheme with variance reduction (DVRSFBF) is proposed for resolving structured monotone inclusion problems. In DVRSFBF, the average gradient is computed periodically in the outer loop, while only cheap sampling is required in the frequently activated inner loop, thus achieving significant speedups when sampling costs cannot be overlooked. The algorithm is fully distributed and it guarantees almost sure convergence under appropriate batch size and strong monotonicity assumptions. Moreover, it exhibits a linear rate with possible biased estimators, which is relatively mild and adopted by many optimization schemes especially for those based on simulations. A numerical study on a class of networked Cournot games reflects the performance of DVRSFBF.

Keywords: Optimal control, Linear systems, Compartmental and Positive systems
Abstract: An optimal control problem on finite-dimensional positive cones is stated. Under a critical assumption on the cone, the corresponding Bellman equation is satisfied by a linear function, which can be computed by convex optimization. A separate theorem relates the assumption on the cone to the existence of minimal elements in certain subsets of the dual cone. Three special cases are derived as examples. The first one, where the positive cone is the set of positive semi-definite matrices, reduces to standard linear quadratic control. The second one, where the positive cone is a polyhedron, reduces to a recent result on optimal control of positive systems. The third special case corresponds to linear quadratic control with additional structure, such as spatial invariance.

Keywords: Optimal control, Linear systems, Robust control
Abstract: The Linear Quadratic Regulator (LQR) and the H-infinity control problems for linear systems are revisited with the objective of deriving a novel algebraic (polynomial) equation alternative to the standard Algebraic Riccati Equation (ARE). Differently from the latter, the former is envisioned to involve the policy alone, in place of the value function as in the ARE. The resulting equation, referred to as the Policy Algebraic Equation, contains nm variables and equations, of order less than or equal to 2n, where n and m denote the dimension of the state and the input, respectively.

Keywords: Optimal control, Linear systems
Abstract: We show that for piecewise affine controlled systems, the bang-bang principle does not hold even when the dynamics is continuous with respect to the state variable. However, we show that there exist minimal time trajectories with extreme controls at the loci where the dynamics is differentiable. We give two examples which exhibit singular arcs exactly at the loci of non-differentiability.

Keywords: Optimal control, LMIs, Time-varying systems
Abstract: We present a simple and effective way to account for non-convex costs and constraints in~state feedback synthesis, and an interpretation for the variables in which state feedback synthesis is typically convex. We achieve this by deriving the controller design using moment matrices of state and input. It turns out that this approach allows the consideration of non-convex constraints by relaxing them as expectation constraints, and that the variables in which state feedback synthesis is typically convexified can be identified with blocks of these moment matrices.

Keywords: Optimal control, Predictive control for linear systems, Optimization
Abstract: An approach based on model predictive control is presented for the scheduling of sowings in an adaptive vertical farm, that is, a pioneering vertical greenhouse where shelf spacing is adjusted automatically according to the growth of crops. We consider the case in which the greenhouse is used for the simultaneous cultivation of multiple types of crops, with plants that may deviate from the ideal growth curve due to suboptimal temperature or humidity (modeled as a system noise). Model predictive control is used to account for such possible deviations and determine the best time instants to perform sowings in the various shelves composing the greenhouse, with the aim of maximizing the production yield. Simulation results are presented in a case study involving the simultaneous cultivation of three different types of crops. They showcase the effectiveness of the proposed method in maximizing production yield while effectively using almost all the vertical space available in the greenhouse, with various control horizons and types of disturbances.

Keywords: Optimal control, Robust control, LMIs
Abstract: We propose a multi-scale approach for computing abstractions of dynamical systems, that incorporates both local and global optimal control to construct a goal-specific abstraction. For a local optimal control problem, we not only design the controller ensuring the transition between every two subsets (cells) of the state space but also incorporate the volume and shape of these cells into the optimization process. This integrated approach enables the design of non-uniform cells, effectively reducing the complexity of the abstraction. These local optimal controllers are then combined into a digraph, which is globally optimized to obtain the entire trajectory. The global optimizer attempts to lazily build the abstraction along the optimal trajectory, which is less affected by an increase in the number of dimensions. Since the optimal trajectory is generally unknown in practice, we propose a methodology based on the RRT* algorithm to determine it incrementally. Finally, we provide a tractable implementation of this algorithm for the optimal control of L-smooth nonlinear dynamical systems.

Keywords: Predictive control for linear systems, Linear parameter-varying systems, Robust control
Abstract: This paper presents a method for robust model predictive control (MPC) of linear parameter varying (LPV) systems considering control policies that are affine functions of the parameter, which is possible when only the `A' and not the `B' matrix depends on the uncertain parameter (LPV-A systems). This is less conservative than formulations in which the policy is restricted to perturbations on a feedback law, as it includes such policies as a special case. State and input constraints are handled efficiently by bounding predicted states in a sequence of polyhedra (i.e. tube MPC), that are parameterised by variables in the online optimisation. The resulting controller can be implemented by online solution of a single quadratic programming problem and can exploit rate bounds on the LPV parameters, which requires a pre-processing step at each iteration. Recursive feasibility and exponential stability are proven and the approach is compared to existing methods in numerical examples drawn from other publications, showing reduced conservatism and improved regions of attraction.

Keywords: Predictive control for linear systems, Linear systems, Algebraic/geometric methods
Abstract: A novel parameterization of the tube associated with the Robust Model Predictive Control law is proposed. The aim is to reduce the computational complexity by describing the tube as a sequence of elastically-scaled zonotopic sets. The developments demonstrate the efficacy within the context of constrained control of a system affected by additive and bounded exogenous disturbances. The underlying set containment conditions using zonotopic sets result in linear complexity.

Keywords: Predictive control for linear systems, Networked control systems, Optimal control
Abstract: This paper addresses the problem of controlling constrained systems subject to disturbances in the case where controller and system are connected over a lossy network. To do so, we propose a novel framework that splits the concept of tube-based model predictive control into two parts. One runs locally on the system and is responsible for disturbance rejection, while the other runs remotely and provides optimal input trajectories that satisfy the system's state and input constraints. Key to our approach is the presence of a nominal model and an ancillary controller on the local system. Theoretical guarantees regarding the recursive feasibility and the tracking capabilities in the presence of disturbances and packet losses in both directions are provided. To test the efficacy of the proposed approach, we compare it to a state-of-the-art solution in the case of controlling a cartpole system. Extensive simulations are carried out with both linearized and nonlinear system dynamics, as well as different packet loss probabilities and disturbances. The code for this work is available at https://github.com/EricssonResearch/Robust-Tracking-MPC-ove r-Lossy-Networks

Keywords: Predictive control for linear systems, Networked control systems, Resilient Control Systems
Abstract: In this paper, a constrained regulation problem for networked control systems, whose plants are described by polytopic linear descriptions with a partial state availability, is considered when the communication medium is unreliable. To address the issue, an ad-hoc state-estimate control architecture has been deployed via a robust model predictive control strategy that is resilient in achieving regulation goals. Additionally, the proposed scheme is designed to prevent communication disconnections in situations where reaching the origin goal is not viable. A final solid numerical example puts in light the effectiveness and the main benefits of the proposed solution.

Keywords: Predictive control for linear systems, Algebraic/geometric methods
Abstract: Solving the explicit model predictive control (MPC) problem entails enumerating a list of critical regions and their ancillary feedback laws. Unfortunately, their number and the time required to compute them increase exponentially with the problem size (state-space model dimension and length of the prediction horizon). We show that when, as it is often the case, the problem's constraints take the form of boxes or zonotopes, the resulting feasible domain can be compactly described as a constrained zonotope. Subsequently, we investigate whether, and under which circumstances, the combinatorial structure of the constrained zonotope interpretation accelerates the computation of the explicit solution.

Keywords: Predictive control for linear systems, Constrained control, Algebraic/geometric methods
Abstract: The maximal positive invariant (MPI) set results from a finite set recurrence instantiated by the intersection of input and state bounds (e.g., the stage constraints of the linear model predictive control problem). When these constraints take the form of hyper-rectangles, zonotopes or constrained zonotopes, the resulting polyhedral MPI set may be succinctly described as a constrained zonotope, eliminating the need for explicit enumeration of its halfspaces. In this paper we discuss the various MPI computation algorithms (with both exact and sufficient stop conditions), recasted in the framework of constrained zonotopes. We analyze one of these variations over a dynamical system whose dimension can be arbitrarily increased in order to asses changes in computation time and storage requirements with respect to the polyhedral case (under the simplifying assumption of closed-loop invertibility of the state matrix).

Keywords: Agents-based systems, Reinforcement learning, Machine learning
Abstract: Off-Policy Prediction (OPP), i.e., predicting the outcomes of a target policy using only data collected under a nominal (behavioural) policy, is a paramount problem in data-driven analysis of safety-critical systems where the deployment of a new policy may be unsafe. To achieve dependable off-policy predictions, recent work on Conformal Off-Policy Prediction (COPP) leverage the conformal prediction framework to derive prediction regions with probabilistic guarantees under the target process. Existing COPP methods can account for the distribution shifts induced by policy switching, but are limited to single-agent systems and scalar outcomes (e.g., rewards). In this work, we introduce MA-COPP, the first conformal prediction method to solve OPP problems involving multi-agent systems, deriving joint prediction regions for all agents' trajectories when one or more ego agents change their policies. Unlike the single-agent scenario, this setting introduces higher complexity as the distribution shifts affect predictions for all agents, not just the ego agents, and the prediction task involves full multi-dimensional trajectories, not just reward values. A key contribution of MA-COPP is to avoid enumeration or exhaustive search of the output space of agent trajectories, which is instead required by existing COPP methods to construct the prediction region. We achieve this by showing that an over-approximation of the true joint prediction region (JPR) can be constructed, without enumeration, from the maximum density ratio of the JPR trajectories. We evaluate the effectiveness of MA-COPP in multi-agent systems from the PettingZoo library and the F1TENTH autonomous racing environment, achieving nominal coverage in higher dimensions and various shift settings.

Keywords: Machine learning, Optimization algorithms, Stochastic systems
Abstract: We consider the problems of estimation and optimization of utility-based shortfall risk (UBSR), which is a popular risk measure in finance. In the context of UBSR estimation, we derive a non-asymptotic bound on the mean-squared error of the classical sample average approximation (SAA) of UBSR. Next, in the context of UBSR optimization, we derive an expression for the UBSR gradient under a smooth parameterization. This expression is a ratio of expectations, both of which involve the UBSR. We use SAA for the numerator as well as denominator in the UBSR gradient expression to arrive at a biased gradient estimator. We derive non-asymptotic bounds on the estimation error, which show that our gradient estimator is asymptotically unbiased. We incorporate the aforementioned gradient estimator into a stochastic gradient (SG) algorithm for UBSR optimization. Finally, we derive non-asymptotic bounds that quantify the rate of convergence of our SG algorithm for UBSR optimization.

Keywords: Stochastic optimal control, Robust control, Uncertain systems
Abstract: Precise control under uncertainty requires a good understanding and characterization of the noise affecting the system. This paper studies the problem of steering state distributions of dynamical systems subject to partially known uncertainties. We model the distributional uncertainty of the noise process in terms of Wasserstein ambiguity sets, which, based on recent results, have been shown to be an effective means of capturing and propagating uncertainty through stochastic LTI systems. To this end, we propagate the distributional uncertainty of the state through the dynamical system, and, using an affine feedback control law, we steer the ambiguity set of the state to a prescribed, terminal ambiguity set. We also enforce distributionally robust CVaR constraints for the transient motion of the state so as to reside within a prescribed constraint space. The resulting optimization problem is formulated as a semi-definite program, which can be solved efficiently using standard off-the-shelf solvers. We illustrate the proposed distributionally-robust framework on a path planning problem and compare with non-robust solutions.

Keywords: Randomized algorithms, Uncertain systems, Robotics
Abstract: With the pervasiveness of Stochastic Shortest-Path (SSP) problems in high-risk industries, such as last-mile autonomous delivery and supply chain management, robust planning algorithms are crucial for ensuring successful task completion while mitigating hazardous outcomes. Mainstream chance-constrained incremental sampling techniques for solving SSP problems tend to be overly conservative and typically do not consider the likelihood of undesirable tail events. We propose an alternative risk-aware approach inspired by the asymptotically-optimal Rapidly-Exploring Random Trees (RRT*) planning algorithm, which selects nodes along path segments with minimal Conditional Value-at-Risk (CVaR). Our motivation rests on the step-wise coherence of the CVaR risk measure and the optimal substructure of the SSP problem. Thus, optimizing with respect to the CVaR at each sampling iteration necessarily leads to an optimal path in the limit of the sample size. We validate our approach via numerical path planning experiments in a two-dimensional grid world with obstacles and stochastic path-segment lengths. Our simulation results show that incorporating risk into the tree growth process yields paths with lengths that are significantly less sensitive to variations in the noise parameter, or equivalently, paths that are more robust to environmental uncertainty. Algorithmic analyses reveal similar query time and memory space complexity to the baseline RRT* procedure, with only a marginal increase in processing time. This increase is offset by significantly lower noise sensitivity and reduced planner failure rates.

Keywords: Autonomous vehicles, Optimal control, Uncertain systems
Abstract: In this paper, we develop an uncertainty-aware decision-making and motion-planning method for an autonomous ego vehicle in forced merging scenarios, considering the motion uncertainty of surrounding vehicles. The method dynamically captures the uncertainty of surrounding vehicles by online estimation of their acceleration bounds, enabling a reactive but rapid understanding of the uncertainty characteristics of the surrounding vehicles. By leveraging these estimated bounds, a non-conservative forward occupancy of surrounding vehicles is predicted over a horizon, which is incorporated in both the decision-making process and the motion-planning strategy, to enhance the resilience and safety of the planned reference trajectory. The method successfully fulfills the tasks in challenging forced merging scenarios, and the properties are illustrated by comparison with several alternative approaches.

Keywords: Autonomous systems, Learning, Uncertain systems
Abstract: We present a novel perspective on the design, use, and role of uncertainty measures for learned modules in an autonomous system. While in the current literature uncertainty measures are produced for standalone modules without considering the broader system context, in our work we explicitly consider the role of decision-making under uncertainty in illuminating how "good" an uncertainty measure is. Our insights are centered around quantifying the ways in which being uncertainty-aware makes a system more robust. Firstly, we use level set generation tools to produce a measure for system robustness and use this measure to compare system designs, thus placing uncertainty quantification in the context of system performance and evaluation metrics. Secondly, we use the concept of specification generation from systems theory to produce a formulation under which a designer can simultaneously constrain the properties of an uncertainty measure and analyze the efficacy of the decision-making-under-uncertainty algorithm used by the system. We apply our analyses to two real-world and complex autonomous systems, one for autonomous driving and another for aircraft runway incursion detection, helping to form a toolbox for an uncertainty-aware system designer to produce more effective and robust systems.

Keywords: Data driven control, Nonlinear systems identification, Identification for control
Abstract: A data-driven ultimate boundedness controller for a nonlinear system is proposed. The controller is designed based on the inverse model of the system identified by Gaussian process regression with state/input measurement data. In particular, the controller actively selects a reference point from the current state with the data used for the identification to achieve a desired ultimate bound. For this reason, the controller is named as the path generating inverse Gaussian process regression (PGIGP) controller. We provide a sufficient condition on the data under which the PGIGP controller guarantees the ultimate boundedness of the closed-loop system with a desired ultimate bound. It is shown that the condition can serve as a guide for data acquisition and, conversely, be employed to establish the best control performance achievable from a given data. The performance of the PGIGP controller is demonstrated through numerical simulations.

Keywords: Data driven control, Robust control, Lyapunov methods
Abstract: This paper proposes a framework to perform verifiably safe control of all discrete-time non-linear systems that are compatible with collected data. Most safety-maintaining control synthesis algorithms (e.g., control barrier functions, density functions) are limited to obtaining theoretical guarantees of safety in continuous-time, even while their implementation on real systems is typically in discrete-time. We first present a sum-of-squares based program to prove the existence of an (acausal) control policy that can safely stabilize all possible data-consistent systems. Causal control policies may be extracted by online optimization, and we provide sufficient conditions for the extraction of this control policy in general scenarios and address as a specific case when convexity assumptions are imposed on the candidate Lyapunov function and safety region descriptor. Discrete-time safe stabilization is demonstrated on two example systems.

Keywords: Data driven control, Sampled-data control
Abstract: This paper investigates the data-driven control (DDC) problem for a class of nonlinear systems satisfying the linear growth condition. The studied system has completely unknown model parameters and mismatched nonlinearity and input. A static linear state-feedback domination controller is proposed to make the closed-loop system globally exponentially stable, which is obtained by offline solving the data-based mixed integer programs (MIPs). Compared to the existing DDC methods, our approach can handle high-order nonlinear systems with nontriangular structures. In contrast to adaptive control methods that require introducing adaptive parameters or Nussbaum functions for handling unknown parameters in the control path, resulting in a dynamic nonlinear controller, our proposed approach only requires offline computation to achieve the desired control objectives using a static linear controller. Two numerical examples are given to illustrate the effectiveness of the proposed method.

Keywords: Data driven control, Stability of nonlinear systems, Lyapunov methods
Abstract: Most existing work on direct data-driven stabilization considers the equilibrium at the origin. When the desired equilibrium is not the origin, existing data-driven approaches often require performing coordinate transformation, or adding integrator action to the controller. As an alternative, this work addresses data-driven state feedback stabilization of any given assignable equilibrium via dissipativity theory. We show that for a polynomial system, if a data-driven stabilizer can be designed to render the origin globally asymptotically stable, then by modifying the stabilizer, we obtain a stabilizer for any given assignable equilibrium.

Keywords: Data driven control, Stability of nonlinear systems, Uncertain systems
Abstract: We consider the problem of designing a controller for an unknown bilinear system using only noisy input-states data points generated by it. The controller should in principle achieve regulation to a given state setpoint and provide a guaranteed basin of attraction. Determining the equilibrium input to achieve that setpoint is not trivial in a data-based setting and we propose the design of a controller in two scenarios. The design takes the form of linear matrix inequalities and is validated numerically for a Cuk converter.

Keywords: Data driven control, Mechatronics, Flexible structures
Abstract: Assembling lattices from discrete building blocks enables the composition of large, heterogeneous, and easily reconfigurable objects with desirable mass-to-stiffness ratios. This type of building system may also be referred to as a digital material, as it is constituted from discrete, error-correcting components. Researchers have demonstrated various active structures and even robotic systems that take advantage of the reconfigurable, mass-efficient properties of discrete lattice structures. However, the existing literature has predominantly used open-loop control strategies, limiting the performance of the presented systems. In this paper, we present a novel approach to feedback control of digital lattice structures, leveraging real-time measurements of the system dynamics. We introduce an actuated voxel which constitutes a novel means for actuation of lattice structures. Our control method is based on the Extended Dynamical Mode Decomposition algorithm in conjunction with the Linear Quadratic Regulator and the Koopman Model Predictive Control. The key advantage of our approach lies in its purely data-driven nature, without the need for any prior knowledge of a system's structure. We illustrate the developed method via real experiments with custom-built flexible lattice beam, showing its ability to accomplish various tasks even with minimal sensing and actuation resources. In particular, we address two problems: stabilization together with disturbance attenuation, and reference tracking.

Keywords: Reinforcement learning, Machine learning, Optimization
Abstract: This paper investigates how to incorporate expert observations (without explicit information on expert actions) into a deep reinforcement learning setting to improve sample efficiency. First, we formulate an augmented policy loss combining a maximum entropy reinforcement learning objective with a behavioral cloning loss that leverages a forward dynamics model. Then, we propose an algorithm that automatically adjusts the weights of each component in the augmented loss function. Experiments on a variety of continuous control tasks demonstrate that the proposed algorithm outperforms various benchmarks by effectively utilizing available expert observations.