Keywords: Cyber-Physical Security, Robotics
Abstract: Enabling autonomy for robotic and cyber-physical systems with provable safety and resilience guarantees has been an ongoing area of research. Despite significant progress over the years, there are still open challenges due to constraints (e.g., safety and time specifications; sensing, computation and communication limitations), and environmental uncertainty. This plenary talk will present some of our recent results and ongoing work on a framework that interconnects control, planning and learning methods towards provably-correct safety-critical robotic and aerospace systems under constraints and uncertainty.

Keywords: Iterative learning control, Adaptive systems, Learning
Abstract: This paper gives a tutorial on iterative learning control nearly five decades after what is widely regarded as the first substantive paper in the literature. The focus is on algorithm development under a number of general headings (linear, optimization, frequency domain, and nonlinear), together with supporting experimental validation/industrial applications and also applications in healthcare.

Keywords: Iterative learning control, Adaptive systems, Learning
Abstract: This presentation introduces the idea of repetitive processes, including ILC as a special case, along with the key results on the control of such processes.

Keywords: Iterative learning control, Adaptive systems, Learning
Abstract: Controller design depends on the particular application at hand. The aim of this tutorial is to provide an overview of the design, analysis, and applications of frequency-domain iterative learning controller (ILC) design. The method is based on frequency response functions, hence it is particularly suitable for systems that are typically modelled in the frequency domain. It is shown how to develop ILC algorithms that converge in a fast and robust way to the limits of the system performance. Indeed, the achievable performance is up to measurement noise level.

Keywords: Iterative learning control, Adaptive systems, Learning 
Abstract: This presentation discusses how to design optimization-based ILC algorithms, with an emphasis on the norm-optimal paradigm. An experimental case study of a robotic arm is used to illustrate the effectiveness of the ideas.

Keywords: Iterative learning control, Adaptive systems, Learning 
Abstract: This paper introduces the concept of using composite energy functions to address nonlinear dynamic systems. To illustrate these ideas, we consider a nonlinear dynamic system that satisfies the global Lipschitz continuity condition. Additionally, the paper explores applications in healthcare, with a focus on stroke rehabilitation.

Keywords: Iterative learning control, Adaptive systems, Learning 
Abstract: New trends in ILC are discussed in this presentation, including data-driven techniques and a discussion of AI in ILC.

Keywords: Human-in-the-loop control, Modeling
Abstract: The paper presents a computational model-based optimization framework for function and attention allocation in collaborative control and decision-making between a human and artificial intelligence (AI). Effective human-AI collaboration (HAC) may depend on structured adaptive function allocation among team members to enhance performance while managing human cognitive limitations, especially attention. Integrating attention allocation is vital for maintaining situation awareness and managing human workload. Various allocation methods rely on heuristics and experimental studies that demand significant resources and domain expertise. To address the function and attention allocation problem in HAC in a systematic way, we propose a computational cognition-work model (CCWM)-based framework. The framework can integrate a qualitative work model and cognitive models to simulate complex team dynamics in temporal semantics. An optimization technique can then improve any task-oriented metrics by exploring the team structure and simulated episodes without requiring exhaustive experimental studies. We present numerical evaluations to demonstrate the proposed framework using a disaster relief drone fleet operation scenario, which provides valuable insights into the early stages of HAC design and the broader domain of HAC.

Keywords: Autonomous vehicles, Transportation networks, Human-in-the-loop control
Abstract: Mixed traffic flows on rings are flows composed of human-driven vehicles and automated vehicles driving on a circular path. A ring is meant to approximate long strings with automated vehicles sparsely inserted into the traffic. Different from existing ring analysis methods based on the equilibrium inter-vehicle distance and equilibrium velocity, this work presents an analysis based on the spacing error and relative velocity error of each vehicle with respect to its predecessor. We refer to such analysis as look-ahead analysis. The benefit of the proposed look-ahead analysis is to directly reflect the feedback variables needed to perform the driving task: indeed, no matter if estimated using visual feedback or available through on-board sensing, the human-driven and automated vehicles exploit feedback variables measured by each vehicle with respect to the preceding one. The analysis focuses on stability of the flow, controllability via the automated vehicle, and impact of disturbances on the flow.

Keywords: Human-in-the-loop control, Stochastic optimal control, Emerging control applications
Abstract: Many cyber-physical-human systems (CPHSs) involve a human decision-maker who acts using recommendations from an artificial intelligence (AI) platform. In such CPHS applications, the human decision-maker may depart from an optimal recommended decision and instead implement a different one for various reasons, resulting in a loss in performance. In this letter, we develop a rigorous framework to overcome this challenge. In our framework, humans may deviate from AI recommendations as they interpret the system's state differently to the AI platform. We establish the structural properties of optimal recommendation strategies and develop an approximate human model (AHM) used by the AI. We provide theoretical bounds on the optimality gap that arises from an AHM and illustrate the efficacy of our results in a numerical example.

Keywords: Autonomous systems, Stochastic optimal control, Neural networks
Abstract: This paper studies the problem of autonomous agents performing Bayesian social learning for sequential detection when the observations of the state belong to a high-dimensional space and are expensive to analyze. Specifically, when the observations are textual, the Bayesian agent can use a large language model (LLM) as a map to get a low-dimensional private observation. The agent performs Bayesian learning and takes an action that minimizes the expected cost and is visible to subsequent agents. We prove that a sequence of such Bayesian agents herd in finite time to the public belief and take the same action disregarding the private observations. We propose a stopping time formulation for quickest time herding in social learning and optimally balance privacy and herding. Structural results are shown on the threshold nature of the optimal policy to the stopping time problem. We illustrate the application of our framework when autonomous Bayesian detectors aim to sequentially identify if a user is a hate speech peddler on an online platform by parsing text observations using an LLM. We numerically validate our results on real-world hate speech datasets. We show that autonomous Bayesian agents designed to flag hate speech peddlers in online platforms herd and misclassify the users when the public prior is strong. We also numerically show the effect of a threshold policy in delaying herding.

Keywords: Human-in-the-loop control, Machine learning, Learning
Abstract: While humans intuitively excel at classifying words according to their connotation, transcribing this innate skill into algorithms remains challenging. We present a human-guided methodology to learn binary word sentiment classifiers from fewer interactions with humans. We introduce a human perception model that relates the perceived sentiment of a word to the distance between the word and the unknown classifier. Our model informs the design of queries that capture more nuanced information than traditional queries solely requesting labels. Together with active learning strategies, our approach reduces human effort without sacrificing learning fidelity. We validate our method through experiments with human data, demonstrating improved accuracy in binary sentiment word classification.

Keywords: Automotive systems, Autonomous vehicles, Flight control
Abstract: In this paper, we propose an optimal motion planning framework to allow a player agent to navigate a multi-agent environment while simultaneously identifying and avoiding potential adversaries attempting to intercept it. The behavior of an adversarial agent pursuing the player agent is defined, and a method for identifying these adversaries in real-time using Gaussian process classification is formulated. A method of real-time path replanning is then proposed and proven to avoid collisions with likely adversarial agents. The algorithm includes functionality to identify non-adversarial independent agents to avoid unnecessary replanning or the freezing robot problem. The efficacy is then shown in simulations.

Keywords: Game theory, Networked control systems, Decentralized control
Abstract: We consider a large population of learning agents that interact noncooperatively by selecting strategies from a common set. Each strategy has a payoff, assigned by a strictly concave potential game. The agents repeatedly revise their strategies according to a learning rule that models how they seek alternatives with higher payoffs. Our objective is to determine when the population learns the Nash equilibrium of the game, meaning its strategy profile asymptotically converges to this equilibrium. Unlike previous work assuming unrestricted strategy switching, here we tackle the case where only certain strategies are accessible from certain others, characterized by a strategy graph that is connected but possibly incomplete. Through Lyapunov's method, we prove that modifications based on KL-divergence to either the payoffs or the learning rules ensure the strategy profile's near-global convergence to the Nash equilibrium. We highlight the practical significance of our findings and provide a numerical validation.

Keywords: Agents-based systems, Game theory, Nonlinear systems
Abstract: We consider seeking generalized Nash equilibria for games with coupled nonlinear constraints over networks. We first revisit a well-known gradient-play dynamics to solve the problem from a passivity-based perspective, and address that the strict monotonicity on pseudo-gradients is a critical assumption to guarantee its convergence. Then we develop a novel passivity-based gradient-play dynamics by introducing parallel feedforward compensators. We prove that the dynamics achieves asymptotic convergence in merely monotone regimes. Moreover, in the absence of coupled constraints, we surprisingly find that the dynamics can deal with hypomonotone games.

Keywords: Game theory, Optimization, Large-scale systems
Abstract: In this paper, we consider the problem of generalized Nash equilibrium (GNE) seeking in a class of contractive population games under a partial-decision information setup and subject to affine equality constraints. Namely, we consider multiple populations, each comprised of a large number of payoff-driven decision makers, and we embed a network topology ruling the exchange of information between the multiple populations. Conceptually, we consider that each population has an associated payoff provider entity, which yields the payoff signals to the agents of its corresponding population. The multiple payoff providers communicate through a (possibly non-complete) network to estimate the non-local information relevant to compute the payoff signals. As the main contribution, we formulate the dynamics of the payoff providers and we provide sufficient conditions to guarantee the asymptotic stability of the set of generalized Nash equilibria of the underlying game. To the best of our knowledge, this is the first paper to address the problem of GNE seeking in population games under partial-decision information.

Keywords: Network analysis and control, Identification, Agents-based systems
Abstract: This paper studies community detection for a nonlinear opinion dynamics model from its equilibria. It is assumed that the underlying network is generated from a stochastic block model with two communities, where agents are assigned with community labels and edges are added independently based on these labels. Agents update their opinions following a nonlinear rule that incorporates saturation effects on interactions. It is shown that clustering based on a single equilibrium can detect most community labels (i.e., achieving almost exact recovery), if the two communities differ in size and link probabilities. When the two communities are identical in size and link probabilities, and the inter-community connections are denser than intra-community ones, the algorithm can achieve almost exact recovery under negative influence weights but fails under positive influence weights. Utilizing fixed point equations and spectral methods, we also propose a detection algorithm based on multiple equilibria, which can detect communities with positive influence weights. Numerical experiments demonstrate the performance of the proposed algorithms.

Keywords: Transportation networks, Game theory, Optimization
Abstract: Congestion pricing is a promising traffic congestion management policy, but has also been criticized for placing outsized financial burdens on low-income users. Credit-based congestion pricing (CBCP) and discount-based congestion pricing (DBCP) policies, which respectively provide travel credits and toll discounts to subsidize low-income users' access to tolled roads, are promising mechanisms for reducing traffic congestion without worsening societal inequities. However, the optimal design and relative merits of CBCP and DBCP policies remain poorly understood. This work studies the effects of deploying CBCP and DBCP policies to route users on multi-lane highway networks with tolled express lanes. We formulate a non-atomic routing game in which a subset of eligible users is granted toll relief via a fixed budget or toll discount, while the remaining ineligible users must pay out-of-pocket. We prove that Nash equilibrium traffic flow patterns exist under any CBCP or DBCP policy. For the setting in which eligible users have time-invariant values of time (VoTs), we provide a convex program to efficiently compute these equilibria. Moreover, for single-edge networks, we establish conditions under which DBCP policies outperform CBCP policies in improving eligible users' express lane access, an equity objective often neglected by existing congestion pricing schemes that price low-income users out of express lanes. Specifically, we identify user and network-dependent parameters that play a key role in determining whether DBCP or CBCP policies are more effective at expanding eligible users' express lane access. Finally, we present empirical results from a CBCP pilot study of the San Mateo 101 Express Lane Project in California. Our empirical results corroborate our theoretical analysis of the impact of deploying credit-based and discount-based policies.

Keywords: Game theory, Network analysis and control, Control of networks
Abstract: We study optimal seeding problems for binary super-modular network games. The system planner’s objective is to design a minimal cost seeding guaranteeing that at least a predefined fraction of the players adopt a certain action in every Nash equilibrium. Since the problem is known to be NPhard and its exact solution would require full knowledge of the network structure, we focus on approximate solutions for large-scale networks with given statistics. In particular, we build on a local mean-field approximation of the linear threshold dynamics that is known to hold true on large-scale locally treelike random networks. We first reduce the optimal intervention design problem to a linear program with an infinite set of constraints. We then show how to approximate the solution of the latter by standard linear programs with finitely many constraints. Our solutions are then numerically validated.

Keywords: Game theory, Cyber-Physical Security, Mean field games
Abstract: We explore a scenario involving two sites and a sequential game between a defender and an attacker, where the defender is responsible for securing the sites while the attacker aims to attack them. Each site holds a loss value for the defender when compromised, along with a probability of successful attack. The defender can reduce these probabilities through security investments at each site. The attacker's objective is to target the site that maximizes the expected loss for the defender, taking into account the defender's security investments. While previous studies have examined security investments in such scenarios, our work investigates the impact of bounded rationality exhibited by the defender, as identified in behavioral economics. Specifically, we consider quantal behavioral bias, where humans make errors in selecting efficient (pure) strategies. We demonstrate the existence of a quantal response equilibrium in our sequential game and analyze how this bias affects the defender's choice of optimal security investments. Additionally, we quantify the inefficiency of equilibrium investments under quantal decision-making compared to an optimal solution devoid of behavioral biases. We provide numerical simulations to validate our main findings.

Keywords: Cyber-Physical Security, Resilient Control Systems, Autonomous vehicles
Abstract: To enhance the robustness of cooperative driving to cyberattacks, we study a controller-oriented approach to mitigate the effect of a class of False-Data Injection (FDI) attacks. By reformulating a given dynamic Cooperative Adaptive Cruise Control scheme (the base controller), we show that a class of new but equivalent controllers (base controller realizations) can represent the base controller. This controller class exhibits the same platooning behavior in the absence of attacks, but in the presence of attacks, their robustness varies with the realization. We propose a prescriptive synthesis framework where the base controller and the system dynamics are written in new coordinates via an invertible coordinate transformation on the controller state. Because the input-output behavior is invariant under coordinate transformations, the input-output behavior is unaffected (so controller realizations do not change the system's closed-loop performance). However, each controller realization may require a different combination of sensors. Subsequently, we obtain the optimal combination of sensors that minimizes the effect of FDI attacks by solving a linear matrix inequality while quantifying the FDI's attack impact through reachability analysis. Through simulation studies, we demonstrate that this approach enhances the robustness of cooperative driving without relying on a detection scheme and maintaining all system properties.

Keywords: Fault diagnosis, Smart cities/houses
Abstract: In this paper, we analyze the leak localization problem in potential flow networks. We present a general model encompassing various physical systems, for example, water distribution networks. In contrast to conventional water distribution network models, our model is neither restricted to any particular type of potential loss function nor to vertex leaks. We consider leak localization via vertex potential analysis, a general description of methods that work by comparing vertex potentials calculated under a leak location hypothesis to measured vertex potentials. We derive conditions on the graph structure of the network and the potential sensor placement, under which it is guaranteed that the leak can be limited, via vertex potential analysis, to a small set of locations. We suggest a bisection method to utilize our conditions. Our conditions are based on one-way edges in graphs, a concept that we introduce. In extension, our results can be used for sensor placement.

Keywords: Cyber-Physical Security, Switched systems, Game theory
Abstract: In this paper, we investigate the problem of securing a system against actuator attacks. Specifically, we employ an unpredictability-based defense algorithm according to the principles of Moving Target Defense, while explicitly considering the worst-case attack that can be launched into the system. Consequently, we allow the system to alternate between different actuating modes, but also between optimal and robust control schemes. Via game-theoretic methods, we derive mixed strategies dictating the probabilities of utilizing different controllers for a given mode of operation, whereas an entropy-augmented optimization problem gives the probabilities of using these different modes. Finally, the overall stability of the system is analyzed, and simulation results showcase the efficacy of our proposed method.

Keywords: Cyber-Physical Security, Power systems, Robust control
Abstract: Sustainability and environmental requirements have driven the shift in electric grids towards renewable and distributed generation, characterized by inverter-based resources (IBRs) including grid-forming inverters (GFMIs). These inverters are critical components of the energy infrastructure, and their cyber-security is crucial with the growing standardization of grid support services. Analyzing the vulnerability of the converter to cyber-attacks is the first step towards developing strategies for a cyber-secure system. This paper presents a novel filter scheme for generating the smallest stealthy damaging false data injection (FDI) undetectable by model-based intrusion detection methods. Simulation results verify the undetectability and effectiveness of the designed FDI method in destabilizing currents and voltages at the Point of Common Coupling.

Keywords: Cyber-Physical Security, Control Systems Privacy, Kalman filtering
Abstract: Driven by ubiquitous digitalization and cyberattacks on critical infrastructure, there is a high interest in research on the security of cyber-physical systems. If an attacker gains access to protected and sensitive information, such as the internal states of a control system, this is considered a breach of confidentiality. Access to sensitive information can be the first step in a larger cyber-attack scheme, such as a stealthy false data injection attack. Considering process and measurement noise in the plant, existing research investigated when an attacker equipped with a Kalman filter can perfectly estimate the internal controller states if the attacker has access to plant measurements and all model parameters. For this estimate to converge, the controller is required to have stable poles. In this paper, we show that if the attacker has access to the control inputs instead of the plant measurements, the controller needs to have stable zeros. Additionally, we demonstrate that an attacker equipped with an Unknown Input Observer, using tools from delayed system inversion, can get a delayed yet perfect estimate of the controller states from the control inputs without knowledge of the plant's parameters and noise characteristics. Lastly, we present simulation results from a three-tank system to showcase the differences in controller state estimation.

Keywords: Switched systems, Robust control, Uncertain systems
Abstract: We introduce a hybrid control system called a hybrid integrator-gain system (HIGS) based integral resonant controller (IRC) to stabilize negative imaginary (NI) systems. A HIGS-based IRC has a similar structure to an IRC, with the integrator replaced by a HIGS. We show that a HIGS-based IRC is an NI system. Also, for a SISO NI system with a minimal realization, we show there exists a HIGS-based IRC such that their closed-loop interconnection is asymptotically stable. Also, we propose a proportional-integral-double-integral resonant controller and a HIGS-based proportional-integral-double-integral resonant controller, and we show that both of them can be applied to asymptotically stabilize an NI system. An example is provided to illustrate the proposed results.

Keywords: Lyapunov methods, Constrained control, Smart grid
Abstract: This paper presents an advanced safety filter based on Control Barrier and Control Lyapunov Functions to smoothly limit the current of an inverter-based Battery Energy Storage System, avoiding converter tripping, operational loss, and potential component failure. The task involves finding Control Barrier and Control Lyapunov Function via Sum-of-Squares optimization to ensure feasibility of the safety filter at every state along the trajectory. We showcase the effectiveness of the implementation through simulations involving a load step at the Point of Common Coupling, and we compare the outcomes with those obtained using a standard vector current controller.

Keywords: Hybrid systems, Switched systems, Mechatronics
Abstract: Projection-based integrators are effectively employed in high-precision systems with growing industrial success. By utilizing a projection operator, the resulting projection-based integrator keeps its input-output pair within a designated sector set, leading to unique freedom in control design that can be directly translated into performance benefits. This paper aims to enhance projection-based integrators by incorporating well-crafted linear filters into its structure, resulting in a new class of projected integrators that includes the earlier ones, such as the hybrid-integrator gain systems (HIGS) as special cases. The extra design freedom in the form of two filters in the input paths to the projection operator and the internal dynamics allows the controller to break away from the inherent limitations of a linear control design. The enhanced performance properties of the proposed structure are formally demonstrated through i) a describing function analysis, ii) mitigation of the gain-loss problem (occurring in e.g., HIGS), and iii) numerical case studies showcasing improved time-domain properties. The describing function analysis is supported by rigorously showing incremental properties of the new filtered projection-based integrators, thereby guaranteeing the steady-state responses to be unique and asymptotically stable.

Keywords: Stability of nonlinear systems, Autonomous systems
Abstract: This paper studies the dynamical properties of closed-loop systems obtained from control barrier function-based safety filters. We provide a sufficient and necessary condition for the existence of undesirable equilibria and show that the Jacobian matrix of the closed-loop system evaluated at an undesirable equilibrium always has a nonpositive eigenvalue. In the special case of linear planar systems and ellipsoidal obstacles, we give a complete characterization of the dynamical properties of the corresponding closed-loop system. We show that for underactuated systems, the safety filter always introduces a single undesirable equilibrium, which is a saddle-point. We prove that all trajectories outside the global stable manifold of such equilibrium converge to the origin. In the fully actuated case, we discuss how the choice of nominal controller affects the stability properties of the closed-loop system. Various simulations illustrate our results.

Keywords: Optimization, Optimization algorithms, Control of networks
Abstract: Online feedback optimization (OFO) enables optimal steady-state operations of a physical system by employing an iterative optimization algorithm as a dynamic feedback controller. When the plant consists of several interconnected sub-systems, centralized implementations become impractical due to the heavy computational burden and the need to pre-compute system-wide sensitivities, which may not be easily accessible in practice. Motivated by these challenges, we develop a fully distributed model-free OFO controller, featuring consensus-based tracking of the global objective value and local iterative (projected) updates that use stochastic gradient estimates. We characterize how the closed-loop performance depends on the size of the network, the number of iterations, and the level of accuracy of consensus. Numerical simulations on a voltage control problem in a direct current power grid corroborate the theoretical findings.

Keywords: Predictive control for nonlinear systems, Constrained control, Intelligent systems
Abstract: Predictive safety filters enable the integration of potentially unsafe learning-based control approaches and humans into safety-critical systems. In addition to simple constraint satisfaction, many control problems involve additional stability requirements that may vary depending on the specific use case or environmental context. In this work, we address this problem by augmenting predictive safety filters with stability guarantees, ranging from bounded convergence to uniform asymptotic stability. The proposed framework extends well-known stability results from model predictive control (MPC) theory while supporting commonly used design techniques. As a result, straightforward extensions to dynamic trajectory tracking problems can be easily adapted, as outlined in this article. The practicality of the framework is demonstrated using an automotive advanced driver assistance scenario, involving a reference trajectory stabilization problem.

Keywords: Network analysis and control, Autonomous vehicles, Decentralized control
Abstract: Merging and splitting of vehicles in a platoon is a basic maneuvering that makes the platoons more scalable and flexible. The main challenges lie in simultaneously ensuring the compositionality of the distributed controllers and the string stability of the platoon. To handle this problem, we propose a control and topology co-design method for vehicular platoons, which enables seamless merging and splitting of vehicular platoons. In particular, we first present a centralized linear matrix inequality (LMI)-based control and topology co-design optimization for vehicular platoons with formal (centralized) disturbance string stability (DSS) guarantee. Then, these centralized DSS constraints are made decentralized by developing an alternative set of sufficient conditions. Using these decentralized DSS constraints and Sylvester’s criterion-based techniques, the said centralized LMI problem is decomposed into a set of smaller decentralized LMI problems that can be solved at each vehicle in a compositional manner, enabling seamless vehicular merging/splitting. Finally, simulation examples are provided to validate the proposed co-design method through a specifically developed simulator.

Keywords: Network analysis and control, Computational methods
Abstract: This paper proposes a new approach for change point detection in multivariate Hawkes processes using Fréchet statistic of a network. The method splits the point process into overlapping windows, estimates kernel matrices in each window, and reconstructs the signed Laplacians by treating the kernel matrices as the adjacency matrices of the causal network. We demonstrate the effectiveness of our method through experiments on both simulated and real-world cryptocurrency datasets. Our results show that our method is capable of accurately detecting and characterizing changes in the causal structure of multivariate Hawkes processes, and may have potential applications in fields such as finance and neuroscience. The proposed method is an extension of previous work on Fréchet statistics in point process settings and represents an important contribution to the field of change point detection in multivariate point processes.

Keywords: Network analysis and control, Control of networks, Communication networks
Abstract: We study the problem of designing scheduling policies for communication networks. This problem is often addressed with max-weight-type approaches since they are throughput-optimal. However, max-weight policies make scheduling decisions based on the network congestion, which can be sometimes unnecessarily restrictive. In this paper, we present a ``schedule as you learn'' (SYL) approach, where we learn an average rate vector, and then select schedules that generate such a rate in expectation. This approach is interesting because scheduling decisions do not depend on the size of the queue backlogs, and so it provides increased flexibility to select schedules based on other criteria or rules, such as serving high-priority queues. We illustrate the results with numerical experiments for a cross-bar switch and show that, compared to max-weight, SYL can achieve lower latency to certain flows without compromising throughput optimality.

Keywords: Network analysis and control, Cooperative control
Abstract: Motivated by empirical research on bias and opinion formation, we formulate a multidimensional nonlinear opinion-dynamical model where agents have individual biases, which are fixed, as well as opinions, which evolve. The dimensions represent competing options, of which each agent has a relative opinion, and are coupled through normalization of the opinion vector. This can capture, for example, an individual’s relative trust in different media. In special cases including where biases are uniform across agents our model achieves consensus, but in general, behaviors are richer and capture multipolar opinion distributions. We examine general fixed points of the system, as well as special cases such as zero biases toward certain options or partitioned decision sets. Lastly, we demonstrate that our model exhibits polarization when biases are spatially correlated across the network, while, as empirical research suggests, a mixed community can mediate biases.

Keywords: Network analysis and control, Distributed control, Control of networks
Abstract: This paper addresses the problem of adaptively controlling the bias parameter in nonlinear opinion dynamics (NOD) to allocate agents into groups of arbitrary sizes for the purpose of maximizing collective rewards. In previous work, an algorithm based on the coupling of NOD with an multi-objective behavior optimization was successfully deployed as part of a multi-robot system in an autonomous task allocation field experiment. Motivated by the field results, in this paper we propose and analyze a new task allocation model that synthesizes NOD with an evolutionary game framework. We prove sufficient conditions under which it is possible to control the opinion state in the group to a desired allocation of agents between two tasks through an adaptive bias using decentralized feedback. We then verify the theoretical results with a simulation study of a collaborative evolutionary division of labor game.

Keywords: Network analysis and control, Game theory, Optimization
Abstract: This paper studies a class of network games with linear-quadratic payoffs and externalities exerted through a strictly concave interaction function. This class of game is motivated by the diminishing marginal effects with peer influences. We analyze the optimal pricing strategy for this class of network game. First, we prove the existence of a unique Nash Equilibrium (NE). Second, we study the optimal pricing strategy of a monopolist selling a divisible good to agents. We show that the optimal pricing strategy, found by solving a bilevel optimization problem, is strictly better when the monopolist knows the network structure as opposed to the best strategy agnostic to network structure. Numerical experiments demonstrate that in most cases, the maximum revenue is achieved with an asymmetric network. These results contrast with the previously studied case of linear interaction function, where a network-independent price is proven optimal with symmetric networks. Lastly, we describe an efficient algorithm for finding the optimal pricing strategy.

Keywords: Sensor fusion, Estimation, Predictive control for linear systems
Abstract: This paper presents a fusion-based output feedback MPC (FOFMPC) approach for multi-sensor systems with state and control constraints. Our approach consists of a fusion estimation procedure and a robust output feedback MPC scheme. For fusion estimation, we adopt a two-layer structure where local observers are designed for all the sensors and operate in parallel and a fusion center collects the states of the individual observers and produces a fusion estimate based on certain weighted fusion criterion. The weights are computed by minimizing the weighted Minkowski sum of the local robust positively invariant (RPI) sets. This fusion estimation procedure is then integrated into the framework of robust output feedback MPC (ROFMPC). We verify the effectiveness of the proposed approach using a double-zone building thermal model.

Keywords: Sensor fusion, Kalman filtering, Algebraic/geometric methods
Abstract: Stochastic inference on Lie groups plays a key role in state estimation problems, such as inertial navigation, visual inertial odometry, pose estimation in virtual reality, etc. A key problem is fusing independent concentrated Gaussian distributions defined at different reference points on the group. In this paper we approximate distributions at different points in the group in a single set of exponential coordinates and then use classical Gaussian fusion to obtain the fused posteriori in those coordinates. We consider several approximations including the exact Jacobian of the change of coordinate map, first and second order Taylor's expansions of the Jacobian, and parallel transport with and without curvature correction associated with the underlying geometry of the Lie group. Preliminary results on SO(3) demonstrate that a novel approximation using parallel transport with curvature correction achieves similar accuracy to the state-of-the-art optimisation based algorithms at a fraction of the computational cost.

Keywords: Sensor fusion, Robotics, Kalman filtering
Abstract: This paper investigates the robot state estimation problem within a non-inertial environment. The proposed state estimation approach overcomes the common assumption of static ground in the system modeling. The process and measurement models explicitly treat the movement of the non-inertial environments without requiring knowledge of its motion in the inertial frame or relying on GPS or sensing environmental landmarks. Further, the proposed state estimator is formulated as an invariant extended Kalman filter (InEKF) with the deterministic part of its process model obeying the group-affine property, leading to log-linear error dynamics. The observability analysis of the filter confirms that the robot's pose (i.e., position and orientation) and velocity relative to the non-inertial environment are observable. Hardware experiments on a humanoid robot moving on a rotating and translating treadmill demonstrate the high convergence rate and accuracy of the proposed InEKF even under significant treadmill pitch sway, as well as large estimation errors.

Keywords: Sensor networks, Agents-based systems, Reinforcement learning
Abstract: This paper introduces an efficient distributed deployment strategy for a network of mobile and stationary sensors with nonidentical sensing and communication radii. A collaborative distributed multi-agent deep reinforcement learning method is proposed to find the best moving direction and step size for each sensor considering the coverage priority. The gradient of the local coverage function is used to generate a fast-converging solution as well as a learning-inspired arbitrary input to enable the network to avoid the local optima. The sensors use their partial observation of the network and field to iteratively relocate themselves to explore the field and learn the optimal policy to increase their local coverage. The efficiency of the proposed strategy in different scenarios is demonstrated by simulations.

Keywords: Sensor networks, Estimation
Abstract: Industrial cyber-physical systems (ICPS) integrate physical processes with computational and communication technologies in industrial settings. With the support of edge computing technology, it is feasible to schedule large-scale sensors for efficient distributed sensing. In the sensing process, observability is the key to obtaining complete system states, and stochastic scheduling is more suitable considering uncertain factors in wireless communication. However, existing works have limited research on observability in stochastic scheduling. Targeting this issue, we propose an observability-based intelligent distributed edge sensing method (OIDM). Deep reinforcement learning (DRL) methods are adopted to optimize sensing accuracy and power efficiency. Based on the system's ability to achieve observability, we establish a bridge between observability and the number of successful sensor transmissions. Novel linear approximations of observability criteria are provided, and probabilistic bounds on observability are derived. Furthermore, these bounds guide the design of action space to achieve a probabilistic observability guarantee in stochastic scheduling. Finally, our proposed method is applied to the estimation of slab temperature in industrial hot rolling process, and simulation results validate its effectiveness.

Keywords: Optimal control, Reinforcement learning, Resilient Control Systems
Abstract: In this paper, we have proposed a resilient reinforcement learning method for discrete-time linear systems with unknown parameters, under denial-of-service (DoS) attacks. The proposed method is based on policy iteration that learns the optimal controller from input-state data amidst DoS attacks. We achieve an upper bound for the DoS duration to ensure closed-loop stability. The resilience of the closed-loop system, when subjected to DoS attacks with the learned controller and an internal model, has been thoroughly examined. The effectiveness of the proposed methodology is demonstrated on an inverted pendulum on a cart.

Keywords: Optimal control, Switched systems
Abstract: In the last decades, many authors provided different notions of impulsive process, seen as a suitably defined limit of a sequence of ordinary processes for a nonlinear control-affine system with unbounded, vector-valued controls. In particular, we refer to the impulsive processes introduced by Karamzin et al. --in which the control is given by a vector measure, a non-negative scalar measure, and a family of so-called attached controls that univocally determine the jumps of the corresponding trajectory-- and to the graph completion processes developed by Bressan and Rampazzo et al. --in which an impulsive trajectory is seen as a spatial projection of a Lipschitzian trajectory in space-time. The equivalence between these notions is the crucial assumption of most results on optimal impulsive control problems, such as existence of an optimal process and necessary/sufficient optimality conditions. In this note we exhibit a counterexample which shows that, in presence of state constraints and endpoint constraints involving the total variation of the impulsive control, this equivalence may fail. Thus, we propose to replace the set of impulsive processes with a smaller class of impulsive processes, that we call admissible, which turns out to be actually in one-to-one correspondence with the set of graph completion processes.

Keywords: Optimal control, Uncertain systems, Reinforcement learning
Abstract: This paper addresses an off-policy Reinforcement learning algorithm for robust linear quadratic regulator (R-LQR) problem of continuous-time linear dynamical systems with parametric uncertainties based on policy iteration framework. A modified algebraic Riccati equation is presented for the R-LQR problem and is further transformed into standard linear quadratic regulator problem. The proposed model-free off policy R-LQR algorithm learns the control policy using generated data samples that obviate the requirement of system dynamics. Numerical simulation examples of spring-mass system with uncertain stiffness are provided to illustrate effectiveness of the approach.

Keywords: Optimization, Optimal control, Optimization algorithms
Abstract: This paper investigates the sensitivity of the solutions of NonLinear Programs (NLPs) with polytopic constraints when the nonlinear continuous objective function is approximated by a PieceWise-Affine (PWA) counterpart. General NLPs are prevalent in optimization-based control of nonlinear systems, but solving them can be computationally expensive, requiring either fast hardware or tractable suboptimal approximations. By leveraging perturbation analysis using a convex modulus, we derive guaranteed bounds on the distance between the optimal solution of the original polytopically-constrained NLP and that of its approximated formulation. Our approach aids in determining criteria for achieving desired solution bounds. Two case studies on the Eggholder function and nonlinear model predictive control of an inverted pendulum demonstrate the theoretical results.

Keywords: Optimal control, Stability of nonlinear systems, Lyapunov methods
Abstract: Given a discounted cost, we study deterministic discrete-time systems whose inputs are generated by policy iteration (PI). We provide novel near-optimality and stability properties, while allowing for non stabilizing initial policies. That is, we first give novel bounds on the mismatch between the value function generated by PI and the optimal value function, which are less conservative in general than those encountered in the dynamic programming literature for the considered class of systems. Then, we show that the system in closed-loop with policies generated by PI are stabilizing under mild conditions, after a finite (and known) number of iterations.

Keywords: Optimal control, Stability of nonlinear systems, Adaptive control
Abstract: This work presents and showcases a novel reinforcement learning agent called Critic As Lyapunov Function (CALF) which is model-free and ensures online environment, in other words, dynamical system stabilization. Online means that in each learning episode, the said environment is stabilized. This, as demonstrated in a case study with a mobile robot simulator, greatly improves the overall learning performance. The base actor-critic scheme of CALF is analogous to SARSA. The latter did not show any success in reaching the target in our studies. However, a modified version thereof, called SARSA-m here, did succeed in some learning scenarios. Still, CALF greatly outperformed the said approach. CALF was also demonstrated to improve a nominal stabilizer provided to it. In summary, the presented agent may be considered a viable approach to fusing classical control with reinforcement learning. Its concurrent approaches are mostly either offline or model-based, like, for instance, those that fuse model-predictive control into the agent.

Keywords: Optimization algorithms, Optimization, Control of networks
Abstract: In this paper, we study a general setup for constrained convex optimisation over time-varying networks. We propose a distributed algorithm, based on the cutting plane method, to address non-smooth optimisation challenges. Cutting plane-based approaches require constraint consensus which is structurally different from established consensus schemes. We bridge this gap by linking the cutting plane-based algorithm with a dynamic average tracking scheme. The distributed cutting plane algorithm is presented and its convergence is analysed. Its performance is investigated through a numerical example.

Keywords: Optimization algorithms, Optimization, Networked control systems
Abstract: In realistic distributed optimization scenarios, individual nodes possess only partial information and communicate over bandwidth constrained channels. For this reason, the development of efficient distributed algorithms is essential. In our paper we addresses the challenge of unconstrained distributed optimization. In our scenario each node's local function exhibits strong convexity with Lipschitz continuous gradients. The exchange of information between nodes occurs through 3-bit bandwidth-limited channels (i.e., nodes exchange messages represented by a only 3-bits). Our proposed algorithm respects the network's bandwidth constraints by leveraging zoom-in and zoom-out operations to adjust quantizer parameters dynamically. We show that during our algorithm's operation nodes are able to converge to the exact optimal solution. Furthermore, we show that our algorithm achieves a linear convergence rate to the optimal solution. We conclude the paper with simulations that highlight our algorithm's unique characteristics.

Keywords: Optimization algorithms, Optimization, Stability of nonlinear systems
Abstract: We propose a novel continuous-time algorithm for inequality-constrained convex optimization inspired by proportional-integral control. Unlike the popular primal-dual gradient dynamics, our method includes a proportional term to control the primal variable through the Lagrange multipliers. This approach has both theoretical and practical advantages. On the one hand, it simplifies the proof of the exponential convergence in the case of smooth, strongly convex problems, with a more straightforward assessment of the convergence rate concerning prior literature. On the other hand, through several examples, we show that the proposed algorithm converges faster than primal-dual gradient dynamics. This paper aims to illustrate these points by thoroughly analyzing the algorithm convergence and discussing some numerical simulations.

Keywords: Optimization algorithms, Optimization
Abstract: A cooperative, multi-agent global optimization problem is considered, where the global cost function is the sum of the agents' private, non-convex costs. In contrast to all previously considered setups, evaluating the private costs involves a global experiment, using a common instance of the decision vector. This is relevant when each agent can only control a part ("subvariable") of the decision vector, but its private cost is also affected by the other subvariables. A novel cooperative optimization method using Set Membership identification and consensus-based techniques is proposed, to make all agents agree on the next global decision vector to be tested. A trade-off between exploitation close to the best point found and exploration around the search set is achieved, even without explicitly sharing the private costs' information. Statistical tests show that the proposed distributed method is competitive with respect to a centralized one.

Keywords: Optimization algorithms, Optimization
Abstract: Decentralized optimization algorithms have recently attracted increasing attention due to its wide applications in all areas of science and engineering. In these algorithms, a collection of agents collaborate to minimize the average of a set of heterogeneous cost functions in a decentralized manner. State-of-the-art decentralized algorithms like Gradient Tracking (GT) and Exact Diffusion (ED) involve communication at each iteration. Yet, communication between agents is often expensive, resource intensive, and can be very slow. To this end, several strategies have been developed to balance between communication overhead and convergence rate of decentralized methods. In this paper, we introduce GT-PGA, which incorporates~GT with periodic global averaging. With the additional PGA, the influence of poor network connectivity in the GT algorithm can be compensated or controlled by a careful selection of the global averaging period. Under the stochastic, nonconvex setup, our analysis quantifies the crucial trade-off between the connectivity of network topology and the PGA period. Thus, with a suitable design of the PGA period, GT-PGA improves the convergence rate of vanilla GT. Numerical experiments are conducted to support our theory, and simulation results reveal that the proposed GT-PGA accelerates practical convergence, especially when the network is sparse.

Keywords: Optimization algorithms, Optimization
Abstract: This work considers the optimization of the composition of functions in a nested form over Riemannian manifolds where each function contains an expectation. This problem type is gaining popularity in applications such as policy evaluation in reinforcement learning or model customization in meta-learning. The standard Riemannian stochastic gradient methods for non-compositional optimization cannot be directly applied as the stochastic approximation of the inner functions creates biases in the gradients of the outer functions. For two-level composition optimization, we present a Riemannian Stochastic Composition Gradient Descent (R-SCGD) method that finds an approximate stationary point, with expected squared Riemannian gradient smaller than epsilon, in O(epsilon^{-2}) calls to the stochastic gradient oracle of the outer function and stochastic function and gradient oracles of the inner function.

Keywords: Stochastic optimal control, Markov processes, Predictive control for nonlinear systems
Abstract: This paper presents first results for near optimality in expectation of the closed-loop solutions for stochastic economic MPC. The approach relies on a recently developed turnpike property for stochastic optimal control problems at an optimal stationary process, combined with techniques for analyzing time-varying economic MPC schemes. We obtain near optimality in finite time as well as overtaking and average near optimality on infinite time horizons.

Keywords: Cyber-Physical Security, Attack Detection, Nonlinear systems
Abstract: As cyber-physical systems form the core of many critical infrastructures, ensuring their safety is essential. The sensor measurements of networked cyber-physical systems can potentially be compromised, resulting in the misbehavior of the overall system. Indeed there have been a number of such attacks. Dynamic Watermarking is a proactive method whose goal is to detect such cyber attacks. It superimposes a small secret stochastic excitation onto signals in the system, such as control inputs of actuators or sensor measurements. Based on an examination of the signals purportedly returned, for example, by the sensors, it determines if the measurements have been tampered with. Previous theory for detection guarantees provided by Dynamic Watermarking has been restricted to linear stochastic systems. This paper examines the Dynamic Watermarking method for nonlinear stochastic systems. We show that Dynamic Watermarking for detecting attacks can be extended to certain systems in backstepping form. We present the analytical proofs, as well as simulation results.

Keywords: Stochastic optimal control, Hybrid systems, Stochastic systems
Abstract: We establish a linear programming formulation for the solution of joint chance constrained optimal control problems over finite time horizons. The joint chance constraint may represent an invariance, reachability or reach-avoid specification that the trajectory must satisfy with a predefined probability. For finite state and action spaces, the solution is exact and our method computationally superior to approaches in the literature. For continuous state or action spaces, our linear programming formulation enables basis function approximations.

Keywords: Mean field games, Stochastic systems, Large-scale systems
Abstract: For sequences of networks embedded in the unit cube [0, 1]^m, (weak) measure limits of sequences of empirical measures of vertex densities (vertexon functions) exist, and the associated (weak) measure limits of sequences of empirical measures of edge densities (graphexon functions) in [0, 1]^{2m} exist, regardless of the sparsity or density of the limit graphs. This paper presents an extension of Graphon Mean Field Game (GMFG) theory to the vertexon-graphexon MFG set-up (denoted GXMFG). Specific second order dynamics are introduced for the inter-node influence mediated by the singular part of a network graphexon measure; this is analyzed in the particular cases of a network limit ring topology and a limit rectangular lattice topology. Existence and uniqueness results are presented for the corresponding GXMFG equations.

Keywords: Stochastic optimal control, Decentralized control, Stochastic systems
Abstract: We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed models and multi-agent models with decentralized information structures. While there exist comprehensive existence and approximations results for the fully observed setup in the literature, few prior research exists on discrete-time approximation results for partially observed models. For decentralized models, even existence results have not received much attention except for specialized models and approximation has been an open problem. Our existence and approximations results lead to the applicability of well-established partially observed Markov decision processes and the relatively more mature theory of discrete-time decentralized stochastic control to be applicable for computing near optimal solutions for continuous-time stochastic control.

Keywords: Stochastic optimal control, Stochastic systems, Optimization
Abstract: Stochastic maximum principle (SMP) specifies a necessary condition for the solution of a stochastic optimal control problem. The condition involves a coupled system of forward and backward stochastic differential equations (FBSDE) for the state and the adjoint processes. Numerical solution of the FBSDE is challenging because the boundary condition of the adjoint process is specified at the terminal time, while the solution should be adaptable to the forward in time filtration of a Wiener process. In this paper, a "time-reversal" of the FBSDE system is proposed that involves integration with respect to a backward in time Wiener process. The time-reversal is used to propose an iterative Monte-Carlo procedure to solves the FBSDE system and its time-reversal simultaneously. The procedure involves approximating the Föllmer's drift and solving a regression problem between the state and its adjoint at each time. The procedure is illustrated for the linear quadratic (LQ) optimal control problem with a numerical example.

Keywords: Data driven control, Iterative learning control, Predictive control for linear systems
Abstract: This work introduces the Data-Enabled Predictive iteRative Control (DeePRC) algorithm, a direct data-driven approach for iterative LTI systems. The DeePRC learns from previous iterations to improve its performance and achieves the optimal cost. By utilizing a tube-based variation of the DeePRC scheme, we propose a two-stage approach that enables safe active exploration using a left-kernel-based input disturbance design. This method generates informative trajectories to enrich the historical data, which extends the maximum achievable prediction horizon and leads to faster iteration convergence. In addition, we present an end-to-end formulation of the two-stage approach, integrating the disturbance design procedure into the planning phase. We showcase the effectiveness of the proposed algorithms on a numerical experiment.

Keywords: Data driven control, Machine learning
Abstract: We present a data-driven control architecture for modifying the kinematics of robots and artificial avatars to encode specific information such as the presence or not of an emotion in the movements of an avatar or robot driven by a human operator. We validate our approach on an experimental dataset obtained during the reach-to-grasp phase of a pick-and-place task.

Keywords: Data driven control, Learning, Machine learning
Abstract: In robotics and reinforcement learning, transfer learning helps to pass the knowledge obtained in the design phase to the deployment phase, allowing for faster adaptation. However, this practice is yet to be fundamentally and theoretically understood for dynamical systems. In this work, we approach the prediction problems for dynamical systems under the transfer learning setup. Specifically, we propose methods that are able to utilize data from different training systems to learn a predictive model for an unknown target system. Feasibility condition and finite sample guarantees are developed. Particularly, through experimental studies, our methods demonstrate the ability to handle hidden model structures such as parameterization redundancy and nonlinearity, which outperforms the system identification method.

Keywords: Data driven control, Machine learning, Lyapunov methods
Abstract: In this paper, we present a data-driven methodology to predict and control the behaviour of nonlinear and non-autonomous systems based on kernel functions. The technique computes the forecasting by means of a linear combination of past data. The weights used to compute the prediction are obtained by solving a convex optimization problem that stems from a novel kriging formulation. A Control Lyapunov Function (CLF) based controller using the presented predictor is also built. Finally, numerical examples of both prediction and control are presented, showing the efficacy of the proposed approach.

Keywords: Data driven control, Machine learning
Abstract: As Artificial Intelligence (AI) techniques continue to advance, the need for explainability becomes increasingly crucial, especially in sensitive or safety-critical domains. eXplainable AI (XAI) has emerged to address this need, aiming to enhance transparency in complex models. While XAI has gained traction in mainstream machine learning, its application in data-driven control systems remains relatively unexplored. This paper introduces a novel concept of explainability tailored for data-driven control, allowing one to design feedback loops from data incorporating prior knowledge and preserving important system properties. Through two case studies, we demonstrate the efficacy of this property-preserving framework in direct and indirect data-driven control system design. This work lays the foundation for further research at the intersection of AI and data-driven control, offering insights into enhancing transparency in complex control systems.

Keywords: Data driven control, Kalman filtering, Predictive control for linear systems
Abstract: Data-driven methods for predictive control rely on input-output data to give a Hankel matrix representation of the space of trajectories. They are poorly suited to situations where both process noise and measurement noise dominate the behaviour whereas Kalman filters optimally estimate system states in this scenario. We derive a data-driven Kalman filter formulation based on the dynamic evolution of Hankel matrix output predictions. This leads to an extended state space model that describes the evolution of both the future inputs and outputs. By applying measurement feedback one arrives at a Kalman filter for the system. The Kalman filter design is performed purely on the basis of the input and output signals and without the need for a specific state-space representation. A benchmark simulation illustrates that the resulting prediction- based control significantly out-performs predictive controllers based on current data-driven methods.

Keywords: Reinforcement learning, Robotics, Learning
Abstract: The Central Pattern Generator (CPG) is adept at generating rhythmic gait patterns characterized by consistent timing and adequate foot clearance. Yet, its open-loop configuration often fails to adjust the system's control performance in response to environmental variations. On the other hand, Reinforcement Learning (RL), celebrated for its model-free properties, has gained significant traction in robotics due to its inherent adaptability and robustness. However, initiating traditional RL approaches from the ground up presents a risk of converging to suboptimal local minima and slow learning convergence. In this paper, we propose a quadruped locomotion framework--called SYNLOCO--by synthesizing CPG and RL, which can ingeniously integrate the strengths of both methods, enabling the development of a locomotion controller that is both stable and natural with partial state observations (e.g., no velocity measurements). To optimize the learning trajectory of SYNLOCO, a two-phase training strategy is presented. Both ablation analysis and experimental comparison are performed using a real quadruped robot under varied conditions, including distinct velocities, terrains, and payload capacities. The experiments showcase SYNLOCO's efficiency in producing consistent and clear-footed gaits across diverse scenarios, despite no velocity measurements. The developed controller exhibits resilience against substantial parameter variations, underscoring its potential for robust real-world applications.

Keywords: Reinforcement learning, Robotics, Simulation
Abstract: We propose a frequency-domain state representation to improve the performance and reduce the computation and data requirements of reinforcement learning. This approach is tailored to tracking tasks of periodic trajectories. We apply the proposed methodology to an active knee prosthesis application. Using the high-fidelity simulator~MuJoCo, we demonstrate significant performance improvements (in terms of Bellman error) for the proposed frequency-domain state representation relative to the current state-of-the-art time-domain state representation used in these applications.

Keywords: Reinforcement learning, Smart grid, Optimization algorithms
Abstract: This paper introduces a novel multi-armed bandits framework, termed Contextual Restless Bandits (CRB), for complex online decision-making. This CRB framework incorporates the core features of contextual bandits and restless bandits, so that it can model both the internal state transitions of each arm and the influence of external global environmental contexts. Using the dual decomposition method, we develop a scalable index policy algorithm for solving the CRB problem, and theoretically analyze the asymptotical optimality of this algorithm. In the case when the arm models are unknown, we further propose a model-based online learning algorithm based on the index policy to learn the arm models and make decisions simultaneously. Furthermore, we apply the proposed CRB framework and the index policy algorithm specifically to the demand response decision-making problem in smart grids. The numerical simulations demonstrate the performance and efficiency of our proposed CRB approaches.

Keywords: Large-scale systems, Reinforcement learning, Optimization
Abstract: Motivated by the emerging paradigm of federated reinforcement learning, we consider a setting involving N agents, where each agent interacts with an environment modeled as a Markov Decision Process (MDP). The agents' MDPs differ in their reward functions, capturing heterogeneous objectives/tasks. The collective goal of the agents is to communicate via a central server to find a policy that maximizes the average of long-term cumulative rewards across environments. The limited existing work on this topic either only provide asymptotic rates, or generate biased policies, or fail to establish any benefits of collaboration. In response, we propose texttt{Fast-FedPG} - a novel federated policy gradient algorithm with a carefully designed bias-correction mechanism. Under a gradient-domination condition, we prove that our algorithm guarantees (i) fast linear convergence with exact gradients, and (ii) sub-linear rates that enjoy a linear speedup w.r.t. the number of agents with noisy, truncated policy gradients. In each case, the convergence is to a globally optimal policy with no heterogeneity-induced bias. In the absence of gradient-domination, we establish convergence to a first-order stationary point at a rate that continues to benefit from collaboration.

Keywords: Machine learning, Human-in-the-loop control, Control applications
Abstract: System identification and optimal control have always contributed to water resources systems planning and management. Although water control problems are commonly formulated as multi-objective Markov Decision Processes, accurately modeling reservoir systems controlled by human operators remains challenging due to the absence of a formal definition of the objective function guiding their behavior. In this paper, we introduce a mixed Reinforcement Learning approach to model the dynamics of multipurpose reservoir systems. Specifically, our method first uses Inverse Reinforcement Learning to extract the tradeoff among competing objectives from historical observations of the reservoir system dynamics. The identified objective function is then used in the formulation of an optimal control problem returning a closed-loop policy which allows the simulation of the observed dynamics of the reservoir system. We demonstrate the potential of the proposed method in a real-world application involving the multipurpose regulation of Lake Como in northern Italy. Results show that our approach effectively infers the tradeoff between flood control and water supply adopted in the observed system’s operation, and yields a control policy that closely approximates the observed system dynamics.

Keywords: Machine learning, Large-scale systems, Communication networks
Abstract: Over-the-Air Computation is a beyond-5G communication strategy that has recently been shown to be useful for the decentralized training of machine learning models due to its efficiency. In this letter, we propose an Over-the-Air federated learning algorithm that aims to provide fairness and robustness through minmax optimization. By using the epigraph form of the problem at hand, we show that the proposed algorithm converges to the optimal solution of the minmax problem. Moreover, the proposed approach does not require reconstructing channel coefficients by complex encoding-decoding schemes as opposed to state-of-the-art approaches. This improves both efficiency and privacy.

Keywords: Distributed parameter systems, Large-scale systems, Flexible structures
Abstract: This study introduces a novel control oriented structure-preserving scheme for discretizing a class of multidimensional linear port-Hamiltonian systems, preserving their inherent structure while enabling the imposition of diverse combinations of boundary inputs, such as generalized velocities, displacements, and tractions. The proposed approach is grounded on the modified Linked Lagrange Multiplier method and the mixed Finite Element Method (FEM), where Dirichlet and Neumann boundary conditions are weakly enforced. Connections with other standard and mixed FEM approaches are also discussed. The proposed scheme is validated through comparisons with commercial software and simulations using a 2D elasticity model as a demonstrative example.

Keywords: Distributed parameter systems, Adaptive control
Abstract: This work considers the adaptive repair rate design of a multi-state reparable system modeled by coupled transport and integro-differential equations. A reparable system is one which can be restored to satisfactory operation by repair actions whenever a failure occurs. The model describes the probabilities of the system in good and failure modes. The objective is to design the adaptive repair functions so that the probability of the system in good mode can be steered to a target distribution. Rigorous analysis on the convergence of tracking error between the plant and the target states is addressed.

Keywords: Computational methods, Large-scale systems, Nonlinear systems
Abstract: We consider the optimal regulation problem for nonlinear control-affine dynamical systems. Whereas the linear-quadratic regulator (LQR) considers optimal control of a linear system with quadratic cost function, we study polynomial systems with polynomial cost functions; we call this problem the polynomial-polynomial regulator (PPR). The resulting polynomial feedback laws provide two potential improvements over linear feedback laws: 1) they more accurately approximate the optimal control law, resulting in lower control costs, and 2) for some problems they can provide a larger region of stabilization. We derive explicit formulas---and a scalable, general purpose software implementation---for computing the polynomial approximation to the value function that solves the optimal control problem. The method is illustrated first on a low-dimensional aircraft stall stabilization example, for which PPR control recovers the aircraft from more severe stall conditions than LQR control. Then we demonstrate the scalability of the approach on a semidiscretization of dimension n=129 of a partial differential equation, for which the PPR control reduces the control cost by approximately 75% compared to LQR for the initial condition of interest.

Keywords: Reduced order modeling, Nonlinear systems, Nonlinear systems identification
Abstract: This paper proposes a new cluster method combined with Dynamic Mode Decomposition with Control (DMDc), and the Proper Orthogonal Decomposition (POD) to construct more accurate reduced order models. DMDc and POD are popular data-driven techniques that extract low-order models from high-dimensional complex dynamic systems. However, these methods are inherently linear, i.e., the data is assumed to belong to linear manifolds. However, this may lead to inaccuracies in the reduced models commensurate with the presence of nonlinearities. To capture the nonlinear behavior, manifold clustering is introduced to group the snapshots obtained by experiments or numerical simulation into several sub-regions based on the underlying non-linear structure. Manifold clustering is a powerful approach for exploratory data analysis, allowing the discovery of patterns and structures that are not apparent in raw high-dimensional data. It does not require knowing the number of clusters and the intrinsic manifold dimensions in advance. Manifold clustering is combined with DMDc and POD to construct the local reduced-order models. Time clustering is applied to the snapshots generated by a nonlinear convective flow governed by the 2D Burgers' equations with boundary actuation. The manifold cluster reduced order model outperforms standard and other cluster-based (K-means) reduced order models.

Keywords: Distributed parameter systems, Optimization algorithms, Identification
Abstract: An extremely efficient technique is developed for maximizing the parameter estimation accuracy for systems modelled by partial differential equations in the presence of correlated measurement noise. The trace of the covariance matrix of the weighted least-squares estimator is employed as the measure of the resulting estimation accuracy. This design criterion is to be minimized by choosing a set of spatiotemporal measurement locations from among a given finite set of candidate locations. To make this inherently combinatorial problem tractable, a novel relaxed convex formulation is proposed. The pivotal role here is played by the decomposition of the noise into uncorrelated and correlated components. Optimal solutions are found via simplicial decomposition which alternates between updating the design weights using a very simple multiplicative algorithm and computing a closed-form solution to a linear programming problem. The sequence of the produced iterates monotonically decreases the value of the original A-optimality design criterion. Since the resulting relaxed solution is a measure on the set of candidate measurement locations and not its specific subset, a sequential conversion to a nearly optimal subset of selected sensors is discussed. A nontrivial simulation experiment is reported to validate the proposed approach.

Keywords: Distributed parameter systems, Modeling, Flexible structures
Abstract: Systems of partial differential equations often admit different Hamiltonian representations, leading to different boundary variables that are either power or energy conjugate. It is shown that any infinite-dimensional Hamiltonian system can be transformed into one with constant symplectic matrix. Alternatively, any passive Hamiltonian system can be converted into one with constant energy storage matrix. The consideration of energy boundary variables points towards a new approach to control by interconnection. All this is illustrated on the example of the elastic rod.

Keywords: Filtering, Estimation, Linear systems
Abstract: The purpose of this paper is to present a filtering algorithm for simultaneous input and state estimation for linear discrete-time systems of any inherent delay. It generalises previous work which was restricted to systems with an inherent delay of less than or equal to one. The algorithm includes the Kalman filter which is itself a special case of a system with an inherent delay of zero. The approach uses a standard characterisation of a system's inherent delay in terms of its rank indices. Two judiciously chosen rank decompositions of a Toeplitz matrix of the system's Markov parameters are used to develop the algorithm. The filter is illustrated on a simple two-mass system with an inherent delay of two.

Keywords: Filtering, Estimation, Stochastic systems
Abstract: The optimal filtering problem for general nonlinear state-observation systems has garnered significant attention in control theory. At its core, optimal filtering involves determining the probability density function of the system state conditioned on historical observations. The Yau-Yau method cite{yau2008real}, a pioneering framework, offers a viable approach with comprehensive theoretical guarantees and practical numerical implementation. Specifically, the Yau-Yau framework comprises two key components: offline solution of the forward Kolmogorov equation (FKE) and online data assimilation updates. The primary challenge lies in efficiently and accurately solving the FKE, as it directly impacts the real-time filtering process. To address this fundamental obstacle, we propose a highly efficient filtering algorithm that combines a FKE solver based on deep neural networks and a PDF approximator using generalized Legendre polynomials. By integrating advanced deep learning techniques with Galerkin approximation, we introduce the logarithmic transformed deep Galerkin approach (LTDG). The numerical simulations showcase the effectiveness and accuracy of our newly proposed algorithm. LTDG demonstrates superior performance compared to other methods, such as the extended Kalman filter and particle filter, and it successfully maintains the high accuracy of the Galerkin spectral method while having fewer online computational burdens.

Keywords: Filtering, Machine learning, Variational methods
Abstract: The nonlinear filtering problem is concerned with finding the conditional probability distribution (posterior) of the state of a stochastic dynamical system, given a history of partial and noisy observations. This paper presents a data-driven nonlinear filtering algorithm for the case when the state and observation processes are stationary. The posterior is approximated as the push-forward of an optimal transport (OT) map from a given distribution, that is easy to sample from, to the posterior conditioned on a truncated observation window. The OT map is obtained as the solution to a stochastic optimization problem that is solved offline using recorded trajectory data from the state and observations. An error analysis of the algorithm is presented under the stationarity and filter stability assumptions, which decomposes the error into two parts related to the truncation window during training and the error due to the optimization procedure. The performance of the proposed method, referred to as optimal transport data-driven filter (OT-DDF), is evaluated for several numerical examples, highlighting its significant computational efficiency during the online stage while maintaining the flexibility and accuracy of OT methods in nonlinear filtering.

Keywords: Information theory and control, Filtering, Optimization
Abstract: In this paper, we analyze the indirect source coding problem with side information at both the encoder and decoder, as well as only at the decoder. We first derive structural properties of the two rate distortion functions (RDFs) for general abstract spaces and identify conditions under which the RDFs coincide. For multivariate jointly Gaussian random variables with square-error fidelity, we establish structural properties of the optimal test channels, show that side information at both the encoder and decoder does not reduce compression, and provide water-filling solutions using parallel Gaussian channel realizations. This paper uses a novel realization theory approach to establish achievability of the converse coding theorem lower bounds of the two RDFs.

Keywords: Estimation, Sensor networks, Kalman filtering
Abstract: A crucial challenge in decentralized systems is state estimation in the presence of unknown inputs, particularly within heterogeneous sensor networks with dynamic topologies. While numerous consensus algorithms have been introduced, they often require extensive information exchange or multiple communication iterations to ensure estimation accuracy. This paper proposes an efficient algorithm that achieves an unbiased and optimal solution comparable to filters with full information about other agents. This is accomplished through the use of information filter decomposition and the fusion of inputs via covariance intersection. Our method requires only a single communication iteration for exchanging individual estimates between agents, instead of multiple rounds of information exchange, thus preserving agents' privacy by avoiding the sharing of explicit observations and system equations. Furthermore, to address the challenges posed by dynamic communication topologies, we propose two practical strategies to handle issues arising from intermittent observations and incomplete state estimation, thereby enhancing the robustness and accuracy of the estimation process. Experiments and ablation studies conducted in both stationary and dynamic environments demonstrate the superiority of our algorithm over other baselines. Notably, it performs as well as, or even better than, algorithms that have a global view of all neighbors.

Keywords: Stochastic systems, Linear systems, Filtering
Abstract: In this paper, we develop a novel time synchronization algorithm for an ensemble of atomic clocks containing a mixture of second order and third order atomic clocks. To design the feedback control, we consider the Kalman filter to estimate the states of the observable subsystem derived from the observable canonical decomposition of the atomic clock ensemble. We prove that, under certain conditions, the time deviation of each clock from the ideal clock behaviour follows the dynamics of the first state of the unobservable subsystem, which is regarded as the synchronization destination of the clocks. In the algorithm, we can adjust the synchronization destination by choosing the basis of the observable subspace in a special manner for the observable canonical decomposition. An example is included to illustrate the efficacy of the algorithm. It has been revealed numerically that altering the control interval appropriately can improve the frequency stability of the clocks.

Keywords: Optimal control, Predictive control for linear systems, Power electronics
Abstract: Currently the large scale adoption of Battery Electric Vehicles (BEVs) is limited due to cost, reliability and lifetime considerations. The power converters, and more specifically their semiconductor switching devices, are the second most likely component to fail in a BEV because of the damage caused by the current-induced temperature cycling. In this paper we propose a novel hybrid frequency and time domain control approach that integrates time-domain performance requirements and frequency-domain reliability requirements, based on a frequency model of the damage. The method is applied to control the motor of an BEVs and minimize the damage experienced by the power converter.

Keywords: Power electronics, Modeling, Nonlinear systems
Abstract: In this paper are presented a novel modeling approach and a passivity-based control scheme to solve the output voltage regulation control problem of a class of Quasi-resonant Converters. Instead of consider classical order reduction arguments, the proposed full order model recovers the Port-Controlled Hamiltonian structure naturally exhibited by the converters. This feature leads to the possibility to propose the implementation of a passive PI control scheme which has been widely recognized to achieve high performances while proving in a formal way its stability properties. In addition, the controller structure is complemented by the inclusion of a static map to use both the frequency and the duty-cycle of the square input signal as control input, guaranteeing a Zero Current Switching operation mode which drastically improve the efficiency of the circuit. The usefulness of the proposed model and control are validated in a numerical setting.

Keywords: Optimal control, Power electronics
Abstract: In power conversion applications, the DC-link capacitor is a critical component prone to degradation and its reliability affects the whole converter lifespan. Two-levels con- verters are commonly operated using pulse patterns designed according to conventional modulation schemes to generate a desired AC voltage. The effect of the switching on the DC-link capacitor current is not considered in the planned pulse pattern. This work proposes an alternative modulation strategy where the patterns are optimized to minimize the RMS current and thus reduce the degradation effect of current ripples on the capacitor. A comparative performance analysis of the reliability-based modulation scheme with respect to the standard practice is carried out based on simulation. The analysis results demonstrate that the proposed method generates the desired voltage output with low levels of harmonic distortion, while significantly decreasing RMS current values and thus enhancing capacitor lifetime.

Keywords: Optimization, Power systems, Computational methods
Abstract: Charging station availability is crucial for a thriving electric vehicle market. Due to budget constraints, locating these stations usually proceeds in phases, which calls for careful consideration of the (random) charging demand growth throughout the planning horizon. This paper integrates user choice behavior into two-stage and multi-stage stochastic programming models for intracity charging station planning under demand uncertainty. We derive a second-order conic representation for the nonlinear, nonconvex formulation by taking advantage of the binary nature of location variables and propose subgradient inequalities to accelerate computation. Numerical results demonstrate the value of employing multi-stage models, particularly in scenarios of high demand fluctuations, increased demand dispersion, and high user sensitivity to the distance-to-recharge.

Keywords: Electrical machine control, Adaptive systems, Power electronics
Abstract: The paper focuses on estimating the rectifier current in slim DC-Link AC drives, known for their reduced DC-Link capacitance. Particularly, at full load, the capacitor's filtering effect on the input voltage diminishes. This allows the estimation of the input voltage from DC-Link voltage measurements. We adopt a simplified model, treating the drive as a DC-Link that supplies a constant power load. Notably, the model accounts for the equivalent series resistance (ESR) of the capacitor, adding intricacy to the design. We propose an adaptive observer to estimate both the rectification current and the input voltage. We design a Luenberger state observer with output injection terms. The study on the observer stability is based on decoupling the state estimation errors from the input estimation errors through swapping design. Theoretical developments are validated through numerical simulations.

Keywords: Lyapunov methods, Nonlinear systems, Power systems
Abstract: This paper analyzes two synchronverters connected in parallel to a common capacitive-resistive load through resistive-inductive power lines. This system is conceptualized as a microgrid with two renewable energy sources controlled using the synchronverter algorithm. It is modeled as an interconnection of three port-Hamiltonian systems, and the dq-coordinates model is derived by averaging the frequencies. Applying the recent Leonov function theory, sufficient conditions to guarantee the global boundedness of the whole system's trajectories are provided. This is necessary to reach the global synchronization of microgrids. Additionally, a numerical example illustrates the potential resonance behavior of the microgrid.

Keywords: Adaptive control, Mechatronics, Identification
Abstract: Dynamic adaptation gain/learning rate have been introduced in the context of adaptation/learning algorithms using scalar adaptation gains/learning rates to accelerate the adaptation transients. This paper shows by means of theoretical analysis, simulations and, experimental results (on an active noise control system) that inserting a dynamic adaptation gain into the recursive least squares algorithm speeds up the adaptation transients in a deterministic environment and the asymptotic convergence in the stochastic case.

Keywords: Adaptive control, Networked control systems, Observers for Linear systems
Abstract: This paper investigates the bipartite time-varying output formation (BTVOF) tracking problem of heterogeneous discrete-time multi-agent systems (MASs). An adaptive distributed observer is constructed for each follower to estimate the state and system dynamics of the leader. Then, two distributed formation controllers with state feedback control and output feedback control are presented based on the proposed observer, respectively. Furthermore, based on Lyapunov stability theory, the sufficient solvability condition for achieving expected formation tracking is obtained. Finally, we perform a numerical simulation involving six quadrotors, and the effectiveness of the proposed BTVOF tracking protocol is verified by comparing the simulation results.

Keywords: Adaptive control, Direct adaptive control, Uncertain systems
Abstract: In this paper, we endow the model adaptive reference control (MRAC) with a novel parameter-dependent input normalization (PIN) to completely eliminate the conventional assumption of the high-frequency gain. Specifically, neither the sign nor the prior knowledge of the upper or lower bounds is required. To this end, we resort to an error augmentation together with a smart design of an adaptive law with a dead zone operation. Global stability in the mean square sense is established with the conventional proof concepts of the augmented error approach. In this way, no persistent excitation requirement is required. Although the system in question is assumed to be unity-relative-degree, the proposed technique can be easily extended to systems of arbitrary relative degrees. Finally, compared with the Nussbaum function-based methods in a numerical experiment, we show that transient behavior in our method is significantly improved.

Keywords: Adaptive control, Estimation, Optimization
Abstract: A second-order Newton-based extremum seeking (SONES) algorithm is presented to estimate directional inflection points for multivariable static maps. The design extends the first-order Newton-based extremum seeking algorithm that drives the system toward its peak point. This work provides perturbation matrices to estimate the second- and third-order derivatives necessary for implementing the SONES. A set of conditions are provided for the probing frequencies that ensure accurate estimation of the derivatives. A differential Riccati filter calculates the inverse of the third-order derivative. The local stability of the new algorithm is proven for general multivariable static maps using averaging analysis. The proposed algorithm ensures uniform convergence toward directional inflection points without requiring information about the curvature of the map and its gradient. Simulation results show the effectiveness of the proposed algorithm.

Keywords: Adaptive control, Adaptive systems, Lyapunov methods
Abstract: This paper investigates a class of continuous-time parameter identifiers with the so-called "dynamic adaptation gain (DAG)", inspired by its discrete-time counterpart [1]. A modular control law is first presented: this helps re-parametrize the system and highlight the required properties for the identifier. Then, a dynamic adaptation gain is constructed using a strictly passive system, strengthened by a feedthrough path: the resulting DAG is input strictly passive. Analysis shows that the identifier with the proposed DAG satisfies specific signal properties. An integrated analysis of both the control law and the identifier establishes boundedness of all closed-loop signals and state convergence. The convergence of the parameter estimate is also established under a persistence of excitation condition. Finally, an example inspired by a path-following problem demonstrates the convergence improvement achieved by the proposed DAG.

Keywords: Adaptive control, Autonomous vehicles, Adaptive systems
Abstract: This work shows how widely adopted longitudinal platooning protocols can be made adaptive via an immersion and invariance (I&I) approach. Such an I&I approach advances the state of the art on adaptive longitudinal platooning, which mostly relies on model reference adaptive control, while also extending the standard I&I design by compensating for exogenous effects from the preceding vehicle in the adaptive law. Two I&I designs are presented and compared with the corresponding state-of-the-art model reference adaptive control designs to demonstrate their effectiveness.

Keywords: Aerospace, Feedback linearization, LMIs
Abstract: This paper presents a cascaded control architecture, based on nonlinear dynamic inversion (NDI), for rigid body attitude control. The proposed controller works directly with the rotation matrix parameterization, that is, with elements of the Special Orthogonal Group SO{3}, and avoids problems related to singularities and non-uniqueness which affect other commonly used attitude representations such as Euler angles, unit quaternions, modified Rodrigues parameters, etc. The proposed NDI-based controller is capable of imposing desired linear dynamics of any order for the outer attitude loop and the inner rate loop, and gives control designers the flexibility to choose higher-order dynamic compensators in both loops. In addition, sufficient conditions are presented in the form of linear matrix inequalities (LMIs) which ensure that the outer loop controller renders the attitude loop almost globally asymptotically stable (AGAS) and the rate loop globally asymptotically stable (GAS). Furthermore, the overall cascaded control architecture is shown to be AGAS in the case of attitude error regulation. Lastly, the proposed scheme is compared with an Euler angles-based NDI scheme from literature for a tracking problem involving agile maneuvering of a multicopter in a high-fidelity nonlinear simulation.

Keywords: Aerospace, Flight control, Neural networks
Abstract: Modern aircraft are designed with redundant control effectors to cater for fault tolerance and maneuverability requirements. This leads to aircraft being over-actuated and requires control allocation schemes to distribute the control commands among control effectors. Traditionally, optimization-based control allocation schemes are used; however, for nonlinear allocation problems, these methods require large computational resources. In this work, an artificial neural network (ANN) based nonlinear control allocation scheme is proposed. The proposed scheme is composed of learning the inverse of the control effectiveness map through ANN, and then implementing it as an allocator instead of solving an online optimization problem. Stability conditions are presented for closed-loop systems incorporating the allocator, and computational challenges are explored with piece-wise linear effectiveness functions and ANN-based allocators. To demonstrate the efficacy of the proposed scheme, it is compared with a standard quadratic programming-based method for control allocation.

Keywords: Aerospace, Learning, Flight control
Abstract: This paper proposes a new dynamics-constrained path planning method for hypersonic vehicles. Due to vehicles’ limited maneuverability, path and dynamics are tightly coupled. To ensure executability, it is required to consider dynamic constraints and trajectory indexes in path planning. This leads to a more intricate problem formulation, and the path search space is high-dimensional and sparse. Existing methods mainly target low-velocity vehicles and rely on simple motion models, making them cannot be directly applied here. We address the problem with a graph learning method. This method begins by modeling the graph-search Markov decision process (MDP) on a Graph Attention Network (GAT) to find a path in the topological graph, which guides the subsequent trajectory generation. On top of the path, it uses a three-dimensional waypoint-crossing navigation (3D WCN) law to generate a trajectory under full dynamics and uncertainties. The GAT is trained using reinforcement learning, where the devised cost function includes both trajectory and path indexes. The trajectory index punishes trajectories that fail safety checks to ensure both the performance and executability of the path. The path index, aimed at eliminating the erratic impact of implicit trajectory representations and uncertainties, is calculated from the mean square error between the path and an optimal reference, thereby improving learning efficiency. Simulation results show superior optimality, adaptability, and milli-second-level computation speed.

Keywords: Aerospace, Sensor fusion, Kalman filtering
Abstract: We propose a Bayesian measurement masking method for global navigation satellite system (GNSS) positioning to mitigate disturbances from multi-path biases and modeling errors. The method removes erroneous GNSS observations to improve performance in downstream positioning algorithms. The measurement masking is posed as a binary classification problem, and solved by sequentially determining the noise statistics of individual pseudo-range measurements in the GNSS observations. Bayesian probabilities of mismatching noise models inform when outlier events such as multipath or non-line-of-sight (NLOS) events occur. We report a classification F1-score of >0.99 when the modeling assumptions are satisfied, and >0.97 when realistic modeling errors are included, both for dynamic and static receiver motion models.

Keywords: Algebraic/geometric methods, Robotics, Aerospace
Abstract: In this work, we address the design of tracking controllers that drive a mechanical system’s state asymptotically towards a reference trajectory. Motivated by aerospace and robotics applications, we consider fully-actuated systems evolving on the broad class of homogeneous spaces (encompassing all vector spaces, Lie groups, and spheres of any dimension). In this setting, the transitive action of a Lie group on the configuration manifold enables an intrinsic description of the tracking error as an element of the state space, even in the absence of a group structure on the configuration manifold itself (e.g., for S²). Such an error state facilitates the design of a generalized control policy depending smoothly on state and time that drives this geometric tracking error to a designated origin from almost every initial condition, thereby guaranteeing almost global convergence to the reference trajectory. Moreover, the proposed controller simplifies naturally when specialized to a Lie group or the n-sphere. In summary, we propose a unified, intrinsic controller guaranteeing almost global asymptotic trajectory tracking for fully-actuated mechanical systems evolving on a broader class of manifolds. We apply the method to an axisymmetric satellite and an omnidirectional aerial robot.

Keywords: Attack Detection, Aerospace, Flight control
Abstract: This work focuses on analyzing the vulnerability of unmanned aerial vehicles (UAVs) to stealthy black-box false data injection attacks on GPS measurements. We assume that the quadcopter is equipped with IMU and GPS sensors, and an arbitrary sensor fusion and controller are used to estimate and regulate the system’s states, respectively. We consider the notion of stealthiness in the most general form, where the attack is defined to be stealthy if it cannot be detected by any existing anomaly detector. Then, we show that if the closed-loop control system is incrementally exponentially stable, the attacker can cause arbitrarily large deviation in the position trajectory by compromising only the GPS measurements. We also show that to conduct such stealthy impact-full attack values, the attacker does not need to have access to the model of the system. Finally, we illustrate our results in a UAV case study.

Keywords: Nonlinear systems, Adaptive control, Robust adaptive control
Abstract: The paper considers some class of dynamical systems that called density systems. For such systems the derivative of quadratic function depends on so-called density function. The density function is used to set the properties of phase space, therefore, it influences the behaviour of investigated systems. A particular class of such systems is previously considered for (in)stability study of dynamical systems using the flow and divergence of a phase vector. In this paper, a more general class of such systems is considered, and it is shown that the density function can be used not only to study (in)stability, but also to set the properties of space in order to change the behaviour of dynamical systems. The development of control laws based on use the density function for systems with known parameters is considered. All obtained results are accompanied by the simulations illustrating the theoretical conclusions.

Keywords: Nonlinear systems, Agents-based systems, Switched systems
Abstract: This paper contributes to the design of an evader controller for the pursuit–evasion problem, where both agents are described by single integrator dynamics. The proposed evader controller only requires a slight knowledge of the pursuer control law and state. The control law guarantees the evasion of the pursuer under a proper design of the evader controller parameters. Moreover, the synthesis of the evader controller is constructive and simple–to–tune since it is in terms of a linear matrix inequality. The stability analysis of the tracking error dynamics is based on a Lyapunov–like function approach. The effectiveness of the proposed evader control design is illustrated through some simulation results.

Keywords: Nonlinear systems, Data driven control, Learning
Abstract: Representing nonlinear dynamical systems using the Koopman Operator and its spectrum has distinct advantages in terms of linear interpretability of the model as well as in analysis and control synthesis through the use of well-studied techniques from linear systems theory. As such, efficient computation of Koopman eigenfunctions is of paramount importance towards enabling such Koopman-based constructions. To this end, several approaches have been proposed in literature, including data-driven, convex optimization, and Deep Learning-based methods. In our recent work, we proposed a novel approach based on path-integrals that allowed eigenfunction computations using a closed-form formula. In this paper, we present several important developments such as finite-time computations, relaxation of assumptions on the distribution of the principal Koopman eigenvalues, as well as extension towards saddle point systems, which greatly enhance the practical applicability of our method.

Keywords: Nonlinear systems, Feedback linearization, Lyapunov methods
Abstract: We study the problem of dynamic partial state-feedback trajectory tracking for a class of minimumphase nonlinear systems in Byrnes-Isidori form using the Lyapunov redesign technique. We use an estimate of the unknown internal state in the feedback linearising control resulting in a dynamic control law. We consider three different approaches, namely, a simulation of the internal dynamics, an observer for the internal state, and model following control (MFC). Each design achieves asymptotic tracking. The observer-based approach and the MFC do so while overcoming the typical problem of unbounded discontinuous control signals if a sufficiently accurate estimate of the initial state of the process is available. Both approaches allow for asymptotic tracking with a continuous control signal if the initial state of the process is known. We demonstrate the tracking capabilities by a numerical example.

Keywords: Nonlinear systems, Filtering, Stochastic optimal control
Abstract: This work proposes a state-coupled model (SCM) for extended object tracking, which treats the orientation and velocity as two dependent variables. With this model, the distribution of multiple measurements is modeled via Gaussian mixture density to match the actual automotive radar or Lidar data. As a result, SCM becomes a highly nonlinear model with multiplicative noise. To handle this challenge, we use the deterministic sampling approach to update the kinematics and orientation information, followed by a constraint condition. And the parameter of extent is estimated under a Bayesian framework with pseudo-measurement. An evaluation is conducted on simulated data, which illustrates that the proposed model and filter are effective.

Keywords: Nonlinear systems, Identification for control, Robotics
Abstract: In this paper, we give sufficient conditions on the input our system for weak regular observability in the general case of landmark-based Simultaneous Localisation and Mapping (SLAM) both with a world-centric and a sensor-centric point of view. We show notably that in the sensor-centric point of view, the dynamics of the robot is not important for our concept of observability and only its state and input trajectories matter. Besides, we prove that tracking circular trajectories imply weak regular observability jointly for 2D systems with several types of commonly used measurements in SLAM.

Keywords: Output regulation, Adaptive control, Linear systems
Abstract: In this paper, we address the error-feedback output regulation problem for linear systems with non-smooth non-periodic exogenous signals. We design an adaptive observer under a relaxed persistence of excitation (PE) condition and we solve the error-feedback problem in the non-smooth case. In addition, we show that this relaxed PE condition is equivalent to a complete observability condition that can be checked textit{a priori} by means of an exogenous excitation (EE) condition. We finally show that, if the exogenous signals are generated by a traditional linear time-invariant implicit model, the EE condition is equivalent to a non-resonance-like condition.

Keywords: Output regulation, Data driven control, Adaptive systems
Abstract: In this paper, we solve the robust output regulation problem (RORP) for a class of nonlinear systems using a data-driven approach to reconstruct the internal model unknown nonlinear continuous map online from some input and output data. The data-driven model is then used to estimate the ideal feed-forward steady-state control inputs obtained by solving the regulator equation instead of implementing it with an extended observer as in previous studies. Secondly, we implement an output feedback stabilizer that does not rely on the complete knowledge of the system but on output measurement of the regulated output, making the proposed approach suitable for systems with modelling errors. Finally, we showed through detailed Lyapunov analysis that under certain conditions the closed-loop system achieves practical asymptotic stability.

Keywords: Output regulation, Linear systems
Abstract: The compensation of internal model is an approach to reduce the stabilization of a plant augmented with an (unstable) internal model to an equivalent stabilization problem for a fictitious plant, whose complexity is the same as that of the plant itself. This note presents a novel approach to construct this fictitious plant in state space, which results in simpler and more transparent procedures than those previously available. The main idea is to employ descriptor systems formalism, which facilitates manipulating systems with potentially non-proper transfer functions. The proposed procedure includes also an additional degree of freedom, which does not affect the complexity of the solution and can be used to tune its properties.

Keywords: LMIs, Output regulation, Lyapunov methods
Abstract: Setpoint regulation for a class of discrete-time nonlinear plants is addressed. The plant under consideration is a feedback interconnection of a square linear system (with the same number of input and output) and a nonlinearity obeying to an incremental quadratic constraint. Within this setting, we design a dynamic feedback controller ensuring, for the closed-loop system, internal exponential stability and output tracking of constant setpoints with tunable performance guarantees. A proportional-integral feedback controller is proposed. Combining Lyapunov theory and the use of a quadratic Lyapunov function, sufficient conditions for the design of the controller are stated in terms of linear matrix inequalities. An optimal controller design algorithm based on semidefinite programming is proposed to design the controller gains to improve transient performance and limit the control effort. Two numerical examples illustrate the application of the proposed methodology.

Keywords: Output regulation, Uncertain systems, Adaptive control
Abstract: Maneuvering control aims to drive the controlled system to achieve a desired motion along a parameterized path, where the path variable, a scalar, tunes additional dynamic tasks. In this paper, we investigate the maneuvering control problem of uncertain nonlinear systems from the perspective of output regulation. Initially, we demonstrate that the maneuvering control problem can be transformed into an output regulation problem. Then, we design a controller for maneuvering control based on the internal model principle and a neural predictor-based dynamic surface control method. We prove that the resulting closed-loop system is input-to-state stable. The effectiveness of proposed theoretical results is verified via a simulation example.

Keywords: Output regulation, Linear systems
Abstract: This paper investigates the general regulator problem with an internal model capable of adapting to the unknown parameters of persistent disturbances and/or reference signals affecting the measured output. Specifically, the proposed architecture based on stable compensators of internal model (CIM) allows to reduce the stabilization problem for an augmented system (the plant plus internal model) to that of a process without the internal model and with the complexity of the plant. Then, it is shown that, under certain conditions on the plant model, the proposed scheme makes the parameters of the internal model affect the closed-loop dynamics affinely. This, in turn, facilitates the incorporation of simple adaptation mechanisms with a global convergence.

Keywords: Agents-based systems, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper proposes a unified framework for the stability analysis of discrete-time nonlinear systems from social networks, including the Friedkin-Johnsen opinion model, two opinion dynamics models in the study of social power, and a general nonlinear polar opinion model. Three novel convergence results are proposed to treat various conditions based on LaSalle invariance principle. Several applications are provided to illustrate the power of the proposed framework.

Keywords: Boolean control networks and logic networks, Stability of nonlinear systems, Discrete event systems
Abstract: This study investigates the impact of multi-bit function perturbations (MFPs) on the steady-state distribution of Boolean networks (BNs), with a focus on robust stability. First, the algebraic formulation of BNs under MFPs is introduced. Then, a novel stability criterion is proposed, demonstrating that monotonically decreasing perturbations can effectively ensure the stability of BNs against MFPs. A significant advantage of this approach is that it enables robust stability assessment without remodeling the perturbed BNs. Applying this methodology to a simplified Boolean model of the lactose operon in emph{Escherichia coli} reveals that MFPs can serve as strategic tools to promote stability within BNs, rather than being merely disruptive.

Keywords: Fuzzy systems, LMIs
Abstract: This paper presents a new method for stability analysis of nonlinear systems through convex modeling. The proposed method consists of two main steps. First, we develop a quadratic optimization program to obtain the scheduling functions as well as the matrix vertices of an exact convex representation of a given nonlinear system. The induced numerical complexity of this new convex modeling can be significantly reduced with respect to the classical sector nonlinearity approach. Second, based on Lyapunov stability theory, we derive relaxed LMI sufficient conditions for nonlinear stability analysis, taking into account the characteristics of the new convex representation. A numerical example is given to illustrate the advantages of the proposal over related existing results.

Keywords: Fuzzy systems, Uncertain systems, Robust control
Abstract: Aggregating results in Multi-Criteria Decision Analysis (MCDA) poses a significant challenge. The common practice within MCDA evaluation is to apply multiple techniques to a single decision problem. This leads to the necessity of determining aggregated results. Despite the development of various aggregation approaches, simplifying the representation of rankings after evaluation, such as calculating mean results or compromise solutions, often neglects critical differences in results. This simplification undermines the value of knowledge presented to decision-makers, providing an incomplete problem overview. To this end, this paper introduces a novel approach to aggregating MCDA results using a fuzzy ranking. Presenting rankings as fuzzy sets, this innovative approach allows for determining membership degrees, revealing the robustness of placing alternatives in specific ranking positions. The proposed aggregating procedure facilitates more informed decision-making without compromising information from various methods or measures, offering comprehensive insights into the influence of different assessment approaches on robustness. The study focuses on a supplier selection evaluation problem, validating the approach using the COmbinative Distance-based ASsessment (CODAS) method and diverse normalization techniques, showcasing its effectiveness in providing decision-makers with a holistic understanding of evaluation results and alternative robustness.

Keywords: Maritime control, Stability of nonlinear systems
Abstract: This letter considers integral {line-of-sight} (LOS) guidance for curved path following for underactuated marine vehicles. {The} proposed guidance scheme renders the resulting closed-loop system input-to-state stable (ISS) with respect to a function of the vehicle's velocities. Moreover, if the forward and sideways velocities are proportional, we show that the origin of the closed-loop system is uniformly globally asymptotically stable (UGAS). Remarkably, these results are derived without the standard assumption of a small crab angle. Furthermore, we discuss how the path parameter should be selected to ensure that the along-track error remains zero for all time, and we show the connection between selecting the path parameter through differential equations and optimization. Finally, we demonstrate the effectiveness of the proposed approach through numerical simulations of an underwater vehicle.

Keywords: Subspace methods, Statistical learning
Abstract: While subspace identification methods (SIMs) are appealing due to their simple parameterization for MIMO systems and robust numerical realizations, a comprehensive statistical analysis of SIMs remains an open problem, especially in the non-asymptotic regime. In this work, we provide a finite sample analysis for a class of SIMs, which reveals that the convergence rates for estimating Markov parameters and system matrices are mathcal{O}(1/sqrt{N}), in line with classical asymptotic results. Based on the observation that the model format in classical SIMs is non-causal because of a projection step, we choose a parsimonious SIM that bypasses the projection step and strictly enforces a causal model to facilitate the analysis, where a bank of ARX models are estimated in parallel. Leveraging recent results from a finite sample analysis of an individual ARX model, we obtain a union error bound for an array of ARX models and proceed to derive error bounds for system matrices using robustness results for the singular value decomposition.

Keywords:
Abstract: The robot dogs that will be presented during CDC2024 are the state of the art of kinematics and sw/hw management of movements and balance. The machines can perform a series of evolutions and movements managed remotely or they can also operate with the help of AI for missions with pre-established objectives.

Keywords:
Abstract: The strong promotion of Artificial Intelligence (AI) as a panacea/menace/smokescreen across so many areas of human endeavor has captured the attention of scientists, policymakers, granting agencies and large fractions of the public. And yet, there is very serious concern both from inside and outside the AI community that the basis for optimism is misplaced, needs rigorous examination, sophisticated regulation, and public oversight. Primarily, there are unmet requirements of quantified performance, interpretation, generalization, etc, etc. Control theory and practice intersects many of the target domains pronounced for AI, and yet has both a more formal methodology and metrics of behavior, bringing the closed-loop firmly into the picture. The rally is aimed at exploring how to achieve this in order to meet these collective challenges and realize the potential opportunities in the control space.

The central themes for consideration are: a) Reclaiming Expertise: Reclaiming agency over Robotics, Automation and technical Decision Sciences. b) Supplying Expertise: Reconstituting AI for Safety using the tools from Control. c) Solving the PhD student dilemma: Why Engineering Science trumps purely ‘Data Driven’ Engineering and can help deliver on some of the promise. d) Stopping the ‘brain drain’ from Engineering to Computer Science and attract talent to work on this more material agenda.

*The lunch sessions are scheduled to start at 12:10 and conclude at 13:10. This timing is intended to allow for a smooth transition between the morning and afternoon sessions, giving participants sufficient time to navigate between consecutive events.*

Keywords:
Abstract: Graduate students, postdoctoral scholars, and early career researchers are warmly invited to a special lunch session. This lunch will be a perfect opportunity to meet new peers and to make a game plan for the rest of your conference agenda. It is also a chance to meet members of the newly formed NextCom committee within the IEEE Control Systems Society (CSS) and learn about upcoming resources, workshops, and networking opportunities aimed at supporting early career members of our community. All are welcome!

Please join us to learn about all the exciting things happening in CSS and explore getting more involved!

*The lunch sessions are scheduled to start at 12:10 and conclude at 13:10. This timing is intended to allow for a smooth transition between the morning and afternoon sessions, giving participants sufficient time to navigate between consecutive events.*

Keywords: Predictive control for linear systems, Predictive control for nonlinear systems
Abstract: This paper provides a comprehensive tutorial on a family of Model Predictive Control (MPC) formulations, known as MPC for tracking, which are characterized by including an artificial reference as part of the decision variables in the optimization problem. These formulations have several benefits with respect to the classical MPC formulations, including guaranteed recursive feasibility under online reference changes, as well as asymptotic stability and an increased domain of attraction. This tutorial paper introduces the concept of using an artificial reference in MPC, presenting the benefits and theoretical guarantees obtained by its use. We then provide a survey of the main advances and extensions of the original linear MPC for tracking, including its non-linear extension. Additionally, we discuss its application to learning-based MPC, discuss optimization aspects related to its implementation.

Keywords: Predictive control for linear systems, Stability of nonlinear systems, Optimization 
Abstract: As with classical MPC, the linear MPC for tracking formulation has been extended to many other control paradigms. This talk presents extensions of the formulation to tracking periodic reference signals, the harmonic MPC for tracking formulation, robust control, economic control and zone control. We highlight how the use of artificial references can also provide substantial benefits to some of the major MPC paradigms.

Keywords: Predictive control for nonlinear systems
Abstract: This talk shows how MPC for tracking and its variants (periodic, robust) can be naturally extended to non-linear systems. Theoretical properties and design aspects for nonlinear systems are covered, including terminal costs, convexity (or lack thereof), tractability of long horizon planning, and robust designs.

Keywords: Predictive control for nonlinear systems, Machine learning, Optimization
Abstract: This talk highlights how MPC for tracking formulations can be a key enabler to address problems beyond tracking in learning-based MPC. We discuss safe exploration in unknown environments, where a large region of attraction is crucial. Furthermore, application to distributed coverage problem in experiments is demonstrated, where tracking MPC enables efficient distributed coordination. We highlight how tracking MPC plays a crucial role to enable such applications, which go beyond tracking.

Keywords: Predictive control for linear systems, Stability of nonlinear systems, Optimization 
Abstract: This talk presents optimization aspects related to how the MPC for tracking formulations presented in the previous talks can be solved online. We discuss the application and tuning of first-order methods, such as ADMM, to efficiently implement these formulations. We focus on leveraging the specific semi-banded structure of the problem and addressing practical aspects such as soft-constrained formulations, warm-start techniques, or unfeasibility detection.

Keywords: Predictive control for linear systems, Stability of nonlinear systems, Optimization
Abstract: This talk provides first-hand insight and examples on the implementation of MPC for tracking. We show how to implement some of the formulations presented in the previous talks using currently available software tools (including both linear and non-linear). We also present results of applications of MPC for tracking formulations, including some real-world applications. The talk highlights both the ease of use of the formulations as well as its good practical performance.

Keywords: Identification, Switched systems, Learning
Abstract: In this paper, we study linear system identification in the setting of switched linear systems, where it is known that the underlying partially-observed linear dynamical system lies within a finite collection of known candidate models. We first consider the problem of identification from a given trajectory, which in this setting reduces to identifying the index of the true model with high probability. We characterize the finite-time sample complexity of this problem by leveraging recent advances in the non-asymptotic analysis of linear least-square methods in the literature. In comparison to the earlier results that assume no prior knowledge of the system, our approach takes advantage of the smaller hypothesis class and leads to the design of a learner with a dimension-independent sample complexity bound. Next, we consider the switching control of linear systems, where there is a candidate controller for each of the candidate models and data is collected through interacting the system with a collection of potentially destabilizing controllers. We develop a criterion that can detect those destabilizing controllers in finite time. By leveraging these results, we propose a data-driven switching strategy that identifies the unknown parameters of the underlying system. We then provide a non-asymptotic analysis of its performance and discuss its implications on the classical method of estimator-based supervisory control.

Keywords: Machine learning, Statistical learning, Data driven control
Abstract: In many control system applications, state constraint satisfaction needs to be guaranteed within a prescribed time. While this issue has been partially addressed for systems with known dynamics, it remains largely unaddressed for systems with unknown dynamics. In this paper, we propose a Gaussian process-based time-varying control method that leverages backstepping and control barrier functions to achieve safety requirements within prescribed time windows. It can be used to keep a system within a safe region or to make it return to a safe region within a limited time window. These properties are cemented by rigorous theoretical results. The effectiveness of the proposed controller is demonstrated in a simulation of a robotic manipulator.

Keywords: Neural networks, Nonlinear systems identification, Control of networks
Abstract: This article addresses the data-driven modeling and predictive control of networked systems. The main contribution is a novel methodology to develop a physics-informed recurrent neural network (PI-RNN) model with guaranteed stability properties. The idea consists in interconnecting multiple RNNs consistently with the known physical topology of the networked system, and in jointly training them while enforcing conditions guaranteeing the input-to-state stability of the PI-RNN model. The stability properties of the proposed PI-RNN model pave the way for the design of (i) a decentralized state observer and (ii) a Nonlinear Model Predictive Control (NMPC) regulator with convergence guarantees. The presented strategies are tested on a realistic large-scale networked system, i.e., a district heating network, demonstrating promising results from both the modeling and the control design perspective.

Keywords: Statistical learning, Learning, Reinforcement learning
Abstract: Model-based reinforcement learning is an effective approach for controlling an unknown system. It is based on a longstanding pipeline familiar to the control community in which one performs experiments on the environment to collect a dataset, uses the resulting dataset to identify a model of the system, and finally performs control synthesis using the identified model. As interacting with the system may be costly and time consuming, targeted exploration is crucial for developing an effective control-oriented model with minimal experimentation. Motivated by this challenge, recent work has begun to study finite sample data requirements and sample efficient algorithms for the problem of optimal exploration in model-based reinforcement learning. However, existing theory and algorithms are limited to model classes which are linear in the parameters. Our work instead focuses on models with nonlinear parameter dependencies, and presents the first finite sample analysis of an active learning algorithm suitable for a general class of nonlinear dynamics. We find that after a short burn-in time, the excess control cost of our algorithm achieves the optimal rate, up to logarithmic factors. We validate our approach in simulation, showcasing the advantage of active, control-oriented exploration in controlling nonlinear systems.

Keywords: Statistical learning, Learning, Data driven control
Abstract: We study the performance of risk-controlling prediction sets (RCPS), an empirical risk minimization-based formulation of conformal prediction, on a single trajectory of data from an unknown stochastic process. Our analysis characterizes the graceful degradation in RCPS performance as data becomes nearly arbitrarily dependent and nonstationary, subject only to a mild requirement that the underlying process is causal. By specializing this analysis, we find that RCPS attains guarantees comparable to those enjoyed on independent and identically distributed data whenever data is generated by an asymptotically stationary and mixing process. We then relate these conditions to system-theoretic properties like contractivity.

Keywords: Statistical learning, Machine learning, Identification
Abstract: We study non-parametric frequency-domain system identification from a finite-sample perspective. We assume an open loop scenario where the excitation input is periodic and consider the Empirical Transfer Function Estimate (ETFE), where the goal is to estimate the frequency response at certain desired (evenly-spaced) frequencies, given input-output samples. We show that under sub-Gaussian colored noise (in time-domain) and stability assumptions, the ETFE estimates are concentrated around the true values. The error rate is of the order of square root of M/Ntot where Ntot is the total number of samples and M is the number of desired frequencies. It also depends multiplicatively on du+(dudy)^(1/2), where du and dy are the dimensions of the input and output signals respectively. This rate remains valid for general irrational transfer functions and does not require a finite order state-space representation. By tuning M, we obtain a Ntot^(-1/3) finite-sample rate for learning the frequency response over all frequencies in the H infinity norm. Our result draws upon an extension of the Hanson-Wright inequality to semi-infinite matrices. We study the finite-sample behavior of ETFE in simulations.

Keywords: Agents-based systems, Autonomous systems, Cooperative control
Abstract: In this paper, we examine networks consisting of multiple interacting agents that have the flexibility to join or leave the network at any moment, which we term open multi-agent systems (OMASs). Expanding upon the recently introduced theoretical framework for analyzing the dynamic characteristics of OMASs, we extend our study to encompass agents with vector states and discrete-time evolution. A key point of our work is the employment of the concept of ”open stability” w.r.t. the infinity norm, which naturally makes the distance between two points in the state independent of the number of agents. This obviates the necessity for distance normalization, as required by the standard Euclidean norm. Within this framework, the main contribution of our work is that of establishing sufficient conditions for the open stability of an OMAS, which include the boundedness of the arrival/departure process and the paracontractivity of the OMAS in the absence of arrivals/departures, thus generalizing existing results for contractive OMASs. To underscore the practical relevance of our theoretical framework, we present the formulation of the dynamic max-consensus protocol for OMASs. Through numerical simulations, we demonstrate the alignment of this protocol with the theoretical findings outlined in this manuscript.

Keywords: Agents-based systems, Autonomous systems, Distributed control
Abstract: In this paper, we consider the problem of distributed average consensus in multi-agent systems, where each agent can come in or move out of the system, possibly multiple times. In the literature, such systems are referred to as open multi-agent systems. A typical goal in such settings is to use an iterative distributed algorithm to calculate the average of some quantities of interest each agent possesses, which can be crucial in many estimation, control, or optimization applications. We consider an open multi-agent setting and propose a distributed algorithm that allows the participating agents to track their average. More specifically, if the set of agents remaining in the computation eventually settles to a certain subset of agents, then the proposed algorithm allows them (under some mild connectivity conditions) to asymptotically reach consensus to the average of the quantities of interest these remaining agents hold. Analysis and numerical examples to illustrate the operation of the proposed algorithm are also provided.

Keywords: Distributed control, Cooperative control, Autonomous robots
Abstract: Preserving the connectivity of the underlying interaction graph in a multi-robot system is a necessary condition for allowing the group of robots to achieve a common task by resorting to only local information. However, in the context of open multi-robot systems, that is, when the number of robots in the team is not fixed, merely preserving connectivity of the current graph does not prevent the loss of connectivity after a robot joins/leaves the group. We present a distributed strategy to achieve biconnectivity, instead of simple connectivity, for a group of robots that allows establishment/deletion of interaction links as well as addition/removal of agents at anytime while guaranteeing that the connectivity, and thus functionality, of the team is always preserved. The proposed approach is completely distributed and embeds into a unique gradient-based control multiple constraints and requirements: (i) limited inter-robot communication ranges, (ii) limited field of view, (iii) desired inter-agent distances, and (iv) collision avoidance. Numerical simulations illustrate the effectiveness of our approach.

Keywords: Cooperative control, Distributed control, Agents-based systems
Abstract: In this paper, we address the distributed average consensus problem over directed networks in open multi-agent systems (OMAS), where the stability of the network is disrupted by frequent agent arrivals and departures, leading to a time-varying average consensus target. To tackle this challenge, we introduce a novel ratio consensus algorithm (OPENRC) based on acknowledgement feedback, designed to be robust to agent arrivals and departures, as well as to unbalanced directed network topologies. We demonstrate that when all active agents execute the OPENRC algorithm, the sum of their state variables remains constant during quiescent epochs when the network remains unchanged. By assuming eventual convergence during such quiescent periods following persistent variations in system composition and size, we prove the convergence of the OPENRC algorithm using column-stochasticity and mass- preservation properties. Finally, we apply and evaluate our proposed algorithm in a simulated environment, where agents are departing from and arriving in the network to highlight its resilience against changes in the network size and topology.

Keywords: Agents-based systems, Optimal control, Control applications
Abstract: This paper considers opinion dynamics on an n-sphere, where leaders steer their followers' opinions to a desirable region on the sphere. In the process of doing so, the leaders may encounter other leaders who influence the opinion of these followers and entice them away. To prevent this, the leader will need to exert additional effort to keep the followers together and away from the influence of the adversarial leaders. This paper formulates this problem as an optimal control problem of choosing a piecewise geodesic path to the opinion goal region while optimizing the cost incurred due to the additional effort required in order to negate the influence of the adversarial leaders. The paper builds upon several well-known concepts such as representation of opinions as vectors on an n-sphere, bounded confidence model that defines the interaction of proximal opinions, and the methods by which leaders try to influence the opinion of their followers and strengthen their hold on them. However, instead of looking at consensus and related concepts, the paper produces novel results on how leaders can successfully steer the opinion of their followers over the spherical manifold. Simulation results are given to illustrate the concepts presented in the paper.

Keywords: Agents-based systems, Autonomous systems, Optimal control
Abstract: Executing computation tasks through cloud–edge collaboration has emerged as a promising method to enhance the quality of service for applications. Typically, cloud servers and edge servers are selfish and rational. Therefore, it is crucial to develop incentive mechanisms that maximize cloud server profit and simultaneously motivate idle edge servers to participate in the task executing process while edge servers, as autonomous agents,

choose their resource-sharing levels by themselves. This paper addresses the challenges of resource limitations and heterogeneity in edge computing by proposing a novel mechanism that integrates contract theory with Stackelberg game properties considering asymmetric information and the autonomous nature of edge servers. To propose an optimal mechanism, we design a linear form of reward function such that the mechanism’s goals are met. The mechanism allows edge servers to autonomously decide their level of resource contribution while ensuring the maximization of the cloud server's utility. The proposed mechanism not only facilitates efficient resource utilization but also guarantees the truthful and rational participation of edge servers. Initially, the proposed mechanism is conceptualized as a non-convex functional optimization with a dual continuum of constraints. However, we illustrate that by deriving an equivalent representation of the constraints, it can be transformed into a convex optimal control problem. Simulation results demonstrate the efficiency of our proposed incentive mechanism approach.

Keywords: Networked control systems
Abstract: The proliferation of cloud computing technologies has paved the way for deploying networked encrypted control systems, offering high performance, remote accessibility and privacy. However, in scenarios where the control algorithms run on third-party cloud service providers, the control's logic might be changed by a malicious agent on the cloud. Consequently, it is imperative to verify the correctness of the control signals received from the cloud. Traditional verification methods, like zero-knowledge proof techniques, are computationally demanding in both proof generation and verification, may require several rounds of interactions between the prover and verifier and, consequently, are inapplicable in real-time control system applications. In this paper, we present a novel computationally inexpensive verifiable computing solution inspired by the probabilistic cut-and-choose approach. The proposed scheme allows the plant's actuator to validate the computations accomplished by the encrypted cloud-based networked controller without compromising the control scheme’s performance. We showcase the effectiveness and real-time applicability of the proposed verifiable computation scheme using a remotely controlled Khepera-IV differential-drive robot.

Keywords: Cyber-Physical Security, Resilient Control Systems, Attack Detection
Abstract: This paper investigates the resilient control design problem of event-triggered Cyber-Physical Systems (CPS) when the system is subject to covert attacks. To avoid continuous communication and save resources, event-triggered schemes are considered on the communication channels between the plant and the Command and Control (C&C) center, on both the input and output channels. By utilizing the notion of auxiliary systems, we have provided an approach to estimate the cyberattack signals and design a resilient controller which can achieve stability of the CPS system. Finally, illustrative examples are provided featuring a Vertical Take-Off and Landing (VTOL) aircraft to demonstrate the effectiveness and capabilities of our approach in mitigating covert attacks.

Keywords: Cyber-Physical Security, Resilient Control Systems, Linear systems
Abstract: This paper presents a novel frequency-domain approach for control input filtering in networked control systems subject to integrity attacks on control inputs. The proposed approach relies on adding linear time-invariant filters to control loops, placed between received control actions and plant actuators, whose aim is to mitigate the impact of attacks on control performance. The filter synthesis problem can be cast in terms of a multi-objective synthesis problem subject to multiple frequency-domain inequality specifications in finite and infinite frequency ranges. The problem is formulated as a convex optimization problem subject to a set of Linear Matrix Inequalities. Simulation results demonstrate the effectiveness of the proposed security-enhancing approach.

Keywords: Estimation, Networked control systems, Sensor networks
Abstract: This paper is concerned with the preservation of privacy in remote state estimation of cyber-physical systems. A privacy-preserving transmission scheduling strategy against eavesdropping is proposed, incorporating three operational modes for sensors: silence, direct transmission, and noise-injected transmission. This strategy is designed to minimize the transmission cost and estimation error covariance for the remote estimator while maximizing the estimation error covariance for eavesdroppers. Threshold structures are demonstrated for optimal transmission schedules in different scenarios. Additionally, a novel correlation between the optimal transmission choice and the magnitude of injection noise is presented, particularly pertinent to scenarios involving direct transmission and transmission with injection noise. This correlation is important in balancing transmission information integrity against privacy concerns. Finally, several numerical examples are presented to demonstrate the effectiveness of the theoretical results.

Keywords: Power systems, Resilient Control Systems, Nonlinear systems
Abstract: Fast charging stations for electric vehicles (EVs) often consist of charging units with multiple modules connected in parallel to achieve high power ratings and can suffer from cyber-attacks in the modern smart grid architecture or measurement/sensor/communication failure. In this paper, an EV charging unit with multiple single active bridge (SAB) converters connected in parallel is considered and a novel nonlinear resilient controller is proposed to achieve accurate power sharing among the converters with inherent current limitation, while enhancing the entire charging unit resilience under abnormal scenarios. By taking into account the dynamic model of the parallel SAB converters and using invariant set and nonlinear ultimate boundedness theories, it is proven that the current injected to the EV battery by each converter is limited below a maximum value at all times. Furthermore, based on the novel dynamic structure of the controller, it is shown that when an unexpected event (e.g. setpoint attack, sensor fault) occurs at one or more converters, the inherent current limiting property protects the corresponding devices and the remaining non-compromised units automatically share the remaining power to the battery, thus enhancing the EV charger resilience. An EV charging unit with 4 modules is simulated to verify the effectiveness and increased resilience of the proposed control approach under setpoint attacks and sensor/measurement faults.

Keywords: Cooperative control, Lyapunov methods, Distributed control
Abstract: This paper presents fully distributed, resilient secondary defense strategies for AC microgrids considering both communication link faults and a broader spectrum of unbounded false data injection (FDI) attacks on control input channels. In contrast to existing solutions that address bounded faults or unbounded attacks on the input channels with bounded first-order time derivatives, the proposed strategies aim to enhance the defense capabilities against polynomially unbounded FDI attacks while the communication links are under faults. Resilient defense strategies for AC microgrids are developed to mitigate the adverse effects of the polynomially unbounded FDI attacks on control input channels and communication link faults, ensuring the stable and resilient operation of AC microgrids. Through rigorous Lyapunov-based stability analysis, the formal certification of the proposed strategies is demonstrated in achieving uniformly ultimately bounded convergence in frequency regulation, voltage containment, and active power sharing in multi-inverter-based AC microgrids. The effectiveness of these resilient strategies is further validated on a modified IEEE 34-bus test feeder system with four inverter-based distributed energy resources.

Keywords: Optimization algorithms, Network analysis and control, Learning
Abstract: This paper investigates a zeroth-order algorithm to solve a distributed weakly convex optimization problem over a multi-agent network, where each agent in the network has access to a local weakly convex objective function. We utilize a pseudo-gradient estimation scheme with orthogonal random directions to estimate the gradient information, which is more general than the existing coordinate descent, discretized gradient descent, and spherical smoothing methods. Moreover, we design a projected pseudo-gradient algorithm with a diminishing step size to achieve the optimal solution. Furthermore, we show the proposed algorithm converges to the optimal solution with an O({ln k}/{sqrt k}) convergence rate from the perspective of the Moreau envelope. Finally, we provide a numerical example to illustrate the effectiveness of the proposed algorithm.

Keywords: Optimization algorithms, Optimization
Abstract: This paper considers a consensus optimization problem, where all the nodes in a network, with access to the zeroth-order information of its local objective function only, attempt to cooperatively achieve a common minimizer of the sum of their local objectives. To address this problem, we develop ZoPro, a zeroth-order proximal algorithm, which incorporates a zeroth-order oracle for approximating Hessian and gradient into a recently proposed, high-performance distributed second-order proximal algorithm. We show that the proposed ZoPro algorithm, equipped with a dynamic stepsize, converges linearly to a neighborhood of the optimum in expectation, provided that each local objective function is strongly convex and smooth. Extensive simulations demonstrate that ZoPro converges faster than several state-of-the-art distributed zeroth-order algorithms and outperforms a few distributed second-order algorithms in terms of running time for reaching given accuracy.

Keywords: Optimization algorithms, Machine learning, Cooperative control
Abstract: The selection of stepsizes has always been an elusive task in distributed optimization and learning. Although some stepsize-automation approaches have been proposed in centralized optimization, these approaches are inapplicable in the distributed setting. This is because in distributed optimization/learning, letting individual agents adapt their own stepsizes unavoidably results in stepsize heterogeneity, which can easily lead to

algorithmic divergence. To solve this issue, we propose an approach that enables agents to adapt their individual stepsizes without any manual adjustments or global knowledge of the objective function. To the best of our knowledge, this is the first algorithm to successfully automate stepsize selection in distributed optimization/learning. Its performance is validated using several machine learning applications, including logistic regression, matrix factorization, and image classification.

Keywords: Optimization
Abstract: This paper introduces a distributed parallel Alternating Direction Method of Multipliers (ADMM) algorithm for solving the distributed constrained optimization problem over directed graphs. To effectively handle the effect of asymmetric information communication on the convergence of the optimization algorithm, a surplus-based averaging consensus algorithm is integrated into the ADMM-based optimization algorithm. Unlike existing distributed ADMM algorithms over directed graphs that focus on the case with solely local constraints, the proposed algorithm can deal with both local constraints and coupling constraints. Under the assumption that the objective function is convex and the underlying graph is strongly connected, the convergence of the surplus-based ADMM to an optimal solution of the distributed constrained problem is proved. Finally, numerical simulations are conducted to validate the effectiveness of the proposed algorithm.

Keywords: Cooperative control, Distributed control, Agents-based systems
Abstract: This paper studies a homogeneous decentralized multi-armed bandit problem, in which a network of multiple agents faces the same set of arms, and each agent aims to minimize its own regret. A fully decentralized upper confidence bound (UCB) algorithm is proposed for a multi-agent network whose neighbor relations are described by a directed graph. It is shown that the decentralized algorithm guarantees each agent to achieve a lower logarithmic asymptotic regret compared to the classic UCB algorithm, provided the neighbor graph is strongly connected. The improved asymptotic regret upper bound is reciprocally related to the maximal size of a local neighborhood within the network. The roles of graph connectivity, maximum local degree, and network size are analytically elucidated in the expression of regret.

Keywords: Optimization algorithms, Machine learning, Computer/Network Security
Abstract: We introduce a novel differentially private algorithm for online federated learning that employs temporally correlated noise to enhance utility while ensuring privacy of continuously released models. To address challenges posed by DP noise and local updates with streaming non-iid data, we develop a perturbed iterate analysis to control the impact of the DP noise on the utility. Moreover, we demonstrate how the drift errors from local updates can be effectively managed under a quasi-strong convexity condition. Subject to an (epsilon, delta)-DP budget, we establish a dynamic regret bound over the entire time horizon, quantifying the impact of key parameters and the intensity of changes in dynamic environments. Numerical experiments confirm the efficacy of the proposed algorithm.

Keywords: Network analysis and control, Identification, Networked control systems
Abstract: This paper deals with the design of Excitation and Measurement Patterns (EMPs) for the identification of dynamic networks, when the objective is to identify only a subnetwork embedded in a larger network. Recent results have shown how to construct EMPs that guarantee identifiability for a range of networks with specific graph topologies, such as trees, loops, parallel networks, or Directed Acyclic Graphs (DAGs). However, an EMP that is valid for the identification of a subnetwork taken in isolation may no longer be valid when that subnetwork is embedded in a larger network. Our main contribution is to exhibit conditions under which it does remain valid, and to propose ways to enhance such EMP when these conditions are not satisfied.

Keywords: Network analysis and control, Identification
Abstract: We analyze the identifiability of directed acyclic graphs in the case of partial excitation and measurement. We consider an additive model where the nonlinear functions located in the edges depend only on a past input, and we analyze the identifiability problem in the class of pure nonlinear functions satisfying f(0)=0. We show that any identification pattern (set of measured nodes and set of excited nodes) requires the excitation of sources, measurement of sinks and the excitation or measurement of the other nodes. Then, we show that a directed acyclic graph (DAG) is identifiable with a given identification pattern if and only if it is identifiable with the measurement of all the nodes. Next, we analyze the case of trees where we prove that any identification pattern guarantees the identifiability of the network. Finally, by introducing the notion of a generic nonlinear network matrix, we provide sufficient conditions for the identifiability of DAGs based on the notion of vertex-disjoint paths.

Keywords: Network analysis and control, Learning, Iterative learning control
Abstract: In this paper, we consider the problem of social learning, where a group of agents embedded in a social network are interested in learning an underlying state of the world. Agents have incomplete, noisy, and heterogeneous sources of information, providing them with recurring private observations of the underlying state of the world. Agents can share their learning experience with their peers by taking actions observable to them, with values from a finite feasible set of states. Actions can be interpreted as samples from the beliefs which agents may form and update on what the true state of the world is. Sharing samples, in place of full beliefs, is motivated by the limited communication, cognitive, and information-processing resources available to agents especially in large populations. Previous work pose the question as to whether learning with probability one is still achievable if agents are only allowed to communicate samples from their beliefs. We provide a definite positive answer to this question, assuming a strongly connected network and a ``collective distinguishability'' assumption, which are both required for learning even in full-belief-sharing settings. In our proposed belief update mechanism, each agent's belief is a normalized weighted geometric interpolation between a fully Bayesian private belief --- aggregating information from the private source --- and an ensemble of empirical distributions of the samples shared by her neighbors over time. By carefully constructing asymptotic almost-sure lower/upper bounds on the frequency of shared samples matching the true state/or not, we rigorously prove the convergence of all the beliefs to the true state, with probability one.

Keywords: Network analysis and control, Model/Controller reduction, Large-scale systems
Abstract: Kron reduction is concerned with the elimination of interior nodes of physical network systems such as linear resistor electrical circuits. In this paper it is shown how this can be extended to networks with nonlinear static relations between the variables associated to the edges of the underlying directed graph.

Keywords: Network analysis and control, Networked control systems, Nonlinear systems
Abstract: We study practical synchronization for heterogeneous networks of nonlinear systems in the presence of bounded perturbations. Under the assumption that the nodes are state semi-passive and the interconnection graph admits a spanning tree, we establish uniform ultimate boundedness of the solutions and, consequently, practical synchronization. That is, we show that the systems’ trajectories approach each other up to a steady state error. The magnitude of this steady-state error can be made arbitrarily small by increasing a scalar coupling gain. The results are shown to hold under the assumption that (at least) a single “well-located” node in the network enjoys some robustness properties. Our theoretical results are fairly general in regards to the topology and are illustrated in simulation on a case-study of networked mobile robots.

Keywords: Network analysis and control, Networked control systems, Time-varying systems
Abstract: This paper investigates strong structural controllability for multi-agent systems (MAS) with dynamically joining and leaving agents. Changing sets of agents can compromise controllability if only a subset of agents is equipped with control inputs. Based on the graph-theoretical concept of zero forcing, conditions are derived under which MAS with joining and leaving agents can maintain strong structural controllability without needing to reconfigure the leader set. Two methods are proposed: The first, employs so-called zero forcing chains, to provide definitive statements on maintaining SSC. The second is based on forcing agents and offers a quick assessment of feasibility for joining or leaving, but confirms SSC only in specific cases.

Keywords: Game theory, Modeling, Optimization
Abstract: A large fraction of total healthcare expenditure occurs due to end-of-life (EOL) care, which means it is important to study the problem of more carefully incentivizing necessary versus unnecessary EOL care because this has the potential to reduce overall healthcare spending. This paper introduces a principal-agent model that integrates a mixed payment system of fee-for-service and pay-for-performance in order to analyze whether it is possible to better align healthcare provider incentives with patient outcomes and cost-efficiency in EOL care. The primary contributions are to derive optimal contracts for EOL care payments using a principal-agent framework under three separate models for the healthcare provider, where each model considers a different level of risk tolerance for the provider. We derive these optimal contracts by converting the underlying principal-agent models from a bilevel optimization problem into a single-level optimization problem that can be analytically solved. Our results are demonstrated using a simulation where an optimal contract is used to price intracranial pressure monitoring for traumatic brain injuries.

Keywords: Game theory, Distributed control, Optimization algorithms
Abstract: This paper studies N-cluster games with second-order dynamics, wherein both local convex set constraints and nonlinear coupled inequality constraints are took into account. The presence of second-order dynamics coupled with constraints puts up obstacles to algorithm design and analysis, since it may be impossible to directly determine the decisions of players based on their control inputs. In order to facilitate autonomous execution of N-cluster game tasks by second-order players, this paper employs state feedback, projection, primal-dual, dynamic average consensus, and passivity methods to design a distributed algorithm, which can regulate the decisions of players to satisfy the set constraints all the time. Additionally, rigorous analysis of the convergence of the proposed algorithm is provided in this paper. Finally, a simulation example is presented to validate the effectiveness of the proposed algorithm.

Keywords: Game theory, Nonlinear systems, Optimization
Abstract: Feedback-evolving games is a framework that models the co-evolution between payoff functions and an environmental state. It serves as a useful tool to analyze many social dilemmas such as natural resource consumption, behaviors in epidemics, and the evolution of biological populations. However, it has primarily focused on the dynamics of a single population of agents. In this paper, we consider the impact of two populations of agents that share a common environmental resource. We focus on a scenario where individuals in one population are governed by an environmentally ``responsible" incentive policy, and individuals in the other population are environmentally ``irresponsible". An analysis on the asymptotic stability of the coupled system is provided, and conditions for which the resource collapses are identified. We then derive consumption rates for the irresponsible population that optimally exploit the environmental resource, and analyze how incentives should be allocated to the responsible population that most effectively promote the environment via a sensitivity analysis.

Keywords: Game theory, Nonlinear systems, Reinforcement learning
Abstract: In this paper, we consider the n times n two-payer zero-sum repeated game in which one player (player X) employs the popular Hedge (also called multiplicative weights update) learning algorithm while the other player (player Y) adopts the myopic best response. The theoretical analysis on the dynamics of such game system is still rare, which is however of promising interests. We investigate the dynamics of such Hedge-myopic system by defining a metric Q(textbf{x}_t), which measures the distance between the stage strategy textbf{x}_t and Nash Equilibrium (NE) strategy of player X. We analyze the trend of Q(textbf{x}_t) and prove that it is bounded and can only take finite values on the evolutionary path when the payoff matrix is rational and the game has an interior NE. Based on this, we prove that the stage strategy sequence of both players are periodic after finite stages and the time-averaged strategy of player Y within one period is an exact NE strategy.

Keywords: Game theory, Learning, Reinforcement learning
Abstract: This paper addresses the challenge of limited observations in non-cooperative multi-agent systems where agents can have partial access to other agents' actions. We present the generalized individual Q-learning dynamics that combine belief-based and payoff-based learning for the networked interconnections of more than two self-interested agents. This approach leverages access to opponents' actions whenever possible, demonstrably achieving a faster (guaranteed) convergence to quantal response equilibrium in multi-agent zero-sum and potential polymatrix games. Notably, the dynamics reduce to the well-studied smoothed fictitious play and individual Q-learning under full and no access to opponent actions, respectively. We further quantify the improvement in convergence rate due to observing opponents' actions through numerical simulations.

Keywords: Game theory, Learning
Abstract: B-Matching is a special case of matching problems where nodes can join multiple matchings with the degree of each node constrained by an upper bound, the node's B-value. The core solution of a bipartite B-matching is both a matching between the nodes respecting the upper bound constraint and an allocation of the weights of the edges among the nodes such that no group of nodes can deviate and collectively gain higher allocation. We present two learning dynamics that converge to the core of the bipartite B-matching problems. The first dynamics are centralized dynamics in the nature of the Hungarian method, which converge to the core in a polynomial time. The second dynamics are distributed dynamics, which converge to the core with probability one. For the distributed dynamics, a node maintains only a state consisting of (i) the aspiration levels for all of its possible matches and (ii) the matches, if any, to which it belongs. The node does not keep track of its history nor is it aware of the environment state. In each stage, a randomly activated node proposes to form a new match and changes its aspiration based on the success or failure of its proposal. At this stage, the proposing node inquires about the aspiration of the node it wants to match with to calculate the feasibility of the match. The environment matching structure changes whenever a proposal succeeds. A state is absorbing for the distributed dynamics if and only if it is in the core of the B-matching.

Keywords: Optimal control, Adaptive control, Nonlinear systems
Abstract: Optimal control-based robust controllers have been extensively studied to mitigate the impacts of disturbances. However, a common limitation of existing methodologies is the incorporation of disturbance information, such as upper bounds, directly into the cost function to design the controller. This practice restricts the applicability of such controllers primarily to problems where disturbance bounds are specified and known a priori. Recognizing the limitation, this paper proposes a novel robust optimal adaptive controller for nonlinear uncertain systems which is formulated without requiring knowledge of the bounds of uncertainty. By employing online parameter estimation, the considered design problem is converted into a nonlinear quadratic optimal control problem. Then an indirect Legendre-Gauss pseudospectral method (ILGPM) is developed to enable efficient numerical implementation. Numerical simulations illustrate the effectiveness of the proposed controller.

Keywords: Optimal control, Control applications
Abstract: Constrained control of the average of outputs of nonlinear systems is sometimes needed to meet regulatory requirements, e.g., to guarantee radio frequency electromagnetic field levels from wireless transmitters, or to guarantee ammonium levels from wastewater treatment plants. Switching control signals have become a concern in some of these applications. To analyze the reason for this, the paper first formulates a general constrained optimal control problem that embeds the average control problem as a special case. The solution to the problem is then outlined using Pontryagin’s maximum principle. The optimal control is computed analytically using logarithmic barrier functions for the control signal constraints. Simulation of the optimizing Euler–Lagrange equations is then used to characterize the constrained average control and output signals over time, which shows that switching occurs due the slow dynamics and the constraints. The paper also contributes by proving that the computed control is optimal, since the second derivative of the Hamiltonian is positive.

Keywords: Optimal control, Automotive systems, Autonomous vehicles
Abstract: The main goal of Eco-Driving (ED) is to maximize for energy efficiency. This study evaluates the energy gains of an ED system for an electric vehicle, obtained from a predictive optimal controller, in a real-world traffic scenario. To this end, a Visual driver Advisory System (VAS) in the form of a personal tablet is used to advise the driver to follow a target eco-speed via a screen. Two Renault Zoe electric cars, one equipped with the different modules for ED and one without, are used to perform field tests on a route between Rueil-Malmaison and Bougival in France. Overall, the eco-driven consumed, on average, 4.6 % less energy than the non-eco-driven car with a maximum of 2 % change in average speed.

Keywords: Optimal control, Autonomous systems, Agents-based systems
Abstract: Quadratic performance indices associated with linear plants offer simplicity and lead to linear feedback control laws, but they may not adequately capture the complexity and flexibility required to address various practical control problems. One notable example is to improve, by using possibly nonlinear laws, on the trade-off between rise time and overshoot commonly observed in classical regulator problems with linear feedback control laws. To address these issues, non-quadratic terms can be introduced into the performance index, resulting in nonlinear control laws. In this study, we tackle the challenge of solving optimal control problems with non-quadratic performance indices using the closed-loop neighboring extremal optimal control (NEOC) approach and homotopy method. Building upon the foundation of the Linear Quadratic Regulator (LQR) framework, we introduce a parameter associated with the non-quadratic terms in the cost function, which is continuously adjusted from 0 to 1. We propose an iterative algorithm based on a closed-loop NEOC framework to handle each gradual adjustment. Additionally, we discuss and analyze the classical work of Bass and Webber, whose approach involves including additional non-quadratic terms in the performance index to render the resulting Hamilton-Jacobi equation analytically solvable. Our findings are supported by numerical examples.

Keywords: Optimal control, Autonomous systems, Game theory
Abstract: An Escort and Defense Scenario is presented in which a high-value Target maneuvers through a high-risk region while being escorted by a mobile, defensive agent. Along this trajectory, an Attacker may be launched from one of several possible launch locations. If a launch occurs, the three agents play out a Target-Attacker-Defender (TAD) differential game in which the Defender attempts to intercept the Attacker at maximal distance from the Target while the Attacker strives to maneuver as close as possible to the Target. Prior to launch, the Target and Defender strive to preposition themselves in advantageous positions to effectively respond to a potential threat while simultaneously moving towards a safety region. An augmented collocation-based direct optimal control method is developed to solve the escort problem by solving and utilizing the value of the TAD differential subgame at each collocation point while simultaneously optimizing the primary optimal control problem.

Keywords: Optimal control, Autonomous vehicles, Stochastic optimal control
Abstract: This paper proposes a path-following policy for linear systems subject to stochastic disturbances. The problem is framed as that of choosing both the control inputs and the trajectory's speed to minimize an infinite-horizon expected quadratic cost taking into account the state and the input. The path is modeled by an exosystem. It is shown that certainty equivalence holds when the path is a straight line. The proposed path-following policy improves the cost of the optimal constant speed trajectory-tracking policy associated with the exosystem. This policy guarantees that the tracking error converges to zero in the absence of disturbances. Numerical examples highlight the advantages of the method with respect to trajectory-tracking.

Keywords: Optimization algorithms, Distributed control, Smart grid
Abstract: This paper presents a fully distributed algorithm for scheduling electric vehicle (EV) charging and discharging to flatten the total load of the grid, while considering constraints on grid transmission capacity. As a fully distributed solution, the proposed algorithm operates without the need for a central unit. Instead, each agent only communicates a single dual variable with its neighboring agents based on a communication graph, and thus no private information is shared. In particular, the algorithm does not rely on initial conditions, ensuring robustness in online changes of operational conditions. Simulation results verify the effectiveness of the proposed algorithm.

Keywords: Optimization algorithms, Distributed control, Large-scale systems
Abstract: In this paper, we focus on the optimization of large-scale multi-agent systems, where agents collaboratively optimize the sum of local objective functions through their own continuous and/or discrete decision variables, subject to global coupling constraints and local constraints. The resulting Mixed-Integer Nonlinear Programmings (MINLPs) are NPhard, non-convex, and large-scale. Therefore, this paper aims to design distributed algorithms to find feasible suboptimal solutions with a guaranteed bound. To this end, considering dual decomposition as an effective method to decompose largescale constraint-coupled optimization problems, we first show, based on the convexification effects of large-scale MINLPs, that the primal solutions from the dual are near-optimal under certain conditions. This expands recent results in Mixed- Integer Linear Programmings (MILPs) to the nonlinear case but requires additional efforts on the proof. Utilizing this result to tighten the coupling constraints, we develop a fully distributed algorithm for the tightened problem, based on dual decomposition and consensus protocols. The algorithm is guaranteed to provide feasible solutions for the original MINLP. Moreover, asymptotic suboptimality bounds are established for the obtained solution. Finally, the efficacy of the method is verified through numerical simulations.

Keywords: Optimization algorithms, Distributed control, Cooperative control
Abstract: This paper presents a novel accelerated distributed algorithm for unconstrained consensus optimization over static undirected networks. The proposed algorithm combines the benefits of acceleration from momentum, the robustness of the alternating direction method of multipliers, and the computational efficiency of gradient tracking to surpass existing state-of-the-art methods in convergence speed, while preserving their computational and communication cost. First, we prove that, by applying momentum on the average dynamic consensus protocol over the estimates and gradient, we can study the algorithm as an interconnection of two singularly perturbed systems: the outer system connects the consensus variables and the optimization variables, and the inner system connects the estimates of the optimum and the auxiliary optimization variables. Next, we prove that, by adding momentum to the auxiliary dynamics, our algorithm always achieves faster convergence than the achievable linear convergence rate for the non-accelerated alternating direction method of multipliers gradient tracking algorithm case. Through simulations, we numerically show that our accelerated algorithm surpasses the existing accelerated and non-accelerated distributed consensus first-order optimization protocols in convergence speed.

Keywords: Optimization algorithms, Electrochemical processes, Nonlinear systems
Abstract: This paper presents a framework for parameter estimation of Solid Oxide Electrolyzer Stack (SOES) model. The complexity of multi-physics in SOES models poses a unique challenge for parameter identification due to the presence of nonlinearities, the large number of parameters, and few available measurements. Consequently, this study presents an enhanced method of parameter estimation, based on the Gauss-Newton optimization algorithm, incorporating a truncated Singular Value Decompostion (SVD) of a normalized sensitivity matrix. This modification prioritizes the update of parameters in the directions of high sensitivity while limiting the condition number of the matrix inverted to choose the step size, thus attenuating the adverse effects of noise and model errors unavoidable in the estimation process. This departure from the conventional approaches, allows a more nuanced and effective identification strategy tailored to the intricacies of SOESs. The proposed method is validated using data from an experimental test bench and compared to other identification methods.

Keywords: Optimization algorithms, Optimal control, Optimization
Abstract: This letter examines the question of finding feasible points to discrete-time optimal control problems. The optimization problem of finding a feasible trajectory is transcribed to an unconstrained optimal control problem. An efficient algorithm, called FP-DDP, is proposed that solves the resulting problem using Differential Dynamic Programming preserving feasibility with respect to the system dynamics in every iteration. Notably, FP-DDP admits global and rapid local convergence properties induced by a combination of a Levenberg-Marquardt method and an Armijo-type line search. An efficient implementation of FP-DDP within acados is compared to established methods such as Direct Multiple Shooting, Direct Single Shooting, and state-of-the-art solvers.

Keywords: Optimization algorithms, Networked control systems, Linear systems
Abstract: This study focuses on the simultaneous minimiza- tion of the quadratic cost and the number of actuations in the control of discrete-time linear systems. The quadratic cost measures conventional transient performance, while penalizing the actuation frequency promotes sparse control and improves energy efficiency. The main challenge in this optimization arises from the intricate interplay between the two optimization variables; control inputs and actuation timings. To avoid this complexity, we introduce a computationally feasible approximation and divide the problem into two stages. The proposed control scheme consists of a feedback controller, whose gains are determined offline, and an online actuation scheduler that decides whether to apply the input at each time step. We also introduce two extensions to further refine the optimization process. Through numerical examples, we evaluate the effectiveness of our method and its extensions. We show that these methods allow to balance the transient performance and the sparsity of the control input.

Keywords: Stochastic optimal control, Mean field games, Linear systems
Abstract: We show in this work how to develop a kernel approach to solve linear-quadratic mean field control problems. We use operator-valued kernels, which is consistent with the fact that we are dealing with an infinite dimensional control problem due to the mean-field term. But the stochastic aspect of the problem brings also a difficulty of a different nature. The kernel is defined over the time variable, and conversely to the deterministic case, information must be considered. Thus the kernel acts on random processes, even for ordinary stochastic control problems. This type of kernels has not appeared previously in the literature. Extensions, like partially observable systems or multiplicative noise, will be considered in the future.

Keywords: Iterative learning control, Stochastic systems, Delay systems
Abstract: In this paper, we consider constant step-size stochastic approximation with delayed updates. For the non-delayed case, it is well known that under appropriate conditions, the discrete-time iterates of stochastic approximation track the trajectory of a continuous-time ordinary differential equation (ODE). For the delayed case, we show in this paper that, under appropriate conditions, the discrete-time iterates track the trajectory of a delay-differential equation (DDE) rather than an ODE. Thus, delayed updates lead to a qualitative change in the behavior of constant step-size stochastic approximation. We present multiple examples to illustrate the qualitative affect of delay and show that increasing the delay is generally destabilizing but, for some systems, it can be stabilizing as well.

Keywords: Reinforcement learning, Stochastic optimal control
Abstract: The field of quickest change detection (QCD) concerns design and analysis of algorithms to estimate in real time the time at which an important event takes place, and identify properties of the post-change behavior. It is shown in this paper that approaches based on reinforcement learning (RL) can be adapted based on any "surrogate information state" that is adapted to the observations. Hence we are left to choose both the surrogate information state process and the algorithm. For the former, it is argued that there are many choices available, based on a rich theory of asymptotic statistics for QCD. Two approaches to RL design are considered:

(i) Stochastic gradient descent based on an actor-critic formulation. Theory is largely complete for this approach: the algorithm is unbiased, and will converge to a local minimum. However, it is shown that variance of stochastic gradients can be very large, necessitating the need for commensurately long run times.

(ii) Q-learning algorithms based on a version of the projected Bellman equation. It is shown that the algorithm is stable, in the sense of bounded sample paths, and that a solution to the projected Bellman equation exists under mild conditions.

Numerical experiments illustrate these findings, and provide a roadmap for algorithm design in more general settings.

Keywords: Stochastic optimal control, Mean field games, Machine learning
Abstract: We study a class of stochastic exchangeable teams with a finite number of decision makers (DMs) and their mean-field limits with infinite DMs. We study teams in the finite population regime under the centralized information structure (IS) and teams in the infinite population setting under the decentralized mean-field information sharing. The paper makes the following main contributions: i) For finite population exchangeable teams, we show existence of an optimal policy that is exchangeable (permutation invariant) and Markovian, obtained via value iterations for an equivalent measure-valued controlled Markov decision problem (MDP); ii) We show that a sequence of exchangeable optimal policies for a finite population setting converges to a symmetric (identical), independent, and decentralized randomized policy for the infinite population problem, guaranteeing existence of an optimal policy that is symmetric, independent, and decentralized optimal policy for the infinite population problem. This certifies optimality of the limiting measure-valued MDP for the representative DM. We also provide a finite approximation model for this MDP; iii) We show that symmetric, independent, decentralized optimal policies are approximately optimal for the corresponding finite-population team with many DMs under the centralized IS; iv) We show that a finite model approximation is near optimal for the mean-field MDP of the representative DM.

Keywords: Stochastic optimal control, Stochastic systems, Markov processes
Abstract: We revisit the work of Mitter and Newton on an information-theoretic interpretation of Bayes' formula through the Gibbs variational principle. This formulation allowed them to pose nonlinear estimation for diffusion processes as a problem in stochastic optimal control, so that the posterior density of the signal given the observation path could be sampled by adding a drift to the signal process. We show that this control-theoretic approach to sampling provides a common mechanism underlying several distinct problems involving diffusion processes, specifically importance sampling using Feynman--Kac averages, time reversal, and Schrödinger bridges.

Keywords: Machine learning, Stochastic optimal control, Robotics
Abstract: Supervised machine learning is powerful. In recent years, it has enabled massive breakthroughs in computer vision and natural language processing. But leveraging these advances for optimal control has proved difficult. Data is a key limiting factor. Without access to the optimal policy, value function, or demonstrations, how can we fit a policy? In this paper, we show how to automatically generate supervised learning data for a class of continuous-time nonlinear stochastic optimal control problems. In particular, applying the Feynman-Kac theorem to a linear reparameterization of the Hamilton-Jacobi-Bellman PDE allows us to sample the value function by simulating a stochastic process. Hardware accelerators like GPUs could rapidly generate a large amount of this training data. With this data in hand, stochastic optimal control becomes supervised learning.

Keywords: Data driven control, LMIs, Power systems
Abstract: In this paper, an online Q-learning algorithm is proposed to address the infinite-horizon guaranteed cost control problem for linear time delay systems with completely unknown dynamics. The developed approach leverages a Lyapunov-Krasovskii functional as the state value function and integrates guaranteed cost control principles. Specifically, based on Bessel-Legendre integral inequality, a Q-function tailored for handling guaranteed cost control in time delay systems is formulated. Furthermore, an integral reinforcement learning method based on an actor/critic approximator framework is used to dynamically estimate the Q-function parameters. Finally, the proposed approach is successfully applied to an interconnected power system.

Keywords: Data driven control, Nonlinear systems identification, Nonlinear output feedback
Abstract: Nonlinear data-driven control strategies, particularly Model Reference Gaussian Process Regression (MR-GPR), have been effective in designing controllers directly from system input/output data, bypassing the need for explicit system modeling. This approach is advantageous for complex nonlinear systems where traditional modeling methods may be inadequate. MR-GPR employs Gaussian Process Regression to provide a non-parametric control method, enhancing adaptability and performance. However, real-world applications present challenges due to variability in system parameters, such as ensuring robustness and consistent performance. To address these challenges, this paper proposes a robust MR-GPR controller incorporating domain randomization to improve adaptability to varying operational conditions. This extension aims to maintain stable performance across diverse settings, mitigating the impact of parameter changes on control efficacy.

Keywords: Data driven control, Nonlinear systems identification, Nonlinear systems
Abstract: Data-driven control methods are gaining importance in control engineering, particularly for nonlinear systems where traditional models fall short. Many approaches rely on predefined libraries of functions, such as polynomials, to approximate system dynamics. These methods assume the selected functions can effectively represent the underlying system, but this assumption may not always hold, potentially impacting the robustness of control designs. The Model Reference Gaussian Process Regression (MR-GPR) controller was introduced to address these issues by using Gaussian Process Regression (GPR) with flexible kernels that adapt to observed data, eliminating the need for predefined function libraries. In this paper, we extend the MR-GPR controller to handle (n)-step controllable systems, allowing for more complex multi-step control tasks. We propose a state feedback control strategy based on real-time system data. Theoretical analysis confirms the stability of the closed-loop system under the MR-GPR controller, and numerical simulations validate its effectiveness.

Keywords: Data driven control, Nonlinear systems identification, Predictive control for nonlinear systems
Abstract: Inverse modeling is the process uncovering the relationships from the system observations to its inputs. It is essential in various fields such as control, robotics, and signal processing. We propose an underline{i}nverse underline{m}odeling method using amortized variational inference based on conditional normalizing underline{flow}s (IMFlow). IMFlow is data-driven and can therefore be applied to black-box environments with limited observability and unknown complexity. Besides, the probabilistic modeling characteristics of conditional normalizing flows allow IMFlow to cope with unknown system uncertainties. We deploy IMFlow as a probabilistic model predictive controller, which estimates the control inputs as stochastic processes based on reference signals and system responses. In addition, we also adjust IMFlow to an online model-free reinforcement learning setting. We demonstrate our proposed method achieves the same accuracy in comparison to the standard model predictive control method using white-box models.

Keywords: Data driven control, Optimal control, Large-scale systems
Abstract: We develop a data-driven approach to Pareto optimal control of large-scale systems, where decision-makers know only their local dynamics. Using reinforcement learning, we design a control strategy that optimally balances multiple objectives. The proposed method guarantees the stability of all the system dynamics and scales well with the total dimension of the system. Experimental results demonstrate the effectiveness of our approach in managing multi-area power systems.

Keywords: Data driven control, Optimal control, Linear systems
Abstract: This paper presents a data-based approach to linear quadratic optimal control design. The system manipulated variable is assumed to have a zero mean uncertainty with a certain covariance, and the true system trajectory is measurable subject to measurement noise. The separation principle in the data-based context is investigated, which reveals that the original problem can be decomposed into an optimal quadratic control problem and an interval-wise trajectory estimation problem that can be designed separately. Algorithms are developed for both the finite and infinite horizon control problem, with the latter proven to be able to asymptotically stabilize the expected value of all trajectories in the controlled behavior. An illustrative example is provided to demonstrate the effectiveness of the proposed approach.

Keywords: Learning, Adaptive control, Optimization algorithms
Abstract: Continuous-time adaptive controllers for systems with a matched uncertainty often comprise an online parameter estimator and a corresponding parameterized controller to cancel the uncertainty. However, such methods are often impossible to implement directly, as they depend on an unobserved estimation error. We consider the equivalent discrete-time setting with a causal information structure, and propose a novel, online proximal point method-based adaptive controller, that under a sufficient excitation (SE) condition is asymptotically stable and achieves finite regret, scaling only with the time required to fulfill the SE. We show the same also for the widely-used recursive least squares with exponential forgetting controller under a stronger persistence of excitation condition.