Keywords: Healthcare and medical systems, Emerging control applications
Abstract: Despite the massive costs and widespread harms of substance use, most individuals with substance use disorders (SUDs) are untreated. Digital therapeutics platforms are an emerging low-cost, low-barrier means to expand treatment. There is a growing body of research showing how treatment providers can identify patients who need SUD support, but very little work addressing how providers should choose the most appropriate treatment for each patient. Because SUD treatment involves months or years of voluntary compliance from the patient, treatment adherence is a critical consideration for the provider. In this paper, we propose algorithms that a provider can use to match the burden-level of daily proposed treatments to the time-varying engagement state of the patient to promote adherence. We propose structured models for a patient’s engagement over time and their treatment adherence decisions. Using these models we pose a stochastic control formulation of the treatment-provider’s burden selection problem. We propose an adaptive control approach that estimates unknown patient parameters as new data are observed. We show that these estimates are consistent and propose algorithms using these estimates to make appropriate treatment recommendations.

Keywords: Healthcare and medical systems, Predictive control for nonlinear systems, Sampled-data control
Abstract: Extensive clinical evidence supports the beneficial role of physical activity in delaying the progression of type-2 diabetes. However, current clinical recommendations remain largely qualitative, failing to account for the patient’s evolving condition and lacking a quantitative framework for real-time, personalized prescriptions. In this work, we propose an original model-based approach to the control of diabetes progression via physical activity, based on a control-theoretical formulation of the benefits of the exercise, leveraging a sampled-data observer-based model predictive control framework. We design the control law on a compact, widespread model of diabetes evolution, whilst the effectiveness of the proposed control strategy is tested in silico by closing the loop on a population of virtual subjects simulated by a different, higher-dimensional model of diabetes regulation under exercise. The validation procedure also accounts for the effect of additional non-idealities, including quantized measurements and disturbances, and clearly shows the efficacy of a suitably designed physical activity to prevent diabetes progression. To the best of our knowledge, this work proposes for the first time an output-feedback approach leveraging physical activity for long-term glucose regulation.

Keywords: Healthcare and medical systems, Modeling, Game theory
Abstract: In many real-world systems, agents interact in complex ways that challenge traditional control and game-theoretic approaches. Building on Stackelberg evolutionary game theory, which models the interplay between a rational leader and an adaptive follower, we introduce a three-agent framework with modulated leadership, where a passive modulator dynamically aligns with either the leader or the follower, thereby influencing the leader’s strategic choices. The main contribution of this work is the application of the proposed framework to a critical biomedical challenge: tumor growth control. Particularly, we develop a novel tumor-immune model capturing evolution, interactions, and environmental factors. We then frame tumor control as a strategic problem involving the physician (leader), the evolving tumor (follower), and the immune system, which can alternatively support treatment and promote tumor survival. Leveraging this framework, we design combination therapies that dynamically adjust chemotherapy and anti-PD1 immunotherapy. Numerical simulations demonstrate the potential of the proposed approach in oncology and, more broadly, in other fields involving dynamic, multi-agent interactions.

Keywords: Biomedical, Compartmental and Positive systems, Control applications
Abstract: A novel interval predictor-based controller is presented to solve the Bispectral Index (BIS) regulation problem in anesthetic procedures where propofol and remifentanil are employed together. The resulting control scheme does not require observing or filtering the drug concentrations on the patient's effect--site from the BIS signal. Knowing a feasible interval of initial conditions is enough to ensure the convergence of the regulation errors to some compact sets around the origin despite perturbations in the pharmacodynamics and pharmacokinetics models. Some simulations, considering different in silico patients, are included to illustrate the implementability of the methods.

Keywords: Biomedical, Identification, Optimization
Abstract: For personalized anesthesia, it is crucial to have accurate pharmacokinetic and pharmacodynamic (PK/PD) models of anesthetic drugs. Typically, total intravenous anesthesia (TIVA) relies on the combined administration of propofol and remifentanil, which affect the bispectral index (BIS). This paper proposes a model reduction method that exploits the use of a fixed administration ratio between propofol and remifentanil, which is consistent with the clinical practice. Using a fourth-order ARX model, we estimate the effect-site concentration and fit the BIS response using a Hill function, thus implicitly considering the coadministration as if it was a single drug. The identification process is performed using a branch and bound approach, leveraging BIS data obtained during the induction of anesthesia. The identified model is then validated during the maintenance phase of anesthesia against models that are usually employed in clinical practice, where propofol and remifentanil are separately modeled. Results obtained on real BIS data show the effectiveness of the proposed approach.

Keywords: Biomedical, Learning, Predictive control for nonlinear systems
Abstract: We present a nonlinear data-driven Model Predictive Control (MPC) algorithm for deep brain stimulation (DBS) for the treatment of Parkinson's disease (PD). Although DBS is typically implemented in open-loop, closed-loop DBS (CLDBS) uses the amplitude of neural oscillations in specific frequency bands (e.g. beta 13-30 Hz) as a feedback signal, resulting in improved treatment outcomes with reduced side effects and slower rates of patient habituation to stimulation. To date, CLDBS has only been implemented in vivo with simple algorithms, such as proportional, proportional-integral, and thresholded switching control. Our approach employs a multi-step predictor based on differences of input-convex neural networks to model the future evolution of beta oscillations. The use of a multi-step predictor enhances prediction accuracy over the optimization horizon and simplifies online computation. In tests using a simulated model of beta-band activity response and data from PD patients, we achieve reductions of more than 20% in both tracking error and control activity in comparison with existing CLDBS algorithms. The proposed control strategy provides a generalizable data-driven technique that can be applied to the treatment of PD and other diseases targeted by CLDBS, as well as to other neuromodulation techniques.

Keywords: Biological systems, Biomedical, Emerging control applications
Abstract: Can border controls and travel restrictions suppress epidemics? This is an important and recurring question for public health policymaking and pandemic preparedness. Although there is a consensus that infections imported from external locations are critical to sustaining local epidemic growth and driving the endemic equilibria of the disease, there remains ongoing debate on the effectiveness of interventions aimed at curbing importations. Most studies contributing to this debate rely on complex metapopulation models that preclude generalisable insights and formal control principles are rarely used. Here we demonstrate how classical control theory applied to an analytic but flexible and widely used transmission model provides compelling evidence that border and travel restrictions fail to stabilise spread. These restrictions always enter as precompensators and hence fail to shift the epidemic poles. Consequently, regions cannot respond to emergent outbreaks in isolation. We find that coordinating interventions across both the local and external regions converts precompensation into feedback control and derive new criteria for overall stability. We further develop formulae specifying how border controls shape performance, such as disease equilibria and total infections. During growing epidemic phases, cooperative control is crucial for stability. Once stability is assured, border control can be sufficient to meet performance targets.

Keywords: Biomedical, Switched systems, Predictive control for linear systems
Abstract: Management of insulin sensitivity variability poses a significant challenge in achieving optimal blood glucose control for Type 1 Diabetes Mellitus (T1DM) patients using Artificial Pancreas (AP) systems. Traditional control strategies, particularly those employing Linear Time-Invariant (LTI) models in Model Predictive Control (MPC), although effective, do not adequately address the pronounced circadian fluctuations in insulin sensitivity. This study proposes an innovative switching MPC strategy leveraging multiple linear models, each corresponding to distinct daily periods (i.e., morning, afternoon, and evening) to dynamically adapt insulin dosing. The flexibility of the switching algorithm allows transitions between models within predefined, physiologically appropriate time windows. Performance evaluation, conducted via simulations using the UVA/Padova T1DM simulator, demonstrates that the proposed switching control strategy substantially reduces hypoglycemic episodes and stabilizes glucose variability compared to traditional single-model MPC approaches. This adaptive methodology shows promise in enhancing the safety and efficacy of glucose management, paving the way for improved quality of life and reduced diabetes-related complications.

Keywords: Data driven control, Statistical learning, Randomized algorithms
Abstract: We consider the problem of repetitive scenario design where one has to solve repeatedly a scenario design problem and can adjust the sample size (number of scenarios) to obtain a desired level of risk (constraint violation probability). We propose an approach to learn on the fly the optimal sample size based on observed data consisting in previous scenario solutions and their risk level. Our approach consists in learning a function that represents the pdf (probability density function) of the risk as a function of the sample size. Once this function is known, retrieving the optimal sample size is straightforward. We prove the soundness and convergence of our approach to obtain the optimal sample size for the class of fixed-complexity scenario problems, which generalizes fully-supported convex scenario programs that have been studied extensively in the scenario optimization literature. We also demonstrate the practical efficiency of our approach on a series of challenging repetitive scenario design problems, including non-fixed-complexity problems, nonconvex constraints and time-varying distributions.

Keywords: Model/Controller reduction, Optimization, Uncertain systems
Abstract: Zonotopes are a widely used set representation in set-based computations due to their compact representation size and their closure under many relevant set operations. However, certain set operations, such as the Minkowski sum, increase the zonotope order, which in turn increases the computational cost of further computations. To address this issue, various order reduction techniques have been proposed, most of which focus on overapproximating the original zonotope. While overapproximations are crucial for safety verification, some applications - such as reachset-conformant identification and backward reachability analysis - require underapproximations (also referred to as inner-approximations). Besides providing a comprehensive survey of existing underapproximative order reduction methods, we propose four novel reduction methods in this paper. We analyze the computational cost of all methods and evaluate the tightness of the resulting underapproximations through numerical experiments on more than 2000 randomly generated zonotopes. The results demonstrate that our proposed methods achieve a favorable balance between computational efficiency and approximation accuracy, making them well-suited for applications in control, estimation, and system identification.

Keywords: Lyapunov methods, Neural networks, Stability of nonlinear systems
Abstract: Analysis of nonlinear autonomous systems typically involves estimating domains of attraction, which has been a topic of extensive research interest for decades. Despite this, accurately estimating domains of attraction for nonlinear systems remains a challenging task, where existing methods are conservative or limited to low-dimensional systems. The estimation becomes even more challenging when accounting for state constraints. In this work, we propose a framework to accurately estimate safe (state-constrained) domains of attraction for discrete-time autonomous nonlinear systems. In establishing this framework, we first derive a new Zubov equation, whose solution corresponds to the exact safe domain of attraction. The solution to the aforementioned Zubov equation is shown to be unique and continuous over the whole state space. We then present a physics-informed approach to approximating the solution of the Zubov equation using neural networks. To obtain certifiable estimates of the domain of attraction from these neural network approximate solutions, we propose a verification framework that can be implemented using standard verification tools (e.g., alpha,!beta-CROWN and dReal). To illustrate its effectiveness, we demonstrate our approach through numerical examples concerning nonlinear systems with state constraints.

Keywords: Data driven control, Statistical learning, Uncertain systems
Abstract: The scenario approach is an established data-driven design framework equipped with a powerful theory that links design complexity to generalization properties. A key feature of this method is that training data are used both for design and for certifying the design’s reliability, without resorting to additional data points. This paper takes a step further: the goal is not only to certify the properties considered at the design stage, but also to guarantee new properties of interest in post-design usage. To this end, we introduce a two-level framework of appropriateness: baseline appropriateness, which guides the design process, and post-design appropriateness, which serves as a criterion for a posteriori evaluation. The paper fully develops a post-design verification framework that works without resorting to additional test data and illustrates it on an H2 problem.

Keywords: Data driven control, Robust control, Formal Verification/Synthesis
Abstract: This paper offers a direct data-driven approach for learning robust control barrier certificates (R-CBCs) and robust safety controllers (R-SCs) for discrete-time input-affine polynomial systems with unknown dynamics under unknown-but-bounded disturbances. The proposed method relies on data from input-state observations collected over a finite-time horizon while satisfying a specific rank condition. Our data-driven scheme enables the synthesis of R-CBCs and R-SCs directly from observed data, bypassing the need for explicit modeling of the system's dynamics and thus ensuring robust system safety against disturbances within an infinite time horizon. The proposed approach is formulated as a sum-of-squares (SOS) optimization problem, providing a structured design framework. Two case studies showcase our method's capability to provide robust safety guarantees for unknown input-affine polynomial systems under bounded disturbances, demonstrating its practical effectiveness.

Keywords: Lyapunov methods, Data driven control, Neural networks
Abstract: We consider the problem of verifying safety for continuous-time dynamical systems. Developing upon recent advancements in data-driven verification, we use only a finite number of sampled trajectories to learn a barrier certificate, namely a function which verifies safety. We train a safety-informed neural network to act as this certificate, with an appropriately designed loss function to encompass the safety conditions. In addition, we provide probabilistic generalisation guarantees from discrete samples of continuous trajectories, to unseen continuous ones. Numerical investigations demonstrate the efficacy of our approach and contrast it with related results in the literature.

Keywords: Reduced order modeling, Data driven control, Formal Verification/Synthesis
Abstract: This paper introduces a data-driven scheme to construct reduced-order models (ROMs) of linear dynamical systems with unknown mathematical models. Our methodology leverages data and establishes similarity relations between output trajectories of unknown systems and their data-driven ROMs via the notion of simulation functions (SFs), capable of formally quantifying their closeness. To achieve this, under a rank condition readily fulfillable using data, we collect only two input-state trajectories from unknown systems to construct both ROMs and SFs, while offering correctness guarantees. We demonstrate that the proposed ROMs derived from data can be leveraged for controller synthesis endeavors while effectively ensuring high-level logic properties over unknown dynamical models. We showcase our data-driven findings across a range of benchmark scenarios involving various unknown physical systems, demonstrating the enforcement of diverse complex properties.

Keywords: Reduced order modeling, Modeling, Model/Controller reduction
Abstract: While approximating dynamical systems, one may be interested in preserving physical insights of specific states of the underlying system, which can be achieved by fixing the output map of the approximated model. In this paper we derive a class of reduced-order models that achieve moment matching while enforcing an output map chosen by the designer. We show that such models can be constructed even without the knowledge of the underlying system by using data generated from some experiments. In doing so, we highlight how the proposed method can be utilized to enforce input-output behavior, such as passivity, negative imaginary, and/or finite gain, in the reduced-order model, thus providing provable guarantees on various user-specified requirements. We demonstrate this aspect through several examples.

Keywords: Agents-based systems, Lyapunov methods, Emerging control applications
Abstract: Many decision-making scenarios in engineering, sociology, and epidemiology, among other fields, can be effectively modeled by a large population of learning agents, captured by an evolutionary dynamics model (EDM), which can possibly act non-cooperatively and interact with an exogenous dynamical system (ES). In these systems, the agents’ collective behavior can drive the ES into undesirable states and significantly compromise its stability and safety. In this paper, we propose a method using control barrier functions to design payoffs that can guide the population’s strategy selection while ensuring safe and stable behavior of the ES. We first address the need to avoid undesirable population states in the EDM. We then extend our approach to coupled EDM-ES systems, where the designed payoffs can prevent undesirable states and guarantee convergence to a target state. Finally, we establish general conditions under which the designed payoffs remain effective across different classes of EDMs, ensuring broad applicability. Numerical simulations validate our approach in both standalone EDMs and coupled EDM-ES systems that satisfy the proposed conditions.

Keywords: Reinforcement learning, Learning
Abstract: Constrained optimization is popularly seen in reinforcement learning (RL) for addressing complex control tasks. From the perspective of dynamic system, iteratively solving a constrained optimization problem can be framed as the temporal evolution of a feedback control system. Classical constrained optimization methods, such as penalty and Lagrangian approaches, inherently use proportional and integral feedback controllers. In this paper, we propose a more generic equivalence framework to build the connection between constrained optimization and feedback control system, for the purpose of developing more effective constrained RL algorithms. Firstly, we define that each step of the system evolution determines the Lagrange multiplier by solving a multiplier feedback optimal control problem (MFOCP). In this problem, the control input is multiplier, the state is policy parameters, the dynamics is described by policy gradient descent, and the objective is to minimize constraint violations. Then, we introduce a multiplier guided policy learning (MGPL) module to perform policy parameters updating. And we prove that the resulting optimal policy, achieved through alternating MFOCP and MGPL, aligns with the solution of the primal constrained RL problem, thereby establishing our equivalence framework. Furthermore, we point out that the existing PID Lagrangian is merely one special case within our framework that utilizes a PID controller. We also accommodate the integration of other various feedback controllers, thereby facilitating the development of new algorithms. As a representative, we employ model predictive control (MPC) as the feedback controller and consequently propose a new algorithm called predictive Lagrangian optimization (PLO). Numerical experiments demonstrate its superiority over the PID Lagrangian method, achieving a larger feasible region up to 7.2% and a comparable average reward.

Keywords: Identification for control, Identification
Abstract: Designing identification experiments for nonlinear systems with only positive excitation is challenging. Large excitations improve the signal-to-noise ratio but risk deviating the system from the desired operating point due to their increased mean. To address this, we propose an algorithm that minimizes the integral of the perturbation, balancing the signal-to-noise ratio and the deviation from the target. We validate this approach with a nonlinear diffusion system identification simulation, which is illustrative for fusion energy applications.

Keywords: Formal Verification/Synthesis, Data driven control, Uncertain systems
Abstract: We propose a data-driven approach to construct barrier certificates for unknown monotone systems using a finite number of system trajectories, while ensuring formal correctness guarantees. Given multiple system trajectories, we construct a family of monotone basis functions that remain non-increasing along all trajectories. Using this class of basis functions, we develop a sampling-based optimization approach to construct barrier certificates, establishing system safety with formal guarantees without requiring additional simulated data or assuming Lipschitz continuity of the system.

Furthermore, for the class of polynomial barrier functions, we show that the sampling-based approach can be efficiently reformulated as a linear program with significantly fewer constraints. Finally, we demonstrate the effectiveness of our approach through numerical examples.

Keywords: Reinforcement learning, Data driven control, Robotics
Abstract: Safe offline reinforcement learning aims to learn policies that maximize cumulative rewards while adhering to safety constraints, using only offline data for training. A key challenge is balancing safety and performance, particularly when the policy encounters out-of-distribution (OOD) states and actions, which can lead to safety violations or overly conservative behavior during deployment. To address these challenges, we introduce Feasibility Informed Advantage Weighted Actor-Critic (FAWAC), a method that prioritizes persistent safety in constrained Markov decision processes (CMDPs). FAWAC formulates policy optimization with feasibility conditions derived specifically for offline datasets, enabling safe policy updates in non-parametric policy space, followed by projection into parametric space for constrained actor training. By incorporating a cost-advantage term into Advantage Weighted Regression (AWR), FAWAC ensures that the safety constraints are respected while maximizing performance. Additionally, we propose a strategy to address a more challenging class of problems that involves tempting datasets where trajectories are predominantly high-rewarded but unsafe. Empirical evaluations on standard benchmarks demonstrate that FAWAC achieves strong results, effectively balancing safety and performance in learning policies from the static datasets.

Keywords: Observers for nonlinear systems, Distributed parameter systems, Traffic control
Abstract: We consider the problem of traffic density estimation with sparse measurements from stationary roadside sensors. Our approach uses Fourier neural operators to learn macroscopic traffic flow dynamics from high-fidelity data. During inference, the operator functions as an open-loop predictor of traffic evolution. To close the loop, we couple the open-loop operator with a correction operator that combines the predicted density with sparse measurements from the sensors. Simulations with the SUMO software indicate that, compared to open-loop observers, the proposed closed-loop observer exhibits classical closed-loop properties such as robustness to noise and ultimate boundedness of the error. This shows the advantages of combining learned physics with real-time corrections, and opens avenues for accurate, efficient, and interpretable data-driven observers.

Keywords: Reinforcement learning, Optimal control
Abstract: This paper proposes tackling safety-critical stochastic Reinforcement Learning (RL) tasks with a sample-based, model-based approach. At the core of the method lies a Model Predictive Control (MPC) scheme that acts as function approximation, providing a model-based predictive control policy. To ensure safety, a probabilistic Control Barrier Function (CBF) is integrated into the MPC controller. To approximate the effects of stochasticies in the optimal control formulation and to fulfil the probabilistic CBF condition, a sample-based approach with guarantees is employed. Furthermore, to counterbalance the additional computational burden due to sampling, a learnable terminal cost formulation is included in the MPC objective. An RL algorithm is deployed to learn both the terminal cost and the CBF constraint. Results from a numerical experiment on a constrained LTI problem corroborate the effectiveness of the proposed methodology in reducing computation time while preserving control performance and safety.

Keywords: Data driven control, Neural networks, Autonomous systems
Abstract: Barrier certificates are real-valued functions used to formally verify the safety of dynamical systems. For systems with unknown dynamics, data-driven barrier certificates have been developed to guarantee safety using only a finite set of data. However, most existing methods rely on knowledge of Lipschitz bound of the system and require a fine discretization of the state set, leading to high sample complexity. In this paper, we propose a novel data-driven framework for learning barrier certificates for unknown monotone systems. Our approach is based on a suitable embedding of barrier certificates into a higher-dimensional space. By leveraging interval analysis, this embedding enables us to establish data-driven safety certificates for monotone systems. Unlike existing methods, our framework is independent of Lipschitz continuity and quantization parameters of the state set, allowing for arbitrary state-space discretization and thereby alleviating extensive sampling requirements. To efficiently construct barrier certificates, we introduce suitable neural network architectures and train them using an appropriately designed loss function. We illustrate our approach through two case studies, demonstrating that our method successfully finds separable embedded barrier certificates while substantially reducing sample complexity compared to conventional neural barrier certificate methods, all while maintaining formal correctness guarantees.

Keywords: Statistical learning, Reinforcement learning, Game theory
Abstract: In this paper, we expand the Bayesian persuasion framework to account for unobserved confounding variables in sender-receiver interactions. While traditional models typically assume that belief updates follow Bayesian principles, real-world scenarios often involve hidden variables that impact the receiver’s belief formation and decision-making. We conceptualize this as a sequential decision-making problem, where the sender and receiver interact over multiple rounds. In each round, the sender communicates with the receiver, who also interacts with the environment. Crucially, the receiver’s belief update is affected by an unobserved confounding variable. By reformulating this scenario as a Partially Observable Markov Decision Process (POMDP), we capture the sender’s incomplete information regarding both the dynamics of the receiver’s beliefs and the unobserved confounder. We prove that finding an optimal observation-based policy in this POMDP is equivalent to solving for an optimal signaling strategy in the original persuasion framework. Furthermore, we demonstrate how this reformulation facilitates the application of proximal learning for off-policy evaluation (OPE) in the persuasion process. This advancement enables the sender to evaluate alternative signaling strategies using only observational data from a behavioral policy, thus eliminating the necessity for costly new experiments.

Keywords: Transportation networks, Game theory, Learning
Abstract: Traditional approaches to modeling and predicting traffic behavior often rely on Wardrop Equilibrium (WE), assuming non-atomic traffic demand and neglecting correlations in individual decisions. However, the growing role of real-time human feedback and adaptive recommendation systems calls for more expressive equilibrium concepts that better capture user preferences and the stochastic nature of routing behavior. In this paper, we introduce a preference-centric route recommendation framework grounded in the concept of Borda Coarse Correlated Equilibrium (BCCE), wherein users have no incentive to deviate from recommended strategies when evaluated by Borda scores—pairwise comparisons encoding user preferences. We develop an adaptive algorithm that learns from dueling feedback and show that it achieves mathcal{O}(T^{frac{2}{3}}) regret, implying convergence to the BCCE under mild assumptions. We conduct empirical evaluations using a case study to illustrate and justify our theoretical analysis. The results demonstrate the efficacy and practical relevance of our approach.

Keywords: Game theory
Abstract: This paper analyzes the problem of communication between boundedly rational senders with heterogeneous objectives and partial information and a full information fully rational receiver. Particularly, we explore a setting where there are K different types of senders with varying levels of available information, degree of strategic behavior, and cognitive hierarchy: i) a non-strategic agent with an honest response (level-0), ii) a k-th level kin [1:K-1] strategic agent that believes the population is Poisson distributed over the lower cognitive types. We model each of these scenarios as a strategic communication of a 2-dimensional source (possibly correlated source and bias components) with quadratic distortion measures. The numerical results we obtained via the proposed method validate our theoretical observations.

Keywords: Game theory, Stochastic systems, Human-in-the-loop control
Abstract: A large share of online content is consumed via social media and other digital platforms. Given the overwhelming amount of content, platforms deploy recommendation systems that operate under uncertainty about user preferences. We introduce a novel information disclosure model of user–platform interaction. The user privately knows his preferences and chooses whether to disclose them to the platform. The platform observes the available content and selects one of them to recommend, aiming to maximize expected engagement. A conflict arises because, when the platform delivers polarizing content, it often increases engagement while reducing user utility. We construct a symmetric Perfect Bayesian Equilibrium characterized by simple threshold functions, and identify conditions under which platforms obtain higher engagement when content creators produce more extreme content. This result helps explain the ongoing escalation of extreme content on social media platforms.

Keywords: Game theory
Abstract: In a two-strategy decision-making problem, a bi-threshold individual adopts a strategy only if the prevalence of adopters lies between her two thresholds. This study investigates the asymptotic behavior of a heterogeneous bi-threshold population, where the thresholds vary among the individuals. First, we show through an example that the proportion of adopters in finite populations may fluctuate. We then use available results in the stochastic approximation theory and approximate the finite discrete dynamics by corresponding continuous-time dynamics. Our analysis shows that the continuous-time dynamics always equilibrate. This implies that the amplitudes of the fluctuations in the proportion of adopters in a finite bi-threshold population almost surely diminish when the population size grows to infinity. The result underlines the advantage of analyzing simpler continuous-time dynamics to get insights into the long-term behavior of finite discrete bi-threshold populations.

Keywords: Game theory, Reinforcement learning, Markov processes
Abstract: Risk-aversion and bounded rationality are two key characteristics of human decision-making. Risk-averse quantal-response equilibrium (RQE) is a solution concept that incorporates these features, providing a more realistic depiction of human decision making in various strategic environments compared to a Nash equilibrium. Furthermore a class of RQE have recently been shown in Mazumdar et al., 2024 to be universally computationally tractable in all finite-horizon Markov games, allowing for the development of multi-agent reinforcement learning algorithms with convergence guarantees. In this paper, we expand upon the study of RQE and analyze their computation in both two-player normal form games and discounted infinite-horizon Markov games. For normal form games we adopt a monotonicity-based approach allowing us to generalize previous results. We first show uniqueness and Lipschitz continuity of RQE with respect to player's payoff matrices under monotonicity assumptions, and then provide conditions on the players' degrees of risk aversion and bounded rationality that ensure monotonicity. We then focus on discounted infinite-horizon Markov games. We define the risk-averse quantal-response Bellman operator and prove its contraction under further conditions on the players' risk-aversion, bounded rationality, and temporal discounting. This yields a Q-learning based algorithm with convergence guarantees for all infinite horizon general-sum games.

Keywords: Human-in-the-loop control, Stochastic optimal control
Abstract: We consider a human-automation team jointly solving binary classification tasks over multiple time stages. At each stage, the automation observes the data for a batch of classification tasks, classifies a subset of them and refers the others to the human. The human's performance depends on task load and fatigue, where fatigue is modeled as a controlled Markov process dependent on the past task loads. We formulate the automation's problem of deciding which tasks to refer as a Markov decision process and present a sampling-based approximate dynamic program that leverages task independence across time and the structure of the recently obtained single-stage optimal allocation policy. We then present a numerical study comparing our solution against a baseline policy that does not explicitly account for fatigue dynamics.

Keywords: Network analysis and control, Agents-based systems, Human-in-the-loop control
Abstract: This article considers Friedkin-Johnsen (FJ) model with an external influencer. The influencer is totally stubborn (i.e., her own opinion never changes), and can par- ticipate in the opinion evolution by adding links to a fixed number of agents in the FJ model. The problem is to investigate how the influencer selects agents to maximize her social power, which represents the influence of her initial opinion on other agents’ final opinions. This problem is shown to be equivalent to maximize the absorbing probability to the influencer for a Markov chain. The solution is analytically characterized for the cases of small and large stubbornness, with a single agent to be selected. Moreover, the social power of the influencer is proved to be monotone and submodular with the selected agent set, and a greedy algorithm is then proposed to generate an approximated solution. Random walks are used to speed up the algorithm for large network size. The effectiveness of the greedy algorithm is further shown by a numerical example.

Keywords: Cooperative control, Autonomous systems, Robotics
Abstract: In this paper, we propose an equilibrium-driven controller that enables a marsupial carrier-passenger robotic system to achieve smooth carrier-passenger separation and then to navigate the passenger robot toward a predetermined target point. To achieve smooth separation and navigation, we design a third-order potential gradient for the passenger's controller as a function of the carrier-passenger and carrier-target distances in the moving carrier robot's coordinate frame, which introduces three equilibrium points in the closed-loop error system, each corresponding to one of the following three scenarios: the passenger robot is (i) staying on the carrier robot, (ii) positioned between the carrier robot and the target, and (iii) reaching the target. Initially, the passenger robot is confined to the carrier robot, namely within the attraction region of the equilibrium point~(i). By appropriately decreasing the carrier-target distance, the equilibrium point (ii) vanishes, and the attraction and repulsion regions of equilibrium points (i) and (iii) change accordingly. This transition guides the passenger robot to leave the equilibrium point (i) and approach the equilibrium point (iii), which enables a smooth carrier-passenger separation and seamless navigation to the target. Furthermore, rigorous convergence analysis is also provided. Finally, 2D and 3D simulations are conducted to demonstrate the effectiveness and adaptability of the proposed controller facing obstacle in the surroundings.

Keywords: Cooperative control, Control over communications, Control of networks
Abstract: For formation control systems based on wireless communications, communication interaction and control performance are highly coupled, while traditional methods do not take this coupling into account. To fill this gap, we study the formation control problem of second-order multi-agent systems (MASs) with uncertain dynamics (affected by unknown but bounded process noises) and commonly-used time division multiple access (TDMA) communications. A formal analysis of system modeling, control-communication codesign, and formation system stability is provided. Firstly, we establish the communication model of MASs, which characterizes the effects of TDMA technology, agent dynamics, and physical layer of communications. Based on the communication model, we propose a codesign of the formation control and the communication power control to achieve bounded formation stability, where the boundedness property of the transmit power and the delay-based formation controller are analyzed. Finally, a numerical simulation is provided to validate the effectiveness of the proposed codesign method.

Keywords: Cooperative control, Intelligent systems, Agents-based systems
Abstract: The safe optimal coordination control problem is addressed for partially-unknown input-constrained air-ground systems under hybrid cyberattacks. Adversaries can launch denial-of-service attacks to prevent data transmission and channel manipulation attacks to tamper with interaction data. A unified distributed observer-based optimal control framework is first proposed for the heterogeneous vehicles. To achieve consensus under hybrid cyberattacks, the Zeno-free switching-type self-triggered observer is constructed based on only viable faulting neighborhood data. Then, optimal input-constrained control policies are learned via an on-policy actor-critic neural network-based learning algorithm. Sufficient conditions to guarantee the stability of the closed-loop system under the modeled hybrid cyberattacks are established. Numerical examples validate the effectiveness of the developed approach.

Keywords: Distributed control, Control of networks, Cooperative control
Abstract: In recent years, numerous extended rigidity theories have been developed by considering different types of local constraints, to adapt to formation control in different scenarios. Among them, the signed angle constraint has attracted considerable interest for its exceptional ability to stabilize coordinate-free formations with high degrees of freedom. Unfortunately, the combinatorial aspects of signed angle rigidity remain largely unexplored. In this paper, we study signed angle rigidity theory in the context of frameworks and prove its complete equivalence to bearing rigidity theory in the plane for the first time. Building on this equivalence, we propose a distributed formation controller based on signed angle measurements only, which achieves global stability as long as the formation framework is infinitesimally signed angle rigid. A simulation example demonstrates the effectiveness of our results.

Keywords: Estimation, Cooperative control, Autonomous systems
Abstract: This paper introduces a framework of localizing and tracking a non-cooperative target by using two pursuers which are assumed to have measurements of self-displacements and interior angles of the triangle formed by the target and the pursuers. First, based on the geometric relationship between the target and pursuers, an {measurement} model is derived from the pursuers' measured interior angles and self-displacements. Combined with the state-transition model, a linear time-varying Kalman filter is designed to estimate the relative position and velocity between the target and the pursuers. Then, a fully actuated control law is designed by using the parametric design approach to achieve the target tracking task, which dynamically adjusts pursuers' trajectories. Finally, simulations demonstrate the proposed framework’s localization and tracking performance for targets with dynamic maneuvering capability.

Keywords: Networked control systems, Optimization, Agents-based systems
Abstract: In this article, we present a constraint-driven control algorithm that minimizes the energy consumption of individual agents and yields an emergent V formation. As the formation emerges from the decentralized interaction between agents, our approach is robust to the spontaneous addition or removal of agents to the system. We start from an analytical model for the trailing upwash behind a fixed-wing uncrewed aerial vehicle (UAV), and we derive the optimal air speed for trailing UAVs to maximize their travel endurance. Next, we prove that simply flying at the optimal airspeed will never lead to emergent flocking behavior and propose a new decentralized “anseroid” behavior that yields emergent V formations. We encode these behaviors in a constraint-driven control algorithm that minimizes the locomotive power of each UAV. Finally, we prove that UAVs initialized in an approximate V or echelon formation will converge under our proposed control law, and we demonstrate that this emergence occurs in real time in simulation and in physical experiments with a fleet of Crazyflie quadrotors despite the noise and disturbances inherent in physical systems.

Keywords: Constrained control, Visual servo control, Nonholonomic systems
Abstract: This paper investigates the leader-follower formation tracking control problem for nonholonomic mobile robots under two critical constraints: the unavailability of robots' position measurements and the absence of inter-robot communication. The lack of positional information renders the formation tracking error immeasurable directly, while the communication restriction prevents the follower from acquiring the leader's real-time velocity. To address these challenges, we propose a novel vision-based feedback control scheme that relies exclusively on monocular visual perception with inherent field-of-view (FOV) constraints. The proposed method achieves the desired formation configuration by regulating visual feature points in the image plane, eliminating the need for both relative pose estimation and the leader's velocity information. Based on the deigned barrier functions, the proposed controller guarantees the leader always remains within the follower's visual coverage domain. Experimental results on multiple differential-drive mobile robots illustrate the effectiveness and practical applicability of the proposed control strategy.

Keywords: Agents-based systems, Autonomous systems, Lyapunov methods
Abstract: Distance-based formation control on minimally rigid graphs often encounters ambiguous shapes, where agents form incorrect shape formations due to multiple equilibrium points in the error dynamics. Recent studies on distance-based controllers, even those that introduce extra control variables, struggle to resolve this fundamental issue fully, typically requiring specific conditions or failing to manage reflections effectively. In this paper, we address both flip and reflection ambiguities in formation control by reexamining core geometric constraints and integrating them with traditional bearing-based formation control methods. We propose a novel controller that guarantees convergence to the correct formation without imposing any additional conditions. Numerical simulations demonstrate the controller's effectiveness in avoiding formation ambiguities.

Keywords: Hybrid systems, Optimization
Abstract: This paper introduces a notion of backward control barrier certificates to synthesize safety controllers for deterministic systems. Barrier certificates and control barrier certificates play a fundamental role in the automated design of controllers to ensure the safety of dynamical systems. The simultaneous search for control barrier certificates and their controllers typically face challenges in automation as they involve quantifier alternation between the states (for all states) and the inputs (there exists an input) as well as bilinearity between the unknown certificate and control input. In this paper, we show that one may simultaneously search for both a certificate and controller effectively for deterministic systems via standard sum-of-squares approaches without the need for quantifier alternation. Here, we treat the input as a disturbance and build an invariant set over the unsafe set of states rather the the initial set. This set is invariant in the backward direction rather than forward, and hence we dub these as backward control barrier certificates. Ensuring that the initial set is not in this invariant set guarantees the existence of a controller to ensure safety. We show how one may automate the search for these certificates, and discuss some strategies to implement their corresponding safety controllers. Finally, we demonstrate the efficacy of our approach with some case studies.

Keywords: Autonomous systems, Nonlinear output feedback, Constrained control
Abstract: We consider the problem of safely exploring a static and unknown environment while learning valid control barrier functions (CBFs) from sensor data. Existing works either assume known environments, target specific dynamics models, or use a-priori valid CBFs, and thus provide limited safety guarantees for general control-affine systems during exploration. We present a method for safely exploring by incrementally composing a global CBF from local CBFs. The challenge here is that local CBFs may not have well-defined end behavior outside their training domain, i.e. local CBFs may be positive (indicating safety) in regions where no training data is available. We show that well-defined end behavior can be obtained when local CBFs are parameterized by compactly-supported radial basis functions. For learning local CBFs, we collect sensor data, e.g. LiDAR capturing obstacles in the environment, and augment it with simulated data from a safe oracle controller. Our work complements recent efforts to learn CBFs from safe demonstrations, where learned safe sets are limited to their training domains, by demonstrating how to grow the safe set over time as more data becomes available. We evaluate our approach on two simulated systems, where our method successfully explores an unknown environment while maintaining safety throughout the entire execution.

Keywords: Autonomous robots, Stochastic systems, Formal Verification/Synthesis
Abstract: Ensuring safety for autonomous robots operating in dynamic environments can be challenging due to factors such as unmodeled dynamics, noisy sensor measurements, and partial observability. To account for these limitations, it is common to maintain a belief distribution over the true state. This belief could be a non-parametric, sample-based representation to capture uncertainty more flexibly. In this paper, we propose a novel form of Belief Control Barrier Functions (BCBFs) specifically designed to ensure safety in dynamic environments under stochastic dynamics and a sample-based belief about the environment state. Our approach incorporates provable concentration bounds on tail risk measures into BCBFs, effectively addressing possible multimodal and skewed belief distributions represented by samples. Moreover, the proposed method demonstrates robustness against distributional shifts up to a predefined bound. We validate the effectiveness and real-time performance (approximately 1kHz) of the proposed method through two simulated underwater robotic applications: object tracking and dynamic collision avoidance.

Keywords: Discrete event systems, Formal Verification/Synthesis, Hybrid systems
Abstract: We investigate the control synthesis problem for continuous-time control-affine systems under a class of multiple reach-avoid (MRA) tasks. Specifically, the MRA task requires the system to reach a series of target regions in a specified order while satisfying state constraints between each pair of target arrivals. This problem is more challenging than standard reach-avoid tasks, as it requires considering the feasibility of future reach-avoid tasks during the planning process. To solve this problem, we define a series of value functions by solving a cascade of time-varying reach-avoid problems characterized by Hamilton-Jacobi variational inequalities. We prove that the super-level set of the final value function computed is exactly the feasible set of the MRA task. Additionally, we demonstrate that the control law can be effectively synthesized by ensuring the non-negativeness of the value functions over time. The effectiveness of the proposed approach is illustrated through two case studies on robot planning problems.

Keywords: Lyapunov methods, Constrained control, Optimization algorithms
Abstract: In this paper, we demonstrate that controllers designed by artificial potential fields (APFs) can be derived from reciprocal control barrier function quadratic program (RCBF-QP) safety filters. By integrating APFs within the RCBF-QP framework, we explicitly establish the relationship between these two approaches. Specifically, we first introduce the concepts of tightened control Lyapunov functions (T-CLFs) and tightened reciprocal control barrier functions (T-RCBFs), each of which incorporates a flexible auxiliary function. We then utilize an attractive potential field as a T-CLF to guide the nominal controller design, and a repulsive potential field as a T-RCBF to formulate an RCBF-QP safety filter. With appropriately chosen auxiliary functions, we show that controllers designed by APFs and those derived by RCBF-QP safety filters are equivalent. Based on this insight, we further generalize the APF-based controllers (equivalently, RCBF-QP safety filter-based controllers) to more general scenarios without restricting the choice of auxiliary functions. Finally, we present a collision avoidance example to clearly illustrate the connection and equivalence between the two methods.

Keywords: Uncertain systems, Distributed control, Constrained control
Abstract: Ensuring safety in multi-agent systems is a significant challenge, particularly in settings where centralized coordination is impractical. In this work, we propose a novel risk-sensitive safety filter for discrete-time multi-agent systems with uncertain dynamics that leverages control barrier functions (CBFs) defined through value functions. Our approach relies on centralized risk-sensitive safety conditions based on exponential risk operators to ensure robustness against model uncertainties. We introduce a distributed formulation of the safety filter by deriving two alternative strategies: one based on worst-case anticipation and another on proximity to a known safe policy. By allowing agents to switch between strategies, feasibility can be ensured. Through detailed numerical evaluations, we demonstrate the efficacy of our approach in maintaining safety without being overly conservative.

Keywords: Constrained control, Cyber-Physical Security, Resilient Control Systems
Abstract: This paper presents a secure safety filter design for constrained nonlinear systems under sensor spoofing attacks. Existing approaches primarily focus on linear systems which limits their applications in real-world scenarios. In this work, we investigate the extension of our previous results to nonlinear systems. We introduce exact observability maps that provide an abstraction independent of any particular state estimation algorithm, and extend them to a secure version capable of handling sensor attacks. The generalization to nonlinear systems also applies to the relaxed observability case, with slightly relaxed guarantees. More importantly, we propose a secure safety filter design in both exact and relaxed cases, which incorporates secure state estimation and a control barrier function-enabled safety filter. The proposed approach provides theoretical guarantees for enforcing state constraints in the presence of sensor attacks. We numerically validate our analysis on a unicycle vehicle equipped with redundant yet partly compromised sensors.

Keywords: Constrained control, Lyapunov methods
Abstract: We consider the problem of meeting multiple state and input constraints simultaneously in nth-order integrator systems for any n >= 1. Our approach consists of a series of recursive filters which result in analytic high-order CBFs for each state constraint, where the recursion is specially designed to enable compatibility of constraints. We further provide a tuning algorithm for the filter parameters and show that it converges in finitely many steps. While we focus our discussions mainly on problems with a single control input, we discuss extensions to multiple inputs as well and validate in simulation on a linearized quadrotor.

Keywords: Variable-structure/sliding-mode control
Abstract: This article proposes a nonlinear solution for stabilizing uncertain linear systems subjected to state constraints and external disturbances. First, a nonlinear control strategy based on Integral Sliding Modes deals with the problem of bounded coupled disturbances. Then, a linear controller provides a solution for uncoupled disturbances and state constraints. The second stage of the controller applies a Composite Lyapunov Function to derive the gains needed to enforce asymptotic convergence to zero while fulfilling state constraints and delimiting the safe set where the system trajectories do not violate these restrictions. Compared to traditional quadratic Barrier functions, Composite Lyapunov functions offer a less conservative estimation of a safety set.

Keywords: Variable-structure/sliding-mode control
Abstract: The paper deals with a well-posedness analysis for discontinuous infinite-dimensional systems, which can be regularized by means of the conventional (finite-dimensional) Filippov method. The study is inspired by sliding mode control theory. Sliding mode controllers for infinite-dimensional systems with finite- and infinite-dimensional sliding surfaces are designed. Theoretical results are supported by an example of an output-based sliding mode control for heat system.

Keywords: Nonlinear output feedback, Variable-structure/sliding-mode control, Lyapunov methods
Abstract: The proposed Lipschitz-continuous arbitrary- relative-degree output-feedback control theoretically exactly compensates for Lipschitz-continuous non-vanishing perturbations. The finite-time stability of the closed-loop system is proved via Lyapunov methods, exploiting the weighted homogeneity features of the control and observer. Moreover, the observer and the controller can be independently designed, establishing a specific separation principle. The feasibility of the method is demonstrated by simulation.

Keywords: Variable-structure/sliding-mode control
Abstract: In this paper, we propose a global second-order dissipative homogeneous observer (DH-observer) that can estimate the states of an uncertain nonlinear system with unknown inputs in finite time. The system must be strongly observable with respect to unknown inputs (UIs), uniformly in the known input. To handle the UIs and non-Lipschitz nonlinearities, the proposed observer combines dissipative and homogeneous properties. The convergence of the DH-observer to the system states is proven through a Lyapunov function. Finally, to showcase the effectiveness of the proposed method, the paper includes an academic example.

Keywords: Nonlinear output feedback, Variable-structure/sliding-mode control, Stability of nonlinear systems
Abstract: Robust finite-time feedback controller, recently introduced by the authors for second-order systems, can be seen as a non-overshooting quasi-continuous sliding mode control. The paper proposes a regularization scheme to suppress inherent chattering due to discontinuity in the origin, in favor of practical applications. A detailed analysis with ISS and iISS proofs are provided along with supporting numerical results.

Keywords: Adaptive systems, Estimation, Identification
Abstract: This paper presents the design of an accelerated adaptive observer for simultaneously estimating states and constant parameters in a class of uncertain nonlinear systems subject to external disturbances. The proposed observer consists of a high-order sliding-mode (HOSM) observer for state estimation and a heavy-ball-based algorithm for identifying unknown constant parameters. Despite the presence of bounded external disturbances that do not satisfy a relative degree one condition, the observer locally estimates both state and parameter vectors in finite time. The closed-loop stability analysis is performed using a Lyapunov function approach, input-to-state stability properties, and a finite-time small-gain theorem. The effectiveness of the proposed estimation algorithm is demonstrated through simulation results.

Keywords: Lyapunov methods, Numerical algorithms, Stability of linear systems
Abstract: An implicit Euler discretization scheme of the control law from [1] is proposed, which preserves all the main properties of the continuous-time counterpart (hyperexponential rate of convergence, compensation of matched disturbances or their attenuation, robustness to measurement noises). Additionally, the sample-and-hold implementation of the control, along with its stability analysis, is provided.

Keywords: Variable-structure/sliding-mode control, Robust control, Nonlinear systems
Abstract: This paper develops a nonsingular predefined-time terminal sliding mode control scheme based on smooth sliding surfaces with artificial time-delays. The proposed approach can achieve predefined-time control for perturbed chains of integrators with matched and mismatched disturbances. Especially, for double integrators, an initial-condition-based sufficient condition is provided to preserve the prescribed-time stability in the presence of bounded matched disturbances.

Keywords: Reinforcement learning, Stability of nonlinear systems, Optimal control
Abstract: Designing a stabilizing controller for nonlinear systems is a challenging task, especially for high-dimensional problems with unknown dynamics. Traditional reinforcement learning algorithms applied to stabilization tasks tend to drive the system close to the equilibrium point. However, these approaches often fall short of achieving true stabilization and result in persistent oscillations around the equilibrium point. In this work, we propose a reinforcement learning algorithm that stabilizes the system by learning a local linear representation of the dynamics. The main component of the algorithm is integrating the learned gain matrix directly into the neural policy. We demonstrate the effectiveness of our algorithm on several challenging high-dimensional dynamical systems. In these simulations, our algorithm outperforms popular reinforcement learning algorithms, such as soft actor-critic (SAC) and proximal policy optimization (PPO), and successfully stabilizes the system. To support the numerical results, we provide a theoretical analysis of the feasibility of the learned algorithm for both deterministic and stochastic reinforcement learning settings, along with a convergence analysis of the proposed learning algorithm. Furthermore, we verify that the learned control policies indeed provide asymptotic stability for the nonlinear systems.

Keywords: Reinforcement learning, Markov processes, Stochastic optimal control
Abstract: In this paper, we present a framework to understand the convergence of commonly used Q-learning reinforcement learning algorithms in practice. Two salient features of such algorithms are: (i)~the Q-table is recursively updated using an agent state (such as the state of a recurrent neural network) which is not a belief state or an information state and (ii)~policy regularization is often used to encourage exploration and stabilize the learning algorithm. We investigate the simplest form of such Q-learning algorithms which we call regularized agent-state-based Q-learning (RASQL) and show that it converges under mild technical conditions to the fixed point of an appropriately defined regularized MDP, which depends on the stationary distribution induced by the behavioral policy. We also show that a similar analysis continues to work for a variant of RASQL that learns periodic policies. We present numerical examples to illustrate that the empirical convergence behavior matches with the proposed theoretical limit.

Keywords: Markov processes, Reinforcement learning, Computer/Network Security
Abstract: With the growing sophistication of cybersecurity threats, artificial intelligence driven defense solutions have gained significant attention, particularly reinforcement learning based approaches, which make sequential decisions based on the latest network information. Despite their success in adaptability, scalability, and real-time response, these methods remain highly vulnerable to adversarial deception—strategic manipulations that distort the defender’s perception of the network security state. This paper introduces a budget-constrained adversarial model, in which an attacker deploys decoys to mislead the defender about the true state of network compromises, disrupting effective and timely decision-making. Deception is formulated as an augmented Markov Decision Process, allowing an intelligent adversary to account for the defender’s perception of network compromises and anticipate defensive actions. An optimal deception policy is derived, enabling the strategic placement of decoys to maximize long-term security degradation while evading detection. A worst-case theoretical bound is established to quantify the long-term impact of deception on network security. Numerical experiments demonstrate that even minimal decoy-based deception significantly weakens network security, particularly against deterministic defense policies.

Keywords: Reinforcement learning, Statistical learning, Decentralized control
Abstract: Recently, there has been a surge of interest in decentralized learning approaches to tackle complex collaborative tasks in multi-agent systems. One of the most promising approaches is multi-agent reinforcement learning (MARL). Yet, as the number of agents becomes larger, the sample complexity in MARL increases exponentially, making scalability a fundamental issue. Networked MARL algorithms can address this issue by leveraging a communication network for information exchange between the agents. For homogeneous network MARL, previous research established a regret upper-bound sqrt{MH^4SAT}. Recent approaches rely on global knowledge about the structure of the communication network, which poses a serious limitation when it is not known or changes depending on the task. In this paper, we overcome this limitation by proposing a novel networked MARL algorithm with an upper-confidence bound (UCB) exploration strategy, called provably efficient local-information networked (PrELIN) MARL, that does not require any global information but only relies on the local interactions between the agents. Furthermore, we derive the regret and sample complexity for our algorithm and show that the regret bound may still remain sublinear.

Keywords: Reinforcement learning, Cooperative control, Autonomous robots
Abstract: Multi-Agent Reinforcement Learning (MARL) has been extensively applied in cooperative control tasks, yet credit assignment remains a critical challenge. Existing approaches predominantly employ global reward sharing mechanisms or rely on monolithic neural network architectures, which may lead to inaccurate credit attribution for individual agents and consequently result in suboptimal learning efficiency. In this paper, we propose the Local Shapley Policy Optimization (LSPO) method to address the multi-agent credit assignment problem. The Shapley values are introduced to reasonably assess the contribution of each agent. Furthermore, we present a method for calculating local Shapley values to address the issue of dimensional explosion, and we theoretically demonstrate that local Shapley values can effectively approximate global Shapley values. Extensive experiments conducted on several cooperative multi-agent benchmarks demonstrate that our approach achieves superior task performance compared to existing state-of-the-art methods. Empirical results reveal that the proposed LSPO framework substantially enhances both learning efficiency and coordination capabilities in complex multi-agent environments.

Keywords: Reinforcement learning, Markov processes
Abstract: Swarm-level task learning provides a basis for learning complex tasks in swarm systems using swarm-level features. We propose a method, SwaRM-L, to learn reward machines (RMs) that encode non-Markovian reward functions. We use reward machines to specify the task and its temporal structure. Our approach enables a swarm of agents to learn an RM that eventually becomes equivalent to ground truth RM (i.e., the specified task) in this environment (agents have no access to the ground truth RM and only learn it through environment interaction). We use generalized moments (GMs) to characterize swarm features and estimate the RM state. Each agent maintains an estimated GM to contribute to the collective learning process. The agents use a gossip algorithm to communicate with neighbors and update their estimated GMs. We prove that our method converges to an optimal policy and learns an equivalent RM to the ground truth RM within the environment. We evaluate our proposed method in three case studies involving forty agents with homogeneous dynamics. Our results demonstrate the effectiveness of our method in learning complex swarm behaviors.

Keywords: Reinforcement learning, Stochastic optimal control, Optimization algorithms
Abstract: Two-time-scale Stochastic Approximation (SA) is an iterative algorithm with applications in reinforcement learning and optimization. Prior finite time analysis of such algorithms has focused on fixed point iterations with mappings contractive under Euclidean norm. Motivated by applications in reinforcement learning, we give the first mean square bound on non linear two-time-scale SA where the iterations have arbitrary norm contractive mappings and Markovian noise. We show that the mean square error decays at a rate of O(1/n^{2/3}) in the general case, and at a rate of O(1/n) in a special case where the slower timescale is noiseless. Our analysis uses the generalized Moreau envelope to handle the arbitrary norm contractions and solutions of Poisson equation to deal with the Markovian noise. By analyzing the SSP Q-Learning algorithm, we give the first O(1/n) bound for an algorithm for asynchronous control of MDPs under the average reward criterion. We also obtain a rate of O(1/n) for Q-Learning with Polyak-averaging and provide an algorithm for learning Generalized Nash Equilibrium (GNE) for strongly monotone games which converges at a rate of O(1/n^{2/3}).

Keywords: Manufacturing systems and automation, Reinforcement learning, Discrete event systems
Abstract: Efficient scheduling significantly impacts productivity and operational performance, particularly in flexible resource allocation scenarios. The Flexible Job Shop Scheduling Problem (FJSP) extends traditional scheduling by allowing each operation to be processed on multiple machines, enhancing practical applicability but increasing computational complexity. Existing methods, ranging from exact solvers to heuristic/metaheuristic algorithms, either incur high computational costs or yield suboptimal solutions. Recent reinforcement learning (RL)-based approaches attempt a balance between optimality and efficiency; however, intrinsic characteristics of FJSP remain underutilized for better solution quality. This paper introduces the Action-Priority Driven Policy Optimization (APDPO) method, explicitly designed to address three critical challenges: achieving superior solutions, improving interpretability via an intuitive policy representation, and enhancing computational efficiency through direct, gradient-free policy updates. A policy improvement theorem ensures performance enhancement under specified conditions. Comparative evaluations confirm that APDPO outperforms existing approaches by delivering better solution quality, reduced computational requirements, and greater decision transparency.

Keywords: Statistical learning, Estimation, Reinforcement learning
Abstract: We consider the off-policy evaluation (OPE) problem in contextual bandits, where the goal is to estimate the value of a target policy using the data collected by a logging policy. Most popular approaches to the OPE are variants of the doubly robust (DR) estimator obtained by combining a direct method (DM) estimator and a correction term involving the inverse propensity score (IPS). Existing algorithms primarily focus on strategies to reduce the variance of the DR estimator arising from large IPS. We propose a new approach called the Doubly Robust with Information borrowing and Context-based switching (DR-IC) estimator that focuses on reducing both bias and variance. The DR-IC estimator replaces the standard DM estimator with a parametric reward model that borrows information from the `closer’ contexts through a correlation structure that depends on the IPS. The DR-IC estimator also adaptively interpolates between this modified DM estimator and a modified DR estimator based on a context-specific switching rule. We give provable guarantees on the performance of the DR-IC estimator. We also demonstrate the superior performance of the DR-IC estimator compared to the state-of-the-art OPE algorithms on a number of benchmark problems.

Keywords: Randomized algorithms, Statistical learning, Optimization algorithms
Abstract: Conformal prediction and scenario optimization constitute two important classes of statistical learning frameworks to certify decisions made using data. They have found numerous applications in control theory, machine learning and robotics. Despite intense research in both areas, and apparently similar results, a clear connection between these two frameworks has not been established. By focusing on the so-called vanilla conformal prediction, we show rigorously how to choose appropriate score functions and set predictor map to recover well-known bounds on the probability of constraint violation associated with scenario programs. We also show how to treat ranking of nonconformity scores as a one-dimensional scenario program with discarded constraints, and use such connection to recover vanilla conformal prediction guarantees on the validity of the set predictor. We also capitalize on the main developments of the scenario approach, and show how we could analyze calibration conditional conformal prediction under this lens. Our results establish a theoretical bridge between conformal prediction and scenario optimization.

Keywords: Statistical learning, Stochastic systems
Abstract: We derive a finite-sample probabilistic bound on the parameter estimation error of a system identification algorithm for Linear Switched Systems. The algorithm estimates Markov parameters from a single trajectory and applies a variant of the Ho-Kalman algorithm to recover the system matrices. Our bound guarantees statistical consistency under the assumption that the true system exhibits quadratic stability. The proof leverages the theory of weakly dependent processes. To the best of our knowledge, this is the first finite-sample bound for this algorithm in the single-trajectory setting.

Keywords: Statistical learning, Data driven control, Optimization
Abstract: We investigate the Probably Approximately Correct (PAC) property of scenario decision algorithms, which refers to their ability to produce decisions with an arbitrarily low risk of violating unknown safety constraints, provided a sufficient number of realizations of these constraints are sampled. While several PAC sufficient conditions for such algorithms exist in the literature---such as the finiteness of the VC dimension of their associated classifiers, or the existence of a compression scheme---it remains unclear whether these conditions are also necessary. In this work, we demonstrate through counterexamples that these conditions are not necessary in general. These findings stand in contrast to binary classification learning, where analogous conditions are both sufficient and necessary for a family of classifiers to be PAC. Furthermore, we extend our analysis to stable scenario decision algorithms, a broad class that includes practical methods like scenario optimization. Even under this additional assumption, we show that the aforementioned conditions remain unnecessary. Furthermore, we introduce a novel quantity, called the dVC dimension, which serves as an analogue to the VC dimension for scenario decision algorithms. We prove that the finiteness of this dimension is a PAC necessary condition for scenario decision algorithms. This allows to (i) guide algorithm users and designers to recognize algorithms that are not PAC, and (ii) contribute to a comprehensive characterization of PAC scenario decision algorithms.

Keywords: Learning, Estimation, Statistical learning
Abstract: In this brief paper, we present a naive aggregation algorithm for a typical learning problem with expert advice setting, in which the task of improving generalization, i.e., model validation, is embedded in the learning process as a sequential decision-making problem. In particular, we consider a class of learning problem of point estimations for modeling high-dimensional nonlinear functions, where a group of experts update their parameter estimates using the discrete-time version of gradient systems, with small additive noise term, guided by the corresponding subsample datasets obtained from the original dataset. Here, our main objective is to provide conditions under which such an algorithm will sequentially determine a set of mixing distribution strategies used for aggregating the experts' estimates that ultimately leading to an optimal parameter estimate, i.e., as a consensus solution for all experts, which is better than any individual expert's estimate in terms of improved generalization or learning performances. Finally, as part of this work, we present some numerical results for a typical case of nonlinear regression problem.

Keywords: Statistical learning, Identification, Stochastic systems
Abstract: Band-limited functions are fundamental objects that are widely used in systems theory and signal processing. In this paper we refine a recent nonparametric, nonasymptotic method for constructing simultaneous confidence regions for band-limited functions from noisy input-output measurements, by working in a Paley-Wiener reproducing kernel Hilbert space. Kernel norm bounds are tightened using a uniformly-randomized Hoeffding's inequality for small samples and an empirical Bernstein bound for larger ones. We derive an approximate threshold, based on the sample size and how informative the inputs are, that governs which bound to deploy. Finally, we apply majority voting to aggregate confidence sets from random subsamples, boosting both stability and region size. We prove that even per-input aggregated intervals retain their simultaneous coverage guarantee. These refinements are also validated through numerical experiments.

Keywords: Learning, Statistical learning, Reinforcement learning
Abstract: This paper presents a new algorithm for neural contextual bandits (CBs) that addresses the challenge of delayed reward feedback, where the reward for a chosen action is revealed after a random, unknown delay. This scenario is common in applications such as online recommendation systems and clinical trials, where reward feedback is delayed because the outcomes or results of a user’s actions (such as recommendations or treatment responses) take time to manifest and be measured. The proposed algorithm, called textit{Delayed NeuralUCB}, uses upper confidence bound (UCB)-based exploration strategy. Under the assumption of independent and identically distributed sub-exponential reward delays, we derive an upper bound on the cumulative regret over T-length horizon, that scales as O(tilde{d} sqrt{T log T}+ tilde{d}^{3/2} D_+log(T)^{3/2}) where tilde{d} denotes the effective dimension of the neural tangent kernel matrix, and D_+ depends on the expected delay Ebb[tau]. We further consider a variant of the algorithm, called Delayed NeuralTS, that uses Thompson Sampling based exploration. Numerical experiments on real-world datasets, such as MNIST and Mushroom, along with comparisons to benchmark approaches, demonstrate that the proposed algorithms effectively manage varying delays and are well-suited for complex real-world scenarios.

Keywords: Reinforcement learning, Optimal control, Statistical learning
Abstract: Inverse reinforcement learning (IRL) usually assumes the reward function model is pre-specified as a weighted sum of features and estimates the weighting parameters only. However, how to select features and determine a proper reward model is nontrivial and experience-dependent. A simplistic model is less likely to explain the intention, while a model with high complexity leads to substantial computation cost and potential overfitting. This paper addresses this trade-off in the model selection for IRL problems by introducing the structural risk minimization (SRM) framework from statistical learning. SRM selects an optimal reward model from a hypothesis set minimizing both estimation error and complexity. To formulate an SRM scheme for IRL, we estimate the policy gradient from given demonstration as the empirical risk, and establish the upper bound of Rademacher complexity as the penalty for hypothesis function classes. The SRM learning guarantee is further presented. In particular, we provide the explicit form for the linear weighted sum setting. Simulations demonstrate the performance and efficiency of our algorithm.

Keywords: Human-in-the-loop control, Agents-based systems, Distributed control
Abstract: Abstract— This paper studies the optimal resource allocation problem within a multi-agent network composed of both autonomous agents and humans. The main challenge lies in the globally coupled constraints that link the decisions of autonomous agents with those of humans. To address this, we propose a novel reformulation that transforms these coupled constraints into decoupled local constraints defined over the system’s communication graph. Building on this reformulation, and incorporating a human response model that captures human-robot interactions while accounting for individual pref- erences and biases, we develop a fully distributed algorithm. This algorithm guides the states of the autonomous agents to equilibrium points which, when combined with the human responses, yield a globally optimal resource allocation. We provide both theoretical analysis and numerical simulations to validate the effectiveness of the proposed approach.

Keywords: Decentralized control, Distributed control, Optimal control
Abstract: The linear quadratic Gaussian (LQG) control problem for the linear wave equation on the unit circle with fully distributed actuation and partial state measurements is considered. An analytical solution to a spatial discretization of the problem is obtained. The main result of this work illustrates that for specific parameter values, the optimal LQG policy is completely decentralized, meaning only a measurement at spatial location i is needed to compute an optimal control signal to actuate at this location. The relationship between performance and decentralization as a function of parameters is explored. Conditions for complete decentralization are related to metrics of kinetic and potential energy quantities and control effort.

Keywords: Decentralized control, Iterative learning control, Optimization
Abstract: Data centers are hitting a bottleneck in available power as the demand for computations increases, and sustainable power management has emerged as a key challenge in green computing. We propose algorithms to scalably manage power across servers and solutions to guarantee and hasten convergence in the power trajectory. This makes power use more efficient and stable, and provides a broad class of pricing functions to discourage excessive power consumption. A foundational framework is established to optimizing power-aware servers. We provide proofs of convergence with guarantees on rates. Simulations support our theory.

Keywords: Decentralized control, Optimal control, Building and facility automation
Abstract: What is the performance cost of using simple, decoupled control policies in inherently coupled systems? Motivated by industrial refrigeration systems, where centralized compressors exhibit economies of scale yet traditional control employs decoupled room-by-room temperature regulation, we address this question through the lens of multi-location inventory control. Here, a planner manages multiple inventories to meet stochastic demand while minimizing costs that are coupled through nonlinear ordering functions reflecting economies of scale. Our main contributions are: (i) a surprising equivalence result showing that optimal stationary base-stock levels for individual locations remain unchanged despite coupling when restricting attention to decoupled strategies; (ii) tight performance bounds for simple decoupled policies relative to optimal coupled policies, revealing that the worst-case ratio depends solely on the degree of nonlinearity in the ordering cost function; and (iii) an analysis of a practical online algorithm that achieves competitive performance without solving complex dynamic programs. Numerical simulations demonstrate that while decoupled policies significantly outperform their worst-case guarantees in typical scenarios, they still exhibit meaningful suboptimality compared to fully coordinated strategies. These results provide actionable guidance for system operators navigating the trade-off between control complexity and operational efficiency in coupled systems.

Keywords: Decentralized control, Distributed control, Time-varying systems
Abstract: This paper proposes a novel protocol for achieving arithmetic average consensus over time-varying directed graphs using broadcast-only communication, where agents cannot selectively address out-neighbors. The proposed approach extends existing methods to compute the dominant left eigenvector of an infinite product of time-varying row-stochastic matrices in a fully distributed manner. By leveraging a local rescaling strategy, our algorithm ensures that agents autonomously compute correction offsets that drive the consensus value toward the arithmetic average of their initial states. The method circumvents the impractical requirement of point-to-point communication and eliminates the need for column stochastic normalization of the weights, which requires each node to know its local out-neighborhood; the proposed method only imposes a mild requirement on knowing an upper bound on the total number of agents. Theoretical convergence guarantees are provided, and simulations confirm the solution’s effectiveness.

Keywords: Decentralized control, Stochastic optimal control, Reinforcement learning
Abstract: Learning-to-Communicate (LTC) in partially observable environments has emerged and received increasing attention in deep multi-agent reinforcement learning, where the control and communication strategies are jointly learned. On the other hand, the impact of communication has been extensively studied in control theory. In this paper, we seek to formalize and better understand LTC by bridging these two lines of work, through the lens of information structures (ISs). To this end, we formalize LTC in decentralized partially observable Markov decision processes (Dec-POMDPs) under the common-information-based framework from decentralized stochastic control, and classify LTC problems based on the ISs before (additional) information sharing. We first show that non-classical LTCs are computationally intractable in general, and thus focus on quasi-classical (QC) LTCs. We then propose a series of conditions for QC LTCs, violating which can cause computational hardness in general. Further, we develop provable planning and learning algorithms for QC LTCs, and show that some examples of QC LTCs satisfying the above conditions can be solved with quasi-polynomial time and samples. Along the way, we also establish some relationship between (strictly) QC IS and the condition of having strategy-independent common-information-based beliefs (SI-CIBs), as well as solving Dec-POMDPs without computationally intractable oracles but beyond those with the SI-CIB condition, which may be of independent interest.

Keywords: Discrete event systems, Automata
Abstract: Recently, the definition of Property-Based Transparency has been proposed in the literature for Discrete Event Systems. Property-Based Transparency is a utility notion that establishes that a legitimate receiver must know if a given property of interest is true in the system before it becomes false. Since the satisfaction of the property of interest is associated with useful states of the system, we call the proposed definition of transparency with respect to the set of useful states State-Based Transparency (ST). In this paper, the ST property is generalized to consider a decentralized approach, leading to the definition of state-based co-transparency. A polynomial-time algorithm to verify state-based co-transparency is proposed. The verification algorithm proposed in this paper can also be applied to the centralized case.

Keywords: Optimization algorithms, Distributed control, Learning
Abstract: We analyze Decentralized Online Optimization algorithms using the Performance Estimation Problem approach which allows, to automatically compute exact worst- case performance of optimization algorithms. Our analysis shows that several available performance guarantees are very conservative, sometimes by multiple orders of magnitude, and can lead to misguided choices of algorithm. Moreover, at least in terms of worst-case performance, some algorithms appear not to benefit from inter-agent communications for a significant period of time. We show how to improve classical methods by tuning their step-sizes, and find that we can save up to 20% on their actual worst-case performance regret.

Keywords: Control of networks, Network analysis and control, Stochastic systems
Abstract: Transmission Neural Networks (TransNNs) proposed by Gao and Caines (2022) serve as both virus spread models over networks and neural network models with tuneable activation functions. This paper establishes that TransNNs provide upper bounds on the infection probability generated from the associated Markovian stochastic Susceptible-Infected- Susceptible (SIS) model with 2^n state configurations where n is the number of nodes in the network, and can be employed as an approximate model for the latter. Based on such an approximation, a TransNN-based receding horizon control approach for mitigating virus spread is proposed and we demonstrate that it allows significant computational savings compared to the dynamic programming solution to Markovian SIS model with 2^n state configurations, as well as providing less conservative control actions compared to the TransNN-based optimal control. Finally, numerical comparisons among (a) dynamic programming solutions for the Markovian SIS model, (b) TransNN-based optimal control and (c) the proposed TransNN-based receding horizon control are presented.

Keywords: Network analysis and control, Cooperative control, Networked control systems
Abstract: This work presents a passivity-based analysis for the nonlinear output agreement problem in network systems over directed graphs. We reformulate the problem as a convergence analysis on the agreement submanifold. First, we establish how passivity properties of individual agents and controllers determine the passivity of their associated system relations. Building on this, we introduce the concept of submanifold-constrained passivity and develop a novel compensation theorem that ensures output convergence to the agreement submanifold. Unlike previous approaches, our approach can analyze the network system with arbitrary digraphs and any passive agents. We apply this framework to analyze the output agreement problem for network systems consisting of nonlinear and passive agents. Numerical examples support our results.

Keywords: Network analysis and control, Nonlinear systems, Agents-based systems
Abstract: This work describes a collective decision-making dynamical process in a multiagent system under the assumption of cooperative higher-order interactions within the community, modeled as a hypernetwork. The nonlinear interconnected system is characterized by saturated nonlinearities that describe how agents transmit their opinion state to their neighbors in the hypernetwork, and by a bifurcation parameter representing the community's social effort. We show that the presence of higher-order interactions leads to the unfolding of a pitchfork bifurcation, introducing an interval for the social effort parameter in which the system exhibits bistability. With equilibrium points representing collective decisions, this implies that, depending on the initial conditions, the community will either remain in a deadlock state (with the origin as the equilibrium point) or reach a nontrivial decision. A numerical example is given to illustrate the results.

Keywords: Network analysis and control, Control of networks, Large-scale systems
Abstract: Graphons generalize graphs and define a limit object of a converging graph sequence. The notion of graphons allows for a generic representation of coupled network dynamical systems. We are interested in approximating optimal switching controls for graphon dynamical systems. To this end, we apply a decomposition approach comprised of a relaxation and a reconstruction step. We extend the sum-up rounding algorithm to operate on a finite partition of the continuous vertex set in a graphon setting and restore integer feasibility for a given relaxed control solution. Finally, we derive an L1 bound for the state variables for structurally similar graphs and approximated switching controls. We verify our claims by simulating the Lotka-Volterra equations on a graph.

Keywords: Network analysis and control, Control of networks
Abstract: Consensus dynamics are ubiquitous in control theory, with applications spanning from technological fields to social sciences. In this Letter, we evaluate the impact of higher-order, many-body interactions on the ability of the network to attain consensus. Modeling multi-body interactions with circulating directed hypergraphs, we provide for the first time global conditions for average consensus in the presence of time-varying and possibly non-smooth coupling protocols. Our results are then applied to study the effect of homophily in opinion dynamics.

Keywords: Compartmental and Positive systems, Network analysis and control, Model Validation
Abstract: Socio-technical systems providing essential services, like water, energy, and transportation, often create captive user segments due to limited alternatives. This can lead to user dissatisfaction and complex social dynamics. Building on a recent dynamic model, this paper extends the analysis of captive user dissatisfaction to networks of interconnected communities, using a directed bipartite leader–follower structure. We investigate the steady-state properties of the model and analyze how leader communities can mitigate or amplify dissatisfaction among their followers. A key insight is that consensus in dissatisfaction levels does not stem from network topology but from the alignment of a Dissatisfaction Index, a new metric introduced in this letter that reflects each community's perception of service quality

Keywords: Network analysis and control, Large-scale systems, Computational methods
Abstract: Optimal sensor placement is essential for state estimation and effective network monitoring. As known in the literature, this problem becomes particularly challenging in large-scale undirected or bidirected cyclic networks with parametric uncertainties, such as water distribution networks (WDNs), where pipe resistance and demand patterns are often unknown. Motivated by the challenges of cycles, parametric uncertainties, and scalability, this paper proposes a sensor placement algorithm that guarantees structural observability for cyclic and acyclic networks with parametric uncertainties. By leveraging a graph-based strategy, the proposed method efficiently addresses the computational complexities of large-scale networks. To demonstrate the algorithm's effectiveness, we apply it to several EPANET benchmark WDNs. Most notably, the developed algorithm solves the sensor placement problem with guaranteed structural observability for the L-town WDN with 1694 nodes and 124 cycles in under 0.1 seconds.

Keywords: Control of networks, Optimization, Lyapunov methods
Abstract: We consider a distributed cloud service deployed at a set of distinct server pools. Arriving jobs are classified into heterogeneous types, in accordance with their setup times which are differentiated at each of the pools. A dispatcher for each job type controls the balance of load between pools, based on decentralized feedback. The system of rates and queues is modeled by a fluid differential equation system, and analyzed via convex optimization. A first, myopic policy is proposed, based on task delay-to-service. Under a simplified dynamic fluid queue model, we prove global convergence to an equilibrium point which minimizes the mean setup time; however queueing delays are incurred with this method. A second proposal is then developed based on proximal optimization, which explicitly models the setup queue and is proved to reach an optimal equilibrium, devoid of queueing delay. Results are demonstrated through a simulation example.

Keywords: Optimization algorithms, Machine learning, Emerging control applications
Abstract: Bilevel optimization involves a hierarchical structure where one problem is nested within another, leading to complex interdependencies between levels. We propose a single-loop, tuning-free algorithm that guarantees anytime feasibility, i.e., approximate satisfaction of the lower-level optimality condition, while ensuring descent of the upper-level objective. At each iteration, a convex quadratically-constrained quadratic program (QCQP) with a closed-form solution yields the search direction, followed by a backtracking line search inspired by control barrier functions to ensure safe, uniformly positive step sizes. The resulting method is scalable, requires no hyperparameter tuning, and converges under mild local regularity assumptions. We establish an O(1/k) ergodic convergence rate in terms of a first-order stationary metric and demonstrate the algorithm’s effectiveness on representative bilevel tasks.

Keywords: Optimization algorithms
Abstract: Optimization algorithms for time-varying cost functions extend classical methods, which cannot track dynamic minima. However, they typically assume access to the partial derivative of the cost function with respect to time. This is an assumption not satisfied in many situations where, although the time-varying parameters can be measured, either their time-derivatives or their model are unknown.

Distributed optimization algorithms for time-varying costs typically inherit these limitations and introduce additional challenges, such as requiring each node to measure the time-varying parameters its local cost depends on. This work addresses these issues by first developing a centralized time-varying optimization algorithm that ensures convergence to a bounded neighborhood of the optimal trajectory. We then extend it to a distributed framework, relaxing the aforementioned measurement requirements. The effectiveness of our approach is validated through numerical examples of varying complexity.

Keywords: Optimization algorithms, Agents-based systems, Communication networks
Abstract: This paper considers distributed online nonconvex optimization with time-varying inequality constraints over a network of agents. For time-varying graphs, we propose a distributed online primal--dual algorithm with compressed communication to efficiently utilize communication resources. We show that the proposed algorithm establishes an mathcal{O}( {{T^{max { {1 - {theta _1},{theta _1}} }}}} ) network regret bound and an mathcal{O}( {T^{1 - {theta _1}/2}} ) network cumulative constraint violation bound, where T is the number of iterations and {theta _1} in ( {0,1} ) is a user-defined trade-off parameter. When Slater's condition holds, the network cumulative constraint violation bound is reduced to mathcal{O}( {T^{1 - {theta _1}}} ). These bounds are comparable to the state-of-the-art results established by existing distributed online algorithms with perfect communication for distributed online convex optimization with (time-varying) inequality constraints. Finally, a simulation example is presented to validate the theoretical results.

Keywords: Algebraic/geometric methods, Optimization algorithms, Optimal control
Abstract: A fundamental issue at the core of trajectory optimization on smooth manifolds is handling the implicit manifold constraint within the dynamics. The conventional approach is to enforce the dynamic model as a constraint. However, we show that this approach leads to significantly redundant operations, as well as being heavily dependent on the state space representation. Specifically, we propose an intrinsic successive convexification methodology for optimal control on smooth manifolds. This so-called iSCvx is then applied to a representative example involving attitude trajectory optimization for a spacecraft subject to non-convex constraints.

Keywords: Optimization algorithms, Machine learning, Stochastic systems
Abstract: Motivated by the growing need for black-box optimization and data privacy, we introduce a collaborative Bayesian optimization (BO) framework that addresses both of these challenges. In this framework agents work collaboratively to optimize a function they only have oracle access to. In order to mitigate against communication and privacy constraints, agents are not allowed to share their data but can share their Gaussian process (GP) surrogate models. To enable collaboration under these constraints, we construct a central model to approximate the objective function by leveraging the concept of Wasserstein barycenters of GPs. This central model integrates the shared models without accessing the underlying data. A key aspect of our approach is a collaborative acquisition function that balances exploration and exploitation, allowing for the optimization of decision variables collaboratively in each iteration. We prove that our proposed algorithm is asymptotically consistent and that its implementation via Monte Carlo methods is numerically accurate. Through numerical experiments, we demonstrate that our approach outperforms other baseline collaborative frameworks and is competitive with centralized approaches that do not consider data privacy.

Keywords: Optimization algorithms, Numerical algorithms, Machine learning
Abstract: This paper studies properties of fixed points of generalised Extra-gradient (GEG) algorithms applied to min- max problems. We discuss connections between saddle points of the objective function of the min-max problem and GEG fixed points. We show that, under appropriate step-size selections, the set of local saddle points (local Nash equilibria) is a subset of locally stable fixed points of GEG. Convergence properties of the GEG algorithm are obtained through a stability analysis of a discrete-time dynamical system. The results when compared to existing methods are illustrated through numerical examples.

Keywords: Optimization algorithms, Robust control
Abstract: We propose a novel approach for analyzing dynamic regret of first-order online convex optimization (OCO) algorithms for strongly convex and Lipschitz-smooth objectives. Crucially, we provide a general analysis that is applicable to a wide range of first-order algorithms that can be expressed as an interconnection of a linear dynamical system in feedback with a first-order oracle. By leveraging Integral Quadratic Constraints (IQCs), we derive a semi-definite program which, when feasible, provides a regret guarantee for the online algorithm. For this, the concept of variational IQCs is introduced as the generalization of IQCs to time-varying monotone operators. Our bounds capture the temporal rate of change of the problem in the form of the path length of the time-varying minimizer and the objective function variation. In contrast to standard results in OCO, our results do not require neither the assumption of gradient boundedness, nor that of a bounded feasible set. Numerical analyses showcase the ability of the approach to capture the dependence of the regret on the function class condition number.

Keywords: Statistical learning, Optimization algorithms, PID control
Abstract: Stopping criteria for Bayesian optimization (BO) automatically terminate the optimization algorithm when a near-optimal solution has likely been reached, avoiding unnecessary expenditure of computational resources. Existing criteria, however, only guarantee accuracy given a well-specified Gaussian process surrogate model, an assumption that often does not hold in practice. We propose a stopping criterion for Bayesian optimization when the mean function of the Gaussian process surrogate model is uncertain, but modeled with a prior. The prior induces a probability distribution over surrogate models, which our criterion uses to evaluate the probability that a model will be stopped under an existing stopping criterion. We demonstrate that our criterion will eventually terminate BO, with high probability of achieving a desired accuracy to the optimal value. Empirical analysis on a controller tuning problem for an autonomous underwater vehicle suggests that our method can terminate BO with high accuracy solutions, particularly for low dimensional problems.

Keywords: Game theory, Agents-based systems
Abstract: The letter studies a multi-cluster aggregative game, where multiple clusters exist and each cluster consists of a group of agents. Each cluster serves as a noncooperative player, and agents within the same cluster aim to minimize the sum of their individual cost functions cooperatively. The local cost function of each agent depends on both its own decision and the aggregate decision of each cluster. The objective is to find the Nash equilibrium (NE) of the game in a distributed manner over time-varying unbalanced graphs. To this end, a distributed discrete-time NE seeking algorithm is developed, incorporating a novel tracking technique to estimate the aggregate decision of each cluster and the push-sum protocol to accommodate time-varying unbalanced graphs. The linear convergence of the proposed algorithm is established rigorously via multi-step contraction analysis and linear systems of inequalities. Finally, numerical simulations of an Energy Internet System validate the effectiveness of the algorithm.

Keywords: Game theory, Decentralized control, Reinforcement learning
Abstract: We address the problem of finding an optimal decentralized control strategy, defined as achieving approximate Correlated Equilibrium, a generalization of Nash equilibrium. Each controller uses a Reinforcement Learning algorithm within a Markov game framework, abstracting continuous states into discrete ones. Controllers observe the system's discrete state and incurred costs, updating their strategies privately. We prove convergence to approximate Correlated Equilibrium. Our algorithm is applied to a water distribution network with two controllers managing a tank's water level in a decentralized setting, using real-world consumption data. Previously, this problem was deemed to be computationally challenging owing to the discrete controls (number of pumps switched on), resulting in a Mixed Integer optimization. The designed controllers aim to ensure stability, safety, and energy minimization. By training over multiple episodes, optimal weighting parameters for objectives are determined using Blackwell's Approachability theorem. Simulation results show the attainment of approximate correlated equilibrium in accordance with our theoretical results.

Keywords: Game theory, Machine learning, Cooperative control
Abstract: We consider a game with N cooperative players that have a global objective. Simple distributed algorithms such as gradient play often converge to a Nash equilibrium (NE). However, a NE typically suffers from poor global performance. Converging to the global optimum requires explicit communication and coordination. In this paper, we propose a new approach to improve the performance at NE. Our method uses machine learning offline training to design games with efficient NE. In particular, we use a dataset of games with parameters coming from a certain distribution (e.g., uniformly random player locations). We then train a deep neural network (DNN) where the input is the local measurement available to the player and the output is the reward parameters that best approximate the global objective. We demonstrate our approach for two %three classes of games: energy games and wireless power control games. Our approach offers significant performance boosts while requiring no communication between the players and no complexity increase in real-time.

Keywords: Mean field games, Large-scale systems, Nonlinear systems
Abstract: The paper studies a nonlinear mean field game (MFG) model based on McKean-Vlasov (MV) approximation, which involves multiple major agents and multiple populations of minor agents. Since the interactions occur not only between major–minor agents but also among major–major agents and minor–minor populations, complicating equilibrium existence analysis, we first cast the MFG as two sets of adjoint stochastic McKean–Vlasov equations coupled via the mean field behavior. Subsequently, a new norm on the product probability measure space is constructed to prove that one set of equations exists solutions, while the existence of solutions to the other set is proved based on Pontryagin stochastic maximum principle. Then, within the established product normed space, Banach's fixed point theorem is utilized to prove the existence of equilibrium for this MFG, which is manifested as solutions to two sets of coupled equations. Finally, an epsilon-Nash equilibrium is proved for the finite agent situation, in which epsilon to 0 while all population sizes go to infty, and a numerical experiment under certain settings is carried out.

Keywords: Mean field games, Game theory, Reinforcement learning
Abstract: Mean Field Games (MFGs) offer a powerful framework for studying large-scale multi-agent systems. Yet, learning Nash equilibria in MFGs remains a challenging problem, particularly when the initial distribution is unknown or when the population is subject to common noise. In this paper, we introduce an efficient deep reinforcement learning (DRL) algorithm designed to achieve population-dependent Nash equilibria without relying on averaging or historical sampling, inspired by Munchausen RL and Online Mirror Descent. The resulting policy is adaptable to various initial distributions and sources of common noise. Through numerical experiments on seven canonical examples, we demonstrate that our algorithm exhibits superior convergence properties compared to state-of-the-art algorithms, particularly a DRL version of Fictitious Play for population-dependent policies. The performance in the presence of common noise underscores the robustness and adaptability of our approach.

Keywords: Game theory, Modeling, Learning
Abstract: Federated learning (FL) offers a decentralized approach to machine learning, where multiple agents collaboratively train a model while preserving data privacy. In this paper, we investigate the decision-making and equilibrium behavior in FL systems, where agents choose between participating in global training or conducting independent local training. The problem is first modeled as a stage game and then extended to a repeated game to analyze the long-term dynamics of agent participation. For the stage game, we characterize the participation patterns and identify Nash equilibrium, revealing how data heterogeneity influences the equilibrium behavior—specifically, agents with similar data qualities will participate in FL as a group. We also derive the optimal social welfare strategy and show that it lies in a neighborhood of Nash equilibrium. In the repeated game, we propose a privacy-preserving, computationally efficient myopic strategy. This strategy enables agents to make practical decisions under bounded rationality and converges to a neighborhood of Nash equilibrium of the stage game in finite time. By combining theoretical insights with practical strategy design, this work provides a realistic and effective framework for guiding and analyzing agent behaviors in FL systems.

Keywords: Boolean control networks and logic networks, Game theory, Algebraic/geometric methods
Abstract: A multi-potential game extends the classical potential game by grouping players into distinct subsets. Within each subset, given that the strategies of players in other subsets remain fixed, the interactions among this subset's players form a potential subgame characterized by its own restricted potential function. In this paper, we first establish that when the conflict relationships among players exist and follow structural balance, the multi-potential game achieves the minimal potential index of two. Then, we prove that the pure strategy Nash equilibria in the multi-potential framework, called the pure strategy multi-potential Nash equilibria, are a subset of those in the original game. Furthermore, we establish the conditions under which the set of pure strategy Nash equilibria in the original game coincides with the set of pure strategy multi-potential Nash equilibria. Finally, we provide an illustrative example to demonstrate our findings.

Keywords: Autonomous vehicles, Predictive control for linear systems, Robotics
Abstract: This paper presents a safe model predictive control (SMPC) framework designed to ensure the satisfaction of hard constraints, for systems perturbed by an external disturbance. Such safety guarantees are ensured, despite the disturbance, by online softening a subset of adjustable constraints defined by the designer. The selection of the constraints to be softened is made online based on a predefined priority assigned to each adjustable constraint. The design of a learning-based algorithm enables real-time computation while preserving the original safety properties.

Simulations results, obtained from an automated driving application, show that the proposed approach provides guarantees of collision-avoidance hard constraints despite the unpredicted behaviors of the surrounding environment.

Keywords: Autonomous vehicles, Automotive systems
Abstract: Pseudo-omnidirectional vehicles exhibit broad application in robotics, yet their inherent complex dynamic characteristics impose fundamental challenges on motion planning and control. In this work, we formulate the trajectory planning as a nonlinear model predictive control problem with bilinear constraints. This formulation effectively addresses multiple model-specific challenges induced by nonlinear kinematics, and features strong generality. To meet real-time computational requirements, we propose an efficient algorithm through structural analysis of the nonlinear optimization problem. The algorithm constructs a sequence of convex optimization subproblems to approximate solutions of the original problem, featuring both implementation simplicity and rapid convergence. Furthermore, the algorithm is proven to converge to stationary points of the original optimization problem. Numerical experiments validate the effectiveness and computational efficiency of our method.

Keywords: Autonomous vehicles, Constrained control, Predictive control for linear systems
Abstract: This paper presents a novel and integrated framework for motion planning and control in time-varying environments. The proposed method combines a time-parameterized convex-lifting-based path planning approach—including state space partitioning, graph generation, pathfinding, and safe corridor generation—with a spatiotemporal formulation to guide motion planning. To ensure dynamic feasibility, the framework is extended with a Model Predictive Control (MPC) scheme that computes safe trajectories within the identified corridors. A replanning strategy is introduced to adapt to the unpredictable behaviors of surrounding vehicles and maintain the feasibility of trajectories. Simulation results in an overtaking scenario demonstrate the effectiveness of the approach, with the ego vehicle successfully performing trajectory tracking and replanning while satisfying dynamic constraints.

Keywords: Constrained control, Autonomous systems, Autonomous vehicles
Abstract: This paper takes a step towards addressing the difficulty of constructing Control Barrier Functions (CBFs) for parallel safety boundaries. A single CBF for both boundaries has been reported to be difficult to validate for safety, and we identify why this challenge is inherent. To overcome this, the proposed method constructs separate CBFs for each boundary. We begin by presenting results for the relative degree one case and then extend these to higher relative degrees using the CBF backstepping technique, establishing conditions that guarantee safety. Finally, we showcase our method by applying it to a unicycle system, deriving a simple, verifiable condition to validate the target CBFs for direct implementation of our results.

Keywords: Output regulation, Autonomous vehicles
Abstract: This paper deals with clothoid tracking via the In ternal Model Principle (IMP). The ability to track these curves impacts efficiency, e.g., traffic flow, number of worked pieces in CNC machines, etc. To solve the clothoid trajectory tracking problem, this paper presents a new design method for IMP-based controllers. The theoretical arguments are corroborated via the simulation of a robotic walker.

Keywords: Autonomous vehicles, Agents-based systems, Lyapunov methods
Abstract: Unmanned aerial vehicles are now widely adopted in various applications, often requiring coexistence with other vehicles. This article proposes a repulsive vector field-based collision avoidance for quadrotors performing independent trajectory-tracking tasks in the same airspace. The trajectory tracking control is designed in the quadrotor’s attitude configuration space and utilizes the Immersion and Invariance technique to compensate for constant unknown disturbances; moreover, it operates with a collision avoidance strategy that is activated when a threshold distance between quadrotors is exceeded. The collision avoidance strategy is based on the direction of a three-dimensional repulsive vector field (RVF) constructed around each vehicle. The trajectory tracking and collision avoidance strategies are experimentally evaluated.

Keywords: Optimization, Cooperative control, Autonomous vehicles
Abstract: This paper presents a persistent control methodology for a sustainably powered host/agent network that executes tip-and-cue oceanographic observing operations within a spatiotemporally evolving environment. Specifically, a renewably powered host vessel simultaneously serves as a recharging platform for an autonomous aerial vehicle (AAV), while also performing broad surveillance of an evolving mission domain. When the AAV is on board the host vessel, the mission trajectory (termed the “nominal” trajectory) is selected based on a clarity-driven ergodic planner. When a location of interest (termed a “tip” location) is detected by the host vessel, the AAV is dispatched to provide detailed observation of that location. This necessitates a replanning operation (of the “rendezvous” trajectory) wherein a rendezvous point is selected to maximize the mutual long-horizon benefit to the host and agent. Because the mutually beneficial rendezvous point will, in general, deviate from the original ergodic trajectory, another replanning operation (of the nominal trajectory) is completed on rendezvous. In this paper, we demonstrate the efficacy of the combination of the ergodic trajectory planner and rendezvous planner for a solar-powered host vessel (the SeaTrac SP-48 ASV) and a quadrotor (Agilicious) agent vehicle. In particular, the combined control system is shown to significantly outperform a line-transect strategy and an ergodic controller wherein the rendezvous point is constrained to lie on the nominal mission trajectory.

Keywords: Formal Verification/Synthesis, Uncertain systems, Nonlinear systems
Abstract: A framework is presented for the verification of Signal Temporal Logic (STL) specifications over continuous-time nonlinear systems under uncertainty. Based on reachability analysis, the proposed method addresses indeterminate satisfaction caused by over-approximated reachable sets or incomplete simulations. STL semantics is extended via Boolean interval arithmetic, enabling the decomposition of satisfaction signals into unitary components with traceable uncertainty markers. These are propagated through the satisfaction tree, supporting precise identification even in nested formulas. To improve efficiency, only the reachable sets contributing to uncertainty are refined, identified through the associated markers. The framework allows online or offline monitoring to adapt to incremental system evolution while avoiding unnecessary recomputation. A case study on a nonlinear oscillator demonstrates a significant reduction in satisfaction ambiguity, highlighting the effectiveness of the approach.

Keywords: Stochastic systems, Statistical learning, Stability of linear systems
Abstract: We present a direct parametrization for continuous-time stochastic state-space models that ensures external stability via the stochastic bounded-real lemma. Our formulation facilitates the construction of probabilistic priors that enforce almost-sure stability, which are suitable for sampling-based Bayesian inference methods. We validate our work with a simulation example and demonstrate its ability to yield stable predictions with uncertainty quantification.

Keywords: Stochastic systems, Lyapunov methods, Optimization
Abstract: Stochastic Barrier Functions (SBFs) certify the safety of stochastic systems by formulating a functional optimization problem, which state-of-the-art methods solve using Sum-of Squares (SoS) polynomials. This work focuses on polynomial SBFs and introduces a new formulation based on Bernstein polynomials and provides a comparative analysis of its theoretical and empirical performance against SoS methods. We show that the Bernstein formulation leads to a linear program (LP), in contrast to the semi-definite program (SDP) required for SoS, and that its relaxations exhibit favorable theoretical convergence properties. However, our empirical results reveal that the Bernstein approach struggles to match SoS in practical performance, exposing an intriguing gap between theoretical advantages and real-world feasibility.

Keywords: Stochastic systems, Information theory and control, Estimation
Abstract: The causal conditional (CC) directed information (DI) can test the Granger causality for stochastic processes. However, for point process networks, the CCDI has only been developed for a trivariate network. In this work, we develop the general multivariate CCDI under Kramer’s framework by developing the CC likelihood, which is characterized by marginal intensity functions. We establish several Granger causality equivalences, unifying and generalizing existing results. Further, for Hawkes networks, we develop an estimation method of CCDI requiring only one point process trajectory, by developing an analytical formula for the marginal intensity functions, and an ergodic theorem for the CCDI.

Keywords: Queueing systems, Stochastic systems, Lyapunov methods
Abstract: We consider a polling system with two queues, where a single server is attending the queues in a cyclic order and requires non-zero switching times to switch between the queues. Our aim is to identify a fairly general and comprehensive class of Markovian switching policies that renders the system stable. Potentially a class of policies that can cover the Pareto frontier related to individual-queue-centric performance measures like the stationary expected number of waiting customers in each queue; for instance, such a class of policies is identified recently for a polling system near the fluid regime (with large arrival and departure rates), and we aim to include that class. We also aim to include a second class that facilitates switching between the queues at the instance the occupancy in the opposite queue crosses a threshold and when that in the visiting queue is below a threshold (this inclusion facilitates the design of `robust' polling systems). Towards this, we consider a class of two-phase switching policies, which include the above mentioned classes. In the maximum generality, our policies can be represented by eight parameters, while two parameters are sufficient to represent the aforementioned classes. We provide simple conditions to identify the sub-class of switching policies that ensure system stability. By numerically tuning the parameters of the proposed class, we illustrate that the proposed class can cover the Pareto frontier for the stationary expected number of customers in the two queues.

Keywords: Neural networks, Lyapunov methods, Stochastic systems
Abstract: Leveraging a stochastic extension of Zubov's equation, we develop a physics-informed neural network (PINN) approach for learning a neural Lyapunov function that captures the largest probabilistic region of attraction (ROA) for stochastic systems. We then provide sufficient conditions for the learned neural Lyapunov functions that can be readily verified by satisfiability modulo theories (SMT) solvers, enabling formal verification of both local stability analysis and probabilistic ROA estimates. By solving Zubov's equation for the maximal Lyapunov function, our method provides more accurate and larger probabilistic ROA estimates than traditional sum-of-squares (SOS) methods. Numerical experiments on nonlinear stochastic systems validate the effectiveness of our approach in training and verifying neural Lyapunov functions for probabilistic stability analysis and ROA estimates.

Keywords: Fault diagnosis, Fault detection, Stochastic systems
Abstract: We study fault identification in discrete-time nonlinear systems subject to additive Gaussian white noise. We introduce a Bayesian framework that explicitly accounts for unmodeled faults under reasonable assumptions. Our approach hinges on a new quantitative diagnosability definition, revealing when passive fault identification (FID) is fundamentally limited by the given control sequence. To overcome such limitations, we propose an active FID strategy that designs control inputs for better fault identification. Numerical studies on a two-water tank system and a Mars satellite with complex and discontinuous dynamics demonstrate that our method significantly reduces failure rates with shorter identification delays compared to purely passive techniques.

Keywords: Formal Verification/Synthesis, Stochastic systems, LMIs
Abstract: We study the problem of constructing controlled supermartingale functions to synthesize feedback laws that guarantee safety properties of stochastic differential equations (SDE) with control inputs. SDEs are widely used to model continuous time stochastic processes with applications ranging from financial markets to biology. In this paper, we extend classic notions from martingale theory for stochastic processes to prove that a given SDE will not exit a safe region over some finite time horizon with high probability. Our notion considers time-varying supermartingale functions that provide sharper probability bounds when compared to those that are time-independent. Furthermore, we study the controlled version of these supermartingales and the problem of synthesizing feedback control law that will maintain the state within a safe set with high probability over a given finite time horizon. We provide a projection-based algorithm for synthesizing polynomial, time-varying controlled supermartingales and corresponding feedback laws using sum-of-square (SOS) programming techniques. We implement our approach on some challenging numerical examples to demonstrate how it can synthesize control feedback laws that provide upper bounds on the probability of safety violations over a given time horizon.

Keywords: Algebraic/geometric methods, Stochastic systems, Numerical algorithms
Abstract: In this paper, we study the numerical simulation of stochastic differential equations (SDEs) on the special orthogonal Lie group SO(n). We propose a geometry-preserving numerical scheme based on the stochastic tangent space parametrization (S-TaSP) method for state-dependent multiplicative SDEs on SO(n). The convergence analysis of the S-TaSP scheme establishes a strong convergence order of O(delta^((1-epsilon)/2)), which matches the convergence order of the previous stochastic Lie Euler-Maruyama scheme while avoiding the computational cost of the exponential map. Numerical simulation illustrates the theoretical results.

Keywords: Predictive control for nonlinear systems, Robotics, Control applications
Abstract: This paper presents a control design approach for underactuated Euler-Lagrange (EL) systems that combines foundational ideas from passivity-based control and energy shaping with Model Predictive Control (MPC). By leveraging the intrinsic energy structure and passivity properties of EL systems, we construct a recursively feasible MPC. This hybrid design guarantees recursive feasibility by enforcing a terminal constraint on the shaped energy and exploiting the forward invariance of energy sub-level sets. We prove that a suitable terminal region always exists under mild system assumptions. The effectiveness of the approach is demonstrated through a case study on snake robot locomotion, highlighting how physical insight and energy-based reasoning can be used transparently throughout the control design process. We discuss the computation of terminal energy levels, and present simulation results validating the approach.

Keywords: Predictive control for nonlinear systems, Robust control, Biotechnology
Abstract: To match the growing demand for bio-methane production, anaerobic digesters need to embrace the co-digestion of different feedstocks; in addition, to improve the techno-economic performance, an optimal and time-varying adaptation of the input diet is required. These operation modes constitute a very hard challenge for the limited instrumentation and control equipment typically installed aboard full-scale plants. A model-based predictive approach may be able to handle such control problem, but the identification of reliable predictive models is limited by the low information content typical of the data available from full-scale plants’ operations, which entail high parametric uncertainty. In this work, the application of a tube-based robust nonlinear model predictive control (NMPC) is proposed to regulate bio-methane production over a period of diet change in time, while warranting safe operation and dealing with uncertainties. In view of its upcoming validation on a true small pilot-scale plant, the NMPC capabilities are assessed via numerical simulations designed to resemble as much as possible the experimental setup, along with some practical final considerations.

Keywords: Predictive control for nonlinear systems, Predictive control for linear systems, Machine learning
Abstract: In Model Predictive Control (MPC), the objective function plays a central role in determining the closed-loop behavior of the system, and must therefore be designed to achieve the desired closed-loop performance. However, in real-world scenarios, its design is often challenging, as it requires balancing complex trade-offs and accurately capturing a performance criterion that may not be easily quantifiable in terms of an objective function. This paper explores preference-based learning as a data-driven approach to constructing an objective function from human preferences over trajectory pairs. We formulate the learning problem as a machine learning classification task to learn a surrogate model that estimates the likelihood of a trajectory being preferred over another. The approach provides a surrogate model that can directly be used as an MPC objective function. Numerical results show that we can learn objective functions that provide closed-loop trajectories that align with the expressed human preferences.

Keywords: Predictive control for nonlinear systems, Optimal control
Abstract: In this work, we propose a Model Predictive Control (MPC) formulation incorporating two distinct horizons: a prediction horizon and a constraint horizon. This approach enables a deeper understanding of how constraints influence key system properties such as suboptimality, without compromising recursive feasibility and constraint satisfaction. In this direction, our contributions are twofold. First, we provide a framework to estimate closed-loop optimality as a function of the number of enforced constraints. This is a generalization of existing results by considering partial constraint enforcement over the prediction horizon. Second, when adopting this general framework under the lens of safety-critical applications, our method improves conventional Control Barrier Function (CBF) based approaches. It mitigates myopic behaviour in Quadratic Programming (QP)-CBF schemes, and resolves compatibility issues between Control Lyapunov Function (CLF) and CBF constraints via the prediction horizon used in the optimization. We show the efficacy of the method via numerical simulations for a safety critical application.

Keywords: Predictive control for nonlinear systems, Numerical algorithms
Abstract: This paper presents a framework for safe navigation in cluttered, unknown environments. The control layer employs a new Nonlinear Model Predictive Control (NMPC) framework based on K-invariant sets, which guarantees that the trajectories of nonlinear systems satisfy safety constraints despite the unknown environment. The recursive feasibility is guaranteed by ensuring the existence of a backup trajectory at each sampling instance. The backup trajectories return to K-invariant sets, which are designed offline but can be parameterized online by leveraging the system's translation and rotation invariance. The planning layer uses an improved convex lifting method, employing the limited information provided by the sensors to find a path. This method is both efficient and exhibits versatile exploration features. It generates an interconnected graph at each computation step, based on which the shortest path to a goal can be extracted, or alternatively, a shortest path to an unexplored region can be identified.

Keywords: Predictive control for nonlinear systems, Robotics, Reinforcement learning
Abstract: In formation control for unmanned aerial vehicles (UAVs), a fleet of drones is arranged in a predefined geometric configuration that must be maintained throughout the flight, while avoiding collisions with other drones and obstacles. In real-world applications, the need for quick deployment of UAV fleets often makes controller parameter tuning a significant challenge. In this paper, we introduce an end-to-end formation controller based on Nonlinear Model Predictive Control (NMPC), enhanced by a reinforcement learning algorithm for optimal hyperparameter tuning. Specifically, we adapt the Policy Gradient with Parameter-based Exploration (PGPE) algorithm to the formation control context. This method offers a fast and scalable solution for parameter tuning that does not require a differentiable controller and can be customized to the specific needs of the deployer. To validate our approach, we conduct simulation experiments using a realistic quadrotor model in a three-dimensional environment with static obstacles. Our results demonstrate the effectiveness and advantages of our method in comparison to state-of-the-art algorithms.

Keywords: Data driven control, Nonlinear systems, Predictive control for nonlinear systems
Abstract: This paper extends the Willems' Fundamental Lemma to nonlinear control-affine systems using the Koopman bilinear realization. This enables us to bypass the Extended Dynamic Mode Decomposition (EDMD)-based system identification step in conventional Koopman-based methods and design controllers for nonlinear systems directly from data. Leveraging this result, we develop a Data-Enabled Predictive Control framework for nonlinear systems with unknown dynamics. A case study demonstrates that our direct data-driven control method achieves improved optimality compared to conventional Koopman-based methods. Furthermore, in examples where an exact Koopman realization with a finite-dimensional lifting function set of the controlled nonlinear system does not exist, our method exhibits advanced robustness to finite Koopman approximation errors compared to existing methods.

Keywords: Emerging control applications, Energy systems, Predictive control for linear systems
Abstract: Control of the density profile based on pellet fueling for the ITER nuclear fusion tokamak involves a multi-rate nonlinear system with safety-critical constraints, input delays, and discrete actuators with parametric uncertainty. To address this challenging problem, we propose a multi-stage MPC (msMPC) approach to handle uncertainty in the presence of mixed-integer inputs. While the scenario tree of msMPC accounts for uncertainty, it also adds complexity to an already computationally intensive mixed-integer MPC (MI-MPC) problem. To achieve real-time density profile controller with discrete pellets and uncertainty handling, we systematically reduce the problem complexity by (1) reducing the identified prediction model size through dynamic mode decomposition with control, (2) applying principal component analysis to reduce the number of scenarios needed to capture the parametric uncertainty in msMPC, and (3) utilizing the penalty term homotopy for MPC (PTH-MPC) algorithm to reduce the computational burden caused by the presence of mixed-integer inputs. We compare the performance and safety of the msMPC strategy against a nominal MI-MPC in plant simulations, demonstrating the first predictive density control strategy with uncertainty handling, viable for real-time pellet fueling in ITER.

Keywords: Nonlinear systems, Hybrid systems, Stability of nonlinear systems
Abstract: This paper develops methods for studying contraction properties of constrained differential inclusions (CDIs) in finite-dimensional settings using non-Euclidean norms and compatible regular pairings. We introduce uniform and non-uniform pre-contraction concepts for CDIs, generalize one-sided Lipschitz conditions for set-valued maps relative to constraint sets, and prove that CDIs satisfying these conditions exhibit suitable contraction properties. The analysis employs jointly continuous regular pairings and constructs surrogate time-varying measurable set-valued maps that, for any given solution to the CDI, select values of the flow-map compatible with the one-sided Lipschitz inequality relative to that solution. This construction guarantees that for each solution there exists a contractive counterpart.

Keywords: Stability of nonlinear systems, Nonlinear systems, LMIs
Abstract: Scaled relative graphs offer a graphical tool for analysis of nonlinear feedback systems. Of specific interest for stability analysis is the scaled graph, a special case of the scaled relative graph, related to non-incremental system properties. Although recently substantial progress has been made in scaled graph analysis, at present their use in multivariable feedback systems is limited by conservatism. In this paper, we aim to reduce this conservatism by introducing multipliers and exploit system structure in the analysis with scaled graphs. In particular, we use weighted inner products to arrive at a weighted scaled graph and combine this with a commutation property to formulate a stability result for multivariable feedback systems. We present a method for computing the weighted scaled graph of Lur'e systems based on solving sets of linear matrix inequalities, and demonstrate a significant reduction in conservatism through an example.

Keywords: Stability of nonlinear systems, Lyapunov methods, Algebraic/geometric methods
Abstract: Since the mid-1990s, State-Dependent Riccati Equation (SDRE) strategies have gained prominence as systematic and effective methodologies for designing nonlinear controllers, observers, and filters. These approaches address many of the limitations inherent in traditional techniques while offering computationally efficient algorithms that have demonstrated remarkable success across a wide range of practical and impactful applications. In this paper, we leverage the State-Dependent-Coefficient (SDC) parametrization to establish a novel global asymptotic (and exponential) stability result for discrete-time nonlinear systems with a controllable SDC representation. The proof is based on geometrical tools and spectral decompositions. Numerical simulations validate the results.

Keywords: Model/Controller reduction, Reduced order modeling, Stability of nonlinear systems
Abstract: Recurrent equilibrium networks (RENs) are effective for learning the dynamics of complex dynamical systems with certified contraction and robustness properties through unconstrained learning. While this opens the door to learning large-scale RENs, deploying such large-scale RENs in real-time applications on resource-limited devices remains challenging. Since a REN consists of a feedback interconnection of linear time-invariant (LTI) dynamics and static activation functions, this article proposes a projection-based approach to reduce the state dimension of the LTI component of a trained REN. One of the two projection matrices is dedicated to preserving contraction and robustness by leveraging the already-learned REN contraction certificate. The other projection matrix is iteratively updated to improve the accuracy of the reduced-order REN based on necessary h2-optimality conditions for LTI model reduction. Numerical examples validate the approach, demonstrating significant state dimension reduction with limited accuracy loss while preserving contraction and robustness.

Keywords: Linear parameter-varying systems, Stability of nonlinear systems, Observers for nonlinear systems
Abstract: This paper investigates the observer-based stabilization problem for a class of disturbance-affected Nonlinear Parameter-Varying (NLPV) systems. We introduce a novel observer-controller framework that leverages Linear Matrix Inequality (LMI) conditions to guarantee robust performance. Unlike previous works that focus solely on state estimation, our approach integrates a controller based on the estimated states to ensure asymptotic stability and optimal disturbance attenuation. Through the deployment of the parameter-dependent Lyapunov (PDL) function and mathcal{H}_infty criterion, a new LMI condition is synthesized to achieve the objective. Further, due to the judicious use of a variant of Young's inequality, the reformulated Lipschitz property, and the matrix multiplier method presented in our previous works, the established LMI encompasses extra degrees of freedom from a feasibility point of view. The performance of the method is validated through simulations on a nonlinear time-varying pendulum system, demonstrating enhanced noise attenuation and faster convergence compared to existing approaches.

Keywords: Stability of nonlinear systems, Markov processes, Delay systems
Abstract: In this paper, we utilize monotonicity to simplify contraction analysis of discrete-time nonlinear time-delay systems with parameters following stochastic processes. First, we extend the concept of almost sure monotonicity to the time-delay systems, which can be verified via analysis of the corresponding prolonged systems. Next, we introduce a novel notion of uniform incremental asymptotic stability (UIAS) in the first moment to the time-delay systems and develop its sufficient condition. By virtue of almost sure monotonicity, if this condition holds, the time-delay systems are UIAS in the first moment for any finite length of delay. Moreover, the proposed result suggests that UIAS in the first moment has a strong connection with stability of the averaged deterministic systems. We formalize this observation for almost surely cooperative systems by establishing the equivalence between UIAS and uniform incremental asymptotic mean stability.

Keywords: LMIs, Stability of nonlinear systems, Fuzzy systems
Abstract: This letter investigates the stability analysis of discrete-time Takagi-Sugeno fuzzy systems by incorporating the dynamics of membership functions (MFs) into an augmented state vector. Using a polytopic representation for the MFs, a novel Lyapunov function that enables the derivation of less conservative linear matrix inequality-based stability conditions is proposed. Unlike existing approaches, the proposed method eliminates the need for predefined MF variation bounds, leveraging their inherent dynamics to refine stability analysis and domain of attraction (DOA) estimation. Numerical examples demonstrate the effectiveness of the proposed approach in reducing conservatism and enlarging the estimated DOA compared to existing methods.

Keywords: LMIs
Abstract: This paper proposes a novel incremental input-to-state stability condition for a discrete-time leaky-integrator echo state network. The derived condition is further utilized for control design through Linear Matrix Inequalities (LMIs). The corresponding observer design LMI condition is also derived. A numerical simulation showcases the effectiveness of the proposed approach.

Keywords: Stability of hybrid systems, Lyapunov methods, Hybrid systems
Abstract: We study the property of fixed-time stability (FxTS) for hybrid dynamical systems (HDS) that combine continuous-time and discrete-time dynamics, possibly modeled via set-valued maps. For such systems, we provide sufficient Lyapunov characterizations for certifying FxTS of general closed sets. In this work, FxTS of a closed set is defined to require two properties: Lyapunov stability and fixed time convergence. For fixed time convergence, the system's trajectories must converge to the set of interest within a finite hybrid time that is uniformly bounded over all initializations. Our main contributions are two-fold: we first show that if the HDS admits solutions that flow for sufficiently long time, as well as a suitable Lyapunov function with a sufficient ``fixed-time'' decrease along flows and nonincrease during jumps, then the HDS will exhibit FxTS. Afterwards, we extend this result by showing that, under a mild dwell time condition, FxTS can be established for HDS for which the Lyapunov function is allowed to mildly increase during jumps. While our theoretical results pave the way for the general analysis of a broad class of fixed-time stable hybrid algorithms and systems, we illustrate them using two representative cases: a hybrid system with dwell-time conditions on the jumps, and systems exhibiting arbitrarily fast switching between a finite number of fixed-time stable vector fields.

Keywords: Power electronics, Algebraic/geometric methods, Hybrid systems
Abstract: Optimal pulse patterns (OPPs) are a modulation method in which the switching angles and levels of a switching signal are computed via an offline optimization procedure to minimize a performance metric, typically the harmonic distortions of the load current. Additional constraints can be incorporated into the optimization problem to achieve secondary objectives, such as the limitation of specific harmonics or the reduction of power converter losses. The resulting optimization problem, however, is highly nonconvex, featuring a trigonometric objective function and constraints as well as both real- and integer-valued optimization variables. This work casts the task of OPP synthesis for a multilevel inverter as an optimal control problem of a hybrid system. This problem is in turn lifted into a convex but infinite-dimensional conic program of occupation measures using established methods in convex relaxations of optimal control. Lower bounds on the minimum achievable harmonic distortion are acquired by solving a sequence of semidefinite programs via the moment-sum-of-squares hierarchy, where each semidefinite program scales in a jointly linear manner with the numbers of permitted switching transitions and inverter voltage levels.

Keywords: Hybrid systems, Lyapunov methods, Stability of hybrid systems
Abstract: This paper studies the problem of control design methods to stabilize a hybrid system under disturbances as an inverse-optimality problem. Sufficient conditions are given to guarantee input-to-state stability of a hybrid system with disturbances only. Next, it is shown that a hybrid system, with inputs and disturbances, can be rendered input-to-state controlled stable under the existence of a control Lyapunov function (CLF) using pointwise minimum norm (min-norm) stabilizing feedback laws. Finally, it is proven that every CLF is a meaningful value function for a two-player zero-sum hybrid game in the context of stability, and that every pointwise min-norm stabilizing control feedback law is optimal for such a game with meaningful cost functionals. The main results are illustrated in a one-degree-of-freedom juggling system.

Keywords: Maritime control, Hybrid systems, Lyapunov methods
Abstract: In this paper we propose discontinuous control laws to globally stabilize a target position for a hovercraft. Equations of motion of the hovercraft are derived through a kinematic model approximation of a dynamic model taken from the literature. Discontinuous feedback laws for the kinematic and the dynamic model are derived and analyzed using a hybrid systems formulation of the closed-loop dynamics. Numerical simulations confirm the theoretical results and validate the kinematic model as a accurate simplification of the dynamic model.

Keywords: Robotics, Stability of hybrid systems, Algebraic/geometric methods
Abstract: We enhance a recently proposed hybrid controller that asymptotically stabilizes a class of closed orbits (so-called oscillations) for mechanical control systems with underactuation degree one. The controller in question enforces virtual holonomic constraint (VHCs) within a parametric family and instantiates new VHCs at certain events so as to asymptotically stabilize the target orbit. In this paper we propose a novel feedback mechanism in the jump map of the hybrid controller ensuring the asymptotic stabilization of oscillations with guaranteed basin of attraction. We demonstrate the new controller with a four degrees-of-freedom robot mimicking a child on a swing. For this robot, we are able to stabilize a large range of oscillations with guaranteed basin of attraction.

Keywords: Stability of hybrid systems, Algebraic/geometric methods
Abstract: This paper develops the hybrid Lyapunov theorem for geometric hybrid dynamical systems, namely hybrid inclusions evolving on intrinsic C1-manifolds. We present various nonsmooth analysis notions, which aid in the formulation of sufficient conditions for existence of solutions. We also present topological definitions of uniform global stability, attractivity, and asymptotic stability of a nonempty, compact set A. Finally, the Lyapunov theorem provides relaxed conditions for uniform global asymptotic stability of the set A. An example illustrates the concepts and results.

Keywords: Stability of hybrid systems, Algebraic/geometric methods
Abstract: This paper establishes a hybrid invariance principle for geometric hybrid dynamical systems, modeled as hybrid inclusions, evolving on intrinsic C1-manifolds. Building on previous results on Lyapunov stability for hybrid systems presented in the companion paper [1], we introduce the theoretical tools required for invariance-based stability analysis. These include set-convergence concepts for manifolds, characterization of nominal well-posedness, and analysis of ω-limit set properties for precompact solutions. Our variational analysis approach preserves the intrinsic geometric structure of the underlying space while handling set-valued dynamics without requiring uniqueness of solutions. This theory particularly addresses challenges in disconnected manifolds and quotient spaces.

Keywords: Hybrid systems, LMIs, Sampled-data control
Abstract: This paper focuses on observer-based control for Lur’e type systems with slope bounded nonlinearities under aperiodic sampling. In this setting, we design a hybrid observer that provides an estimate of the state based only on aperiodic samples of the plant output. The estimate provided by the observer is used in a sampled-data feedback control architecture. By introducing a timer state, the closed-loop system is modeled as a hybrid dynamical system. By using a quadratic timer-dependent Lyapunov function, we provide sufficient conditions in the form of timer-dependent inequalities ensuring closed-loop asymptotic stability. By relying on an affine dependence on the timer variable, these conditions are turned into a finite collection of linear matrix inequalities that can be used to design the observer gain. A numerical example illustrates the application of the proposed results.

Keywords: Optimal control, Numerical algorithms, Predictive control for nonlinear systems
Abstract: Direct collocation (DC) is a widely used method for solving dynamic optimization problems (DOPs), but its implementation simplicity and computational efficiency are limited for challenging problems. For DOPs involving singular arcs, DC solutions often exhibit significant fluctuations along the singular arc, accompanied by large residual errors between collocation points, where the dynamic constraints are enforced as equality constraints. In this paper, we introduce the direct transcription method of integrated residual regularized direct collocation (IRRDC). This approach enforces dynamic constraints using a combination of point-wise residual constraints (expressed as either equalities or inequalities) and a penalty term on the integrated residual error, which helps reduce errors between collocation points. IRRDC retains the implementation simplicity of DC while improving both solution accuracy and efficiency, particularly for challenging problem types. Through the examples, we demonstrate that for problems where traditional DC results in excessive fluctuations, IRRDC effectively suppresses fluctuations and yields solutions with greater accuracy - at least two orders of magnitude lower in various error measures in relation to the dynamic and path constraints.

Keywords: Optimal control, Constrained control, Systems biology
Abstract: In this letter, we study optimal vaccination in a heterogeneous metapopulation epidemic model with state and mixed control-state constraints and a linear cost on the control, which better reflects real‐world expenses compared to quadratic penalties. Within a general SIR / SEIR K-patch framework, we formulate the resulting control problem on a finite horizon, prove the existence of optimal control, and derive the necessary conditions via Pontryagin's maximum principle extended to handle both shipment and per-region capacity limits. We use Pontryagin's result to rule out the existence of singular arcs and obtain a complete characterization of the optimal policy as bang-bang, with at most one switching time per region per week. Numerical experiments on networks of 3 to 8 cities confirm that the predicted switching structure arises from the linear cost assumption, and we compare these findings with optimal quadratic-cost policies.

Keywords: Optimal control, Numerical algorithms
Abstract: An adversarial setting is considered where two agents influence in turn a dynamical system using continuous, scalar actions. One agent seeks to maximize an infinite-horizon value, while the other aims to minimize it, leading to a sequential noncooperative game. To search for the minimax-optimal solution, we introduce Minimax Optimistic Planning for Continuous actions (MOPC), an algorithm that iteratively refines the most promising regions of the space of action sequences. Our analysis shows that MOPC provides computable bounds on the solution quality. Furthermore, we prove these bounds converge at well-characterized rates as computation grows, where the convergence rates are driven by a measure of problem complexity. We show how to pose in our framework -- and thereby near-optimally solve -- a class of problems in which opinions in a social network are influenced by two competing marketers. In experiments on such problems, MOPC outperforms uniform allocation over time.

Keywords: Optimal control, Neural networks, Embedded systems
Abstract: Real-time optimal control remains a fundamental challenge in robotics, especially for nonlinear systems with stringent performance requirements. As one of the representative trajectory optimization algorithms, the iterative Linear Quadratic Regulator (iLQR) faces limitations due to their inherently sequential computational nature, which restricts the efficiency and applicability of real-time control for robotic systems. While existing parallel implementations aim to overcome the above limitations, they typically demand additional computational iterations and high-performance hardware, leading to only modest practical improvements. In this paper, we introduce Quattro, a transformer-accelerated iLQR framework employing an algorithm-hardware co-design strategy to predict intermediate feedback and feedforward matrices. It facilitates effective parallel computations on resource-constrained devices without sacrificing accuracy. Experiments on cart-pole and quadrotor systems show an algorithm-level acceleration of up to 5.3x and 27x per iteration, respectively. When integrated into a Model Predictive Control (MPC) framework, Quattro achieves overall speedups of 2.8x for the cart-pole and 17.8x for the quadrotor compared to the one that applies traditional iLQR. Transformer inference is deployed on FPGA to maximize performance, achieving further up to 20.8x speedup over prevalent embedded CPUs with over 11x power reduction than GPU and low hardware resource overhead.

Keywords: Hybrid systems, Stability of nonlinear systems, Optimal control
Abstract: This work presents an innovative robust attitude tracking controller for fully actuated rigid bodies. The approach relies on the modified Rodrigues parameters (MRP) description, which is a double covering of the three-dimensional rotation group SO(3), and the feedback linearization technique to formulate a Lyapunov-based quadratic programming problem. The resulting optimal controller ensures the exponential decay of the attitude tracking errors with minimum control effort and operates within a hybrid system structure to overcome the topological obstructions of the rotation group, rendering the attitude tracking dynamics globally exponentially stable on the MRP space. By combining this MRP-based optimal controller with a hybrid dynamic path-lifting mechanism, this novel approach yields an equivalent tracking result on SO(3). Furthermore, the resulting closed-loop system is nominally robust to perturbations, including external disturbances, parameter variations, or measurement noise. Numerical simulations demonstrate the performance and efficiency of the optimal hybrid strategy.

Keywords: Constrained control, Stability of nonlinear systems, Optimal control
Abstract: Safe predefined-time stability characterizes parameter-dependent nonlinear dynamical systems whose trajectories starting in a given set of admissible states remain in the set of admissible states and converge to an equilibrium point in a predefined time. In this paper, we develop a game-theoretic control framework to address a robust optimal safe predefined-time stabilization problem for parameter-dependent nonlinear affine dynamical systems subject to an adversary with nonquadratic cost functionals. In particular, the robust optimal safe predefined-time stabilization problem is formulated as a two-player zero-sum differential game, wherein the controller is a minimizing player and the adversary is a maximizing player. Sufficient conditions for the existence of a saddle-point solution to the zero-sum game and closed-loop system safe predefined-time stability are derived. Specifically, safe predefined-time stability of the closed-loop system is guaranteed via a barrier Lyapunov function satisfying a differential inequality while serving as a solution to the steady-state Hamilton-Jacobi-Isaacs equation ensuring Nash equilibrium. Finally, an illustrative numerical example is provided to demonstrate the efficacy of the developed framework.

Keywords: Formal Verification/Synthesis, Optimal control, Hybrid systems
Abstract: As control systems grow in complexity, abstraction-based methods have become essential for designing controllers with formal guarantees. However, a key limitation of these methods is their reliance on discrete-time models, typically obtained by discretizing continuous-time systems with a fixed timestep. This discretization leads to two major problems: when the timestep is small, the abstraction includes numerous stuttering and spurious trajectories, making controller synthesis suboptimal or even infeasible; conversely, a large time step may also render control design infeasible due to a lack of flexibility. In this work, drawing inspiration from Reinforcement Learning concepts, we introduce temporal abstractions, which allow for a flexible timestep. We provide a method for constructing such abstractions and formally establish their correctness in controller design. Furthermore we show how to apply these to optimal control under reachability specifications. Finally we showcase our methods on two numerical examples, highlighting that our approach leads to controllers that achieve a lower worst-case control cost.

Keywords: Hybrid systems, LMIs, Formal Verification/Synthesis
Abstract: We propose an approach to trajectory optimization for piecewise polynomial systems based on the recently proposed graphs of convex sets framework. We instantiate the framework with a convex formulation of optimal control based on occupation measures, resulting in a convex relaxation resembling the discrete shortest-paths linear program that can be solved efficiently to global optimality. This approach improves scalability to large numbers of discrete modes compared to the NP-hard mixed- integer formulation. We use this to plan trajectories under linear temporal logic specifications, comparing the computed cost lower bound to a nonconvex optimization approach with fixed mode sequence. In our numerical experiments, we find that this bound is typically in the vicinity of the nonconvex solution. While the method inherits the limitations of semidefinite programming, the runtime speedup is significant compared to the often intractable mixed-integer formulation. Our implementation is available at https://github.com/ebuehrle/hpoc.

Keywords: Cooperative control, Reinforcement learning, Resilient Control Systems
Abstract: This tutorial paper provides an overview of distributed multi-armed bandit problems in networks of multiple agents. Each agent repeatedly selects an action from a fixed set of choices and receives a random reward, aiming to balance exploration and exploitation. The agents are allowed to communicate only with their neighbors, where the neighbor relations are described by a possibly time-varying graph. Two main settings are considered. Two main settings are discussed. In the first setting, agents observe identical reward distributions for each action. We present two distributed algorithms based on the classical UCB and KL-UCB methods. It is shown that each agent can achieve a lower logarithmic regret compared to the standard single-agent case, as long as the agent has at least one neighbor and the communication graph is strongly connected. The improvement in regret is related to the size of the agent's local neighborhood and the structure of the network. These algorithms can be further modified to be fully resilient to adversarial agents who may inject untrustworthy information, using communication redundancy. In the second setting, agents observe different reward distributions for the same actions. The goal is to minimize cumulative expected regret with respect to the true rewards, defined as the average of all agents' mean rewards. A distributed algorithm is introduced that guarantees optimal regret performance for each agent when the network remains jointly connected over time. All proposed algorithms operate in a fully distributed manner and do not require any global knowledge of the network. This tutorial highlights the theoretical foundations, algorithmic designs, and resilience properties of distributed bandit algorithms, with an emphasis on performance guarantees under limited communication and local observations. Open problems and possible future directions are also discussed.

Keywords: Quantum information and control, Optimization algorithms, Stochastic optimal control
Abstract: This paper presents a gradient-based optimization approach to solve optimal control problems in linear quantum systems. Utilizing an algebraic Lyapunov equation, we derive the partial derivatives of the cost function with respect to the steady-state covariance matrix. Furthermore, we formulate the gradient for the cost function when applied to a coherent quantum Linear Quadratic Gaussian (LQG) control problem. Through numerical simulations, we verify that our method effectively solves the LQG control problem.

Keywords: Quantum information and control, Robust control, Uncertain systems
Abstract: Robust control design in open quantum systems under parametric uncertainty is essential for practical quantum applications. In this work, we propose an efficient algorithm for identifying robust control solutions in high-dimensional quantum systems. The algorithm reduces the computational complexity by utilizing the Suzuki-Trotter expansion to accelerate the solution of system dynamics. Our simulation results demonstrate that the proposed algorithm can achieve control solutions of high fidelity and robustness with improved efficiency.

Keywords: Quantum information and control
Abstract: The estimation of all the parameters in an unknown quantum state or measurement device, known as quantum state tomography (QST) and quantum detector tomography (QDT), is fundamental to the characterization and control of quantum systems. This paper presents a unified state space framework for QST and QDT based on observable measurement results at different evolution moments, applicable to both closed and Markovian open quantum systems. We derive lower bounds on the number of sampling points required for a unique estimation and analyze the computational complexity and mean squared error (MSE) scaling. As the estimator may yield unphysical results, we introduce new correction techniques to enforce physicality, which may also improve the infidelity scaling. The effectiveness of the proposed methods is validated through numerical simulations.

Keywords: Quantum information and control
Abstract: Information control and online estimation are studied in this paper, focusing on unital quantum systems. To infer the value of an unknown parameter characterizing a quantum state, control is applied to the quantum system and then a measurement is taken. To optimize the estimation procedure, control is selected with the aim of maximizing the Fisher information, about the unknown parameter, extracted from the measurement. We introduce the notion of ``controlled Fisher information (CFI)’’ to describe the maximum Fisher information attainable by control. A method for computing the CFI is designed and an analytic upper bound on the CFI is derived. Necessary and sufficient conditions for the admissibility (i.e., parameter independence) of controls achieving the CFI are given. For scenarios in which the admissibility conditions are not met, we devise an online joint estimation and control algorithm that, with high probability, achieves the CFI after a sufficient number of measurements.

Keywords: Quantum information and control, Markov processes
Abstract: In this paper, we study the ergodic theorem for infinite-dimensional quantum Markov semigroups, originally introduced by Frigerio and Verri in 1982, and its latest version developed by Carbone and Girotti in 2021. We provide a sufficient condition that ensures exponential convergence towards the positive recurrent subspace, a well-known result for irreducible quantum Markov semigroups in finite-dimensional Hilbert spaces. Several illustrative examples are presented to demonstrate the application of the ergodic theorem. Moreover, we show that the positive recurrent subspace plays a crucial role in the study of global asymptotic stability.

Keywords: Quantum information and control, Emerging control applications
Abstract: We develop a method for controlling the ensemble states of a two-level quantum system on the Bloch sphere. A spectrum of resonance frequencies parametrizes the ensemble. The evolution of the ensemble states under externally applied controls is uniquely determined by their respective resonance frequencies. Therefore, the control fields must be designed to simultaneously evolve the ensemble states across the domain of resonance frequencies. The objective is to evolve the discretized quantum ensemble states from an arbitrary initial dispersion to a targeted final configuration on the Bloch sphere. The proposed method holds potential for broad applicability in areas such as nuclear magnetic resonance (NMR), electron spin resonance (ESR), quantum computing and information processing, sampling-based learning control (SLC), quantum sensing and metrology, and quantum error correction.

Keywords: Quantum information and control, Markov processes, Lyapunov methods
Abstract: In this paper, we present several refinements of the results in [1] concerning feedback stabilization of invariant subspaces of quantum trajectories. First, we consider a more realistic dynamics that accounts for potential measurement imperfections. Second, the strong controllability assumption can be replaced by a simpler and more natural condition whenever the channel is primitive on its minimal invariant subspaces. Finally, we show that the control strategy can be based on the estimated trajectory (starting from an estimated initial state) while still ensuring exponential stabilization towards the target subspace at the same rate, thereby demonstrating the robustness of the approach.

Keywords: Quantum information and control
Abstract: Quantum metrology utilizes quantum resources to enhance the precision of parameter estimation. A central objective is to maximize the Quantum Fisher information (QFI), which determines the ultimate estimation precision based on the Cram´er-Rao bound. In this work, we formulate QFI optimization as an optimal control problem and apply Pontryagin's Minimum Principle (PMP) to derive the optimal control protocol. We show that when the control amplitude is constrained below a certain threshold, the optimal solution reduces to a constant control. This strategy enables the QFI to scale quadratically with the total evolution time T, thereby improving estimation precision with longer probe durations.

Keywords: Data driven control, Nonlinear systems identification, Robust control
Abstract: We derive novel deterministic bounds on the approximation error of data-based bilinear surrogate models for unknown nonlinear systems. The surrogate models are constructed using kernel-based extended dynamic mode decomposition to approximate the Koopman operator in a reproducing kernel Hilbert space. Unlike previous methods that require restrictive assumptions on the invariance of the dictionary, our approach leverages kernel-based dictionaries that allow us to control the projection error via pointwise error bounds, overcoming a significant limitation of existing theoretical guarantees. The derived state- and input-dependent error bounds allow for direct integration into Koopman-based robust controller designs with closed-loop guarantees for the unknown nonlinear system. Numerical examples illustrate the effectiveness of the proposed framework.

Keywords: Data driven control, Cooperative control, Output regulation
Abstract: Given an autonomous exosystem, generating a reference output, we consider the problem of designing a distributed data-driven control law for a family of discrete time heterogeneous LTI agents, connected through a directed graph, in order to synchronize the agents’ outputs to the reference one. Agents split into two categories: leaders, with direct access to the exosystem output, and followers, that only receive information from their neighbors. All agents aim to achieve output synchronization by means of a state feedback that makes use of their own states as well as of an estimate of the exogenous system state, provided by an internal state observer. Such an observer has a different structure for leaders and followers. Necessary and sufficient conditions for the existence of a solution are first derived in the model-based set-up and then in a data-driven context.

Keywords: Stability of nonlinear systems, Data driven control, Formal Verification/Synthesis
Abstract: Incremental input-to-state stability (delta-ISS) offers a robust framework to ensure that small input variations result in proportionally minor deviations in the state of a nonlinear system. This property is essential in practical applications where input precision cannot be guaranteed. However, analyzing delta-ISS demands precise knowledge of system dynamics to assess the state's incremental response to input changes, posing a challenge in real-world scenarios where mathematical models are unknown. In this work, we develop a data-driven approach to design delta-ISS Lyapunov functions together with their corresponding delta-ISS controllers for continuous-time input-affine nonlinear systems with polynomial dynamics, ensuring the delta-ISS property is achieved without requiring knowledge of the system dynamics. In our data-driven scheme, we collect only two sets of input-state trajectories from sufficiently excited dynamics. By fulfilling a specific rank condition, we design delta-ISS controllers using the collected samples through formulating a sum-of-squares optimization program. The effectiveness of our data-driven approach is evidenced by its application to a physical case study.

Keywords: Formal Verification/Synthesis, Neural networks, Lyapunov methods
Abstract: In this paper, we introduce a novel framework for simultaneously learning and verifying certificates for uncertain nonlinear control affine systems, utilizing robust control Lyapunov barrier functions (rCLBF) to jointly certify stability and safety. To ensure dense verification of Lyapunov (barrier) conditions in continuous state space, we exploit the special structure of the derivative condition of rCLBF to derive a novel specification that could be densely verified with neural network (NN) verifiers, endowing such certificates with formal guarantees. We first over-approximate the verification domain for the derivative condition and then use counter-example guided abstraction refinement (CEGAR) to eliminate spurious counterexamples (false-negatives) generated by the NN verifiers to prevent exponential blow-up of the verification problem and improve the efficiency of the verification process. We demonstrate the efficacy of our framework in generating certificates with formal guarantees on challenging benchmarks.

Keywords: Data driven control, Formal Verification/Synthesis, Stochastic systems
Abstract: We study the automated abstraction-based synthesis of correct-by-construction control policies for stochastic dynamical systems with unknown dynamics. Our approach is to learn an abstraction from sampled data, which is represented in the form of a finite Markov decision process (MDP). In this paper, we present a data-driven technique for constructing finite-state interval MDP (IMDP) abstractions of stochastic systems with unknown nonlinear dynamics. As a distinguishing and novel feature, our technique only requires (1) noisy state-input-state observations and (2) an upper bound on the system's Lipschitz constant. Combined with standard model-checking techniques, our IMDP abstractions enable the synthesis of policies that satisfy probabilistic temporal properties (such as "reach-while-avoid") with a predefined confidence. Our experimental results show the effectiveness and robustness of our approach.

Keywords: Data driven control, Predictive control for nonlinear systems, Stability of nonlinear systems
Abstract: For nonlinear (control) systems, extended dynamic mode decomposition (EDMD) is a popular method to obtain data-driven surrogate models. Its theoretical foundation is the Koopman framework, in which one propagates observable functions of the state to obtain a linear representation in an infinite-dimensional space. In this work, we prove practical asymptotic stability of a (controlled) equilibrium for EDMD-based model predictive control, in which the optimization step is conducted using the data-based surrogate model. To this end, we derive novel bounds on the estimation error that are proportional to the norm of state and control. This enables us to show that, if the underlying system is cost controllable, this stabilizablility property is preserved. We conduct numerical simulations illustrating the proven practical asymptotic stability.

Keywords: Feedback linearization, Data driven control, Predictive control for nonlinear systems
Abstract: This article contributes a theoretical framework for data-driven feedback linearization of nonlinear control-affine systems. We unify the traditional geometric perspective on feedback linearization with an operator-theoretic perspective involving the Koopman operator. We first show that if the distribution of the control vector field and its repeated Lie brackets with the drift vector field is involutive, then there exists an output and a feedback control law for which the Koopman generator is finite-dimensional and locally nilpotent. We use this connection to propose a data-driven algorithm ‘Koopman generator-based feedback linearization (KGFL)’ for feedback linearization of single-input systems. Particularly, we use experimental data to identify the state transformation and control feedback from a dictionary of functions for which feedback linearization is achieved in a least-squares sense. We also propose a single-step data-driven formula which can be used to compute the linearizing transformations. When the system is feedback linearizable and the chosen dictionary is complete, our data-driven algorithm provides the same solution as model-based feedback linearization. Finally, we provide numerical examples for the data-driven algorithm and compare it with model-based feedback linearization. We also numerically study the effect of the richness of the dictionary and the size of the dataset on the effectiveness of feedback linearization.

Keywords: Compartmental and Positive systems, Data driven control, Networked control systems
Abstract: This paper proposes a localized data-driven distributed controller design method for positive stabilization of network systems. In particular, we consider a network system consisting of multiple unknown agents, each of which has a local controller to be designed. In addition, we assume that each agent has access to the local time-series data and communicates with neighboring agents. In this situation, we address the problem of finding a protocol in which each agent autonomously updates its local controller such that the entire network system is positive and stable. In this paper, we provide a solution to the problem based on the projected consensus algorithm.

Keywords: Computational methods, Autonomous systems
Abstract: Interval refinement is a technique for reducing the conservatism of traditional interval based reachability methods by lifting the system to a higher dimension using new auxiliary variables and exploiting the introduced structure through a refinement procedure. We present a novel, efficiently scaling, automatic refinement strategy based on a subspace sampling argument and motivated by reducing the number of interval operations through sparsity. Unlike previous methods, we guarantee that refined bounds shrink as additional auxiliary variables are added. This additionally encourages automation of the lifting phase by allowing larger groups of auxiliary variables to be considered. We implement our strategy in JAX, a high-performance computational toolkit for Python and demonstrate its efficacy on several examples, including regulating a multi-agent platoon to the origin while avoiding an obstacle.

Keywords: Cooperative control, Agents-based systems, Networked control systems
Abstract: This paper explores the application of Hamilton’s rule to altruistic decision-making in multi-agent systems. Inspired by biological altruism, we introduce a framework that evaluates when individual agents should incur costs to benefit their neighbors. By adapting Hamilton’s rule, we define agent "fitness" in terms of task productivity rather than genetic survival. We formalize altruistic decision-making through a graph-based model of multi-agent interactions and propose a solution using collaborative control Lyapunov functions. The approach ensures that altruistic behaviors contribute to the collective goal-reaching efficiency of the system. We illustrate this framework on a multi-agent way-point navigation problem, where we show through simulation how agent importance levels influence altruistic decision-making, leading to improved coordination in navigation tasks.

Keywords: Autonomous robots, Constrained control, Output regulation
Abstract: We consider nonconvex obstacle avoidance where a robot described by nonlinear dynamics and a nonconvex shape has to avoid potentially nonconvex obstacles. Obstacle avoidance is a fundamental problem in robotics and well studied in control. However, existing solutions are computationally expensive (e.g., model predictive controllers), neglect nonlinear dynamics (e.g., graph-based planners), use diffeomorphisms to obtain convex problems (e.g., for star shapes), or are conservative as convex overapproximations are used. We instead provide an efficient yet non-conservative solution for general nonconvex obstacles.Our approach is based on efficient computation of the distance between the robot and obstacles via sampling-based distance functions. We quantify the sampling error and show that, for certain systems, sampling-based distance functions are valid nonsmooth control barrier functions. We also study how to deal with disturbances on the robot dynamics in our setting. Finally, we illustrate our method on a robot navigation task involving an omnidirectional robot and nonconvex obstacles. We also analyze performance and computational efficiency of our controller as a function of the number of samples.

Keywords: Optimization, Sensor networks, Optimization algorithms
Abstract: Sensor networks play a critical role in many situational awareness applications. In this paper, we study the problem of determining sensor placements to balance coverage and connectivity objectives over a target region. Leveraging algebraic graph theory, we formulate a novel optimization problem to maximize sensor coverage over a spatial probability density of event likelihoods while adhering to connectivity constraints. To handle the resulting non-convexity under constraints, we develop an augmented Lagrangian-based gradient descent algorithm inspired by recent approaches to efficiently identify points satisfying the Karush-Kuhn-Tucker (KKT) conditions. We establish convergence guarantees by showing necessary assumptions are satisfied in our setup, including employing Mangasarian-Fromowitz constraint qualification to prove the existence of a KKT point. Numerical simulations under different probability densities demonstrate that the optimized sensor networks effectively cover high-priority regions while satisfying desired connectivity constraints.

Keywords: Learning, Optimization, Agents-based systems
Abstract: We address distributed learning problems over undirected networks. Specifically, we focus on designing a novel ADMM-based algorithm that is jointly computation- and communication-efficient. Our design guarantees computational efficiency by allowing agents to use stochastic gradients during local training. Moreover, communication efficiency is achieved as follows: i) the agents perform multiple training epochs between communication rounds, and ii) compressed transmissions are used. We prove exact linear convergence of the algorithm in the strongly convex setting. We corroborate our theoretical results by numerical comparisons with state of the art techniques on a classification task.

Keywords: Cooperative control, Optimal control, Cyber-Physical Security
Abstract: In this article, a fixed-time convergent reinforcement learning (RL) algorithm is proposed to accomplish the secure formation control of a second-order multi-agent system (MAS) under the false data injection (FDI) attack. To alleviate the FDI attack on the control signal, a zero-sum graphical game is introduced to analyze the attack-defense process, in which the secure formation controller intends to minimize the common performance index function, whereas the purpose of the attacker is the opposite. Attaining the optimal secure formation control policy located at the Nash equilibrium depends on solving the game-associated coupled Hamilton-Jacobi-Isaacs equation. Taking into account fixed-time convergence, a critic-only online RL algorithm with the experience replay technique is designed. Meanwhile, the corresponding convergence and stability proofs are provided. A simulation example is presented to show the effectiveness of the devised scheme.

Keywords: Resilient Control Systems, Distributed control, Networked control systems
Abstract: This extended abstract presents a resilient control framework for multi-agent systems operating under Denial of Service (DoS) attacks. The proposed approach includes two core components: a real-time connectivity detection algorithm that identifies disruptions using only local neighbor information, and a dynamic multi-layer communication architecture that activates hidden backup layers when connectivity is lost. The framework is fully distributed, scalable, and requires no global coordination. The distributed framework is validated on a 1000-node Erdõs-Rényi network and an IEEE 123-bus power system, demonstrating improved resilience, consensus accuracy, and reduced control effort under persistent DoS conditions.

Keywords: Boolean control networks and logic networks, Networked control systems
Abstract: The Boolean Kalman Filter and associated Boolean Dynamical System Theory have been proposed to study the spread of infection on computer networks. Such models feature a network where attacks propagate through, an intrusion detection system that provides noisy signals of the true state of the network, and the capability of the defender to clean a subset of computers at any time. The Boolean Kalman Filter has been used to solve the optimal estimation problem, by estimating the hidden true state given the attack-defense dynamics and noisy observations. However, this algorithm is intractable because it runs in exponential time and space with respect to the network size. We address this feasibility problem by proposing a mean-field estimation approach, which is inspired by the epidemic modeling literature. Although our approach is heuristic, we prove that our estimator exactly matches the optimal estimator in certain non-trivial cases. We conclude by using simulations to show both the run-time improvement and estimation accuracy of our approach.

Keywords: Game theory, Learning, Autonomous systems
Abstract: This paper considers online bandit games with arbitrary delays, where the cost functions of all self-interested players are time-varying. Specifically, the players lack an explicit model of the game and can only learn their actions based on the sole available feedback of delayed cost values. To address this challenging setting, a novel learning algorithm named Cumulative Bandit Online Learning with arbitrary delays (CBOL-ad) is proposed. We conduct regret analysis for time-varying games where the player-specific problem is convex, explicitly revealing the influence of time delays and game structure on the regret bound. In particular, under certain delay conditions, our bound can achieve the same order as that in online bandit optimization problems without considering delays. In addition, numerical simulations are provided to illustrate the algorithmic performance.

Keywords: Optimization algorithms, Agents-based systems, Large-scale systems
Abstract: In this paper, we tackle the problem of distributed optimization over directed networks in open multi-agent systems (OMAS), where agents may dynamically join or leave, causing persistent changes in network topology and problem dimension. These disruptions not only pose significant challenges to maintaining convergence and stability in distributed optimization algorithms, but could also break the network topology into multiple clusters, each one associated with its own set of objective functions. To address this, we propose a novel Open Distributed Optimization Algorithm with Gradient Tracking (OPEN-GT), which employs: (a) a dynamic mechanism for detecting active out-neighbors through acknowledgement messages, and (b) a fully distributed max-consensus procedure to spread information regarding agent departures, in possibly unbalanced directed networks. We show that when all active agents execute OPEN-GT, the optimization process in each formed cluster remains consistent, while the agents converge to their cluster-wide optimal solution if there exists a time after which the network remains unchanged. Finally, we validate our approach in a simulated environment with dynamically changing agent populations, demonstrating its resilience to network variations and its ability to support distributed optimization under OMAS dynamics.

Keywords: Game theory, Agents-based systems, Biologically-inspired methods
Abstract: Multiplayer pursuit-evasion games are a promising framework for studying inter-agent coordination in domains like autonomous navigation and aerial defence. However, many models rely on explicit coordination, a simplistic assumption that does not adequately reflect the complex communication environment of real-world problems. We elucidate this gap by formulating a multiplayer nonlinear differential game using a physics-based spatio-temporal swarming model to explore how indirect communication drives spatial coordination. We introduce novel locally optimal state feedback strategies, derived via a Hamilton-Jacobi-Bellman approach, that promote synchronisation amongst peers to enhance coordination using mutual observation without requiring explicit communication. Finally, we demonstrate that these strategies harness spatio-temporal dynamics to improve team performance in a three-player scenario, establishing a strong foundation for future numerical evaluations.

Keywords: Game theory, Mean field games, Control of networks
Abstract: In this paper, we study a model of network formation in large populations. Each agent can choose the strength of interaction (i.e. connection) with other agents to find a Nash equilibrium. Different from the recently-developed theory of graphon games, here each agent's control depends not only on her own index but also on the index of other agents. After defining the general model of the game, we focus on a special case with piecewise constant graphs and we provide optimality conditions through a system of forward-backward stochastic differential equations. Furthermore, we show the uniqueness and existence results. Finally, we provide numerical experiments to discuss the effects of different model settings.

Keywords: Game theory, Reinforcement learning, Autonomous vehicles
Abstract: Markov games (MGs) provide a mathematical foundation for multi-agent reinforcement learning (MARL), enabling self-interested agents to learn their optimal policies while interacting with others in a shared environment. However, due to the complexities of an MG problem, seeking (Markov perfect) Nash equilibrium (NE) is often very challenging for a general-sum MG. Markov potential games (MPGs), which are a special class of MGs, have appealing properties such as guaranteed existence of pure NEs and guaranteed convergence of gradient play algorithms, thereby leading to desirable properties for many MARL algorithms in their NE-seeking processes. However, the question of how to construct MPGs has been open. This paper provides sufficient conditions on the reward design and on the Markov decision process (MDP), under which an MG is an MPG. Numerical results on autonomous driving applications are reported.

Keywords: Distributed control, Cooperative control, Networked control systems
Abstract: The advancements in resilient consensus research have largely relied on the assumption of a timeinvariant agent set, limiting their applicability in dynamic environments. This paper extends the resilient consensus problem to a more dynamic setting by incorporating agent arrivals and departures, modeling an open multi-agent system. To ensure resilient consensus in such systems, we propose a novel algorithm that separately designs update schemes for cooperative remaining agents and newly joining agents. Furthermore, we establish and analyze the sufficient condition for resilient consensus under malicious attacks in open multi-agent systems. Numerical examples are provided to validate the effectiveness of our theoretical findings.

Keywords: Game theory
Abstract: We investigate the relationship between the team-optimal solution and the Nash equilibrium (NE) to assess the impact of strategy deviation on team performance. As a working use case, we focus on a class of flow assignment problems in which each source node acts as a cooperating decision maker (DM) within a team that minimizes the team cost based on the team-optimal strategy. In practice, some selfish DMs may prioritize their own marginal cost and deviate from NE strategies, thus potentially degrading the overall performance. To quantify this deviation, we explore the deviation bound between the team-optimal solution and the NE in two specific scenarios: (i) when the team-optimal solution is unique and (ii) when multiple solutions do exist. This helps DMs analyze the factors influencing the deviation and adopting the NE strategy within a tolerable range. Furthermore, in the special case of a potential game model, we establish the consistency between the team-optimal solution and the NE. Once the consistency condition is satisfied, the strategy deviation does not alter the total cost, and DMs do not face a strategic trade-off. Finally, we validate our theoretical analysis through some simulation studies.

Keywords: Agents-based systems, Autonomous systems, Distributed control
Abstract: Decentralized estimation of centroid and common reference frame in multi-agent systems is a challenging problem, particularly when agents do not have access to global positioning data. This challenge intensifies in Open Multi-Agent Systems (OMAS), where the network composition dynamically changes due to agents joining or leaving, causing fluctuations in the number of participants. This paper presents a novel, decentralized gossip-based algorithm that enables agents in OMAS to collaboratively estimate both the centroid and a common reference frame in a 2-D environment where the number of agents may fluctuate over time. Notably, our approach remains robust despite noisy distance measurements and intermittent participation, as it relies on asynchronous, local pairwise interactions. Designed to accommodate the dynamic nature of network topologies, our algorithm can be employed for real-world applications where agents can join or leave the system due to failures, resource limitations or external environmental factors.

Keywords: Large-scale systems, Distributed control
Abstract: Multi-agent systems need to satisfy complex tasks where collaboration among agents is required. These tasks are often specified using formal methods such as Linear Temporal Logic (LTL). However, enforcing LTL constraints in multi-agent systems presents computational challenges, particularly in large-scale systems, since the size of the joint multi-agent state space scales exponentially with the number of agents. In this paper, we develop a scalable and efficient framework to assign agents to swarms to satisfy LTL specifications. The central idea of our approach is to map multi-agent planning to a combinatorial optimization problem of assigning agents to swarms during each time step, which reduces computational complexity. Further, we demonstrate that swarm assignments can be computed using submodular optimization, leading to a computationally efficient greedy algorithm with a provable (1-frac{1}{e}) optimality guarantee. We extend our approach to a fully distributed scenario, enabling agents to form swarms in a decentralized manner while preserving near-optimal performance. The proposed framework is validated through theoretical analysis and simulations. Compared to a mixed-integer programming-based baseline, our approach is over 10^6 times faster when there are more than 10 agents, while still guaranteeing the satisfaction of LTL formula.

Keywords: Optimization, Machine learning, Robotics
Abstract: WiFi-based sensing is a promising tool for activity recognition, localization, and human-robot interaction, enabled by the rich Channel State Information (CSI) of modern standards. However, the high dimensionality of CSI, often hundreds of subcarriers, creates challenges in computation, communication, and storage. We propose a submodular optimization framework for subcarrier selection, formulating the task as a combinatorial problem and exploiting the diminishing-returns property of submodular functions. A simple greedy algorithm provides near-optimal selection with theoretical guarantees, while analysis of a convex relaxation shows optimal solutions are sparse and boundary-optimal. We validate our method on the multimodal RoboMNIST dataset for robot activity recognition, showing that it reduces CSI dimensionality while preserving high accuracy and outperforming heuristic baselines. These results demonstrate the effectiveness of submodular optimization for scalable and resource-efficient WiFi sensing in robotic and IoT systems.

Keywords: Optimization, Optimization algorithms
Abstract: We study a distributed framework for solving submodular maximization under a partitioned matroid constraint. A group of agents are connected by a graph and each agent needs to choose a subset from its local ground set. The goal of the agents is to maximize a global objective function that is submodular with respect to the union of the sets chosen by all the agents. We propose a distributed algorithm to solve the problem that lets the agents communicate over the graph and return a global solution that is a 1/(1+c) approximation of the optimal solution in a finite number of communication rounds among the agents, where c is the curvature of the submodular function. We further consider an online setting of the problem where the global objective function can change over a time horizon T. We propose an online distributed algorithm for this setting with frac{1}{1+c}-regret that scales as sqrt{T}.

Keywords: Optimization, Optimization algorithms, Numerical algorithms
Abstract: This paper introduces Rewired Sequential Greedy (ResQue Greedy), an enhanced approach for submodular maximization under cardinality constraints. By integrating a novel set curvature metric within a lattice-based framework, ResQue Greedy identifies and corrects suboptimal decisions made by the standard sequential greedy algorithm. Specifically, a curvature-aware rewiring strategy is employed to dynamically redirect the solution path, leading to improved approximation performance over the conventional sequential greedy algorithm without significantly increasing computational complexity. Numerical experiments demonstrate that ResQue Greedy achieves tighter near-optimality bounds compared to the traditional sequential greedy method.

Keywords: Autonomous robots, Cooperative control, Agents-based systems
Abstract: This paper presents a multi-team collaboration strategy based on Hamilton's rule from ecology that facilitates resource allocation among multiple teams, where agents are considered as shared resource among all teams that must be allocated appropriately. We construct an algorithmic framework that allows teams to make bids for agents that consider the costs and benefits of transferring agents while also considering relative mission importance for each team. This framework is applied to a multi-team coverage control mission to demonstrate its effectiveness. It is shown that the necessary criteria of a mission evaluation function are met by framing it as a function of the locational coverage cost of each team with respect to agent gain and loss, and these results are illustrated through simulations.

Keywords: Autonomous systems, Autonomous robots, Cooperative control
Abstract: We conduct research on a distributed task assignment algorithm, which is one of the distributed decision-making methods in multi-agent system. We consider a mission in which multi-agents transport multiple packages. In particular, we deal with a problem in which one agent can carry multiple packages depending on its loading capacity, and there are heavy packages which require multiple agents to transport. To achieve the goal, we combine two methods. The first is Grouping, a method of calculating the groups of packages that one agent can carry at a time. The second is two types of lists which record task start time of agents. In consequence, we decrease the task start time and agents' moving distance. We also consider transportation control, by using consensus based control to second-order system.

Keywords: Robotics, Autonomous systems, Optimal control
Abstract: This work proposes a novel multi-robot task allocation framework for robots that can switch between multiple modes, e.g., flying, driving, or walking. We first provide a method to encode the multi-mode property of robots as a graph, where the mode of each robot is represented by a node. Next, we formulate a constrained optimization problem to decide both the task to be allocated to each robot as well as the mode in which the latter should execute the task. The robot modes are optimized based on the state of the robot and the environment, as well as the energy required to execute the allocated task. Moreover, the proposed framework is able to encompass kinematic and dynamic models of robots alike. Furthermore, we provide sufficient conditions for the convergence of task execution and allocation for both robot models.

Keywords: Formal Verification/Synthesis, Computational methods, Autonomous systems
Abstract: In this paper, we present pyspect, a Python toolbox that simplifies the use of reachability analysis for temporal logic problems. Currently, satisfying complex requirements in cyber-physical systems requires significant manual effort and domain expertise to develop the underlying reachability programs. This high development effort limits the broader adoption of reachability analysis for complex verification problems. To address this, pyspect provides a method-agnostic approach to performing reachability analysis for verifying a temporal logic specification via temporal logic trees (TLTs). It enables the specification of complex safety and liveness requirements using high-level logic formulations that are independent of any particular reachability technique or set representation. As a result, pyspect allows for the comparison of different reachability implementations, such as Hamilton-Jacobi and Hybrid Zonotope-based reachability analysis, for the same temporal logic specification. This design separates the concerns of implementation developers (who develop numerical procedures for reachability) and end-users (who write specifications). Through a simple vehicle example, we demonstrate how pyspect simplifies the synthesis of reachability programs, promotes specification reusability, and facilitates side-by-side comparisons of reachability techniques for complex tasks.

Keywords: Nonlinear output feedback
Abstract: Control barrier functions (CBFs) offer a powerful tool for enforcing safety specifications in control synthesis. This paper deals with the problem of constructing valid CBFs. Given a second-order system and any desired safety set with linear boundaries in the position space, we construct a provably control-invariant subset of this desired safety set. The constructed subset does not sacrifice any positions allowed by the desired safety set, which can be nonconvex. We show how our construction can also meet safety specification on the velocity. We then demonstrate that if the system satisfies standard Euler- Lagrange systems properties then our construction can also handle constraints on the allowable control inputs. We finally show the efficacy of the proposed method in a numerical example of keeping a 2D robot arm safe from collision.

Keywords: Uncertain systems, Machine learning, Estimation
Abstract: Certifying safety in dynamical systems is crucial, but barrier certificates — widely used to verify that system trajectories remain within a safe region — typically require explicit system models. When dynamics are unknown, data-driven methods can be used instead, yet obtaining a valid certificate requires rigorous uncertainty quantification. For this purpose, existing methods usually rely on full-state measurements, limiting their applicability. This paper proposes a novel approach for synthesizing barrier certificates for unknown systems with latent states and polynomial dynamics. A Bayesian framework is employed, where a prior in state-space representation is updated using output data via a targeted marginal Metropolis–Hastings sampler. The resulting samples are used to construct a barrier certificate through a sum-of-squares program. Probabilistic guarantees for its validity with respect to the true, unknown system are obtained by testing on an additional set of posterior samples. The approach and its probabilistic guarantees are illustrated through a numerical simulation.

Keywords: Constrained control, Data driven control, Identification
Abstract: Control Barrier Functions (CBFs) offer a framework for ensuring set invariance and designing constrained control laws. However, crafting a valid CBF relies on system-specific assumptions and the availability of an accurate system model, underscoring the need for systematic data-driven synthesis methods. This paper introduces a data-driven approach to synthesizing a CBF for discrete-time LTI systems using only input-output measurements. The method begins by computing the maximal control invariant set using an input-output data-driven representation, eliminating the need for precise knowledge of the system’s order and explicit state estimation. The proposed CBF is then systematically derived from this set, which can accommodate multiple input-output constraints. Furthermore, the proposed CBF is leveraged to develop a minimally invasive safety filter that ensures recursive feasibility with an adaptive decay rate. To improve clarity, we assume a noise-free dataset, though the method can be extended using data-driven reachability to capture noise effects and handle uncertainty. Finally, the effectiveness of the proposed method is demonstrated on an unknown time-delay system.

Keywords: Formal Verification/Synthesis, Hybrid systems, Optimization
Abstract: Barrier certificates, a form of state invariants, provide an automated approach for safety verification of dynamical systems. Similar to barrier certificates, recent works explore the notion of closure certificates, a form of transition invariants, to verify dynamical systems against ω-regular properties including safety. A closure certificate, defined over state pairs of a dynamical system, is a real-valued function whose zero superlevel set characterizes an inductive transition invariant of the system. The search for such a certificate can be effectively automated by assuming it to be within a specific template class, e.g. a polynomial of a fixed degree, and then using optimization techniques such as sum-of-squares programming (SOS) or satisfiability-modulo-theory solvers (SMT) to find it. Unfortunately, one may not be able to find such a certificate for a fixed template. In such a case, one must change the template, e.g. increase the degree of the polynomial. In this paper we consider a notion of multiple closure certificates, dubbed interpolation-inspired closure certificates. An interpolation-inspired closure certificate consists of a set of functions which jointly over-approximate a transition invariant by first considering one-step transitions, then two, and repeat until a transition invariant is obtained. The advantage of interpolation-inspired closure certificates is that they allow us to prove properties, even when a single function for a fixed template cannot be found using standard approaches. We present an SOS programming approach to search for these set of functions and demonstrate the effectiveness of our proposed method for verifying safety and persistence (or refuting recurrence) over case studies.

Keywords: Constrained control
Abstract: This letter shows how to design control barrier functions for underactuated and fully-actuated Euler-Lagrange systems subject to state and input constraints. The proposed method uses passivity-based considerations to limit the total energy available to the system and ensure constraint satisfaction. The approach can handle multiple state and input constraints regardless of relative degree.