Keywords: Agents-based systems, Autonomous systems, Decentralized control
Abstract: We consider a cooperative pursuit setting involving multiple pursuers and a single evader. The pursuers are required to collectively enclose the evader in their convex hull throughout the game when the evader's strategy is unknown to the pursuers. While prior works have considered this problem in the two-dimensional space, we generalize their results to the unbounded three-dimensional space. We propose a partition of the evader-centered space such that if each pursuer adheres to its assigned constrained region, then the evader is guaranteed to lie within the convex hull defined by the positions of the pursuers. Each pursuer then solves a robust model predictive control (MPC) problem treating the strategy of the evader as the uncertain variable and by including the partition related constraints. The resulting robust optimization problem is solved in a decentralized manner by the pursuers and guarantees enclosure of the evader. Simulation results demonstrate the effectiveness of the proposed framework.

Keywords: Automotive systems, Communication networks, Markov processes
Abstract: We propose a scenario model predictive controller (SMPC) for determining the acceleration of an autonomous vehicle (AV) based on decisions made by human-driven vehicle (HDV) obstacles in a traffic intersection. The collective behavior of the HDVs is modeled as a decision process between Markovian agents, and the time-dependent probabilities of scenarios in which the agents decide to occupy the intersection are predicted. The SMPC then determines the AV acceleration that minimizes an expected scenario cost formulated using the transient probability predictions. In simulation, we show how using transient instead of stationary scenario probabilities determines the AV’s acceleration based on a trade-off between a current observation and a predicted environment behavior.

Keywords: Building and facility automation, Control applications, Smart grid
Abstract: Water heating accounts for 18% of energy consumption and 15% of greenhouse gas (GHG) emissions in U.S. residential buildings. Electric heat pump water heaters (HPWHs) are more energy efficient than electric resistance water heaters, but widespread adoption could strain the grid. Moreover, successful decarbonization requires aligning HPWH operation with electricity generated from clean energy sources like solar and wind. This work demonstrates the effectiveness of a multi-objective economic model predictive control (MPC) in minimizing electricity costs, GHG emissions, and comfort violations for a laboratory HPWH unit over three consecutive days. An automated tuning approach for the MPC cost function is presented, along with a discussion of the practical implementation challenges of the proposed MPC framework. Experimental results show that MPC reduces electricity costs by 31% and GHG emissions by 46% compared to a typical HPWH rule-based control strategy.

Keywords: Chemical process control, Predictive control for nonlinear systems, Machine learning
Abstract: This work proposes the implementation of encryption to machine learning-based model predictive control to enhance cybersecurity without significant performance losses. Tracking model predictive control with respect to an unstable steady state for a model chemical process serves as the control group. Results are compared to the same process with the Pallier cryptosystem applied with varying degrees of precision. We determine the stabilizability of the design, including the impacts of quantization loss and sample-and-hold control. Results of the simulated system are compared to demonstrate the impact of quantization losses on system performance.

Keywords: Constrained control, Machine learning
Abstract: Recently, there has been a resurgence in research focused on using neural networks (NNs) to approximate model predictive control (MPC) for fast optimization, achieving great practical success. However, the relation between the computational complexity of NNs and control performance remains an open question. In this paper, we establish explicit upper bounds on the width and depth of ReLU NNs for linear MPC approximation with constraint satisfaction and stability guarantees. These bounds are expressed in terms of the desired approximation error, the problem size of the multi-parameter quadratic program and its Lipschitz constant, as well as the shape of the constraint set. We find that increasing the depth is more effective than increasing the width. Our work provides a theoretical guide for designing NN approximations of MPC.

Keywords: Control software, Predictive control for linear systems, Robust control
Abstract: As society advances, increasingly complex systems pose new challenges for control engineers. To address these challenges, researchers continually develop state-of-the-art control techniques. This creates a demand for software tools that integrate these advanced methods and allow users to utilize them with ease.

In this paper, we aim to enhance the MPT+ toolbox, an extension of the widely used Multi-Parametric Toolbox~(MPT), with a novel implicit tube Model Predictive Control (MPC) approach. This approach avoids geometric set operations and enables robust MPC designs for large-scale problems.

It will be shown that MPT+ allows users to apply this control strategy within a few lines in a user-friendly manner. Moreover, via a case study, we demonstrate that the implicit tube MPC approach enables us to design robust controllers for more complex systems, which were previously computationally intractable with the commonly used rigid tube MPC methods.

Keywords: Hierarchical control, Predictive control for linear systems
Abstract: This paper presents two complementary approaches for reducing the computational complexity of the upper-level controller in a two-level vertical hierarchical Model Predictive Control (MPC) framework for systems with multiple dynamic timescales. First, inner-approximations are used to produce simplified output constraint sets, reducing the number of inequality constraints. Second, the fast and slow dynamic timescales are used to determine a reduced input space, reducing the number of decision variables. A systematic analysis of the number of design variables and constraints is provided and the improved scalability of the proposed approach is demonstrated using a simulated thermal system example.

Keywords: Fuzzy systems, Learning, Identification for control
Abstract: Model predictive control (MPC) is a powerful control technique for online optimization using system model-based predictions over a finite time horizon. However, the computational cost MPC requires can be prohibitive in resource-constrained computer systems. This paper presents a fuzzy controller synthesis framework guided by MPC. In the proposed framework, training data is obtained from MPC closed-loop simulations and is used to optimize a low computational complexity controller to emulate the response of MPC. In particular, autoregressive moving average (ARMA) controllers are trained using data obtained from MPC closed-loop simulations, such that each ARMA controller emulates the response of the MPC controller under particular desired conditions. Using a Takagi-Sugeno (T-S) fuzzy system, the responses of all the trained ARMA controllers are then weighted depending on the measured system conditions, resulting in the Fuzzy-Autoregressive Moving Average (F-ARMA) controller. The effectiveness of the trained F-ARMA controllers is illustrated via numerical examples.

Keywords: Manufacturing systems, Materials processing, Reduced order modeling
Abstract: Roll-to-roll (R2R) mechanical transfer is an advanced technology that facilitates continuous, etchant-free transfer of printed electronics and two-dimensional (2D) materials on flexible substrates. It involves dry peeling of fabricated devices or material from a donor substrate and transferring them to a target substrate, both being flexible and known as webs. Achieving high-quality transfer requires precise regulation of the tensions in these webs, necessitating a controller that handles process nonlinearities, disturbances, and input constraints. To address these challenges, we propose a nonlinear model predictive control (MPC) approach for R2R mechanical peeling. However, current R2R peeling models are either too computationally expensive to use or lack the accuracy needed for real-time application. This study introduces a fast, accurate nonlinear model of R2R peeling and integrates it into an MPC framework. A case study on the dry transfer of chemical vapor deposition (CVD) graphene is used to demonstrate the effectiveness of this proposed nonlinear MPC approach.

Keywords: Optimization, Robust control, Predictive control for nonlinear systems
Abstract: We consider actuator attacks where an adversary applies an optimized control sequence to drive the system's output away from its nominal value while aiming to remain undetected. In response, we propose a robust model predictive defense (RMPD) strategy where an anticipatory defender foresees such an adversary. Our RMPD approach is formulated as a min-max problem, accounting for the set of the adversary's feasible control inputs when available and being conservative otherwise, allowing for different prediction horizons of the defender and adversary. We present a novel exact convex reformulation approach for our RMPD problem with a rectangular feasibility region. The novel exact reformulation is benchmarked against a relaxed formulation using an ellipsoidal feasibility region of the adversary's control inputs, which is solved using the S-procedure. Numerical experiments on a coupled pendulum validate our RMPD's effectiveness and show the exact approach's improved efficiency in terms of required control energy compared to the relaxed approach.

Keywords: Predictive control for linear systems, Constrained control, Optimal control
Abstract: Model predictive control (MPC) has emerged as a promising strategy for the control of lithium-ion batteries due mainly to its capability for real-time constraint handling. However, classical implementations of MPC cannot guarantee stability, thus limiting its practical application. In addition, classical linear MPC relies on the computation of a constrained quadratic program at every time step, the computation of which may become burdensome when long horizons and numerous constraints are involved. The present paper applies a ”dual mode” variation of MPC which reduces the necessity of implementing a quadratic program and provides assured stability of operation, at the cost of introducing a degree of conservatism.

Keywords: Predictive control for nonlinear systems, Constrained control
Abstract: Robust Model Predictive Control (MPC) for nonlinear systems is a problem that poses significant challenges, as highlighted by the diversity of approaches proposed in the last decades. Often compromises concerning computational load, conservatism, generality, or implementation complexity have to be made. This work provides a practical balance by proposing a novel receding-horizon robust MPC formulation for nonlinear discrete-time systems. By explicitly accounting for how disturbances and linearization errors are propagated through the nonlinear dynamics, a constraint tightening-based formulation is obtained, with guarantees of robust constraint satisfaction. The proposed controller relies on iteratively solving a Nonlinear Program (NLP) to optimize system operation and the required constraint tightening. Numerical experiments show the effectiveness of the proposed controller for three nonlinear systems. Comparisons with two existing techniques show scenario-dependent tradeoffs, though the proposed controller consistently generates less conservative error sets.

Keywords: Predictive control for nonlinear systems, Data driven control, Adaptive control
Abstract: A major challenge in autonomous flights is unknown disturbances, which can jeopardize safety and lead to collisions, especially in obstacle-rich environments. This paper presents a disturbance-aware motion planning and control framework designed for autonomous aerial flights. The framework is composed of two key components: a disturbance-aware motion planner and a tracking controller. The disturbance-aware motion planner consists of a predictive control scheme and a learned model of disturbances that is adapted online. The tracking controller is designed using contraction control methods to provide safety bounds on the quadrotor behaviour in the vicinity of the obstacles with respect to the disturbance-aware motion plan. Finally, the algorithm is tested in simulation scenarios with a quadrotor facing strong crosswind and ground-induced disturbances.

Keywords: Predictive control for nonlinear systems, Machine learning, Optimization algorithms
Abstract: We present a method, which allows efficient and safe approximation of model predictive controllers using kernel interpolation. Since the computational complexity of the approximating function scales linearly with the number of data points, we propose to use a scoring function which chooses the most promising data. To further reduce the complexity of the approximation, we restrict our considerations to the set of closed-loop reachable states. That is, the approximating function only has to be accurate within this set. This makes our method especially suited for systems, where the set of initial conditions is small. In order to guarantee safety and high performance of the designed approximated controller, we use reachability analysis based on Monte Carlo methods.

Keywords: Predictive control for nonlinear systems, Neural networks, Identification for control
Abstract: We present a reformulation of the model predictive control problem using a Legendre basis. To do so, we use a Legendre representation both for prediction and optimization. For prediction, we use a neural network to approximate the dynamics by mapping a compressed Legendre representation of the control trajectory and initial conditions to the corresponding compressed state trajectory. We then reformulate the optimization problem in the Legendre domain, and demonstrate methods for including optimization constraints. We present simulation results demonstrating that our implementation provides a speedup of 31-40 times for comparable or lower tracking error with or without constraints on a benchmark task.

Keywords: Uncertain systems, Predictive control for linear systems, Reinforcement learning
Abstract: A reinforcement learning (RL)-based robust model predictive control (MPC) is proposed to achieve on-line learning and control for a system with unknown dynamics. A temporal difference (TD) RL method is applied to learn the unknown dynamics by utilizing real-time measurements of input-state data. With the learned system matrices of RL, robust MPC is used to provide effective control actions and approximate the value function for RL. The convergence of learning unknown parameters is discussed based on two constructed parameter sets. A stability condition is derived to connect the Lyapunov function matrices between two consecutive learning steps and input-to-state stability of the unknown system is analyzed. Simulation results are presented to validate the effectiveness of the proposed approach.

Keywords: Predictive control for nonlinear systems, Stability of nonlinear systems, Optimization algorithms
Abstract: This letter analyzes the contraction property of the nonlinear systems controlled by suboptimal model predictive control (MPC) using the continuation method. We propose a contraction metric that reflects the hierarchical dynamics inherent in the continuation method. We derive a pair of matrix inequalities that elucidate the impact of suboptimality on the contraction of the optimally controlled closed-loop system. A numerical example is presented to verify our contraction analysis. Our results are applicable to other MPCs than stabilization, including economic MPC.

Keywords: Predictive control for nonlinear systems, Stability of nonlinear systems, Optimal control
Abstract: This work analyzes the stability properties of a nonlinear Model Predictive Control (MPC) scheme for avoidance. This control approach introduces an extra penalty for avoidance within the nonlinear tracking MPC framework. We demonstrate that, under a mild assumption on the avoidance penalty, the closed-loop system is Input-to-State Stable (ISS) with respect to this penalty. Furthermore, we discuss the conditions under which asymptotic stability can be achieved and present a simplified scheme with relaxed terminal constraints. To illustrate the effectiveness of the proposed strategy, we apply it to the control of a van der Pol oscillator subjected to non-convex constraints.

Keywords: Predictive control for nonlinear systems, Autonomous systems, Statistical learning
Abstract: The control of a single agent in complex and uncertain multi-agent environments requires careful consideration of the interactions between the agents. In this context, this paper proposes a dual model predictive control (MPC) method using Gaussian process (GP) models for multi-agent systems. While Gaussian process MPC (GP-MPC) has been shown to be effective in predicting the dynamics of other agents, current methods do not consider the influence of the control input on the covariance of the predictions, and hence lack the dual control effect. Therefore, we propose a dual MPC that directly optimizes the actions of the ego agent, and the belief of the other agents by jointly optimizing their state trajectories as well as the associated covariance while considering their interactions through a GP. We demonstrate our GP-MPC method in a simulation study on autonomous driving, showing improved prediction quality compared to a baseline stochastic MPC. The results show that GP-MPC can learn the interactions between the agents online, demonstrating the potential of GPs for dual MPC in uncertain and unseen scenarios.

Keywords: Machine learning, Optimal control, Robotics
Abstract: This paper presents a novel approach to learning free terminal time closed-loop control for robotic manipulation tasks, enabling dynamic adjustment of task duration and control inputs to enhance performance. We extend the supervised learning approach, namely solving selected optimal open-loop problems and utilizing them as training data for a policy network, to the free terminal time scenario. Three main challenges are addressed in this extension. First, we introduce a marching scheme that enhances the solution quality and increases the success rate of the open-loop solver by gradually refining time discretization. Second, we extend the QRnet in cite{QRnet} to the free terminal time setting to address discontinuity and improve stability at the terminal state. Third, we present a more automated version of the initial value problem (IVP) enhanced sampling method from previous work cite{IVP_Enhanced} to adaptively update the training dataset, significantly improving its quality. By integrating these techniques, we develop a closed-loop policy that operates effectively over a broad domain with varying optimal time durations, achieving near globally optimal total costs. The appendix and videos are available at https://deepoptimalcontrol.github.io/FreeTimeManipulator.

Keywords: Iterative learning control, Linear systems, LMIs
Abstract: This paper considers the class of 2D linear systems where a linear differential equation governs information propagation in one of the two directions, and in the other, the updating is governed by a difference equation; in some of the literature, these systems are referred to as mixed or hybrid. Linear repetitive processes are a particular case of such systems that have been used in several areas, such as iterative learning control. Stability margins for these systems have received attention in the literature. In this paper, new results on these margins for differential linear repetitive processes are derived and applied to iterative learning control law design. Sufficient conditions for computing the required stability margins are formulated using linear matrix inequalities, which can be readily adapted to design the required controllers. Finally, the applications of the developed results to design of ILC scheme for a typical actuator in a tracking servo system is presented to demonstrate the effectiveness of the design and highlight its advantages over existing alternatives.

Keywords: Machine learning, Adaptive control, Robust adaptive control
Abstract: Functional electrical stimulation (FES) is widely used for rehabilitating individuals with total or partial limb paralysis, but challenges like muscle fatigue and discomfort limit its effectiveness. Hybrid exoskeletons combine the rehabilitative benefits of exoskeletons and FES, while mitigating the drawbacks of each. However, despite hybrid exoskeletons being highly effective in rehabilitation, the dynamics associated with these systems are complex. Deep neural networks (DNNs) can approximate these complex hybrid exoskeleton dynamics; however, they traditionally lack stability guarantees and robustness, hindering their application in real-world systems. Moreover, traditional training methods (e.g., gradient descent) require an extensive dataset and offline training, further hindering a DNNs practical use. Therefore, this paper presents an innovative Lyapunov-based adaptation law, with a gradient descent-like structure, that is designed to update all layer weights of a DNN in real-time for a DNN-based hybrid exoskeleton control framework. To promote user comfort and safety, a saturation limit was implemented on the DNN-based FES controller and the excess control effort was redirected to the exoskeleton. A Lyapunov-based stability analysis was performed on the DNN-based hybrid exoskeleton control system to prove global asymptotic trajectory tracking. A numerical simulation of the designed controller was performed to validate the results.

Keywords: Machine learning, Learning, Time-varying systems
Abstract: In recent years, there has been a growing interest in integrating linear state-space models (SSM) in deep neural network architectures of foundation models. This is exemplified by the recent success of Mamba, showing better performance than the state-of-the-art Transformer architectures in language tasks. Foundation models, like e.g. GPT-4, aim to encode sequential data into a latent space in order to learn a compressed representation of the data. The same goal has been pursued by control theorists using SSMs to efficiently model dynamical systems. Therefore, SSMs can be naturally connected to deep sequence modeling, offering the opportunity to create synergies between the corresponding research areas. This paper is intended as a gentle introduction to SSM-based architectures for control theorists and summarizes the latest research developments. It provides a systematic review of the most successful SSM proposals and highlights their main features from a control theoretic perspective. Additionally, we present a comparative analysis of these models, evaluating their performance on a standardized benchmark designed for assessing a model’s efficiency at learning long sequences.

Keywords: Machine learning, Adaptive control, Robust adaptive control
Abstract: Abstract--- Recent studies suggest that deep neural networks (DNNs) have the potential to outperform neural networks (NNs) in approximating complex dynamics, which may enhance the tracking performance of a control system. However, unlike NN-based nonlinear control systems, designing update policies for the inner-layer weights of a DNN using Lyapunov-based stability methods is problematic since the inner-layer weights are nested within activation functions. Traditional DNN training methods (e.g., gradient descent) may improve the approximation capability of a DNN and thus could enhance a DNN-based controller’s tracking performance; however, traditional DNN-based control approaches lack stability guarantees and may be ineffective in training the DNN without large data sets, which could hinder the tracking performance. In this work, a DNN-based control structure is developed for a hybrid exoskeleton, which combines a rehabilitative robot with functional electrical stimulation (FES). The proposed control system updates the DNN weights in real-time and a rigorous Lyapunov-based stability analysis is performed to ensure semi-global asymptotic trajectory tracking, even without the presence of a data set. Specifically, a Lyapunov-based update law is developed for the output-layer DNN weights and Lyapunov-based constraints are established for the adaptation laws of the inner-layer DNN weights. Additionally, the DNN-based FES controller was designed to be saturated to increase the comfort and safety of the participant.

Keywords: Machine learning, Modeling, Nonlinear systems identification
Abstract: This paper is centered around the approximation of dynamical systems by means of Gaussian processes. To this end, trajectories of such systems must be collected to be used as training data. The measurements of these trajectories are typically noisy, which implies that both the regression inputs and outputs are corrupted by noise. However, most of the literature considers only noise in the regression outputs. In this paper, we show how to account for the noise in the regression inputs in an extended Gaussian process framework to approximate scalar and multidimensional systems. We demonstrate the potential of our framework by comparing it to different state-of-the-art methods in several simulation examples.

Keywords: Machine learning, Predictive control for nonlinear systems, Neural networks
Abstract: Learning-based offline training methods can effectively reduce the computational demands of real-time optimization-based control. However, existing approaches that rely on imitation or differentiable programming may fail if a well-tuned MPC controller or differentiable plant dynamics are unavailable. To address this, we propose a zeroth-order optimization approach for training neural controllers using input-output prediction sequences or even only bandit feedback. To improve the training efficiency, we introduce a mixed gradient computation scheme that first applies a zeroth-order gradient estimation with the bandit controller performance, and then applies a backpropagation to the neural controller network. Additionally, we propose a training procedure that relies on the closed-loop interaction between the neural controller network and a black-box plant model to generate training data with better distribution, thereby positively impacting training effectiveness. Finally, we evaluate the proposed method through extensive simulation experiments for tracking and stabilization control tasks with high-dimensional, nonlinear, and strongly coupled plants. The controller shows generalizability when implemented in plants with parameter drifts, which makes it suitable for transferring from simulation to real-world applications.

Keywords: Neural networks, Machine learning, Nonlinear systems identification
Abstract: This paper introduces the temporally-consistent bilinearly recurrent autoencoder (tcBLRAN), a Koopman operator based neural network architecture for modeling a control-affine nonlinear control system. The proposed method extends traditional Koopman autoencoders (KAE) by incorporating bilinear recurrent dynamics that are consistent across predictions, enabling accurate long-term forecasting for control-affine systems. This overcomes the roadblock that KAEs face when encountered with limited and noisy training datasets, resulting in a lack of generalizability due to inconsistency in training data. Through a blend of deep learning and dynamical systems theory, tcBLRAN demonstrates superior performance in capturing complex behaviors and control systems dynamics, providing a superior data-driven modeling technique for control systems and outperforming the state-of-the-art Koopman bilinear form (KBF) learned by autoencoder networks.

Keywords: Reinforcement learning, Automotive control
Abstract: Collision avoidance is a critical component of automotive safety systems and a key feature of autonomous vehicles. This paper focuses on the lateral control aspect of collision avoidance. The complexity of vehicle dynamics, environmental uncertainties, and the need for real-time computation present significant challenges to effective collision avoidance control. State-of-the-art approaches, such as Model Predictive Control (MPC), provide optimal solutions but are difficult to calibrate, suffer from high computational demands, and can be unstable when simplified models are used. This study investigates an alternative approach using reinforcement learning (RL). We demonstrate how an RL agent, trained in a simulated environment, can be successfully deployed and perform aggressive maneuvers in a real vehicle without additional training. The RL agent’s performance is compared to state-of-the-art controllers, showing competitive results with lower computational requirements.

Keywords: Reinforcement learning, Autonomous robots, Estimation
Abstract: In robotic navigation, continuous localization can be inefficient in certain scenarios, such as underwater searches. For example, an underwater agent surfacing to localize too often hinders it from searching for critical items underwater, such as black boxes from crashed aircraft. On the other hand, if the agent never localizes, it may fail to find the items due to inadvertently leaving the search area or entering hazardous, restricted areas. Motivated by such scenarios, we explore approaches to help an agent determine ``when to localize?''. We propose a bi-criteria optimization approach to determine optimal localization timing, which balances localization frequency with failure probability. Our method, RiskRL, employs a Particle Filter and a constrained Reinforcement Learning (RL) framework, which uses a recurrent Soft Actor-Critic (SAC) network. This approach improves our previous POMDP-based solution by reducing computational complexity and eliminating the need for complete transition and observation models. Experimental results demonstrate that RiskRL showed a 26% increase in success rates in unseen environments over our baselines. The implementation is available on GitHub: https://github.com/raaslab/when-to-localize-riskrl

Keywords: Reinforcement learning, Markov processes, Automata
Abstract: We present LARL-RM (Large language model-generated Automaton for Reinforcement Learning with Reward Machine) algorithm to encode high-level knowledge into reinforcement learning using automaton to expedite the reinforce- ment learning. Our method uses large language models (LLM) to obtain high-level domain-specific knowledge using prompt engineering instead of providing the reinforcement learning (RL) algorithm directly with the high-level knowledge that requires an expert to encode the automaton. We use chain- of-thought and few-shot methods for prompt engineering and demonstrate that our method works using these approaches. Additionally, LARL-RM allows for fully closed-loop reinforcement learning without the need for an expert to guide and supervise the learning since LARL-RM can use the LLM directly to generate the required high-level knowledge for the task at hand. Moreover, we demonstrate LARM-RM robustness to LLM hallucination and show the theoretical guarantee of our algorithm to converge to an optimal policy. We show that LARL-RM speeds up the convergence by implementing our method in two case studies and compare it to other RL methods.

Keywords: Reinforcement learning, Networked control systems, Autonomous systems
Abstract: In this paper, we propose an adaptive event-triggered reinforcement learning control for continuous-time nonlinear systems, subject to bounded uncertainties, characterized by complex interactions. Specifically, the proposed method is capable of jointly learning both the control policy and the communication policy, thereby reducing the number of parameters and computational overhead when learning them separately or only one of them. By augmenting the state space with accrued rewards that represent the performance over the entire trajectory, we show that accurate and efficient determination of triggering conditions is possible without the need for explicit learning triggering conditions, thereby leading to an adaptive non-stationary policy. Finally, we provide several numerical examples to demonstrate the effectiveness of the proposed approach.

Keywords: Reinforcement learning, Observers for Linear systems, Optimal control
Abstract: Data-driven reinforcement learning (RL) algorithms primarily rely on the system data that may take the input-state or the input-output form. Input-output form is preferred by controls practitioners as it relaxes the sensing requirements compared to the full-state feedback case. The design of output feedback data-driven learning techniques is, however, a challenging control problem. Compared to the input-state form, the learning complexity is often significantly higher. This is a result of higher-order reinforcement learning equations that require deep historic input-output datasets, which then translates to higher exploration requirements and increased sample complexity. We present here a reduced-order parameterization that is tailored for RL-based control applications towards obtaining reduced-order RL learning equations. While the results are general to RL-based optimal control of LTI systems, we consider the classic Q-learning LQR algorithm as a proof-of-concept. It is shown that the parameterization is exact in N steps, where this N is always smaller than the full-order case. An iterative Q-learning algorithm based on the proposed reduced-order parameterized learning equation is presented, whose convergence to the nominal LQR solution is shown analytically. Finally, numerical simulation results are presented to show the effectiveness of the reduced-order method compared to existing full-order output feedback RL control approaches.

Keywords: Stochastic systems, Identification, Linear systems
Abstract: We study the problem of system identification for stochastic continuous-time dynamics, based on a single finite-length state trajectory. We present a method for estimating the possibly unstable open-loop matrix by employing properly randomized control inputs. Then, we establish theoretical performance guarantees showing that the estimation error decays with trajectory length, a measure of excitability, and the signal-to-noise ratio, while it grows with dimension. Numerical illustrations that showcase the rates of learning the dynamics, will be provided as well. To perform the theoretical analysis, we develop new technical tools that are of independent interest. That includes non-asymptotic stochastic bounds for highly non-stationary martingales and generalized laws of iterated logarithms, among others.

Keywords: Reinforcement learning, Stochastic systems, Optimal control
Abstract: This work considers the problem of transferring a policy in the context of reinforcement learning. Specifically, we consider training a policy in a reduced order system and deploying it in the full state system. The motivation for this training strategy is that simulating full-state systems with complex dynamics may take excessive time. While transfer learning alleviates this issue, the transfer guarantees depend on the discrepancy between the two systems. In this work, we consider a class of cascade dynamical systems, where a subset of the state variables influences the dynamics of the remaining states but not vice-versa. We refer to the former as internal states. The reinforcement learning agent learns a policy using a model that ignores the internal states, treating them instead as commanded inputs. In deploying the policy in the full-state system, classic controllers (e.g., a PID) handle the dynamics of the internal states so that they track the reference signal provided by the reinforcement learning policy. The cascade structure allows us to provide transfer guarantees that depend on the stability of the inner loop controller. Numerical experiments on a quadrotor support the theoretical findings.

Keywords: Machine learning, Statistical learning, Neural networks
Abstract: Closed-loop control of nonlinear dynamical systems with partial-state observability demands expert knowledge of a diverse, less standardized set of theoretical tools. Moreover, it requires a delicate integration of controller and estimator designs to achieve the desired system behavior. To establish a general controller synthesis framework, we explore the Decision Transformer (DT) architecture. Specifically, we first frame the control task as predicting the current optimal action based on past observations, actions, and rewards, eliminating the need for a separate estimator design. Then, we leverage the pre-trained language models, i.e., the Generative Pre-trained Transformer (GPT) series, to initialize DT and subsequently train it for control tasks using low-rank adaptation (LoRA). Our comprehensive experiments across five distinct control tasks, ranging from maneuvering aerospace systems to controlling partial differential equations (PDEs), demonstrate DT's capability to capture the parameter-agnostic structures intrinsic to control tasks. DT exhibits remarkable zero-shot generalization abilities for completely new tasks and rapidly surpasses expert performance levels with a minimal amount of demonstration data. These findings highlight the potential of DT as a foundational controller for general control applications.

Keywords: Autonomous systems, Lyapunov methods, Reinforcement learning
Abstract: Recent methods using Reinforcement Learning (RL) have proven to be successful for training intelligent agents in unknown environments. However, current state-of-the-art RL methods require large amounts of data to learn a specific task, leading to unreasonable costs when deploying the agent to collect data in real-world applications. In this paper, we take a control-theoretic approach to improve sample efficiency in RL. Our approach has two key components. First, we build upon recent work which shows that the inclusion of a Control Lyapunov Function (CLF) in the reward function can enable RL algorithms to learn with substantially lower discount factors, and therefore require less data to train. To do this “reward shaping” requires access to a CLF, however, and computational methods to compute a CLF do not scale well to high-dimensional systems. Therefore, the primary contribution of our work is the development of a new variety of CLF-like functions, which we term Decomposed Control Lyapunov Functions (DCLFs). We show that these functions can be used in the same way as CLFs for RL reward shaping, yet are more readily computable in higher-dimensional cases via a system decomposition technique. Through multiple examples, we demonstrate the effectiveness of this approach, where our method finds a policy to successfully land a quadcopter with less than half the amount of real-world data required by two state-of-the-art RL algorithms.

Keywords: Automotive control, Autonomous systems, Reinforcement learning
Abstract: Deep reinforcement learning is a promising technique that can help create autonomous agents. However, it is still an open problem how one can create a controller with robust operation for real-world automotive systems. The difficulty lies in either sample efficiency for real-world learning or developing a good enough simulator for training. This paper addresses the latter, proposing a method that provides a solution to the sim-to-real gap through domain randomization, learning with disturbances, and observation preprocessing. The method is validated on a small-scale F1TENTH-type test vehicle, that is trained to race autonomously in a fully end-to-end manner. It is demonstrated that the training process results in a policy that can drive the car safely even over the grip limit.

Keywords: Robotics, Constrained control, Adaptive control
Abstract: This article presents a closed-form adaptive control- barrier-function (CBF) approach for satisfying state constraints in systems with parametric uncertainty. This approach uses a sampled-data recursive-least-squares algorithm to estimate the unknown model parameters and construct a nonincreasing upper bound on the norm of the estimation error. Together, this estimate and upper bound are used to construct a CBF-based constraint that has nonincreasing conservativeness. Furthermore, if a persistency of excitation condition is satisfied, then the CBF-based constraint has vanishing conservativeness in the sense that the CBF-based constraint converges to the ideal constraint corresponding to the case where the uncertainty is known. In addition, the approach incorporates a monotonically improving estimate of the unknown model parameters—thus, this estimate can be effectively incorporated into a desired control law. We demonstrate constraint satisfaction and performance using a nonholonomic robot.

Keywords: Vision-based control, Visual servo control, Robotics
Abstract: The problem of control based on vision measurements (bearings) has been amply studied in the literature; however, the problem of addressing the limits of the field of view of physical sensors has received relatively less attention (especially for agents with non-trivial dynamics). The technical challenge is that, as in most vision-based control approaches, a standard approach to the problem requires knowing the distance between cameras and observed features in the scene, which is not directly available. Instead, we present a solution based on a Control Barrier Function (CBF) approach that uses a splitting of the original differential constraint to effectively remove the dependence on the unknown measurement error. Compared to the current literature, our approach gives strong robustness guarantees against bounded distance estimation errors. We showcase the proposed solution with the numerical simulations of a double integrator and a quadrotor tracking a trajectory while keeping the corners of a rectangular gate in the camera field of view.

Keywords: Automotive control, Control applications, Optimal control
Abstract: This paper examines the problem of preventing frontal collisions between road vehicles. The paper focuses on the concept of achieving graceful safety control in the context of vehicle collision avoidance, in the sense of ensuring safety when possible, and degrading gracefully otherwise. This is achieved through a novel nonlinear barrier constraint which provides a multi-layered safety guarantee: it ensures safe spacing when possible, and avoids collision even when the safe spacing constraint is breached. The proposed graceful control approach is illustrated using a first-order barrier constraint. Then a second-order version of the constraint is presented that enables tracking a desired jerk signal while assuring graceful safety.

Keywords: Autonomous systems, Optimization
Abstract: A fundamental and classical problem in mobile autonomous systems is maintaining the safety of autonomous agents during deployment. Prior literature has presented techniques using control barrier functions (CBFs) to achieve this goal. These prior techniques utilize CBFs to keep an isolated point in state space away from the unsafe set. However, various situations require a non-singleton set of states to be kept away from an unsafe set. Prior literature has addressed this problem using nonsmooth CBF methods, but no prior work has solved this problem using only "smooth" CBF methods. This paper addresses this gap by presenting a novel method of applying CBF methods to non-singleton parameterized convex sets. The method ensures differentiability of the squared distance function between ego and obstacle sets by leveraging a form of the log-sum-exp function to form strictly convex, arbitrarily tight overapproximations of these sets. Safety-preserving control inputs can be computed via convex optimization formulations. The efficacy of our results is demonstrated through multi-agent simulations.

Keywords: Constrained control, Autonomous robots, Autonomous systems
Abstract: This paper introduces the notion of a Rate-Tunable Control Barrier Function (RT-CBF), which allows for online tuning of the response of CBF-based controllers by adapting the parameters of the class-K function that is involved in the CBF condition. In contrast to most existing approaches that use a fixed class-K function to ensure safety, we propose an online adaptation law for the parameters of class-K function of the CBF condition. The algorithm can also incorporate multiple higher-order CBFs and adapt their class-K parameters for feasibility, albeit without guarantees in theory. The simulation results verify that online adaptation helps improve the system’s response in terms of the resulting trajectories being closer to a nominal reference while still being safe.

Keywords: Constrained control, Autonomous robots, Lyapunov methods
Abstract: We propose a novel zero-order control barrier function (ZOCBF) for sampled-data systems to ensure system safety. Our formulation generalizes conventional control barrier functions and straightforwardly handles safety constraints with high-relative degrees or those that explicitly depend on both system states and inputs. The proposed ZOCBF condition does not require any differentiation operation. Instead, it involves computing the difference of the ZOCBF values at two consecutive sampling instants. We propose three numerical approaches to enforce the ZOCBF condition, tailored to different problem settings and available computational resources. We demonstrate the effectiveness of our approach through a collision avoidance example and a rollover prevention example on uneven terrains.

Keywords: Constrained control, Autonomous systems, Estimation
Abstract: This paper develops a new framework for preventing localization failures in mobile systems that estimate their state using measurements. Safety is guaranteed by imposing that the nonlinear least squares optimization, solved in modern localization algorithms, remains well-conditioned. Specifically, the eigenvalues of the Hessian matrix are made to be positive via two methods that leverage control barrier functions to achieve safe set invariance. The proposed method is not constrained to any specific measurement or system type, offering a general solution to the safe mobility with localization problem. The efficacy of the approach is demonstrated on a system provided range-only and heading-only measurements for localization.

Keywords: Constrained control, Optimization algorithms, Lyapunov methods
Abstract: Obtaining a controlled invariant set is crucial for safety-critical control with control barrier functions (CBFs) but is non-trivial for complex nonlinear systems and constraints. Backup control barrier functions allow such sets to be constructed online in a computationally tractable manner by examining the evolution (or flow) of the system under a known backup control law. However, for systems with unmodeled disturbances, this flow cannot be directly computed, making the current methods inadequate for assuring safety in these scenarios. To address this gap, we leverage bounds on the nominal and disturbed flow to compute a forward invariant set online by ensuring safety of an expanding norm ball tube centered around the nominal system evolution. We prove that this set results in robust control constraints which guarantee safety of the disturbed system via our Disturbance-Robust Backup Control Barrier Function (DR-bCBF) solution. Additionally, the efficacy of the proposed framework is demonstrated in simulation, applied to a double integrator problem and a rigid body spacecraft rotation problem with rate constraints.

Keywords: Constrained control, Stability of nonlinear systems, Adaptive control
Abstract: In this work, we explore the application of barrier states (BaS) in the realm of safe nonlinear adaptive control. Our proposed framework derives barrier states for systems with parametric uncertainty, which are augmented into the uncertain dynamical model. We employ an adaptive nonlinear control strategy based on a control Lyapunov functions approach to design a stabilizing controller for the augmented system. The developed theory shows that the controller ensures safe control actions for the original system while meeting specified performance objectives. We validate the effectiveness of our approach through simulations on diverse systems, including a planar quadrotor subject to unknown drag forces and an adaptive cruise control system, for which we provide comparisons with existing methodologies.

Keywords: Constrained control
Abstract: Enforcing multiple constraints based on the concept of control barrier functions (CBFs) is a remaining challenge because each of the CBFs requires a condition on the control inputs to be satisfied which may easily lead to infeasibility problems. The problem becomes even more challenging with input constraints and disturbances. In this paper, we consider enforcement of bounding box constraints for a second order system under limited control authority and input disturbances. To solve the constrained control problem, we apply multiple robust control barrier functions (RCBFs) which, in general, do not provide a feasible solution to the problem. However, we derive conditions on how to select the RCBF parameters to guarantee that a feasible solution always exists.

Keywords: Constrained control, Predictive control for nonlinear systems, Robotics
Abstract: This paper presents a model predictive control (MPC) with recursive feasibility guarantees for robots with LiDAR sensory measurements. Control barrier functions (CBF) are incorporated within the MPC framework to shorten the prediction horizon of MPC compared to the standard MPC, effectively reducing computational complexity while ensuring the avoidance of unsafe sets. A CBF is synthesized from 2D LiDAR data points by first clustering obstacles using the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm and then fitting an ellipse to each cluster using the OpenCV library’s fitting tool. The resulting CBFs from each obstacle are subsequently unified and integrated into the MPC framework. The recursive feasibility of the proposed MPC is analyzed and guaranteed by choosing an appropriate terminal set. The effectiveness of the approach is demonstrated through simulations in the Gazebo robotic simulator using ROS 2, followed by experimental validation with a unicycle-type robot equipped with a LiDAR sensor.

Keywords: Distributed control, Mean field games, Constrained control
Abstract: Control Barrier Functions (CBFs) are an effective methodology to ensure safety and performative efficacy in real-time control applications such as power systems, resource allocation, autonomous vehicles, robotics, etc. This approach ensures safety independently of the high-level tasks that may have been pre-planned off-line. For example, CBFs can be used to guarantee that a vehicle will remain in its lane. However, when the number of agents is large, computation of CBFs can suffer from the curse of dimensionality in the multi-agent setting. In this work, we present Mean-field Control Barrier Functions (MF-CBFs), which extends the CBF framework to the mean-field (or swarm control) setting. The core idea is to model a population of agents as probability measures in the state space and build corresponding control barrier functions. Similar to traditional CBFs, we derive safety constraints on the (distributed) controls but now relying on the differential calculus in the space of probability measures.

Keywords: Hybrid systems, Optimization
Abstract: Barrier certificates provide an effective automated approach to verifying the safety of dynamical systems. A barrier certificate is a real-valued function over states of the system whose zero level set separates the unsafe region from all possible trajectories starting from a given set of initial states. Typically, the system dynamics must be nonincreasing in the value of the barrier certificate with each transition. Thus, the states of the system that are nonpositive with respect to the barrier certificate act as an over-approximation of the reachable states. The search for such certificates is typically automated by first fixing a template of functions and then using optimization and satisfiability modulo theory (SMT) solvers to find them. Unfortunately, it may not be possible to find a single function in this fixed template. To tackle this challenge, we propose the notion of interpolation-inspired barrier certificate. Instead of a single function, an interpolation-inspired barrier certificate consists of a set of functions such that the union of their sublevel sets over-approximate the reachable set of states. We show how one may find such interpolation-inspired barrier certificates of a fixed template, even when we fail to find standard barrier certificates of the same template. We present sum-of-squares (SOS) programming as a computational method to find this set of functions and demonstrate effectiveness of this method over a case study.

Keywords: Lyapunov methods, Constrained control, Control applications
Abstract: In this letter, we address the critical trade-off between safety and performance in control systems by establishing the contractivity of a class of nonlinear systems driven by control barrier function (CBF)-based online feedback optimization. First, we derive a closed-form solution for the control system driven by a CBF-based controller with vector-valued safety constraints. Next, we introduce sufficient design conditions based on the properties of a baseline controller and CBF parameters to ensure both safety and contractivity of the closed-loop system. Under these conditions, we demonstrate the existence of an exponentially stable equilibrium within the safe set and provide an explicit term for the rate of convergence. Building upon these results, we propose a feedback motion planning algorithm that guarantees a global region of attraction within non-convex search areas through a tree of contractive controllers. The contractive nature of our approach ensures robustness against perturbations, making it suitable for dynamic and uncertain environments.

Keywords: Stochastic systems, Hybrid systems, Automata
Abstract: This paper addresses the problem of synthesizing controllers that enforce properties expressed by Universal Co-Büchi Automata (UCA) over stochastic control systems. Our approach introduces a notion of Stochastic Co-Büchi Control Barrier Certificates (SCBC), which, together with their associated controllers, ensure that specific regions in the state set are visited only a limited number of times during the system’s evolution. The SCBC is formulated over a hybrid domain that combines the system’s state, the UCA’s state, and a counter variable that tracks the number of visits to the UCA’s accepting states. We require the SCBC to satisfy a supermartingale condition, thereby, enforcing the property expressed by the UCA on the stochastic control system without any restriction over the time horizon. Additionally, we propose a method for constructing SCBCs and corresponding controllers that guarantee the enforcement of UCA properties over stochastic control systems with formal probabilistic guarantee. The practical applicability of our approach is demonstrated through a case study involving a stochastic three-tank system, whose dynamics is both nonlinear and influenced by noise.

Keywords: Optimal control, Lyapunov methods, Predictive control for nonlinear systems
Abstract: Dynamic obstacle avoidance is a challenging topic for optimal control and optimization-based trajectory planning problems. Many existing works use Control Barrier Functions (CBFs) to enforce safety constraints for control systems. CBFs are typically formulated based on the distance to obstacles, or integrated with path planning algorithms as a safety enhancement tool. However, these approaches usually require knowledge of the obstacle boundary equations or have very slow computational efficiency. In this paper, we propose a framework based on model predictive control (MPC) with discrete-time high-order CBFs (DHOCBFs) to generate a collision-free trajectory. The DHOCBFs are first obtained from convex polytopes generated through grid mapping, without the need to know the boundary equations of obstacles. Additionally, a path planning algorithm is incorporated into this framework to ensure the global optimality of the generated trajectory. We demonstrate through numerical examples that our framework allows a unicycle robot to safely and efficiently navigate tight, dynamically changing environments with both convex and nonconvex obstacles. By comparing our method to established CBF-based benchmarks, we demonstrate superior computing efficiency, length optimality, and feasibility in trajectory generation and obstacle avoidance.

Keywords: Robust control, Uncertain systems, Autonomous systems
Abstract: In safety-critical control, managing safety constraints with high relative degrees and uncertain obstacle dynamics pose significant challenges in guaranteeing safety performance. Robust Control Barrier Functions (RCBFs) offer a potential solution, but the non-smoothness of the standard RCBF definition can pose a challenge when dealing with multiple derivatives in high relative degree problems. As a result, the definition was extended to the marginally more conservative smooth Robust Control Barrier Functions (sRCBF). Then, by extending the sRCBF framework to the CBF backstepping method, this paper offers a novel approach to these problems. Treating obstacle dynamics as disturbances, our approach reduces the requirement for precise state estimations of the obstacle to an upper bound on the disturbance, which simplifies implementation and enhances the robustness and applicability of CBFs in dynamic and uncertain environments. Then, we validate our technique through an example problem in which an agent, modeled using a kinematic unicycle model, aims to avoid an unknown moving obstacle. The demonstration shows that the standard CBF backstepping method is not sufficient in the presence of a moving obstacle, especially with unknown dynamics. In contrast, the proposed method successfully prevents the agent from colliding with the obstacle, proving its effectiveness.

Keywords: Traffic control, Cooperative control, Reinforcement learning
Abstract: The control of mixed-autonomy platoons comprising both connected and automated vehicles (CAVs) and human-driven vehicles (HDVs) has attracted increasing research attention. Multi-agent reinforcement learning (MARL) appears to be a promising control strategy for its capability of managing complex scenarios in real time. However, current research on MARL-based mixed-autonomy platoon control suffers from the following limitations. First, existing MARL approaches lack theoretical safety guarantees, as they typically address safety by adding penalty terms for violations in the reward function, which does not ensure safety due to the black-box nature of RL. Second, few studies have explored the cooperative safety of multi-CAV platoons, where CAVs can be coordinated to further enhance the system-level safety involving the safety of both CAVs and HDVs. To address these limitations, we (i) design cooperative control barrier function (CBF) candidates to enhance system-level safety and (ii) integrate the safety constraints into a Multi-Agent Reinforcement Learning (MARL) framework through a differentiable quadratic programming layer. Simulation results show that our proposed control strategy can effectively enhance the system-level safety of a mixed-autonomy platoon through the cooperation of multiple CAVs.

Keywords: Constrained control, Lyapunov methods
Abstract: This paper presents a novel approach for synthesizing control barrier functions (CBFs) from high relative degree safety constraints: Rectified CBFs (ReCBFs). We begin by discussing the limitations of existing High-Order CBF approaches and how these can be overcome by incorporating an activation function into the CBF construction. We then provide a comparative analysis of our approach with related methods, such as CBF backstepping. Our results are presented first for safety constraints with relative degree two, then for mixed-input relative degree constraints, and finally for higher relative degrees. The theoretical developments are illustrated through simple running examples and an aircraft control problem.

Keywords: Nonholonomic systems, Optimal control, Robotics
Abstract: A turn constrained vehicle is initially located inside a region described as a convex polygon and desires to escape in minimum time. First, the method of characteristics is used to describe the time-optimal strategies for reaching a line. Next, the approach is extended to polygons constructed of a series of line segments. Using this construction technique, the min-time path to reach each edge is obtained; the resulting minimum of the set of optimal trajectories is then selected for escaping the polygon.

Keywords: Predictive control for nonlinear systems, Robotics, Robust control
Abstract: Image-based visual servoing (IBVS) applications for autonomous underwater vehicles (AUVs) face significant challenges, including frequent recalibration and lack of constraint handling ability. This paper introduces a novel nonlinear model predictive control (NMPC) approach that integrates the Broyden method for uncalibrated IBVS and incorporates the min-max strategy to tolerate the errors in Jacobian matrix estimation. Our proposed min-max NMPC-IBVS framework estimates the Jacobian matrix online, allowing for continuous adaptation to the underwater environment without the need for prior calibration. This approach significantly enhances computational efficiency and robust control performance, enabling real-time uncalibrated applications. A rigorous proof of recursive feasibility is provided in this work, ensuring that our NMPC-IBVS method consistently finds feasible optimal solutions that satisfy all constraints over time. Simulation results show that the proposed method is able to respect all design constraints in the AUV IBVS control and achieve robust stability with boosted computational efficiency.

Keywords: Robotics, Estimation, Predictive control for nonlinear systems
Abstract: Robots rely on motion planning to navigate safely and efficiently while performing various tasks. In this paper, we investigate motion planning through Bayesian inference, where motion plans are inferred based on planning objectives and constraints. However, existing Bayesian motion planning methods often struggle to explore low-probability regions of the planning space, where high-quality plans may reside. To address this limitation, we propose the use of heavy-tailed distributions---specifically, Student's-t distributions---to enhance probabilistic inferential search for motion plans. We develop a novel sequential single-pass smoothing approach that integrates Student's-t distribution with Monte Carlo sampling. A special case of this approach is ensemble Kalman smoothing, which depends on short-tailed Gaussian distributions. We validate the proposed approach through simulations in autonomous vehicle motion planning, demonstrating its superior performance in planning, sampling efficiency, and constraint satisfaction compared to ensemble Kalman smoothing. While focused on motion planning, this work points to the broader potential of heavy-tailed distributions in enhancing probabilistic decision-making in robotics.

Keywords: Robotics, Uncertain systems, Kalman filtering
Abstract: In recent years, an increasing number of service robots have been deployed in human environments. The growing complexity of modern robotic systems and the environments in which they operate necessitates careful consideration of safety, increasing the challenges associated with socially responsible navigation. Moreover, localization measurements are often affected by random noise, which can lead to unsafe behavior if not addressed in the control design. Therefore, it is crucial to ensure robustness against measurement noise. This paper addresses navigation challenges by presenting a robust CBF-based STL motion planning algorithm, integrated with an unscented Kalman filter. This methodology mitigates the effects of state disturbances and measurement noise, ensuring task completion at any time within a specified time interval for dynamic systems subject to nonlinear velocity constraints and collision avoidance. A simulation study is conducted to validate the methodology, demonstrating its ability to ensure safety.

Keywords: Mechanical systems/robotics, Optimal control, Robotics
Abstract: In this paper, a “risk set” is introduced within the safe set of a safety-critical nonlinear tracking controller to en- hance performance while avoiding overly conservative behavior. Utilizing a barrier function approach within a quadratic cost framework, this set is incorporated into the State-Dependent Riccati Equation (SDRE) for the inner control loop of Uncrewed Aerial Vehicles (UAVs), ensuring safe flight control of quadro- tors. Additionally, a sliding mode controller is designed for the outer loop to generate the desired trajectory for the inner loop. The risk set further enables systematic switching of weighting adjustments in the nonlinear tracking SDRE controller to balance system performance and safety requirements. This approach is compared to that for scenarios without switching weighting adjustments, and the results demonstrate that the proposed controller satisfies safety constraints while delivering improved performance.

Index Terms— safety critical tracking controller, SDRE, UAVs, time-varying desired trajectory, bar- rier functions

Keywords: Robotics, Reinforcement learning, Optimal control
Abstract: Nonlinear model predictive control (NMPC) is typically restricted to short, finite horizons to limit the computational burden of online optimization. As a result, global planning frameworks are frequently necessary to avoid local minima when using NMPC for navigation in complex environments. By contrast, reinforcement learning (RL) can generate policies that minimize the expected cost over an infinite-horizon and can often avoid local minima, even when operating only on current sensor measurements. However, these learned policies are usually unable to provide performance guarantees (e.g., on collision avoidance), especially when outside of the training distribution. In this paper, we augment Probably Approximately Correct NMPC (PAC-NMPC), a sampling-based stochastic NMPC algorithm capable of providing statistical guarantees of performance and safety, with an approximate perception-based value function trained via RL. We demonstrate in simulation that our algorithm can improve the long-term behavior of PAC-NMPC while outperforming other approaches with regards to safety for both planar car dynamics and more complex, high-dimensional fixed-wing aerial vehicle dynamics. We also demonstrate that, even when our value function is trained in simulation, our algorithm can successfully achieve statistically safe navigation on hardware using a 1/10th scale rally car in cluttered real-world environments using only current sensor information.

Keywords: Distributed control, Large-scale systems, Optimal control
Abstract: Robotic swarms, also known as very large-scale robotic (VLSR) systems, have numerous applications in complex tasks. However, as the number of robots increases, the complexity of motion control and energy costs escalate rapidly. In addressing this problem, our previous studies have formulated various methods employing macroscopic and microscopic approaches. These methods enable microscopic robots to adhere to a reference Gaussian mixture model (GMM) distribution observed at the macroscopic scale. As a result, optimizing the macroscopic level will result in an optimal overall result. However, all these methods require systematic and global generation of Gaussian components (GCs) within obstacle-free areas to construct the GMM trajectories. This work utilizes centroidal Voronoi tessellation to generate GCs methodically. Consequently, it demonstrates performance improvement while also ensuring consistency and reliability.

Keywords: Flight control, Reinforcement learning, Robotics
Abstract: Bird-sized flapping-wing robots offer significant potential for agile flight in complex environments, but achieving agile and robust trajectory tracking remains a challenge due to the complex aerodynamics and highly non-linear dynamics inherent in flapping-wing flight. In this work, a learning-based control approach is introduced to unlock the versatility and adaptiveness of flapping-wing flight. We propose a model-free reinforcement learning (RL)-based framework for a high degree-of-freedom (DoF) bird-inspired flapping-wing robot that allows for multimodal flight and agile trajectory tracking. Stability analysis was performed on the closed-loop system comprising of the flapping-wing system and the RL policy. Additionally, simulation results demonstrate that the RL-based controller can successfully learn complex wing trajectory patterns, achieve stable flight, switch between flight modes spontaneously, and track different trajectories under various aerodynamic conditions.

Keywords: Emerging control applications, Reinforcement learning, Robotics
Abstract: Recent advancements in reinforcement learning have enabled the development of model-free controllers for a wide range of applications, including quadrotor aerial vehicles. However, a significant limitation of learning-based controllers is their inability to account for variations in system parameters, leading to sub-optimal performance when deployed in realworld. Typically, these variations stem from the inaccurate synthetic training environment. To address this limitation, this work develops an integrated control architecture that combines a reinforcement learning agent with a model reference adaptive controller (MR2C) for linear systems. This hybrid approach allows the controller to adapt to changes in system dynamics in real time. The proposed framework is validated through both simulations and real flight experiments using a Parrot Mambo quadrotor aerial vehicle for a hovering task. In these experiments, model uncertainties is introduced by varying the quadrotor’s mass and flying near the ground to incorporate unmodeled ground effects. Results demonstrate a significant improvement in tracking performance when handling parametric variations along-with faster rise-times with the MR2C controller compared to the conventional controllers and standalone learning-based approaches.

Keywords: Robotics, Sensor fusion, Autonomous robots
Abstract: Real time wildfire prediction and estimation is paramount in the practice of wildland fire management. In this paper we propose a method of autonomous wildland fire monitoring via integrated path planning and sensor fusion. Accurate localized measurements made by a Unmanned Aerial System (UAS) equipped with a radiometric infrared camera are fused with a low fidelity Reaction Diffusion fire model using an Ensemble Kalman Filter. We develop a reward function defined by the variance of the EnKF. We pose a fixed time horizon reward maximization path planning problem to determine what sequence of measurements the UAS is to take, and solve it using a dynamic programming algorithm. We demonstrate the closed loop performance of the path planning and sensor fusion framework using real infrared video of a prescribed prairie burn to simulate a wildland fire.

Keywords: Robotics, Autonomous robots, Control system architecture
Abstract: This paper presents a nonlinear control strategy for an aerial cooperative payload transportation system consisting of two quadrotor UAVs rigidly connected to a payload. The system includes human physical interaction facilitated by an admittance control. The proposed control framework integrates an adaptive Backstepping controller for the position subsystem and a Fast Nonsingular Terminal Sliding Mode Control (FNTSMC) for the attitude subsystem to ensure asymptotic stabilization. The admittance controller interprets the interaction forces from the human operator, generating reference trajectories for the position controller to ensure accurate tracking of the operator’s guidance. The system aims to assist humans in payload transportation, providing both stability and responsiveness. The robustness and effectiveness of the proposed control scheme in maintaining system stability and performance under various conditions are presented.

Keywords: Robotics, Fault detection, Mechanical systems/robotics
Abstract: We present a conceptual framework for an autonomous safety mechanism designed to enhance the reliability of Unmanned Aerial Vehicles (UAVs) that use Visual-Inertial Odometry (VIO) for state estimation. As UAVs increasingly interact with the public, such safety mechanisms are crucial to reducing the likelihood and severity of accidents. VIO drift, which occurs when accumulated estimation errors cause discrepancies between the UAV’s perceived and actual position, poses a significant risk to safe operation. To address this challenge, we propose a Kalman filter-based approach for detecting VIO drift events. Upon detection, the envisioned safety mechanism is designed to adjust state estimation by integrating onboard gyroscope measurements and thrust commands for short durations, aiming to enhance stability and prevent potential crashes before initiating a controlled landing. While this framework provides the foundation for a real-time safety mechanism, the implementation and experiment focus on validating the drift detection component in an offline setting using real UAV flight data. The results demonstrate the effectiveness of the detection method in identifying VIO drift scenarios, highlighting its potential for future real-time applications.

Keywords: Agents-based systems, Cooperative control, Networked control systems
Abstract: This article presents a novel time-coordination algorithm based on event-triggered communication to ensure multiple UAVs progress along their desired paths in coordination with one another. In the proposed algorithm, a UAV transmits its progression information to its neighbor UAVs only when a decentralized trigger condition is satisfied. Consequently, it significantly reduces the volume of inter-vehicle communications required to achieve the goal compared with the existing algorithms based on continuous communication. With such intermittent communications, it is shown that a decentralized coordination controller guarantees exponential convergence of the coordination error to a neighborhood of zero. Furthermore, a lower bound on the difference between two consecutive event-triggered times is provided showing that the Zeno behavior is excluded with the proposed algorithm. Lastly, simulation results validate the efficacy of the proposed algorithm.

Keywords: Autonomous systems, Multivehicle systems, Cooperative control
Abstract: Formation flying, particularly in swarms, has become a prominent area of interest for UAV applications. While rigid formation shapes offer several advantages, they also increase the likelihood of detection by optical or radar systems. Drawing inspiration from materials science and condensed matter physics, this paper proposes a method to provide to a formation an amorphous appearance, thereby reducing the swarm’s detectability. The proposed technique involves the introduction of a virtual obstacle that performs stochastic movements within the formation. Two simple metrics are introduced to evaluate the effectiveness of this approach: one assesses the formation’s resemblance to a rigid body, while the other measures the internal velocity correlation among the UAVs. Simulation results validate the effectiveness of this method.

Keywords: Networked control systems, Communication networks
Abstract: This paper explores the application of multi-loop Wireless Networked Control Systems (WNCS) for managing Unmanned Aerial Vehicle (UAV) formations. A multi-antenna base station (BS) employs beamforming (BF) technology to communicate with and control multiple UAVs simultaneously. Given the inherent uncertainty and stochastic nature of wireless channels and control system dynamics, we propose a joint communication-control strategy to ensure an accurate, stable, and efficient system. Our objective is to minimize the weighted sum of state error and communication costs while maintaining system stability and adhering to transmission power constraints by adapting BF weight vectors. The benefits of the BF design are twofold: first, a good trade-off between communication costs and control stability is achieved due to the array gain from the multi-antenna controller. Second, the generation of multiple beams to control multiple UAVs simultaneously effectively resolves scheduling issues, enhancing the efficiency of limited radio spectrum and energy consumption. To address the resulting non-trivial probabilistic optimization problem, we adopt the Bernstein outage probability theorem to construct a semidefinite program relaxation. Simulation results confirm the effectiveness of the proposed joint strategy.

Keywords: Optimal control, Flight control, Aerospace
Abstract: In this paper, we address the Unmanned Aerial Vehicle trajectory (UAV) tracking and obstacle avoidance problem, by proposing a novel Chebyshev pseudospectral method-based Model Predictive Control (MPC) formulation that is real-time implementable. In this formulation, a continuous-time integral form of the quadratic tracking error cost function is considered and its exact solution is obtained by the Clenshaw-Curtis quadrature rule. The state and control histories are approximated by Lagrange interpolating polynomials, with their coefficients as decision variables. These polynomials are proven to yield smooth control histories, unlike piecewise constant control inputs in the standard MPC. The collocation method is used to satisfy the dynamics and avoid computationally expensive numerical integration. This also allows us for a longer prediction horizon with the same number of decision variables without affecting the computational speed. The MPC is designed by considering the translational dynamics for determining acceleration inputs, which are subsequently used by the low-level controller to obtain desired thrust, orientation, and angular speeds. The performance of the proposed MPC is validated by indoor experiments.

Keywords: Kalman filtering, Uncertain systems, Mechanical systems/robotics
Abstract: In this paper, we consider a position estimation problem for an unmanned aerial vehicle (UAV) equipped with both proprioceptive sensors, i.e. IMU, and exteroceptive sensors, i.e. GPS and a barometer. We propose a data-driven position estimation approach based on a robust estimator which takes into account that the UAV model is affected by uncertainties and thus it belongs to an ambiguity set. We propose an approach to learn this ambiguity set from the data.

Keywords: Robust adaptive control, Adaptive systems, Robotics
Abstract: This paper introduces an adaptive-neuro geometric control for a centralized multi-quadrotor cooperative transportation system, which enhances both adaptivity and disturbance rejection. Our strategy is to coactively tune the model parameters and learn the external disturbances in real-time. To realize this, we augmented the existing geometric control with multiple neural networks and adaptive laws, where the estimated model parameters and the weights of the neural networks are simultaneously tuned and adjusted online. The Lyapunov-based adaptation guarantees bounded estimation errors without requiring either pre-training or the persistent excitation (PE) condition. The proposed control system has been proven to be stable in the sense of Lyapunov under certain preconditions, and its enhanced robustness under scenarios of disturbed environment and model-unmatched plant was demonstrated by numerical simulations.

Keywords: Aerospace, Autonomous systems, Multivehicle systems
Abstract: We consider the problem of transporting multiple packages from an initial location to a destination location in a windy urban environment using a team of SUAVs. Each SUAV carries one package. We assume that the wind field is unknown, but wind speed can be measured by SUAVs during flight. The SUAVs fly sequentially one after the other, measure wind speeds along their trajectories, and report the measurements to a central computer. The overall objective is to minimize the total travel time of all SUAVs, which is in turn related to the number of SUAV traversals through the environment. For a discretized environment modeled by a graph, we describe a method to estimate wind speeds and the time of traversal for each SUAV path. Each SUAV traverses a minimum-time path planned based on the current wind field estimate. We study cases of static and time-varying wind fields with and without measurement noise. For each case, we demonstrate via numerical simulation that the proposed method finds the optimal path after a minimal number of traversals.

Keywords: Fault tolerant systems, Flight control
Abstract: Fault-tolerant control of a quadrotor in extreme conditions, such as rotor failure and strong winds, is exceptionally challenging due to its underactuated nature, strong mismatched disturbances, and highly nonlinear multi-input and multi-output properties. This letter proposes a reduced attitude control approach that combines a model-based extended state observer (MB-ESO) and mismatched disturbance decoupling to control a quadrotor under strong winds and complete loss of two opposing rotors. Our MB-ESO based control provides a new theoretical framework for more general nonlinear systems by utilizing all measurable outputs, thereby maximizing the use of all available information to design a robust controller. Testing in a high-fidelity simulator shows that our approach outperforms the state-of-the-art Incremental Nonlinear Dynamic Inversion method.

Keywords: Optimization, Autonomous systems, Robotics
Abstract: Online trajectory generation for unmanned aerial vehicles has attracted increased interest for a variety of autonomous applications. This paper explores the use of non-uniform B-splines to produce improved time-optimal trajectories in comparison to contemporary work that uses uniform B-splines. This paper introduces the non-uniform B-spline matrix form and the transformation of non-uniform B-spline control points to other parametric representations such as MINVO and B'ezier control points. Conversion between control point representations allows the optimization to take advantage of the piecewise continuity of B-splines, as well as the tighter convex bounding properties of MINVO and B'ezier curve representations. The results show that non-uniform B-spline trajectories significantly outperform uniform B-splines trajectories in time optimality and path length. However, computational performance is degraded when optimizing for non-uniform B-splines, thus requiring computational improvements for online use.

Keywords: Autonomous vehicles, Optimal control, Optimization
Abstract: In this paper, we propose a bi-stage optimal trajectory planning method to address the challenges of planning in unstructured environments. In the first stage, we leverage the incremental sampling method, optimal rapidly-exploring random tree star, combined with the Reeds-Shepp curve, to identify a feasible path in complex environments. The generated path is then smoothed using cubic spline interpolation. In the second stage, we address collision avoidance from multiple obstacles by first reducing the obstacle regions and then transforming the non-convex obstacle constraints into convex form through dual reformulation. The formulated optimal control problem is transcribed into a nonlinear programming and solved using a direct method, embedding the smoothed first-stage path as a feasible initial guess to generate optimal trajectories. We demonstrate the applicability of our approach in diverse environments, showing reduced computation time compared to representative methods due to fewer collision evaluations and improved feasibility. Additionally, the generated trajectories account for passenger comfort while minimizing travel time.

Keywords: Robotics, Feedback linearization, Reduced order modeling
Abstract: This work presents the study on the controllability and input-output stability of an underactuated soft robotic exoskeleton digit, made of three individual soft actuator segments, for guiding a human finger to track desired trajectories. A quasi-static analytical model of the underactuated soft robot interacting with a human finger was developed in terms of the joint variables, i.e. the arc length and bending angle. The analytical equations were investigated to determine the existence of control solutions (actuation pressure) in the joint space of each soft actuator. A nonlinear time-discrete state-space model was derived based on the quasi-static motion of the coupled human-soft-robot model. The underactuated system's controllability, partial feedback linearizability, and stability were studied. A partial feedback linearizing control law was derived, and a secondary controller was designed to guarantee the asymptotic stability of the input-output linearized subsystem. Then, the local stability of the zero dynamics was shown, and necessary and sufficient conditions were determined. The controller performance in following desired trajectories was validated in the simulation studies and compared to a standard PID controller.

Keywords: Machine learning, Numerical algorithms, Optimization
Abstract: The recent success of diffusion-based generative models in image and natural language processing has ignited interest in diffusion-based trajectory optimization for nonlinear control systems. Existing methods cannot, however, handle the nonlinear equality constraints necessary for direct trajectory optimization. As a result, diffusion-based trajectory optimizers are currently limited to shooting methods, where the nonlinear dynamics are enforced by forward rollouts. This precludes many of the benefits enjoyed by direct methods, including flexible state constraints, reduced numerical sensitivity, and easy initial guess specification. In this paper, we present a method for diffusion-based optimization with equality constraints. This allows us to perform direct trajectory optimization, enforcing dynamic feasibility with constraints rather than rollouts. To the best of our knowledge, this is the first diffusion-based optimization algorithm that supports the general nonlinear equality constraints required for direct trajectory optimization.

Keywords: Manufacturing systems, Optimal control, Materials processing
Abstract: Metal additive manufacturing (AM) opens the possibility for spatial control of as-fabricated microstructure and properties. However, since the solid state diffusional transformations that drive microstructure outcomes are governed by nonlinear ODEs in terms of temperature, which is itself governed by PDEs over the entire part domain, solving for the system inputs needed to achieve desired microstructure distributions has proven difficult. In this work, we present a trajectory optimization approach for spatial control of microstructure in metal AM, which we demonstrate by controlling the hardness of a low-alloy steel in electron beam powder bed fusion (EB-PBF). To this end, we present models for thermal and microstructural dynamics. Next, we use experimental data to identify the parameters of the microstructure transformation dynamics. We then pose spatial microstructure control as a finite-horizon optimal control problem. The optimal power field trajectory is computed using an augmented Lagrangian differential dynamic programming (AL-DDP) method with GPU acceleration. The resulting time-varying power fields are then realized on an EB-PBF machine through an approximation scheme. Measurements of the resultant hardness shows that the optimized power field trajectory is able to closely produce the desired hardness distribution.

Keywords: Optimal control, Optimization algorithms, Optimization
Abstract: Nonconvex trajectory optimization is at the core of designing trajectories for complex autonomous systems. A challenge for nonconvex trajectory optimization methods, such as sequential convex programming, is to find an effective warm-starting point to approximate the nonconvex optimization with a sequence of convex ones. We introduce a first-order method with filter-based warm-starting for nonconvex trajectory optimization. The idea is to first generate sampled trajectories using constraint-aware particle filtering, which solves the problem as an estimation problem. We then identify different locally optimal trajectories through agglomerative hierarchical clustering. Finally, we choose the best locally optimal trajectory to warm-start the prox-linear method, a first-order method with guaranteed convergence. We demonstrate the proposed method on a multi-agent trajectory optimization problem with linear dynamics and nonconvex collision avoidance. Compared with sequential quadratic programming and interior-point method, the proposed method reduces the objective function value by up to approximately 96% within the same amount of time for a two-agent problem, and 98% for a six-agent problem.

Keywords: Reinforcement learning, Process Control, Hierarchical control
Abstract: Traditional Reinforcement Learning (RL) methods typically assume that actions are executed instantly, and that the agent receives immediate feedback in terms of state and reward information. However, in many real-world systems, delays in action execution and feedback are common, making these assumptions impractical. While some existing RL methods address scenarios with constant and known delays, this paper proposes a hybrid RL approach designed to handle unknown but bounded delays. The approach introduces two operational modes, each managed by a specialized agent. Initially, a delay-safe agent guarantees system stability regardless of delays, ensuring safety during a delay identification phase. Once the delay is estimated, control is transferred to a delay-informed agent that optimizes performance based on the identified delay. The transition between the two agents is managed through a convex combination of their actions, with increasing emphasis on the delay-informed agent as the accuracy of the delay estimator improves.

Keywords: Reinforcement learning, Stochastic optimal control, Computational methods
Abstract: In this study, we consider the application of max-plus-linear approximators for Q-function in offline reinforcement learning of discounted Markov decision processes. In particular, we incorporate these approximators to propose novel fitted Q-iteration (FQI) algorithms with provable convergence. Exploiting the compatibility of the Bellman operator with max-plus operations, we show that the max-plus-linear regression within each iteration of the proposed FQI algorithm reduces to simple max-plus matrix-vector multiplications. We also consider the variational implementation of the proposed algorithm which leads to a per-iteration complexity that is independent of the number of samples.

Keywords: Reinforcement learning, Transportation networks, Smart cities/houses
Abstract: As Machine Learning grows in popularity across various fields, equity has become a key focus for the AI community. However, fairness-oriented approaches are still underexplored in smart mobility. Addressing this gap, our study investigates the balance between performance optimization and algorithmic fairness in shared micromobility services providing a novel framework based on Reinforcement Learning. Exploiting Q-learning, the proposed methodology achieves equitable outcomes in terms of the Gini index across different areas characterized by their distance from central hubs. Through vehicle rebalancing, the provided scheme maximizes operator performance while ensuring fairness principles for users, reducing iniquity by up to 85% while only increasing costs by 30% (w.r.t. applying no equity adjustment). A case study with synthetic data validates our insights and highlights the importance of fairness in urban micromobility (source code: https://github.com/mcederle99/FairMSS.git)

Keywords: Optimal control, Reinforcement learning, Linear systems
Abstract: This paper presents a framework that integrates model-based and model-free optimal control methods for continuous-time linear systems. The control design starts with a controller derived from an available nominal model, and is subsequently augmented with a learned (model-free) controller component. We formulate an optimality-based condition that determines the switching instant from the purely model-based controller to the composite controller, which contains both model-based and model-free components. We do not impose any requirements on the model's precision; the switching condition accounts for real-time model mismatch, ensuring that a more misaligned model results in a faster switching time. Finally, to obtain the model-free component in the composite controller, we employ off-policy reinforcement learning (RL) where trajectory data is used to learn the optimal control augmentation.

Keywords: Optimal control, Reinforcement learning
Abstract: This paper addresses the infinite horizon optimal tracking control problem for partially uncertain control-affine nonlinear discrete-time (DT) systems, where the control input dynamics are known. Multi-layer critic and actor neural networks (MNNs) are utilized for online estimation of the infinite horizon value function and optimal control input. The NN weights are tuned online using a direct temporal difference error (TDE)-driven learning approach, which modifies the singular values of the gradient with respect to the NN weights to accelerate their convergence. The critic NN uses a novel experience replay technique to improve sample efficiency without introducing biased TDEs and guarantee the persistence of excitation (PE) condition. The tracking error and weight estimation errors are shown to be uniformly ultimately bounded (UUB) using Lyapunov analysis. The performance of the optimal tracking control scheme with experience replay is evaluated on a two-link robot manipulator and contrasted with model predictive control scheme with known dynamics.

Keywords: Computational methods, Evolutionary computing, Reinforcement learning
Abstract: The combination of Large Language Models (LLMs), systematic evaluation, and evolutionary algorithms has enabled breakthroughs in combinatorial optimization and scientific discovery. We propose to extend this powerful combination to the control of dynamical systems, generating interpretable control policies capable of complex behaviors. With our novel method, we represent control policies as programs in standard languages like Python. We evaluate candidate controllers in simulation and evolve them using a pre-trained LLM. Unlike conventional learning-based control techniques, which rely on black-box neural networks to encode control policies, our approach enhances transparency and interpretability. We still take advantage of the power of large AI models, but only at the policy design phase, ensuring that all system components remain interpretable and easily verifiable at runtime. Additionally, the use of standard programming languages makes it straightforward for humans to finetune or adapt the controllers based on their expertise and intuition. We illustrate our method through its application to the synthesis of an interpretable control policy for the pendulum swing-up and the ball in cup tasks. We make the code available at https://github.com/muellerlab/synthesizing_interpretable_control_policies.git

Keywords: Machine learning, Constrained control, Neural networks
Abstract: This tutorial paper focuses on safe physics-informed machine learning in the context of dynamics and control, providing a comprehensive overview of how to integrate physical models and safety guarantees. As machine learning techniques enhance the modeling and control of complex dynamical systems, ensuring safety and stability remains a critical challenge, especially in safety-critical applications like autonomous vehicles, robotics, medical decision-making, and energy systems. We explore various approaches for embedding and ensuring safety constraints, such as structural priors, Lyapunov functions, Control Barrier Functions, predictive control, projections, and robust optimization techniques, ensuring that the learned models respect stability and safety criteria. Additionally, we delve into methods for uncertainty quantification and safety verification, including reachability analysis and neural network verification tools, which help validate that control policies remain within safe operating bounds even in uncertain environments. The paper includes illustrative examples demonstrating the implementation aspects of safe learning frameworks that combine the strengths of data-driven approaches with the rigor of physical principles, offering a path toward the safe control of complex dynamical systems.

Keywords: Autonomous systems, Constrained control, Stochastic optimal control
Abstract: Physical systems in the real world are usually constrained due to different considerations. These constraints are closely related to the system safety and stability. In this letter we investigate the optimal stabilization control problem of stochastic time-delay systems under safety constraints. We first follow the Razumikhin approach to propose stochastic control Lyapunov and barrier functions, which result in the closed-form controllers for the stabilization and safety control individually. Next, based on the modification of the quadratic programming, an optimization problem is established to address the stabilization control under safe constraints. The optimal controller is derived explicitly in a switching form to tradeoff the stabilization and safety requirements. Finally, a numerical example is presented to illustrate the proposed control strategy.

Keywords: Autonomous systems, Cooperative control, Modeling
Abstract: As the transportation industry increasingly shifts towards electric vehicles (EVs), optimizing energy efficiency becomes crucial due to EVs' ongoing range limitations. Adaptive Cruise Control (ACC) improves driver comfort and safety by maintaining a preset speed and adjusting it to keep safe following distances between EVs. However, ACC tends to be less efficient and slower to react in traffic compared to Cooperative Adaptive Cruise Control (CACC), which leverages vehicle-to-vehicle (V2V) communication for quicker and smoother traffic flow. Therefore, this paper introduces a novel, practical, and energy-efficient Lyapunov-based CACC approach that guarantees safe following distances and facilitates smooth vehicle platooning for EVs. To address the absence of an accurate EV model, we initially performed experimental tests to derive a third-order differential equation representing EV acceleration dynamics and defined a novel controller for it. The proposed controller’s performance was then assessed, with an emphasis on energy efficiency and string stability, through both simulations and experimental validation.

Keywords: Autonomous systems, Mechatronics, Networked control systems
Abstract: Connectivity preservation is an important aspect of the formation control problem as it is necessary for the agents to be able to sense their respective neighbors' information to maintain the geometric shape and accomplish the desired task. Recently, bearing-only formation control problem has garnered significant research interest, albeit in the absence of any connectivity preservation constraint. Towards that end, we study the problem of bearing-only formation control in presence of connectivity and safety constraints for single and double integrator agents. Circular disk shaped agents are considered and two relative bearing vectors are sensed such that the angle information can be extracted. We construct a barrier Lyapunov function which then exploits these angles to enforce required constraints for maintaining connectivity and safety among neighbors. We propose gradient based bearing-only controllers and carry out the stability analyses. Simulation examples are provided to illustrate the effectiveness of the control laws.

Keywords: Autonomous vehicles, Distributed control, Observers for nonlinear systems
Abstract: This paper presents a distributed high-gain observer for vehicle platoons, where each observer ensures exponential convergence of full state estimation using local measurements and a strongly connected directed communication graph. Unlike previous studies, our approach incorporates the nonlinear longitudinal dynamics of the platoon, taking air resistance into account. By employing the Lipschitz condition and leveraging the structure of the nonlinearity, we derive a less conservative inequality for the nonlinear estimation error. Gain matrices are computed by solving linear matrix inequalities based on observability decomposition. Simulation results for a platoon validate the effectiveness of the proposed observer, showing improved convergence rates and estimation accuracy.

Keywords: Feedback linearization, Autonomous systems, Robotics
Abstract: In this letter we derive a tracking controller based on feedback linearization for a miniature blimp controlled by co-located fans on an undermounted gondola. We prove that feedback linearization of the underactuated blimp induces nontrivial zero dynamics corresponding to lightly damped oscillations in pitch and roll. To mitigate these oscillations we use high-order control barrier functions (HOCBFs) to limit the maximum allowable pitch and roll at the expense of tracking error. Experimental results are presented for three illustrative trajectories which demonstrate that the proposed controller outperforms a well-tuned LQR controller and a baseline nonlinear MPC controller, while the attitude oscillations are theoretically and empirically shown to be bounded by the HOCBFs.

Keywords: Autonomous vehicles, Variable-structure/sliding-mode control, Automotive control
Abstract: An approach for automated driving in highway scenarios based on Sliding Mode Control (SMC) methodologies supported by the use of Artificial Potential (APF) fields is presented. The use of APF allows us to propose an effective SMC solution based on the gradient tracking (GT) principle. In fact, SMC-GT approaches regard the APF gradient lines as the desired trajectories and the gradient is interpreted as a desired velocity to be tracked by the vehicle. In this regard, a novel formulation of the APF functions is introduced that exploits a sequence of attractive quadratic functions centered on the target points of the planned trajectory. In this way, the computation of the gradient is more effective and leads to more accurate trajectory tracking and convergence results. Furthermore, the use of the Super-Twisting (STW) algorithm improves the smoothness properties of the velocity reference and the control action. Extensive simulation tests as well as a sensitivity analysis show the effectiveness and the robustness properties of the proposed approach.

Keywords: Healthcare and medical systems
Abstract: Mixed venous oxygen saturation (SvO2) can play a pivotal role for patient monitoring and treatment in critical care and cardiopulmonary medicine. Unfortunately, its continuous measurement requires the use of invasive pulmonary artery catheters. This letter presents a novel population-informed personalized Gaussian sum extended Kalman filtering (PI-P-GSEKF) approach to continuous SvO2 estimation from arterial oxygen saturation (SpO2) measurement. The main challenge in SvO2 estimation is large inter-individual variability in the cardiopulmonary dynamics, which seriously deteriorates the efficacy of standard EKF. To cope with this challenge, we employ the GSEKF in which individual EKFs are designed using a mathematical model of cardiopulmonary dynamics whose operating points are selected from (i) population-level generative sampling (thus “population-informed”) and (ii) Markov chain Monte Carlo (MCMC) sampling based on a one-time SpO2-SvO2 measurement (thus “personalized”). Using the experimental data collected from 8 hypoxia trials in 4 large animals, we showed the ability of the PI-P-GSEKF to estimate SvO2 from SpO2 in comparison with its PI-EKF (EKF with population-level generative sampling as the source of process noise) and PI-GSEKF (GSEKF with population-level generative sampling alone) counterparts (average SvO2 root-mean-squared error: PI-EKF 4.7%, PI-GSEKF 4.3%, PI-P-GSEKF 3.0%). We also showed that population-level generative sampling and MCMC sampling both had respective roles in improving SvO2 estimation accuracy. In sum, the PI-P-GSEKF demonstrated its proof-of-principle to enable non-invasive continuous SvO2 estimation.

Keywords: Identification for control, Machine learning, Metabolic systems
Abstract: Glucose modelling is crucial for an efficient management of Type 1 Diabetes (T1D). In recent years Neural Networks (NNs) techniques, in particular Long Short-Term Memory networks (LSTMs), have shown very promising performances in glucose prediction. However, to use them in control application, these NNs have to be able to correctly learn the effect of each single input from the data. Considering that meals and insulin boluses characterizing the datasets occur simultaneously, the pure NNs are often unable to distinguish their effects. This work proposes two Physics-LSTM hybrid Models (PhLSTMs) merging a classical Black Box model (BB) with two LSTM-based approaches in order to obtain satisfactory prediction capabilities for single impulse responses to insulin/meal. The resulting PhLSTMs have been tested on a case study of 2 in silico patients showing satisfactory glycemic prediction performances both on real-life datasets and impulse responses to insulin/meal.

Keywords: Computational methods, Human-in-the-loop control, Biomedical
Abstract: Accurate upper extremity (UE) movement classification is essential for effective prosthetic design and rehabilitation. However, distinguishing movements involving simultaneous shoulder and elbow motions is challenging due to the complexity presented by the multiple degrees of freedom (DOF). This study proposes an innovative homogeneous dynamic classifier selection system (HDCS) that accurately distinguishes between single DOF (sDOF) and multi-DOF (mDOF) movements. It comprises two main components: a feature-optimized multiple classifier system (MCS) with either linear discriminant analysis (LDA) or decision tree (DT) classifiers and a dynamic classifier selection (DCS) mechanism. On average, the HDCS significantly improves the accuracy from a baseline model by 3.3% (p < 0.05) for the 5-movement classification scenario and 2.5% (p < 0.05) for the 4-movement classification scenarios. The proposed HDCS outperformed individual ML across various movement paradigms, demonstrating its adaptability to a wide range of UE motions, and offering advantages for prosthetics and rehabilitation devices in biomedical engineering

Keywords: Biomedical, Kalman filtering, Control applications
Abstract: The anesthetic drug propofol is commonly used to control hypnotic depth (suppression of awareness) in patients undergoing surgery or intensive care. In addition to manual titration, a model-based open-loop feed-forward strategy called target-controlled infusion (TCI) has attained some clinical popularity. Research on closed-loop control, with awareness estimates derived from an electroencephalogram (EEG), has proven feasible through several extensive clinical studies over the past decades. While TCI is vulnerable to model imperfections, closed-loop control is susceptible to corrupt measurements. By combining Kalman-filter-based state estimation with model predictive control (MPC), we introduce a novel anesthetic dosing regimen that can transition seamlessly between TCI and closed-loop control, thus constituting an adequate trade-off between model and measurement reliance. We introduce this regimen and provide a realistic simulation example that highlights its strengths compared to pure TCI or closed-loop control of propofol infusion.

Keywords: Optimal control, Communication networks, Networked control systems
Abstract: Dynamical flow networks have emerged in the scientific community as a valuable tool to model the time-varying behavior of transportation and communication engineering systems. In this work, we present some results on the minimum-time optimal control of single-commodity dynamical flow networks, showing how it is possible to identify some structural properties relating optimality conditions and graph topology through a mathematical tool defined as the optimality matrix. Additionally, proving some results on the admissibility of solutions, which have significant implications for the activation of control inputs within the optimal time interval, leading to some derivations regarding the number of switching instants in the control strategy.

Keywords: Optimal control, Computational methods
Abstract: With a view to generalizing existing max-plus and min-plus methods so as to use quadratic basis functions with a non-uniform Hessian, the underlying dual space representation of dynamic programming founded on the semiconvex transform is generalized via the quadratic transform. Using this generalization, an illustrative iteration is proposed as the foundation of a new max-plus method for the solution of a class of optimal control problems.

Keywords: Optimal control, Predictive control for nonlinear systems, Lyapunov methods
Abstract: This paper considers the infinite horizon optimal control problem for nonlinear systems. Under the condition of nonlinear controllability of the system to any terminal set containing the origin and forward invariance of the terminal set, we establish a regularized solution approach consisting of a ``finite free final time" optimal transfer problem to the terminal set, which renders the set globally asymptotically stable. Further, we show that the approximations converge to the optimal infinite horizon cost as the size of the terminal set decreases to zero. We also perform the analysis for the discounted problem and show that the terminal set is asymptotically stable only for a subset of the state space and not globally. The theory is empirically evaluated on various nonholonomic robotic systems to show that the cost of our approximate problem converges and the transfer time into the terminal set is dependent on the initial state of the system, necessitating the free final time formulation. We also do comparisons of our free-final time approach with nonlinear MPC.

Keywords: Optimal control, Computational methods, Variational methods
Abstract: A class of finite time horizon optimal control problems with nonlinear dynamics and non-quadratic costs is considered. Stat-quad duality is used to transform the problem into a canonical form. A derivative-free numerical method that only uses fixed-point iterations is devised to solve it efficiently, the convergence of which is limited only by the existence of the staticizing control process (arg stat). For problems with mild and low-dimensional nonlinearities, this leads to dimension reduction of the control space. A 4-D and a 25-D control problem are solved to demonstrate its accuracy and scalability.

Keywords: Optimal control, Hybrid systems, Switched systems
Abstract: Nonsmooth phenomena, such as abrupt changes, impacts, and switching behaviors, frequently arise in real-world systems and present significant challenges for traditional optimal control methods, which typically assume smoothness and differentiability. These phenomena introduce numerical challenges in both simulation and optimization, highlighting the need for specialized solution methods. Although various applications and test problems have been documented in the literature, many are either overly simplified, excessively complex, or narrowly focused on specific domains. On this canvas, this paper proposes two novel tutorial problems that are both conceptually accessible and allow for further scaling of problem difficulty. The first problem features a simple ski jump model, characterized by state-dependent jumps and sliding motion on impact surfaces. This system does not involve control inputs and serves as a testbed for simulating nonsmooth dynamics. The second problem considers optimal control of a special type of bicycle model. This problem is inspired by practical techniques observed in BMX riding and mountain biking, where riders accelerate their bike without pedaling by strategically shifting their center of mass in response to the track’s slope.

Keywords: Optimal control, Randomized algorithms, Optimization algorithms
Abstract: We present randomized reconstruction approaches for optimal solutions to mixed-integer elliptic PDE control systems. Approximation properties and relations to sum-up rounding are derived using the cut norm. This enables us to dispose of space-filling curves required for sum-up rounding. Rates of almost sure convergence in the cut norm and the SUR norm in control space as well as almost sure H^1 convergence in state space are shown.

Keywords: Power systems, Differential-algebraic systems, Reduced order modeling
Abstract: Power systems transients are commonly modeled via a system of large nonlinear differential-algebraic equations (NDAEs). The complexity and size of power systems models are further increasing with the integration of renewables and other distributed energy resources. To allow scalable real-time monitoring and control, the field of model order reduction (MOR) is becoming more crucial. Although recent literature in power system MOR has proposed various MOR techniques, they simplify the NDAE models to often linear ODE ones. Consequently, the algebraic variables and constraints (power/current balance equations) are neglected and reduction is only carried out for dynamic variables. Ignoring algebraic states in the reduction process means a significant part of the system remains untouched, limiting the effectiveness of MOR. To that end, we propose MOR for complete NDAE power system models. The presented approach can significantly reduce the number of both dynamic and algebraic variables. Case studies on the IEEE 39-bus system validates the performance of the proposed approach. Numerical simulations show that the system can be represented using a far fewer number of states while providing input-output behavior similar to the original NDAE model.

Keywords: Power systems, Stability of linear systems, Optimization
Abstract: While the effect of network topology, line impedances, and (synthetic) inertia/damping coefficients of synchronous generators and distributed energy resources (DER) on dynamic performance have been extensively investigated, the impact of the power network operating point has not been studied. This work proposes a novel semi-definite program (SDP)-based optimal power flow (OPF) formulation to find a more stable generator dispatch. Different stability metrics widely used in industry can be captured by a carefully selected mcH_2-norm of a linear time-invariant (LTI) system obtained from swing dynamics. Under practical approximations, the dependence of this norm on the operating point can be captured by a convex model. This allows selecting the operating point to minimize stability metrics, generation costs, or combinations thereof through an SDP-based OPF. A two-stage approach can also be followed, where first, active power setpoints are decided by a standard OPF and reactive power setpoints are decided subsequently by the proposed OPF. Pareto optimality analysis carried out to study the relative trade-off between generation and stability costs reveals that a significant improvement in stability can be obtained with small increments in generation cost. Dynamic simulations on the IEEE 68-bus system corroborate that the found OPF schedules feature improved dynamic behavior.

Keywords: Power systems, Stochastic optimal control, Kalman filtering
Abstract: By incorporating the unscented Kalman filter (UKF) into the stochastic model predictive control (SMPC) architecture, a UKF-SMPC framework is formulated to solve the load frequency control (LFC) problem of a power system subject to wind resources and load disturbances. To suppress the frequency deviations resulting from the stochastic uncertainties and reduce the mechanical power cost, a finite-horizon constrained optimization problem is formulated to maintain the stability of the power system and improve the overall performance. Considering the stochastic nature of wind resources and load disturbances, the UKF is incorporated into the SMPC directly to estimate the states and to propagate the mean and covariance of the states forward in time by taking the state estimation errors and additive noise from the disturbances into consideration. The statistical description including the mean and covariance estimates of the state provided by the UKF are employed to reformulate the cost function and chance constraints. By resorting to the Chebyshev-Cantelli inequality, the chance constraints on the load frequency deviation are reformulated as deterministic ones, which are subsequently linearized at the cost of additional conservativeness. To guarantee the convergence and recursive feasibility of the UKF-SMPC framework, two kinds of terminal constraints are applied, that is, “robust horizon” and Lyapunov equation. By resorting to the Schur complement, the finite-horizon constrained optimization problem is recast as a linear one with a set of linear matrix inequalities (LMIs), which yields an SDP. Simulation results validate the effectiveness of our approach.

Keywords: Power systems, Smart grid, Differential-algebraic systems
Abstract: Increasing digitalization has made smart grids vulnerable to cyberattacks. False Data Injection Attacks (FDIAs), a type of cyberattacks, against power system monitoring and state estimation (SE) methods have been extensively studied for AC steady-state and simplified transient model of multi-machine power grids. However, the vulnerability of the more realistic (and complex) Nonlinear Differential Algebraic Equation (NDAE) models, favored in the industry for their accuracy, remains by and large unexplored. This paper investigates FDIAs targeting SE of NDAE models, which preserve crucial coupling between dynamic and algebraic power flow constraints. We formulate two novel FDIA methods: (i) an optimization-based approach and (ii) an algorithmic method leveraging worst-case attacks on intrusion detectors. Both methods ensure attack stealthiness and physical plausibility by adhering to NDAE model constraints. We compare these constraint-aware attacks to their constraint-unaware counterparts, quantifying the inherent resilience of NDAE models to FDIAs. We evaluate different intrusion detection methods and their impact on allowable attack magnitudes. The proposed attack strategies are validated through simulations on the IEEE 39-bus system.

Keywords: Power systems, Optimization, Machine learning
Abstract: Timely and effective load shedding in power systems is critical for maintaining supply-demand balance and preventing cascading blackouts. To eliminate load shedding bias against specific regions in the system, optimization-based methods are uniquely positioned to help balance between economic and fairness considerations. However, the resulting optimization problem involves complex constraints, which can be time-consuming to solve and thus cannot meet the real-time requirements of load shedding. To tackle this challenge, in this paper we present an efficient machine learning algorithm to enable millisecond-level computation for the optimization-based load shedding problem. Numerical studies on both a 3-bus toy example and a realistic RTS-GMLC system have demonstrated the validity and efficiency of the proposed algorithm for delivering fairness-aware and real-time load shedding decisions.

Keywords: Stability of nonlinear systems, Power systems, Lyapunov methods
Abstract: This paper presents a passivity-based stability analysis of power systems comprised of Park synchronous generator models. While existing research often utilizes simplified generator models, we focus on the Park model that is known for its high level of detail. We extend existing equilibrium-independent passivity analysis for the stability of power systems with this detailed model. This approach enables a more accurate assessment of power system stability by considering the complex dynamics captured by the Park model, which are often neglected in simplified representations. Numerical simulations confirm the validity of our analysis.

Keywords: Energy systems, Control applications, Robust adaptive control
Abstract: With the growing use of PV systems in renewable energy, maximum power point tracking (MPPT) technology is vital for enhancing energy conversion efficiency. However, traditional MPPT algorithms struggle to find the global maximum power point (GMPP) in complex, dynamically changing environments and under partial shading. This paper introduces an improved Newton extremum seeking control (ESC) MPPT algorithm that employs an extreme value search for local extremes and a multi-peak mechanism for global extremes, preventing entrapment at local maxima and ensuring maximum power output. Simulations demonstrate that the algorithm rapidly adapts to irradiation and temperature changes, accurately tracking the GMPP in multi-peak scenarios, and improving the PV system's energy conversion rate. It offers faster convergence and higher accuracy than traditional methods, making it ideal for complex environments.

Keywords: Energy systems, Optimal control, Control applications
Abstract: Airport rental car facilities present a significant potential for decarbonization through the adoption of electric vehicles. This transition will likely be accompanied by the deployment of fast charging infrastructure, necessary for keeping vehicle inventory and dwell times low. In this context, distributed or behind-the-meter resources such as stationary battery storage and PV generation are proposed as a solution for reducing costs and improving resiliency. The impact of load uncertainty on the optimal control and design of these systems, however, is a critical topic that has been less explored. In this work, the control and design of behind-the-meter resources under load uncertainty from a large-scale electrified rental car facility is investigated. An Economic Model Predictive Control model is presented, along with stochastic extensions. Different control policies and forecasting methods are compared, demonstrating that chance constraints can be employed to improve performance with limited forecasting. System design is shown to significantly impact control performance, indicating that larger batteries are necessary for less optimal policies. Finally, the effect of control policy choice on the optimal system design is evaluated.

Keywords: Distributed control, Power systems
Abstract: This letter proposes a novel, fully distributed, Safety-Guaranteed and Attack-Resilient (SGAR) cooperative control framework for AC microgrids, addressing polynomially unbounded false data injection (PU-FDI) attacks on control input channels. Unlike existing methods that address either transient-state safety, without considering malicious cyberattacks, or steady-sate resilience under attacks, without considering safety constraints, our SGAR control framework guarantees both safety and resilience, ensuring that system states remain within predefined safety bounds even during attack initiation—a critical aspect overlooked in prior research. Given the reduction of network inertia by increasing the penetration of inverted-based renewables, large overshooting and intense fluctuations are more likely to occur during transients caused by disturbances and cyberattacks. To mitigate these risks, the proposed SGAR control framework enhance resilient capabilities against PU-FDI attacks, maintaining safe system trajectories for both frequency and voltage throughout the transient response. Through rigorous certification using Control Barrier Functions (CBFs) and Control Lyapunov Function (CLF), we formally certify the strategies to achieve safety guarantees and maintain uniformly ultimately bounded (UUB) convergence in frequency regulation and voltage containment}, as well as active power sharing across multi-inverter-based AC microgrids. Numerical simulation studies verify the effectiveness of the proposed control protocols, demonstrating improved system reliability, safety, and resilience under adverse conditions.

Keywords: Sensor fusion, Smart grid, Kalman filtering
Abstract: This paper explores the privacy-preserving power estimation problem in DC microgrids with plug-and-play functionality. To safeguard the privacy of data associated with distributed generation units, a novel differentially private distributed fusion filtering algorithm is proposed. Under some mild assumptions, such as collective observability and bounded system parameters, the security, optimality of the gain matrix, consistency, and stability of the designed filters are ensured. Finally, simulation experiments demonstrate the effectiveness of the proposed algorithm.

Keywords: Game theory, Reinforcement learning, Distributed control
Abstract: This paper formulates multiplayer multiagent differential games to design the optimal containment control of multiplayer multiagent systems. Distributed Nash equilibrium (simplified as Nash) control policies are designed within the games using only local agents' trajectories, enabling all players to implement optimal control simultaneously. The solvability of the games and the asymptotic stability of the local error system are ensured. A data-driven integral reinforcement learning (RL) algorithm is devised to compute the distributed Nash control online by leveraging system trajectories without knowing explicit system dynamics. Finally, simulation results verify the effectiveness of the designed games and algorithms.

Keywords: Game theory, Autonomous systems, Optimization
Abstract: Existing multi-agent motion planners face scalability challenges with the number of agents and route plans that span long time horizons. We tackle these issues by introducing additional abstraction by interpolating agent trajectories with natural cubic splines and leveraging existing results that under some natural assumptions, the resulting game has the structure of a potential game. We prove a simultaneous gradient descent method using independent per-agent step sizes is guaranteed to converge to a local Nash equilibrium. Compared with recent iLQR-based potential game solvers, our method solves for local Nash equilibrium trajectories faster in games with up to 52 agents, and we demonstrate scalability to long horizons.

Keywords: Game theory, Machine learning, Robotics
Abstract: We consider the problem of learning Nash equilibrial policies for two-player risk-sensitive collision-avoiding interactions. Solving the Hamilton-Jacobi-Isaacs equations of such general-sum differential games in real time is an open challenge due to the discontinuity of equilibrium values on the state space. A common solution is to learn a neural network that approximates the equilibrium Hamiltonian for given system states and actions. The learning, however, is usually supervised and requires a large amount of sample equilibrium policies from different initial states in order to mitigate the risks of collisions. This paper claims two contributions towards more data-efficient learning of equilibrium policies: First, instead of computing Hamiltonian through a value network, we show that the equilibrium co-states have simple structures when collision avoidance dominates the agents' loss functions and system dynamics is linear, and therefore are more data-efficient to learn. Second, we introduce theory-driven active learning to guide data sampling, where the acquisition function measures the compliance of the predicted co-states to Pontryagin's Maximum Principle. On an uncontrolled intersection case, the proposed method leads to more generalizable approximation of the equilibrium policies, and in turn, lower collision probabilities, than the state-of-the-art under the same data acquisition budget.

Keywords: Game theory, Agents-based systems
Abstract: When multiple agents are engaged in a network of conflict, some can advance their competitive positions by forming alliances with each other. However, the costs associated with establishing an alliance may outweigh the potential benefits. This study investigates costly alliance formation in the framework of coalitional Blotto games, in which two players compete separately against a common adversary and are able to collude by exchanging resources with one another. Previous work has shown that both players in the alliance can mutually benefit if one player unilaterally donates, or transfers, a portion of their budget to the other. In this letter, we consider a variation where the transfer of resources is inherently inefficient, meaning that the recipient of the transfer only receives a fraction of the donation. Our findings reveal that even in the presence of inefficiencies, mutually beneficial transfers are still possible. More formally, our main result provides necessary and sufficient conditions for the existence of such transfers, offering insights into the robustness of alliance formation in competitive environments with resource constraints.

Keywords: Game theory, Control applications, Mean field games
Abstract: In this paper, we propose a graphon game model to understand how rumor (such as fake news) propagates in large populations that are interacting on a network and how different policies affect the spread. We extend the SKIR model that is used to model rumor propagation and implement individual controls and weighted interactions with other agents to have controlled dynamics. The agents aim to minimize their own expected costs non-cooperatively. We give the finite player game model and the limiting graphon game model to approximate the Nash equilibrium in the population. We give the graphon game Nash equilibrium as a solution to a continuum of ordinary differential equations (ODEs) and give existence results. Finally, we give a numerical approach and analyze examples where we use piecewise constant graphon.

Keywords: Game theory, Control of networks, Data driven control
Abstract: Targeted interventions in games present a challenging problem due to the asymmetric information available to the regulator and the agents. This note addresses the problem of steering the actions of self-interested agents in quadratic network games towards a target action profile. A common starting point in the literature assumes prior knowledge of utility functions and/or network parameters. The goal of the results presented here is to remove this assumption and address scenarios where such a priori knowledge is unavailable. To this end, we design a data-driven dynamic intervention mechanism that relies solely on historical observations of agent actions and interventions. Additionally, we modify this mechanism to limit the amount of interventions, thereby considering budget constraints. Analytical convergence guarantees are provided for both mechanisms, and a numerical case study further demonstrates their effectiveness.

Keywords: Distributed control, Agents-based systems, Optimal control
Abstract: This paper presents an asynchronous formulation of sensitivity-based nonlinear distributed optimal control. The execution is asynchronous in the sense that agents perform algorithmic steps independently of each other and are allowed to use outdated data. Under the assumption of a bounded delay, it is shown that the linear convergence rate of the synchronous execution reduces to R-linear convergence in the asynchronous setting. The algorithm is analyzed in numerical simulations and the execution time is evaluated on distributed hardware with networked communication. The results show that the slower convergence rate is compensated by the reduced execution time when compared to the synchronous approach.

Keywords: Distributed control, Cooperative control, Output regulation
Abstract: A general multi-partite output regulation problem (MORP) for heterogeneous linear multi-agent systems (MASs) is formulated and solved by realizing it as the cooperative output regulation problem (CORP). In our previous work [1], we formulated the MORP, in which the objective is to design a distributed control law such that each follower sharing the same partition term asymptotically tracks a predefined scalar multiple of a reference while ensuring the internal stability of the closed-loop system. The goal of the problem formulated in this paper is more general; that is, a left matrix multiple is considered instead of a scalar multiple of the reference. In some applications, such as the formation tracking of networked mobile robots, the output of each agent tracking a scalar multiple of the reference imposes a limitation on the MAS's maneuverability due to the multi-output nature of the application. The primary motivation behind our problem formulation is to overcome this limitation and equip MASs with enhanced operational capability. To demonstrate this, an experiment is conducted with a MAS composed of nonholonomic mobile robots.

Keywords: Distributed control, Large-scale systems, Networked control systems
Abstract: We study phenomena where some eigenvectors of a graph Laplacian are largely confined in small subsets of the graph. These localization phenomena are similar to those generally termed Anderson Localization in the Physics literature, and are related to the complexity of the structure of large graphs in still unexplored ways. Using perturbation analysis and pseudo-spectrum analysis, we explain how the presence of localized eigenvectors gives rise to fragilities (low robustness margins) to unmodelled node or link dynamics. Our analysis is demonstrated by examples of networks with relatively low complexity, but with features that appear to induce eigenvector localization. The implications of this newly-discovered fragility phenomenon are briefly discussed.

Keywords: Distributed control, LMIs, Lyapunov methods
Abstract: This study addresses the centralized synthesis of distributed controllers using linear matrix inequalities (LMIs). Sparsity constraints on control gains of distributed controllers result in conservatism via the convexification of the existing methods such as the extended LMI method. In order to mitigate the conservatism, we introduce a novel LMI formulation for this problem, utilizing the clique-wise decomposition method from our previous work on continuous-time systems. By reformulating the sparsity constraint on the gain matrix within cliques, this method achieves a broader solution set. Also, the analytical superiority of our method is confirmed through numerical examples.

Keywords: Distributed control, Optimization, Optimization algorithms
Abstract: In Bayesian optimization, a black-box function is maximized via the use of a surrogate model. We apply distributed Thompson sampling, using a Gaussian process as a surrogate model, to approach the multi-agent Bayesian optimization problem. In our distributed Thompson Sampling implementation, each agent receives sampled points from neighbors, where the communication network is encoded in a graph; each agent utilizes a Gaussian process to model the objective function. We demonstrate a theoretical bound on Bayesian Simple Regret, where the bound depends on the size of the largest complete subgraph of the communication graph. Unlike in batch Bayesian optimization, this bound is applicable in cases where the communication graph amongst agents is constrained. When compared to sequential Thompson sampling, our bound guarantees faster convergence with respect to time as long as there is a fully connected subgraph of at least two agents. We confirm the efficacy of our algorithm with numerical simulations on traditional optimization test functions, illustrating the significance of graph connectivity on improving regret convergence.

Keywords: Modeling, Cooperative control, Adaptive systems
Abstract: In this study, cooperative conveyance by sandwiching a dolly with multiple mobile robots is considered. A velocity response approximation (VRA) is proposed to consider the dynamics of the robots and the dolly, in addition to their kinematics, while maintaining model simplicity and reducing challenges related to parameter estimation. Since the VRA relies on the condition that all robots hold the dolly correctly, we introduce a control barrier function-based assist controller to ensure the validity of the model. Assuming state measurements are performed more frequently than control updates, a parameter ataptation algorithm is proposed to estimate the time-varying parameters of the VRA. Several numerical examples demonstrate the effectiveness of the proposed method.

Keywords: Maritime control, Autonomous systems, Lyapunov methods
Abstract: Critical marine operations, such as offshore inspections and border patrols, require timely execution. Drawing inspiration from interceptor guidance techniques, this work presents a novel method for steering unmanned surface vessels (USVs) using a true-proportional-navigation philosophy. The proposed approach enables USVs to reach their destinations while adhering to specific time constraints. We first establish the framework and derive the necessary dynamics, allowing us to exploit the time-varying speed of USVs to arrive at the desired location at the desired time. The concept of time-to-go relative to the desired position is formulated and managed through a coordinated effort involving the vessels' longitudinal acceleration and yaw rate. Control inputs for both channels are then determined using a Lyapunov stability-based approach and a control allocation technique within a nonlinear framework, ensuring the strategy remains effective even with significant initial deviations. The proposed controller is validated through numerical simulations, demonstrating satisfactory performance.

Keywords: Multivehicle systems, Automotive control, Robotics
Abstract: Occluded areas restrict the field of view for autonomous vehicles, thus posing significant safety risks. Although cooperative perception via vehicle connectivity may mitigate these risks, gradual adoption of connected vehicles (CV) means such hazards will persist in mixed traffic in the foreseeable future. To enable safe autonomous overtaking in occluded zones under sparse CV deployment, this paper proposes a game-theoretical decision-making strategy using coordinated active perception. First, a risk assessment strategy based on reachability analysis quantifies maximum risk sets from potential occluded vehicles and derives safe drivable regions for the ego CV. Building on this, the coordinated active perception method employs automated CVs as active sensors that dynamically adjust their motion/pose to expand visibility and share information via wireless communications, thus minimizing overtaking conservativeness. Finally, with occluded vehicle intentions obtained via coordinated active perception, the ego CV interactively determines its maneuver based on a potential differential game, which is transformed into a centralized optimal control problem and solved using an iterative Linear Quadratic Regulator. Simulation results demonstrate the potential enhancement of overtaking efficiency through coordinated active perception, while the proposed interaction strategy ensures robust safety and adaptability across diverse overtaking scenarios.

Keywords: Stochastic optimal control, Fault detection, Information theory and control
Abstract: We formulate a stochastic zero-sum game over continuous-time dynamics to analyze the competition between the attacker, who tries to covertly misguide the vehicle to an unsafe region, versus the detector, who tries to detect the attack signal based on the observed trajectory of the vehicle. Based on Girsanov's theorem and the generalized Neyman-Pearson lemma, we show that a constant bias injection attack as the attacker's strategy and a likelihood ratio test as the detector's strategy constitute the unique saddle point of the game. We also derive the first-order and the second-order exponents of the type II error as a function of the data length.

Keywords: Optimal control, Optimization, Power systems
Abstract: Hybrid electric vehicles (HEV) enable reduction of emissions without sacrificing consumer expected range and drivability. The diversification of the powertrain with multiple power sources allows downsizing the internal combustion engine and implementing optimal energy management strategies. The interaction among components of an HEV are key to the overall efficiency. Therefore, efficiency potential is lost if this interdependence is neglected during the powertrain design by focusing on individual optimization of component specifications. This work formulates and solves a co-design problem by integrating the energy management with the optimal powertrain and drivetrain component sizing for a hybrid powertrain equipped with an opposed piston (OP) engine in a series architecture. Our novel approach develops a model for an OP engine and integrates battery capacity degradation into the co-design problem. The optimal solution allows for a minimally sized engine that accounts for the average power requirements, and a large enough battery to provide fast power dynamics.

Keywords: Maritime control, Control applications, Autonomous systems
Abstract: Uncrewed Surface Vessels (USVs) are increasingly employed in critical marine operations, such as offshore monitoring and cargo transport, where autonomous docking is essential for mission success. In this paper, we introduce a novel approach to autonomous docking wherein the motion control algorithm guides the USV to the docking station while maintaining the orientation required to dock and reducing vehicle speed for safe docking. Control inputs for the vehicle are derived using sliding mode control within a nonlinear framework that guarantees the validity of the proposed controller even in scenarios with significant deviations. Owing to the inherent robustness of sliding mode control techniques, the proposed controller is robust against matched uncertainties. The proposed algorithm's effectiveness is validated through numerical simulations, demonstrating satisfactory performance in achieving precise docking of USV from various initial and final geometries.

Keywords: Optimization algorithms, Stability of nonlinear systems, PID control
Abstract: We propose a generalized feedback system architecture to synthesize and analyze a class of continuous-time dynamics that solves linearly constrained quadratic programs (LCQPs) in real time. In particular, the architecture executes gradient descent on the cost function, invokes a deliberately engineered sector-bounded nonlinearity to penalize excursions in states away from the feasible space, and involves a family of linear controllers, including proportional-integral (PI) control and its variants to shape dynamic performance and minimize steady-state error. We establish the correspondence of the equilibria of the continuous-time dynamics with stationary points of the LCQP and provide numerical tests for stability leveraging the Kalman-Yakubovich-Popov lemma. We also comment on how specific variants of PI control present similarities to widely studied methods, including the augmented Lagrangian method, the primal-dual method, and gradient-flow dynamics for penalty program reformulations. Theoretical results are validated through simulations to benchmark performance and elucidate the design choices that emerge from the analysis.

Keywords: Optimization, Stability of nonlinear systems, Optimization algorithms
Abstract: The low-rank matrix factorization problem serves as a building block in many modern learning tasks including matrix recovery and neural networks training. For a symmetric version of the associated non-convex optimization problem, we characterize equilibrium points of the gradient flow dynamics and examine their local and global stability properties. Our proof exploits a novel change of variables that brings the underlying dynamics into a cascade connection of three subsystems whose analysis is considerably simpler than the analysis of the original system. We show that one of these subsystems is governed by autonomous dynamics which are decoupled from the rest of the system. In the over-parameterized regime, this subsystem is associated with the excess parameters; our analysis reveals its stability with an O(1/t) asymptotic convergence rate that captures the slow dynamics. For the other two subsystems, we utilize a Lyapunov-based approach to establish their exponential convergence under mild assumptions.

Keywords: Optimization algorithms, Robust control, Numerical algorithms
Abstract: This paper presents a discrete-time passivity-based analysis of the gradient descent method for a class of functions with sector-bounded gradients. Using a loop transformation, it is shown that the gradient descent method can be interpreted as a passive controller in negative feedback with a very strictly passive system. The passivity theorem is then used to guarantee input-output stability, as well as the global convergence, of the gradient descent method. Furthermore, provided that the lower and upper sector bounds are not equal, the input-output stability of the gradient descent method is guaranteed using the weak passivity theorem for a larger choice of step size. Finally, to demonstrate the utility of this passivity-based analysis, a new variation of the gradient descent method with variable step size is proposed by gain-scheduling the input and output of the gradient.

Keywords: Optimization algorithms, Optimization, Robust control
Abstract: In this paper, we develop an online optimization algorithm for solving a class of nonconvex optimization problems with a linearly varying optimal point. The global convergence of the algorithm is guaranteed using the circle criterion for the class of functions whose gradient is bounded within a sector. Also, we show that the corresponding Luré-type nonlinear system involves a double integrator, which demonstrates its ability to track a linearly varying optimal point with zero steady-state error. The algorithm is applied to solving a time-of-arrival based localization problem with constant velocity and the results show that the algorithm is able to estimate the source location with zero steady-state error.

Keywords: Optimization algorithms, Robust control
Abstract: The interpretation of iterative optimization algorithms as dynamical systems has led to a variety of advances in their analysis and design using tools from control. In this paper, we identify a structure of dynamical systems that arises naturally in a variety of optimization algorithms, and we show how to take advantage of this structure in system analysis. In particular, first-order optimization algorithms consist of the gradient of the objective in feedback with graded dynamical systems, which are systems whose signal spaces decompose as direct sums that are not mixed by the system dynamics.

Keywords: Optimization, Robust adaptive control, Uncertain systems
Abstract: We propose an adaptive method for online time-varying (TV) convex optimization, termed L1 adaptive optimization (L1-AO). TV optimizers utilize a prediction model to exploit the temporal structure of TV problems, which can be inaccurate in the online implementation. Inspired by L1 adaptive control, the proposed method augments an adaptive update law to estimate and compensate for the uncertainty from the prediction inaccuracies. The proposed method provides performance bounds of the error in the optimization variables and cost function, allowing efficient and reliable optimization for TV problems. Numerical simulation results demonstrate the effectiveness of the proposed method for online TV convex optimization.

Keywords: Constrained control, Optimization, Aerospace
Abstract: This paper introduces a novel control strategy for agile spacecraft attitude control, addressing reaction wheel-related input and state constraints. An optimal-decay control Lyapunov function quadratic program stabilizes the system and mitigates chattering at low sampling frequencies, while control barrier functions enforce hard state constraints. Numerical simulations validate the method's practicality and efficiency for real-time agile spacecraft attitude control.

Keywords: Aerospace, Autonomous systems, Cooperative control
Abstract: In this paper, we present a multi-agent framework for intercepting an incoming fast-moving intruder by slower agents. Given the pose of the intruder and a known target, agents compute the feasible region of intruder trajectories though an interception plane in 3D space. They then deploy themselves into a 3D formation that increases the likelihood of intruder interception in the feasible region. Results are verified in Monte-Carlo simulations for given intruder characteristics, and show that the proposed method reduces the miss distance to less than 0.2 meters for at least one agent in the swarm for more than 99% of simulated intruders that follow a proportional navigation scheme, even when intruders have significant speed advantage over the agents.

Keywords: Aerospace, Flight control, Feedback linearization
Abstract: In this paper, we present a control design methodology, stability criteria and performance bounds for autonomous helicopter aerial refueling. Autonomous aerial refueling is particularly difficult due to the aerodynamic interaction between the wake of the tanker, the contact-sensitive nature of the maneuver, and the uncertainty in drogue motion. Since the probe tip is located significantly away from the helicopter's center of gravity, its position/velocity is strongly sensitive to the helicopter's attitude/angular rates. In addition, the fact that the helicopter is operating at high speeds to match the velocity of the tanker forces it to maintain a particular orientation, making the docking maneuver especially challenging. In this paper, we propose a novel outer-loop position controller that incorporates the probe position and velocity into the feedback loop. The position and velocity of the probe tip depend both on the position/velocity and on the attitude/angular rates of the aircraft. We derive analytical guarantees for docking performance in terms of the uncertainty of the drogue motion and the angular acceleration of the helicopter, using the ultimate boundedness property of the closed loop error dynamics. Simulations are performed on a high-fidelity UH60 helicopter model with a high-fidelity drogue motion under wind effects to validate the proposed approach for realistic refueling scenarios. These high-fidelity simulations reveal that the proposed control methodology yields an improvement of 36% in the 2-norm docking error compared to the existing standard controller.

Keywords: Flight control, Control applications
Abstract: We implement a conflict detection and 4D flight path resolution algorithm on quadrotor platforms. 4D flight paths are based on 5th-order polynomial interpolating splines that fully define the flight path's position, velocity, and acceleration at each moment in time between waypoints. Our detection and resolution algorithms operate in a distributed manner with multiple quadrotors using 8Gb Raspberry Pis. We control the quadrotors to the 4D flight paths with Pixhawk autopilots using PX4 software. The results of this work show the capability of our flight path model to effectively separate autonomous vehicles with flight paths that their control systems are capable of tracking.

Keywords: Flight control, Fault tolerant systems, Feedback linearization
Abstract: This paper presents a path-following controller for a quadrotor system to guarantee safe maneuvers, in terms of forward path invariance, in the presence of cyber-physical attacks. We assume that an adversarial agent can control any one of the rotors through a false data injection (FDI) type of attack. A feedback controller is designed using transverse feedback linearization which guarantees that the system follows a class of smooth curves under FDI attacks. Our proposed controller is computationally efficient, with a closed-form analytical expression, that not only mitigates the effect of bounded malicious signal but also ensures mission success. We provide theoretical guarantees of forward path invariance under FDI attacks with realistic assumptions and demonstrate the effectiveness of our approach through simulation.

Keywords: Distributed parameter systems, Fluid flow systems, Control applications
Abstract: We perform various numerical tests to study the effect of (boundary) stenosis on blood flow stability, employing a detailed and accurate, second-order finite-volume scheme for numerically implementing a partial differential equation (PDE) model, using clinically realistic values for the artery's parameters and the blood inflow. The model consists of a baseline 2x2 hetero-directional, nonlinear hyperbolic PDE system, in which, the stenosis' effect is described by a pressure drop at the outlet of an arterial segment considered. We then study the stability properties (observed in our numerical tests) of a reference trajectory, corresponding to a given time-varying inflow (e.g., a periodic trajectory with period equal to the time interval between two consecutive heartbeats) and stenosis severity, deriving the respective linearized system and constructing a Lyapunov functional. Due to the fact that the linearized system is time varying, with time-varying parameters depending on the reference trajectories themselves (that, in turn, depend in an implicit manner on the stenosis degree), which cannot be derived analytically, we verify the Lyapunov-based stability conditions obtained, numerically. Both the numerical tests and the Lyapunov-based stability analysis show that a reference trajectory is asymptotically stable with a decay rate that decreases as the stenosis severity deteriorates.

Keywords: Distributed parameter systems, Sampled-data control, Lyapunov methods
Abstract: This article introduces an observer-based periodic event-triggered control (PETC) strategy for boundary control of a system characterized by 2times2 linear hyperbolic partial differential equations (PDEs). An anti-collocated actuation and sensing configuration is considered, and an exponentially convergent observer for state estimation from boundary data is designed. Initially, a continuous-time dynamic event-triggering mechanism requiring constant monitoring of the triggering function is developed. This mechanism is subsequently adapted into a periodic event-triggering scheme, which necessitates only periodic monitoring to identify when the control input needs updating. The underlying control approach is the PDE backstepping boundary control, implemented in a zero-order hold manner between events. This result marks a substantial improvement over conventional observer-based continuous-time event-triggered control for linear coupled hyperbolic PDEs by removing the requirement for constant monitoring of the triggering function. With the triggering function evaluated periodically, the closed-loop system is inherently free from Zeno behavior. It is demonstrated that under the proposed PETC, the closed-loop system globally exponentially converges to zero in the spatial L^2 norm. A simulation study illustrating the theoretical results is presented.

Keywords: Distributed parameter systems, Fluid flow systems, Uncertain systems
Abstract: In this paper, we consider the problem of regulating the outlet pressure of gas flowing through a pipeline subject to uncertain and variable outlet flow. Gas flow through a pipe is modeled using the coupled isothermal Euler equations, with the Darcy–Weisbach friction model used to account for the loss of gas flow momentum. The outlet flow variation is generated by a periodic linear dynamic system, which we use as a model of load fluctuations caused by varying customer demands. We first linearize the nonlinear equations around the equilibrium point and obtain a 2-by-2 coupled hyperbolic partial differential equation (PDE) system expressed in canonical form. Using an observer-based PDE backstepping controller, we demonstrate that the inlet pressure can be manipulated to regulate the outlet pressure to a setpoint, thus compensating for fluctuations in the outlet flow. Furthermore, we extend the observer-based controller to the case when the outlet flow variation is uncertain within a bounded set. In this case, the controller is also capable of regulating the outlet pressure to a neighborhood of the setpoint by manipulating the inlet pressure, even in the presence of uncertain fluctuations in the outlet flow. We provide numerical simulations to demonstrate the performance of the controller.

Keywords: Traffic control, Optimization, Automotive systems
Abstract: Intersections pose critical challenges in traffic management, where maintaining operational constraints and ensuring safety are essential for efficient flow. This paper investigates the effect of intervention timing in managing the autonomous agents while satisfying constraints like safe separation distance, avoiding collisions at intersections, and minimizing the delay to improve the overall system efficiency. We introduce control regions, represented as circles around the intersection, which refers to the timing of interventions by a centralized control system when agents approach the intersection. We use a mixed-integer linear programming (MILP) approach to optimize the system's performance. A simulation study is conducted to analyze the effectiveness of early and late control measures, focusing on the safe, efficient, and robust management of agent movement within the control regions.

Keywords: Switched systems, Biomedical, Decentralized control
Abstract: Mechanical ventilators are complex mechatronic devices that are essential for patients who are unable to breathe independently. The aim of this paper is to develop a systematic control method that achieves accurate tracking of both the pressure and flow to ensure comfortable breathing for the patient. This is achieved by using a feedback design procedure technique based on static decoupling and the factorized Nyquist criterion. Furthermore, switching controllers are introduced that allow for improved baseflow tracking performance. The presented control method is implemented in a real ventilator and it is demonstrated that the tracking performance is improved by conducting an experimental case-study.

Keywords: Optimal control, Computational methods, Fluid flow systems
Abstract: A computational framework for the solution of optimal control problems governed by a parabolic partial differential equation is presented. The continuous space-time optimal control problem is transcribed into a sparse nonlinear programming problem (NLP) through state and control parameterization. In particular, a multi-interval flipped Legendre-Gauss-Radau (fLGR) collocation method is implemented for temporal discretization alongside a Galerkin finite element spatial discretization. The finite element discretization allows for a reduction in problem size and avoids the redefinition of constraints required under a previous method. Further, a generalization of a Kirchoff transformation is performed to handle nonlinearities in the variational description of the problem. Lastly, it is demonstrated on a numerical example that the use of a multi-interval fLGR temporal discretization can lead to a reduction in the required number of collocation points to compute accurate values of the optimal objective in comparison to other methods.

Keywords: Learning, Robust control, LMIs
Abstract: Assessing data informativity, determining whether the measured data contains sufficient information for a specific control objective, is a fundamental challenge in data-driven control. In noisy scenarios, existing studies deal with system noise and measurement noise separately, using quadratic matrix inequalities. Moreover, the analysis of measurement noise requires restrictive assumptions on noise properties. To provide a unified framework without any restrictions, this study introduces data perturbation, a novel notion that encompasses both existing noise models. It is observed that the admissible system set with data perturbation does not meet preconditions necessary for applying the key lemma in the matrix S-procedure. Our analysis overcomes this limitation by developing an extended version of this lemma, making it applicable to data perturbation. Our results unify the existing analyses while eliminating the need for restrictive assumptions made in the measurement noise scenario.

Keywords: Stability of linear systems
Abstract: This document surveys the most widely used metrics for assessing the stability margin of linear time-invariant scalar and multivariable closed-loop systems. These metrics are presented in a uniform mathematical framework that highlights their frequency-by-frequency quantification. The author proposes a set of desirable properties for stability-margin metrics, against which the surveyed metrics are assessed. In addition, this document introduces a novel metric, called extended stability margin, that addresses a gap found in all the other stability margins surveyed: A certain class of multivariable closed-loop systems can be at the brink of instability by a minimal perturbation in the plant, yet none of the other stability-margin metrics is able to detect such a near-instability condition.

Keywords: Uncertain systems, Robust adaptive control, Output regulation
Abstract: Acoustic noise attenuation is crucial in both industrial and everyday applications, particularly for low-frequency disturbances that passive methods cannot effectively mitigate. This paper presents a model-free active noise control (ANC) strategy inspired by the internal model principle (IMP), specifically designed for periodic noise in duct systems. We demonstrate significant attenuation of low-frequency noise while ensuring input-to-state stability, all without prior model information regarding the acoustic system's parameters or structure. More importantly, the proposed method robustly tolerates frequency estimation errors and measurement noise, as evidenced by theoretical analysis and experimental validation. Finally, benchmark experiments reveal the adaptability and generality of our approach within unknown environments and diverse systems. Our findings highlight the potential of IM-based model-free ANC strategies for practical applications in complex acoustic environments.

Keywords: Variable-structure/sliding-mode control, Robust control, Robust adaptive control
Abstract: Linear time-varying state feedback controllers are proposed for a double integrator subject to bounded disturbances in each equation. It is shown that, for the double integrator, the first state converges to zero at a hyperexponential rate (faster than any exponential decay) uniformly with respect to the disturbances, while the second state approaches the negative of an unmatched perturbation. These results are applied to design a novel time-varying differentiator.

Keywords: Uncertain systems, Stability of nonlinear systems, Computational methods
Abstract: A critical property of dynamical systems for real-world applications is their stability characterization, that is, their region of attraction. When accounting for systems with parametric uncertainty, it is crucial to obtain what is instead the robust region of attraction, i.e., the set of initial conditions of a dynamical system such that the system's asymptotic stability is guaranteed in the presence of uncertainties. Multiple methods to estimate the robust region of attraction are presented in the literature, mostly resorting to Lyapunov theory and converse results. In this paper, we present algorithms to estimate the robust region of attraction through an iterative approach based on invariant sets, sum-of-squares relaxations and some asymptotic stability certificates distinct from the ones present in Lyapunov's theory, namely an extension of the Bendixson-Dulac criterion. Furthermore, we demonstrate the applicability of this method to polynomial dynamical systems with constrained uncertainties, enabling the determination of a robust region of attraction inner estimate.

Keywords: Hierarchical control, Linear systems, Stability of linear systems
Abstract: In this paper, a dynamic self-triggered mechanism based on the hierarchical strategy is proposed for linear systems to reduce communication resource utilization. Specifically, a hierarchical structure is employed, where the upper trigger layer calculates the maximum allowable triggering interval value while ensuring system stability. To further reduce the number of triggers, a dynamic variable is introduced for the construction of the dynamic self-triggered mechanism, which greatly reduces the waste of communication resources. Within each inter-execution interval computed in the upper layer, the lower control layer determines the optimal control input by minimizing the given objective function. Through the collaboration between the triggering layer and the control layer, the number of system samples is significantly reduced, and the convergence time of the system state trajectory is shortened. The effectiveness of this proposed method is demonstrated through a numerical simulation example.

Keywords: Automotive control, Optimal control, Modeling
Abstract: Autonomous vehicles capable of safely operating beyond the stable handling limits would be able to perform a broader array of maneuvers during emergencies, thereby enhancing overall safety. This paper presents an approach to autonomous drifting using Koopman-based dynamics models, which globally linearize the nonlinear system dynamics. By leveraging the Koopman operator, we efficiently generate these dynamics models with only 2.2 minutes of offline data. The generated models are integrated with Model Predictive Control to achieve computationally efficient optimization and high tracking accuracy. Additionally, real-time adaptation using online data is employed to reduce model error and enhance trajectory tracking performance. Experimental results demonstrate that the proposed method effectively enables autonomous drifting, offering a significant improvement in control precision and computational efficiency compared to traditional approaches. This work opens up new possibilities for advanced autonomous driving applications where high-speed and precise control are crucial.

Keywords: Predictive control for nonlinear systems, Adaptive control, Optimal control
Abstract: We study a problem of simultaneous system identification and model predictive control of nonlinear systems. Particularly, we provide an algorithm for systems with unknown residual dynamics that can be expressed by Koopman operators. Such residual dynamics can model external disturbances and modeling errors, such as wind and wave disturbances to aerial and marine vehicles, or inaccurate model parameters. The algorithm has finite-time near-optimality guarantees and asymptotically converges to the optimal non-causal controller. Specifically, the algorithm enjoys sublinear dynamic regret, defined herein as the suboptimality against an optimal clairvoyant controller that knows how the unknown dynamics will adapt to its states and actions. To this end, we assume the algorithm is given Koopman observable functions such that the unknown dynamics can be approximated by a linear dynamical system. Then, it employs model predictive control based on the current learned model of the unknown dynamics. This model is updated online using least squares in a self-supervised manner based on the data collected while controlling the system. We validate our algorithm in physics-based simulations of a cart-pole aiming to maintain the pole upright despite inaccurate model parameters.

Keywords: Process Control, Machine learning, Constrained control
Abstract: We propose a data-driven robust reference governor (DRRG) framework to enforce output constraints in nonlinear systems with unknown dynamics and uncertainties. Leveraging the Koopman operator, DRRG maps complex dynamics into a higher-dimensional linear space for effective data-driven prediction. A data-driven maximal admissible set (MAS) and a min-max optimization problem are then formulated to compute robust reference signals. The proposed approach enhances robustness against uncertainties while preserving existing control architectures and simplifying controller tuning. Finally, a numerical simulation demonstrates its effectiveness.

Keywords: Predictive control for nonlinear systems, Data driven control, Reinforcement learning
Abstract: This paper develops the machinery of Koopman-based Model Predictive Control (KMPC) design, where the Koopman derived model is unable to capture the real nonlinear system perfectly. We then propose to use an MPC-based reinforcement learning within the Koopman framework combining the strengths of MPC, Reinforcement Learning (RL), and the Koopman Operator (KO) theory for an efficient data-driven control and performance-oriented learning of complex nonlinear systems. We show that the closed-loop performance of the KMPC is improved by modifying the KMPC objective function. In practice, we design a fully parameterized KMPC and employ RL to adjust the corresponding parameters aiming at achieving the best achievable closed-loop performance.

Keywords: Learning, Nonlinear systems identification, Identification for control
Abstract: This work presents a data-driven Koopman operator-based modeling method using a model averaging technique. While the Koopman operator has been used for data-driven modeling and control of nonlinear dynamics, it is challenging to accurately reconstruct unknown dynamics from data and perform different decision-making tasks, mainly due to its infinite dimensionality and difficulty of finding invariant subspaces. We utilize ideas from a Bayesian inference-based model averaging technique to devise a data-driven method that first populates multiple Koopman models starting with a feature extraction using neural networks and then computes point estimates of the posterior of predicted variables. Although each model in the ensemble is not likely to be accurate enough for a wide range of operating points or unseen data, the proposed weighted linear embedding model combines the outputs of model ensemble aiming at compensating the modeling error of each model so that the overall performance will be improved.

Keywords: Nonlinear systems identification, Predictive control for linear systems, Mechanical systems/robotics
Abstract: Online optimal control of quadrupedal robots would enable them to plan their movement in novel scenarios. Linear Model Predictive Control (LMPC) has emerged as a practical approach for real-time control. In LMPC, an optimization problem with a quadratic cost and linear constraints is formulated over a finite horizon and solved on the fly. However, LMPC relies on linearizing the equations of motion (EOM), which may lead to poor solution quality. In this paper, we use Koopman operator theory and the Extended Dynamic Mode Decomposition (EDMD) to create a linear model of the system in high dimensional space, thus retaining the nonlinearity of the EOM. We model the aerial phase and ground contact phases using different linear models. Then, using LMPC, we demonstrate bounding, trotting, and bound-to-trot and trot-to-bound gait transitions in level and rough terrains. The main novelty is the use of Koopman operator theory to create hybrid models of a quadrupedal system and demonstrate the online generation of multiple gaits and gaits transitions.

Keywords: Distributed parameter systems, Estimation, Adaptive systems
Abstract: This paper proposes a method to adaptively estimate static maps, representing spatially varying functions, by viewing them as solutions to elliptic partial differential equations (PDEs). The elliptic PDEs are assumed to have a known source function with an unknown source intensity and an unknown solution. By embedding the elliptic PDE into a parabolic PDE, a state and parameter estimator based on both the Kalman filter design and adaptive observer method is proposed. For both cases, the stability and convergence properties are presented. The benefit for this embedding approach is that the estimator for static maps no longer assumes a series expansion of the unknown spatially varying function with a prescribed and known series limit. Numerical results using both a Kalman filter and an adaptive observer demonstrate the proposed identifiers.

Keywords: Reduced order modeling, Fluid flow systems, Control applications
Abstract: This paper presents a method to design feedback boundary controllers for nonlinear partial differential equations (PDEs) based on cluster reduced-order modeling. First, k-means clustering is introduced for snapshot solutions generated by the full-order simulation. The snapshots are grouped into several subregions over space, where the behavior has different features. Proper orthogonal decomposition (POD) is then applied locally within each cluster, which causes discontinuities of the POD basis at the cluster interfaces. Discontinuous-Galerkin (DG) projection is introduced to construct a more accurate reduced-order model than global POD. Open-loop simulations of the full and reduced order models with application to a convection-diffusion flow are compared to validate the efficiency of cluster reduced-order modeling. A linear quadratic regulator problem is formulated for the boundary control based on the reduced-order model and then applied to the full order flow. The efficiency of the proposed method is demonstrated in reduced and full order simulations.

Keywords: Distributed parameter systems, Delay systems, Observers for nonlinear systems
Abstract: We study the state estimation problem for a ODE-PDE delayed cascaded coupling. The delayed output of a nonlinear system in triangular form is the boundary condition of a 1-D linear parabolic system at one side, and the available measurement is taken at the opposite side. The observer of the parabolic system is based on the modal decomposition approach and it is finite dimensional. The nonlinear observer compensates the delay in the measurement. We prove that under a suitable delay bound the estimation error is bounded even when the dynamic of the parabolic system is unbounded.

Keywords: Model/Controller reduction, Fluid flow systems, Distributed parameter systems
Abstract: This paper presents an efficient Chorin-projection reduced order model based on proper orthogonal decomposition (POD-ROM) for control of Navier-Stokes equations modeling viscous incompressible flows. The proposed POD-ROM is a velocity-pressure reduced-order model but decouples the computation of reduced-order velocity and pressure. Furthermore, it does not require satisfaction of the so-called inf-sup condition for mixed POD subspaces with the help of in-built pressure stabilization provided by the projection method without adding extra stabilization terms. We derive error estimates for the state, adjoint and control variables. Numerical studies are performed to discuss the accuracy and performance of the new Chorin- projection reduced order model in the simulation of control of flow past a backward-facing step channe

Keywords: Distributed parameter systems, Quantized systems, Switched systems
Abstract: We solve the global asymptotic stability problem of an unstable reaction-diffusion Partial Differential Equation (PDE) subject to input delay and state quantization developing a switched predictor-feedback law. To deal with the input delay, we reformulate the problem as an actuated transport PDE coupled with the original reaction-diffusion PDE. Then, we design a quantized predictor-based feedback mechanism that employs a dynamic switching strategy to adjust the quantization range and error over time. The stability of the closed-loop system is proven properly combining backstepping with a small-gain approach and input-to-state stability techniques, for deriving estimates on solutions, despite the quantization effect and the system's instability. We also extend this result to the input quantization case.

Keywords: Distributed parameter systems, Lyapunov methods, Systems biology
Abstract: Age-structured models describe the dynamic behaviors of populations over time and result in integro-partial differential equations (IPDEs). These models are useful to represent a multitude of processes in biotechnology or economics. Single population models are, for example, used to control the harvest rate of a chemostat in order to maximize the yield of a process. Age-structured population models with more than one species, leading to coupled IPDEs, are relevant for epidemics or ecology, but have received little attention in the literature. In this work, we present a model for two interacting populations in a predator-prey setup, whose input is the inevitable harvesting of both species, which represents a challenge for stabilization. The model is transformed to a system of two coupled ordinary differential equations (ODEs) actuated by the input, and two autonomous but exponentially stable integral delay equations (IDEs). The controllable ODE is stabilized through a weighted Control Lyapunov Function (CLF) feedback. We establish that the CLF-based control law exclusively derived from the ODE system dynamics, locally stabilizes the transformed ODE-IDE system.

Keywords: Transportation networks, Cooperative control, Game theory
Abstract: Network-based complex systems are inherently interconnected, with the design and performance of subnetworks being interdependent. However, the decisions of self-interested operators may lead to suboptimal outcomes for users. In this paper, we consider the question of what cooperative mechanisms can benefit both operators and users simultaneously. We address this question in a game theoretical setting, integrating both non-cooperative and cooperative game theory. During the non-cooperative stage, subnetwork decision-makers strategically design their local networks. In the cooperative stage, the co-investment mechanism and the payoff-sharing mechanism are developed to enlarge collective benefits and fairly distribute them. A case study of the Sioux Falls network is conducted to demonstrate the efficiency of the proposed framework. The impact of this interactive network design on environmental sustainability, social welfare and economic efficiency is evaluated, along with an examination of scenarios involving regions with heterogeneous characteristics.

Keywords: Lyapunov methods, Stability of nonlinear systems, Spacecraft control
Abstract: In this paper, we develop new necessary and sufficient Lyapunov conditions for fixed time stability that subsume the classical fixed time stability results presented in the literature as special cases and provide an optimized estimate of the settling time bound that is less conservative than the existing results. The results are then used to address the problem of fixed time stabilization of a rigid spacecraft.

Keywords: Genetic regulatory systems, Markov processes, Reinforcement learning
Abstract: This paper presents a deep reinforcement learning framework for designing optimal intervention policies in Gene Regulatory Networks (GRNs) under partial observability. Existing methods often assume full observability of the system states, which is unrealistic in practice due to incomplete or noisy gene expression data. To address these limitations, we extend Boolean network models to include partial observability. The uncertainty in gene expression data and stochasticity in gene activities impacted by interventions are captured through the posterior distribution of states, called the belief state. We formulate the optimal intervention policy over the belief space, maximizing long-term rewards by reducing harmful gene activations while accounting for system and data uncertainties. Deep reinforcement learning, particularly deep Q-network (DQN), is developed to enable approximation of the optimal intervention policy at scale. Our analytical results demonstrate that the method converges to the optimal dynamic programming solution if the uncertainty in the gene state disappears. Numerical experiments on a melanoma gene regulatory network demonstrate the efficacy of the proposed approach, showing improved performance compared to existing methods in maintaining desirable system states and reducing the activation of cancer-related genes.

Keywords: Power systems, Distributed control, Control of networks
Abstract: This paper presents a novel dissipativity-based distributed droop-free control approach for voltage regulation in DC microgrids (MGs) comprised of an interconnected set of distributed generators (DGs), loads, and power lines. First, we describe the closed-loop DC MG as a networked system where the sets of DGs and lines (i.e., subsystems) are interconnected via a static interconnection matrix. This interconnection matrix demonstrates how the disturbances, inputs, and outputs of DGs and lines are connected with each other. Each DG has a local controller and a distributed global controller. To design the latter, we use the dissipativity properties of the subsystems and formulate a linear matrix inequality (LMI) problem. To support the feasibility of this problem, we next identify a set of necessary local conditions, which we then enforce in a specifically developed LMI-based local controller design process. In contrast to existing DC MG control solutions, our approach proposes a unified framework for co-designing the distributed controller and communication topology. As the co-design process is LMI-based, it can be efficiently implemented and evaluated. The proposed solution's effectiveness in terms of voltage regulation and current sharing is verified by simulating an islanded DC MG in a MATLAB/Simulink environment under different scenarios, such as load changes and topological constraint changes, and then comparing the performance with a recent droop control solution.

Keywords: Linear systems, Identification, Optimization
Abstract: This paper studies the linear system identification problem in the general case where the disturbance is sub-Gaussian, correlated, and possibly adversarial. First, we consider the case with noncentral (nonzero-mean) disturbances for which the ordinary least-squares (OLS) method fails to correctly identify the system. We prove that the l_1-norm estimator accurately identifies the system under the condition that each disturbance has equal probabilities of being positive or negative. This condition restricts the sign of each disturbance but allows its magnitude to be arbitrary. Second, we consider the case where each disturbance is adversarial with the model that the attack times happen occasionally but the distributions of the attack values are arbitrary. We show that when the probability of having an attack at a given time is less than 0.5 and each attack spans the entire space in expectation, the l_1-norm estimator prevails against any adversarial noncentral disturbances and the exact recovery is achieved within a finite time. These results pave the way to effectively defend against arbitrarily large noncentral attacks in safety-critical systems.

Keywords: Distributed control, Linear systems, Observers for Linear systems
Abstract: We propose a convex controller synthesis framework for a large class of constrained linear systems, including those described by (deterministic and stochastic) partial differential equations and integral equations, commonly used in fluid dynamics, thermo-mechanical systems, quantum control, or transportation networks. Most existing control techniques rely on a (finite-dimensional) discrete description of the system, via ordinary differential equations. Here, we work instead with more general (infinite-dimensional) Hilbert spaces. This enables the discretization to be applied after the optimization (optimize-then-discretize). Using output-feedback SLS, we formulate the controller synthesis as a convex optimization problem. Structural constraints like sensor and communication delays, and locality constraints, are incorporated while preserving convexity, allowing parallel implementation and extending key SLS properties to infinite dimensions. The proposed approach and its benefits are demonstrated on a linear Boltzmann equation.

Keywords: Agents-based systems, Vision-based control, Decentralized control
Abstract: Localization via Pose Graph Optimization is a relevant task in both robotics and computer vision. Most formulations rely on the availability of relative rotations between nodes. However, this is a non-trivial assumption, as in many practical settings (e.g., vision-based camera systems) estimating relative rotations is hard. State-of-the-art methods for pose averaging such as SE-Sync are based on relaxations and are able to obtain low cost solutions, but require relative rotations. In this paper, we present an SE-Sync-style solution for translation-only pose averaging. Our method is based on an application of the Riemannian staircase, and we analyze the potential symmetries introduced by degenerate cases (nodes with only two translation measurements). In particular, we show that our method outperforms other baseline algorithms (such as block coordinate descent) in terms of precision and accuracy. The improvements are particularly apparent in the case of a random initial guess.

Keywords: Optimization algorithms, Distributed control, Autonomous robots
Abstract: Inter-robot loop closure detection is crucial in Collaborative Simultaneous Localization and Mapping (CSLAM) as it integrates individual robot trajectories into a unified map. However, accurately identifying all overlapping observations among a large group of robots poses significant challenges due to the limited computational and communication resources of multi-robot systems. Existing approaches typically rely on central nodes to facilitate data exchange and become resource-intensive as the problem size increases, resulting in substantial overhead and limiting scalability. To address this issue, we propose a distributed method that decentralizes the execution of resource-adaptive algorithms, enabling efficient data transmission without relying on a central node. Experimental results show that the proposed method is computationally efficient, outperforming benchmark algorithms while closely matching the performance of centralized counterparts.

Keywords: Observers for nonlinear systems, Autonomous systems, Robotics
Abstract: This paper investigates the estimation problem of the pose (orientation and position) and linear velocity of a rigid body, as well as the landmark positions, using an inertial measurement unit (IMU) and a monocular camera. First, we propose a globally exponentially stable (GES) linear time-varying (LTV) observer for the estimation of body-frame landmark positions and velocity, using IMU and monocular bearing measurements. Thereafter, using the gyro measurements, some landmarks known in the inertial frame and the estimates from the LTV observer, we propose a nonlinear pose observer on SO(3)times mathbb{R}^3. The overall estimation system is shown to be almost globally asymptotically stable (AGAS) using the notion of almost global input-to-state stability (ISS). Interestingly, we show that with the knowledge (in the inertial frame) of a small number of landmarks, we can recover (under some conditions) the unknown positions (in the inertial frame) of a large number of landmarks. Numerical simulation results are presented to illustrate the performance of the proposed estimation scheme.

Keywords: Algebraic/geometric methods, Lyapunov methods, Control of networks
Abstract: This work analyzes and develops some fundamental results for attitude consensus control of a network of rigid-body vehicles, considered as a multi-agent rigid body system (MARBS). The system is analyzed using a full rigid body dynamics model on TSO(3) for each vehicle (agent) in the network. Therefore, the state space of the system is TSO(3)^N, where N is the number of vehicles. Attitude synchronization control laws for each vehicle in the network to reach a consensus attitude with zero angular velocity are obtained, using a Morse-Lyapunov function. Some fundamental results on equilibria of the network under these attitude consensus control laws are obtained. We show that unlike cooperative control of multi-agent systems with highly simplified dynamics models for agents, like point particles or unicycles where the state space of the dynamics is modeled as a vector space, there are multiple equilibrium solutions possible for attitude consensus control laws for a MARBS with dynamics on TSO(3)^N. Further, the number of equilibria depends on the network graph topology. This is followed by numerical simulation results for two different network graphs, which show this network control framework to be effective in obtaining attitude consensus.

Keywords: Estimation, Networked control systems, Autonomous robots
Abstract: This paper introduces a novel algorithm for consistent Cooperative Visual-Inertial Navigation (CVIN) tailored for multi-robot systems using matrix Lie groups. The approach enables multiple robots to collaboratively estimate both their states and environmental features by integrating their own sensor data with shared information from neighboring robots. This shared information, which consists of commonly observed environmental features, creates geometric constraints between robots, thereby enhancing the individual robots' state estimation accuracy. The proposed CVIN algorithm extends the Invariant Extended Kalman Filter (IEKF) from single-robot localization to a multi-robot framework, leveraging the geometric constraints between robots to further improve localization accuracy. Thanks to the inherent properties of invariant error, the algorithm naturally maintains the consistency of the multi-robot localization system, as rigorously proven through an observability analysis. Moreover, the algorithm is fully distributed, relying solely on each robot's local measurements and information shared by one-hop communication neighbors. This structure ensures both robustness and scalability. Extensive Monte Carlo simulations demonstrate the superior performance of the proposed method in accurately estimating robot states and environmental features in 3D environments.

Keywords: Algebraic/geometric methods, Kalman filtering, Aerospace
Abstract: This paper proposes a novel geometric nonlinear filter for attitude and bias estimation on the Special Orthogonal Group SO(3) using matrix measurements. The structure of the proposed filter is similar to that of the continuous-time deterministic multiplicative extended Kalman filter (MEKF). The main difference with the MEKF is the inclusion of curvature correction terms in both the filter gain and gain update equations. These terms ensure that the proposed filter, named the Generalized SO(3)-MEKF, renders the desired equilibrium of the estimation error system to be almost globally uniformly asymptotically stable (AGUAS). More precisely, the attitude and bias estimation errors converge uniformly asymptotically to zero for almost all initial conditions except those where the initial angular estimation error equals pi radians. Moreover, in the case of small estimation errors, the proposed generalized SO(3)-MEKF simplifies to the standard SO(3)-MEKF with matrix measurements. Simulation results indicate that the proposed filter has similar performance compared to the latter. Thus, the main advantage of the proposed filter over the MEKF is the guarantee of (almost) global uniform asymptotic stability.

Keywords: Hybrid systems, Lyapunov methods, Power electronics
Abstract: We consider the problem of globally controlling a DC-DC Buck converter. A constrained switched differential inclusion model is derived to capture all possible modes of operation of the Buck converter circuit. In contrast with the traditional pulse-width modulation control method, a control Lyapunov function-based controller is designed. A hybrid closed-loop system is formulated to combine the continuous dynamics of the converter circuit and the hybrid dynamics of the controller. The hybrid closed-loop system induces global asymptotic stabilization of a desired output voltage setpoint. Simulations are provided to validate the results.

Keywords: Hybrid systems, Automata
Abstract: We consider the problem of verifying discrete-time dynamical systems against properties specified as universal timed co-Buchi automata. Universal co-Buchi automata capture many important properties such as safety, liveness, and those specified in Linear Temporal Logic. However, such automata do not consider timing constraints. We thus consider the problem of verification against timed automata which combine such properties with timing constraints. Our verification approach relies on first constructing a product of the system, the states of the timed automata, and the valuations of the clocks of the automata. To show that the traces of the system are accepted by the automata, we seek to ensure that they correspond to runs that visit a set of co-Buchi states only finitely often. To prove this, we append a counter value to the product that keeps track of the number of visitations to relevant co-Buchi states. We then search for a so-called timed co-Buchi barrier certificate (TCBC) to show that these states are visited only finitely often. We present a satisfiability modulo theory (SMT)-based approach to find these certificates. Finally, we demonstrate our approach on some case studies.

Keywords: Hybrid systems, Stability of hybrid systems, Biological systems
Abstract: The importance of pulse modulated regulation in non-basal testosterone secretion in males has motivated research to understand properties of periodic solutions to impulsive Goodwin's oscillators (IGOs). Despite the fruitful work studying IGOs, a result certifying robustness of asymptotically stable hybrid limit cycles is not available. In this paper, the IGO with a third-order continuous part of GnRH-LH-Te axis in males is described and analyzed within a hybrid systems framework. The notion of hybrid limit cycle is introduced and results on their existence are proposed for the IGO system. A sufficient condition related to Schur stability of the Jacobian of a hybrid Poincare map is presented for stability of hybrid limit cycles. In addition, we establish robustness properties to perturbations of stable hybrid limit cycles and validate the results numerically.

Keywords: Control applications, Reinforcement learning, Hybrid systems
Abstract: The operation of complex systems often requires balancing of conflicting goals. This also applies to the adsorption cooling facade system (AFS) that provides cooling power through a solar-driven adsorption-desorption cycle. Operation of the AFS is described with discrete states, called phases, that represent the different operational modes depending on, e. g., user requirements and weather conditions. In this hybrid system, transitions between phases are subject to state constraints, such as pressure conditions, and the state of switching valves represented by binary inputs. Using the example of the adsorption facade, this work adapts its hybrid system model to deep reinforcement learning (DRL) ensuring compliance of phase transitions with possible switching constraints. Exploiting this technique, an operational strategy based on DRL is developed. The choice of reward function is explained in detail, as it is crucial to the success of DRL-based policies. To illustrate the benefits of the DRL-based policy, it is compared to a conventional state-of-the-art rule-based operational strategy using a high-fidelity simulation model.

Keywords: Systems biology, Stochastic systems, Hybrid systems
Abstract: Recent studies combining cell size dynamics measurements with mathematical modeling have uncovered how living cells regulate their division rates to buffer statistical size fluctuations around a target setpoint. Building on previous work, we propose a framework based on stochastic hybrid systems (SHS), where cell size grows according to an arbitrary ordinary differential equation, and division occurs probabilistically at a size-dependent rate. We propose different forms of division rates that capture the distribution of added size from cell birth to division, implementing size control via three models: Adder (division occurs when the added size reaches a prescribed threshold); sizer (division occurs when the cell reaches a prescribed size) or adder-sizer mixture models. The proposed division rates are corroborated with data from exponentially growing bacterial cells. We further show how solving the underlying Chapman-Kolmogorov equation for the SHS provides transient solutions for the cell size distribution and its statistical moments, which can be further tested through experimental tracking of cell size over time. In summary, this contribution provides novel theoretical tools for characterizing cell size homeostasis mechanisms in diverse proliferating cell types, offering experimentally testable predictions and a framework that can be generalized to model clonal expansion of cells.

Keywords: Lyapunov methods, Stability of hybrid systems, Stability of nonlinear systems
Abstract: In the setting of continuous-time, discrete-time, and hybrid systems, including differential inclusions and difference inclusions, relaxations are given for Lyapunov functions to establish uniform global pre-asymptotic stability (UGAS) of compact sets. It is shown that for a compact set, if there exist a Lyapunov function and two lower semicontinuous functions that are positive definite with respect to the compact set and whose negations are upper bounds on the rate of change of the Lyapunov function during flows and jumps, respectively, then the compact set is UGAS. Under additional regularity conditions, conditions sufficient to show that a compact set is UGAS are further weakened to merely require the rate of change of the Lyapunov function is negative definite. Simplified conditions on hybrid time domains, compared to existing results, are given to establish that a set is UGAS for hybrid systems when the Lyapunov function is merely nonincreasing during either flows or jumps.

Keywords: Data driven control, Compartmental and Positive systems, Control of networks
Abstract: The paper presents and analyzes an adaptive data-driven controller that learns the optimal processing rate in a multi-unit processing network in the presence of disturbances. We formulate an optimization problem of linear cost, linear dynamics for the processing network model and an affine constraint on the dispatcher policy. A data-driven linear equation is constructed, based on which the online dispatcher policy is updated. An upper bound on the gap between the optimal cost and the cost incurred by the data-driven controller is extracted.

Keywords: Data driven control, Constrained control, Switched systems
Abstract: We present a novel direct data-driven method for computing constraint-admissible positive invariant sets for general nonlinear systems with compact constraint sets. Our approach employs machine learning techniques to lift the state space and approximate invariant sets using finite data. The invariant sets are parameterized as sublevel-sets of linear functions in the lifted space, which is suitable for control applications. We provide probabilistic guarantees of invariance through scenario optimization, with probability bounds on robustness against the uncertainty inherent in the data-driven framework. As the amount of data increases, these probability bounds approach 1. We use our invariant sets to design a model-free switched controller for nonlinear systems, which selects constraintenforcing controllers from a set of existing controllers. We demonstrate the practicality of our method by applying it to a nonlinear autonomous driving lane-keeping scenario.

Keywords: Building and facility automation, Reinforcement learning, Optimal control
Abstract: This paper introduces an innovative data-driven control methodology for variable air volume (VAV) HVAC systems in non-residential buildings with fluctuating occupancy patterns. A reinforcement learning approach leveraging model-free optimal linear quadratic control principles is developed. The algorithm utilizes Bellman dynamic programming to derive a quality function that enables control decisions based entirely on system-generated data, encompassing both building dynamics and occupant behavior. The approach is validated through simulations conducted on a recently installed HVAC-VAV system in a university building. Results demonstrate the method's effectiveness in maintaining optimal thermal comfort while minimizing airflow requirements of VAV units, directly correlating with reduced energy consumption per room. This approach proves particularly valuable in non-residential settings where occupancy levels vary significantly and unpredictably throughout the day.

Keywords: Data driven control, Linear systems, Optimal control
Abstract: This paper presents a data-driven approach to assess the controllability of discrete fractional-order systems using the nabla difference operator. The paper also proposes a data-driven approximation method for these systems, grounded in behavioral system theory. Instead of identifying a model that replicates the input-output dynamics, a direct modification was applied to the collected input-output data by solving a Hankel-structured low-rank approximation problem. The performance of the approximated nonparametric model in system response simulation is compared with that of an ARX model. The results indicate that the proposed method has similar simulation performance with improved accuracy. Furthermore, leveraging this approximation, many data driven controllers can be designed. As an example, a data-driven predictive control strategy is designed and applied to a discrete fractional-order system. Simulation results demonstrate that the controller successfully drives the system outputs to the desired positions.

Keywords: Data driven control, Optimization
Abstract: We consider the problem of forward reachability analysis of a black-box nonlinear system, using only the data from the system. We propose a method that computes an ellipsoidal set that tightly over-approximates the true reachable set using convex optimization. Exploiting the fact that a linear approximation of a nonlinear system is not unique, we find conditions of a linear time-varying system that approximates the nonlinear system such that its reachable set is guaranteed to include the reachable set of the unknown nonlinear system, textcolor{red}{ assuming that the Lipchitz coefficient of the nonlinear system is known.} Then, we formulate a convex optimization problem that jointly searches for the parameters of the linear system and its ellipsoidal over-approximate reachable set based only on the data to minimize the growth rate of the reachable set while ensuring the ellipsoid over-approximates the true reachable set. We demonstrate the advantages of the proposed method via two illustrative examples: an autonomous nonlinear system and the TRAF22 benchmark system, and compare the results with other state-of-the-art algorithms.