Keywords: Intelligent systems

Keywords: Predictive control for nonlinear systems, Stochastic optimal control, Autonomous systems
Abstract: In this work, we design a model predictive controller (MPC) for systems to satisfy Signal Temporal Logic (STL) specifications when the system dynamics are partially unknown, and only a nominal model and past runtime data are available. Our approach uses Gaussian process regression to learn a stochastic, data-driven model of the unknown dynamics, and manages uncertainty in the STL specification resulting from the stochastic model using Probabilistic Signal Temporal Logic (PrSTL). The learned model and PrSTL specification are then used to formulate a chance-constrained MPC. For systems with high control rates, we discuss a modification for improving the solution speed of the control optimization. In simulation case studies, our controller increases the frequency of satisfying the STL specification compared to controllers that use only the nominal dynamics model.

Keywords: Biomedical, Hybrid systems, Predictive control for linear systems
Abstract: This paper describes an optimized behavioral intervention Healthy Mom Zone (HMZ) for managing gestational weight gain featuring sequential decision-making using Hybrid Model Predictive Control (HMPC). Dynamical models incorporating both behavioral and physiological aspects of the problem are presented and estimated from HMZ participant data via constrained semi-physical modeling. Daily measurements are provided to a controller that ultimately makes judicious (though infrequent) augmentations on categorical dosages of healthy eating and physical activity intervention components. Consequently, an HMPC algorithm is required which must follow a logical sequence of control actions conforming to practical requirements. A case study shows the benefits relative to a conventional "IF-THEN" approach. The computational framework (both modeling and control) serves as the basis for the Healthy Mom Zone 2.0 intervention currently being evaluated in a randomized clinical trial (NIH R01DK134863, NCT05807594) at Penn State University.

Keywords: Stochastic optimal control, Randomized algorithms, Optimization
Abstract: This paper presents Robust samplE-based coVarIance StEering (REVISE), a multi-query algorithm that generates robust belief roadmaps for dynamic systems navigating through spatially dependent disturbances modeled as a Gaussian random field. Our proposed method develops a novel robust sample-based covariance steering edge controller to safely steer a robot between state distributions, satisfying state constraints along the trajectory. Our proposed approach also incorporates an edge rewiring step into the belief roadmap construction process, which provably improves the coverage of the belief roadmap. When compared to state-of-the-art methods [1], [2], REVISE improves median plan accuracy (as measured by Wasserstein distance between the actual and planned final state distribution) by 10x in multi-query planning and reduces median plan cost (as measured by the largest eigenvalue of the planned state covariance at the goal) by 2.5x in single-query planning for a 6DoF system. Our code is available at https://acl.mit.edu/REVISE/.

Keywords: Stochastic optimal control, Modeling, Manufacturing systems
Abstract: Dry methane reforming (DRM) offers a promising avenue to convert the greenhouse gases such as methane (CH4) and carbon dioxide (CO2) into syngas for synthesizing long-chain alkanes. However, catalyst deactivation remains a significant obstacle, primarily due to the deep cracking of CH4 and CO2 and the metal-catalyzed formation of carbon whiskers on the catalyst surface. To tackle this challenge, we introduce a comprehensive multiscale model that analyzes DRM reactions on nickel surfaces and predicts carbon whisker formation. This model integrates microscopic kinetic Monte Carlo (kMC) simulations for surface reactions, mesoscopic pelletscale modeling for whisker growth prediction, and macroscopic modeling to account for packed bed porosity and reactor pressure drop. Moreover, by applying time-steppers using the gap-tooth scheme, we significantly enhance computational efficiency, reducing computational time from 17 days to 6 hours with minimal loss in accuracy for whisker length prediction. Controlling catalyst deactivation is crucial for maintaining the efficiency and economic viability of the DRM process. Implementing optimal operation for reactor’s jacket temperature enables effectively mitigating catalyst deactivation and prolonging catalyst lifespan. Our multiscale model not only aids in understanding the factors affecting catalyst deactivation and surface reaction kinetics but also provides a robust framework for implementing MPC strategies to optimize and control the DRM process.

Keywords: Control system architecture, Sampled-data control, Predictive control for nonlinear systems
Abstract: Inner-outer-loop control is widely used for controlling mechanical systems with time-scale separation. Model Predictive Control (MPC) is a popular technique for systems that require command following with state and control constraints. We present a conversion algorithm for MPC-based inner- and outer-loop control by accounting for timing intricacies. For uncertain systems, we apply inner-outer-loop control based on predictive cost adaptive control (PCAC) to flight-control examples, with and without the conversion algorithm. Numerical examples show that inner-outer-loop PCAC improves command following and constraint satisfaction when the conversion algorithm is used. The investigation in this paper is numerical, and thus the contribution is technological rather than theoretical.

Keywords: Mechanical systems/robotics, Stochastic optimal control, Kalman filtering
Abstract: Modeling and controlling cable-driven snake robots is a challenging problem due to nonlinear mechanical properties such as hysteresis, variable stiffness, and unknown friction between the actuation cables and the robot body. This challenge is more significant for snake robots in ophthalmic surgery applications, such as the Improved Integrated Robotic Intraocular Snake (I2RIS), given its small size and lack of embedded sensory feedback. Data-driven models take advantage of global function approximations, reducing complicated analytical models' challenge and computational costs. However, their performance might deteriorate in case of new data unseen in the training phase. Therefore, adding an adaptation mechanism might improve these models' performance during snake robots' interactions with unknown environments. In this work, we applied a model predictive path integral (MPPI) controller on a data-driven model of the I2RIS based on the Gaussian mixture model (GMM) and Gaussian mixture regression (GMR). To analyze the performance of the MPPI in unseen robot-tissue interaction situations, unknown external disturbances and environmental loads are simulated and added to the GMM-GMR model. These uncertainties of the robot model are then identified online using a radial basis function (RBF) whose weights are updated using an extended Kalman filter (EKF). Simulation results demonstrated the robustness of the optimal control solutions of the MPPI algorithm and its computational superiority over a conventional model predictive control (MPC) algorithm.

Keywords: Optimal control, Optimization algorithms, Constrained control
Abstract: Many practical applications of optimal control are subject to real-time computational constraints. When applying model predictive control (MPC) in these settings, respecting timing constraints is achieved by limiting the number of iterations of the optimization algorithm used to compute control actions at each time step, resulting in so-called suboptimal MPC. This paper proposes a suboptimal MPC scheme based on the alternating direction method of multipliers (ADMM). With a focus on the linear quadratic regulator problem with state and input constraints, we show how ADMM can be used to split the MPC problem into iterative updates of an unconstrained optimal control problem (with an analytical solution), and a dynamics-free feasibility step. We show that using a warm-start approach combined with enough iterations per time-step, yields an ADMM-based suboptimal MPC scheme which asymptotically stabilizes the system and maintains recursive feasibility.

Keywords: Predictive control for linear systems, Human-in-the-loop control, Nonlinear systems identification
Abstract: This paper presents an ultrasound-informed recursive Koopman Model Predictive Control (RK-MPC) framework for enhancing gait assistance in individuals with impaired locomotion. By leveraging muscle activation and fatigue estimates derived from ultrasound imaging, the proposed approach improves the performance of predictive control strategies for functional electrical stimulation (FES) systems. The Koopman operator is used to linearize the nonlinear gait dynamics in a higher-dimensional space. We develop the framework for phase-specific dynamics for the stance and swing phases. We incorporate ultrasound-derived muscle fatigue measurements to design update laws for the Koopman operator to improve prediction accuracy. We present our simulation results for ankle motion control to track a desired gait trajectory while monitoring a prescribed fatigue level.

Keywords: Predictive control for linear systems, Linear parameter-varying systems, LMIs
Abstract: This paper introduces a novel LMI-based approach for synthesizing robust controllers for polytopic time-varying systems within the Model Predictive Control (MPC) framework. Compared to existing methods in the literature addressing the same problem, the proposed approach offers the following improvements and innovations for the offline design: the use of parameter-dependent Lyapunov matrices; the design of controllers of arbitrary order; the simultaneous design of all controller matrices; and the incorporation of the scalar arising from the S-procedure, which typically requires a time-consuming linear search, into the synthesis conditions affinely, allowing it to be optimized jointly with the other variables. The synthesis procedure is formulated as an iterative algorithm where LMIs are solved at each iteration. The effectiveness of the proposed strategy is demonstrated through numerical examples, highlighting its feasibility and superior performance, particularly in achieving larger invariant regions compared to existing methods.

Keywords: Predictive control for linear systems, Numerical algorithms, Linear systems
Abstract: In almost all algorithms for Model Predictive Control (MPC), the most time-consuming step is to solve some form of Linear Quadratic (LQ) Optimal Control Problem (OCP) repeatedly. The commonly recognized best option for this is a Riccati recursion based solver, which has a time complexity of mathcal{O}(N(n_x^3 + n_x^2 n_u + n_x n_u^2 + n_u^3)). In this paper, we propose a novel textit{Brunovsky Riccati Recursion} algorithm to solve LQ OCPs for Linear Time Invariant (LTI) systems. The algorithm transforms the system into Brunovsky form, formulates a new LQ cost (and constraints, if any) in Brunovsky coordinates, performs the Riccati recursion there, and converts the solution back. Due to the sparsity (block-diagonality and zero-one pattern per block) of Brunovsky form and the data parallelism introduced in the cost, constraints, and solution transformations, the time complexity of the new method is greatly reduced to mathcal{O}(n_x^3 + N(n_x^2 n_u + n_x n_u^2 + n_u^3)) if N threads/cores are available for parallel computing.

Keywords: Predictive control for linear systems, Robust control, Aerospace
Abstract: This paper introduces a robust Model Predictive Control approach in which a shrinking prediction horizon and a system input parameterization are exploited to control a linear system with set-bounded disturbances while satisfying state and control constraints. By exploiting input parameterization, the number of decision variables in the optimal control problem and the computational time can be reduced. The simulated spacecraft rendezvous maneuver is used to highlight the potential of the proposed approach for practical applications.

Keywords: Predictive control for linear systems, Robust control, Optimal control
Abstract: Model Predictive Control (MPC) is a powerful control strategy; however, its reliance on online optimization poses significant challenges for implementation on systems with limited computational resources. One possible approach to address this issue is to shorten the prediction horizon and adjust the conventional MPC formulation to enlarge the region of attraction. However, these methods typically introduce additional computational load. Recently, steady-state-aware MPC has been introduced to ensure output tracking and convergence to a given desired steady-state configuration while maintaining constraint satisfaction at all times without adding extra computational load. Despite its promising performance, steady-state-aware MPC does not account for external disturbances, which can significantly limit its applicability to real-world systems. This paper aims to advance the method further by enhancing its robustness against external disturbances. To achieve this, we adopt the tube-based design framework, which decouples nominal trajectory optimization from robust control synthesis, thereby requiring no additional online computational resources. Theoretical guarantees of the proposed methodology are shown analytically, and its effectiveness is assessed through simulations and experimental studies on a Parrot Bebop 2 drone.

Keywords: Predictive control for nonlinear systems, Iterative learning control, Constrained control
Abstract: For nonlinear systems that can be written in pseudo-linear form, we use iterative model predictive control (IMPC) for receding-horizon optimization. Pseudo-linear models, which are written in terms of state-dependent-coefficients (SDC’s), are widely used with the state-dependent Riccati equation. IMPC iterates over the horizon by updating the future control and state sequences until the future control sequence converges. To facilitate convergence, this paper explores the effectiveness of four root-finding methods and compares their performance with fixed-point iteration. At each iteration, a quadratic programming problem is solved using the current SDC’s to obtain a full-state-feedback controller. To assess the effectiveness of the root-finding methods, each technique is used to stabilize a collection of benchmark nonlinear systems.

Keywords: Predictive control for nonlinear systems, Optimal control, Computational methods
Abstract: Nonlinear Model Predictive Control (NMPC) is a powerful tool that can be applied to various control objects. However, it is known that the behavior realized by the NMPC varies greatly depending on the results of tuning the weight parameters of the objective function. In recent years, some methods have been proposed to automatically tune the weight parameters by utilizing tools such as machine learning, but these methods require a large number of prior trials. We propose a method to tune time-varying stage cost weight parameters in the NMPC for reference tracking without prior trials so as to minimize the tracking error. We formulate the NMPC as a min-max problem with the weight parameters as the maximizer and the control input as the minimizer, in order to optimize the weight parameter and the control input simultaneously. Furthermore, treating this optimization problem as a nonlinear receding-horizon differential game, we develop a real-time optimization algorithm. The effectiveness of the proposed method was confirmed by numerical simulations. We confirmed that the proposed method improves the control performance over the NMPC using fixed weight parameters designed by Bayesian optimization.

Keywords: Predictive control for nonlinear systems, Optimal control, Numerical algorithms
Abstract: This paper presents a novel framework for the continuation method of model predictive control based on optimal control problem with a nonsmooth regularizer. Via the proximal operator, the first-order optimality inclusion relation is reformulated into an equation system, to which the continuation method is applicable. In addition, we present constraint qualifications that ensure the well-posedness of the proposed equation system. A numerical example is also presented that demonstrates the effectiveness of our approach.

Keywords: Predictive control for nonlinear systems, Process Control, Lyapunov methods
Abstract: Cyberattacks may be performed on process control systems due to their integration of networking and computing with physical systems. Prior work in our group has developed detection strategies for nonlinear systems under sensor, actuator, and combined sensor and actuator attacks which can ensure, under characterizable conditions, that attacks can be detected before they cause safety issues. However, this work did not take into account the potential that an attacker could attempt to provide data to a process that causes an attack to remain undetected but that also is consistent with different process dynamics than those which the process has. This could be problematic if process models are consistently updated online based on the most recent process data, such that skewed data during routine process operation (or during operations designed to produce informative data for model updates) could lead to new models being developed for model-based controllers that are, in a sense, ``specified'' by the attacker. This work provides steps toward understanding how to prevent an attacker from achieving these types of data poisoning attacks for processes under Lyapunov-based economic model predictive control (LEMPC).

Keywords: Control applications, Power systems, Process Control
Abstract: This article proposes an efficient strategy for the real-time implementation of mixed-integer model predictive control (MI-MPC) in the context of an Electric Arc Furnace (EAF) application. Based on the EAF dynamical model and the discrete nature of the control inputs, we formulate mixed-integer programming problems within a predictive control framework to design the transformer power control system. To address the computational challenges of the MIP approaches, we introduce a model-informed optimal rounding strategy, which approximates the solution to the MI-MPC problem, delivering a sub-optimal solution that satisfies system constraints, enhances tracking performance and reduces computation time. The proposed MPC implementations are compared and evaluated using a Software-in-the-Loop (SIL) EAF simulator.

Keywords: Iterative learning control, Process Control, Optimization
Abstract: This paper presents a one-step predictive run-to-run controller (R2R-MPC) for the automation of mechanical serial sectioning (MSS), a destructive material analysis process. To address the inherent uncertainty and disturbances in the MSS process, a robust closed-loop approach is presented. The robust R2R-MPC models the uncertainty of the MSS process using a linear differential inclusion. As an analytical model of the MSS process is unavailable, the differential inclusion is identified from historical data. The R2R-MPC is posed as an optimization problem that computes incremental changes to the control input which minimize the worst-case material removal errors. This optimization-based controller is combined with a run-to-run controller to provide integral action that rejects constant disturbances and tracks constant reference removal rates. To demonstrate the efficacy of our robust R2R-MPC, we present simulation results which compare the presented controller with a conventional non-robust R2R.

Keywords: Predictive control for nonlinear systems, Stability of hybrid systems, Robotics
Abstract: Computing the receding horizon optimal control of nonlinear hybrid systems is typically prohibitively slow, limiting real-time implementation. To address this challenge, we propose a layered Model Predictive Control (MPC) architecture for robust stabilization of hybrid systems. A high level “hybrid” MPC is solved at a slow rate to produce a stabilizing hybrid trajectory, potentially sub-optimally, including a domain and guard sequence. This domain and guard sequence is passed to a low level “fixed mode” MPC which is a traditional, time-varying, state-constrained MPC that can be solved rapidly, e.g., using nonlinear programming (NLP) tools. A robust version of the fixed mode MPC is constructed by using tracking error tubes that are not guaranteed to have finite size for all time. Using these tubes, we demonstrate that the speed at which the fixed mode MPC is re-calculated is directly tied to the robustness of the system, thereby justifying the layered approach. Finally, simulation examples of a five link bipedal robot and a controlled nonlinear bouncing ball are used to illustrate the formal results.

Keywords: Autonomous robots, Sensor networks, Control applications
Abstract: This paper investigates the problem of informative path planning for a mobile robotic sensor network in spatially temporally distributed mapping. The robots are able to gather noisy measurements from an area of interest during their movements to build a Gaussian process (GP) model of a spatio-temporal field. The model is then utilized to predict the spatio-temporal phenomenon at different points of interest. To spatially and temporally navigate the group of robots so that they can optimally acquire maximal information gains while their connectivity is preserved, we propose a novel multi-step prediction informative path planning optimization strategy employing our newly defined local cost functions. By using the dual decomposition method, it is feasible and practical to effectively solve the optimization problem in a distributed manner. The proposed method was validated through synthetic experiments utilizing real-world data sets.

Keywords: Reinforcement learning, Intelligent systems, Robotics
Abstract: Understanding a reinforcement learning policy, which guides state-to-action mappings to maximize rewards, necessitates an accompanying explanation for human comprehension. In this paper, we introduce a set of linear temporal logic formulae designed to provide explanations for policies, and an algorithm for searching through those formulae for the one that best explains a given policy. Our focus is on explanations that elucidate both the ultimate objectives accomplished by the policy and the prerequisite conditions it upholds throughout its execution. The effectiveness of our proposed approach is illustrated through a simulated game of capture-the-flag and a car-parking environment.

Keywords: Machine learning, Stochastic systems, Uncertain systems
Abstract: This study introduces a novel theoretical framework for analyzing heteroscedastic Gaussian processes (HGPs) that identify unknown systems in a data-driven manner. Although HGPs effectively address the heteroscedasticity of noise in complex training datasets, calculating the exact posterior distributions of the HGPs is challenging, as these distributions are no longer multivariate normal. This study derives the exact means, variances, and cumulative distributions of the posterior distributions. Furthermore, the derived theoretical findings are applied to a chance-constrained tracking controller. After an HGP identifies an unknown disturbance in a plant system, the controller can handle chance constraints regarding the system despite the presence of the disturbance.

Keywords: Machine learning, Systems biology, Data driven control
Abstract: Existing methods for controlling circadian rhythms are based on standard mathematical models that are not personalized, as they are fitted to population-wide data. Moreover, the standard models are expressed in terms of variables, such as the core body temperature, which is not practically measurable. In this paper, we introduce a data-driven modeling approach that can use generic biometric measurements. We train a machine learning model with low-dimensional latent state representations and affine dynamics. Without any prior knowledge of the real system, we show that this model has long-horizon prediction power and outperforms traditional system identification methods. Applied to the optimal circadian entrainment problem, our model can calculate optimal control inputs that achieve over 80% of the performance of solutions computed directly from the ground truth model.

Keywords: Machine learning, Traffic control
Abstract: Accurate pedestrian intention estimation is crucial for the safe navigation of autonomous vehicles (AVs) and hence attracts a lot of research attention. However, current models often fail to adequately consider dynamic traffic signals and contextual scene information, which are critical for realworld applications. This paper presents a Traffic-Aware SpatioTemporal Graph Convolutional Network (TA-STGCN) that integrates traffic signs and their states (Red, Yellow, Green) into pedestrian intention prediction. Our approach introduces the integration of dynamic traffic signal states and bounding box size as key features, allowing the model to capture both spatial and temporal dependencies in complex urban environments. The model surpasses existing methods in accuracy. Specifically, TA-STGCN achieves a 4.75% higher accuracy compared to the baseline model on the PIE dataset, demonstrating its effectiveness in improving pedestrian intention prediction.

Keywords: Machine learning, Energy systems
Abstract: As the number of electric vehicles (EVs) continues to grow, the demand for charging stations is also increasing, leading to challenges such as long wait times and insufficient infrastructure. High-precision forecasting of EV charging demand is crucial for efficient station management, to address some of these challenges. This paper presents an approach to predict the charging demand at 15-minute intervals for the day ahead using a multivariate long short-term memory (LSTM) network with an attention mechanism. Additionally, the model leverages explainable AI techniques to evaluate the influence of various factors on the predictions, including weather conditions, day of the week, month, and any holiday. SHapley Additive exPlanations (SHAP) are used to quantify the contribution of each feature to the final forecast, providing deeper insights into how these factors affect prediction accuracy. As a result, the framework offers enhanced decision-making for infrastructure planning. The efficacy of the proposed method is demonstrated by simulations using the test data collected from the EV charging stations at California State University, Long Beach.

Keywords: Mechanical systems/robotics, Mechatronics, Sensor fusion
Abstract: This paper develops a 2D position estimation system for a robot moving over a plane using an actively controlled magnetic field. The position estimation system consists of two magnetic sensors on the moving robot and an actively controlled external rotating permanent magnet. The orientation of the external magnet is controlled to roughly point at the robot and a narrow localized magnetic field map is developed using deep learning around the pointing direction of the magnet. Using the magnetic fields at both sensors, the radial and polar positions of the robot are estimated using an unscented Kalman filter. The orientation of the magnet is then more finely controlled to point precisely at one of the magnetic sensors. This enables the design of an asymptotically stable nonlinear observer that provides enhanced accuracy in the radial position estimation of the robot. Experimental results are presented on the performance of the estimation system for a robot moving in a 2D plane.

Keywords: Reinforcement learning, Automata, Machine learning
Abstract: Offline reinforcement learning (RL) learns a policy from a fixed batch of data. However, the overestimation of the values rooted in the out-of-distribution actions limits the applicability of offline RL. It results in methods constraining or regularizing the learned policy based on the dataset. Hence, offline RL algorithms try to tackle this issue by adding a secondary component, e.g., new hyperparameters or generative models. Our proposed method aims to tackle this problem by taking a new perspective using two approaches on the offline RL. We tend to use deterministic finite automoton (DFA) to learn Offline-DFA (offline deterministic finite automaton) or ARM-DFA (association rule mining DFA) from the trajectories in the dataset without implementing any secondary component or constraining or regularizing the learned policy. This means that the learned policy does not face out-of-distribution actions. We use the Offline-DFA (or ARM-DFA) to guide the policy learning process based on the dataset’s trajectories. We propose a novel method called automata learning for offline RL with q-learning (ALOQ), which implements this perspective into the offline RL. We show the convergence guarantee of our proposed method to an optimal policy. In practice, Offline- DFA (or ARM-DFA) is used to learn the ground truth task, i.e., encoded by a reward machine, which allows the agent to learn an optimal policy. We demonstrate the performance of our proposed method by comparing it to the existing offline RL methods, converging to an optimal policy faster than the existing offline RL methods.

Keywords: Reinforcement learning, Control applications, Neural networks
Abstract: The design of feedback controllers is undergoing a paradigm shift from modelic (i.e., model-driven) control to datatic (i.e., data-driven) control. Canonical form of state space model is an important concept in modelic control systems, exemplified by Jordan form, controllable form and observable form, whose purpose is to facilitate system analysis and controller synthesis. In the realm of datatic control, there is a notable absence in the standardization of data-based system representation.

This paper for the first time introduces the concept of textit{canonical data form} for the purpose of achieving more effective design of datatic controllers. In a control system, the data sample in canonical form consists of a textit{transition} component and an textit{attribute} component. The former encapsulates the plant dynamics at the sampling time independently, which is a tuple containing three elements: a state, an action and their corresponding next state. The latter describes one or some artificial characteristics of the current sample, whose calculation must be performed in an online manner. The attribute of each sample must adhere to two requirements: (1) causality, ensuring independence from any future samples; and (2) locality, allowing dependence on historical samples but constrained to a finite neighboring set. The purpose of adding attribute is to offer some kinds of benefits for controller design in terms of effectiveness and efficiency. To provide a more close-up illustration, we present two canonical data forms: temporal form and spatial form, and demonstrate their advantages in reducing instability and enhancing training efficiency in two datatic control systems.

Keywords: Reinforcement learning, Machine learning, Statistical learning
Abstract: We consider a sequential multi-task problem, where each task is modeled as the stochastic multi-armed bandit with K arms. We assume the bandit tasks are adjacently similar in the sense that the difference between the mean rewards of the arms for any two consecutive tasks is bounded by a parameter. We propose two algorithms (one assumes the parameter is known while the other does not) based on UCB to transfer reward samples from preceding tasks to improve the overall regret across all tasks. Our analysis shows that transferring samples reduces the regret as compared to the case of no transfer. We provide empirical results for our algorithms, which show performance improvement over the standard UCB algorithm without transfer and a naive transfer algorithm

Keywords: Reinforcement learning, Machine learning, Neural networks
Abstract: Effective exploration is critical for reinforcement learning (RL) to understand the environment and achieve high performance, especially in sparse reward tasks. Existing methods often depend on task-specific prior knowledge for exploration guidance, which is unavailable in complex tasks, or rely on additional policy objectives to encourage exploration, which leads to sub-optimal solutions. This paper addresses these limitations by introducing a dual-policy guided exploration (DPE) mechanism, which learns an extra policy to promote the sample collection in unfamiliar state areas. In this mechanism, we define a curiosity signal using the random network distillation technique, which evaluates the state familiarity to the agent. Then, a separate courage policy is trained by maximizing the accumulated discounted curiosity signal, which aims to explore unfamiliar areas and discover potential high-value behaviors. By incorporating this mechanism into the distributional RL framework, we propose a Distributional Soft Actor-Critic algorithm for Sparse reward tasks (DSAC-S), which comprises three modules: dual-policy guided exploration, curiosity signal learning, and actor-critic training. We validate its effectiveness in several sparse reward tasks, including MountainCar, Sparse HalfCheetah, and IDSim. The results demonstrate that DSAC-S successfully conquers these tasks with its enhanced exploration ability, while the two baselines either fail to solve these problems or exhibit poor performance.

Keywords: Autonomous systems, Machine learning, Aerospace
Abstract: Path planning for Mars rovers presents significant challenges due to the diverse terrain, ranging from easily navigable areas to hazardous zones. Traditional methods typically classify terrain simply as passable or impassable, failing to account for the nuances of more moderately challenging areas. In this paper, we introduce a tiered terrain-aware path planning strategy, employing few-shot learning to classify and segment Martian terrain into levels of difficulty. The few-shot learning model, trained on Earth, is sent to the rover, enabling real-time processing of images from satellites or helicopters. The flexibility of few-shot learning, which requires minimal data and training time, enables quick updates and redeployment of the policy when necessary. Then, a modified A* path planning algorithm is proposed to generate paths on the tiered terrain maps. This algorithm takes into account the classified terrain tiers, allowing the rover to dynamically adjust its path based on real-time assessments. By integrating few-shot learning with the modified A* algorithm, the rover is equipped to make real-time intelligent decisions, enhancing its ability to navigate complex terrains effectively. Simulation results demonstrate the rover’s enhanced capability to navigate complex terrains, illustrating the effectiveness and flexibility of this integrated approach.

Keywords: Optimization algorithms, Optimization, Distributed parameter systems
Abstract: This paper addresses the challenge of online learning in contexts where agents accumulate disparate data and use different local learning algorithms due to resource constraints. We introduce the Switched Online Learning Algorithm (SOLA), designed to solve the heterogeneous online learning problem by fusing updates from diverse agents through a dynamic switching mechanism contingent upon their respective performance and available resources. We theoretically analyze the design of the selecting mechanism to ensure that the regret of SOLA is bounded. Our findings show that the number of changes in selection needs to be bounded by a parameter dependent on the performance of the different local algorithms. Additionally, two numerical experiments are presented to emphasize the effectiveness of SOLA, first on an online linear regression problem and then on an online classification problem with the MNIST dataset.

Keywords: Automotive control, Predictive control for nonlinear systems, Machine learning
Abstract: Autonomous drifting is a complex challenge due to the highly nonlinear dynamics and the need for precise real-time control, especially in uncertain environments. To address these limitations, this paper presents a hierarchical control framework for autonomous vehicles drifting along general paths, primarily focusing on addressing model inaccuracies and mitigating computational challenges in real-time control. The framework integrates Gaussian Process (GP) regression with an Alternating Direction Method of Multipliers (ADMM)-based iterative Linear Quadratic Regulator (iLQR). GP regression effectively compensates for model residuals, improving accuracy in dynamic conditions. ADMM-based iLQR not only combines the rapid trajectory optimization of iLQR but also utilizes ADMM's strength in decomposing the problem into simpler sub-problems. Simulation results demonstrate the effectiveness of the proposed framework, with significant improvements in both drift trajectory tracking and computational efficiency. Our approach reduced the RMSE lateral error by 38% and achieved a computation time approximately 25% of that required by IPOPT.

Keywords: Autonomous robots, Distributed control, Machine learning
Abstract: Swarms of Unmanned Aerial Vehicles (UAVs) have demonstrated enormous potential in many industrial and commercial applications. However, before deploying UAVs in the real world, it is essential to ensure they can operate safely in complex environments, particularly where communication is not available. To address this challenge, we propose a control-aware learning-based trajectory prediction algorithm that can enable communication-free UAV swarm control in a cluttered environment. Specifically, our proposed algorithm can enable each UAV to predict the planned trajectories of its neighbors in scenarios without communication. The predicted planned trajectories will serve as input to a distributed model predictive control (DMPC) approach. The proposed algorithm combines (1) a trajectory prediction model based on EvolveGCN, a Graph Convolutional Network (GCN) that can handle dynamic graphs, and (2) a KKT-informed training approach that applies the Karush-Kuhn-Tucker (KKT) conditions in the training process to encode DMPC information into the trained neural network. We evaluate our proposed algorithm in a funnel-like environment. The results show that the proposed algorithm outperforms state-of-the-art benchmarks, providing close-to-optimal control performance even when communication is unavailable.

Keywords: Machine learning, Reinforcement learning, Algebraic/geometric methods
Abstract: We consider the continuous-time temporal difference (TD) learning dynamics with nonlinear value function approximations, where there is a slim understanding of the convergence properties in irreversible regimes. Motivated by Krener's linearization idea ala Lie-brackets, we obtain conditions on the approximating value function and irreversibility coefficients under which the TD dynamics behaves close to a gradient flow. We show that our conditions lead to a set of partial differential equations, and study the existence of solutions using the algebraic invertibility of differential operators. Whenever a solution exists, using a perturbation analysis, we provide a stability result for nonlinear TD dynamics. As a by-product, we state the implications of the results for the classical case of linear approximations, where our conditions are algebraic, and easily verifiable.

Keywords: Reinforcement learning, Optimal control, Statistical learning
Abstract: Least-squares Temporal Difference (LSTD) is a model-free method to estimate the cost/value function of a Markov Decision Process (MDP). Interestingly, it is equivalent to estimating the state transition probabilities of the MDP through Maximum Likelihood (ML) and computing the cost/value function with (model-based) policy evaluation. However, LSTD does not incorporate prior knowledge of the state transition probabilities of the MDP. This paper proposes a data-based method, coined MAP-LSTD, that incorporates such prior knowledge without explicitly estimating the model. This method is analogous to LSTD in the sense that it is equivalent to estimating the state transition probabilities (model) through Maximum a Posteriori (MAP) instead of ML and computing the cost function with policy evaluation. Moreover, as LSTD, it can be implemented recursively. The key to our method is to model the unknown state distributions with Dirichlet distributions and encapsulate prior knowledge in the parameters of these distributions. An important concomitant is that the prior can be obtained from previous data samples and/or simulated samples, leveraging a model/simulator. Through an example, we illustrate how our method can speed up the convergence of the estimated cost function and reduce the estimation error significantly by exploiting prior knowledge derived from a model/simulator.

Keywords: Reinforcement learning, Reduced order modeling, Distributed parameter systems
Abstract: Dimensionality reduction is crucial for controlling nonlinear partial differential equations (PDE) through a ``reduce-then-design'' strategy, which identifies a reduced-order model and then implements model-based control solutions. However, inaccuracies in the reduced-order modeling can substantially degrade controller performance, especially in PDEs with chaotic behavior. To address this issue, we augment the reduce-then-design procedure with a policy optimization (PO) step. The PO step fine-tunes the model-based controller to compensate for the modeling error from dimensionality reduction. This augmentation shifts the overall strategy into reduce-then-design-then-adapt, where the model-based controller serves as a warm start for PO. Specifically, we study the state-feedback tracking control of PDEs that aims to align the PDE state with a specific constant target subject to a linear-quadratic cost. Through extensive experiments, we show that a few iterations of PO can significantly improve the model-based controller performance. Our approach offers a cost-effective alternative to PDE control using end-to-end reinforcement learning.

Keywords: Identification, Statistical learning, Linear systems
Abstract: We study the problem of learning clusters of partially observed linear dynamical systems from multiple input-output trajectories. This setting is particularly relevant when there are limited observations (e.g., short trajectories) from individual data sources, making direct estimation challenging. In such cases, incorporating data from multiple related sources can improve learning. We propose an estimation algorithm that leverages different data requirements for the tasks of clustering and system identification. First, short impulse responses are estimated from individual trajectories and clustered. Then, refined models for each cluster are jointly estimated using multiple trajectories. We establish end-to-end finite sample guarantees for estimating Markov parameters and state space realizations and highlight trade-offs among the number of observed systems, the trajectory lengths, and the complexity of the underlying models.

Keywords: Reinforcement learning
Abstract: This paper considers an online reinforcement learning algorithm that leverages pre-collected data (passive memory) from the environment for online interaction. We show that using passive memory improves performance and further provide theoretical guarantees for regret that turns out to be near-minimax optimal. Results show that quality of passive memory determines sub-optimality of the incurred regret. The proposed approach and results hold in both continuous and discrete state-action spaces.

Keywords: Machine learning, Optimal control, Aerospace
Abstract: Effective trajectory generation is essential for reliable on-board spacecraft autonomy. Among other approaches, learning-based warm-starting represents an appealing paradigm for solving the trajectory generation problem, effectively combining the benefits of optimization- and data-driven methods. Current approaches for learning-based trajectory generation often focus on fixed, single-scenario environments, where key scene characteristics, such as obstacle positions or final-time requirements, remain constant across problem instances. However, practical trajectory generation requires the scenario to be frequently reconfigured, making the single-scenario approach a potentially impractical solution. To address this challenge, we present a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging high-capacity transformer neural networks capable of learning from multimodal data sources. Specifically, our approach integrates transformer-based neural network models into the trajectory optimization process, encoding both scene-level information (e.g., obstacle locations, initial and goal states) and trajectory-level constraints (e.g., time bounds, fuel consumption targets) via multimodal representations. The transformer network then generates near-optimal initial guesses for non-convex optimization problems, significantly enhancing convergence speed and performance. The framework is validated through extensive simulations and real-world experiments on a free-flyer platform, achieving up to 30% cost improvement and 80 % reduction in infeasible cases with respect to traditional approaches, and demonstrating robust generalization across diverse scenario variations.

Keywords: Network analysis and control, Agents-based systems, Fault tolerant systems
Abstract: We study the problem of resilient leader-follower consensus in multi-agent systems (MASs) where some of the agents may malfunction. The objective is for nonfaulty agents to reach consensus on a reference value broadcast by leader agents despite possible misbehaviors of adversarial agents. To this end, we utilize the multi-hop weighted mean subsequence reduced (MW-MSR) algorithm to achieve the goal in multi-agent networks with directed topologies. We characterize a necessary and sufficient condition on graph structures for our algorithm to succeed, which is expressed in a novel notion of robust following graphs. With one-hop communication, our condition is tighter than the ones in the related resilient leader-follower consensus works. With multi-hop communication, we can have an even more relaxed graph condition for our algorithm to succeed. Lastly, we present numerical examples to verify the effectiveness of our algorithm.

Keywords: Network analysis and control, Control of networks, Cooperative control
Abstract: In this paper, we study the resilient distributed vector consensus problem in networks where agents have imprecise knowledge of their neighbors' states and must operate in the presence of adversarial agents. Unlike existing methods that assume exact state observations, we consider a more realistic scenario where each normal agent's knowledge of its neighbors' states is restricted to a bounded region in mathbb{R}^d, referred to as the imprecision region. The size of this region reflects the degree of uncertainty and dynamically shrinks as agents update their states and approach consensus. We analyze the required decrease in the imprecision region at each iteration to ensure consensus, even in the presence of adversarial agents and state imprecision. By leveraging novel geometric insights, we present a method to ensure that each normal agent can consistently find a point within the convex hull of its normal neighbors' true states, despite not knowing these states exactly. We also provide numerical evaluations to demonstrate the practical effectiveness of our approach in distributed optimization contexts.

Keywords: Agents-based systems, Autonomous systems, Cooperative control
Abstract: This paper presents a distributed control framework for leader-follower multi-agent systems with general linear dynamics to achieve consensus alongside prescribed transient performance. The multi-agent system is in a leader-follower configuration with only a set of agents selected as leaders and controlled via external inputs. In particular, we propose a distributed control framework comprising a prescribed performance controller for the leaders, while the followers are governed solely by a consensus protocol. When the decay rate of the performance functions is within a sufficient bound, we show that this distributed control law achieves consensus for the entire multi-agent system, with the guarantee that the trajectories of the consensus errors remain bounded by a time-varying prescribed transient performance function. Finally, the proposed results are illustrated through numerical examples.

Keywords: Agents-based systems, Game theory, Distributed control
Abstract: In multi-agent systems with coupled objectives and/or constraints, agents may misreport information to achieve individual gains. This issue is exacerbated when agents possess local decision-making power, such as in multi-agent trajectory planning, where the increased autonomy amplifies individual benefits at the expense of a higher social cost. To overcome this problem, we leverage the Vickrey-Clarke-Grove (VCG) framework and propose a strategyproof, two-stage mechanism. We further extend this mechanism to prioritized planning and prevent agents from manipulating their priority.

Keywords: Autonomous systems, Agents-based systems
Abstract: This paper examines the consensus problem of multi-agent systems with asymmetric higher-order interactions amongst neighboring agents, where the asymmetric matrix-valued inter-agent coupling mechanism is employed to model the higher-order interactions amongst neighboring agents, and the associated pairwise interaction is also supposed to be irreciprocal. First, we reveal the fundamental mechanism of how asymmetric higher-order interactions influence the error elimination between a pair of neighboring agents according to the asymmetric/skew-symmetric decomposition of square matrices. Second, we provide sufficient conditions to guarantee the convergence of asymmetric HOI networks from the network topology perspective. By examining the null space of the Laplacian matrix associated with asymmetric HOI networks, we finally present necessary/sufficient conditions for achieving average consensus on asymmetric HOI networks.

Keywords: Optimal control, Estimation, Autonomous systems
Abstract: This paper considers the problem of localizing a set of nodes in a wireless sensor network where both the node positions and communication model parameters are unknown. We assume that a multi-agent system moves in formation through the environment, taking measurements of the Received Signal Strength, and seek a controller that optimizes a performance metric based on the Fisher Information Matrix. We propose a two-stage formation-based receding horizon approach that alternates between estimating the parameters and determining where to move and how to scale the formation to maximally inform the estimation problem. We apply a Dynamic Programming approach to solve the multi-stage look ahead control problem of the first stage, followed by a Particle Swarm Optimization algorithm to determine the best formation configuration in the second stage. We demonstrate our approach using different formation structures and compare it against multiple baselines.

Keywords: Cooperative control, Agents-based systems, Adaptive control
Abstract: This paper presents a distributed adaptive control strategy for multi-agent systems with heterogeneous time-varying nonlinear dynamics, integrating obstacle and collision avoidance mechanisms. We propose an adaptive control strategy designed to ensure leader-following formation consensus while effectively managing collision and obstacle avoidance using potential functions. By integrating neural network (NN) based estimation and adaptive tuning laws, the proposed strategy ensures consensus, safety and stability in leader-following formations under fixed topologies. The NNs enhance the adaptability of the proposed architecture, enabling agents to learn and adjust to environmental and system changes.

Keywords: Cooperative control, Distributed control, Output regulation
Abstract: In this paper, we investigate the bipartite formation tracking control problem with output regulation for heterogeneous multi-agent systems (HMAS) under distributed communication. The HMAS consists of linear and nonlinear agents, where we utilize a leader-follower structure and the properties of the Laplacian matrix to design the formation controller. A distributed observer is designed to address the issue about the lack of leader state information. We propose a consensus based control protocol to achieve bipartite formation and tracking of the desired reference trajectory. System stability is verified using Lyapunov stability theory and asymptotic stability is guaranteed under the basic output regulation requirements. Some simulation results are provided to demonstrate the feasibility and effectiveness of the designed controller.

Keywords: Cooperative control, Optimization, Human-in-the-loop control
Abstract: In this study, we explore a collaborative multi-agent stochastic linear bandit setting involving a network of (N) agents that communicate locally to minimize their collective regret while keeping their expected cost under a specified threshold (tau). Each agent encounters a distinct linear bandit problem characterized by its own reward and cost parameters, i.e., local parameters. The goal of the agents is to determine the best overall action corresponding to the average of these parameters, or so-called global parameters. In each round, an agent is randomly chosen to select an action based on its current knowledge of the system. This chosen action is then executed by all agents, then they observe their individual rewards and costs. We propose a safe distributed upper confidence bound algorithm, so called textit{MA-OPLB}, and establish a high probability bound on its (T)-round regret. MA-OPLB utilizes an accelerated consensus method, where agents can compute an estimate of the average rewards and costs across the network by communicating the proper information with their neighbors. We show that our regret bound is of order mathcal{O}left(frac{d}{tau-c_0}frac{log(NT)^2}{sqrt{ N}}sqrt{frac{T}{log(1/|lambda_2|)}}right), where lambda_2 is the second largest (in absolute value) eigenvalue of the communication matrix, and tau-c_0 is the known cost gap of a feasible action. We also experimentally show the performance of our proposed algorithm in different network structures.

Keywords: Decentralized control, Cooperative control, Agents-based systems
Abstract: In this paper, a novel Density-Driven Formation Control (D2FC) framework is presented for the formation control of swarm robots, driven by a given reference density map. This work is specifically designed for search and rescue (SAR) missions. Leveraging principles from optimal transport theory and decentralized control, our approach enables a swarm of agents to distribute themselves according to a given density distribution, focusing on high-priority areas without the need for centralized coordination. By integrating the virtual leader paradigm and formation, the swarm adapts efficiently to cover areas of interest. To improve the area coverage efficiency for achieving high detection rate of victims on a disaster site, a new information update model is proposed. Simulation results demonstrate that swarms utilizing the proposed D2FC method significantly outperform uncoordinated swarms in victim detection across various scenarios. The proposed approach enhances the effectiveness of SAR operations and can be extended to other applications requiring coordinated multi-agent systems.

Keywords: Reinforcement learning, Agents-based systems, Machine learning
Abstract: We introduce an approach called Multi-Agent State Abstraction with Causal dynamics and Reward Machine Learning (Multi-SACReM), designed to enhance the efficiency of reinforcement learning (RL) in multi-agent environments. By integrating causal information through state abstraction, Multi-SACReM enables agents to learn more effectively in complex, dynamic settings. In our framework, each agent employs a reward machine, to optimize its behavior. Multi-SACReM identifies and eliminates redundant causal relationships, allowing agents to focus on essential interactions and optimize policies accordingly. Our approach works in both decentralized settings, where agents learn independently using local observations, and centralized RL environments. In centralized RL, a controller coordinates agents using global information to optimize collective behavior. We evaluate Multi-SACReM through case studies comparing decentralized versus centralized approaches. Results show Multi-SACReM effectively learns causal models with reward machines, comparing decentralized versus centralized approaches in multi-agent settings.

Keywords: Switched systems, Optimal control, Hybrid systems
Abstract: This paper formulates and addresses the linear quadratic regulator problem concerning asynchronously switched multi-agent systems comprising agents with different dynamics. The objective is to derive closed-form solutions that minimize an overall performance index instead of numerical solutions or sub-optimal solutions. The mixed-integer programming problem is transformed into a continuous optimization problem suitable for solutions based on the maximum principle. We decompose the co-optimization problem involving switching variables coupled with control inputs into multiple mutually independent sub-problems and derive analytical bang-bang type solutions for the switching variables. Additionally, the conditions for the existence of distributed solutions for optimal control and switching functions are presented. Finally, numerical examples are conducted to validate the effectiveness of the proposed method.

Keywords: Reinforcement learning, Automata, Machine learning
Abstract: We study the use of large language models (LLMs) to integrate high-level knowledge in stochastic games using reinforcement learning with reward machines to encode non-Markovian and Markovian reward functions. In non- cooperative games, one challenge is to provide agents with knowledge about the task efficiently to speed up the convergence to an optimal policy. We aim to provide this knowledge in the form of deterministic finite automata (DFA) generated by LLMs(LLM-generated DFA). Additionally, we use reward machines (RMs) to encode the temporal structure of the game and the non-Markovian or Markovian reward functions. Our proposed algorithm, LLM-generated DFA for Multi-agent Reinforcement Learning with Reward Machines for Stochastic Games (StochQ-RM), can learn an equivalent reward machine to the ground truth reward machine (specified task) in the environment using the LLM-generated DFA. Additionally, we propose DFA-based q-learning with reward machines (DBQRM) to find the best responses for each agent using Nash equilibrium in stochastic games. Despite that the LLMs are known to hallucinate, we show that our method is robust and guaranteed to converge to an optimal policy. Furthermore, we study the performance of our proposed method in three case studies.

Keywords: Networked control systems, Network analysis and control, Linear systems
Abstract: In this paper, we consider the problem of distributed reachable set computation for multi-agent systems (MASs) interacting over an undirected, stationary graph. A full state-feedback control input for such MASs depends no only on the current agent's state, but also of its neighbors. However, in most MAS applications, the dynamics are obscured by individual agents. This makes reachable set computation, in a fully distributed manner, a challenging problem. We utilize the ideas of polytopic reachable set approximation and generalize it to a MAS setup. We formulate the resulting sub-problems in a fully distributed manner and provide convergence guarantees for the associated computations. The proposed algorithm's convergence is proved for two cases: static MAS graphs, and time-varying graphs under certain restrictions.

Keywords: Optimal control, Reinforcement learning, Cooperative control
Abstract: Avoiding collisions between unmanned aerial vehicles (UAVs) during formation is a challenge in leaderfollower formation control. This paper presents a two-stage optimal formation control scheme. Firstly, an end-to-end motion planning method is designed for collision-free motion during the formation process, utilizing deep reinforcement learning. Specifically, an actor-critic network with two hidden layers is constructed, incorporating an obstacle avoidance module based on Bounding Volume Hierarchy (BVH) trees. A novel composite reward function, consisting of target penalty and safety penalty, is proposed to guide the training of the actor-critic network. Furthermore, a low-complexity consensus protocol is developed to synchronize the state of the followers and leaders, and the system is rigorously proven to be asymptotically stable. Finally, the effectiveness of the proposed method is validated through simulations involving a set of UAVs.

Keywords: Optimization, Optimization algorithms, Cooperative control
Abstract: This paper investigates distributed zeroth-order feedback optimization in multi-agent systems with coupled constraints, where each agent operates its local action vector and observes only zeroth-order information to minimize a global cost function subject to constraints in which the local actions are coupled. Specifically, we employ two-point zeroth-order gradient estimation with delayed information to construct stochastic gradients, and leverage the constraint extrapolation technique and the averaging consensus framework to effectively handle the coupled constraints. We also provide convergence rate and oracle complexity results for our algorithm, characterizing its computational efficiency and scalability by rigorous theoretical analysis. Numerical experiments are conducted to validate the algorithm's effectiveness.

Keywords: Networked control systems, Uncertain systems, Distributed control
Abstract: This paper introduces a novel framework for multi-agent trajectory optimization under unknown deterministic uncertainty. Many systems are affected by deterministic disturbances, such as environmental effects, system degradation, etc., which cannot be accurately modeled using stochastic signals. Therefore, it is crucial to develop trajectory optimization frameworks that ensure safety despite these disturbances. To this end, we focus on solving a multi-agent trajectory optimization problem involving robust constraints, such as collision avoidance, that must be satisfied for all possible realizations of uncertainty lying in an ellipsoidal set. Conventional robust optimization techniques that are used to address such problems are computationally expensive and struggle when dealing with numerous constraints. To overcome this, we propose tighter approximations of robust constraints that significantly reduce computational complexity without compromising safety. Furthermore, leveraging these constraint approximations, we introduce a distributed robust optimization framework for decentralized multi-agent robust trajectory optimization based on the Alternating Direction Method of Multipliers (ADMM). This framework allows agents to optimize their trajectories without sharing control parameters or system information (e.g., dynamics), thereby preserving data security. The effectiveness of the proposed robust constraint approximations and the scalability of the presented distributed framework are demonstrated through simulation experiments with up to 164 agents.

Keywords: Decentralized control, Distributed control, Large-scale systems
Abstract: In this work, we consider the problem of coordinating a collection of n^th-order integrator systems. The coordination is achieved through the novel serial consensus design; this control design achieves a stable closed-loop system while adhering to the constraint of only using local and relative measurements. Earlier work has shown that second-order serial consensus can stabilize a collection of double integrators with scalable performance conditions independent of the number of agents and topology. This paper generalizes these performance results to an arbitrary order n ≥ 1. The derived performance bounds depend on the condition number, measured in the vector-induced maximum matrix norm, of a general diagonalizing matrix. We precisely characterize how a minimal condition number can be achieved. Third-order serial consensus is illustrated through a case study of PI-controlled vehicular formation, where the added integrators are used to mitigate the effect of unmeasured load disturbances. The theoretical results are illustrated through examples.

Keywords: Robotics, Reduced order modeling, Emerging control applications
Abstract: A major challenge in soft robotics is model-based control. Many common models for soft robots suffer a loss of mechanical structure when model reduction is performed. This is a significant problem for control, since many control methods, e.g. energy-based control, rely heavily on this structure. In this paper, we present an alternative finite dimensional reduction method that preserves the mechanical structure of the model of a soft robot. Using a form of finite element Galerkin projection, our approach incorporates this reduction process into a variational framework that is suitable for control applications. As a proof-of-principle, we present the model of a planar soft robot with preserved mechanical structure and a simple control paradigm. With this method, soft robots can be described in a form that is suitable for control strategies that depend on their mechanical structure, such as energy-based control.

Keywords: Robotics, Decentralized control, Adaptive control
Abstract: We propose reaching steps towards the real-time strain control of multiphysics, multiscale continuum soft robots. To study this problem fundamentally, we ground ourselves in a model-based control setting enabled by mathematically precise dynamics of a soft robot prototype. Poised to integrate, rather than reject, inherent mechanical nonlinearities for embodied compliance, we first separate the original robot dynamics into separate subdynamics --- aided by a perturbing time-scale separation parameter. Second, we prescribe a set of stabilizing nonlinear backstepping controllers for regulating the resulting subsystems' strain dynamics. Third, we study the interconnected singularly perturbed system by analyzing and establishing its stability. Fourth, our theories are backed up by fast numerical results on a single arm of the Octopus robot arm. We demonstrate strain regulation to equilibrium, in a significantly reduced time, of the whole-body reduced-order dynamics of an infinite degrees-of-freedom soft robot. This paper communicates our thinking within the backdrop of embodied intelligence: it informs our conceptualization, formulation, computational setup, and yields improved control performance for infinite degrees-of-freedom soft robots.

Keywords: Robotics, Autonomous systems, Control applications
Abstract: This paper introduces a novel approach based on density functions for safe navigation in dynamic environments with time-varying obstacles and targets. We analytically construct time-varying density functions to address the challenges of dynamic safe navigation. Using this approach, we develop a safe feedback controller that ensures safety and almost everywhere stability in dynamic environments. The proposed framework is demonstrated on complex robotic systems, including the safe navigation of a quadruped robot with time-varying obstacles and sensor uncertainty, as well as a robotic arm tracking a time-varying target with static obstacles.

Keywords: Robotics, Control applications, Lyapunov methods
Abstract: We present LQR-CBF-RRT*, an incremental sampling-based algorithm for offline motion planning. Our framework leverages the strength of Control Barrier Functions (CBFs) and Linear Quadratic Regulators (LQR) to generate safety-critical and optimal trajectories for general affine control systems. This work uses CBF for safety guarantees and LQRs for optimal control synthesis during edge extensions. Traditional CBF methods involve Quadratic Programs (QPs), which add computational overhead and can sometimes be infeasible. Conversely, LQR-based controllers typically employ first-order Taylor approximations for nonlinear systems, necessitating consistent recalculations. To enhance motion planning efficiency, our framework directly verifies CBF constraints during the planning process, thereby eliminating the need for QP solutions. Additionally, we cache optimal LQR gain matrices in a hash table to bypass re-computation during local linearizations in the rewiring phase. To further boost sampling efficiency, we integrate the Cross-Entropy Method. Our results demonstrate that the proposed planner outperforms existing algorithms in computational efficiency and exhibits robust performance in real-world experiments.

Keywords: Robotics, Estimation, Sensor fusion
Abstract: To achieve dynamic legged locomotion, real-time acquisition of the robot floating base state is critical. However, current full-coupled state estimation solutions may suffer from convergence issues. To address this, a two-stage state estimator that uses proprioceptive sensors is proposed to estimate the floating base's velocity and pose. The approach decomposes the floating-base state estimation problem into base orientation estimation and base velocity and position estimation, resulting in a two-stage multi-sensor fusion algorithm with stability guarantees. Additionally, a covariance inflation method is introduced to consider the influences of contact switching by adjusting noise covariances. The proposed state estimator has been successfully implemented and verified on a real quadrupedal robot, supporting dynamic motions such as multi-gaited locomotion and push recovery.

Keywords: Robotics, Hybrid systems, Optimal control
Abstract: In this paper, we present a novel control framework to achieve robust push recovery on bipedal robots while locomoting. The key contribution is the unification of hybrid system models of locomotion with a reduced-order model predictive controller determining: foot placement, step timing, and ankle control. The proposed reduced-order model is an augmented Linear Inverted Pendulum model with zero moment point coordinates; this is integrated within a model predictive control framework for robust stabilization under external disturbances. By explicitly leveraging the hybrid dynamics of locomotion, our approach significantly improves stability and robustness across varying walking heights, speeds, step durations, and is effective for both flat-footed and more complex multi-domain heel-to-toe walking patterns. The framework is validated with high-fidelity simulation on Cassie, a 3D underactuated robot, showcasing real-time feasibility and substantially improved stability. The results demonstrate the robustness of the proposed method in dynamic environments.

Keywords: Robotics, Optimal control, Biologically-inspired methods
Abstract: There has been significant advancement in legged robot's agility where they can show impressive acrobatic maneuvers, such as parkour. These maneuvers rely heavily on posture manipulation. To expand the stability and locomotion plasticity, we use the multi-modal ability in our legged-aerial platform, the Husky Beta, to perform thruster-assisted walking. This robot has thrusters on each of its sagittal knee joints which can be used to stabilize its frontal dynamic as it walks. In this work, we perform a simulation study of quadruped narrow-path walking with Husky Beta, where the robot will utilize its thrusters to stably walk on a narrow path. The controller is designed based on a centroidal dynamics model with thruster and foot ground contact forces as inputs. These inputs are regulated using a QP solver to be used in a model predictive control framework. In addition to narrow-path walking, we also perform a lateral push-recovery simulation to study how the thrusters can be used to stabilize the frontal dynamics of our robot.

Keywords: Robotics, Optimization, Energy systems
Abstract: The paper considers energy-oriented trajectory optimization of a two-link robot inspired by thermodynamics,particularly on the second law. A formulation of thermodynamic principles is developed within the framework of dynamical systems, extending these principles for use in multi-domain systems. The approach involves periodic motions, with the cyclic-averaged subsystem energies replacing temperature in an extended definition of entropy generation. In a representative problem, the robot arm moves between two arbitrary points under the influence of an external force and performs work by repeatedly elevating loads. The trajectories to be optimized were parameterized with a Fourier decomposition. Simulation results indicate that the proposed cost function effectively minimizes energy consumption with some advantages over direct minimization of the supplied energy. Specifically, only partial knowledge of the dissipation characteristics is needed, and the proposed cost function exhibits improved convexity properties near the optimal point in comparison to energy supply.

Keywords: Robotics, Predictive control for nonlinear systems, Optimization
Abstract: Existing loco-manipulation frameworks employ nonlinear models that require complex commands and parameters that are difficult to plan and adjust for application on different tasks. This work presents a model predictive control (MPC) framework for dynamic loco-manipulation that enables versatile whole-body motion while ensuring safety. Our approach simplifies the user command to object trajectory only, then optimizes for whole-body motion for precise object tracking, which can be applied to disparate tasks without formulation change. This approach also allows for predictive collision avoidance for both the robot and the object, both during and after contact between the object and the robot system. We demonstrated the efficient model dynamics and collision avoidance method of this unified formulation through simulation results on the Unitree Aliengo with a custom-made arm, showing the ability to transport objects weighing 20% of the robot mass with precision of within 3 cm error from the final position. Further, our proposed method is able to throw an object at a moving target traveling away at 1 m/s. Leveraging this prediction accuracy, we demonstrated "flight phase" collision avoidance without a pre-planned trajectory by throwing a 1 kg object over a 1.2 m wall.

Keywords: Robotics, Reduced order modeling, Biologically-inspired methods
Abstract: In this study, our aim is to evaluate the effectiveness of thruster-assisted steep slope walking for the Husky Carbon, a quadrupedal robot equipped with custom-designed actuators and plural electric ducted fans, through simulation prior to conducting experimental trials. Thruster-assisted steep slope walking draws inspiration from wing-assisted incline running (WAIR) observed in birds, and intriguingly incorporates posture manipulation and thrust vectoring, a locomotion technique not previously explored in the animal kingdom. Our approach involves developing a reduced-order model of the Husky robot, followed by the application of an optimization-based controller utilizing collocation methods and dynamics interpolation to determine control actions. Through simulation testing, we demonstrate the feasibility of hardware implementation of our controller.

Keywords: Robotics, Reinforcement learning, Optimization
Abstract: We propose a novel hierarchical reinforcement learning framework for quadruped locomotion over challenging terrain. Our approach incorporates a two-layer hierarchy in which a high-level policy (HLP) selects optimal goals for a low-level policy (LLP). The LLP is trained using an on-policy actor-critic RL algorithm and is given footstep placements as goals. We propose an HLP that does not require any additional training or environment samples and instead operates via an online optimization process over the learned value function of the LLP. We demonstrate the benefits of this framework by comparing it with an end-to-end reinforcement learning (RL) approach. We observe improvements in its ability to achieve higher rewards with fewer collisions across an array of different terrains, including terrains more difficult than any encountered during training.

Keywords: Robotics, Visual servo control
Abstract: This paper introduces the Visual Inverse Kinematics (VIK) problem to fill the gap between robot Inverse Kinematics (IK) and visual servoing (VS) control. IK aims to find joint configurations that achieve an explicitly given end effector pose. VS aims to drive the joint configurations to achieve an end effector position that is assumed to be kinematically feasible but is specified by vision constraints. Our proposed problem, VIK, aims to find a joint configuration that is kinematically feasible and achieves a specific image configuration. With respect to IK, we do not have a given end effector pose, and with respect to VS, we do not necessarily have an exact desired image configuration, but only vision-based constraints. In this work, we develop a formulation of the VIK problem with a Field of View (FoV) constraint, enforcing the visibility of an object from a camera on the robot. Our proposed solution is based on the idea of adding a virtual kinematic chain connecting the physical robot and the object; the FoV constraint is then equivalent to a joint angle kinematic constraint. Along the way, we introduce multiple vision-based cost functions to fulfill different objectives. We solve this formulation of the VIK problem using a method that involves a semidefinite program (SDP) constraint followed by a rank minimization algorithm. The performance of this method for solving the VIK problem is validated through simulations.

Keywords: Modeling, Robotics, Mechatronics
Abstract: In this work we present a dynamic model for a novel compliant worm robot, that is designed to travel in corrugated pipe environments. The robot uses the anisotropic interaction between its fins and pipe ridges to achieve locomotion when its body undergoes cable-driven peristaltic movement. The model, represented as a hybrid system, incorporates the deformation of the fins during contact and accounts for the switching logic of the anchoring positions. With the actuated body length as input, the model captures the movement dynamics and continuous locomotion of the robot. The model is used to optimize the gait, specifically the waveform of the body length change, for achieving the trade-off between travel speed and power efficiency, and to optimize fin design, revealing that large anisotropy enhances both travel speed and energy savings. The approach is supported with simulation results.

Keywords: Optimal control, Biologically-inspired methods, Robotics
Abstract: Soft robots offer a frontier in robotics with enormous potential for safe human-robot interaction and agility in uncertain environments. A stepping stone towards unlocking their potential is a control theory tailored to soft robotics, including a principled framework for gait design. We analyze the problem of optimal gait design for a soft crawling body – the crawler. The crawler is an elastic body with the control signal defined as actuation forces between segments of the body. We consider the simplest such crawler: a two-segmented body with a passive mechanical connection modeling the viscoelastic body dynamics and a symmetric control force modeling actuation between the two body segments. The model accounts for the nonlinear asymmetric friction with the ground, which together with the symmetric actuation forces enable the crawler’s locomotion. Using a describing-function analysis, we show that when the body is forced sinusoidally, the optimal actuator contraction frequency corresponds to the body’s natural frequency when operating with only passive dynamics. We then use the framework of Optimal Periodic Control (OPC) to design optimal force cycles of arbitrary waveform and the corresponding crawling gaits. We provide a hill-climbing algorithm to solve the OPC problem numerically. Our proposed methods and results inform the design of optimal forcing and gaits for more complex and multi-segmented crawling soft bodies.

Keywords: Lyapunov methods, Robotics, Mechanical systems/robotics
Abstract: This paper presents a novel approach for 3D trajectory tracking of underactuated autonomous underwater vehicles (AUVs). Unlike previous works, this method does not require the position tracking error to be zero but instead allows it to take a nonzero constant value. This setup resembles a vehicle being towed by a rope, where one end follows the reference trajectory. In this towing-type tracking scheme, the vehicle's desired attitude (defined by its pitch and yaw angles) is aligned with the polar and azimuth angles from the vehicle to a target point on the reference trajectory. Notably, this desired attitude is state-dependent, distinguishing it from the polar and azimuth angles of the reference trajectory's tangent. However, this paper demonstrates that, at least in certain special cases, the proposed tracking method ensures the vehicle's desired attitude converges to the reference trajectory's tangent. Numerical studies are conducted to validate the effectiveness of the proposed scheme.

Keywords: Optimal control, Predictive control for linear systems, Robotics
Abstract: This paper proposes a robot action planning scheme that provides an efficient and probabilistically safe plan for a robot interacting with an unconcerned human--someone who is either unaware of the robot's presence or unwilling to engage in ensuring safety. The proposed scheme is predictive, meaning that the robot is required to predict human actions over a finite future horizon; such predictions are often inaccurate in real-world scenarios. One possible approach to reduce the uncertainties is to provide the robot with the capability of reasoning about the human's awareness of potential dangers. This paper discusses that by using a binary variable, so-called danger awareness coefficient, it is possible to differentiate between concerned and unconcerned humans, and provides a learning algorithm to determine this coefficient by observing human actions. Moreover, this paper argues how humans rely on predictions of other agents' future actions (including those of robots in human-robot interaction) in their decision-making. It also shows that ignoring this aspect in predicting human's future actions can significantly degrade the efficiency of the interaction, causing agents to deviate from their optimal paths. The proposed robot action planning scheme is verified and validated via extensive simulation and experimental studies on a LoCoBot WidowX-250.

Keywords: Mechanical systems/robotics, Autonomous robots, Robotics
Abstract: Recent advancements in applying machine learning to bipedal robots have demonstrated significant potential. However, the need for more comprehensive Artificial Intelligence (AI) testing environments has become increasingly evident, as current evaluations are constrained by limited datasets, making real-world testing essential. This paper presents Stilt-bot, an AI test bed specifically designed to support skill transfer across various robotic platforms, regardless of changes in size, shape, or actuator power. Inspired by how humans adapt their gait throughout growth, Stilt-bot employs a simple yet versatile 6 degrees-of-freedom (DOF) design and a prismatic sliding mechanism that enhance its agility and reduce weight. This configuration allows easy modifications to its height, mass, and power, providing a flexible and intuitive platform for evaluating AI-based walking control strategies. Both simulation and experimental results confirm Stilt-bot’s capability for stable flat-ground walking, demonstrating its effectiveness as a test bed for developing robust AI-driven bipedal locomotion.

Keywords: Nonholonomic systems, Lyapunov methods, Autonomous vehicles
Abstract: We present a safety-critical controller for the problem of stabilization for force-controlled nonholonomic mobile robots. The proposed control law is based on the constructions of control Lyapunov functions (CLFs) and control barrier functions (CBFs) for cascaded systems. To address nonholonomicity, we design the nominal controller that guarantees global asymptotic stability and local exponential stability for the closed-loop system in polar coordinates and construct a strict Lyapunov function valid on any compact sets. Furthermore, we present a procedure for constructing CBFs for cascaded systems, utilizing the CBF of the kinematic model through integrator backstepping. Quadratic programming is employed to combine CLFs and CBFs to integrate both stability and safety in the closed loop. The proposed control law is time-invariant, continuous along trajectories, and easy to implement. Our main results guarantee both safety and local asymptotic stability for the closed-loop system.

Keywords: Optimal control, Mechanical systems/robotics, Hybrid systems
Abstract: Energy efficiency is a critical factor in the performance and autonomy of quadrupedal robots. While previous research has focused on mechanical design and actuation improvements, the impact of gait parameters on energetics has been less explored. In this paper, we hypothesize that gait parameters, specifically duty factor, phase shift, and stride duration, are key determinants of energy consumption in quadrupedal locomotion. To test this hypothesis, we modeled the Unitree A1 quadrupedal robot and developed a locomotion controller capable of independently adjusting these gait parameters. Simulations of bounding gaits were conducted in Gazebo across a range of gait parameters at three different speeds: low, medium, and high. Experimental tests were also performed to validate the simulation results. The findings demonstrate that optimizing gait parameters can lead to significant reductions in energy consumption, enhancing the overall efficiency of quadrupedal locomotion. This work contributes to the advancement of energy-efficient control strategies for legged robots, offering insights directly applicable to commercially available platforms.

Keywords: Electrical machine control, Optimization, Estimation
Abstract: Servo systems, one of the backbones of modern manufacturing, are supposed to move as fast as possible for high productivity. Due to the inaccurate information on torque capacity, conventional trajectory generation methods are either overly conservative, compromising yield, or violate dynamical feasibility, compromising quality. This work proposes a method to address these shortcomings. Stable adaptive estimation of the servomotor model parameters is first performed, then torque capacity constraints are established as analytical functions of the motor speed based on parameter estimates, and finally, a computationally efficient algorithm is developed to reshape an aggressive (dynamically infeasible) trajectory into a feasible one. Theoretical analysis and numerical simulation validate the effectiveness of the proposed method.

Keywords: Optimization algorithms, Computational methods, Aerospace
Abstract: This paper investigates and compares the convergence performance of two widely used Nonlinear Programming (NLP) solvers—MATLAB’s texttt{fmincon} and IPOPT— for solving multi-impulse cislunar trajectory optimization problems. The problem is formulated as a minimum-fuel (or minimum-Delta v) trajectory optimization for a transfer between two Distant Retrograde Orbits (DROs) in the Circular Restricted Three-Body Problem (CR3BP). Derivatives of the objective and constraints, required by the solvers, are computed using three methods: a) solver’s built-in finite-difference (FD) method, b) the complex-step based (CX) method, and c) an analytical method. The results demonstrate that while analytical derivatives provide the fastest convergence, the CX method achieves nearly the same performance, despite being significantly easier to compute than the analytical derivatives. The CX method outperforms the FD method in terms of derivative accuracy and its impact on the convergence performance of the solvers (i.e., total number of iterations and function evaluations). Results indicate that IPOPT exhibits faster convergence compared to texttt{fmincon}, when CX and analytical derivatives are used.

Keywords: Switched systems, Estimation, Optimization algorithms
Abstract: We consider the problem of using an autonomous agent to persistently monitor a collection of targets distributed in a given environment. We generalize existing work by allowing the agent's dynamics to vary throughout the environment, leading to a hybrid dynamical system. This introduces an additional layer of complexity towards the planning portion of the problem: we must not only identify in which order to visit the points of interest, but also in which order to traverse the regions. We propose a tailored global path planner and prove that it is not only probabilistically complete, but converges in probability to a time-optimal solution. We then design an offline sequence planner together with an online trajectory optimizer. Simulations validate the results.

Keywords: Optimal control, Robotics, Aerospace
Abstract: In this paper, motion planning for a Dubins vehicle on a unit sphere to attain a desired final location is considered. The radius of the Dubins path on the sphere is lower bounded by r, where r represents the radius of the tightest left or right turn the vehicle can take on the sphere. Noting that r in (0, 1) and can affect the trajectory taken by the vehicle, it is desired to determine the candidate optimal paths for r ranging from nearly zero to close to one to attain a desired final location. In a previous study, this problem was addressed, wherein it was shown that the optimal path is of type CG, CC, or a degenerate path of the CG and CC paths, which includes C, G paths, for r leq frac{1}{2}. Here, C in {L, R} denotes an arc of a tight left or right turn of minimum turning radius r, and G denotes an arc of a great circle. In this paper, the candidate paths for the same problem are generalized to model vehicles with a larger turning radius. In particular, it is shown that the candidate optimal paths are of type CG, CC, or a degenerate path of the CG and CC paths for r leq frac{sqrt{3}}{2}. Noting that at most two LG paths and two RG paths can exist for a given final location, this paper further reduces the candidate optimal paths by showing that only one LG and one RG path can be optimal, yielding a total of seven candidate paths for r leq frac{sqrt{3}}{2}. Additional conditions for the optimality of CC paths are also derived in this study.

Keywords: Aerospace, Hierarchical control, Lyapunov methods
Abstract: Space robots, i.e., spacecraft with robotic manipulators, are essential for in-orbit servicing and debris removal. The high actuation redundancy of these systems allows for the imposition of multiple tasks with different priority levels. This work presents a singularity-free task-priority design for trajectory tracking in space robots, effectively removing the stringent assumption that the manipulator operates within the feasible workspace (free of kinematic singularities). The design, based on a modular hierarchical architecture, ensures end-effector tracking takes priority over tasks like maintaining a safe distance and a proper attitude of the spacecraft base. The proposed control law uses a 6D matrix representation of rigid motion that unifies attitude and position, offering invariance to inertial frame changes, making it ideal for space robotics.

Keywords: Constrained control, Flight control, Control applications
Abstract: This paper considers the problem of trajectory tracking for quadrotors operating in wind conditions that result in propeller thrust saturation. To address this problem, an antiwindup compensator (AWC) is developed to reduce the tracking performance degradation and destabilizing effects from thrust saturation. Relationships are derived showing how the tracking error and AWC states are influenced by the wind disturbance and saturation, and how the influences depend on the controller and AWC gains. As a result, these gains can be tuned to achieve desired performance levels. Simulation results are presented to validate the effectiveness of the proposed method.

Keywords: Lyapunov methods, Autonomous systems
Abstract: The work presented addresses the tracking control problem for a class of marine vehicles characterized by the type of propulsion system used and resulting impact on dynamical behavior. Specifically, we consider movement in the horizontal plane of vehicles that use a rudder for steering (or, equivalently, vectored thrust), as opposed to differential thrust, or lateral thrusters. The corresponding system dynamics are underactuated, featuring fewer independent control inputs than degrees of freedom. They also typically display a non-minimal phase nature (in the nonlinear sense), where desirable equilibria of the internal (non-actuated) dynamics prove unstable. We present solutions to this problem, which rely on careful design of the set of errors tracked by the external dynamics, guaranteeing uniform ultimate boundedness of tracking errors.

Keywords: Statistical learning, Adaptive control, Machine learning
Abstract: Neural networks are widely used to approximate unknown functions in control. A common neural network architecture uses a single hidden layer (i.e. a shallow network), in which the input parameters are fixed in advance and only the output parameters are trained. A common argument asserts that if output parameters exist to approximate the unknown function with sufficient accuracy, then desired control performance can be achieved. A long-standing theoretical gap was that no conditions existed to guarantee that, for the fixed input parameters, the required accuracy could be obtained by training the output parameters. Our recent work has partially closed this gap by demonstrating that if input parameters are chosen randomly, then for any sufficiently smooth function, with high-probability there are output parameters resulting in O((1/m)^1/2) approximation errors, where m is the number of neurons. However, some applications, notably continuous-time value function approximation, require that the network approximates the both the unknown function and its gradient with sufficient accuracy. In this paper, we show that randomly generated input parameters and trained output parameters result in gradient errors of O((log(m)/m)^1/2), and additionally, improve the constants from our prior work. We show how to apply the result to policy evaluation problems.

Keywords: Statistical learning, Algebraic/geometric methods, Uncertain systems
Abstract: This paper introduces Manifold-Guided Stabilizing Control (MGSC), a novel approach to synthesizing stabilizing controllers for nonlinear dynamical systems using diffusion models. Our method formulates control synthesis as a search for the closest asymptotically stable vector field within a learned manifold of stable dynamics. We train a diffusion model on a dataset of asymptotically stable vector fields and employ Tweedie’s estimate to iteratively adjust control parameters, ensuring convergence to a stabilizing controller. This formulation enables zero-shot stabilization for previously unseen systems with significantly reduced computational cost. Our numerical experiments demonstrate that MGSC achieves stabilization in just 16 seconds, compared to 2 minutes in prior work, while generalizing effectively across different nonlinear control problems. These results highlight the potential of diffusion models as a powerful tool for fast, data-driven control synthesis.

Keywords: Statistical learning, Identification for control, Estimation
Abstract: This paper revisits the set membership identification for linear control systems and establishes its convergence rates under relaxed assumptions on (i) the persistent excitation requirement and (ii) the system disturbances. In particular, instead of assuming persistent excitation exactly, this paper adopts the block-martingale small-ball condition enabled by randomly perturbed control policies to establish the convergence rates of SME with high probability. Further, we relax the assumptions on the shape of the bounded disturbance set and the boundary-visiting condition. Our convergence rates hold for disturbances bounded by general convex sets, which bridges the gap between the previous convergence analysis for general convex sets and the existing convergence rate analysis for L-infinity balls. Further, we validate our convergence rates by several numerical experiments.

Keywords: Statistical learning, Lyapunov methods, Stability of nonlinear systems
Abstract: Theoretical properties and data-driven learning of the Koopman operator, which represents nonlinear dynamics as a linear mapping on a properly defined functional spaces, have become key problems in nonlinear system identification and control. However, Koopman operators that are approximately learned from snapshot data may not always accurately predict the system evolution on long horizons. In this work, by defining the Koopman operator on a space of weighted continuous functions and learning it on a weighted reproducing kernel Hilbert space, the Koopman operator is guaranteed to be contractive and the accumulated error is bounded. The weighting function, assumed to be known a priori due to the knowledge on system stability, has an exponential decay with the flow. Under such a construction, the Koopman operator learned from data is used to estimate (i) Lyapunov functions for globally asymptotically stable dynamics, and (ii) Zubov-Lyapunov functions that characterize the domain of attraction. For these estimations, probabilistic error bounds are derived.

Keywords: Stochastic optimal control, Statistical learning, Stochastic systems
Abstract: This paper addresses the numerical solution of backward stochastic differential equations (BSDEs) arising in stochastic optimal control. Specifically, we investigate two BSDEs: one derived from the Hamilton-Jacobi-Bellman equation and the other from the stochastic maximum principle. For both formulations, we analyze and compare two numerical methods. The first utilizes the least-squares Monte-Carlo (LSMC) approach for approximating conditional expectations, while the second leverages a time-reversal (TR) of diffusion processes. Although both methods extend to nonlinear settings, our focus is on the linear-quadratic case, where analytical solutions provide a benchmark. Numerical results demonstrate the superior accuracy and efficiency of the TR approach across both BSDE representations, highlighting its potential for broader applications in stochastic control

Keywords: Markov processes, Estimation, Statistical learning
Abstract: Energy efficiency of buildings is considered to be one of the major means of achieving the net-zero carbon goal around the world. The big part of the energy savings are supposed to be coming from optimizing the operation of the building heating, ventilation, and air conditioning (HVAC) systems. There is a natural trade-off between the energy efficiency and the indoor comfort level, and finding an optimal operating schedule/regime requires knowing the occupancy of different spaces inside of the building. Moreover, the COVID-19 pandemic has also revealed the need to sustain the high quality of the indoor air in order to reduce the risk of spread of infection. Occupancy detection from indoor sensors is thus an important practical problem. In the present paper, we propose detection of occupancy based on the carbon dioxide measurements inside the building. In particular, a new approach based on the, so-called, switching auto-regressive process with Markov regime is presented and justified by the physical model of the carbon dioxide concentration dynamics. We demonstrate the efficiency of the method compared to simple Hidden Markov approaches on simulated and real-life data. We also show that the model is flexible and can be generalized to account for different ventilation regimes, simultaneously detecting the occupancy and the ventilation rate.

Keywords: Network analysis and control, Control of networks
Abstract: This paper considers the problem of designing non-pharmaceutical intervention (NPI) strategies, such as masking and social distancing, to slow the spread of a viral epidemic. We formulate the problem of jointly minimizing the infection probabilities of a population and the cost of NPIs based on a Susceptible-Infected-Susceptible (SIS) propagation model. To mitigate the complexity of the problem, we consider a steady-state approximation based on the quasi-stationary (endemic) distribution of the epidemic, and prove that the problem of selecting a minimum-cost strategy to satisfy a given bound on the quasi-stationary infection probabilities can be cast as a submodular optimization problem, which can be solved in polynomial time using the greedy algorithm. We carry out experiments to examine effects of implementing our NPI strategy on propagation and control of epidemics on a Watts-Strogatz small-world graph network. We find the NPI strategy reduces the steady state of infection probabilities of members of the population below a desired threshold value.

Keywords: Networked control systems, Biological systems, Time-varying systems
Abstract: Numerous complex systems, such as those arisen in ecological networks, genomic contact networks, and social networks, exhibit higher-order and time-varying characteristics, which can be effectively modeled using temporal hypergraphs. However, analyzing and controlling temporal hypergraphs poses significant challenges due to their inherent time-varying and nonlinear nature, while most existing methods predominantly target static hypergraphs. In this article, we generalize the notions of controllability and observability to temporal hypergraphs by leveraging tensor and nonlinear systems theory. Specifically, we establish tensor-based rank conditions to determine the weak controllability and observability of directed, weighted temporal hypergraphs. The proposed framework is further demonstrated with synthetic and real-world examples.

Keywords: Networked control systems, Control over communications, Filtering
Abstract: We consider a remote estimation setup, where data packets containing sensor observations are transmitted over a Gilbert-Elliot channel to a remote estimator, and design scheduling policies that minimize a risk-sensitive cost, which is equal to the expected value of the exponential of the cumulative cost incurred during a finite horizon, that is the sum of the cumulative transmission power consumed, and the cumulative squared estimation error. More specifically, consider a sensor that observes a discrete-time autoregressive Markov process, and at each time decides whether or not to transmit its observations to a remote estimator using an unreliable wireless communication channel after encoding these observations into data packets. Modeling the communication channel as a Gilbert-Elliot channel allows us to take into account the temporal correlations in its fading. We pose this dynamic optimization problem as a Markov decision process (MDP), and show that there exists an optimal policy that has a threshold structure, i.e., at each time t it transmits only when the current channel state is good, and the magnitude of the current ``error'' exceeds a certain threshold.

Keywords: Networked control systems, Decentralized control, Cooperative control
Abstract: Hierarchical cyclic pursuit has garnered significant attention among researchers in recent years. Along similar lines, this work aims to study ring digraphs, with a hierarchical "necklace" structure, involving antagonistic interactions. In the first part, identical antagonistic interactions, represented by negative edge weights, within macro vertices are considered. A study on the effect of such negative interactions on the consensus of agents is presented and bounds on negative gain, for consensus, are derived. Subsequently, heterogeneous antagonistic interactions are considered and the effect of these interactions on the consensus of the agents is analyzed. The set of points where consensus is achievable, when the edge weights are varied within the derived permissible bounds, is also investigated. Finally, numerical examples and corresponding simulations are presented to validate the analytical results.

Keywords: Networked control systems, Kalman filtering
Abstract: In this paper, we propose a novel approach for designing physical watermarks in cyber-physical systems (CPSs) with a focus on detection rate. While physical watermarks have been recognized as effective defenses against replay attacks, existing methods lack direct metrics to evaluate the detection performance. We address this gap by deriving a direct mapping between the watermark parameters and the detection rate. Our results show that, under specific conditions, physical watermarks can be effectively designed based on detection and false alarm rates. We provide a comprehensive set of conditions for this design, offering a rigorous and quantifiable framework for assessing the efficacy of watermark-based replay attack detection methods. Finally, extensive simulations are conducted to demonstrate the effectiveness of the proposed method.

Keywords: Computational methods
Abstract: Signal denoising is a fundamental task in signal processing that aims to extract the true underlying signal from noisy observations. Existing signal denoising methods, such as wavelet transform and Fourier-based filtering, suffer from low computational efficiency and potential loss of important signal components during reduction. Moreover, determining the optimal threshold for singular value selection remains a challenge in traditional techniques that are based on singular value decomposition. In this paper, we introduce an efficient, non-iterative algorithm for signal denoising that leverages two noisy observations. By constructing Hankel matrices from these observations, the proposed method establishes a threshold using the largest singular value of their difference, effectively separating true signal components from noise without the need for iterative optimization. We validate the approach on both synthetic data and real-world measurements, including smartphone sensor readings and displacement data from a micro-positioning system with a piezoelectric actuator.

Keywords: Mechanical systems/robotics
Abstract: This paper presents a higher-dimensional state-space model for predicting the nonlinear dynamics of a cable slab using Koopman operators. Currently, cable slab dynamics is a problem in precision motion systems, as the cable can impart unwanted vibrations or disturbances onto the motion stage. Currently, there is no definitive analytical model of the cable slab dynamics. The development process involved systematically evaluating various Koopman observable functions to minimize tracking errors. The model achieves an error margin of approximately ±1% for the given motion range and is robust enough to predict the movement of untrained acyclic randomized cable slab motions. Through a Koopman Operator approach the analytical formulation of the nonlinear cable slab dynamics is approximated. The model is verified through experimental results.

Keywords: Mechatronics, Stability of nonlinear systems, Control applications
Abstract: This paper presents a novel design for finite-time position control of quadrotor Unmanned Aerial Vehicles (UAVs). A robust, finite-time, nonlinear feedback controller is introduced to reject bounded disturbances in tracking tasks. The proposed control framework differs conceptually from conventional controllers that utilize Euler angle parameterization for attitude and adhere to the traditional hierarchical inner-outer loop design. In standard approaches, the translational controller and the corresponding desired attitude are computed first, followed by the design of the attitude controller based on time-scale separation between fast attitude and slow translational dynamics. In contrast, the proposed control scheme is quaternion-based and utilizes a transit feed-forward term in the attitude dynamics that anticipates the slower translational subsystem. Robustness is achieved through the use of continuously differentiable sliding manifolds. The proposed approach guarantees semi-global finite-time stability, without requiring time-scale separation. Finally, numerical simulation results are provided to demonstrate the effectiveness of the proposed controller.

Keywords: H-infinity control, LMIs, Mechatronics
Abstract: In multi-axis precision positioning systems, the plant dynamics are typically decoupled into single-input, single-output (SISO) channels to enable a decentralized controller structure. However, static decoupling often fails to provide sufficient decoupling performance in the mid-to-high frequency range, which limits the system's performance. Dynamic decoupling has been demonstrated as a promising tool for addressing cross-coupling effects while preserving the use of SISO feedback and feedforward controllers. However, existing model-based dynamic decoupling approaches cannot offer sufficient robustness with respect to model uncertainties. While data-driven methods have been developed, they often lack a stability guarantee and cannot be used for hardware-control co-optimization purposes. This paper proposes a novel model-based dynamic decoupling controller synthesis method for precision motion systems with model uncertainties explicitly considered through a robust H-infinity optimization, which can be transformed into Linear Matrix Inequalities (LMI) for solving. Simulation using a lightweight three-axis precision motion system is presented for evaluation.

Keywords: Mechatronics, Nonlinear output feedback, Observers for nonlinear systems
Abstract: The deformable mirror (DM) is a critical component in adaptive optic systems. However, controlling and modeling it is challenging due to its nonlinear, coupled, and position-dependent dynamics, which arise from the interaction of actuators and mechanical coupling across multiple axes. This paper presents a decoupled control strategy for the DM that addresses the issue of unknown system dynamics and position dependency. The proposed approach incorporates a proportional controller as an inner control loop and an output feedback controller as an outer control loop, combining state feedback with an extended high-gain observer (EHGO). The controller design does not require prior knowledge of the mirror’s dynamics and can be implemented independently for each actuator and axis. Simulations were conducted to assess the controller’s performance, demonstrating its effectiveness in tracking desired motions along both the X and Y axes, even in the presence of unknown dynamics.

Keywords: Mechatronics, Nonlinear output feedback, Observers for nonlinear systems
Abstract: Reluctance actuators (RAs) present a promising alternative to Lorentz actuators for next-generation positioning and scanning systems, such as wafer scanners and deformable mirrors. However, RAs face challenges such as nonlinear output force characteristics, negative stiffness due to gap dependency, and nonlinear magnetic hysteresis. This paper proposes a current control strategy based on an extended high-gain observer to linearize the dynamic behavior of an RA driving a motion system. The controller integrates a PI controller for tracking the desired current and utilizes the extended high-gain observer to estimate unknown uncertainties and nonlinearities in the actuator dynamics, thereby enhancing system robustness and linearizing the actuator’s behavior. The effectiveness of the proposed controller is demonstrated through simulation and experimental testing. Both simulation and experimental results confirm the controller’s ability to linearize the RA and mitigate the negative stiffness effects caused by position dependency. The experimental results demonstrate the system’s linear behavior under the proposed controller, and the controller also increases the system’s frequency bandwidth to f_{BW}=35 Hz compared to f_{BW}=9.5 Hz for the system without a controller.

Keywords: Biomedical, Biological systems, Control system architecture
Abstract: Regulation of the cardiovascular system is a critical component of patient care in the operating room and intensive care unit. One such goal is to regulate arterial blood pressure (ABP) to ensure end-organ perfusion. This regulation is managed by clinicians through manual titration of medications or fluids, an approach prone to human error. Physiological closed-loop control systems automate this titration and have significant potential to improve performance of cardiovascular system management. Many such control systems have been developed utilizing varied control strategies (e.g., rule-based methods, proportional-integral-derivative control, and model predictive control, etc.). These systems are often designed and evaluated without consideration of mechanistic cardiovascular states. In this work, we aim to motivate why mechanistic cardiovascular states must be considered in the design and evaluation of closed-loop systems for cardiovascular control. We use control of ABP to ensure end-organ perfusion as an illustrative example. We show theoretically and with recordings from a swine cardiovascular system model that similar mean ABP values often result from different cardiovascular states. We then present an example layered control architecture which explicitly incorporates mechanistic cardiovascular states in its decision-making scheme. At the high layer is selection of the control targets and actuators informed by mechanistic estimates of cardiovascular states, systemic resistance and cardiac output. At the low layer is a set of real-time controllers with actuators that regulate mean ABP either directly or through regulation of mechanistic cardiovascular states. We then present simulations of a representative high layer and its interaction with a representative low layer within our proposed control architecture. Overall, we assert that closed-loop systems for cardiovascular control should incorporate mechanistic cardiovascular states into their design and evaluation.

Keywords: Modeling, Biomedical, Hybrid systems
Abstract: This paper presents a data driven global linear model of steady state walking dynamics. Instantaneous whole body angular momentum is a physics informed aggregate quantity used as a marker for dynamic balance during locomotion. Gait dynamics are often modeled as hybrid and nonlinear. We propose using Koopman Operators to model the gait stability dynamics with a global, linear model. This is achieved by augmenting the whole body angular momentum state variables with learned observables, or basis functions, such that the dynamics look linear in the lifted dimension. Considering that the gait dynamics are periodic, a regularization term that encourages the state transition matrix to be orthonormal is added to the loss term when learning the observables. This forces a periodic model to be learned and prevents the likelihood of unstable poles. A low average MSE was obtained over 2 gait cycles for different population types, each with slightly differing gait dynamics. Furthermore, this linear representation enables the use of linear analysis tools that could have clinical implications for assessing the gait of different patient groups.

Keywords: Biomedical, Modeling, Human-in-the-loop control
Abstract: We introduce a novel dynamic modeling paradigm to represent fatigue and recovery processes in muscles by dynamic extension of the classical 3-element Hill model. A memristor and a capacitor are used in a series circuit arrangement, with a voltage source representing muscle activation. This model is shown to capture the fundamental features of fatigue accumulation and recovery in response to arbitrary motion, load and activation profiles, in contrast with other work assuming specific input shapes such as constants or periodic functions. Further, the basic model is shown to capture the gradual increase in activation spectral amplitudes with fatigue progression, which is an experimental fact. The paper shows how the fatigue modeling element is integrated into larger musculoskeletal dynamic models. Possible modifications and extensions aimed at more flexibility are also suggested.

Keywords: Emerging control applications, Predictive control for linear systems, Identification for control
Abstract: Control systems engineering has contributed to a paradigm shift in behavioral science and medicine. Among the applications of control systems engineering in behavioral medicine includes understanding, on an individual level, the dynamics of behavior change and leveraging this knowledge to deliver optimized, personalized interventions. These principles are the foundation of the control optimization trial (COT) framework that aims to facilitate the dissemination of data-driven, control-oriented behavioral interventions and consequently improve individual and public health. YourMove (ClinicalTrials.gov ID NCT05598996), a first-of-its-kind COT study, is an intervention to increase physical activity in sedentary adults and the culmination of years of research into the effectiveness of system identification and model predictive control (MPC) design in behavior change interventions. This paper summarizes the methods utilized in YourMove and provides promising preliminary results for illustrative participants in the ongoing study. The results presented are consistent with scenarios simulated in prior work and validate the COT framework as an effective tool for delivering personalized closed-loop interventions. In particular, results from this study demonstrate the performance and robustness of a three-degree-of-freedom Kalman filter-based hybrid model predictive control (3DoF-KF HMPC) algorithm in real-world settings.

Keywords: Machine learning, Fault detection, Genetic regulatory systems
Abstract: A mRNA-protein gene regulatory network is a differential equation model for gene expression that incorporates the dynamics of both mRNA and protein expressions. This paper addresses the challenges of using High-Dimensional Low-Sample-Size (HDLSS) data set for early disease detection and re-stabilization in mRNA-protein gene regulatory networks. For the first time, we demonstrate that detecting the pre-disease stage of mRNA-protein gene regulatory networks is possible using only the HDLSS data of either mRNA or protein. After detecting the pre-disease stage, it is crucial to prevent disease progression at that point. From a control engineering perspective, this prevention can be achieved by enhancing the system's stability, a process called as re-stabilization. We demonstrate that the key nodes for re-stabilization in the mRNA-protein gene regulatory network manifest as mRNA-protein duals. The intervention strategy, whether suppression or promotion, is identical for the mRNA and protein components of each key dual. Furthermore, we present a Lyapunov equation-based system identification method to estimate the system matrix using the HDLSS data set. From the estimated system matrix, the key nodes for re-stabilization can be identified. The effectiveness of the proposed method has been validated via numerical examples.

Keywords: Biomedical, Predictive control for linear systems, PID control
Abstract: In this paper we propose a technique to enhance the performance of a Proportional-Integral-Derivative (PID)-based control structure for Depth-of-Hypnosis control in total intravenous anesthesia when set-point changes are required during the maintenance phase. In particular, the PID controller, tuned for disturbance rejection, is integrated with a feedforward action based on Model Predictive Control (MPC). A tuning parameter determines the aggressiveness of the controller, thus allowing the anesthesiologist to select the most appropriate transient response depending on the kind of patient and of surgery. Simulation results show that the method is useful in providing an effective tool for the anesthesiologist to interact with the control system.

Keywords: Adaptive control, Robust adaptive control
Abstract: A multi-variable adaptive controller is derived as the explicit solution to a minimax dynamic game. The minimizing player selects the control action as a function of past state measurements and inputs. The maximizing player selects disturbances and model parameters for the underlying linear time-invariant dynamics. This leads to a Bellman equation that can be solved explicitly for the case with unitary B-matrix known up to a sign and no input penalty. The minimizing policy is a dual controller that optimizes the tradeoff between exploration and exploitation.

Keywords: Algebraic/geometric methods, Observers for nonlinear systems, Kalman filtering
Abstract: Linear observed systems on manifolds are a special class of nonlinear systems whose state spaces are smooth manifolds but possess properties similar to linear systems. Such properties can be characterized by preintegration and exact linearization with Jacobians independent of the linearization point. Non-biased IMU dynamics in navigation can be constructed into linear observed settings, leading to invariant filters with guaranteed behaviors such as local convergence and consistency. In this letter, we establish linear observed property for systems evolving on a smooth manifold through the connection structure endowed upon this space. Our key findings are the existence of linear observed systems on manifolds poses constraints on the curvature of the state space, beyond requiring the dynamics to be compatible with some connection-preserving transformations. Specifically, the flat connection case reproduces the characterization of linear observed systems on Lie groups.

Keywords: Algebraic/geometric methods
Abstract: In this work, we provide a global condition for contraction with respect to an invariant Riemannian metric on reductive homogeneous spaces. Using left-invariant frames, vector fields on the manifold are horizontally lifted to the ambient Lie group, where the Levi-Civita connection is globally characterized as a real matrix multiplication. By linearizing in these left-invariant frames, we characterize contraction using matrix measures on real square matrices, avoiding the use of local charts. Applying this global condition, we provide a necessary condition for a prescribed subset of the manifold to possibly admit a contracting system, which accounts for the underlying geometry of the invariant metric. Applied to the sphere, this condition implies that no great circle can be contained in a contraction region. Finally, we apply our results to compute reachable sets for an attitude control problem.

Keywords: Optimal control, Reinforcement learning, Nonlinear output feedback
Abstract: This paper develops a data-based approach to the closed-loop control of nonlinear dynamical systems with a partial nonlinear observation model. We propose an ``information-state" based approach to rigorously transform the partially observed problem into a fully observed problem where the information-state consists of the past several observations and control inputs. We further show the equivalence of the transformed and the initial partially observed optimal control problems and provide the conditions to solve for the deterministic optimal solution. We develop a data-based generalization of the iterative Linear Quadratic Regulator (ILQR) for partially-observed systems using a local linear time-varying model of the information-state dynamics approximated by an Autoregressive–moving-average (ARMA) model that is generated using only the input-output data. This open-loop trajectory optimization solution is then used to design a local feedback control law, and the composite law then provides an optimum solution to the partially observed feedback design problem. The efficacy of the developed method is shown by controlling complex high dimensional nonlinear dynamical systems in the presence of model and sensing uncertainty.

Keywords: Identification for control, Nonlinear systems identification, Optimal control
Abstract: In this paper, we study the use of state-of-the-art nonlinear system identification techniques for the optimal control of nonlinear systems. We show that the nonlinear system identification problem is equivalent to estimating the generalized moments of an underlying sampling distribution and is bound to suffer from ill-conditioning and variance when approximating a system to high order, requiring samples combinatorial-exponential in the order of the approximation. We show that the iterative identification of ``local" linear time varying (LTV) models around the current estimate of the optimal trajectory, coupled with a suitable optimal control algorithm such as iterative LQR (ILQR), alleviates these issues and is sufficient to accurately solve the underlying optimal control problem.

Keywords: Human-in-the-loop control, Automotive control, Automotive systems
Abstract: The introduction of drive-by-wire systems and advances in vehicle automation have enabled new possibilities of what a vehicle can do. Research has pushed vehicle automation to a level that only skilled drivers can achieve in racing and drifting, and decoupled systems can act as a guardian, protecting the driver in extreme situations. Introducing these systems has the potential to improve safety of the car and driver combined system, but could also lead to deskilling and over-reliance by the driver. This manuscript presents an approach to address this through leveraging task decomposition to teach advanced driving skills, namely drifting a circular donut. Using Model Predictive Control, task decomposition is achieved by allowing drivers to learn steering and throttle control independently, enabling the driver to isolate learning individual skills to enhance skill acquisition. A user study was conducted with 11 participants to experimentally validate the approach. Results reflect that the group subject to task decomposition demonstrated enhanced mastery of the skill.

Keywords: Differential-algebraic systems, Estimation, Control applications
Abstract: An observer is developed for the nonlinear descriptor system dynamics of parallel connected lithium-ion battery packs. The observer estimates the states of the individual cells in the pack with stability and existence guarantees provided through linear matrix inequalities. When evaluated on the urban dynamometer driving schedule, the proposed observer performed well, with root mean squared errors (RMSEs) of 0.0072 and 0.0054 in the state-of-charges as well as 0.3A and 0.28A in the currents for cells 1 and 2, respectively. The results also demonstrated the value of incorporating contact resistances in the model. In particular, with the inclusion of a contact resistance of 0.02 Ohm, the mismatch in currents grew by 9.69% for cell 1 and by 8.55% for cell 2. These results highlight the potential of implementing cell-level state estimation and control in large battery packs accounting for cell-to-cell variability and contact resistances.

Keywords: Energy systems, Modeling, Identification
Abstract: This contribution presents a parameter identification methodology for the accurate and fast estimation of model parameters in a pseudo-two-dimensional (P2D) battery model. The methodology consists of three key elements. First, the data for identification is inspected and specific features herein that need to be captured are included in the model. Second, the P2D model is analyzed to assess the identifiability of the physical model parameters and propose alternative parameterizations that alleviate possible issues. Finally, diverse operating conditions are considered that excite distinct battery dynamics which allows the use of different low-order battery models accordingly. Results show that, under low current conditions, the use of low-order models achieve parameter estimates at least 500 times faster than using the P2D model at the expense of twice the error. However, if accuracy is a must, these estimated parameters can be used to initialize the P2D model and perform the identification in half of the time.

Keywords: Fault detection, Energy systems, Estimation
Abstract: Faults in lithium-ion batteries can diminish performance and shorten lifespan. One notable electrochemical fault arises from the loss of lithium concentration in the electrolyte, which can lead to degradation over time. This paper studies the detection and estimation of electrolyte lithium concentration degradation fault in battery cells. The proposed model-based fault detection scheme consists of two cascaded closed-loop Partial Differential Equation (PDE) observers. The first observer, state observer, estimates the distributed lithium concentration in the electrolyte. The second observer, fault estimator, utilizes the concentration estimated by the first observer to detect and estimate the electrolyte lithium concentration fault. The scheme is validated through simulations conducted on a LiFePO4 battery cell under an Urban Dynamometer Driving Schedule (UDDS) current profile.

Keywords: Estimation, Adaptive systems, Energy systems
Abstract: All-Solid-State Batteries (ASSBs) offer enhanced safety and higher energy density compared to conventional Lithium-ion Batteries (LiBs), but their thermal management is challenging due to time-varying thermal properties. The thermal behavior of ASSBs is modeled by five Ordinary Differential Equations (ODEs) representing the temperatures of the case surface (near the cathode and anode), cathode, electrolyte, and anode. These temperatures are driven by heat from the battery, derived from an electrochemical model using two Partial Differential Equations (PDEs) for Li+ ions concentration. This study presents an adaptive observer that adjusts thermal conductivities in real-time, accurately estimating ASSB temperatures. Simulations demonstrate that the observer effectively tracks time-varying conductivities, with estimation errors converging to zero and improving thermal management accuracy.

Keywords: Fault diagnosis, Energy systems, Estimation
Abstract: Safe and reliable operation of lithium-ion battery packs depends on effective fault diagnosis. However, model-based approaches often encounter two major challenges: high computational complexity and extensive sensor requirements. To address these bottlenecks, this paper introduces a novel approach that harnesses the structural properties of battery packs, including cell uniformity and the sparsity of fault occurrences. We integrate this approach into a Moving Horizon Estimation (MHE) framework and estimate fault signals such as internal and external short circuits and faults in voltage and current sensors. To mitigate computational demands, we propose a hierarchical solution to the MHE problem. The proposed solution breaks up the pack-level MHE problem into smaller problems and solves them efficiently. Finally, we perform extensive simulations across various battery pack configurations and fault types to demonstrate the effectiveness of the proposed approach. The results highlight that the proposed approach simultaneously reduces the computational demands and sensor requirements of fault diagnosis.

Keywords: Energy systems, Estimation, Identification
Abstract: Accurate estimation of state of health (SOH) is critical for battery applications. Current model-based SOH estimation methods typically rely on low C-rate constant current tests to extract health parameters like solid phase volume fraction and lithium-ion stoichiometry, which are often impractical in real-world scenarios due to time and operational constraints. Additionally, these methods are susceptible to modeling uncertainties that can significantly degrade the estimation accuracy, especially when jointly estimating multiple parameters. In this paper, we present a novel reference voltage-based method for robust battery SOH estimation. This method utilizes the voltage response of a battery under a predefined current excitation at the beginning of life (BOL) as a reference to compensate for modeling uncertainty. As the battery degrades, the same excitation is applied to generate the voltage response, which is compared with the BOL trajectory to estimate the key health parameters accurately. The current excitation is optimally designed using the Particle Swarm Optimization algorithm to maximize the information content of the target parameters. Simulation results demonstrate that our proposed method significantly improves parameter estimation accuracy under different degradation levels, compared to conventional methods relying only on direct voltage measurements. Furthermore, our method jointly estimates four key SOH parameters in only 10 minutes, making it practical for real-world battery health diagnostics, e.g., fast testing to enable battery repurposing.

Keywords: Energy systems, Optimal control, Control applications
Abstract: Tab cooling offers more uniform temperature distribution across the battery because of high thermal conductivity, while surface cooling is more effective at removing heat due to a larger cooling contact area. This work formulates the optimal integration of tab and surface cooling methods to synergise their unique strengths. This optimal integration control problem is solved within the model predictive control (MPC) framework, leading to minimised thermal gradients and average temperature rise. The proposed MPC scheme, which includes an advanced thermal model containing the cooling system's converter switching mechanism, is evaluated against conventional side-only and base-only %electric vehicle battery cooling schemes under the urban dynamometer driving schedule. Results demonstrate significant thermal performance improvements of the proposed MPC scheme over the conventional cooling methods.

Keywords: Power systems, Predictive control for linear systems, Optimal control
Abstract: Self-powered control systems use harvested dis- turbance energy to actuate the controller. These systems are of interest, for example, in vibration suppression applications where the control force cannot rely on outside energy. The control input is constrained to prevent the system from ex- hausting the stored energy, and by the energy storage capacity. In this paper we use Model Predictive Control (MPC) to minimize an integral-quadratic performance objective while meeting the enforced energy constraints. The limitation on the maximum stored energy results in a non-convex control optimization problem. We apply convex-concave techniques to the non-convex constraint to conservatively bound the stored energy. It follows that convex-concave MPC can be used to find the optimal solution to these self-powered control problems. This technique is demonstrated in a simple example vibration suppression problem.

Keywords: Optimal control, Constrained control, Power electronics
Abstract: In self-powered control technologies the sensors, actuators, and control algorithm are entirely powered by harvesting energy from system disturbances. Because they are energy autonomous, self-powered control systems are ideal for disturbance rejection applications where access to external power is limited, and have shown great promise in many civil, mechanical, and biomedical engineering applications. This work expands on previous efforts in the field regarding the synthesis of feedback laws for self-powered control systems. Using a passive linear time-invariant (LTI) feedback law as a basis for design, a multi-objective optimal control design method for self-powered feedback control is presented. First, frequency domain methods are employed to find an LTI self-powered control law; then, a nonlinear self-powered feedback law is designed with performance which is guaranteed to improve on that of the LTI law. The proposed methodology is demonstrated in a design example involving a tuned vibration absorber attached to a two degree-of-freedom structure.

Keywords: Energy systems, Hybrid systems
Abstract: Blue economy industries like aquaculture are expanding further offshore to leverage the ocean's vast scale. However, this shift demands reliable, regular power independent of land-based grids. This study introduces a bi-level optimization framework for the design and operation of a hybrid OTEC-diesel system equipped with battery energy storage to power offshore fish farms. The upper-level optimization aims to minimize the levelized cost of energy (LCOE) and ensure continuous operation by optimizing the battery size within the constraints of energy storage. The objective of the lower-level optimization is to minimize energy waste while also addressing an environmental goal, which is to decrease the unnecessary mixing of cold and warm water in the Rankine cycle of the ocean thermal energy conversion (OTEC) system. In our study, we investigated two system configurations: first, a traditional setup using only a diesel generator; second, an OTEC/Diesel/BESS hybrid configuration was evaluated under two scenarios for different fish farm sizes. The results show that scaling up the fish farm and hybrid OTEC-diesel system with energy storage significantly lowers the LCOE. Furthermore, regulating the working fluid’s mass flow rate cuts energy waste by nearly 20% compared to single-level optimization, which reduces environmental impact, and positions this hybrid system as a sustainable and cost-effective alternative to diesel-powered fish farms.

Keywords: Hierarchical control, Predictive control for linear systems, Energy systems
Abstract: Electrifying industrial processes powered by renewable energy (RE) presents unique control challenges due to the multi-timescale dynamics and signals inherent in these systems. The coupling of the fast-timescale dynamics of the RE subsystems and the slow-timescale dynamics of industrial end-use systems creates a complex control problem. Further, variability in RE resources in both fast and slow timescales introduces an additional challenge for the storage dispatch that the controller must also consider. This paper presents a multi-timescale model predictive controller (MT-MPC) designed for an off-grid RE-powered electrified ammonia plant. The MT-MPC effectively manages storage dispatch and provides dynamic control of the end-use subsystem at slow timescales while also managing control of the fast-timescale RE subsystem with the time-varying RE resource input. The MT-MPC outperforms two single-timescale MPCs designed for only the fast or slow timescales, showing that the electrified ammonia plant benefits from multi-timescale control and that the MT-MPC is a promising controller to address that need.

Keywords: Energy systems, Power systems, Optimization
Abstract: The increasing integration of distributed energy resources (DERs) and the rising frequency of extreme weather events present significant challenges to the resilience of distribution systems. Virtual Power Plants (VPPs) offer a promising solution by providing the grid with flexible demand or generation support. This flexibility can be quantified by Dynamic Operating Envelopes (DOEs), which define the upper and lower power bounds of distributed assets. This paper investigates the impact of heat and cold waves on DOEs in distribution systems that incorporate VPPs. VPPs are modeled using high-fidelity, weather-dependent simulations of load and DERs to capture how extreme temperatures affect both demand and generation. Realistic weather data is used to explore how these conditions change the system conditions and then propagate to DOE results. This paper employs a modified IEEE 123-bus system to simulate heat and cold waves in King County, Washington State. Results indicate that extreme weather events decrease the DOEs and limit system flexibility. However, VPP integration enhances operational flexibility and improves resilience, although the benefits are not uniformly distributed across all nodes in the system.

Keywords: Networked control systems, Game theory, Distributed control
Abstract: This paper studies the distributed generalized Nash equilibrium seeking problem for aggregative games with coupling constraints, where each player optimizes its strategy depending on its local cost function and the estimated strategy aggregation. The information transmission in distributed networks may go beyond bandwidth capacity and eventuate communication bottlenecks. Therefore, we propose a novel communication-efficient distributed generalized Nash equilibrium seeking algorithm, in which the communication efficiency is improved by event-triggered communication and information compression methods. The proposed algorithm saves the transmitted rounds and bits of communication simultaneously. Specifically, by developing precise step size conditions, the proposed algorithm ensures provable convergence, and is proven to achieve (0,delta)-differential privacy with a stochastic quantization scheme. In the end, simulation results verify the effectiveness of the proposed algorithm.

Keywords: Reinforcement learning, Game theory, Markov processes
Abstract: We consider a multi-agent system under the control of a central planner (CP). We model the system as a hierarchical multi-agent reinforcement learning (MARL) problem in which a state process is influenced by both the CP's control and the agents' actions. Each agent learns a local policy to maximize the local value function, which may or may not be aligned with the CP's control objective. The CP's goal is therefore to find a global policy such that the agents learn an equilibrium joint policy that maximizes the CP's value function. We first show that this problem is equivalent to a Stackelberg game. This equivalence can be used to derive game-theoretic properties about the desired Stackelberg equilibrium and can be used to solve for optimal policies directly if a model is known. If no game model is given, then we propose a Monte Carlo (MC)-based reinforcement learning (RL) method based on the hierarchical game structure. We demonstrate that under standard RL assumptions, this method can approximate solutions to the desired Stackelberg game. This procedure is validated in simulations on synthetic games resembling social welfare problems.

Keywords: Distributed control, Reinforcement learning, Game theory
Abstract: Learning in games provides a powerful framework to design control policies for self-interested agents that may be coupled through their dynamics, costs, or constraints. We consider the case where the dynamics of the coupled system can be modeled as a Markov potential game. In this case, distributed learning ensures agents' control policies converge to a Nash equilibrium. However, standard algorithms like natural policy gradient require global state and action knowledge, which does not scale well with more agents. We show that by limiting information flow to local neighborhoods, we can still converge to near-optimal policies. If a game’s global cost function can be decomposed into local costs that align with agent policies at equilibrium, this approach benefits team coordination. We demonstrate this with a sensor coverage problem.

Keywords: Mean field games, Game theory, Control applications
Abstract: We model a Stackelberg game in a power market with rational consumers and a benevolent regulator as a mean-field game. The Stackelberg leader, who is a government regulator, sets the grid distribution fees so as to maximize the total welfare of the consumers, while also ensuring the solvency of the electricity producers and while satisfying renewable production targets. The Stackelberg followers, who are rational prosumers of electricity, maximize their personal utilities by choosing their individual Photovoltaics investments that provides an alternative to buying electricity from the grid, and hence can also produce electricity. With the representative prosumer’s demand evolving as an Ornstein-Uhlenbeck process, we find a closed form mean-field game approximation to prosumer’s optimal strategy, and use that to calculate the optimal fees set by the regulator. Using these we numerically investigate and explain the influence of various market conditions on the optimal distribution fees.

Keywords: Modeling, Game theory
Abstract: We explore the evolution of trust and trustworthiness in bounded rational agents via the Logit dynamics. We focus on the N-player trust game, where multiple investors and trustees interact and make decisions based on their level of trust and trustworthiness, respectively. We characterize the equilibria of the dynamics under different parameter regimes and establish their stability properties. We show how parameters associated with rationality, incentives, and synergy influence the emergence of trustworthy behavior. Additionally, we show when investors’ contributions are synergistic, then the dynamics may exhibit a limit cycle emerging from a Hopf bifurcation. We illustrate our theoretical results and provide additional insights into these dynamics through numerical simulations.

Keywords: Evolutionary computing, Network analysis and control, Game theory
Abstract: Cooperation plays a fundamental role is societal and biological evolution, and the population structure can significantly influence the dynamics of cooperative behaviors. While mathematical frameworks have existed for studying evolutionary games on graphs with pairwise interactions, exploration of strategy evolution on hypergraphs is relatively limited despite the widespread occurrence of multi-group (higher-order) interactions. Here, we propose the higher-order strategy update pattern based on two-stage selection of a imitating object, governing the propagation of cooperation on hypergraphs. Aiming at two specific update mechanisms, we establish mathematically rigorous conditions for cooperation in public goods games on any weighted hypergraph, achieved by adapting the higher-order random walk into the coalescence theory. These analytical conditions are in good agreement with the experimental consequences of Monte Carlo simulations, regarding both the random hypergraphs and the empirical higher-order networks. Furthermore, we surprisingly identify that one of the higher-order rule of strategy updates – selecting a high-performing group and then selecting a random member in it for imitation – can profoundly promotes the emergence of cooperation. These findings underscore a crucial role of higher-order interactions in modeling and controlling social and biological systems, favoring collective cooperation.

Keywords: Autonomous systems, Cooperative control, Aerospace
Abstract: The Center for Autonomous Air Mobility and Sensing (CAAMS) is a new NSF Industry University Collaborative Research Center (IUCRC) created by the University of Colorado Boulder, Brigham Young University, Penn State University, Sinclair Community College, Texas A&M University, the University of Michigan, and Virginia Tech to address the unmet, precompetitive research needs of the aviation industry as it moves toward the design and deployment of increasingly autonomous systems. Together with industry and government partners, CAAMS researchers are developing solutions that address the critical technical challenges relevant to autonomous air mobility and sensing. This tutorial will describe the vision, goals, and structure of CAAMS. The tutorial will identify critical research thrust areas where needed advances will help the aviation industry close significant gaps in performance, safety, reliability, and sustainability. Finally, the tutorial will share results from current research and development efforts that are closing these gaps.

Keywords: Decentralized control, Agents-based systems, Game theory
Abstract: In this paper, we propose a multilateral negotiation protocol for multi-agent task allocation problems. The protocol extends the Monotonic Concession Protocol (MCP), which was originally limited to two-player settings. For more than two players settings, we generalize several key concepts of MCP, including concession, agreement, and risk. Specifically, by introducing a notion of generalized risk, agents can systematically rank among various task allocations. The generalized risk metric is then employed to form a Nash strategy, leading the negotiation towards a task partition that maximizes the Nash product of the agents' utilities. The proposed mechanism ensures that the negotiation process is fully distributed, with agents acting in a completely multilateral manner, meaning their actions, roles, state, and information are symmetric. We demonstrate the performance and features of the proposed protocol through a three-player task negotiation example, which shows that the negotiation monotonically converges towards a task allocation that maximizes the Nash product as the negotiation round progresses.

Keywords: Decentralized control, Distributed control
Abstract: The bittide mechanism enables logically synchronous computation across distributed systems by leveraging the continuous frame transmission inherent to wired networks such as Ethernet. Instead of relying on a global clock, bittide uses a decentralized control system to adjust local clock frequencies, ensuring all nodes operate with a consistent notion of time by utilizing elastic buffers at each node to absorb frequency variations. This paper presents an analysis of the steady-state occupancy of these elastic buffers, a critical factor influencing system latency. Using a fluid model of the bittide system, we prove that buffer occupancy converges and derive an explicit formula for the steady-state value in terms of system parameters, including network topology, physical latencies, and controller gains. This analysis provides valuable insights for optimizing buffer sizes and minimizing latency in bittide-based distributed systems.

Keywords: Optimal control, Distributed control, Numerical algorithms
Abstract: Mean-field control (MFC) problems aim to find the optimal policy to control massive populations of interacting agents. These problems are crucial in areas such as economics, physics, and biology. We consider the non-local setting, where the interactions between agents are governed by a suitable kernel. For N agents, the interaction cost has O(N^2) complexity, which can be prohibitively slow to evaluate and differentiate when N is large. To this end, we propose an efficient primal-dual algorithm that utilizes basis expansions of the kernels. The basis expansions reduce the cost of computing the interactions, while the primal-dual methodology decouples the agents at the expense of solving for a moderate number of dual variables. We also demonstrate that our approach can further be structured in a multi-resolution manner, where we estimate optimal dual variables using a moderate N and solve decoupled trajectory optimization problems for large N. We illustrate the effectiveness of our method on an optimal control of 5000 interacting quadrotors.

Keywords: Agents-based systems, Optimization algorithms, Distributed control
Abstract: In this paper, we consider a distributed Riemannian optimization problem with personalization, where cost functions are defined on manifolds and rely on a decision variable comprising both global (shared) and local (private) components. This problem structure arises in various contexts where the global part encapsulates common characteristics across nodes, while the local part represents personalized features specific to individual nodes within the network. To solve this problem, we propose a personalized distributed Riemannian gradient tracking method where each node updates its private variables locally while performing distributed gradient tracking on the shared component with neighboring nodes to facilitate coordination, leading to greater algorithm flexibility and robustness against data heterogeneity. Leveraging the Riemannian consensus mechanism and the law of cosines for non-Euclidean geometry, we prove that the proposed algorithm achieves a suboptimality gap of O(1/sqrt{K}) for geodesically convex objective functions. Numerical examples are conducted to verify the effectiveness of the proposed algorithm within the realm of Riemannian optimization.

Keywords: Automata, Discrete event systems, Fault detection
Abstract: This paper extends the results on repeated fault diagnosability of monolithic discrete-event systems to systems with decentralized structure. To this end, the definition of K-codiagnosability, which basically consists of ensuring that at least one local diagnoser is able to accurately determine whether a fault event has occurred at least K times is introduced. Moreover, a necessary and sufficient condition for K-codiagnosability of regular languages is presented and an algorithm with polynomial-time complexity is proposed for its verification. A numerical examples illustrates the efficiency of the proposed method.

Keywords: Distributed parameter systems, Uncertain systems, Predictive control for nonlinear systems
Abstract: This article describes a systematic method for propagating and controlling the effects of parametric uncertainties in population balance models (PBMs) using polynomial chaos expansions. PBMs are an important element in the model-based control of many particulate manufacturing processes. Direct stochastic Galerkin projections of PBMs with uncertain parameters allow for structure-preserving uncertainty quantification of the population distribution across the full intrinsic variable domain. With the proposed method, more accurate real-time risk analysis to ensure safe and controlled particulate manufacturing under distributed parameter system model constraints becomes accessible on compute-limited hardware. The method in particular shows a considerable computational speedup and memory usage decrease relative to conventional sampling-based methods, enabling fast on-line model predictive control implementations. The method also retains full population distribution information in contrast to less-accurate, structure-reducing techniques such as the method of moments. Throughout, we discuss approaches to mitigate common numerical accuracy losses appearing in the discretization of PBMs.

Keywords: Optimal control, Automotive control, Automotive systems
Abstract: Model predictive control has emerged as an effective approach for real-time optimal control of connected and automated vehicles. However, the nonlinear dynamics of vehicle and traffic systems make accurate modeling and real-time optimization challenging. Learning-based control offers a promising alternative through offline control policy learning and low computational cost real-time inference. In vehicle and traffic systems, designing an effective learning control framework is nontrivial, as optimal control depends on both ego vehicle's state and predicted states of surrounding vehicles. This work proposes a traffic adaptive learning control method that allows the control strategy to intelligently adapt to varying traffic conditions. By leveraging an augmented state-space formulation, the predicted states of surrounding vehicles are transcribed to only alter initial conditions. This approach ensures that the system dynamics remain independent of surrounding vehicle trajectories to facilitate learning of a time-invariant system. Additionally, a physics-informed learning control framework is introduced that integrates the value function and its derivatives into a unified loss function. By incorporating system dynamics information into the learning process, the proposed method reduces the required training data and time while enhancing robustness and efficiency. The proposed control method is applied to car-following scenarios in real-world data calibrated simulation environments. The results show that this learning control approach alleviates real-time computational demands while achieving car-following behaviors comparable to model-based methods, leading to a 9% energy savings.

Keywords: Multivehicle systems, Predictive control for nonlinear systems
Abstract: Large-spacing truck platooning strikes a balance between maintaining the benefits of platooning—such as the fuel efficiency and emissions reduction—while addressing safety, flexibility, and operational challenges of close-spacing systems. To improve the performance of large-spacing truck platooning under windy conditions, a nonlinear model predictive controller (NMPC) is developed. This controller ensures platooning safety while optimizing fuel efficiency and reducing tailpipe nitrogen oxides (NOx) emissions. Simulations of a two-truck platoon with a 3-sec time gap under windy conditions were conducted, using models validated with on-road experimental data. The results demonstrate that the proposed NMPC effectively keeps spacing errors within the preset safety buffer. Moreover, the follower truck achieves a fuel saving of 5.4% in the Alberta Highway 2 driving cycle with headwinds, and tailpipe NOx emissions are reduced by up to 56.8% by suppressing rapid engine torque changes. This study offers valuable insights into the feasibility of deploying large-spacing platoons in windy environments.

Keywords: Automotive control, Constrained control, Human-in-the-loop control
Abstract: This manuscript presents a control barrier function based approach to shared control for preventing a vehicle from entering the part of the state space where it is unrecoverable. The maximal phase recoverable ellipse is presented as a safe set in the sideslip angle-yaw rate phase plane where the vehicle's state can be maintained. An exponential control barrier function is then defined on the maximal phase recoverable ellipse to promote safety. Simulations demonstrate that this approach enables safe drifting,that is, driving at the handling limit without spinning out. Results are then validated for shared control drifting with an experimental vehicle in a closed course. The results show the ability of this shared control formulation to maintain the vehicle's state within a safe domain in a computationally efficient manner, even in extreme drifting maneuvers.

Keywords: Multivehicle systems, Formal verification/synthesis, Automotive control
Abstract: We consider the coordination of a fleet of tractor trucks to manage trailers in a large warehouse complex and propose an approach that leverages Metric Temporal Logic (MTL) to describe missions to be executed. Each mission includes multiple tasks, such as reaching a trailer, connecting to it, moving it to a sequence of specific warehouse regions, such as loading docks, internal holding areas, and departure parking lots, and eventually disconnecting from it. The electric-powered tractor trucks must also be recharged by visiting charging stations. The MTL formulation avoids an operator manually designing a mission specification, which can quickly become unfeasible with many requests and possible assignments of tractor trucks. MTL specifications and motion dynamics are formulated as a mixed integer linear programming (MILP) approach, where the cost function includes performance objectives such as minimizing the trailer motions and energy-efficient usage. Since missions are added and removed during operation and to also reduce the computation time, we modify the method to allow for a receding horizon approach that allows for partial satisfaction of the MTL specification and uses the cost function to favor the progress towards completion of partially satisfied specifications. We compare different MILP formulations in simulations.

Keywords: Automotive control, Robust control, Machine learning
Abstract: Designing controllers for obtaining optimal performance from a multi-fuel compression ignition engine is a challenging problem. Multiple actuators are used to achieve desired combustion phasing across various operating conditions. Traditionally, feedforward maps are used, which are built by performing experiments at different operating conditions in steady-state. In recent years, the use of data-driven models to construct FF maps has gained traction. However, performance using FF maps while moving between steady-state points and in the presence of disturbance is not guaranteed. This paper explores the design of feedback controllers for reducing the transients observed due to uncertain actuator dynamics while moving between steady-state points and in the presence of disturbance. Robust control framework was used for controller design, and simulations representing scenarios that can be seen during the engine operation were used to demonstrate the effectiveness of the developed controllers.

Keywords: Nonholonomic systems, Modeling, Mechanical systems/robotics
Abstract: Trajectory tracking with an autonomous unicycle is considered in three-dimensional space. It is shown that with the appropriate choice of pseudo-velocities the lateral and longitudinal dynamics and control can be decoupled at the linear level. Linear state feedback controllers are designed separately for lateral and longitudinal subsystems and these controllers are tested simultaneously for the nonlinear model via numerical simulations.

Keywords: Optimization, Machine learning, Aerospace
Abstract: Rapid and accurate measurement of the temperature field is crucial in temperature control systems. However, conventional CCD-based thermometry treats the choice between dual-channel and tri-channel information as an either-or decision. To combine the advantages of both methods, this paper proposes an ensemble learning-based two-color and three-color joint thermometry method. The entire process, from spectral radiation reception to pixel intensity generation, was analyzed to understand its impact on selecting the optimal thermometry formula. By integrating channel reliability, noise parameters, and coefficients of variation in both spatial and colorimetric domains, a random forest model was used to predict the optimal thermometry formula. Experimental results demonstrate that the proposed method effectively selects the optimal thermometry method for each thermographic image.

Keywords: Optimization algorithms, Optimization, Agents-based systems
Abstract: Recently, Nesterov proposed an implementable tensor method for unconstrained convex optimization problems, and showed that the p-th order algorithm converges at a rate of O( epsilon^{-1/p+1} ) , where p is the degree of Taylor approximation. In this paper, we extend this high-order method to affine constrained convex optimization problems. Under the same assumption for the objective function, we show that the p-th order primal-dual algorithm converges. Moreover, we introduce two inner-loop algorithms for the practical implementation of the proposed high-order primal-dual method. Extensive numerical experiments show that, as long as the step size of dual gradient ascent updates is not too small, the proposed high-order primal-dual method outperforms vanilla primal-dual methods with low-order primal updates. However, if using optimized number of inner loops without accurately solving the inner problem in primal updates, algorithms with 2nd- and 3rd-order primal updates yield comparable results.

Keywords: Optimization, Optimization algorithms, Machine learning
Abstract: This paper aims to enhance the use of the Frank-Wolfe (FW) algorithm for training deep neural networks. Similar to any gradient-based optimization algorithm, FW suffers from high computational and memory costs when computing gradients for DNNs. This paper introduces the application of the recently proposed projected forward gradient (Projected-FG) method to the FW framework, offering reduced computational cost similar to backpropagation and low memory utilization akin to forward propagation. Our results show that trivial application of the Projected-FG introduces non-vanishing convergence error due to the stochastic noise that the Projected-FG method introduces in the process. This noise results in an non-vanishing variance in the Projected-FG estimated gradient. To address this, we propose a variance reduction approach by aggregating historical Projected-FG directions. We demonstrate rigorously that this approach ensures convergence to the optimal solution for convex functions and to a stationary point for non-convex functions. These convergence properties are validated through a numerical example, showcasing the approach’s effectiveness and efficiency.

Keywords: Optimization algorithms, Optimization, Cooperative control
Abstract: We revisit the so-called distributed two-time-scale stochastic gradient method for solving a strongly convex optimization problem over a network of agents in a bandwidth-limited regime. In this setting, the agents can only exchange the quantized values of their local variables using a limited number of communication bits. Due to quantization errors, the existing best known convergence results of this method can only achieve a suboptimal rate mathcal{O}(1/sqrt{k}), while the optimal rate is mathcal{O}(1/k) under no quantization, where k is the time iteration. The main contribution of this paper is to address this theoretical gap, where we study a sufficient condition and develop an innovative analysis and step-size selection to achieve the optimal convergence rate mathcal{O}(1/k) for the distributed gradient methods given any number of quantization bits. We provide numerical simulations to illustrate the effectiveness of our theoretical results.

Keywords: Optimization algorithms, Optimization, Numerical algorithms
Abstract: This paper presents an enhanced Trust Region Method (TRM) for Sequential Linear Programming (SLP) designed to improve the initial feasible solution to a constrained nonlinear programming problem while maintaining the interim solutions feasibility throughout the SLP iterations. The method employs a polytopic sub-approximation of the feasible region, defined around the interim solution as a level set based on variable limits for the linearization error. This polytopic feasible region is established by using a trust region that ensures that maximum limits of the linearization errors are respected. The method adaptively adjusts the size of the feasible region during iterations to achieve convergence to a local optimum by employing variable linearization error limits. Local convergence is attained by reducing the size of the trust radius. A case study illustrates the effectiveness of the proposed method, which is compared to the benchmark TRM that uses heuristic limits on the permissible changes in manipulated variables.

Keywords: Optimization algorithms, Optimization
Abstract: Many classical and modern machine learning algorithms require solving optimization tasks under orthogonality constraints. Solving these tasks with feasible methods requires a gradient descent update followed by a retraction operation on the Stiefel manifold, which can be computationally expensive. Recently, an infeasible retraction-free approach, termed the landing algorithm, was proposed as an efficient alternative. Motivated by the common occurrence of orthogonality constraints in tasks such as principle component analysis and training of deep neural networks, this paper studies the landing algorithm and establishes a novel linear convergence rate for smooth non-convex functions using only a local Riemannian PŁ condition. Numerical experiments demonstrate that the landing algorithm performs on par with the state-of-the-art retraction-based methods with substantially reduced computational overhead.

Keywords: Cooperative control, PID control, Networked control systems
Abstract: A fractional proportional-integral-derivative (PID) attitude controller for multi agent rigid-body systems is proposed in order to achieve attitude synchronization or balancing. The controller is designed on the tangent bundle of the special orthogonal group (SO(3)) and employs states comprising the rotation matrices and angular velocities of the bodies along with proportional, fractional derivative, and fractional integral feedback of these states. The performance of the closed-loop responses are evaluated numerically for both cases of synchronization and balancing.

Keywords: Spacecraft control, Markov processes, Autonomous systems
Abstract: As space activities expand within the cislunar environment, developing efficient refueling strategies becomes essential for sustaining long-term missions. The multi-spacecraft refueling problem focuses on optimizing fuel consumption to extend mission lifetimes. A multi-fidelity modeling approach addresses varying levels of precision, using the two-body problem for initial trajectory estimation, the circular restricted three-body problem for reference orbits and transfers, and the bicircular restricted four-body problem with solar radiation pressure for realistic station-keeping costs. The refueling problem is formulated as an infinite-horizon Markov decision process (MDP), optimizing the refueling strategy while preventing fuel depletion. The refueling process consists of two phases: first, the refueler transfers into a temporary phasing orbit, followed by a final phasing maneuver to rendezvous with the target spacecraft. We map the available transfers between orbits using a fast sampling technique and select the phasing orbit to minimize fuel consumption within a limited synodic period. The case study samples orbits from the Lyapunov and Halo families around the Lagrange points of the Earth-Moon system, L1, L2, and L3. Simulations demonstrate that the MDP-based planning achieves a 36% reduction in fuel consumption compared to a greedy strategy.

Keywords: Spacecraft control, Optimal control
Abstract: This paper presents an optimal control solution for an aerocapture vehicle with two control inputs, bank angle and angle of attack, referred to as augmented bank angle modulation (ABAM). We derive the optimal control profiles using Pontryagin’s Minimum Principle, validate the result numerically using the Gauss pseudospectral method (implemented in GPOPS), and introduce a novel guidance algorithm, ABAMGuid, for in-flight decision making. High-fidelity Monte Carlo simulations of a Uranus aerocapture mission demonstrate that ABAMGuid can greatly improve capture success rates and reduce the propellant needed for orbital correction following the atmospheric pass.

Keywords: Spacecraft control, Adaptive control, Lyapunov methods
Abstract: The stability and accuracy of satellite missions in space are significantly reliant on satellite attitude control. This paper presents a comprehensive study of satellite attitude tracking using Coulombic actuators and back-stepping adaptive control using signal-chasing analysis. Mathematical models which describe the dynamics and kinematics of satellite motion, and the features of the Coulombic actuator are discussed. The Lyapunov function is used to show that the closed-loop system is stable and that it is converging toward the desired equilibrium point. To manage parameter uncertainty, the backstepping adaptive control approach is used, assuring robustness and stability. Finally, extensive simulation results are presented to evaluate the performance of the proposed approach.

Keywords: Optimal control, Spacecraft control, Constrained control
Abstract: In astrodynamics and space flight mechanics problems, it is known that the choice of coordinates (and/or elements) has a significant influence on the convergence performance of the Hamiltonian/indirect boundary-value problems (HBVPs). It is oftentimes required to enforce the orbit-departure (or orbit-insertion) boundary conditions in terms of the classical orbital elements (COEs), which are more intuitive compared to the modified equinoctial elements (MEEs). In trade studies, it is quite common to seek an answer to the following question: What is the best orbit geometry to which a spacecraft can be transferred? We propose an easy-to-implement approach to enforcing orbit-departure/insertion transversality conditions using the costate mapping theory. This approach allows practitioners to formulate HBVPs using the set of MEEs (which is advantageous from a computational performance point of view) while enforcing the boundary conditions using the more intuitive COEs. Application of the method is demonstrated for generating minimum-fuel low-thrust trajectories between geocentric elliptical and near-circular orbits.

Keywords: Adaptive control, Spacecraft control, Closed-loop identification
Abstract: We consider detumbling for a rigid spacecraft with 3, 2, or 1 torque actuators. The inertia matrix of the spacecraft is assumed to be unknown, and the axes of the torque actuators and angular-rate sensors are assumed to be mutually orthogonal but are otherwise unknown. Predictive cost adaptive control (PCAC) is applied to this system using online system identification and receding-horizon optimization. PCAC is applied to Euler's equation as a sampled-data controller. The goal of this numerical investigation is to assess the effectiveness of PCAC for fully and underactuated detumbling with high modeling uncertainty and hard constraints on the torque magnitude and torque rate.

Keywords: Process Control, Chemical process control, Biotechnology
Abstract: The 21st century society faces significant sustainability challenges, including climate change, resource depletion, and environmental degradation. Microalgae have emerged as a versatile and sustainable solution to mitigate these problems. The microalgae production process involves harnessing their ability to convert sunlight, carbon dioxide, and nutrients (contaminants in wastewater media or agriculture effluents) into valuable biomass through photosynthesis, thus providing a synergy between biomass production and environmental remediation. When microalgae are produced, the culture conditions, including temperature, light intensity, nutrient concentration, dissolved oxygen, and pH, are critical factors that must be carefully controlled to ensure optimal microalgae growth. The growth dynamics of microalgae is influenced by a multitude of interacting factors that, together with the biological nature of microalgae and the variability of weather conditions, pose a significant barrier to the implementation of effective control algorithms. This tutorial paper summarizes the results of more than 20 years of experience working in the scale-up and development of modeling, control, and optimization methods for microalgae production systems. Theoretical and experimental results at industrial scale will be presented to show the potential of microalgae production systems and how control engineering solutions can contribute to make them more competitive in the market for bioproduct generation and wastewater regeneration.

Keywords: Chemical process control, Process Control
Abstract: When optimization control approaches are developed to maximize biomass production in open photobiorreactos, one of the main challenges is to have reliable online sensors to measure the biomass concentration in real time. Typically, biomass concentration is measured using turbidimeters, which allow the concentration of the culture to be related to its turbidity. These techniques are inadequate in open reactors due to the high level of contamination that these systems are subject to, and the concentration estimation is strongly affected. This presentation will introduce a novel sensor for monitoring and control of biomass concentration suitable for open photobioreactors. The sensor includes a procedure for measuring biomass concentration in microalgae cultures that transforms a measured absorption signal into the final biomass concentration of the culture. The device and procedure allow online measurement of the concentration of microalgae biomass (including cyanobacteria) in microalgae cultures at any scale, from laboratory to closed reactors of 3,000 L and open reactors of up to 100 m 3 in volume. The developed device presents a high robustness against the contamination that occurs in open reactors. An international patent was obtained for this sensor, which underlines its relevance and potential impact on the sector. In this presentation, not only the biomass estimator device is presented, but also its validation in a real semi-industrial microalgae plant, what supports its effectiveness in real operating conditions. Moreover, the integration of culture concentration measurement into the control algorithms will be represented, demonstrating an important advance in the industrial automation of this type the microalgae production systems.

This work has been financed by the following projects: PID2023-150739OB-I00 project financed by the Spanish Ministry of Science and also by the European Union (Grant agreement ID: 101060991, REALM).

Keywords: Chemical process control, Process Control, Control applications
Abstract: The solar photo-Fenton process operated in raceway pond reactors (RPRs) has garnered significant attention as a promising advanced oxidation technology for the simultaneous removal of pathogens and organic microcontaminants contained in conventional UWWTP secondary effluents. Its feasibility has been proved from a physical, operational, and economical perspective at demonstration scale under real operating environments. Furthermore, recent research trends are exploring the integration of non-conventional secondary and tertiary treatments, such as microalgae-based wastewater treatment, and solar photo-Fenton (as a quaternary treatment), to reduce operating costs and mitigate the carbon footprint of the treatment. Nevertheless, further research is still required to address the automation and optimization of large-scale solar photo-Fenton plants, as no commercial equipment has been still developed for the real-time measurement of microcontaminant concentration (target variable). In addition, reaction stage indicators (such as hydrogen peroxide) have limitations in their online quantification using commercial probes, due to the interacting wastewater matrix effect and relatively low but rapid concentration changes. As such, the quantification of these variables requires sample analysis using sophisticated and highly-sensitive analytical equipment. This leads to significant delays in measurement, making it difficult to control and optimize the treatment in real time when environmental conditions (UVA irradiance) or inlet wastewater composition suddenly change. Therefore, control and optimization techniques should be oriented towards the use of reliable and accurate kinetic models to bridge the lack of real-time measuring equipment. The presentation will showcase the kinetic modelling of different solar photo-Fenton demonstration plants. It will present the use of these models for control and optimization purposes at different levels of complexity and accuracy, advantages and disadvantages being discussed in each case.This work has been partially financed by: PROJECT TED2021-130458B-I00, CAFIRA PROJECT PID2023-152519OB-I00 and INACUA PROJECT PLSQ_00330.

Keywords: Robust control, Adaptive control, Mechatronics
Abstract: This paper introduces a novel H_{infty}/mu-based indirect adaptive robust control (ARC) strategy aiming to better handle high-order dynamics often neglected by conventional rigid-body control designs. The proposed strategy integrates H∞/μ robust control with an adaptive mechanism to adapt the parameters of the modeled structural uncertainties and disturbances, and then effectively compensate for them. With a decoupled design that separates adaptation from feedback, the proposed strategy allows for high adaptation rates without exciting flexible modes or destabilizing the overall system. Validation on a motor-driven servo table with significant flexible modes demonstrates that the proposed strategy effectively enhances track performance compared to traditional methods, including direct-type ARC and PID control. Simulations and comparative experiments confirm the superior performance of the proposed strategy.

Keywords: Robust control, Mechatronics, Adaptive control
Abstract: To advance theoretical solutions and address limitations in modeling complex servo-driven actuation systems experiencing high non-linearity and load disturbances, this paper aims to design a practical model-free generic robust control (GRC) framework for these mechanisms. This framework is intended to be applicable across all actuator systems encompassing electrical, hydraulic, or pneumatic servomechanisms, while also functioning within complex interactions among dynamic components and adhering to control input constraints. In this respect, the state-space model of actuator systems is decomposed into smaller subsystems that incorporate the first principle equation of actuator motion dynamics and interactive energy conversion equations. This decomposition operates under the assumption that the comprehensive model of the servo-driven actuator system and energy conversion, uncertainties, load disturbances, and their bounds are unknown. Then, the GRC employs subsystem-based adaptive control strategies for each state-variant subsystem separately. Despite control input constraints and the unknown interactive system model, the GRC-applied actuator mechanism ensures uniform exponential stability and robustness in tracking desired motions. It features straightforward implementation, experimentally evaluated by applying it to two industrial applications.

Keywords: Robust control, Observers for nonlinear systems, Lyapunov methods
Abstract: Dynamical equations representing an electro-hydraulic system (EHS) is composed of hydraulic, mechanical, and electrical subsystems. As a consequence, monitoring and/or controlling an EHS system requires the measurement of multiple state variables, which in turn mandates the use of additional sensory equipment. In particular, in systems where multiple hydraulic actuators are required, such as robotic arms or excavators, each additional sensor would increase the overall manufacturing and maintenance cost. In this study, we propose a robust observer/controller formulation to eliminate the need of velocity measurements in EHSs. The proposed formulation solely rely on nominal model information. The stability of the closed--loop system is ensured via novel Lyapunov-based arguments and the global uniform ultimate boundedness of the position tracking error and the velocity observation error are established. The effectiveness of the method is evaluated through simulation studies.

Keywords: Robust control, Optimal control, Adaptive control
Abstract: Partial shading conditions (PSC) and dynamic environmental variations often result in multi-peak power-voltage characteristics in photovoltaic (PV) systems. Traditional maximum power point tracking (MPPT) techniques struggle to track the maximum power point (MPP) under these conditions. To address this challenge, this paper presents an enhanced extremum seeking control strategy designed to improve MPPT performance in complex environments. The proposed employs an adaptive search mechanism that responds quickly to changes, effectively avoiding local maxima and ensuring global power optimization. Simulation results demonstrate that the algorithm achieves faster convergence and higher power stability under shading and changing environmental conditions. This method provides an efficient solution for maximizing energy utilization in PV systems operating in challenging conditions, offering significant potential for practical applications.

Keywords: Robust control, Time-varying systems, Output regulation
Abstract: We prove finite-time input-to-state stability estimates for discrete-time time-varying linear systems in closed loop with sample-data control laws. The upper bounds for the norms of the states in the estimates are suprema of the uncertainties over time intervals of constant finite lengths. We cover output feedback stabilization and input delays. We combine our results with a trajectory based approach, to prove novel global exponential input-to-state stability estimates for nonlinear systems with state delays. We illustrate our work using a dynamics containing an unknown nonlinearity and an unknown state delay.

Keywords: Robust control, Time-varying systems, Uncertain systems
Abstract: The paper presents a novel approach to synthesize robust controllers for nonlinear systems along perturbed trajectories. The approach linearizes the system with respect to a reference trajectory. In contrast to existing methods rooted in robust linear time-varying synthesis, the approach accurately includes perturbations that drive the system away from the reference trajectory. Hence, the controller obtained in the linear framework provides a significantly more robust nonlinear performance. The calculation of the controller is derived from robust synthesis approaches rooted in the integral quadratic constraints framework. The feasibility of the approach is demonstrated on a pitch tracker design for a space launcher.

Keywords: Stochastic systems, Optimal control, Uncertain systems
Abstract: To address deviations from expected performance in stochastic systems, we propose a risk-sensitive control synthesis method to minimize certain risk measures over the limiting stationary distribution. Specifically, we extend Worst-case Conditional Value-at-Risk (W-CVaR) optimization for Linear Time-invariant (LTI) systems to handle nonzero-mean noise and affine controllers, using only the first and second moments of noise, which enhances robustness against model uncertainty. Highlighting the strong coupling between the linear and bias terms of the controller, we reformulate the synthesis problem as a Bilinear Matrix Inequality (BMI), and propose an alternating optimization algorithm with guaranteed convergence. Finally, we demonstrate the numerical performance of our approach in two representative settings, which shows that the proposed algorithm successfully synthesizes risk-sensitive controllers that outperform the naive LQR baseline.

Keywords: Hybrid systems, Autonomous robots
Abstract: Path Planning for stochastic hybrid systems presents a unique challenge of predicting distributions of future states subject to a state-dependent dynamics switching function. In this work, we propose a variant of Model Predictive Path Integral Control (MPPI) to plan kinodynamic paths for such systems. Monte Carlo may be inaccurate when few samples are chosen to predict future states under state-dependent disturbances and lacks computational efficiency when scaling to more particles. We employ recently proposed Unscented Transform-based methods to capture stochasticity in the states as well as the state-dependent switching surfaces. This is in contrast to previous works that perform switching based only on the mean of predicted states. We focus our motion planning application on the navigation of a mobile robot in the presence of dynamically moving agents whose responses are based on sensor-constrained attention zones. We evaluate our framework on a simulated mobile robot and show faster convergence to a goal without collisions when the robot exploits the hybrid human dynamics versus when it does not.

Keywords: Distributed control, Cooperative control, Linear systems
Abstract: This letter studies the Friedkin-Johnsen (FJ) model with diminishing competition, or stubbornness. The original FJ model assumes fixed competition that is manifested through a constant weight that each agent gives to its initial opinion in addition to its contribution through a consensus dynamic. This letter investigates the effect of diminishing competition on the convergence point and speed of the FJ dynamics. We show that, if the competition is uniform across agents and vanishes asymptotically, the convergence point coincides with the nominal consensus reached with no competition. However, the diminishing competition slows down convergence according to its own rate of decay. We evaluate this phenomenon analytically and provide upper and lower bounds on the convergence rate. If competition is not uniform across clients, we show that the convergence point may not coincide with the nominal consensus point. Finally, we evaluate and validate our analytical insights numerically.

Keywords: Fault tolerant systems, Linear systems, Constrained control
Abstract: Cyber-physical systems are prone to sensor attacks that can compromise safety. A common approach to synthesizing controllers robust to sensor attacks is secure state reconstruction (SSR)---but this is computationally expensive, hindering real-time control. In this paper, we take a safety-critical perspective on mitigating severe sensor attacks, leading to a computationally efficient solution. Namely, we design feedback controllers that ensure system safety by directly computing control actions from past input-output data. Instead of fully solving the SSR problem, we use conservative bounds on a control barrier function (CBF) condition, which we obtain by extending the recent eigendecomposition-based SSR approach to severe sensor attack settings. Additionally, we present an extended approach that solves a smaller-scale subproblem of the SSR problem, taking on some computational burden to mitigate the conservatism in the main approach. Numerical comparisons confirm that the traditional SSR approaches suffer from combinatorial issues, while our approach achieves safety guarantees with greater computational efficiency.

Keywords: Stochastic optimal control, Iterative learning control, Reinforcement learning
Abstract: Risk-sensitive control balances performance with resilience to unlikely events in uncertain systems. This paper introduces ergodic-risk criteria, which capture long-term cumulative risks through probabilistic limit theorems. By ensuring the dynamics exhibit strong ergodicity, we demonstrate that the time-correlated terms in these limiting criteria converge even with potentially heavy-tailed process noises as long as the noise has a finite fourth moment. Building upon this, we proposed the ergodic-risk constrained policy optimization which incorporates an ergodic-risk constraint to the classical Linear Quadratic Regulation (LQR) framework. We then propose a primal-dual policy optimization method that optimizes the average performance while satisfying the ergodic-risk constraints. Numerical results demonstrate that the new risk-constrained LQR not only optimizes average performance but also limits the asymptotic variance associated with the ergodic-risk criterion, making the closed-loop system more robust against sporadic large fluctuations in process noise.

Keywords: Autonomous systems, Constrained control, Learning
Abstract: We present an interactive motion planner that integrates online learning of human driver preferences with parametric control barrier functions. Using stochastic models with Gaussian disturbances to capture human-driven vehicle behavior uncertainty, we update parameters in real-time parameter by Kalman filtering while ensuring safety by control barrier functions. A case study on highway lane-changing tasks demonstrates improved traffic flow, reduced disruptions, and lighter actuation effort compared to non-adaptive algorithms.

Keywords: Model/Controller reduction, Estimation, Distributed parameter systems
Abstract: This paper extends a Finite Difference model reduction method to the Euler-Bernoulli beam equation with fully clamped boundary conditions. The corresponding partial differential equation (PDE) is exactly observable in the energy space with a single boundary observer in arbitrarily short observation times. However, standard Finite Difference spatial discretization fails to achieve uniform exact observability as the mesh parameter approaches zero, with minimal observation time potentially depending on the filtering parameter. To address this, we propose a Finite Difference algorithm incorporating an averaging operator and discrete multipliers, leveraging Haraux’s theorem on the spectral gap to ensure uniform observability. This approach eliminates the need for artificial viscosity or Fourier filtering. Our method achieves uniform observability for arbitrarily small times with dual observers—the tip moment and average tip velocity—mirroring results from mixed Finite Elements applied to the wave equation with homogeneous Dirichlet boundary conditions, where dual controllers converge to the single controller of the PDE model [Castro, Micu—Numerische Mathematik'06]. Our reduced model is applicable to more complex systems involving Euler-Bernoulli beam equations.

Keywords: Distributed parameter systems, Kalman filtering
Abstract: This paper utilizes computational geometry methods to partition the sensor spatial distribution in order to arrive at realistic sensor distributions. An ideal sensor representing a sensor distribution with enhanced observability and filter performance properties is approximated by a network of sensor distributions that describe available sensing devices. Such approximate sensor distributions take the form of spatial delta functions representing realistic sensing devices. The approximation is made feasible via computational geometry methods using a modification of the Centroidal Voronoi Tessellations (CVT). This modified version of CVT ensures that the ideal sensor distribution is partitioned into cells of equal areas and a single pointwise sensor is placed in each cell. Once the ideal sensor distribution is approximated, an optimal filter is subsequently designed for the original process resulting in a state estimator using realistic sensing devices. Extensive numerical studies on a parabolic PDE examine the various aspects of the decomposition using a single and multiple pointwise sensors.

Keywords: Distributed parameter systems, Flexible structures, Numerical algorithms
Abstract: We use Linear Quadratic Regulation (LQR) to stabilize a bending and twisting beam by boundary control . The thin beam is rectangular and of moderate to high aspect ratio. This is the first step in using boundary actuation to stabilize a fluttering wing. The next step will be to add a model of the aerodynamic forces and moments that result ffom the motions of the wing. We leave this to the future.

We find a full state feedback that atabilizes the bending and torsion of the beam using two point actuators located at the root of the beam. One actuator delivers a bending moment to the beam and the other delivers a twisting torque to the beam. The model is the Euler Bernouli beam equation inertially coupled to the wave equation.

We express the optimal cost as a Fredholm quadratic form. Using integration by parts and completing the square we derive ta Riccati partial differential equation (RPDE) that the kernel of the optimal cost weakly satisfies. This RPDE is elliptic with a quadratic nonlinearity. We seek a continuous, weak solution. The continuity of the solution is important because it confirms that the optimal linear feedback has moved all the open loop eigenvalues off the imaginary axis and into the open left half plane.

We use Fourier methods involving two families of eigenfunctions, the family of eigenfunctions of the second order spatial partial differential operator subject to Neumann boundary conditions and the family of eigenfunctions of the fourth order spatial partial differential operator subject to modified cantilever boundary conditions. In this way we convert the RPDE into an infinite dimensional Algebraic Riccati Equation (ARE). The ARE has four coupled blocks corresponding to the two families of eigenfunctions in the first or second position. We close with an example where the infinite dimensional ARE is approximated by a sixteen dimensional ARE.

Keywords: Distributed parameter systems, Cooperative control, Traffic control
Abstract: This paper investigates the consensus problem of hyperbolic multi-agent systems (MASs) under Markov switching topologies. Firstly, we propose a boundary feedback consensus protocol for the hyperbolic MASs of conservation laws, in which the communication topology of the agents is subject to a Markov chain. Secondly, by employing the Lyapunov approach, we provide the consensus analysis under Markov switching topologies, obtaining sufficient conditions w.r.t. the boundary control matrices, Laplacian matrices and generator of Markov process for ensuring the exponential mean-square consensus. To simplify the inequality conditions, we further combine spectral decomposition techniques with the Lyapunov approach to establish sufficient conditions w.r.t. Laplacian eigenvalues, which are more tractable. Finally, we apply the proposed boundary consensus protocol to synchronize a multi-lane road traffic flow system under Markov switching topologies, and present numerical simulation results to illustrate the effectiveness of the boundary consensus protocol.

Keywords: Distributed parameter systems, Adaptive control, Numerical algorithms
Abstract: This paper proposes a closed-form adaptive tracking control approach for linear heat equations with unknown parameters to achieve full temperature profile tracking by leveraging Fourier regularization and bi-orthogonal series. A state predictor which copies the plant with unknown parameters replaced by their estimates is built and an adaptive law is designed to estimate the unknown parameters. The state predictor is decomposed into two subsystems for tracking control synthesis: the first subsystem involves terms from the original heat equation, while the second subsystem is simpler and can be reformulated as a standard heat equation. Specifically, the first subsystem is regarded as an unforced PDE whose terminal states always follow the desired temperature profile such that its initial condition can be calculated by solving the backward heat equation at every time step. To address the blow-up issue in backward calculation, a Fourier regularization scheme is explored to cut off the higher-order Fourier modes and an appropriate tradeoff between approximation accuracy and robustness is achieved. Given the solutions from the first subsystem, the initial condition for the second subsystem can be subsequently calculated. We propose a numerical algorithm to calculate a set of bi-orthogonal series offline and employ them to compute the boundary control function that drives the second subsystem to zero at every time step. Combining these two subsystems, it guarantees that the overall system follows the desired temperature profile. We demonstrate that the proposed closed-form adaptive tracking control algorithm achieves full temperature profile tracking with <2% error.

Keywords: Distributed parameter systems, Linear systems, Chemical process control
Abstract: This work addresses the design of an output regulator for a three-dimensional transport-reaction model in cylindrical coordinates described by a first-order hyperbolic partial differential equation. By utilizing the Laplace transform in space and time a closed-form semigroup operator is derived to capture the system’s evolution in time, and the Lyapunov equation is solved to address the stability analysis. An output regulator is designed to track periodic signals generated by a finite-dimensional exosystem. This is achieved by solving the continuous-time Sylvester output regulation equation. Numerical simulations demonstrate the controller’s efficiency in properly tracking the desired signal.

Keywords: PID control, Mechatronics, Process Control
Abstract: It is not uncommon for graduate students on the mechatronics side of the control world to treat the Proportional plus Integral plus Derivative (PID) controller with a certain amount of disdain. This is not surprising since most control texts from this end of the control world treat PIDs as simple, basic structures, to be quickly replaced by more advanced methods. To that end, these texts devote only a handful of pages to the subject. It seems that – at least in the mechatronics world – PIDs are considered too simple for much interest in academia while practicing engineers do not seem to care why they were working.

This is a far cry from the treatment of PIDs in the chemical and bio-process control worlds (CPC and BPC, respectively). At this end of the control spectrum, PID controllers are studied in far more depth obtaining entire books or book series. Despite this volumetric expansion of material, it seems that in the latter worlds, many of the issues and concerns one sees in the mechatronic world are treated as obscure corner cases. Depending upon the teaching text, issues of sampling and digital representation may have been completely omitted. There were other surprises. While PIDs were almost universal and standard, they were almost never unified or standardized. Furthermore, what seemed to limit performance was not the structure of the controller itself, but the lack of accurate system/process models based on repeated physical system measurements.

However, the mechatronic and process PID goals and foibles were not that different once one considered the different system, time constant, and measurement constraints. We will discuss these issues with the goal of getting a more unified view of PIDs across our application domains. We will provide a handful of common PID forms and show how they are related, so that we can approach any PID structure with the same analytical approach. We will finally look forward to how PIDs can be used, not only as a fundamental teaching tool for explaining control outside of our research circles, but as a critical component for advanced control methods.

Keywords: PID control, Mechatronics, Process Control
Abstract: A key feature of most Proportional-Integral-Derivative (PID) controllers is the need to prevent integrator windup in the event of control signal saturation. Providing a means of anti-windup can be viewed as a main motivator for having the integrator block separated out from the rest of the controller. The PID literature contains several popular methods for implementing anti-windup, including integrator reset, back calculation, and integrator clamping (also known as conditional integration). The effectiveness of these methods depends upon when the saturation occurs and how the PID is implemented (in continuous or discrete time). Some methods make sense in either domain, while some are only feasible in a discrete-time implementation. This talk will discuss the different methods in the context of which are most popular in the process control world versus which are used in the mechatronic control world.

Keywords: PID control, Mechatronics, Process Control
Abstract: Loop shaping typically refers to using the frequency response function (FRF) of a system to guide the compensation so that the overall loop (open or closed) takes on a desired shape. One simple loop shaping metric is to make the open-loop response resemble that of an integrator to as high a frequency for which one can maintain phase margin. Not only does this show up across multiple applications, but its simplicity makes it easier to explain and transfer into industrial products. Furthermore, if the loop gain crossover is chosen to guarantee sufficient gain margin, typically around 60◦ or more, then the closed-loop response resembles a first-order low-pass filter (LPF) with minimal peaking. While this is a popular way to work in the mechatronic control world, frequency response methods are far less prevalent in the process control world. However, it can be shown that Internal Model Control (IMC) is algebraically performing loop shaping. We will compare and unify these different views of loop shaping for PID controllers.

Keywords: Kalman filtering, Stability of nonlinear systems, Filtering
Abstract: We analyze the stochastic stability of a class of generic Kalman-type nonlinear filtering algorithms with the aim of providing practically verifiable conditions. We investigate conditions under which the Kalman gain matrix and the approximate error covariance matrix are bounded in nonlinear systems. By stochastic Lyapunov stability theory, we show that the estimation error for the nonlinear filter is bounded exponentially in mean square, with its norm bounded in probability. The analysis in this work applies to popular nonlinear Kalman filter extensions such as the extended Kalman filter (EKF) and the constant-gain Kalman filter (CGKF).

Keywords: Estimation, Filtering, Kalman filtering
Abstract: This letter introduces a Kalman Filter framework for systems with process noise and measurements characterized by state-dependent, nonlinear conditional means and covariances. Estimating such general nonlinear models is challenging because traditional methods, such as the Extended Kalman Filter, linearize only functions – not noise – and require state-independent covariances. These limitations often necessitate Bayesian approaches that rely on specific distribution assumptions. To address these challenges, we propose a framework that employs a recursive least squares method that relies solely on conditional means and covariances, eliminating the need for explicit probability distributions. By applying first-order linearizations and incorporating targeted modifications to manage state dependence, the filter simplifies implementation, reduces computational demands, and provides a practical solution for systems that deviate from the assumptions underlying traditional Kalman filters. Simulation results on a compartmental model demonstrate performance comparable to sequential Monte Carlo methods while significantly lowering computational costs, effectively addressing real-world challenges of scalability and precision.

Keywords: Estimation, Hybrid systems, Algebraic/geometric methods
Abstract: This paper addresses the problem of global attitude filtering on the special orthogonal group SO(3). A hysteresis-based hybrid switching strategy is used to switch between two filters that operate in different regions of SO(3). The first filter is the passive complementary filter, whereas the second filter is designed using an appropriately chosen Morse function. To this end, a novel approach to design Morse functions on SO(3) is proposed. The proposed hybrid filter is shown to be input-to-state stable with respect to measurement noise. Simulations validate the stability properties of the hybrid filter.

Keywords: Stability of nonlinear systems, Robust control, Lyapunov methods
Abstract: Asymptotic disturbance rejection (equivalently tracking) for nonlinear systems has been studied only in qualitative terms (the state is asymptotically stable under bounded disturbances). We show how to prove quantitative performance guarantees for the nonlinear servomechanism problem. Our technique originates by applying a gain inequalities point of view to an emph{ad fontes} reexamination of the linear problem: what is the nonlinear equivalent of a sensitivity transfer function with a zero at the origin? We answer: a nonlinear input-output system is high-pass if its output is stable with respect to the emph{derivative} of the input. We first show that definition generalizes high-pass resistor-capacitor circuit analysis to accommodate nonlinear resistors. We then show that this definition generalizes the steady-state disturbance rejection property of integral feedback controllers for linear systems. The theoretical payoff is that low-frequency disturbance rejection is captured by a quantitative, non-asymptotic output cost bound. Finally, we raise theoretical questions about compositionality of nonlinear operators.

Keywords: Markov processes, Stability of nonlinear systems, Reinforcement learning
Abstract: We obtain conditions for Wasserstein regularity of non-linear filters, which serve as reduced MDP (Markov Decision Process) models for Partially Observable Markov Decision Process (POMDPs). In particular, we obtain general and unified conditions which recover several recent studies. These conditions lead to (i) unique ergodicity and (ii) geometric ergodicity of control-free non-linear filter kernels, as well as (iii) existence of optimal solutions and (iv) their rigorous approximations and Q-learning for discounted cost as well as average cost optimal stochastic control problems. Specifically, we identify complementary conditions under which finite-window policies remain near-optimal and establish refined error bounds. While previous work assumed transition kernels to be continuous in total variation and Wasserstein regularity of measurement channels with moduli of continuity, we provide a general conditions which both recovers the aforementioned and also shows that Wasserstein regularity of the transition kernel and total variation continuity of the measurement channels with moduli of continuity is sufficient for existence, approximations and robustness results involving POMDPs.

Keywords: Neural networks, Machine learning, Aerospace
Abstract: This tutorial highlights the engineering aspects of developing AI-based control systems. It renders the whole systems engineering spectrum from safety assessment to requirements engineering, from design to implementation, from verification to deployment and further more certification and operation of the aerospace products that include control systems that utilize AI techniques. The position paper not only provides a examples of AI-based safety-critical control systems in aerospace, it also shares the opinions of the authors about the important techniques that are being developed or to be developed at each above mentioned activities of systems engineering.

Keywords: Output regulation, Data driven control, Stability of nonlinear systems
Abstract: This paper addresses the semi-global output regulation problem for continuous-time linear systems with input saturation and unknown dynamics. First, we employ a low-gain technique to design a state-feedback linear control law such that the control input operates within the linear region of the actuator. Then, taking it as the linear part, we construct a composite nonlinear feedback (CNF) control law, consisting of both linear and nonlinear parts, to improve the transient performance of the closed-loop system. Without requiring prior knowledge of the system dynamics or an initial stabilizing control policy, we propose a novel adaptive dynamic programming (ADP) learning algorithm. This algorithm learns both the linear part and the nonlinear part of the CNF control law using the same set of data. In addition, the algorithm uses single-layer filters, eliminating the need for integral operations during the learning process. Finally, the effectiveness of the proposed algorithm is demonstrated by an illustrative example.

Keywords: Statistical learning, Behavioural systems, Identification for control
Abstract: In this paper, we study the noise sensitivity of the semidefinite program (SDP) used in the direct data-driven infinite horizon linear quadratic regulator (LQR) problem for discrete-time linear time-invariant systems. While this SDP is shown to find the true LQR controller in the noise-free setting, we show that it leads to a trivial solution when data is corrupted by noise, even when the noise is arbitrarily small. Hence, a “certainty equivalence” approach that uses the original SDP with noisy data is not appropriate.

Keywords: Robust adaptive control, Stability of nonlinear systems, Nonlinear systems identification
Abstract: This work presents a computationally efficient approach to data-driven robust contracting controller synthesis for polynomial control-affine systems based on a sum-of-squares program. In particular, we consider the case in which a system alternates between periods of high-quality sensor data and low-quality sensor data. In the high-quality sensor data regime, we focus on robust system identification based on the data informativity framework. In low-quality sensor data regimes we employ a robustly contracting controller that is synthesized online by solving a sum-of-squares program based on data acquired in the high-quality regime, so as to limit state deviation until high-quality data is available. This approach is motivated by real-life control applications in which systems experience periodic data blackouts or occlusion, such as autonomous vehicles undergoing loss of GPS signal or solar glare in machine vision systems. We apply our approach to a planar unmanned aerial vehicle model subject to an unknown wind field, demonstrating its uses for verifiably tight control on trajectory deviation.

Keywords: Predictive control for linear systems, Data driven control, Identification for control
Abstract: We propose a novel data-driven stochastic model predictive control framework for uncertain linear systems with noisy output measurements. Our approach leverages multi-step predictors to efficiently propagate uncertainty, ensuring chance constraint satisfaction. In particular, we present a strategy to identify multi-step predictors and quantify the associated uncertainty using a surrogate (data-driven) state space model. Then, we utilize the derived distribution to formulate a constraint tightening that ensures chance constraint satisfaction despite the parametric uncertainty. A numerical example highlights the reduced conservatism of handling parametric uncertainty in the proposed method compared to state-of-the-art solutions.

Keywords: Predictive control for nonlinear systems, Data driven control, Machine learning
Abstract: This paper presents a kernelized offset-free data-driven predictive control scheme for nonlinear systems. Traditional model-based and data-driven predictive controllers often struggle with inaccurate predictors or persistent disturbances, especially in the case of nonlinear dynamics, leading to tracking offsets and stability issues. To overcome these limitations, we employ kernel methods to parameterize the nonlinear terms of a velocity model, preserving its structure and efficiently learning unknown parameters through a least squares approach. This results in a offset-free data-driven predictive control scheme formulated as a nonlinear program, but solvable via sequential quadratic programming. We provide a framework for analyzing recursive feasibility and stability of the developed method and we demonstrate its effectiveness through simulations on a nonlinear benchmark example.

Keywords: Fault diagnosis, Identification, Subspace methods
Abstract: This paper proposes a data-driven approach to design a fault isolation filter to isolate actuator faults in discrete linear time-invariant systems. It is shown that a fault isolation filter can be obtained directly from the system input and output data without any knowledge about the system model. The subspace identification technique is applied to obtain the fault detectability indices and the fault detectability matrix directly from the data. The design procedure involves a singular value decomposition and an LU factorization with partial pivoting. The identified fault isolation filter is able to isolate multiple faults with only a single filter, which significantly reduces the online computational efforts needed for fault isolation. A simulation example is given to illustrate the proposed datadriven approach.

Keywords: Reinforcement learning, Optimal control, Iterative learning control
Abstract: Reinforcement Learning for control of an unknown nonlinear dynamical system is equivalent to the search for an optimal feedback law utilizing only data from the simulations/ rollouts of the dynamical system. Most RL techniques search over a complex global nonlinear feedback parametrization, making them suffer from high training times and variance. Instead, we advocate a model-based reinforcement learning (MBRL) framework that searches over a local feedback representation consisting of an open-loop sequence and an associated optimal linear feedback law. We show that this approach results in highly efficient training, the answers obtained are globally optimum, repeatable with negligible variance, and hence reliable, and the resulting closed performance is superior to the state-of-the-art RL techniques that search for a global nonlinear feedback law. Finally, if we replan whenever required, akin to Model Predictive Control (MPC), which is feasible due to the fast and reliable local solution, it allows us to recover the optimal global feedback law.

Keywords: Reinforcement learning, Nonlinear output feedback, Optimal control
Abstract: This talk develops a model-based Reinforcement Learning approach to the closed-loop control of nonlinear dynamical systems with a partial nonlinear observation model. We propose an “information-state” based approach to rigorously transform the partially observed problem into a fully observed problem where the information-state consists of the past several observations and control inputs. We further show the equivalence of the transformed and the initial partially observed optimal control problems and provide the conditions to solve for the deterministic optimal solution. We develop a data-based generalization of the iterative Linear Quadratic Regulator (ILQR) for the RL of partially-observed systems using a local linear time-varying model of the information-state dynamics approximated by an Autoregressive–moving-average (ARMA) model that is generated using only the input-output data. This open-loop trajectory optimization solution is then used to design a local feedback control law, and the composite law then provides an optimum solution to the partially observed feedback design problem. The efficacy of the developed method is shown by controlling complex high dimensional nonlinear dynamical systems in the presence of model and sensing uncertainty.

Keywords: Optimal control, Stability of nonlinear systems, Predictive control for nonlinear systems
Abstract: In this talk, we consider the infinite horizon optimal control problem for nonlinear systems. Under the conditions of controllability of the linearized system around the origin, and nonlinear controllability of the system to a terminal set containing the origin, we establish an approximate regularized solution approach consisting of a “finite free final time” optimal transfer problem to the terminal set, and an infinite horizon linear regulation problem within the terminal set, that is shown to render the origin globally asymptotically stable. Further, we show that the approximations converge to the true optimal cost function as the size of the terminal set decreases to zero. The talk also discusses the extension to nonholonomic systems, which are not linearly controllable and show similar guarantees. The proposed data-based approach is empirically evaluated on the robotics problems to show that the finite time transfer is far shorter than the effective horizon required to solve the infinite horizon problem without the proposed regularization. We also do comparisons of our approach with nonlinear Model Predictive Control.

Keywords: Large-scale systems, Reduced order modeling, Optimal control
Abstract: This talk will introduce a reduced order model based reinforcement learning (MBRL) approach, utilizing the Iterative Linear Quadratic Regulator (ILQR) algorithm for the optimal control of nonlinear partial differential equations (PDEs). The approach proposes a novel modification of the ILQR technique: it uses the Method of Snapshots to identify a reduced order Linear Time Varying (LTV) approximation of the nonlinear PDE dynamics around a current estimate of the optimal trajectory, utilizes the identified LTV model to solve a time-varying reduced order LQR problem to obtain an improved estimate of the optimal trajectory along with a new reduced basis, and iterates till convergence, in an ‘Iteratively-Reduce-then-Control (IRTC)’ manner. The proposed approach is tested on the viscous Burger’s equation and two phase-field models for microstructure evolution in materials, and the results show that there is a significant reduction in the computational burden over the standard ILQR approach, without significantly sacrificing performance.