Keywords: Nonlinear systems identification, Lyapunov methods, Machine learning
Abstract: An approximate optimal policy is developed for a pursuing agent to indirectly regulate an evading agent coupled by an uncertain interaction dynamic. Approximate dynamic programming is used to design a controller for the pursuing agent to optimally influence the evading agent to a goal location. Since the interaction dynamic between the agents is unknown, integral concurrent learning is used to update a Lyapunov-based deep neural network to facilitate sustained learning and system identification. A Lyapunov-based stability analysis is used to show uniformly ultimately bounded convergence. Simulation results demonstrate the performance of the developed method.

Keywords: Nonlinear output feedback, Learning, Nonlinear systems
Abstract: We introduce a framework for accelerating the computation of a backstepping controller in PDE control. We learn the nonlinear operator from the plant parameter and PDE solution to the boundary control with a (deep) neural network. We provide closed-loop stability guarantees (semiglobal exponential) under a NN-approximation of the feedback law. While, in the existing PDE backstepping, finding a feedback law requires the solution to multiple integral equations and operations for both the gain and control input value, the neural operator (NO) approach we propose learns the mapping from the functional coefficients of the plant PDE and PDE system state to the boundary control value by employing a sufficiently high number of offline numerical solutions to the analytical feedback control law. We prove the existence of a DeepONet approximation with arbitrarily high accuracy, of the exact nonlinear continuous operator mapping between the PDE coefficient functions and PDE system state into a control feedback law. Once proven to exist, learning of the NO is standard, completed "once and for all" (never online) and the control feedback equation doesn't need to be solved ever again, for both any new functional coefficient and PDE system state that does not exceed the magnitude of the coefficients and states used in training. Simulation illustrations are provided and the code is available on github.

Keywords: Optimal control, Optimization, Nonlinear systems
Abstract: The aim of this paper is to assess the effect of the presence of an opponent in a class of finite-horizon differential games described by scalar linear differential equations and quadratic cost functionals in which the state is penalized only at the terminal time. The contribution of the other player is quantitatively characterized by comparing the solutions of the underlying Riccati differential equations for the optimal control (in the absence of the opponent) and of the differential game. In the case of open-loop Nash equilibria, this effect can be characterized in closed form, since an analytic expression for the solutions of the coupled asymmetric differential Riccati equations can be computed. For feedback Nash equilibria a closed-form solution to the related coupled symmetric differential Riccati equations cannot be determined. Therefore an estimate of the solution is provided by relying on a functional approximation approach, allowing to characterize the effect of the presence of an opponent also in this setting.

Keywords: Data driven control, Learning, Optimal control
Abstract: One problem that arises in control engineering is that of controlling a system for which the dynamics are unknown. Such a problem favors a data-driven approach, such as can be done through the use of the Koopman operator. We present a switched Koopman model, applicable to systems with discrete sets of inputs, that gives rise to an optimal control problem with a piecewise affine value function. This structure provides an efficient representation and enables a heuristic pruning algorithm that avoids the exponential complexity of finding the true optimal solution. We use density-like observables that are defined through the notion of entity-based systems: systems whose state is composed of a possibly varying number of entities that can be grouped into classes of like-entities. This encompasses many systems, including arcade games, a common benchmark used in reinforcement learning. We find that our approach requires much less training than commonly used for reinforcement learning. The Koopman approach also has the advantage of being agnostic to the control objective, which allows the objective to be changed without needing to retrain the model.

Keywords: Neural networks, Optimal control, Learning
Abstract: In this work, we leverage physics-informed neural networks (PINNs) to approximately solve the infinite-horizon optimal control problem for nonlinear systems. Specifically, since PINNs are generally able to solve a class of partial differential equations, they can be employed to approximate the value function in the infinite-horizon optimal control problem, via solving the associated steady-state Hamilton-Jacobi-Bellman (HJB) equation. However, the issue with such a direct approach is that the steady HJB equation generally yields more than one solution, hence directly employing PINNs to solve it can lead to divergence of the method. To tackle this problem, we instead apply PINNs to a finite-horizon variant of the steady-state HJB equation which has a unique solution, and which uniformly approximates the infinite-horizon optimal value function as the horizon increases. A method to verify whether the selected horizon is large enough is also provided, as well as an algorithm to increase it with reduced computations if it is not. Unlike conventional methods, the proposed approach does not require knowledge of a stabilizing controller, the execution of computationally expensive iterations, or polynomial basis functions for approximation.

Keywords: Machine learning, Robust control, Uncertain systems
Abstract: Safety-critical cyber-physical systems require control strategies whose worst-case performance is robust against adversarial disturbances and modeling uncertainties. In this paper, we present a framework for approximate control and learning in partially observed systems to minimize the worst-case discounted cost over an infinite time horizon. We model disturbances to the system as finite-valued uncertain variables with unknown probability distributions. For problems with known system dynamics, we construct a dynamic programming (DP) decomposition to compute the optimal control strategy. Our first contribution is to define information states that improve the computational tractability of this DP for a class of problems with observable incurred costs at each time instance. Our second contribution proposes approximate information states that can be constructed or learned directly from observed data for these problems. We derive bounds on the performance loss of the resulting approximate control strategy and illustrate the effectiveness of our approach in partially observed decision-making problems with a numerical example.

Keywords: Stability of nonlinear systems, Network analysis and control, Large-scale systems
Abstract: This letter is concerned with a compositional data-driven approach for stability certificate of interconnected homogeneous networks with (partially) unknown dynamics while providing 100% correctness guarantees (as opposed to probabilistic confidence). The proposed framework enjoys input-to-state stability (ISS) properties of subsystems described by ISS Lyapunov functions. In our data-driven scheme, we first reformulate the corresponding conditions of ISS Lyapunov functions as a robust optimization program (ROP). Due to appearing unknown dynamics of subsystems in the constraint of ROP, we propose a scenario optimization program (SOP) by collecting data from trajectories of each unknown subsystem. We solve SOP and construct an ISS Lyapunov function for each subsystem with unknown dynamics. We accordingly leverage a compositional technique based on max-type small-gain reasoning and construct a Lyapunov function for an unknown interconnected network based on ISS Lyapunov functions of individual subsystems. We demonstrate the efficacy of our data-driven approach over a room temperature network containing 1000 rooms with unknown dynamics. Given collected data from each unknown room, we verify that the unknown interconnected network is globally asymptotically stable (GAS) with 100% correctness guarantee.

Keywords: Data driven control, Lyapunov methods, Machine learning
Abstract: Recent advances in learning-based control leverage deep function approximators, such as neural networks, to model the evolution of controlled dynamical systems over time. However, the problem of learning a dynamics model and a stabilizing controller persists, since the synthesis of a stabilizing feedback law for known nonlinear systems is a difficult task, let alone for complex parametric representations that must be fit to data. To this end, we propose Control with Inherent Lyapunov Stability (CoILS), a method for jointly learning parametric representations of a nonlinear dynamics model and a stabilizing controller from data. To do this, our approach simultaneously learns a parametric Lyapunov function which intrinsically constrains the dynamics model to be stabilizable by the learned controller. In addition to the stabilizability of the learned dynamics guaranteed by our novel construction, we show that the learned controller stabilizes the true dynamics under certain assumptions on the fidelity of the learned dynamics. Finally, we demonstrate the efficacy of CoILS on a variety of simulated nonlinear dynamical systems.

Keywords: Sampled-data control, Iterative learning control, Behavioural systems
Abstract: We introduce an adaptive refinement procedure for smart and scalable abstraction of dynamical systems. Our technique relies on partitioning the state space depending on the observation of future outputs. However, this knowledge is dynamically constructed in an adaptive, asymmetric way. In order to learn the optimal structure, we define a Kantorovich-inspired metric between Markov chains, and we use it to guide the state partition refinement. Our technique is prone to data-driven frameworks, but not restricted to.

We also study properties of the above mentioned metric between Markov chains, which we believe could be of broader interest. We propose an algorithm to approximate it, and we show that our method yields a much better computational complexity than using classical linear programming techniques.

Keywords: Neural networks, Autonomous systems, Nonlinear systems
Abstract: In this paper, we present a contraction-guided adaptive partitioning algorithm for improving interval-valued robust reachable set estimates in a nonlinear feedback loop with a neural network controller and disturbances. Based on an estimate of the contraction rate of over-approximated intervals, the algorithm chooses when and where to partition. Then, by leveraging a decoupling of the neural network verification step and reachability partitioning layers, the algorithm can provide accuracy improvements for little computational cost. This approach is applicable with any sufficiently accurate open-loop interval-valued reachability estimation technique and any method for bounding the input-output behavior of a neural network. Using contraction-based robustness analysis, we provide guarantees of the algorithm's performance with mixed monotone reachability. Finally, we demonstrate the algorithm's performance through several numerical simulations and compare it with existing methods in the literature. In particular, we report a sizable improvement in the accuracy of reachable set estimation in a fraction of the runtime as compared to state-of-the-art methods.

Keywords: Formal Verification/Synthesis, Vision-based control, Machine learning
Abstract: This paper studies the problem of designing a certified vision-based state estimator for autonomous landing systems. In such a system, a neural network (NN) processes images from a camera to estimate the aircraft's relative position with respect to the runway. We propose an algorithm to design such NNs with certified properties in terms of their ability to detect runways and provide accurate state estimation. At the heart of our approach is the use of geometric models of perspective cameras to obtain a mathematical model that captures the relation between the aircraft states and the inputs. We show that such geometric models enjoy mixed monotonicity properties that can be used to design state estimators with certifiable error bounds. We show the effectiveness of the proposed approach using an experimental testbed on data collected from event-based cameras.

Keywords: Stochastic systems, Data driven control, Network analysis and control
Abstract: This work proposes a compositional data-driven technique for the construction of finite Markov decision processes (MDPs) for large-scale stochastic networks with unknown mathematical models. Our proposed framework leverages dissipativity properties of subsystems and their finite MDPs using a notion of stochastic storage functions (SStF). In our data-driven scheme, we first build an SStF between each unknown subsystem and its data-driven finite MDP with a certified probabilistic confidence. We then derive dissipativity-type compositional conditions to construct a stochastic bisimulation function (SBF) between an interconnected network and its finite MDP using data-driven SStF of subsystems. Accordingly, we formally quantify the probabilistic distance between trajectories of an unknown large-scale stochastic network and those of its finite MDP with a guaranteed confidence. We illustrate the efficacy of our data-driven results over a room temperature network composing 100 rooms with unknown models.

Keywords: Robotics, Autonomous robots, Uncertain systems
Abstract: 5G networks have the potential to provide external sensor data and offload computations for future industrial mobile robots. To enable these benefits while maintaining safety, we propose a modular architecture, including an onboard safety filter for the velocity control loop. The safety filter leverages robust control barrier functions to guarantee safety from collisions under bounded localization uncertainty. Initial experiments are performed to quantify the localization uncertainty and generate suitable bounds for the safety filter. We then derive the safety filter, and analyze its conservatism numerically. Finally, the method is demonstrated in experiments using an ABB Mobile YuMi® Research Platform robot.

Keywords: Vision-based control, Machine learning, Autonomous systems
Abstract: We consider perception-based control using state estimates that are obtained from high-dimensional sensor measurements via learning-enabled perception maps. However, these perception maps are not perfect and result in state estimation errors that can lead to unsafe system behavior. Stochastic sensor noise can make matters worse and result in estimation errors that follow unknown distributions. We propose a perception-based control framework that i) quantifies estimation uncertainty of perception maps, and ii) integrates these uncertainty representations into the control design. To do so, we use conformal prediction to compute valid state estimation regions, which are sets that contain the unknown state with high probability. We then devise a sampled-data controller for continuous-time systems based on the notion of measurement robust control barrier functions. Our controller uses idea from self-triggered control and enables us to avoid using stochastic calculus. Our framework is agnostic to the choice of the perception map, independent of the noise distribution, and to the best of our knowledge the first to provide probabilistic safety guarantees in such a setting. We demonstrate the effectiveness of our proposed perception-based controller for a LiDAR-enabled F1/10th car.

Keywords: Control applications, Data driven control, Uncertain systems
Abstract: A novel approach for robust controller synthesis, which models uncertainty as an elliptical set, is proposed in the paper. Given a set of frequency response functions of linear time-invariant (LTI) multiple-input multiple-output (MIMO) systems, the approach determines the `best' linear nominal model and the corresponding elliptical uncertainty set, which is consistent with the data. Using a novel split representation, the uncertainty set is represented as an equivalent integral quadratic constraint (IQC). Finally, this IQC is integrated into a data-driven frequency-domain controller synthesis method using convex optimisation. The proposed method is used to design a controller, which is robust against mechanical uncertainties, for a hybrid micro-disturbance isolation platform for space applications. The experimental results show that the proposed method provides a `tighter' uncertainty set and improves attenuation performance compared to classical methods that use disk uncertainty.

Keywords: Network analysis and control, Networked control systems, Autonomous vehicles
Abstract: The emergence of advanced perception algorithms has opened the possibility of achieving long-term autonomy in vehicle platooning. We develop a framework to assess the risk of misperception due to noisy observations from the environment. Each vehicle is assumed to rely on a perception unit to comprehend its surrounding environment and estimate other vehicles' positions and velocity. The Expected Shortfall (Average Value-at-Risk) measure is employed to assess the risk of collision between vehicle pairs and the risk of violating traffic laws for each vehicle under possible misperceptions. Obtaining an explicit expression for the risk measure allows us to further explore the potential trade-offs between the overall misperception-induced risks and network architecture. Using our framework, we demonstrate how misperception of highway traffic signs may cause phenomena similar to tailgating and quantify how it affects the risk of such events. Our result can also be applied to identify vehicles with high fragility to misperception and to increase the robustness of the platoon concerning overall risk measures in the presence of misperception. Our theoretical results are validated by extensive simulation.

Keywords: Formal Verification/Synthesis, Autonomous robots
Abstract: Compared to safe obstacle avoidance in the position space, ensuring the safety of the attitude system in today's aerial vehicle operations is more challenging due to the non-Euclidean nature of the attitude space and the underactuated nature of the system. To address this issue, we first propose a Geometric Exponential Barrier Condition (GEBC) to produce barrier certificates on the manifold, by which attitude safety requirements can be encoded globally into the verification problems of attitude control systems. Then, we use exponential coordinates to characterize GEBCs, which makes them descriable in terms of Quantifier Free Real Arithmetic Logic (QF-NRA) and efficiently solvable by SMT solvers. A performance criterion is further discussed where we propose an effective algorithm to construct safe operational regions with different controllers, which can help with nominal controller selection and tuning. Finally, we demonstrate our approach in a quadrotor system and analyze the safe performance of two PD controllers on the proposed safe operation criteria.

Keywords: Autonomous vehicles, Hybrid systems, Stochastic optimal control
Abstract: This paper presents a scenario-based hybrid model predictive control (MPC) design approach for cooperative adaptive cruise control (CACC) in mixed-autonomy traffic environments with uncertainties stemming from unexpected maneuvers of human-driven vehicles. Different from the past works that consider one possible realization of uncertainty, the proposed approach here considers multiple scenarios based on the uncertainty description that varies with the relative location of the human-driven vehicles (HVs) and the connected and automated vehicles (CAVs). For each scenario, a mixed integer quadratic programming problem is formulated for the control of CAVs with four operating modes, namely free following, braking, danger, and lane change. Each CAV's operating mode is determined based on the predictive information it receives from its predecessors and the anticipated behaviors of surrounding HVs. All the scenarios are handled simultaneously using the scenario-based MPC approach for a robust CACC. Simulations in a mixed-autonomy traffic system including two lanes demonstrate that the proposed scenario-based hybrid MPC approach significantly reduces deviations from the desired spacing policy and the desired velocity in the platoon during unexpected human-driven vehicle maneuvers, compared with the past work, particularly, a discrete hybrid stochastic (DHSA) MPC approach.

Keywords: Autonomous vehicles, Optimization, Evolutionary computing
Abstract: The trajectory-based traveling salesman problem represents an extension of the classical traveling salesman problem, aimed at determining the most optimal trajectory that passes through a designated set of points. This paper introduces a novel formulation, termed the Energy-Optimal Trajectory-Based Traveling Salesman Problem (EOTB-TSP), which is grounded in an innovative energy assessment model. This model takes into account the intricate dynamics of unmanned aerial vehicles (UAVs). In addition, the EOTB-TSP is cast as a bilevel optimization challenge. To tackle this complex problem, we introduce a modified genetic algorithm specifically tailored for its resolution. To validate the effectiveness of our proposed approach, we conduct a series of experiments and apply it to real-world scenarios. Our evaluation and comparative analyses unequivocally demonstrate the high efficiency of our method in minimizing energy consumption.

Keywords: Autonomous vehicles, Optimal control, Optimization algorithms
Abstract: In this work, a combined task and motion planner for a tractor and a set of trailers is proposed and it is shown that it is resolution complete and resolution optimal. The proposed planner consists of a task planner and a motion planner that are both based on heuristically guided graph-search. As a step towards tighter integration of task and motion planning, we use the same heuristic that is used by the motion planner in the task planner as well. We further propose to use the motion planner heuristic to give an initial underestimate of the motion costs that are used as costs during the task planning search, and increase this estimate gradually by using the motion planner to verify the cost and feasibility of actions along paths of interest. To limit the time spent in the motion planner, the use of time and cost limits to pause or prematurely abort the motion planner is proposed, which does not affect the resolution completeness or resolution optimality. The planner is evaluated on numerical examples and the results show that the proposed planner can significantly reduce the execution time compared to a baseline resolution optimal task and motion planner.

Keywords: Autonomous vehicles, Optimal control, Predictive control for nonlinear systems
Abstract: In this paper we treat optimal trajectory planning for an autonomous vehicle (AV) operating in dense traffic, where vehicles closely interact with each other. To tackle this problem, we present a novel framework that couples trajectory prediction and planning in multi-agent environments, using distributed model predictive control. A demonstration of our framework is presented in simulation, employing a trajectory planner using non-linear model predictive control. We analyze performance and convergence of our framework, subject to different prediction errors. The results indicate that the obtained locally optimal solutions are improved, compared with decoupled prediction and planning.

Keywords: Autonomous vehicles, Predictive control for linear systems
Abstract: This paper proposes a model predictive control framework to design autonomous rendezvous operations for terrestrial and space missions in the scope of spacecraft and drones, in the presence of obstacles in an unknown environment. Vehicles equipped with a LiDAR collecting points associated with the obstacles are considered. By proposing an efficient method to compute the smallest ellipse that contains the collected LiDAR points, this work is able to correct, in run time, the vehicle trajectory to avoid collision and reach the rendezvous target. Finally, simulations of the proposed approach to show its effectiveness are presented.

Keywords: Networked control systems, Optimization, Constrained control
Abstract: How to effectively communicate over wireless networks characterized by link failures is central to understanding the fundamental limits in the performance of a networked control system. In this paper, we study the online remote control of linear-quadratic Gaussian systems over unreliable wireless channels (with random packet drops), where the controller is a priori oblivious to the cost parameters. We first reformulate the problem using a semi-definite program and consequently compute a stabilizing policy from its solution. We then derive a O(sqrt(T)) regret bound (against a best offline policy in hindsight) for a projected online gradient algorithm, where T is the length of the horizon of interest. In the process, we introduce finite time notions of the classical mean-square stability, which may be of independent interest. Finally, we provide a numerical example to validate the theoretical results, demonstrating the limitations induced by lossy communication on the control performance.

Keywords: Optimization, Robust control, Optimal control
Abstract: Direct policy search has achieved great empirical success in reinforcement learning. Recently, there has been increasing interest in studying its theoretical properties for continuous control, and fruitful results have been established for linear quadratic regulator (LQR) and linear quadratic Gaussian (LQG) control that are smooth and nonconvex. In this paper, we consider the standard H∞ robust control for output feedback systems and investigate the global optimality of direct policy search. Unlike LQR or LQG, the H∞ cost function is nonsmooth in the policy space. Despite the lack of smoothness and convexity, our main result shows that for a class of non-degenerate stabilizing controllers, all Clarke stationary points of H∞ robust control are globally optimal and there is no spurious local minimum. Our proof technique is motivated by the idea of differentiable convex liftings (DCL), and we extend DCL to analyze the nonsmooth and nonconvex H∞ robust control via convex reformulation. Our result sheds some light on the analysis of direct policy search for solving nonsmooth and nonconvex robust control problems.

Keywords: Data driven control, Predictive control for linear systems, Networked control systems
Abstract: This paper presents a novel approach for data-driven self-triggered state feedback control of unknown linear systems using noisy data gathered offline. The self-triggering mechanism determines the next triggering time by checking whether the difference between the predicted state and the current state is significant or not. However, when the system matrices are unknown, the challenge lies in characterizing the distance between future states and the current state using only data. To address this, we put forth a data-driven online optimization problem for trajectory prediction by using noisy input-state data. Its optimal solution, together with another unknown parameter that reflects the open-loop divergence rate, is shown sufficient for explicitly quantifying the distance. Moreover, a data-driven set-based over-approximating algorithm using matrix zonotopes is subsequently proposed to upper-bound the {open-loop} divergence rate. Leveraging the optimal solution and the upper bound, a self-triggering mechanism is devised for state feedback control systems, which is proven to ensure input-to-state stability. Numerical examples are presented to validate the effectiveness of the proposed method.

Keywords: Learning, Optimization, Sampled-data control
Abstract: Policy optimization (PO), an essential approach of reinforcement learning for a broad range of system classes, requires significantly more system data than indirect (identification-followed-by-control) methods or behavioral-based direct methods even in the simplest linear quadratic regulator (LQR) problem. In this paper, we take an initial step towards bridging this gap by proposing the data-enabled policy optimization (DeePO) method, which requires only a finite number of sufficiently exciting data to iteratively solve the LQR problem via PO. Based on a data-driven closed-loop parameterization, we are able to directly compute the policy gradient from a batch of persistently exciting data. Next, we show that the nonconvex PO problem satisfies a projected gradient dominance property by relating it to an equivalent convex program, leading to the global convergence of DeePO. Moreover, we apply regularization methods to enhance the certainty-equivalence and robustness of the resulting controller and show an implicit regularization property. Finally, we perform simulations to validate our results.

Keywords: Data driven control, Observers for Linear systems, Vision-based control
Abstract: We study the problem of representation learning for control from partial and potentially high-dimensional observations. We approach this problem via direct latent model learning, where one directly learns a dynamical model in some latent state space by predicting costs. In particular, we establish finite-sample guarantees of finding a near-optimal representation function and a near-optimal controller using the directly learned latent model for infinite-horizon time-invariant Linear Quadratic Gaussian (LQG) control. A part of our approach to latent model learning closely resembles MuZero, a recent breakthrough in empirical reinforcement learning, in that it learns latent dynamics implicitly by predicting cumulative costs. A key technical contribution of this work is to prove persistency of excitation for a new stochastic process that arises from our analysis of quadratic regression in our approach.

Keywords: Learning, Optimization algorithms, Adaptive systems
Abstract: Classical paradigms for distributed learning, such as federated or decentralized gradient descent, employ consensus mechanisms to enforce homogeneity among agents. While these strategies have proven effective in i.i.d. scenarios, they can result in significant performance degradation when agents follow heterogeneous objectives or data. Distributed strategies for multitask learning, on the other hand, induce relationships between agents in a more nuanced manner, and encourage collaboration without enforcing consensus. We develop a generalization of the exact diffusion algorithm for subspace constrained multitask learning over networks, and derive an accurate expression for its mean-squared deviation when utilizing noisy gradient approximations. We verify numerically the accuracy of the predicted performance expressions, as well as the improved performance of the proposed approach over alternatives based on approximate projections.

Keywords: Distributed parameter systems
Abstract: This paper presents an entirely different approach for the synchronization of identical networked infinite dimensional systems. With a large class of infinite dimensional systems representing partial differential equations (PDEs), the concept of a functional form of the consensus protocol used for synchronization is applied here and incorporates spatial derivatives and spatial averages of the differences of the PDE states. This leads to spatial PD-type of consensus protocols for synchronization of PDEs. When the networked PDEs are tasked with following a leader, also described by a PDE of the same type, an added component of the controller is incorporated to ensure leader following. The proposed PD-coupling in the synchronization control of infinite dimensional systems attains a new form for the finite dimensional case, where now a temporal PID coupling in the consensus protocol is implemented. Simulation studies for both the infinite and the finite dimensional cases are included to demonstrate the effects of the non-traditional coupling in the synchronization control of networked systems.

Keywords: Distributed parameter systems, Cooperative control, Traffic control
Abstract: This paper investigates the consensus problem of networked hyperbolic partial differential equation (PDE) systems for reaching an agreement over the whole spatial domain via event-triggered boundary feedback control. Consensus controllers are proposed for each PDE system based on the boundary information of its neighboring systems, where both centralized and distributed event-triggered strategies are designed in order to reduce the controller updating frequency. By employing the Lyapunov technique, sufficient conditions with respect to system matrices, event-triggered conditions and the undirected communication topology are obtained to ensure consensus of the networked systems, and it is proved that the Zeno behavior can also be avoided. Finally, the consensus control of a three-lane freeway traffic flow system modeled by Aw-Rascle-Zhang Equations is given as an application example, and the numerical simulation is carried out to validate the theoretical results.

Keywords: Backstepping, Distributed parameter systems
Abstract: This paper presents a backstepping control design method of stabilization unstable 1D reaction-diffusion system, where the input is a 1D function on an edge of a rectangle, the distal system is a 1D reaction-diffusion PDE on the opposite edge of the rectangle, and the actuator dynamics in between are a 2D heat PDE on the rectangle between the opposite edges. A novel invertible integral transformation is introduced and the resulting controller with feedback of both PDEs' states (the distal 1D and the interior 2D states). We define a new Lyapunov function that contains cosine coefficients to prove the exponential stability in H^2 norm of the closed-loop system. Finally, the theoretical result is illustrated by simulations on a numerical example.

Keywords: Distributed parameter systems, Stability of linear systems, Network analysis and control
Abstract: In this work, we study the exponential stability of a system of linear Korteweg-de Vries (KdV) equations interconnected through the boundary conditions on a star-shaped network structure. On each branch of the network, we define a linear KdV equation defined on a bounded domain (0,ℓj) or the half-line (0,∞). We start by proving well-posedness using semigroup theory and then some hidden regularity results. Then, we state the exponential stability of the linear KdV equation by acting with a damping term on not all the branches. This is proved by using compactness argument deriving a suitable observability inequality.

Keywords: Reduced order modeling, Computational methods, Fluid flow systems
Abstract: In this paper, we propose a new pressure-stabilized proper orthogonal decomposition reduced order model (POD- ROM) for the control of viscous incompressible flows. It is a velocity-pressure ROM that uses pressure modes as well to compute the reduced order pressure needed for instance in the control drag and lift forces on bodies in the flow. We also propose and analyze a decoupled time-stepping scheme that uncouples the computation of velocity and pressure variables. It allows us at each time step to solve linear problems, uncoupled in pressure and velocity, which can greatly improve computational efficiency. Numerical studies are performed to discuss the accuracy and performance of the new pressure- stabilized ROM in the simulation of control of flow past a forward-facing step channel

Keywords: Estimation, Distributed parameter systems, Healthcare and medical systems
Abstract: Uncertainty caused by parameter randomness in a system modeling the relationship between alcohol concentration level in the blood (BAC) or breath (BrAC) and transdermal alcohol concentration (TAC) measured on the surface of the skin by a wearable, non-invasive, electro-chemical biosensor is considered. The parameter-dependent impulse response function (IRF) and system output in the form of a convolution are expressed in terms of an analytic semigroup of operators with regularly dissipative generator set in a Gelfand triple of Hilbert spaces. The Fr'echet derivative of the analytic semigroup in this setting is used to study the variation in the response function resulting from the uncertainties in the parameters described by probability distributions whose statistics depend on regression models with covariates as predictor variables. Finite dimensional approximation of the infinite dimensional state space system and the multi-variate delta method for nonlinear functions of random vectors with asymptotically normal distributions are used to obtain approximating uniform confidence bands for the IRF and the TAC output signal. Convergence of the approximations and three different techniques for obtaining the confidence bands are analyzed and compared.

Keywords: Identification, Estimation
Abstract: This paper deals with the identification of graphical autoregressive moving-average (ARMA) models with latent variables. Combining sparse structural characteristics of the graphical model with low-rank modeling of the latent variables, a sparse plus low-rank based iterative identification algorithm is proposed. The topological information embedded in the sparse AR dynamics is estimated from a regularized Yule-Walker optimization problem, which is then treated as prior graphical structure constraint. The latent-variable plus MA part is identified by solving a convex constrained trace norm minimization problem. Based on the MA part estimate and the structural constraint, the graphical AR estimates are updated by the sparse plus low-rank optimization framework and are then used for the update of the latent-variable plus MA part. The effectiveness of the proposed method is illustrated through a simulation study.

Keywords: Identification, Linear systems
Abstract: Hyper-parameter estimation is one of the fundamental issues for kernel-based regularized system identification methods. Empirical Bayes (EB) estimator and Stein's unbiased risk estimator (SURE) are two popular hyper-parameter estimators, but they both have advantages and disadvantages. Specifically, EB is not asymptotically optimal in the mean squared error (MSE) sense but SURE is, while SURE is more sensitive to ill-conditioned regression matrices but EB is more robust. In this paper, to find a better estimator by combining their strength and mitigating their weakness, we propose a family of hyper-parameter estimators by linking EB and SURE estimators together through an index. The finite sample and asymptotic properties of this family of estimators have been established. The Monte Carlo simulation results show that there does exist a ``middle" hyper-parameter estimator in this family that is superior to the EB and SURE.

Keywords: Nonlinear systems identification
Abstract: Let the observation be generated by the nonlinear ARX (NARX) systems. A new method for variable selection of the nonlinear function within the system at any interested points is introduced. In contrast to most of the existing results, the new method is not based on optimizing a certain criterion, and estimates from the variable selection algorithm given in this paper are easy to update computationally in comparison with the criterion-optimization-based methods when new data arrive. Under reasonable conditions the estimates are proved to converge to the true contributing variables with probability one.

Keywords: Identification, Estimation, Computational methods
Abstract: When applying the Bayesian manifold regularization method to function estimation problem with manifold constraints, the direct implementation has computational complexity mathcal{O}(N^3), where N is the number of input-output data measurements. This becomes particularly costly when N is large. In this paper, we propose a more efficient implementation based on the Kalman filter and smoother using a state-space model realization of the underlying Gaussian process. Moreover, we explore the sequentially semi-separable structure of the Laplacian matrix and the posterior covariance matrix. Our proposed implementation has computational complexity mathcal{O}(N) and thus can be applied to large data problems. We exemplify the effectiveness of our proposed implementation through numerical simulations.

Keywords: Machine learning, Agents-based systems, Traffic control
Abstract: This paper considers the hub-based platoon coordination problem in the large-scale transportation network, to promote cooperation among trucks and optimize the overall efficiency of the transportation network. We design a distributed communication model for transportation networks and transform the problem into a Dec-POMDP (Decentralized-Partial Observable Markov Decision Process). We then propose the A-QMIX deep reinforcement learning algorithm to solve the problem, which adopts centralized training and distributed execution and hence provides a reliable model for trucks to make quick decisions using only partial information. Finally, we carry out experiments with 100 and 1000 trucks in the transportation network of the Yangtze River Delta region in China to demonstrate the effectiveness and real-time performance of the proposed algorithm.

Keywords: Network analysis and control, Stochastic systems, Control of networks
Abstract: The sparsity or compressibility of network spread states in a graph-spectrum basis is examined, in the context of a stochastic model for influence/spread known as the voter model. In particular, first- and second- moments are characterized, for the graph-spectrum basis components contained in the voter-model state. These formal characterizations, as well as an asymptotic analysis of the voter model, are used to relate compressibility with the network's graph. An illustrative example as well as a simulation of a larger-scale model are included.

Keywords: Optimal control, Switched systems, Numerical algorithms
Abstract: We propose two new optimistic planning algorithms for nonlinear hybrid-input systems, in which the input has both a continuous and a discrete component, and the discrete component must respect a dwell-time constraint. Both algorithms select sets of input sequences for refinement at each step, along with a continuous or discrete step to refine (split). The dwell-time constraint means that the discrete splits must keep the discrete mode constant if the required dwell-time is not yet reached. Convergence rate guarantees are provided for both algorithms, which show the dependency between the near-optimality of the sequence returned and the computational budget. The rates depend on a novel complexity measure of the dwell-time constrained problem. We present simulation results for two problems, an adaptive-quantization networked control system and a model for the COVID pandemic.

Keywords: Optimal control, Switched systems, Optimization
Abstract: Control problems with Finite-State Machines (FSM) are often solved using integer variables, leading to a mixed-integer optimal control problem (MIOCP). This paper proposes an alternative method to describe a subclass of FSMs using complementarity constraints and time-freezing. The FSM from this subclass is built up by a sequence of states where a transition between the states is triggered by a single switching function. This can be looked at as a cascade of hysteresis loops where a memory effect is used to maintain the active state of the state machine. Based on the reformulation for hybrid systems with a hysteresis loop, a method is developed to reformulate this subclass in a similar fashion. The approach transforms the original problem into a Piecewise Smooth System (PSS), which can be discretized using the recently developed Finite Elements with Switch Detection, allowing for high-accuracy solutions. The reformulation is compared to a mixed-integer formulation from the literature on a time-optimal control problem. This work is a first step towards the general reformulation of FSMs into nonsmooth systems without integer states.

Keywords: Optimal control, Uncertain systems, Aerospace
Abstract: We consider the problem of robust trajectory optimization of constrained discrete-time linear systems under bounded uncertainty. This paper extends our previous work on robust trajectory planning for linear systems with control-dependent uncertainties to a more general class of uncertainties, including nominal-state-dependent uncertainties. In particular, we show that if strong duality holds for the robust state constraints, and if the uncertainties are bounded by convex elementwise nonnegative functions of the nominal state and control, the robust constraints can be equivalently reformulated as deterministic convex constraints, enabling globally optimal solutions with no conservatism. We first use convex duality theory to reformulate robust linear inequality state constraints as deterministic biconvex constraints . We then exploit the elementwise nonnegativity of the uncertainty bounds to remove the biconvexity in closed form, resulting in convex deterministic constraints that can be handled by off-the-shelf solvers. These two lemmas are our main contribution, and lead to our final result, and equivalent convex reformulation of the original robust optimization problem which allows efficient trajectory optimization solutions under control and state dependent uncertainties. We then demonstrate the practical applicability of our method via numerical simulations.

Keywords: Optimal control
Abstract: We consider an optimal control problem on infinite horizon with a vanishing discounting factor, state sufficient conditions of optimality and illustrate them with examples.

Keywords: Optimal control, Iterative learning control, Linear systems
Abstract: This paper presents a model-free, real-time, data-efficient Q-learning-based algorithm to solve the H_{infty} control of linear discrete-time systems. The computational complexity is shown to reduce from mathcal{O}(underline{q}^3) in the literature to mathcal{O}(underline{q}^2) in the proposed algorithm, where underline{q} is quadratic in the sum of the size of state variables, control inputs, and disturbance. An adaptive optimal controller is designed and the parameters of the action and critic networks are learned online without the knowledge of the system dynamics, making the proposed algorithm completely model-free. Also, a sufficient probing noise is only needed in the first iteration and does not affect the proposed algorithm. With no need for an initial stabilizing policy, the algorithm converges to the closed-form solution obtained by solving the Riccati equation. A simulation study is performed by applying the proposed algorithm to real-time control of an autonomous mobility-on-demand (AMoD) system for a real-world case study to evaluate the effectiveness of the proposed algorithm.

Keywords: Optimal control, Game theory, Linear systems
Abstract: Open-loop and feedback potential differential games for multi-agent systems are considered in this paper. Constructive sufficient conditions under which a linear quadratic differential game constitutes a potential differential game are provided. The conditions enable the construction of associated optimal control problems that yield (at significantly reduced computational complexity) solutions (in terms of open-loop and feedback Nash equilibrium strategies) of the original differential game. The results are demonstrated on a practically-motivated example that concerns spacecraft formation control.

Keywords: Optimization, Markov processes, Robotics
Abstract: Markov chains have been increasingly used to define persistent robotic surveillance schemes. Motivations for this design choice include their easy implementation, unpredictable surveillance patterns, and their well-studied mathematical background. However, applying previous results to scenarios with multiple agents can significantly increase the dimension of the problem, leading to intractable algorithms. In this work we analyze the hitting time minimization problem for multiple agents moving over a finite graph. We exploit the structure of this problem to propose a tractable algorithm to design Markov chains to cover the graph with multiple interacting agents. Using mathematical analysis, we provide guarantees for the convergence of our proposed solution. Also, through numerical simulations, we show the performance of our approach compared to the current state of art in multi-agent scenarios.

Keywords: Optimization, Numerical algorithms, Optimal control
Abstract: We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is O(n) for each termination check assuming a bounded problem domain. We show that this domain restriction can be relaxed for problems whose data satisfies a simple rank condition, in which case each check requires an O(n^2) solve using a linear system that must be factored only once at the root node. We further show how this approach can be used in hybrid model predictive control as long as system inputs are bounded. Numerical results show that our method leads to a moderate reduction in the total iterations required for branch-and-bound conic solvers with interior-point based subsolvers.

Keywords: Optimization, Numerical algorithms, Uncertain systems
Abstract: Chance-constrained optimization problems are generally complicated by uncertainty, nonsmoothness, and nonconvexity. Yet, non-asymptoic rates and complexity guarantees for computing an epsilon-global minimizer remain open questions. We consider a subclass of problems in which the probability is defined as mathbb{P}left{ zeta mid zeta in K(x) right}, K(x) is a set defined as K(x) , = , { zeta in Kscr , mid , c(x,zeta) leq 1} where c(x,bullet) is a positively homogenous function for any x in Xscr and Kscr is a nonempty, compact, and convex set, symmetric about the origin. We make two contributions in this context. (i) First, when zeta admits a log-concave density on Kscr, the chance-constrained formulation admits an equivalent convex representation and under a suitable regularity condition, the necessary and sufficient conditions are captured by a monotone inclusion with a compositional expectation-valed operator. (ii) Second, when zeta admits a uniform density, we present a variance-reduced stochastic proximal-point framework for resolving the monotone inclusion and provide amongst the first rate and complexity guarantees for such a problem.

Keywords: Optimization, Optimal control, Stochastic optimal control
Abstract: Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a primal-dual convex optimization method that can be tailored to the geometry of the optimization problem at hand through the choice of a strongly convex potential function. It arises as a basic primitive in a variety of applications, including large-scale optimization, machine learning, and control. This paper proposes a variational formulation of mirror descent and of its stochastic variant, mirror Langevin dynamics. The main idea, inspired by the classic work of Brezis and Ekeland on variational principles for gradient flows, is to show that mirror descent emerges as a closed-loop solution for a certain optimal control problem, and the Bellman value function is given by the Bregman divergence between the initial condition and the global minimizer of the objective function.

Keywords: Optimization, Optimal control, Learning
Abstract: Over the past few years, differentiable optimization has gained interest within machine learning, control, and robotics communities. It consists in computing the derivatives of the solutions of a given optimization problem which can then be used in learning algorithms. Until now, dedicated approaches have been proposed to compute the derivatives of various optimization problems (LPs, QPs, SOCPs, etc.). However, these approaches assume the problems are well-conditioned, limiting textit{de facto} their application to general optimal control problems (OCP) widely used in robotics. In this work, we focus on the differentiation of such problems. We notably introduce a differentiable proximal formulation of equality-constrained LQR problems that accurately solves rank-deficient problems. This proximal formulation allows us to compute accurate gradients even in the case of problems that do not satisfy the standard linear independence constraint qualification (LICQ). Because any optimal control problem can be cast as an equality-constrained LQR problem in the vicinity of the optimal solution, we show that our robust LQR derivatives computation can be exploited to obtain the derivatives of general optimal control problems. We demonstrate the effectiveness of our approach in dynamics learning and parameter identification experiments in both linear and nonlinear optimal control problems.

Keywords: Optimization, Optimization algorithms, Agents-based systems
Abstract: In this paper, we study unconstrained distributed optimization strongly convex problems, in which the exchange of information in the network is captured by a directed graph topology over digital channels that have limited capacity (and hence information should be quantized). Distributed methods in which nodes use quantized communication yield a solution at the proximity of the optimal solution, hence reaching an error floor that depends on the quantization level used; the finer the quantization the lower the error floor. However, it is not possible to determine in advance the optimal quantization level that ensures specific performance guarantees (such as achieving an error floor below a predefined threshold). Choosing a very small quantization level that would guarantee the desired performance, requires information packets of very large size, which is not desirable (could increase the probability of packet losses, increase delays, etc) and often not feasible due to the limited capacity of the channels available. In order to obtain a communication-efficient distributed solution and a sufficiently close proximity to the optimal solution, we propose a quantized distributed optimization algorithm that converges in a finite number of steps and is able to adjust the quantization level accordingly. The proposed solution uses a finite-time distributed optimization protocol to find a solution to the problem for a given quantization level in a finite number of steps and keeps refining the quantization level until the difference in the solution between two successive solutions with different quantization levels is below a certain pre-specified threshold. Therefore, the proposed algorithm progressively refines the quantization level, thus eventually achieving low error floor with a reduced communication burden. The performance gains of the proposed algorithm are demonstrated via illustrative examples.

Keywords: Neural networks, Predictive control for linear systems, Control applications
Abstract: The internal combustion engine faces a severe energy conservation and emission reduction challenge. In this regard, low-temperature combustion technology is a profitable solution, allowing for pollutant emission reduction while improving engine efficiency. However, the process is complex, and the cycles are mutually coupled, making it a huge challenge to stabilize the entire process behavior. Also, the model mismatch and the inherent stochasticity of the process bring considerable difficulties to the application of control technology. In this work, we propose a deep learning-based generative model to learn the distribution of system uncertainties. The uncertainty information is considered in the model predictive control (MPC) strategy design. We adopt the disturbance-affine stochastic MPC (sMPC) and transform the chance-constrained MPC problem into some tractable optimization problems. The results show better closed-loop performance with smaller output variance, given the prior knowledge of uncertainty realization from the proposed generative model.

Keywords: Neural networks, Predictive control for nonlinear systems, Linear parameter-varying systems
Abstract: Artificial neural networks (ANN) have been shown to be flexible and effective function estimators for the identification of nonlinear state-space models. However, if the resulting models are used directly for nonlinear model predictive control (NMPC), the resulting nonlinear optimization problem is often overly complex due to the size of the network, requires the use of high-order observers to track the states of the ANN model, and the overall control scheme does not exploit the available autograd tools for these models. In this paper, we propose an efficient approach to auto-convert ANN state-space models to linear parameter-varying (LPV) form and solve predictive control problems by successive solutions of linear model predictive problems, corresponding to quadratic programs (QPs). Furthermore, we show how existing deep-learning methods, such as SUBNET that uses a state encoder, enable efficient implementation of MPCs on identified ANN models. Performance of the proposed approach is demonstrated by a simulation study on an unbalanced disc system.

Keywords: Neural networks, Robotics, Lyapunov methods
Abstract: In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. This paper develops an analytic end-to-end learning algorithm for deep fully connected neural networks(FNN). Unlike existing end-to-end deep learning methods, the convergence analysis of the proposed method is assured based on smooth sigmoid activation functions and thus avoiding any potential chattering problem, but the proposed method does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm.

Keywords: Neural networks, Robust control, Optimization
Abstract: The reliable deployment of neural networks in control systems requires rigorous robustness guarantees. In this paper, we obtain tight robustness certificates over convex attack sets for min-max representations of ReLU neural networks by developing a convex reformulation of the nonconvex certification problem. This is done by "lifting" the problem to an infinite-dimensional optimization over probability measures, leveraging recent results in distributionally robust optimization to solve for an optimal discrete distribution, and proving that solutions of the original nonconvex problem are generated by the discrete distribution under mild boundedness, nonredundancy, and Slater conditions. As a consequence, optimal (worst-case) attacks against the model may be solved for exactly. This contrasts prior state-of-the-art that either requires expensive branch-and-bound schemes or loose relaxation techniques. Experiments on robust control and MNIST image classification examples highlight the benefits of our approach.

Keywords: Neural networks, Robust control, Uncertain systems
Abstract: With the increase in data availability, it has been widely demonstrated that neural networks (NN) can capture complex system dynamics precisely in a data-driven manner. However, the architectural complexity and nonlinearity of the NNs make it challenging to synthesize a provably safe controller. In this work, we propose a novel safety filter that relies on convex optimization to ensure safety for a NN system, subject to additive disturbances that are capable of capturing modeling errors. Our approach leverages tools from NN verification to over-approximate NN dynamics with a set of linear bounds, followed by an application of robust linear MPC to search for controllers that can guarantee robust constraint satisfaction. We demonstrate the efficacy of the proposed framework numerically on a nonlinear pendulum system.

Keywords: Neural networks, Stochastic systems, Nonlinear systems
Abstract: The problem of function approximation by neural dynamical systems has typically been approached in a top-down manner: Any continuous function can be approximated to an arbitrary accuracy by a sufficiently complex model with a given architecture. This can lead to high-complexity controls which are impractical in applications. In this paper, we take the opposite, constructive approach: We impose various structural restrictions on system dynamics and consequently characterize the class of functions that can be realized by such a system. The systems are implemented as a cascade interconnection of a neural stochastic differential equation (Neural SDE), a deterministic dynamical system, and a readout map. Both probabilistic and geometric (Lie-theoretic) methods are used to characterize the classes of functions realized by such systems.

Keywords: Large-scale systems, Control system architecture, Distributed control
Abstract: Complex networks pose significant challenges for design of control systems. This paper proposes a partitioning method for large-scale networks to be employed for use in decentralized and distributed control. The paper is divided into two parts. In the first part, a new formulation of the partitioning problem considering both computational and communication costs associated with control is established, and the controllability of the subsystems are also taken into account. In the second part, an effective algorithm is developed to find the solution to the network decomposition problem. The proposed approach is illustrated on a water distribution system.

Keywords: Large-scale systems, Model/Controller reduction, Numerical algorithms
Abstract: Nonlinear feedback design via state-dependent Riccati equations is well

established but unfeasible for large-scale systems because of computational costs. If the system can be embedded in the class of linear parameter-varying (LPV) systems with the parameter dependency being affine-linear, then the nonlinear feedback law has a series expansion with constant and precomputable coefficients. In this work, we propose a general method to approximating nonlinear systems such that the series expansion is possible and efficient even for high-dimensional systems. We lay out the application for the stabilization of incompressible Navier-Stokes equations, discuss the numerical solution of the involved matrix valued equations, and confirm the performance of the approach in a numerical example.

Keywords: Large-scale systems, Robotics, Autonomous robots
Abstract: This paper presents a solution to lifelong Multi-Agent Path Finding (MAPF) problems for long and narrow-corridor environments. In this setting, robots need to navigate conflict-free paths while adapting to new goals. We propose an algorithm called Conflict-Free Node-To-Robot Scheduling (CFNRS), which effectively coordinates the paths of robots on a given graph in a constrained environment. The algorithm assigns nodes of the graph, ensuring no conflicts with other robots. In particular, we introduce a Deadlock-Detection and Resolution mechanism to find and resolve conflicts and ensure conflict-free paths. We have introduced a problem-reduction technique for improved efficiency. The proposed algorithms are evaluated through simulations in narrow-corridor environments and compared to existing state-of-the-art MAPF solvers, demonstrating their validity and effectiveness in ensuring that robots can navigate conflict-free paths.

Keywords: Large-scale systems, Stability of nonlinear systems, Autonomous systems
Abstract: Geometric pattern formation is an important emergent behavior in many applications involving large-scale multi-agent systems, such as sensor networks deployment and collective transportation. Attraction/repulsion virtual forces are the most common control approach to achieve such behavior in a distributed and scalable manner. Nevertheless, for most existing solutions only numerical and/or experimental evidence of their convergence is available. Here, we revisit the problem of achieving pattern formation in spaces of any dimension, giving sufficient conditions to prove analytically that under the influence of appropriate virtual forces, a large-scale multi-agent swarming system locally converges towards a stable and robust rigid lattice configuration. Our theoretical results are complemented by exhaustive numerical simulations confirming their effectiveness and estimating the region of asymptotic stability of the rigid lattice configuration.

Keywords: Large-scale systems, Stochastic systems, Lyapunov methods
Abstract: In this paper, we address the problem of steering the distribution of oscillators all receiving the same control input to a given desired distribution. In a large population limit, the distribution of oscillators can be described by a probability density. Then, our problem can be seen as an ensemble control problem with a constraint on the steady-state density. In particular, we consider the case where oscillators are subjected to stochastic noise. One of the difficulties of this problem is that due to the stochasticity, it is generally impossible to design a control law under which oscillators converge to a target density exactly. To avoid this issue, we first give an alternative target density that is close enough to the original target. The modified target is carefully designed via a periodic input so that the distribution of oscillators can converge to it by an appropriate control strategy. Next, we construct a controller that decreases the Kullback–Leibler divergence between the distribution of oscillators and the modified target combining a periodic input and feedback control. We exhibit some convergence results for our proposed method. The effectiveness of the proposed method is demonstrated by a numerical example.

Keywords: Optimal control, Large-scale systems, Predictive control for linear systems
Abstract: This paper investigates dynamical optimal transport of underactuated linear systems over an infinite time horizon. In our previous work, we proposed to integrate model predictive control and the celebrated Sinkhorn algorithm to perform efficient dynamical transport of agents. However, the proposed method requires the invertibility of input matrices, which severely limits its applicability. To resolve this issue, we extend the method to (possibly underactuated) controllable linear systems. In addition, we ensure the convergence properties of the method for general controllable linear systems. The effectiveness of the proposed method is demonstrated by a numerical example.

Keywords: Distributed control, Agents-based systems, Autonomous robots
Abstract: This paper aims to design a distributed algorithm based on Doppler effect that allows multiple robots to achieve a consensus on their velocities. Instead of relying on a direct measurement of robots’ exact or relative velocities, which are usually challenging under denied environments (i.e., underwater), the novelty of our approach stems from utilizing the sound frequency as a medium to coordinate velocities among robots. Such a mechanism is achieved by exploiting the Doppler effect and establishing an equivalence between the velocity consensus of the robots and the frequency consensus of the sound they broadcast/receive. To address scalability and interference issues for large-scale systems, our use of the Doppler effect is broadcast-based, unlike most traditional Doppler devices that are reflection-based. Building on this, we develop a fully distributed algorithm for multi-robot flocking for the one-dimensional case, where the only control input for each robot is the sound frequencies it locally receives. The designed controller leads to highly nonlinear system dynamics. We employ a linearization method to theoretically prove the local convergence of the system to the desired equilibrium for velocity consensus. Simulations demonstrate the effectiveness of the proposed approach.

Keywords: Distributed control, Constrained control, Output regulation
Abstract: In this paper, we consider the problem of output bounded controller design for a parallel-flow heat exchanger described by 2times 2 coupled linear hyperbolic partial differential equations (PDEs) of balance laws. We aim to drive the internal fluid outlet temperature to track a reference trajectory by manipulating the external fluid velocity. Due to physical limitations, this manipulated variable has to be bounded to avoid a laminar regime. Consequently, the control problem becomes bounded and bilinear. Based on the set-invariance concept and the Lyapunov's stability theory, first, we design a bounded state feedback controller. Then, since only boundary measurements are available, we synthesize an output feedback controller, and demonstrate the exponential stability of the closed-loop system. Finally, simulation results are provided to illustrate the performance of the proposed control technique.

Keywords: Distributed control, Constrained control, Robust control
Abstract: We present a set-theoretical characterization of a bound on the maximal portion that an agent can cede of its input variable to another agent. By ceding control authority, agents can decompose coupling variables into public and private parts, which is of interest in situations of partial cooperation. In particular, sufficient conditions under which the non-existence of the maximum robust control invariant set is guaranteed are provided, expressed in terms of support functions and the dominant system eigenvalue. Finally, the results are illustrated via stable and unstable example systems with different coupling.

Keywords: Distributed control, Control applications, Networked control systems
Abstract: In this paper, distributed optimal solutions are designed for networked multiagent pursuit-evasion (MPE) games for capture and formation control. In the games, the pursuers aim to minimize the distance from their target evaders while the evaders attempt to maximize it, and at the same time, all players desire to maintain cohesion with their teammates. The goals of agents are obviously reflected in the obtained optimal control strategies which consist of an attracting term and/or a repelling term. Nash equilibrium is obtained by means of optimal strategies using the solutions of the HJI equations. Furthermore, three scenarios are considered in the MPE game: one-pursuer-one-evader, multiple-pursuer-oneevader, and multiple-pursuer-multiple-evader, where sufficient conditions are given for pursuers in achieving capture or formation control with ultimate zero or bounded errors. It is shown that the conditions depend on the structure of the communication graph, the parameters in the controllers, and the expected formation configurations. Finally, both simulations and real flight experiments successfully demonstrate the effectiveness of the proposed strategies.

Keywords: Distributed control, Cooperative control, Networked control systems
Abstract: A multi-agent system is a system consisting of multiple agents that can make a global decision autonomously. So far, many studies have been conducted under the assumption that all agents behave cooperatively. On the other hand, in a large-scale multi-agent system such as a connected vehicle network, each agent must be owned by different owners. Thus, in such a system, the agents do not necessarily work cooperatively and each agent pursues its utility. Such a multi-agent system is modeled as a system driven by the exchange of certain values. In this paper, we address a consensus problem for a multi-agent system driven by the exchange between tokens and information. Unlike the typical consensus control, we assume that each agent has tokens and collects information from its neighbors in exchange for tokens. To this system, we derive a necessary and sufficient condition for the system to achieve consensus. This condition is characterized by the number of tokens and the network structure. Moreover, we disclose that the consensus value is given by the left eigenvector of the Perron matrix associated with the initial token distribution. Finally, we discuss the convergence property for several specific network structures.

Keywords: Distributed control, Formal Verification/Synthesis, Agents-based systems
Abstract: In this paper, we propose a computationally efficient symbolic controller synthesis technique for multi-agent systems. The paper focuses on synthesizing distributed controllers enforcing local temporal logic specifications along with global safety specifications for multi-agent systems. To solve the problem in a computationally efficient way, we leverage the concept of control barrier functions. In particular, we use a three-step bottom-up approach: first, the symbolic controllers for individual agents are synthesized to enforce local temporal logic specifications, then we use a notion of control barrier functions for symbolic models to compose controlled agent systems by removing unsafe transitions, and finally, we synthesize controller for the reduced composed system to ensure the satisfaction of local temporal logic specifications while ensuring global safety specification. The effectiveness of our approach is demonstrated on a multi-robot system by comparing it with the conventional monolithic symbolic control approaches.

Keywords: Networked control systems, Optimization algorithms, Agents-based systems
Abstract: This manuscript proposes novel distributed algorithms for solving the dynamic consensus problem in discrete-time multi-agent systems on three different objective functions: the average, the maximum, and the median. In this problem, each agent has access to an external time-varying scalar signal and aims to estimate and track a function of all the signals by exploiting only local communications with other agents. By recasting the problem as an online distributed optimization problem, the proposed algorithms are derived based on the distributed implementation of the alternating direction method of multipliers (ADMM) and are thus amenable to a unified analysis technique. A major contribution is that of proving linear convergence of these ADMM-based algorithms for the specific dynamic consensus problems of interest, for which current results could only guarantee sub-linear convergence. In particular, the tracking error is shown to converge within a bound, whereas the steady-state error is zero. Numerical simulations corroborate the theoretical findings, empirically show the robustness of the proposed algorithms to re-initialization errors, and compare their performance with that of state-of-the-art algorithms.

Keywords: Networked control systems, Sampled-data control, Cooperative control
Abstract: This work presents a novel approach for achieving state synchronization of homogeneous LTI agents to a trajectory generated by a prescribed reference generator under intermit- tent and asynchronous communication. The proposed protocol involves emulating “ideal” global analog dynamics at each agent to generate the control signal between samples. Each agent transmits the centroid state of its local emulator rather than its own state vector, which is used to update the emulators at the receiving end. The paper guarantees synchronization with a prescribed reference generator under mild assumptions on the system’s structure, persistency of connectivity, and uniform boundedness of sampling intervals. Additionally, the controller parameters are independent of the sampling interval, allowing it to be designed without any a priori knowledge of the sampling sequence. Lastly, a simplified and scalable implementation whose dimension is independent of the number of agents is also proposed.

Keywords: Networked control systems, Sensor networks, Filtering
Abstract: We consider the problem of optimally scheduling transmissions for remote estimation of a discrete-time autoregressive Markov process that is driven by white Gaussian noise. A sensor observes this process, and then decides to either encode the current state of this process into a data packet and attempts to transmit it to the estimator over an unreliable wireless channel modeled as a Gilbert-Elliott channel~cite{Gilbert1960capacity}~cite{Chakravorty2017structure}~cite{yao2022age}, or does not send any update. Each transmission attempt consumes lambda units of transmission power, and the remote estimator is assumed to be linear. The channel state is revealed only via the feedback (ACKslash NACK) of a transmission, and hence the channel state is not revealed if no transmission occurs. The goal of the scheduler is to minimize the expected value of an infinite-horizon cumulative discounted cost, in which the instantaneous cost is composed of the following two quantities: (i)~squared estimation error, (ii) transmission power. We posed this problem as a partially observable Markov decision process (POMDP), in which the scheduler maintains a belief about the current state of the channel, and makes decisions on the basis of the current value of the error e(t) (defined in~eqref{def:error_evolve}), and the belief state.~To aid its analysis, we introduce an easier-to-analyze ``folded POMDP.'' We then analyze this folded POMDP and show that there is an optimal scheduling policy that has threshold structure, i.e. for each value of the error e, there is a threshold bust(e) such that when the error is equal to e, this policy transmits only when the current belief state is greater than bust(e).

Keywords: Networked control systems, Stability of hybrid systems, Nonlinear systems
Abstract: This paper studies the stabilisation problem for a class of nonlinear systems with two time scales, where only a single communication channel is available to allocate both low and high-frequency transmissions from slow and fast subsystems, respectively. A clock mechanism is proposed to govern the transmissions, and the closed-loop system is modelled by a hybrid singularly perturbed system. Singular perturbation based analysis is used to obtain individual maximum allowable transmission intervals for both slow and fast transmissions, and also to guarantee semi-global practical asymptotic stability with respect to the minimum allowable transmission interval of slow transmissions. We illustrate the results via a numerical example.

Keywords: Networked control systems, Stochastic systems, Stability of hybrid systems
Abstract: Stability in networked control systems has been typically addressed in the deterministic setting using the concepts of Maximum Allowable Transmit Interval (MATI) and Maximum Allowable Delay (MAD). This work looks to extend the analysis to the stochastic setting by giving conditions for uniform stability in probability when the communication delay follows a probability distribution that includes values over the deterministic MAD. Analytical conditions are given and validated through simulations using a concrete example.

Keywords: Networked control systems
Abstract: This paper presents a Lyapunov function-based control strategy for networked control systems (NCS) affected by variable time delays and data loss. A special focus is put on the reduction of the computational complexity. The crucial step to achieving computational efficiency involves defining a specific buffering mechanism that introduces an additional delay not larger than one sampling period. This allows representing the buffered NCS as a switched system, thus leading to a significant simplification of the NCS model and subsequent controller synthesis. The novel approach does not only circumvent the need for any over-approximation technique, but also leads to a strongly decreased number of optimization variables and linear matrix inequalities, allowing hereby greater flexibility with respect to additional degrees of freedom affecting the transient behavior. The performance and computational efficiency of the control strategy are demonstrated in a simulation example.

Keywords: Observers for Linear systems, Distributed control, Autonomous systems
Abstract: A simply structured distributed estimator is described for estimating the state of a continuous-time, jointly observable, input free, multi-channel linear system whose sensed outputs are distributed across a fixed multi-agent network. The estimator is then extended to non-stationary networks whose neighbor graphs switch according to a switching signal with a dwell time, or switch arbitrarily under appropriate assumptions. The estimator is guaranteed to solve the problem, provided a network-widely shared gain is sufficiently large. The lower bound of the gain is derived. This is accomplished by appealing to the “split-spectrum” approach and exploiting several well-known properties of invariant subspace. The proposed estimators are inherently resilient to abrupt changes in the number of agents and communication links in the inter-agent communication graph upon which the algorithms depend, provided the network is redundantly strongly connected and redundantly jointly observable.

Keywords: Observers for Linear systems, Estimation
Abstract: This paper presents a recursive ellipsoidal set-membership state estimation algorithm for discrete-time linear time-varying (LTV) models with additive bounded disturbances affecting state evolution and sporadic measurement equations. The state vector is subject to linear equality and/or inequality constraints, which are mathematically viewed as additional measurements. A novel approach is developed considering the unprecedented fact that, owing to equality constraints, the ellipsoid characterizing all possible values of the state vector has a zero volume and its shape matrix is non invertible. A new size criterion, the pseudo-volume, is introduced and minimized in both the prediction and correction phases.

Keywords: Observers for Linear systems, LMIs, Uncertain systems
Abstract: This paper investigates interval observer design for uncertain discrete-time linear switched systems under unknown inputs and a known switching signal. The approach introduces weighting matrices which allow one to relax design difficulties caused by classical coordinate transformation. To improve the accuracy, an L_{infty} method minimizing the peak-to-peak gain is employed to reduce the influence of unknown uncertainties. The interval observer gains are computed by solving Linear Matrix Inequality (LMI) formulated based on multiple quadratic Lyapunov functions under average dwell time switching signals.

Keywords: Observers for Linear systems, Optimization algorithms, Information theory and control
Abstract: The study of theoretical conditions for recovering sparse signals from compressive measurements has received a lot of attention in the research community. In parallel, there has been a great amount of work characterizing conditions for the recovery of both the state and the input to a linear dynamical system (LDS), including a handful of results on recovering sparse inputs. However, existing sufficient conditions for recovering sparse inputs to an LDS are conservative and hard to interpret, while necessary and sufficient conditions have not yet appeared in the literature. In this work, we provide (1) the first characterization of necessary and sufficient conditions for the existence and uniqueness of sparse inputs to an LDS, (2) the first necessary and sufficient conditions for a linear program to recover both an unknown initial state and a sparse input, and (3) simple, interpretable recovery conditions in terms of the LDS parameters. We conclude with a numerical validation of these claims and discuss implications and future directions.

Keywords: Observers for Linear systems, Robust control, Uncertain systems
Abstract: For linear systems with uncertainties and external disturbances, we present an uncertainty estimation and composite control (UECC) to achieve reference tracking and uncertainty estimation simultaneously. In this proposed UECC, we extract the uncertainty information from the error dynamics equation instead of the system dynamics. By reformulating the error dynamics equation as an algebraic equation using auxiliary variables, the need to measure state derivatives in the estimator and controller design is eliminated. Unlike time-delay control (TDC) and uncertainty and disturbance estimator (UDE) methods, we avoid the use of time delay and additional filtering operations to circumvent the noise amplification and oscillations in the control signal. Comparative simulations are provided to verify the effectiveness of the proposed method.

Keywords: Robust control, Uncertain systems, Optimization
Abstract: This paper presents a tractable framework for data-driven synthesis of robustly safe control laws. Given noisy experimental data and some priors about the structure of the system, the goal is to synthesize a state feedback law such that the trajectories of the closed loop system are guaranteed to avoid an unsafe set even in the presence of unknown but bounded disturbances (process noise). The main result of the paper shows that for polynomial dynamics, this problem can be reduced to a tractable convex optimization by combining elements from polynomial optimization and the theorem of alternatives. This optimization provides both a rational control law and a density function safety certificate. These results are illustrated with numerical examples.

Keywords: Robust control, Uncertain systems, Stability of nonlinear systems
Abstract: In this paper, we address the problem of safe and robust stabilization for a class of uncertain nonlinear systems. The key idea is to employ the disturbance observer (DOB) to a nominal safety-critical controller designed for the control Lyapunov-barrier function (CLBF). The DOB estimates and compensates the lumped disturbance that represents all the effect of model uncertainty and disturbance to the system approximately but as accurately as possible. As a result, only a small perturbation remains in the control loop, which can be dealt with as long as the nominal closed-loop system is input-tostate safe (ISSf) in a sense. To ensure the ISSf property without restriction on the CLBF, we propose a modified version of the Sontag’s universal formula as a nominal controller. This preliminary study verifies the validity of the proposed approach for 2nd-order nonlinear systems, but with mathematical analysis and simulations for the inverted pendulum on a cart.

Keywords: Robust control, Vision-based control, Fault tolerant systems
Abstract: When we rely on deep-learned models for robotic perception, we must recognize that these models may behave unreliably on inputs dissimilar from the training data, compromising the closed-loop system's safety. This raises fundamental questions on how we can assess confidence in perception systems and to what extent we can take safety-preserving actions when external environmental changes degrade our perception model's performance. Therefore, we present a framework to certify the safety of a perception-enabled system deployed in novel contexts. To do so, we leverage robust model predictive control (MPC) to control the system using the perception estimates while maintaining the feasibility of a safety-preserving fallback plan that does not rely on the perception system. In addition, we calibrate a runtime monitor using recently proposed conformal prediction techniques to certifiably detect when the perception system degrades beyond the tolerance of the MPC controller, resulting in an end-to-end safety assurance. We show that this control framework and calibration technique allows us to certify the system's safety with orders of magnitudes fewer samples than required to retrain the perception network when we deploy in a novel context on a photo-realistic aircraft taxiing simulator. Furthermore, we illustrate the safety-preserving behavior of the MPC on simulated examples of a quadrotor. We open-source our simulation platform and provide videos of our results at our project page: https://tinyurl.com/fallback-safe-mpc.

Keywords: Process Control, Robust control, Optimal control
Abstract: This paper investigates a possibility to improve the control performance of a laboratory-scale heat exchanger by introducing the convex-lifting-based robust controller into the closed-loop system. Robust model predictive control (MPC) design serves as the relevant reference control strategy. The improvements are expected in both, increased performance of the control trajectories and, simultaneously, reduced computational complexity. The performance is analyzed subject to the reference tracking implemented on the laboratory plate heat exchanger. This plant has a nonlinear and asymmetric behavior, affected by various uncertain parameters. The case study investigates the robust MPC, tunable convex-lifting-based robust control, and convex-lifting-based robust control with approximated control law. This paper also extends the convex-lifting-based robust control with approximated control law to provide the robust stability guarantees. The experimental case study evaluates and analyses various apprehensible criteria, e.g., energy consumption, carbon footprint, and computational demands.

Keywords: Robust control, Stability of linear systems, LMIs
Abstract: This paper derives a new and unifying state-space characterisation for the entire class of real, rational, proper Output Negative Imaginary (ONI) systems, allowing poles on the imaginary axis even at the origin. The proposed result captures the existing versions of the NI state-space characterisations, particularly the ones that apply to the NI systems with poles at the origin. A necessary and sufficient LMI condition has been derived to test the strict/non-strict ONI properties of an LTI system with a given minimal state-space realisation. The LMI-based characterisation offers easy and convenient execution due to the easily accessible SDP solver packages. Finally, a necessary and sufficient internal stability theorem is also derived for a positive feedback ONI systems interconnection containing pole(s) at the origin. The proposed stability result specialises to the earlier versions when the earlier assumptions are imposed. Numerical examples are given to show the usefulness of the proposed theoretical results.

Keywords: Decentralized control, Robust control, Distributed control
Abstract: It is well-known that linear quadratic regulators (LQR) enjoy guaranteed stability margins, whereas linear quadratic Gaussian regulators (LQG) do not. In this letter, we consider systems and compensators defined over directed acyclic graphs. In particular, there are multiple decision-makers, each with access to a different part of the global state. In this setting, the optimal LQR compensator is dynamic, similar to classical LQG. We show that when sub-controller input costs are decoupled (but there is possible coupling between sub-controller state costs), the decentralized LQR compensator enjoys similar guaranteed stability margins to classical LQR. However, these guarantees disappear when cost coupling is introduced.

Keywords: Smart grid, Optimization, Neural networks
Abstract: Volt/VAR control rules enable distributed energy resources (DER) to autonomously regulate voltage in distribution grids. The Volt/VAR rules provisioned by the IEEE Standard 1547 take on a piecewise-linear shape. However, its maximum slope is upper bounded to ensure stability, and that may hamper its voltage regulation performance. This limitation can be surpassed by adding a memory term to the control rule, and thus, obtaining a so-termed incremental control rule. This letter aims to optimally customize the shape of incremental rules across buses to attain desirable voltage profiles. Albeit this task can be posed as a bilevel program, we pursue a more scalable approach by reformulating it as a deep learning task. The idea is that Volt/VAR dynamics can be captured by a recursive neural network (RNN). Interestingly, the RNN weights correspond to the parameters of the control rule; the RNN input to the grid loading conditions; and the RNN output to the equilibrium voltages. Therefore, the optimal rule parameters can be found upon training the RNN so its output (equilibrium voltages) approach unity. Training is performed by feeding the RNN with representative scenarios of the anticipated grid loading conditions. The RNN depth depends on the settling time of Volt/VAR dynamics. Because the discrete-time Volt/VAR dynamics can be viewed as iterations of a proximal gradient descent (PGD) algorithm, we also leverage Nesterov's accelerated PGD iterations to reduce the RNN depth. The RNN is never implemented in the field. Training this RNN is equivalent to solving the optimal rule design in a more computationally efficient manner. Analytical findings and numerical tests corroborate that the proposed solution can be neatly adapted to single- and multi-phase feeders. The proposed approach could be of general interest in designing piecewise-linear controllers acting on linear plants.

Keywords: Smart grid, Optimization algorithms, Control Systems Privacy
Abstract: The electric vehicle (EV) industry is rapidly evolving owing to advancements in smart grid technologies and charging control strategies. While EVs are promising in decarbonizing the transportation system and providing grid services, their widespread adoption has led to notable and erratic load injections that can disrupt the normal operation of power grid. Additionally, the unprotected collection and utilization of personal information during the EV charging process cause prevalent privacy issues. To address the scalability and data confidentiality in large-scale EV charging control, we propose a novel decentralized privacy-preserving EV charging control algorithm via state obfuscation that 1) is scalable w.r.t. the number of EVs and ensures optimal EV charging solutions; 2) achieves privacy preservation in the presence of honest-but-curious adversaries and eavesdroppers; and 3) is applicable to eliminate privacy concerns for general multi-agent optimization problems in large-scale cyber-physical systems. The EV charging control is structured as a constrained optimization problem with coupled objectives and constraints, then solved in a decentralized fashion. Privacy analyses and simulations demonstrate the efficiency and efficacy of the proposed approach.

Keywords: Smart grid, Optimization algorithms
Abstract: This paper investigates a multi-objective distributed resource allocation problem, where the economic cost including the transmission loss, and the environmental pollution are taken into account simultaneously. To settle this problem, a Pareto-based zeroth-order fast distributed optimization algorithm is proposed, which can always balance the overall energy demand with generation. In the algorithm design, the acceleration idea of the momentum method is tailored for the gradient estimation update, which gives a more accurate descent direction. Moreover, the unknown effect causes the gradient of the objective function to be unavailable and only the function values to be observed. Different from the gradient-based methods, a zeroth-order method is proposed to solve the distributed resource allocation problem with gradient estimation. Furthermore, the convergence of the designed algorithm is proved theoretically, and the convergence rate of linear speedup can be achieved. Finally, numerical simulations verify the validity and applicability of the proposed algorithm.

Keywords: Smart grid, Optimization algorithms, Energy systems
Abstract: With the growth of energy and transportation demand, the integrated energy dispatching of ship power grid has become the focus of researchers. The optimization technique is used to reduce the total energy consumption and pollutant emissions of ships, optimizing the ship power generation planning. The purpose is to achieve environmental protection and energy saving while ensuring the continuous and reliable power supply of ships. However, heterogeneous ship microgrid poses new challenges to integrated energy dispatch. This paper proposes an integrated energy scheduling scheme that integrates photovoltaic, wind power, diesel engine, gas turbine, and battery for a heterogeneous multienergy ship microgrid. Under the system constraints, a multi-objective optimal scheduling model including operating costs and pollutant emissions is established, then the gravity search algorithm is applied to solve such an issue. The simulation results show that the scheme can effectively reduce the cost of energy consumption and pollutant emissions of ships, improving the economy, reliability and energy conservation, which verify the advantages of the proposed scheme.

Keywords: Smart grid, Power systems, Energy systems
Abstract: Increasing penetrations of electric vehicles (EVs) presents a large source of flexibility, which can be used to assist balancing the power grid. The flexibility of an individual EV can be quantified as a convex polytope and the flexibility of a population of EVs is the Minkowski sum of these polytopes. In general computing the exact Minkowski sum is intractable. However, exploiting symmetry in a restricted but significant case, enables an efficient computation of the aggregate flexibility. This results in a polytope with exponentially many vertices and facets with respect to the time horizon. We show how to use a lifting procedure to provide a representation of this polytope with a reduced number of facets, which makes optimising over more tractable. Finally, a disaggregation procedure that takes an aggregate signal and computes dispatch instructions for each EV in the population is presented. The complexity of the algorithms presented is independent of the size of the population and polynomial in the length of the time horizon. We evaluate this work against existing methods in the literature, and show how this method guarantees optimality with lower computational burden than existing methods.

Keywords: Smart grid, Power systems, Optimization
Abstract: Meeting demand for automotive battery resources is predicted to be costly from both economic and environmental perspectives. To minimize costs, battery resources should be deployed as efficiently as possible. A potential source of inefficiency in battery deployment is that the batteries of personal vehicles are typically much larger than necessary to meet most daily mobility needs. In this paper, we consider whether battery resources can be used more efficiently in a setting where drivers, in addition to having personal vehicle batteries, have access to a shared battery resource. More precisely, we consider the problem of minimizing aggregate battery capacity in settings with and without a shared resource subject to the requirement that driver commuting needs are met with high reliability. To assess capacity reduction potential, we quantify the difference in deployed battery capacity in settings with and without a shared resource in a case study using real-world longitudinal mobility data from Puget Sound, Washington. We find that access to a shared battery resource can substantially reduce deployed battery capacity. Furthermore, relative reductions in battery capacity increase with number of drivers and the level of reliability desired.

Keywords: Statistical learning, Autonomous vehicles, Iterative learning control
Abstract: We investigate methods to provide safety assurances for autonomous agents that incorporate predictions of other, uncontrolled agents' behavior into their own trajectory planning. Given a learning-based forecasting model that predicts agents' trajectories, we introduce a method for providing probabilistic assurances on the model's prediction error with calibrated confidence intervals. Through quantile regression, conformal prediction, and reachability analysis, our method generates probabilistically safe and dynamically feasible prediction sets. We showcase their utility in certifying the safety of planning algorithms, both in simulations using actual autonomous driving data and in an experiment with Boeing vehicles.

Keywords: Statistical learning, Estimation
Abstract: The effectiveness of non-parametric, kernel-based methods for function estimation comes at the price of high computational complexity, which hinders their applicability in adaptive, model-based control. Motivated by approximation techniques based on sparse spectrum Gaussian processes, we focus on models given by regularized trigonometric linear regression. This paper provides an analysis of the performance of such an estimation set-up within the statistical learning framework. In particular, we derive a novel bound for the sample error in finite-dimensional spaces, accounting for noise with potentially unbounded support. Next, we study the approximation error and discuss the bias-variance trade-off as a function of the regularization parameter by combining the two bounds.

Keywords: Statistical learning, Distributed parameter systems, Adaptive systems
Abstract: We analyze the convergence of online regularized learning algorithm based on dependent and non-stationary online data streams for the nonparametric regression problem in reproducing kernel Hilbert space (RKHS). We show that the algorithm achieves mean-square convergence if the algorithm gain and regularization parameter are chosen appropriately, the online data streams are weakly dependent and satisfy the eigenvalue-wise persistence of excitation condition. Especially, for the case with independent but non-identically distributed online data streams, we give more intuitive convergence conditions on the drifts of the probability measures induced by the data.

Keywords: Statistical learning, Identification, Learning
Abstract: Modern data sets, such as those in healthcare and e-commerce, are often derived from many individuals or systems but have insufficient data from each source alone to separately estimate individual, often high-dimensional, model parameters. If there is shared structure among systems however, it may be possible to leverage data from other systems to help estimate individual parameters, which could otherwise be non-identifiable. In this paper, we assume systems share a latent low-dimensional parameter space and propose a method for recovering d-dimensional parameters for N different linear systems, when there are only T

Keywords: Statistical learning, Linear systems, Uncertain systems
Abstract: Inspired by the work of Tsiamis et al. [1], in this paper we study the statistical hardness of learning to stabilize linear time-invariant systems. Hardness is measured by the number of samples required to achieve a learning task with a given probability. The work in [1] shows that there exist system classes that are hard to learn to stabilize with the core reason being the hardness of identification. Here we present a class of systems that can be easy to identify, thanks to a non-degenerate noise process that excites all modes, but the sample complexity of stabilization still increases exponentially with the system dimension. We tie this result to the hardness of co-stabilizability for this class of systems using ideas from robust control.

Keywords: Statistical learning, Machine learning, Adaptive control
Abstract: In this paper, we consider learning and control problem in an unknown Markov jump linear system (MJLS) with perfect state observations. We first establish a generic upper bound on regret for any learning based algorithm. We then propose a certainty equivalence-based learning algorithm and show that this algorithm achieves a regret of O(sqrt(T)log(T)) relative to a certain subset of the sample space. As part of our analysis, we revisit the switched least squares system identification algorithm of [1], [2] for autonomous MJLS and generalize it to controlled MJLS, establishing strong consistency and almost sure rates of convergence of this method.

Keywords: Stability of nonlinear systems, Nonholonomic systems
Abstract: The problem of transforming a locally asymptotically stabilizing time-varying control law to a globally stabilizing one with accelerated finite/fixed-time convergence is studied. The solution is based on an extension of the theory of homogeneous systems to the setting where the symmetry and stability properties only hold with respect to a part of the state variables. The proposed control design advances the kind of approaches first studied in [M'Closkey&Murray,1997], and relies on the implicit Lyapunov function framework. Examples of finite-time and nearly fixed-time stabilization of a nonholonomic integrator are reported.

Keywords: Stability of nonlinear systems, Lyapunov methods, Neural networks
Abstract: The paper is about the computation of the principal spectrum of the Koopman operator (i.e., eigenvalues and eigenfunctions). The principal eigenfunctions of the Koopman operator are the ones with the corresponding eigenvalues equal to the eigenvalues of the linearization of the nonlinear system at an equilibrium point. The main contribution of this paper is to provide a novel approach for computing the principal eigenfunctions using a path-integral formula. Furthermore, we provide conditions based on the stability property of the dynamical system and the eigenvalues of the linearization towards computing the principal eigenfunction using the path-integral formula. Further, we provide a Deep Neural Network framework that utilizes our proposed path-integral approach for eigenfunction computation in high-dimension systems. Finally, we present simulation results for the computation of principal eigenfunction and demonstrate their application for determining the stable and unstable manifolds and constructing the Lyapunov function.

Keywords: Stability of nonlinear systems, Nonlinear systems, Lyapunov methods
Abstract: In this paper, the concept of control Leonov functions is introduced, and it is shown that their information is enough to design continuous and periodic controllers that provide boundedness of the state for a class of multistable state-periodic systems. These feedback control laws are based on a mild adaptation of Sontag's universal formula and a kind of small control property. The proposed method is illustrated via application in a microgrid.

Keywords: Stability of nonlinear systems, Lyapunov methods, Nonlinear systems
Abstract: We present novel sufficient conditions for the global stability of an equilibrium in the case of nonlinear dynamics with analytic vector fields. These conditions provide stability criteria that are directly expressed in terms of the Taylor expansion coefficients of the vector field (e.g. in terms of first order coefficients, maximal coefficient, sum of coefficients). Our main assumptions is that the vector field components be holomorphic, and the linearized system be locally exponentially stable and diagonalizable. These results are based on the properties of the Koopman operator defined on the Hardy space on the polydisc.

Keywords: Stability of nonlinear systems, Lyapunov methods
Abstract: The problem of global asymptotic stabilization by state feedback is considered for time-invariant bilinear non-homogeneous control systems in the complex space. For such systems, the possibility of applying the second Lyapunov method is proved, which is not valid for general nonlinear complex systems. The approach uses the Barbashin--Krasovsky--La Salle theorem on global asymptotic stability. Sufficient conditions for global asymptotic stabilization of a bilinear non-homogeneous complex system by real state feedback are obtained. Finally, an example of using the obtained results is presented.

Keywords: Stability of nonlinear systems, Numerical algorithms, Lyapunov methods
Abstract: Lyapunov's direct method is a powerful tool that provides a rigorous framework for stability analysis and control design for dynamical systems. A critical step that enables the application of the method is the existence of a Lyapunov function V---a function whose value monotonically decreases along the trajectories of the dynamical system. Unfortunately, finding a Lyapunov function is often tricky and requires ingenuity, domain knowledge, or significant computational power. At the core of this challenge is the fact that the method requires every sub-level set of V (V_{leq c}) to be forward invariant, thus implicitly coupling the geometry of V_{leq c} and the trajectories of the system. In this paper, we seek to disentangle this dependence by developing a direct method that substitutes the concept of invariance with the more flexible notion of recurrence. A set is (tau-)recurrent if every trajectory that starts in the set returns to it (within tau seconds). We show that, under mild conditions, the recurrence of sub-level sets V_{leq c} is sufficient to guarantee stability and introduce the appropriate stronger notions to obtain asymptotic stability and exponential stability. We further provide a GPU-based algorithm to verify whether V satisfies such recurrence conditions up to an arbitrarily small neighborhood of the equilibrium.

Keywords: Predictive control for linear systems, Optimal control, Subspace methods
Abstract: We introduce the notion of implicit predictors, which characterize the input-(state)-output prediction behavior underlying a predictive control scheme, even if it is not explicitly enforced as an equality constraint (as in traditional model or subspace predictive control). To demonstrate this concept, we derive and analyze implicit predictors for some basic data-driven predictive control (DPC) schemes, which offers a new perspective on this popular approach that may form the basis for modified DPC schemes and further theoretical insights.

Keywords: Predictive control for linear systems, Optimization, Model/Controller reduction
Abstract: Model predictive control presents remarkable po- tential for the optimal control of dynamic systems. However, the necessity for an online solution to an optimal control problem often renders it impractical for control systems with limited computational capabilities. To address this issue, specialized dimensionality reduction techniques designed for optimal con- trol problems have been proposed. In this paper, we introduce a methodology for designing a low-dimensional subspace that provides an ideal representation for a predefined finite set of high-dimensional optimizers. By characterizing the subspace as an element of a specific Riemannian manifold, we leverage the unique geometric structure of the subspace. Subsequently, the optimal subspace is identified through optimization on the Riemannian manifold. The dimensionality reduction for the model predictive control scheme is achieved by confining the search space to the optimized low-dimensional subspace, enhancing both efficiency and applicability.

Keywords: Predictive control for linear systems, Optimization, Optimal control
Abstract: Lipschitz constants for linear MPC are useful for certifying inherent robustness against unmodeled disturbances or robustness for neural network-based approximations of the control law. In both cases, knowing the minimum Lipschitz constant leads to less conservative certifications. Computing this minimum Lipschitz constant is trivial given the explicit MPC. However, the computation of the explicit MPC may be intractable for complex systems. The paper discusses a method for efficiently computing the minimum Lipschitz constant without using the explicit control law. The proposed method simplifies a recently presented mixed-integer linear program (MILP) that computes the minimum Lipschitz constant. The simplification is obtained by exploiting saturation and symmetries of the control law and irrelevant constraints of the optimal control problem.

Keywords: Predictive control for linear systems, Stochastic optimal control, Constrained control
Abstract: We investigate model predictive control (MPC) formulations for linear systems subject to i.i.d. stochastic disturbances with bounded support and chance constraints. Existing stochastic MPC formulations with closed-loop guarantees can be broadly classified in two separate frameworks: i) using robust techniques; ii) feasibility preserving algorithms. We investigate two particular MPC formulations representative for these two frameworks called robust-stochastic MPC and indirect feedback stochastic MPC. We provide a qualitative analysis, highlighting intrinsic limitations of both approaches in different edge cases. Then, we derive a unifying stochastic MPC framework that naturally includes these two formulations as limit cases. This qualitative analysis is complemented with numerical results, showcasing the advantages and limitations of each method.

Keywords: Predictive control for linear systems, Stochastic optimal control, Stochastic systems
Abstract: We present a Stochastic Model Predictive Control (SMPC) framework for linear systems subject to Gaussian disturbances. In order to avoid feasibility issues, we employ a recent initialization strategy, optimizing over an interpolation of the initial state between the current measurement and previous prediction. By also considering the variance in the interpolation, we can employ variable-size tubes, to ensure constraint satisfaction in closed-loop. We show that this novel method improves control performance and enables following the constraint closer, then previous methods. Using a DC-DC converter as numerical example we illustrated the improvement over previous methods.

Keywords: Predictive control for linear systems, Uncertain systems, Markov processes
Abstract: Autonomous systems operate in environments that can be observed only through noisy measurements. Thus, controllers should compute actions based on their beliefs about the surroundings. In these settings, we design a Model Predictive Controller (MPC) based on a continuous-state Linear Time-Invariant (LTI) system model operating in a discrete-state environment described by a Hidden Markov Model (HMM). Environment constraints are modeled as chance constraints and environment observations can be asynchronous with system state measurements and controller updates. We show how to approximate the solution of the MPC problem defined over the space of feedback policies by optimizing over a trajectory tree, where each branch is associated with an environment measurement. The proposed approach guarantees chance constraint satisfaction and recursive feasibility. Finally, we test the proposed strategy on navigation examples in partially observable environments, where the proposed MPC guarantees chance constraint satisfaction.

Keywords: Autonomous robots, Nonholonomic systems, Feedback linearization
Abstract: Differential drive robots that can be modeled as a kinematic unicycle are a standard mobile base platform for many service and logistics applications. Safe and smooth autonomous motion around obstacles is a crucial skill for unicycle robots to perform diverse tasks in complex environments. A classical control approach for unicycle control is feedback linearization using a headway point at a fixed headway distance in front of the unicycle. The unicycle headway control brings the headway point to a desired goal location by embedding a linear headway reference dynamics, which often results in an undesired offset for the actual unicycle position. In this paper, we introduce a new unicycle headway control approach with an adaptive headway distance that overcomes this limitation, i.e., when the headway point reaches the goal the unicycle position is also at the goal. By systematically analyzing the closed-loop unicycle motion under the adaptive headway controller, we design analytical feedback motion prediction methods that bound the closed-loop unicycle position trajectory and so can be effectively used for safety assessment and safe unicycle motion design around obstacles. We present an application of adaptive headway motion control and motion prediction for safe unicycle path following around obstacles in numerical simulations.

Keywords: Autonomous robots, Networked control systems, Optimization
Abstract: In a recent research program, we have undertaken the investigation of robotic traffic management problems arising when a fleet of networked mobile robots is employed in the support of certain coverage tasks that take place in physically constricted environments. But our past investigation of these problems is restricted to the case where the guidepath networks supporting the robot traffic have a dendritic topology. In the current work, we extend the investigation of the considered problems to the case where the underlying guidepath networks have an arbitrary topology. We provide (i) detailed descriptions of the considered problems in this new operational setting, (ii) analytical characterizations of these problems that take the form of integer programming formulations, and (iii) strong combinatorial relaxations for the derived formulations that are applicable to larger problem instances. A numerical experiment presented in the last part of the manuscript demonstrates and assesses the efficacy and the tractability of the analytical developments. We also notice that the undertaken extension of the past results is a nontrivial task, for the reasons that are explained in the manuscript.

Keywords: Autonomous robots, Agents-based systems
Abstract: In this paper, we consider the problem of a swarm traveling between two points as fast as possible in an unknown environment cluttered with obstacles. Potential applications include search-and-rescue operations where damaged environments are typical. We present swarm generalizations, called SwarmCom, SwarmBug1, and SwarmBug2, of the classical path generation algorithms Com, Bug1, and Bug2. These algorithms were developed for unknown environments and require low computational power and memory storage, thereby freeing up resources for other tasks. We show the upper bound of the worst-case travel time for the first agent in the swarm to reach the target point for SwarmBug1. For SwarmBug2, we show that the algorithm underperforms in terms of worst-case travel time compared to SwarmBug1. For SwarmCom, we show that there exists a trivial scene for which the algorithm will not halt, and it thus has no performance guarantees. Moreover, by comparing the upper bound of the travel time for SwarmBug1 with a universal lower bound for any path generation algorithm, it is shown that in the limit when the number of agents in the swarm approaches infinity, no other algorithm has strictly better worst-case performance than SwarmBug1 and the universal lower bound is tight.

Keywords: Autonomous robots, Optimal control, Distributed control
Abstract: This paper studies the motion coordination control problem for multiple mobile robots under a common workspace and reach-avoid tasks. Using abstraction-based techniques, we combine the offline and online control methods to propose a distributed motion coordination control strategy. In the offline control strategy, we partition the workspace to derive the graph for the offline planning, and construct the symbolic abstraction for each robot to design the individual controller offline. In the online control strategy, we provide a detection mechanism to check the existence of the potential robot collision, and implement the constructed symbolic abstraction and graph-searching techniques to resolve the robot collision. The combination of the offline and online control strategies results in the overall motion coordination control strategy for all robots.

Keywords: Autonomous robots, Sensor fusion, Kalman filtering
Abstract: In this paper, we propose a comprehensive solution for 3-D active target tracking with multiple robots in a fully distributed setting. Here multiple robots cooperatively estimate their own states and the target’s state and actively plan their motions to achieve better estimation of the target. For cooperative localization and target state estimation, each robot maintains a state vector consisting of its own state, the target’s state, and its own cloned history states. The challenge of localizing moving robots in 3-D is addressed by using multirobot cooperative visual-inertial odometry algorithm, which improves the estimation accuracy by using environmental common feature measurements. Each robot’s target measurement (if available) and its neighbors’ target estimators are then exploited for estimation updates. To preserve and update the correlations between the target and robot states while limiting the influence of bad target estimates on localization accuracy, the Schmidt-Kalman Filter framework is adopted. For motion planning, a gradient-based approach that uses differentiable field-of-view and potential functions is employed to achieve efficient and accurate active target tracking while avoiding collisions and maintaining communication connectivity. Numerous simulations show that our proposed algorithm provides an accurate and efficient solution for cooperative localization and active target tracking.

Keywords: Autonomous robots, Variable-structure/sliding-mode control, Constrained control
Abstract: This paper contributes to the design of a robust control strategy for the trajectory tracking problem in perturbed unicycle mobile robots. The proposed strategy comprises the design of a robust control law, which is based on an Integral Sliding–Mode Control (ISMC) approach together with an interval predictor–based state–feedback controller and a Model Predictive Control (MPC) scheme. The robust controller deals with some perturbations in the kinematic model, and with state and input constraints that are related to restrictions on the workspace and saturated actuators, respectively. The proposed approach guarantees the exponential convergence to zero of the tracking error. Furthermore, the performance of the proposed approach is validated through some simulations.

Keywords: Extremum seeking, Optimization
Abstract: We present a multivariable extremum seeking (ES) algorithm for static and dynamic maps that achieves unbiased convergence to the optimum exponentially, referred to as exponential ES. The conventional ES approach, which uses constant amplitude sinusoids, results in steady-state oscillations around the optimum and is unable to guarantee unbiased convergence. In contrast, our ES approach employs exponential decay and growth functions to gradually decrease the amplitude of the perturbation signal and increase the amplitude of the demodulation signal, respectively. This eliminates the steady-state oscillation. To achieve unbiased convergence, we choose an adaptation gain that is sufficiently larger than the decay rate of the perturbation so that the learning process outpaces the perturbation’s waning. The stability analysis is based on state transformation, averaging, and singular perturbation methods applied to the transformed system resulting in local stability of the transformed system as well as local exponential stability of the original system. For numerical simulation, we consider the problem of source seeking by a 2D velocity actuated point.

Keywords: Extremum seeking, Adaptive systems, Optimization
Abstract: The paper presents an extremum-seeking scheme in which the dither is adaptively tuned to deal with non-convex cost functions. The adaptation law decreases the dither when local cost function trends are easily visible from output data. Contrarily, when the cost function does not have a dominant trend, the dither is increased to enrich the output data. This adaptive scheme can give advantages in practical applications when a conservatively large dither implies unnecessary high energy to optimise a cost function corrupted by non-uniform state-dependent disturbances. Numerical comparisons confirm the superior performance of the proposed solution.

Keywords: Extremum seeking, Machine learning, Data driven control
Abstract: We present a novel type of sampled-data extremum-seeking control (ESC) aimed at speeding up convergence to the optimum and reducing the number of costly performance measurements in practical applications. The approach uses collected output measurements to construct online an approximation of the system’s steady-state performance function using kernel-based function approximation. In regions where this approximation is detected to be sufficiently accurate, the proposed approach utilizes it to determine the search direction and compute a suitable optimizer gain for the update step. In regions where the approximation is not yet accurate, additional data is collected and employed in a ‘standard’ ESC update step, while also using it to refine the approximation of the performance function. By using the approximation of the performance function to determine the search direction and optimizer gain when possible, the number of required performance measurements and parameter update steps can be significantly reduced, e.g., with respectively 75% and 45% in our simulation study involving a static cost function.

Keywords: Extremum seeking, Delay systems, Robust control
Abstract: In this article, we introduce a time-delay approach to gradient-based extremum seeking (ES) in the continuous domain for n-dimensional (nD) static quadratic maps. As in the recently introduced (for 2D maps in the continuous domain), we transform the system to the time-delay one (neutral type system). This system is O(varepsilon)-perturbation of the averaged linear ODE system. We further explicitly present the neutral system as the linear ODE, where O(varepsilon)-terms are considered as disturbances with distributed delays of the length of the small parameter varepsilon. Quantitative (for uncertain map) and qualitative (for unknown map) practical stability analyses are provided by employing a variation of constants formula that greatly simplifies the results compared to the previously used Lyapunov-Krasovskii (L-K) method. The new approach also simplifies the conditions and improves the results. Examples from the literature illustrate the efficiency of the new approach, allowing essentially large uncertainty of the Hessian matrix with bounds on varepsilon that are not too small.

Keywords: Extremum seeking, Optimization algorithms
Abstract: We introduce a type of safe extremum seeking (ES) controller, which minimizes an unknown objective function while also maintaining practical positivity of an unknown barrier function. We show semi-global practical asymptotic stability of our algorithm and present an analogous notion of practical safety. The dynamics of the controller are inspired by the quadratic program (QP) based safety filter designs which, in the literature, are more commonly used in cases where the barrier function is known. Conditions on the barrier and objective function are explored showing that non convex problems can be solved. A Lyapunov argument is proposed to achieve the main results of the paper. Finally, an example is given of the algorithm which solves the constrained optimization problem.

Keywords: Extremum seeking, Output regulation, Adaptive systems
Abstract: The Nussbaum gain approach has been the standard technique in solving unknown control direction problems. In this paper, we propose control laws composed of extremum seeking control, an internal model, and a compensator signal to solve the robust practical output regulation problem of a second-order system subject to an unknown control direction. Using the Lie bracket approximation technique, we show that the closed-loop system is bounded and the origin is (epsilon)Semi-global Practical Uniform Asymptotic Stable. Finally, we illustrate the effectiveness of the proposed approach with a numerical example of a Van der Pol system.

Keywords: Cooperative control, Stochastic systems, Optimization
Abstract: In this work we consider mean field type control problems with multiple species that have different dynamics. We formulate the discretized problem using a new type of entropy-regularized multimarginal optimal transport problems where the cost is a decomposable structured tensor. A novel algorithm for solving such problems is derived, using this structure and leveraging recent results in entropy-regularized optimal transport. The algorithm is then demonstrated on a numerical example in robot coordination problem for search and rescue, where three different types of robots are used to cover a given area at minimal cost.

Keywords: Stochastic optimal control, Stochastic systems
Abstract: Gradients in temperature and particle concentration fuel many processes in the physical and biological world. In the present work we study a thermodynamic engine powered by anisotropic thermal excitation (that may be due to e.g., a temperature gradient), and draw parallels with the well-known principle of impedance matching in circuit theory, where for maximal power transfer, the load voltage needs to be half of that of the supplying power source. We maximize power output of the thermodynamic engine at steady-state and show that the optimal reactive force is precise half of that supplied by the anisotropy.

Keywords: Filtering, Optimization, Statistical learning
Abstract: This paper is concerned with the theoretical and computational development of a new class of nonlinear filtering algorithms called optimal transport particle filters (OTPF). The algorithm is based on a recently introduced variational formulation of the Bayes' rule, which aims to find the Brenier optimal transport map between the prior and the posterior distributions as the solution to a stochastic optimization problem. On the theoretical side, the existing methods for the error analysis of particle filters and stability results for optimal transport map estimation are combined to obtain uniform error bounds for the filter's performance in terms of the optimization gap in solving the variational problem. The error analysis reveals a bias-variance trade-off that can ultimately be used to understand if/when the curse of dimensionality can be avoided in these filters. On the computational side, the proposed algorithm is evaluated on a nonlinear filtering example in comparison with the ensemble Kalman filter (EnKF) and the sequential importance resampling (SIR) particle filter.

Keywords: Identification, Estimation, Learning
Abstract: The contribution of this paper is a generalized formulation of correctional learning using optimal transport, which is about how to optimally transport one mass distribution to another. Correctional learning is a framework developed to enhance the accuracy of parameter estimation processes by means of a teacher-student approach. In this framework, an expert agent, referred to as the teacher, modifies the data used by a learning agent, known as the student, to improve its estimation process. The objective of the teacher is to alter the data such that the student's estimation error is minimized, subject to a fixed intervention budget. Compared to existing formulations of correctional learning, our novel optimal transport approach provides several benefits. It allows for the estimation of more complex characteristics as well as the consideration of multiple intervention policies for the teacher. We evaluate our approach on two theoretical examples, and on a human-robot interaction application in which the teacher's role is to improve the robots performance in an inverse reinforcement learning setting.

Keywords: Optimal control, Computational methods, Neural networks
Abstract: We present a novel computational framework for density control in high-dimensional state spaces. The considered dynamical system consists of a large number of indistinguishable agents whose behaviors can be collectively modeled as a time-evolving probability distribution. The goal is to steer the agents without collision from an initial distribution to reach (or approximate) a given target distribution within a fixed time horizon at minimum cost. To tackle this problem, we propose to model the drift as a nonlinear reduced-order model, such as a deep network, and enforce the matching to the target distribution at terminal time either strictly or approximately using the Wasserstein metric. The resulting saddle-point problem can be solved by an effective numerical algorithm that leverages the excellent representation power of deep networks and fast automatic differentiation for this challenging high-dimensional control problem. A variety of numerical experiments were conducted to demonstrate the performance of our method.

Keywords: Stochastic optimal control, Stochastic systems, Uncertain systems
Abstract: We consider the finite horizon optimal steering of the joint state probability distribution subject to the angular velocity dynamics governed by the Euler equation. The problem and its solution amounts to controlling the spin of a rigid body via feedback, and is of practical importance, for example, in angular stabilization of a spacecraft with stochastic initial and terminal states. We clarify how this problem is an instance of the optimal mass transport (OMT) problem with bilinear prior drift. We deduce both static and dynamic versions of the Eulerian OMT, and provide analytical and numerical results for the synthesis of the optimal controller.

Keywords: Discrete event systems, Automata, Fault diagnosis
Abstract: In this paper, we address the problem of failure diagnosis in timed discrete-event systems modeled by timed automata. While existing works on this topic typically focus on failures modeled as particular events, many complex applica- tions, especially time-critical systems, require the ability to identify time-sensitive failures associated with real-time information rather than just the occurrence of events at any time. To address this challenge, we propose the use of metric interval temporal logic (MITL) with continuous semantics on Boolean signals to formally describe time-sensitive failures. We introduce a novel concept called time-sensitive diagnosability (TS-diagnosability) to characterize whether or not any violation of the MITL task (i.e., failure) can be determined within a finite time elapsing. Furthermore, we provide a necessary and sufficient condition for verifying TS-diagnosability. Our results offer a more general framework for failure diagnosis of timed discrete-event systems

Keywords: Learning, Formal Verification/Synthesis, Markov processes
Abstract: Reinforcement learning (RL) is a popular paradigm for synthesizing controllers in environments modeled as Markov Decision Processes (MDPs). The RL formulation assumes that users define local rewards that depend only on the current state (and action), and learning algorithms seek to find control policies that maximize cumulative rewards along system trajectories. An implicit assumption in RL is that policies that maximize cumulative rewards are desirable as they meet the intended control objectives. However, most control objectives are emph{global} properties of system trajectories, and meeting them with local rewards requires tedious, manual and error-prone process of hand-crafting the rewards. We propose a new algorithm for automatically inferring local rewards from high-level task objectives expressed in the form of emph{symbolic automata} (SA); a symbolic automaton is a finite state machine where edges are labeled with symbolic predicates over the MDP states. SA subsume many popular formalisms for expressing task objectives, such as discrete-time versions of Signal Temporal Logic (STL). We assume that a model-free RL setting, i.e., we assume emph{no prior knowledge} of the system dynamics. We give theoretical results that establish that an optimal policy learned using our shaped rewards also maximizes the probability of satisfying the given SA-based control objective. Our approach is empirically compared with other RL methods that use temporal logic and automata-based control objectives, and evaluate them based on training convergence time and the probability of satisfying finite horizon control objectives.

Keywords: Formal Verification/Synthesis
Abstract: Temporal logics (TLs) have been widely used to formalize interpretable tasks for cyber-physical systems. Time Window Temporal Logic (TWTL) has been recently proposed as a specification language for dynamical systems. In particular, it can easily express robotic tasks, and it allows for efficient, automata-based verification and synthesis of control policies for such systems. In this paper, we define two quantitative semantics for this logic, and two corresponding monitoring algorithms, which allow for real-time quantification of satisfaction of formulas by trajectories of discrete-time systems. We demonstrate the new semantics and their runtime monitors on numerical examples.

Keywords: Formal Verification/Synthesis, Discrete event systems
Abstract: In time-critical systems, such as air traffic control systems, it is crucial to design control policies that are robust to timing uncertainty. Recently, the notion of asynchronous temporal robustness (ATR) was proposed to capture the robustness of a system trajectory against individual time shifts in its sub-trajectories. In a multi-robot system, this may correspond to individual robots being delayed or early. Control synthesis under ATR constraints is challenging and has not yet been addressed. In this paper, we propose a efficient control synthesis method under ATR constraints which are defined with respect to simple safety or complex signal temporal logic specifications. Given an ATR bound, we compute a sequence of control inputs so that the specification is satisfied by the system as long as each sub-trajectory is shifted not more than the ATR bound. We avoid combinatorially exploring all shifted sub-trajectories by first identifying redundancy between them. We capture this insight by the notion of instant-shift pair sets, and then propose an optimization program that enforces the specification only over the instant-shift pair sets. We show soundness and completeness of our method and analyze its computational complexity. Finally, we present various illustrative case studies.

Keywords: Formal Verification/Synthesis, Optimization algorithms, Optimal control
Abstract: This study considers the control problem with signal temporal logic (STL) specifications. Prior works have adopted smoothing techniques to address this problem within a feasible time frame and solve the problem by applying sequential quadratic programming (SQP) methods naively. However, one of the drawbacks of this approach is that solutions can easily become trapped in local minima that do not satisfy the specification. In this study, we propose a new optimization method, termed CCP-based SQP, based on the convex-concave procedure (CCP). Our framework includes a new robustness decomposition method that decomposes the robustness function into a set of constraints, resulting in a form of difference of convex (DC) program that can be solved efficiently. We solve this DC program sequentially as a quadratic program by only approximating the disjunctive parts of the specifications. Our experimental results demonstrate that our method has a superior performance compared to the state-of-the-art SQP methods in terms of both robustness and computational time.

Keywords: Formal Verification/Synthesis, Autonomous robots
Abstract: We consider the problem of autonomous exploration in search of targets while respecting a fixed energy budget. The robot is equipped with an incremental-resolution symbolic perception module wherein the perception of targets in the environment improves as the robot's distance from targets decreases. We assume no prior information about the number of targets, their locations, and possible distribution within the environment. This work proposes a novel decision-making framework for the resulting constrained sequential decision-making problem by first converting it into a reward maximization problem on a product graph computed offline. It is then solved online as a Mixed-Integer Linear Program (MILP) where the knowledge about the environment is updated at each step, combining automata-based and MILP-based techniques. We demonstrate the efficacy of our approach with the help of a case study and present empirical evaluation in terms of expected regret. Furthermore, the runtime performance shows that online planning can be efficiently performed for moderately-sized grid environments.

Keywords: Networked control systems, Data driven control, Sampled-data control
Abstract: This paper considers data-driven control of unknown linear discrete-time systems under a self-triggered transmission scheme. While self-triggered control has received much attention in the literature, its design and implementation typically require explicit model knowledge. Due to the difficulties in obtaining accurate models and the abundance of data in applications, this paper proposes a novel data-driven self-triggered control scheme for unknown systems. To this end, we begin by presenting a model-based self-triggered scheme (STS) in form of quadratic matrix inequalities, on the basis of an equivalent switched system representation. Combining the model-based triggering law and a data-based system representation, a data-driven STS is developed leveraging pre-collected input-state data for predicting the next transmission instant while ensuring system stability. A data-based method for co-designing the controller gain and the triggering matrix is then provided. Finally, a numerical simulation showcases the efficacy of STS in reducing transmissions as well as practicality of the proposed co-design methods.

Keywords: Biomedical, Sampled-data control, Nonlinear systems
Abstract: In this paper, an event-based quantized sampled-data control strategy is proposed for the plasma glucose regulation problem in Type 2 diabetic patients. In particular, the proposed event-triggered digital glucose regulator is designed by exploiting a nonlinear time-delay model of the glucose-insulin regulatory system which takes into account the subcutaneous infusion of insulin. It is proved that the provided quantized sampled-data glucose controller, updated via a proposed event-based mechanism, guarantees the semi-global practical stability property of the related closed--loop tracking error system, with arbitrarily small steady--state tracking error. The stabilization in the sample--and--hold sense theory is used as a tool to prove the results. An approximation scheme based on first--order splines is used in order to cope with the problem of the possible non--availability in the buffer of the value of the system variables at some past times which are needed for the implementation of the proposed digital controller. The possible non-uniform quantization of the input/output channels as well as the case of time--varying sampling periods are included in the theory here developed. The validation of the proposed glucose control strategy is carried out via simulations.

Keywords: Robotics, Nonlinear systems, Network analysis and control
Abstract: This paper focuses on the event-triggered tracking control problem for Hamiltonian-based flexible-joints robots. Based on Hamiltonian theory and the delay system approach, an event-triggered modular control strategy is proposed to guarantee that the link and motor generalized position can track the target signal asymptotically while reducing the waste of transmission resources. The modular controller designed in this paper takes fully into account the structural characteristics of the flexible-joints robots. Moreover, the difficulties of tracking control and event-triggered control caused by the high coupling of Hamiltonian systems are overcome in the design process. This extends the existing theoretical results for flexible-joints robots and effectively improves energy efficiency. The validity of the proposed strategy is verified by a simulation example of a flexible joint robot.

Keywords: Backstepping, Hybrid systems, Stability of linear systems
Abstract: This paper provides a novel self-triggered boundary control (STBC) strategy for a class of reaction-diffusion PDEs with Robin actuation using infinite-dimensional backstepping boundary control. Our goal is to offer a solution for the continuous monitoring of triggering functions in conventional event-triggered control. We propose a method for converting a certain class of continuous-time dynamic event-triggers that require continuous monitoring to self-triggers that proactively compute the time of the next event at the current event time using the knowledge of the available system states and dynamics. We achieve this by designing a positively and uniformly lower-bounded function which, when evaluated at the current event time, outputs the waiting time until the next event. The control input is updated only at events indicated by the self-trigger and is applied in a zero-order hold fashion between two events. We establish the closed-loop system well-posedness under the proposed STBC approach. Furthermore, we prove that the global L^2-exponential convergence to zero under continuous-time event-triggered boundary control (CETBC) is preserved under the proposed STBC approach. We provide a simulation result that validates the theoretical claims.

Keywords: Distributed control, Optimal control, Cooperative control
Abstract: It is a challenging problem to achieve fast distributed optimal consensus for Euler–Lagrangian (EL) systems meanwhile economizing communication resources. To solve the problem, a novel predefined-time distributed optimal consensus strategy for EL systems is proposed by applying time-base generator (TBG) and dynamic event-triggered mechanism, which can reach the optimal consensus in a completely distributed manner. It is proven that the algorithm can converge in predefined time by Lyapunov energy function and Zeno behavior is avoided. A dynamic event-triggered method is designed which event-triggered thresholds are replaced by dynamic variables. The numerical simulation is given to show the effectiveness and superiority of the algorithm.

Keywords: Distributed parameter systems, Stability of nonlinear systems, Lyapunov methods
Abstract: Although switching-based stabilization of 1D parabolic systems was investigated by employing one actuator moving in spatial domain in our recent paper [18], this method increases the system cost since actuator and sensor switching happens at fixed time regardless of whether the switching is necessary or not. To further reduce operating and production cost, in the present paper, switching-based dynamic event-triggered control law is studied to stabilize the parabolic PDE systems via output-dependent switching law. Constructive exponential stability conditions are established by using Lyapunov method. A numerical example shows the effectiveness of the proposed methods.

Keywords: Supervisory control, Discrete event systems, Automata
Abstract: Supervisor localization can be applied to distribute a monolithic supervisor into local supervisors. Performing supervisor localization can be computationally costly. In this work, we consider systems that evolve over time. We study how to reuse the results from a previous supervisor localization, to more efficiently compute local supervisors when the system is adapted. We call this approach transformational supervisor localization, and present algorithms for the procedure. The efficiency of the procedure is experimentally evaluated.

Keywords: Discrete event systems
Abstract: Opacity is an information-flow property capturing privacy from observers that are aware of a system’s dynamics. The potential for an observer with perfect recall to reason about long histories of the system poses a challenge for opacity verification. In this paper, we address this challenge by proposing a new notion of opacity over automata, called bounded memory opacity, with respect to an observer with a bounded memory. We show that verifying this weaker notion of opacity has reduced computational complexity compared to general opacity (co-NP vs. PSPACE). Furthermore, we present a corresponding verification algorithm using an encoding to the Boolean satisfiability problem (SAT). We demonstrate this approach on randomly generated automata as well as a web server load-hiding example.

Keywords: Discrete event systems, Automata, Estimation
Abstract: We investigate the state estimation problem under a decentralized observation architecture. More specifically, we consider a discrete event system, modeled by a nondeterministic finite automaton, whose behavior is partially observed and recorded at a set of observation sites with distinct capabilities. When prompted, these observation sites send their sequences of observations to a coordinator that fuses and analyzes this information to estimate the specific system states of interest (current- and initial-states). The notion of S-builder is introduced to systematically infer possible (totally ordered) sequences of observations and an algorithm is proposed for constructing a synchronizer in a breadth-first search manner to efficiently perform current-state estimation. With slight extensions, the synchronizer construction algorithm can be also applied towards initial-state estimation.

Keywords: Discrete event systems, Supervisory control, Data driven control
Abstract: In this paper we develop a data-driven approach for supervisory control of discrete-event systems (DES). We consider a setup in which models of DES to be controlled are unknown, but a set of data concerning the behaviors of DES is available. We propose a new concept of data-informativity, which captures the notion that the available data set contains sufficient information such that a valid supervisor may be constructed for a family of DES models that all can generate the data set. We then characterize data-informativity with a necessary and sufficient condition, based on which we design an algorithm for its verification. Moreover, if the data set fails to be informative, we propose two related new concepts of limited data-informativity and informatizability. Their characterization conditions and verification algorithms are also presented.

Keywords: Discrete event systems, Automata, Markov processes
Abstract: This paper is about state estimation in a class of labeled timed probabilistic automata. In detail, we consider continuous time Markov processes where the occurrence of some transitions produces observable events. Such observations can be used to update and refine the state estimation. In this setting, we discuss how a logical state estimation approach can be used to characterize the probabilistic state estimation whenever a new event is observed or when the system evolves without producing new observations (silent closure). The main results of the paper show that the final behaviour, as the silent closure goes to infinity, cannot be characterized only in terms of the graphical structure of the underlying automaton but also depends on the values of the firing rates.

Keywords: Petri nets, Discrete event systems, Modeling
Abstract: Deadlock-freeness is a basic property of dynamical systems that ensures that at least one process of the system can operate indefinitely. Given that a system is deadlock-free if at any reachable state there is at least one process that can operate, the reachability space, i.e. the set of states that can be reached, and deadlock-freeness are closely related. This paper focuses on some fundamental properties of the reachability space of Flexible Nets, a modeling formalism that can easily account for uncertain parameters. After showing that the reachability space is convex, a sufficient condition for deadlock-freeness is derived.

Keywords: Adaptive control, Neural networks, Stability of nonlinear systems
Abstract: Recurrent neural networks (RNNs) are a dynamic mapping that can capture time-varying, accumulative effects in a sequence that static, feedforward neural networks (NNs) cannot. Long short-term memory (LSTM) NNs are a type of RNNs that have gained recent popularity because the cell structure allows them to retain long-term information more efficiently than traditional RNNs. Existing results develop LSTM-based controllers to compensate for uncertainties in nonlinear systems. However, these results use discrete-time LSTMs with offline-trained weights. In this paper, a Lyapunov-based LSTM controller is developed for general Euler-Lagrange systems. Specifically, an Lb-LSTM is implemented in the control design to adaptively estimate uncertain model dynamics, where the weight estimates of the LSTM cell are updated using Lyapunov-based adaptation laws. This allows the LSTM cell to adapt to system uncertainties without requiring offline training. A Lyapunov-based stability analysis yields uniform ultimate boundedness (UUB) of the tracking errors and LSTM state and weight estimation errors. Simulations indicate the developed Lb-LSTM-based controller yielded significant improvement in tracking and function approximation performance when compared to several DNN examples.

Keywords: Learning, Optimization, Uncertain systems
Abstract: This paper presents a probabilistic framework for the safe control of linear systems under parametric uncertainties. To this end, a two-layer learning-enabled controller is presented that unifies the experience-replay model learning and the scenario-based optimization. The inner loop leverages the scenario optimization to impose probabilistic stability and safety specifications through sampling from a control Lyapunov function and a control barrier function, respectively. Each sample represents a plausible system dynamics realization within the range of the uncertainties. To quantify the model uncertainty and, thus, to facilitate a proactive and goal-oriented sampling of safety and stability constraints, an experience replay-based model learning is presented in the outer loop. The exponentially fast convergent guarantees of the presented approach and the quantification of the exponential rate using the collected data allow us to quantify the ambiguity set for the system parameters based on the data informativeness. The quantified modeling error acts as a vanishing perturbation to the true dynamics, from which samples can be taken at a specific frequency to solve an optimization problem in the inner loop. The presented approach provides safety and stability guarantees with high probability, even during learning. Simulation is used to depict the efficacy of the proposed approach.

Keywords: Game theory, Optimization, Autonomous systems
Abstract: Examining the behavior of multi-agent systems is vitally important to many emerging distributed applications - game theory has emerged as a powerful tool set in which to do so. The main approach of game-theoretic techniques is to model agents as players in a game, and predict the emergent behavior through the relevant Nash equilibria. The virtue from this viewpoint is that by assuming that self-interested decision-making processes lead to Nash equilibrium, system behavior can then be captured by Nash equilibrium without studying the decision-making processes explicitly. This approach has seen success in a wide variety of domains, such as sensor coverage, traffic networks, auctions, and network coordination. However, in many other problem settings, Nash equilibria are not necessarily guaranteed to exist or emerge from self-interested processes. Thus the main focus of the paper is on the study of sink equilibria, which are defined as the attractors of these decision-making processes. By classifying system outcomes through a global objective function, we can analyze the resulting approximation guarantees that sink equilibria have for a given game. Our main result is an approximation guarantee on the sink equilibria through defining an introduced metric of misalignment, which captures how uniform agents are in their self-interested decision making. Overall, sink equilibria are naturally occurring in many multi-agent contexts, and we display our results on their quality with respect to two practical problem settings.

Keywords: Optimization, Optimization algorithms, Agents-based systems
Abstract: In this paper, we focus on an asynchronous distributed optimization problem. In our problem, each node is endowed with a convex local cost function, and is able to communicate with its neighbors over a directed communication network. Furthermore, we assume that the communication channels between nodes have limited bandwidth, and each node suffers from processing delays. We present a distributed algorithm which combines the Alternating Direction Method of Multipliers (ADMM) strategy with a finite time quantized averaging algorithm. In our proposed algorithm, nodes exchange quantized valued messages and operate in an asynchronous fashion. More specifically, during every iteration of our algorithm each node (i) solves a local convex optimization problem (for the one of its primal variables), and (ii) utilizes a finite-time quantized averaging algorithm to obtain the value of the second primal variable (since the cost function for the second primal variable is not decomposable). We show that our algorithm converges to the optimal solution at a rate of O(1/k) (where k is the number of time steps) for the case where the local cost function of every node is convex and not-necessarily differentiable. Finally, we demonstrate the operational advantages of our algorithm against other algorithms from the literature.

Keywords: Learning, Stochastic optimal control, Neural networks
Abstract: Improvement of exploration and exploitation using more efficient samples is a critical issue in reinforcement learning algorithms. A basic strategy of a learning algorithm is to facilitate indiscriminate exploration of the entire environment state space, as well as to encourage exploration of rarely visited states rather than frequently visited ones. Under this strategy, we propose a new method to boost exploration through an intrinsic reward, based on the measurement of a state's novelty and the associated benefit of exploring the state, collectively called plausible novelty. By incentivizing exploration of plausible novel states, an actor-critic (AC) algorithm can improve its sample efficiency and, consequently, its training performance. The new method is verified through extensive simulations of continuous control tasks in MuJoCo environments, using a variety of prominent off-policy AC algorithms.

Keywords: Machine learning, Learning
Abstract: There is a need for robust Reinforcement Learning (RL) algorithms that can cope with model misspecification, parameter uncertainty, disturbances, etc. Risk-sensitive methods offer an approach to developing robust RL algorithms by hedging against undesirable outcomes in a probabilistic manner. The Probabilistic Graphical Model (PGM) framework offers systematic exploration for risk-sensitive RL. In this paper, we bridge the Markov Decision Process (MDP) and the PGM frameworks. We exploit the equivalence of optimizing a certain risk-sensitive criterion in the MDP formalism with optimizing a log-likelihood objective in the PGM formalism. By utilizing this equivalence, we offer an approach for developing risk-sensitive algorithms by leveraging the PGM framework. We explore the Expectation-Maximization (EM) algorithm under the PGM formalism. We show that risk-sensitive policy gradient methods can be obtained by applying sampling-based approaches to the EM algorithm, e.g., Monte-Carlo EM, with the log-likelihood. We show that Monte-Carlo EM leads to a risk-sensitive Monte-Carlo policy gradient algorithm. Our simulations illustrate the risk-sensitive nature of the resulting algorithm.

Keywords: Discrete event systems
Abstract: This work proposes a data-driven framework to synthesize safety controllers for nonlinear systems with finite input sets and unknown mathematical models. The proposed scheme leverages new notions of multiple control barrier certificates (M-CBC) and provides controllers ensuring the safety of systems with confidence 1. While there may not exist a common control barrier certificate with a fixed template, our proposed technique adaptively partitions the state set to potentially find M-CBC of the same template for different regions. In the proposed data-driven framework, we first cast our proposed conditions of M-CBC as a robust optimization program (ROP). Given that the unknown model appears in some of the constraints of the ROP, we propose a sampling approach for collecting data and provide a scenario optimization program (SOP) associated with the proposed ROP. We solve the resulted SOP and construct M-CBC together with safety controllers for the unknown system with 100% correctness guarantee. We apply our results to a nonlinear jet engine compressor with unknown dynamics to illustrate the efficacy of our data-driven approach. In the case study, we show that while there exists no common polynomial-type control barrier certificate of a given degree, there exist polynomial-type M-CBC of the same degree by partitioning the state set to different regions.

Keywords: Formal Verification/Synthesis, Identification, Uncertain systems
Abstract: Reachability analysis is a powerful tool for computing the set of states or outputs reachable for a system. While previous work has focused on systems described by state-space models, we present the first methods to compute reachable sets of ARMAX models - one of the most common input-output models originating from data-driven system identification. The first approach we propose can only be used with dependency-preserving set representations such as symbolic zonotopes, while the second one is valid for arbitrary set representations but relies on a reformulation of the ARMAX model. By analyzing the computational complexities, we show that both approaches scale quadratically with respect to the time horizon of the reachability problem when using symbolic zonotopes. To reduce the computational complexity, we propose a third approach that scales linearly with respect to the time horizon when using set representations that are closed under Minkowski addition and linear transformation and that satisfy that the computational complexity of the Minkowski sum is independent of the representation size of the operands. Our numerical experiments demonstrate that the reachable sets of ARMAX models are tighter than the reachable sets of equivalent state space models in case of unknown initial states. Therefore, this methodology has the potential to significantly reduce the conservatism of various verification techniques.

Keywords: Maritime control, Autonomous robots, Predictive control for nonlinear systems
Abstract: Low-propulsion vessels can take advantage of powerful ocean currents to navigate towards a destination. Recent results demonstrated that vessels can reach their destination with high probability despite forecast errors. However, these results do not consider the critical aspect of safety of such vessels: because their propulsion is much smaller than the magnitude of surrounding currents, they might end up in currents that inevitably push them into unsafe areas such as shallow waters, garbage patches, and shipping lanes. In this work, we first investigate the risk of stranding for passively floating vessels in the Northeast Pacific. We find that at least 5.04% would strand within 90 days. Next, we encode the unsafe sets as hard constraints into Hamilton-Jacobi Multi-Time Reachability to synthesize a feedback policy that is equivalent to re-planning at each time step at low computational cost. While applying this policy guarantees safe operation when the currents are known, in realistic situations only imperfect forecasts are available. Hence, we demonstrate the safety of our approach empirically with large-scale realistic simulations of a vessel navigating in high-risk regions in the Northeast Pacific. We find that applying our policy closed-loop with daily re-planning as new forecasts become available reduces stranding below 1% despite forecast errors often exceeding the maximal propulsion. Our method significantly improves safety over the baselines and still achieves a timely arrival of the vessel at the destination.

Keywords: Formal Verification/Synthesis, Learning, Randomized algorithms
Abstract: This paper is concerned with data-driven reachability analysis of discrete-time nonlinear systems without any dynamical model. We use only a number of observations of trajectories of the system to estimate the actual reachable set. With the data set, using the Support Vector Data Description (SVDD) technique, we propose a sample-based approximation method to solve the reachability analysis problem, which can be considered as a one-class classification problem. Under the framework of scenario optimization, we then derive over-approximations of the reachable set in a probabilistic sense with Lipschitz continuity and other regularity conditions. Finally, we demonstrate the proposed method on a numerical example.

Keywords: Formal Verification/Synthesis, Neural networks, Optimal control
Abstract: We consider the problem of learning a neural network controller for a system required to satisfy a Signal Temporal Logic (STL) specification. We exploit STL quantitative semantics to define a notion of robust satisfaction. Guaranteeing the correctness of a neural network controller is a difficult problem that received a lot of attention recently. We provide a general procedure to construct a set of trainable High Order Control Barrier Functions (HOCBFs) enforcing the satisfaction of formulas in a fragment of STL. We use the BarrierNet, implemented by a differentiable Quadratic Program (dQP) with HOCBF constraints, as the last layer of the neural network controller, to guarantee the satisfaction of the STL formulas. We train the HOCBFs together with other neural network parameters to further improve the robustness of the controller. Simulation results demonstrate that our approach ensures satisfaction and outperforms existing algorithms.

Keywords: Formal Verification/Synthesis, Uncertain systems, Large-scale systems
Abstract: In this paper, we present an approach for designing correct-by-design controllers for cyber-physical systems composed of multiple dynamically interconnected uncertain systems. We consider networked discrete-time uncertain nonlinear systems with additive stochastic noise and model parametric uncertainty. Such settings arise when multiple systems interact in an uncertain environment and only observational data is available. We address two limitations of existing approaches for formal synthesis of controllers for networks of uncertain systems satisfying complex temporal specifications. Firstly, whilst existing approaches rely on the stochasticity to be Gaussian, the heterogeneous nature of composed systems typically yields a more complex stochastic behavior. Secondly, exact models of the systems involved are generally not available or difficult to acquire. To address these challenges, we show how abstraction-based control synthesis for uncertain systems based on sub-probability couplings can be extended to networked systems. We design controllers based on parameter uncertainty sets identified from observational data and approximate possibly arbitrary noise distributions using Gaussian mixture models whilst quantifying the incurred stochastic coupling. Finally, we demonstrate the effectiveness of our approach on a nonlinear package delivery case study with a complex specification, and a platoon of cars.

Keywords: Power systems, Game theory, Robust control
Abstract: We develop investment approaches to secure electric power systems against load attacks where a malicious intruder (the attacker) covertly changes reactive power setpoints of loads to push the grid towards voltage instability while the system operator (the defender) employs reactive power compensation (RPC) to prevent instability. Extending our previously reported Stackelberg game formulation for this problem, we develop a robust-defense sequential algorithm and a novel genetic algoirhtm that provides scalability to large-scale power system models. The proposed methods are validated using IEEE prototype power system models with time-varying load uncertainties, demonstrating that reliable and robust defense is feasible unless the operator's RPC investment resources are severely limited relative to the attacker's resources.

Keywords: Cyber-Physical Security
Abstract: The impacts of zero dynamics attacks and perfectly undetectable cyber-attacks cannot be observed in outputs of cyber-physical systems (CPS). Adversaries are capable of executing these cyber-attacks and leading the CPS to undesirable trajectories while remaining undetected. In this paper, we introduce and formally introduce the notion of security effort (SE) as a novel security metric for CPS that determines the minimum number of actuators and sensors that should be secured and kept attack free in order to prevent adversaries from executing zero dynamics attacks, covert attacks, and controllable attacks. Moreover, since zero dynamics attacks, covert attacks, and controllable attacks belong to weakly unobservable and controllable weakly unobservable subspaces of the CPS, conditions under which these subspaces become zero are obtained and investigated. An illustrative numerical simulation is provided to demonstrate the effectiveness of our proposed security measure.

Keywords: Sensor fusion, Fault tolerant systems, Cyber-Physical Security
Abstract: This paper proposes a secure state estimation scheme with asynchronous non-periodic measurements for continuous LTI systems under false data attacks on measurement transmission channels. Each sensor transmits the measurement information in a triple comprised of its sensor index, the time-stamp, and the measurement value to the fusion center via unprotected communication channels. A malicious attacker can corrupt a subset of sensors by (i) manipulating the time- stamp and the measurement value; (ii) blocking transmitted measurement triples; or (iii) injecting fake measurement triples. To deal with such attacks, we propose a secure state estimator by designing decentralized local estimators and fusing all the local states by the median operator. The local estimators receive the sampled measurements and update their local state in an asynchronous manner, while the fusion center triggers the fusion and generates a secure estimation in the presence of a local update. We prove that local estimators of benign sensors are unbiased with stable error covariance. Moreover, the fused secure estimation error has bounded expectation and covariance against at most p corrupted sensors as long as the system is 2p-sparse observable. The efficacy of the proposed scheme is demonstrated through a benchmark example of the IEEE 14-bus system.

Keywords: Attack Detection, Cyber-Physical Security, Uncertain systems
Abstract: In this paper, a set-based detector is developed and analyzed based on the propagation of nominal (no attacks), actual (possibly with attacks), and perceived (corrupted by attacks) residuals due to bounded system and measurement noise uncertainties. The set of stealthy attacks (attacks that raise no alarms) is characterized and the impact of these stealthy attacks on the system state is quantified. When implemented through the set tools of constrained zonotopes, this approach provides accurate, time-varying attack-reachable sets that can be used to evaluate the safety and performance of systems in adversarial conditions. These tools are demonstrated through two case studies.

Keywords: Fault detection, Power systems, Estimation
Abstract: Motivated by the need to localise faults along electrical power lines, this paper adopts a frequency-domain approach to parameter estimation for an infinite-dimensional linear dynamical system with one spatial variable. Since the time of the fault is unknown, and voltages and currents are measured at only one end of the line, distance information must be extracted from the post-fault transients. To properly account for high-frequency transient behaviour, the line dynamics is modelled directly by the Telegrapher’s equation, rather than the more commonly used lumped-parameter approximations. First, the governing equations are non-dimensionalised to avoid ill-conditioning. A closed-form expression for the transfer function is then derived. Finally, nonlinear least-squares optimisation is employed to search for the fault location. Requirements on fault bandwidth, sensor bandwidth and simulation time-step are also presented. The result is a novel end-to-end algorithm for data generation and fault localisation, the effectiveness of which is demonstrated via simulation.

Keywords: Cyber-Physical Security, Data driven control, Resilient Control Systems
Abstract: This study investigates the vulnerability of direct data-driven control to adversarial attacks in the form of a small but sophisticated perturbation added to the original data. The directed gradient sign method (DGSM) is developed as a specific attack method, based on the fast gradient sign method (FGSM), which has originally been considered in image classification. DGSM uses the gradient of the eigenvalues of the resulting closed-loop system and crafts a perturbation in the direction where the system becomes less stable. It is demonstrated that the system can be destabilized by the attack, even if the eigenvalues of the original closed-loop matrix with the clean data are aligned far from the unstable region. To increase the robustness against the attack, regularization methods that have been developed to deal with random disturbances are considered. Their effectiveness is evaluated by numerical experiments using an inverted pendulum model.

Keywords: Autonomous vehicles, Predictive control for nonlinear systems, Robotics
Abstract: This work proposes a nonlinear model predictive control (NMPC) strategy for robot navigation in cluttered unknown environments using polynomial zonotopes. The infor- mation provided by a laser sensor is used in the computation of the collision-free area. The procedure splits the area into convex subregions which are converted into polynomial zonotopes (PZs) to generate constraints for the NMPC optimal control problem. The PZ is a set representation that can describe polytopes using fewer constraints than conventional half-space representations, thus being more efficient while maintaining the accuracy equiv- alent to the polytopic case. Numerical experiments demonstrate the advantages of the proposed strategy.

Keywords: Autonomous vehicles, Traffic control, Transportation networks
Abstract: This paper addresses the challenge of generating optimal vehicle flow at the macroscopic level. Although several studies have focused on optimizing vehicle flow, little attention has been given to ensuring it can be practically achieved. To overcome this issue, we propose a route-recovery and eco-driving strategy for connected and automated vehicles (CAVs) that guarantees optimal flow generation. Our approach involves identifying the optimal vehicle flow that minimizes total travel time, given the constant travel demands in urban areas. We then develop a heuristic route-recovery algorithm to assign routes to CAVs. Finally, we present an efficient coordination framework to minimize the energy consumption of CAVs while safely crossing intersections. The proposed method can effectively generate optimal vehicle flow and potentially reduce travel time and energy consumption in urban areas.

Keywords: Autonomous vehicles
Abstract: Occlusions at intersections threaten the safety of autonomous driving, especially in urban scenarios. Phantom vehicles are usually considered to mitigate the risks associated with the occlusions, and their states may be inferred from observable vehicles. This paper extends these methods to deal with multiple occlusions and multiple inferences. A multi-occlusion aware model allows for the concerns of occluded areas and the configuration of phantom vehicles. The evidence theory fuses multiple estimations to enhance occlusion inferences. Simulations for intersection scenarios are conducted to demonstrate the effectiveness of the proposed method for improving traffic efficiency.

Keywords: Autonomous vehicles
Abstract: The determination of robot trajectories that can achieve reactive collision avoidance with moving objects is of great interest. In cluttered environments with tight spaces, it becomes important to consider the shapes of the objects in computing these trajectories. The literature largely models the shapes of the objects as circles, and this can make the avoidance maneuvers very conservative, especially when the objects are elongated and/or non-convex. In this paper, we model the shapes of the objects using combinations of quadric surfaces or polygons, and employ a collision cone approach to achieve reactive collision avoidance, in the presence of measurement noise. The collision avoidance design employs a dual-loop control architecture where the inner loop uses a dynamic inversion-based method and the outer loop uses Linear Matrix Inequalities (LMIs). Simulation results demonstrating the the collision avoidance laws for dynamic, heterogeneous quadric and polygonal surfaces are presented.

Keywords: Autonomous vehicles
Abstract: The capturability analysis of the pulsed guidance law using differential game theory is presented in this paper. The engagement of an interceptor with pulsed guidance constraint and a maneuvering target with bounded acceleration is considered. The differential game guidance laws for both the interceptor and the target are proposed. Then the engagement is converted into four different cases according to the guidance law parameters, and the capture boundary conditions for these four cases are given as functions of the guidance law parameters and the system parameters, including acceleration constraints, the acceptable miss distance and initial values of the engagement. Afterwards, the capture zone is given according to the boundary conditions and is decided by the guidance law parameters and the system parameters. Finally, various specific examples show that the interception can be guaranteed with lower acceleration constraint ratio and higher guidance law parameter ratio.

Keywords: Aerospace, Optimal control, Autonomous vehicles
Abstract: This paper presents an indirect solution method for state-constrained optimal control problems to address the long-standing issue of discontinuous control and costate under state constraints. It is known in optimal control theory that a state inequality constraint introduces discontinuities in control and costate, rendering the classical indirect solution methods ineffective. This study re-examines the necessary conditions of optimality for a class of state-constrained optimal control problems, and shows the uniqueness of the optimal control that minimizes the Hamiltonian under state constraints, which leads to a unifying form of the necessary conditions. The unified form of the necessary conditions opens the door to addressing the issue of discontinuities in control and costate by modeling them via smooth functions. This paper exploits this insight to transform the originally discontinuous problems to smooth two-point boundary value problems that can be solved by existing nonlinear root-finding algorithms. This paper also shows the formulated solution method to have an anytime algorithm-like property, and then numerically demonstrates the solution method by an optimal orbit control problem.

Keywords: Agents-based systems, Building and facility automation, Learning
Abstract: Robotic sortation centers use mobile robots to sort packages by their destinations. The destination-to-sort- location (chute) mapping can significantly impact the volume of packages that can be sorted by the sortation floor. In this work, we propose a multi-agent reinforcement learning method to solve large-scale chute mapping problems with hundreds of agents (the destinations). To address the exponential growth of the state-action space, we decompose the joint action-value function as the sum of local action-value functions associated with the individual agents. To incorporate robot congestion effects on the rates at which packages are sorted, we couple the local action-value functions through the states of destinations mapped to nearby chutes on the sortation floor. We show that our proposed framework can solve large chute mapping problems and outperforms static or reactive policies that are commonly used in practice in robotic sortation facilities.

Keywords: Stochastic optimal control, Game theory
Abstract: Until recently, efficient policy iteration algorithms for zero-sum Markov games that converge were unknown. Therefore, model-based RL algorithms for such problems could not use policy iteration in the planning modules of the algorithms. In an earlier paper, we showed that a convergent policy iteration algorithm can be obtained by using a commonly used technique in RL called lookahead. However, the algorithm could be applied to the function approximation setting only in the special case of linear MDPs (Markov Decision Processes). In this paper, we obtain performance bounds for policy-based RL algorithms for general settings, including one where policy evaluation is performed using noisy samples of (state, action, reward) triplets from a single sample path of a given policy.

Keywords: Learning, Optimization, Numerical algorithms
Abstract: We consider the problem of estimation of an unknown real valued function with real valued input by an agent. The agent exists in 3D Euclidean space. It is able to traverse in a 2D plane while the function is depicted in a 2D plane perpendicular to the plane of traversal. By viewing the function from a given position, the agent is able to collect a data point lying on the function. By traversing through the plane while paying a control cost, the agent collects a finite set of data points. The set of data points are used by the agent to estimate the function. The objective of the agent is to find a control law which minimizes the control cost while estimating the function optimally. We formulate a control problem for the agent incorporating an inference cost and the control cost. The control problem is relaxed by finding a lower bound for the cost function. We present a kernel based linear regression model to approximate the cost-to-go and use the same in a control algorithm to solve the relaxed optimization problem. We present simulation results comparing the proposed approach with greedy algorithm based exploration.

Keywords: Identification, Agents-based systems, Computational methods
Abstract: In this work, we study the inverse problem of identifying complex flocking dynamics in a domain cluttered with obstacles. We get inspiration from animal flocks moving in complex ways with capabilities far beyond what current robots can do. Owing to the difficulty of observing and recovering the trajectories of the agents, we focus on the dynamics of their probability densities, which are governed by partial differential equations (PDEs), namely compressible Euler equations subject to non-local forces. We formulate the inverse problem of learning interactions as a PDE-constrained optimization problem of minimizing the squared Hellinger distance between the histogram of the flock and the distribution associated to our PDEs. The numerical methods used to efficiently solve the PDE-constrained optimization problem are described. Realistic flocking data are simulated using the Boids model of flocking agents, which differs in nature from the reconstruction models used in our PDEs. Our analysis and simulated experiments show that the behavior of cohesive flocks can be recovered accurately with approximate PDE solutions.

Keywords: Statistical learning, Linear systems, Optimization
Abstract: We address the problem of learning linear system models by observing multiple trajectories from systems with differing dynamics. This framework encompasses a collaborative scenario where several systems seeking to estimate their dynamics are partitioned into clusters according to system similarity. Thus, the systems within the same cluster can benefit from the observations made by the others. Considering this framework, we present an algorithm where each system alternately estimates its cluster identity and performs an estimation of its dynamics. This is then aggregated to update the model of each cluster. We show that under mild assumptions, our algorithm correctly estimates the cluster identities and achieves an epsilon-approximate solution with a sample complexity that scales inversely with the number of systems in the cluster, thus facilitating a more efficient and personalized system identification.

Keywords: Decentralized control, Learning, Cooperative control
Abstract: Decentralized multi-agent reinforcement learning (MARL) is an inherently difficult problem because agents can have individual, unique objectives and no direct incentive to cooperate. Conflicts often arise over bottlenecks in the environment, such as a shared key or an intersection, where multiple agents need to access a single resource. To resolve these conflicts, we propose the use of a shared scheduling protocol. A scheduling protocol coordinates agent behavior such that one agent is allowed to greedily use the resource while the others are required to wait. In particular, we are interested in decentralized scheduling protocols that can be implemented independently by each agent without a centralized controller. We present three protocols and prove that they resolve conflicts when obeyed by all agents. In training, agents learn to obey the protocol as violations incur a penalty. Experimental results show that scheduling protocols increase the performance of multi-agent training fivefold compared to baseline decentralized MARL.

Keywords: Quantum information and control
Abstract: Quantum process tomography (QPT) is a fundamental task to characterize the dynamics of quantum systems. In contrast to standard QPT, ancilla-assisted process tomography (AAPT) framework introduces an extra ancilla system such that a single input state is needed. In this paper, we extend the two-stage solution, a method originally designed for standard QPT, to perform AAPT. Our algorithm has O(Md_A^2d_B^2) computational complexity where M is the type number of the measurement operators, d_A is the dimension of the quantum system of interest, and d_B is the dimension of the ancilla system. Then we establish an error upper bound and further discuss the optimal design on the input state in AAPT. A numerical example on a phase damping process demonstrates the effectiveness of the optimal design and illustrates the theoretical error analysis.

Keywords: Robust control, Quantum information and control
Abstract: Recent achievements in quantum control have resulted in advanced techniques for designing controllers for applications in quantum communication, computing, and sensing. However, the susceptibility of such systems to noise and uncertainties necessitates robust controllers that perform effectively under these conditions to realize the full potential of quantum devices. The time-domain log-sensitivity and a recently introduced robustness infidelity measure (RIM) are two means to quantify controller robustness in quantum systems. The former can be found analytically, while the latter requires Monte-Carlo sampling. In this work, the correlation between the log-sensitivity and the RIM for evaluating the robustness of single excitation transfer fidelity in spin chains and rings in the presence of dephasing is investigated. We show that the expected differential sensitivity of the error agrees with the differential sensitivity of the RIM, where the expectation is over the error probability distribution. Statistical analysis also demonstrates that the log-sensitivity and the RIM are linked via the differential sensitivity, and that the differential sensitivity and RIM are highly concordant. This unification of two means (one analytic and one via sampling) to assess controller robustness in a variety of realistic scenarios provides a first step in unifying various tools to model and assess robustness of quantum controllers.

Keywords: Quantum information and control, Stochastic systems, Mean field games
Abstract: Following Kolokoltsov’s work [1], we present an extension of mean-field control theory in quantum framework. In particular such an extension is done naturally by considering the Belavkin quantum filtering and control theory in a mean- field setting. In this setting, the dynamics is described by a controlled Belavkin equation of McKean-Vlasov type. We prove the well-posedness of such an equation under imperfect measurement records. Furthermore, we show under purification assumption the propagation of chaos for perfect measurements. Finally, we apply particle methods to simulate the mean- field Belavkin equation and we provide numerical simulations showing the stabilization of the mean-field Belavkin equation by a feedback control strategy towards a chosen target state.

Keywords: Quantum information and control, Optimal control
Abstract: The transmission of flying qubits carried by itinerant photons is fundamental in quantum communication networks. To physically match the receiver system, the single photons must be prepared in proper shapes, and this leads to a variety of flying-qubit control problems. In this paper, we introduce the optimal control theory to the shaping of single photons, where the control to be optimized are coherent driving fields. We design gradient-based algorithms to minimize the shape difference between the emitted and desired single photons. Simulation results show that the optimization can achieve high-fidelity in the generation of decaying photon shapes with a two-level atom. However, its performance is limited and hence has to be combined with incoherence controls when the target shape has a rising part.

Keywords: Quantum information and control, Robust control, Lyapunov methods
Abstract: We investigate the impact of undesired Markovian couplings to external systems on the stabilization towards pure states or subspaces of continuously monitored quantum systems under open-loop and feedback control protocols. In particular, we provide sufficient conditions on perturbations to maintain stability of the target and demonstrate the boundedness in mean of the solutions of perturbed systems under open-loop protocols. Effect on the stability of feedback protocols is also analyzed. This work is a step toward a comprehensive robustness analysis of measurement-based control of quantum systems with respect to model perturbations.

Keywords: Quantum information and control, Reduced order modeling
Abstract: Consider an open quantum system governed by a Gorini–Kossakowski–Sudarshan–Lindblad master equation with two times-scales: a fast one, exponentially converging towards a linear subspace of quasi-equilibria; a slow one resulting from small decoherence and Hamiltonian dynamics. Usually adiabatic elimination is performed in the Schrödinger picture. We propose here a Heisenberg formulation where the invariant operators attached to the fast decay dynamics towards the quasi-equilibria subspace play a key role. Based on geometric singular perturbations, asymptotic expansions of the Heisenberg slow dynamics and of the fast invariant linear subspaces are proposed. They exploit Carr's approximation lemma from center-manifold and bifurcation theory. Second-order expansions are detailed and shown to ensure preservation, up to second-order terms, of the complete positivity for the slow propagator on a slow timescale. Such expansions can be exploited numerically to derive reduced-order dynamical models.

Keywords: Linear systems, Optimal control, Game theory
Abstract: In this paper, we consider the feedback representation of open-loop saddle points for irregular linear-quadratic (LQ) two-person zero-sum games, where the control weighting matrices in the cost functional are only semidefinite. The existence of an open-loop saddle point is characterized by the solvability of a system of constrained linear forward-backward differential equations (FBDEs), together with a convexity-concavity condition. In classical zero-sum games, the feedback representation is obtained by decoupling the FBDEs through the regular solution to a Riccati differential equation. But the associated Riccati equation cannot be used to decouple the FBDEs to obtain the feedback representation of open-loop saddle points. The essential differences between regular and irregular LQ zero-sum games are investigated. The irregular feedback representation can be derived from two equilibrium conditions in two different layers by using the ``two-layer optimization" approach.

Keywords: Optimization algorithms, Game theory, Variational methods
Abstract: We study generalized games with full row rank equality constraints and we provide a strikingly simple proof of strong monotonicity of the associated KKT operator. This allows us to show linear convergence to a variational equilibrium of the resulting primal-dual pseudo-gradient dynamics. Next, we propose a fully-distributed algorithm with linear convergence guarantee for aggregative games under partial-decision information. Based on these results, we establish stability properties for online GNE seeking in games with time-varying cost functions and constraints. Finally, we illustrate our findings numerically on an economic dispatch problem for peer-to-peer energy markets.

Keywords: Optimal control, Game theory, Intelligent systems
Abstract: We propose a hybrid approach that combines Hamilton-Jacobi (HJ) reachability and mixed-integer optimiza- tion for solving a reach-avoid game with multiple attackers and defenders. The reach-avoid game is an important problem with potential applications in air traffic control and multi- agent motion planning; however, solving this game for many attackers and defenders is intractable due to the adversarial nature of the agents and the high problem dimensionality. In this paper, we first propose an HJ reachability-based method for solving the reach-avoid game in which 2 attackers are playing against 1 defender; we derive the numerically convergent optimal winning sets for the two sides in environments with obstacles. Utilizing this result and previous results for the 1 vs. 1 game, we further propose solving the general multi-agent reach-avoid game by determining the defender assignments that can maximize the number of attackers captured via a Mixed Integer Program (MIP). Our method generalizes previous state- of-the-art results and is especially useful when there are fewer defenders than attackers. We validate our theoretical results in numerical simulations.

Keywords: Supervisory control, Game theory, Discrete event systems
Abstract: In recent years, there was an increasing interest in the connections between supervisory control theory and reactive synthesis. As the two fields use similar techniques there is great hope that technologies from one field could be used in the other. In this spirit, we provide an alternative reduction from the supervisor synthesis problem to solving B"uchi games via games with a non-blocking objective. Our reduction is more compact and uniform than previous reductions. As a consequence, it gives an asymptotically better upper bound on the time complexity of the supervisory control synthesis problem. Our reduction also breaks a widely held belief about the impossibility of reduction of supervisory control synthesis problem to a game with linear winning condition.