Keywords: Power electronics
Abstract: Battery management systems (BMSs) rely on empirical models, in the form of equivalent circuit models, thanks to their mathematical simplicity and low computational burden. However, empirical models undergo extensive calibration efforts, and they lack in transferability across chemistries. In addition, the inability to predict electrochemical internal states and account for degradation dynamics usually lead to ill usability of the battery system, possibly resulting in inaccurate state of health (SOH) estimations diverging over time. An advanced BMS design that can observe, and control internal variables of the battery system is imperative to overcome these limitations, enabling long-lasting, safer, and cost-effective battery systems for the fast-growing energy market. Physics-based battery models have been regarded as one of the appropriate modeling frameworks to be integrated into the next-generation BMS. In model-based estimation, available input/output sensor information (e.g., current, voltage, and temperature) are used along with a mathematical representation of the battery dynamics to estimate the internal states. The purpose of this tutorial paper is to review implementation challenges of physics-based battery models and provides an overview of the latest research trends focusing on numerical algorithms and observer designs for hardware implementation of physics-based battery models towards the advanced BMS.

Keywords: Power electronics
Abstract: This presentation will briefly explore some recent developments in the areas of lithium-sulfur (Li-S) battery modeling, parameter estimation, state estimation, and control. Widely recognized for their potential to provide attractive specific energy levels safely and inexpensively, Li-S batteries are governed by chains of multiple nonlinear redox and selfdischarge reaction dynamics. This makes it important to develop novel battery management algorithms specifically tailored for the unique characteristics of Li-S batteries, particularly in contrast to their more traditional lithium-ion counterparts. The presentation will highlight some of the progress, as well as some of the key challenges and opportunities, in this emerging research domain.

Keywords: Power electronics
Abstract: BMS require models of their cells to be able to compute estimates of state-of-charge, state-of-health, state-of-power, and state-of-energy. Physics-based models (PBMs) allow a BMS to control the cell to physical limits but it is difficult to estimate the parameter values for these PBMS. This tutorial describes a process whereby nondestructive lab tests can be used to estimate all identifiable PBM parameter values. First, the models must be reformulated to be identifiable. Second, lab tests are designed to isolate subsets of the parameters. Third, these tests are implemented and the lab data are regressed back to parameter estimates. Results from simulation and from tests performed on physical cells will be presented.

Keywords: Power electronics
Abstract: As demand grows for electrified propulsion systems, the ability to predict the response of battery cells in vehicle offers engineers a pathway to virtually test and calibrate controls systems, such as the Battery Management System (BMS). In support of General Motor’s traction battery efforts, a method has been derived to describe the electrochemical performance of a battery cell through the combination of a modified Newman Pseudo 2- Dimensional model and a three electrode apparatus. The model is typically applied to simulate the electrochemical and transport processes within a battery cell to predict the negative electrode potential and positive electrode potential with respect to a reference electrode. This offers the ability to consider the probability of lithium plating during a DCFC event virtually, and calibrate controls accordingly. Additionally, during a high rate charging or discharging event, significant heat generation in the battery cell is seen to occur, and virtually considering the impact of heat generation on the thermal management system and associated controls is necessary to drive robust design. In this discussion, we will give an overview of the electrochemical tools in use to drive virtual cell design considering the internal transport processes in the cell and associated heat generation, and offer an overview of typical outputs that can be considered for engineers focused on controls and calibration.

Keywords: Power electronics
Abstract: A Battery Management System (BMS) is an embedded system that monitors and controls batteries and the health its cells to deliver power to an electric vehicle (EV), Hybrid Electric Vehicle (HEV) or Plug-In Hybrid Electric Vehicle (PHEV). A BMS provides voltage, temperature and current monitoring, battery state of charge (SoC) and cell balancing of batteries, among other functions. A Hardware-In-the-Loop (HIL) test system for a BMS emulates the battery cells in a real-time environment, allowing test engineers and technicians to simulate faults (e.g., open circuits, short circuits, over/under-voltage, etc.), temperature variations (under/over-temperature), and isolation resistance in multiple test scenarios using different communication interfaces (e.g., LIN, CAN, FlexRay, etc.). For a deterministic execution, a real-time operating system (RTOS) can be used in conjunction with multi-core CPUs; for better performance, field-programmable gate arrays (FPGAs) can be used for both, parallelized real-time model simulation and high-speed communications (e.g., Xilinx® Aurora™). In this tutorial, we will discuss how to maximize the performance of BMS HIL test and/or PEHIL simulation systems using multi-core CPUs running RTOSes, FPGAs and COTS technologies, providing real-time, deterministic, and parallel code execution with sub nano-second resolution to support the design, prototype, and test of the next generation of EVs.

Keywords: Learning, Optimization, Markov processes
Abstract: This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example.

Keywords: Learning, Machine learning, Optimization
Abstract: In this paper, we investigate online convex optimization (OCO) with squared {l}_{2} norm switching cost, which has great applicability but very little work has been done on it. Specifically, we provide a new theoretical analysis in terms of dynamic regret and lower bounds for the case when loss functions are strongly-convex and smooth or only smooth. We show that by applying the advanced Online Multiple Gradient Descent (OMGD) and Online Optimistic Mirror Descent (OOMD) algorithms that are originally proposed for classic OCO, we can achieve state-of-the-art performance bounds for OCO with squared {l}_{2} norm switching cost. Furthermore, we show that these bounds match the lower bound.

Keywords: Machine learning, Power systems, Robotics
Abstract: Reinforcement learning is well-studied under discrete actions. Integer actions setting is popular in the industry yet still challenging due to its high dimensionality. To this end, we study reinforcement learning under integer actions by incorporating the Soft Actor-Critic (SAC) algorithm with an integer reparameterization. Our key observation for integer actions is that their discrete structure can be simplified using their comparability property. Hence, the proposed integer reparameterization does not need one-hot encoding and is of low dimensionality. Experiments show that the proposed SAC under integer actions is as good as the continuous action version on robot control tasks and outperforms Proximal Policy Optimization on power distribution systems control tasks.

Keywords: Game theory, Output regulation, H-infinity control
Abstract: Multiple leader and follower graphical games constitute challenging problems for aerospace and robotics applications. One of the challenges arises from having a different number of followers and leaders. We overcome this by employing an output containment error system that results in a formulation where outputs of all followers are proved to converge the convex hull spanned by the outputs of leaders. Another challenge is to design distributed Nash equilibrium control strategies for such games, which cannot be achieved with traditional quadratic cost functional formulation. Therefore, a modified cost functional that provides both Nash and distributed control strategies in the sense that each follower uses the state information of its own and neighbors, is presented. Furthermore, an mathcal{L}_2 gain bound of the output containment error system that experiences unbounded disturbances is investigated by a recently developed class of digraphs that results in a semi-simple pinned graph Laplacian.

Keywords: Delay systems, Distributed parameter systems
Abstract: This paper focuses on the prediction-based stabilization of a linear system subject to a random input delay.Modeling the delay as a finite-state Markov process, it proves that a constant time-horizon prediction enables robust compensation of the delay, provided the horizon prediction is sufficiently close to the delay values in average. Simulation results emphasize the practical relevance of this condition.

Keywords: Decentralized control, Game theory, Stochastic systems
Abstract: A static output feedback (SOF) strategy for robust incentive Stackelberg games with a large population for mean-field stochastic systems is investigated. First, the saddle point equilibrium condition of external disturbance and control strategy is derived based on stochastic algebraic matrix equations (SAMEs). Then, a centralized SOF incentive Stackelberg strategy is derived through restructuring the follower's strategies and the leader's incentive strategy. Moreover, to avoid the high dimension of design procedure, a new designing algorithm of low-dimensional approximation SOF incentive Stackelberg strategy is proposed. The difference in the equilibrium values between using the centralized SOF incentive Stackelberg strategy and using the low-dimensional approximation SOF incentive Stackelberg strategy is proved. Finally, a numerical example with a large population size demonstrates the effectiveness of the proposed approximation SOF incentive Stackelberg strategy.

Keywords: Networked control systems, Cooperative control, Distributed control
Abstract: When a team of agents, such as unmanned aerial/underwater vehicles, are operating in 3-dimensional space, their coordinated action in pursuit of a cooperative task generally requires all agents to either share a common coordinate frame or know the orientations of their coordinate axes with regard to the global coordinate frame. Given the coordinate axes that are initially unaligned, this work proposes an orientation alignment scheme for multiple agents with a type of leader-following graph typologies using only inter-agent directional vectors, and the direction measurements to one or more landmarks of the first two agents. The directional vectors are expressed in the agents' body-fixed coordinate frames and the proposed alignment protocol works exclusively with the directional vectors without the need of a global coordinate frame common to all agents or the construction of the agents' orientation matrices. Under the proposed alignment scheme, the orientations of the agents converge almost globally and asymptotically to the orientation of the leader agent. Finally, numerical simulations are also given to illustrate the effectiveness of the proposed method.

Keywords: Networked control systems, Decentralized control, Cooperative control
Abstract: This paper presents an alternative approach to the study of distance rigidity in networks of mobile agents, based on a subframework scheme. The advantage of the proposed strategy lies in expressing framework rigidity, which is inherently global, as a set of local properties. Also, we show that a framework's normalized rigidity eigenvalue degrades as the graph's diameter increases. Thus, the rigidity eigenvalue associated to each subframework arise naturally as a local rigidity metric. A decentralized subframework-based controller for maintaining rigidity using only range measurements is developed, which is also aimed to minimize the network's communication load. Finally, we show that the information exchange required by the controller is completed in a finite number of iterations, indicating the convenience of the proposed scheme.

Keywords: Cooperative control, Networked control systems, Autonomous robots
Abstract: We consider the problem of collectively transporting multiple objects by multiple agents. The objective is to find the optimal matching between the objects and agents that minimizes the energy of the overall system. We show that combining a proximal gradient method with continuous relaxation yields a distributed algorithm which converges to a near-optimal solution for the associated optimization problem. Furthermore, by using this solution as an initial solution, a distributed negative-cycle canceling algorithm, which monotonically decreases the matching cost at each step, provides the globally optimal solution for the problem. Numerical simulations demonstrate the performance on practical problems.

Keywords: Network analysis and control, Networked control systems, Distributed control
Abstract: In this paper, we study the problem of resilient average consensus for second-order multi-agent systems with misbehaving agents. General types of misbehaviors are considered, including false data injection attacks and accidental faults. The difficulties of this problem are to detect errors in a distributed way and accurately compensate two-dimension state errors by one-dimension acceleration input. We first provide sufficient conditions for second-order average consensus. Then, we design detection methods via two-hop communication information and propose schemes to compensate errors accurately in a distributed way inspired by sufficient conditions. Hence, we propose a finite input-errors detection-compensation-based consensus algorithm (FIDC). Considering infinite attacks on input, velocity and position, an extension named IADC is proposed with a fault-tolerance mechanism. We prove that the proposed algorithms allow agents to asymptotically achieve second-order average consensus. Finally, extensive simulations are conducted to verify the effectiveness of the algorithms.

Keywords: Network analysis and control, Optimization algorithms, Uncertain systems
Abstract: Network reliability, the probability that the connectivity between given nodes holds in a stochastic network, is one of the most fundamental metrics in infrastructure networks, and many methods have been developed to evaluate it for a given network. In reality, however, to know the network connectivity more definitively, partial observation of the network is often performed, and depending on its results, the estimation accuracy of the connectivity greatly changes. In this paper, we discuss the problem of finding an optimal subset of edges whose states should be observed to accurately know the connectivity of the network. In this problem, we aim at decreasing the uncertainty of connectivity, in other words, bringing the network reliability closer to 0 or 1 by performing an appropriate partial observation. When the observable edges are cost-constrained, the problem is formulated as a combinatorial optimization and can be shown to be NP-hard. To solve this problem, we propose a heuristic algorithm that combines the cross entropy method, the Monte Carlo method, and the network reliability evaluation method via binary decision diagrams (BDD). To verify the effectiveness of the proposed algorithm, we conducted numerical evaluations using open datasets of network topology, demonstrating that the proposed method outperforms the baseline method for most cases and is robust in terms of parameter settings. As a case study, we also applied the proposed method to the problem of installing surveillance cameras in a disaster-prone road network and conducted numerical experiments.

Keywords: Control of networks, Linear systems, Agents-based systems
Abstract: In this paper, the directed threshold graphs (DTGs) and their Laplacian controllability issues are studied. The graphs are constructed by a sequence of graph operations that include `union' and `directed join', where the directed join means in adding a new node, the direction of the new edges are either from the new node to all existing ones or the other way around. It is shown that the Laplacian spectra of the graphs can be readily identified and Laplacian eigenspaces fully characterized, and thus the binary control matrices can be determined to render the graphs Laplacian controllable. In particular, simple algorithms are proposed to facilitate the Laplacian eigenspace analysis and to select the nodes to which the minimum number of controllers can be connected to ensure the Laplacian controllability. Examples are provided to illustrate the proposed results.

Keywords: Distributed parameter systems, Network analysis and control, Control of networks
Abstract: In this paper we study the leader-synchronization problem for a class of partial differential equations with boundary control and textit{in-domain} coupling. We describe the problem in an abstract formulation and then we apply the results to a network of parabolic PDEs. In particular, a subset of the followers is connected to the leader through a boundary control, while interconnection among the followers is enforced by distributed couplings inside the domain. The analysis of this framework is based on matrix inequalities that give sufficient conditions for the selection of the control parameters in order to attain synchronization. Last, some numerical examples are reported to illustrate and corroborate the theoretical findings.

Keywords: Linear systems, Adaptive control, Estimation
Abstract: This paper studies the output regulation problem for discrete-time linear systems subject to an uncertain exosystem. The design involves two online estimation algorithms. The first one is to estimate the unknown parameters of the exosystem, and the second one is to estimate the solution to the regulator equations based on the estimated system matrix of the exosystem. Combining these two algorithms gives rise to an iterative solution that converges exponentially to some exact solution to the regulator equations. Finally, by integrating these two algorithms with the feedforward control approach, both dynamic state feedback and dynamic output feedback control laws are synthesized to solve the problem.

Keywords: Distributed control, Communication networks, Optimization algorithms
Abstract: There has been an approximation-based distributed optimization algorithm that solves univariate non-convex problems to arbitrary precision. The key idea is to construct approximations of local objectives and address a more structured approximate version of the problem. By representing diverse local objectives with compressed coefficients vectors of approximations, such algorithms enjoy gradient-free iterations but face severe security issues when adversaries occur. In this paper, we propose a resilient approximation-based distributed nonconvex optimization algorithm termed R-ADOA to defend attacks from malicious nodes. First, errors caused by adversaries are quantified and unified as the perturbation of coefficient vectors of approximations. Next, we propose a filtering mechanism and resilient stopping mechanism to limit errors arising in consensus-based iterations. Finally, an upper bound of the deviations of the obtained solutions from optimal solutions is given based on the eigenvalue perturbation theory of matrices. Numerical experiments are provided to illustrate the effectiveness of our algorithm. Compared to existing resilient distributed optimization algorithms, R-ADOA addresses non-convex problems, converges exponentially fast, and contains explicit bounds for the deviations of solutions.

Keywords: Large-scale systems, Optimal control, Predictive control for nonlinear systems
Abstract: We consider the optimal control problem of steering an agent population to a desired distribution over an infinite horizon. This is an optimal transport problem over a dynamical system, which is challenging due to its high computational cost. In this letter, we propose Sinkhorn MPC, which is a dynamical transport algorithm combining model predictive control and the so-called Sinkhorn algorithm. The notable feature of the proposed method is that it achieves cost-effective transport in real time by performing control and transport planning simultaneously. In particular, for linear systems, we reveal the fundamental properties of Sinkhorn MPC such as ultimate boundedness and asymptotic stability.

Keywords: Discrete event systems, Automata
Abstract: In this paper, we investigate the verification of detectability, a fundamental state estimation property, for partially-observed discrete event systems (DES). Existing works on this topic mainly focus on untimed DES. In many applications, however, real-time information is critical for the purpose of system analysis. To this end, in this paper, we investigate the verification of detectability for timed DES modeled by time automata. Two notions of detectability, strong detectability and weak detectability, are studied in the dense-time setting, which characterize detectability by time elapsing rather than events steps. We show that verifying strong detectability for timed system is decidable by providing a verifiable necessary and sufficient condition. Furthermore, we show that weak detectability is undecidable in the timed setting by reducing the language universality problem for timed automata to this verification problem. Our results extend the detectability analysis of DES from the untimed setting to a timed setting.

Keywords: Cooperative control
Abstract: Consensus in multi-agent dynamical systems is prone to be sabotaged by the adversary, which has attracted much attention due to its key role in broad applications. In this paper, we study a new false data injection (FDI) attack design problem, where the adversary with limited capability aims to maximize the consensus convergence error by compromising a subset of agents and manipulating their local high-dimensional states. We first formulate the FDI attack design problem as a combinatorial optimization problem, which is NP-hard. Then, based on the submodularity optimization theory, we provide the necessary and sufficient conditions to guarantee the submodularity of the objective function, which satisfies the property of diminishing marginal returns. In other words, the profit of adding an extra agent to the compromised set decreases as that set becomes larger. With this property, we exploit the greedy scheme which aims to find the optimal compromised agent set to produce the maximum convergence error when adding one extra agent to that set each time. Thus, an FDI attack selection algorithm is formed to obtain the near-optimal subset of the compromised agents. Furthermore, we derive the analytical suboptimality bounds and the worst-case running time under the proposed algorithm. Extensive simulation results are conducted to show the effectiveness of the proposed algorithm.

Keywords: Automotive control, Optimal control
Abstract: In this paper, a computationally efficient two-path non-linear optimal control allocation method is proposed to improve the yaw stability of four-wheel-independent-steering, four-wheel-drive vehicles. The virtual controller output is allocated using an optimization problem to compute each wheel's steering and traction commands at every controller time step. The optimization problem is solved by running a sequential quadratic programming (SQP) procedure, which may take some time to obtain satisfactory results. The proposed two-path control structure is derived from a more complex single-path allocation problem where torque allocation and steering correction optimal solutions are calculated concurrently. In this separated two-path control structure, computational load due to the complexity of the single block problem is reduced. In real applications, each problem can be run in parallel on different controllers of the vehicle controller network, which decreases the execution time with near-optimal results. The performance and speed comparisons of both approaches are studied using detailed vehicle simulations.

Keywords: Distributed control, Cooperative control
Abstract: A design framework recently has been developed to stabilize interconnected multiagent systems in a distributed manner, and systematically capture the architectural aspect of cyber-physical systems. Such a control theoretic framework, however, results in a stabilization protocol which is passive with respect to the cyber attacks and conservative regarding the guaranteed level of resiliency. We treat the control layer topology and stabilization gains as the degrees of freedom, and develop a mixed control and cybersecurity design framework to address the above concerns. From a control perspective, despite the agent layer modeling uncertainties and perturbations, we propose a new step-by-step procedure to design a set of control sublayers for an arbitrarily fast switching of the control layer topology. From a proactive cyber defense perspective, we propose a satisfiability modulo theory formulation to obtain a set of control sublayer structures with security considerations, and offer a frequent and fast mutation of these sublayers such that the control layer topology will remain unpredictable for the adversaries. We prove the robust input-to-state stability of the two-layer interconnected multiagent system, and validate the proposed ideas in simulation.

Keywords: Automotive control, Estimation, Autonomous vehicles
Abstract: The tire-road friction coefficient conveys critical information for advanced vehicular active safety systems to significantly enhance driving safety and maneuverability. In this paper, a look-up table based method is proposed to estimate the tire-road friction coefficient of each driving wheel in real time. The table that represents the relations among the normalized longitudinal tire force, tire slip ratio and slip angle, and the tire-road friction coefficient is constructed off-line by collecting data from common onboard sensors under different driving scenarios. By applying the perspective projection procedure, only data from two different road surfaces are required, which facilitates table construction. Then the table is used for estimating the tire-road friction coefficient in real time and the results are post-processed by the Kalman filter to render smooth and reliable estimates. Simulations in the CarSim-Simulink environment are conducted to verify the satisfactory performance of the proposed estimation method for various driving scenarios and sudden changes of road conditions.

Keywords: Traffic control, Optimal control, Predictive control for linear systems
Abstract: In large cities, metro lines are often saturated and impacted by sudden events to the point that some stations in the network become overcrowded and multiple trains are seriously delayed, causing the increase of passengers' waiting time. This disservice can be reduced by rescheduling the metro traffic and by adding backup trains in storage lines to be used when the service level largely decreases. In this paper, a novel control strategy, called Integrated Model Predictive Control, is proposed that combines both timetable rescheduling and backup trains allocation. In particular, a state-space model is adopted to describe the evolution of the train traffic dynamics and the model predictive control method is applied to obtain an optimal controller such that the rescheduled departure time and headway deviations from the nominal timetable are minimized. In the case where no feasible controller exists because of extensive delays, we propose an event-triggered process to automatically add backup trains into the operation plan such that the timetable can recover quickly. Finally, numerical examples are provided to demonstrate the effectiveness of the proposed control method.

Keywords: Power systems, Optimization algorithms
Abstract: Existing algorithms to solve alternating-current optimal power flow (AC-OPF) often exploit linear approximations to simplify system models and accelerate computations. In this paper, we improve a recent hierarchical OPF algorithm, which rested on primal-dual gradients evaluated in a linearized distribution power flow model. Specifically, we identify a risk of voltage violation arising from the model linearization, and propose a more accurate gradient evaluation method to eliminate that risk. We further develop a hierarchical primal-dual algorithm to solve OPF based on the proposed gradient evaluation method. Numerical results on IEEE networks show that our algorithm can enhance voltage safety with satisfactory computational efficiency.

Keywords: Optimization, Flight control
Abstract: We solve the minimum-thrust optimal trajectory generation problem for the transition of a tiltwing Vertical Take-Off and Landing (VTOL) aircraft using convex optimisation. The method is based on a change of differential operator that allows us to express the simplified point-mass dynamics along a prescribed path and formulate the original nonlinear problem in terms of a pair of convex programs. A case study involving the Airbus A3 Vahana VTOL aircraft is considered for forward and backward transitions. The presented approach provides a fast method to generate a safe optimal transition for a tiltwing VTOL aircraft that can further be leveraged online for control, and guidance purposes.

Keywords: Fault detection, Manufacturing systems, Markov processes
Abstract: With the advent of industry 4.0, machine learning (ML) methods have mainly been applied to design condition-based maintenance strategies to improve the detection of failure precursors and forecast degradation. However, in real-world scenarios, relevant features unraveling the actual machine conditions are often unknown, posing new challenges in addressing fault diagnosis problems. Moreover, ML approaches generally need ad-hoc feature extractions, involving the development of customized models for each case study. Finally, the early substitution of key mechanical components to avoid costly breakdowns challenge the collection of sizable significant data sets to train fault detection (FD) systems. To address these issues, this paper proposes a new unsupervised FD method based on double deep-Q network (DDQN) with prioritized experience replay (PER). We validate the effectiveness of the proposed algorithm on real data obtained from a steel plant placed in the south of Italy. Lastly, we compare the performance of our method with other FD methods showing its viability.

Keywords: Machine learning, PID control, Process Control
Abstract: Many industrial processes utilize proportional-integral-derivative (PID) controllers due to their practicality and often satisfactory performance. The proper controller parameters depend highly on the operational conditions and process uncertainties. This dependence brings the necessity of frequent tuning for real-time control problems due to process drifts and operational condition changes. This study combines the recent developments in computer sciences and control theory to address the tuning problem. It formulates the PID tuning problem as a reinforcement learning task with constraints. The proposed scheme identifies an initial approximate step-response model and lets the agent learn dynamics off-line from the model with minimal effort. After achieving a satisfactory training performance on the model, the agent is fine-tuned on-line on the actual process to adapt to the real dynamics, thereby minimizing the training time on the real process and avoiding unnecessary wear, which can be beneficial for industrial applications. This sample efficient method is applied to a pilot-scale multi-modal tank system. The performance of the method is demonstrated by setpoint tracking and disturbance regulatory experiments.

Keywords: Traffic control, Agents-based systems, Intelligent systems
Abstract: We propose an event-triggered framework for deciding the traffic light at each lane in a mixed autonomy scenario. We deploy the decision after a suitable delay, and events are triggered based on the satisfaction of a predefined set of conditions. We design the trigger conditions and the delay to increase the vehicles' throughput. This way, we achieve full exploitation of autonomous vehicles (AVs) potential. The ultimate goal is to obtain vehicle-flows led by AVs at the head. We formulate the decision process of the traffic intersection controller as a deterministic delayed Markov decision process, i.e., the action implementation and evaluation are delayed. We propose a Reinforcement Learning based model-free algorithm to obtain the optimal policy. We show - by simulations - that our algorithm converges, and significantly reduces the average wait-time and the queues length as the fraction of the AVs increases. Our algorithm outperforms our previous work cite{salvato2021control} by a quite significant amount.

Keywords: Neural networks, Supervisory control, Nonholonomic systems
Abstract: The paper proposes a novel training approach for a neural network to perform switching among an array of computationally generated stochastic optimal feedback controllers. The training is based on the outputs of off-line computed lookup-table metric (LTM) values that store information about individual controller performances. Our study is based on a problem of bicycle kinematic model navigation through a sequence of gates and a more traditional approach to the training is based on kinematic variables (KVs) describing the bicycle-gate relative position. We compare the LTM and KV based training approaches to the navigation problem and find that the LTM training has a faster convergence with less variations than the KV based training. Our results include numerical simulations illustrating the work of the LTM trained neural network switching policy.

Keywords: Optimization, Optimization algorithms, Learning
Abstract: We study the dynamic regret in online convex optimization (OCO), where the cost functions are revealed sequentially over time. Prior studies on the dynamic regret of OCO algorithms often require the cost functions to be Lipschitz continuous. However, the costs functions that arise in many applications may not satisfy this condition. In this work, we analyze the performance of Online Multiple Mirror Descent (OMMD), which can handle non-Lipschitz cost functions. OMMD is based on mirror descent but uses multiple mirror descent steps per online round. We first derive two upper bounds on the dynamic regret based on the path length and squared path length, and we further derive a third upper bound based on the cumulative optimal cost, which can be much smaller than the path length or squared path length especially when the sequence of minimizers fluctuates over time. We show that the dynamic regret of OMMD scales linearly with the minimum among the path length, squared path length, and cumulative optimal cost.

Keywords: Optimization algorithms, Statistical learning, Machine learning
Abstract: In this work, multiplicative stochasticity is applied to the learning rate of stochastic optimization algorithms, giving rise to stochastic learning-rate schemes. In-expectation theoretical convergence results of Stochastic Gradient Descent equipped with this novel stochastic learning rate scheme under the stochastic setting, as well as convergence results under the online optimization settings are provided. Empirical results consider the case of an adaptively uniformly distributed multiplicative stochasticity equipped with a stochastic learning rate.

Keywords: Hybrid systems, Stochastic systems, Formal verification/synthesis
Abstract: This work is concerned with the safety controller synthesis of stochastic hybrid systems, in which continuous evolutions are described by stochastic differential equations with both Brownian motions and Poisson processes, and instantaneous jumps are governed by stochastic difference equations with additive noises. Our proposed framework leverages the notion of control barrier certificates (CBC), as a discretization-free approach, to synthesize safety controllers for stochastic hybrid systems while providing safety guarantees in finite time horizons. In our proposed scheme, we first provide an augmented framework to characterize each stochastic hybrid system containing continuous evolutions and instantaneous jumps with a unified system covering both scenarios. We then introduce an augmented control barrier certificate (ACBC) for augmented systems and propose sufficient conditions to construct an ACBC based on CBC of original hybrid systems. By utilizing the constructed ACBC, we quantify upper bounds on the probability that the stochastic hybrid system reaches certain unsafe regions in a finite time horizon. The proposed approach is verified over a nonlinear case study.

Keywords: Discrete event systems, Automata
Abstract: This paper considers the problem of optimal sensor schedule for remote state estimation of discrete-event systems. In this setting, the sensors observes events from the plant transmits them to the receiver or estimator selectively. The transmission mechanism decides to transmit or not to transmit the observed information, according to an information transmission policy and the receiver needs to have sufficient information for the purpose of decision making. To solve this problem, we first constructed a non-deterministic dynamic observer that contains all feasible information transmission policies. Then we show that the information updating rule of the dynamic observer indeed yields the state estimate from the receiver's point of view. Finally, we propose an approach to extract a specific information transmission policy, realized by a finite-state automaton, from the dynamic observer while satisfying some desired observation property. To reduce transmission-related costs, we also require that the sensors transmit events as less as possible. A running example is provided to illustrate the proposed procedures.

Keywords: Mechatronics, Hybrid systems
Abstract: According to the well-known loop-shaping control design approach, the steady-state precision of control systems can be improved by stacking integrators. However, due to the waterbed effect in linear control systems, such an action will worsen the transient response by increasing overshoot and creating wind-up problems. This paper presents a new architecture for rest control systems that can significantly decrease the overshoot and create a no-overshoot performance even in the presence of stacked integrators. The steady-state analysis of the proposed system will also show that improved precision expected due to stacked integrators can be achieved as well. A numerical simulation study is presented to verify the results, and the tuning guide is presented.

Keywords: Hybrid systems, Embedded systems, Discrete event systems
Abstract: The notion of opacity refers to a condition that external observers cannot distinguish between secret and nonsecret states, which is important for security of a plant. On the other hand, the authors have recently proposed a ”symbolic control barrier function” that enforces forward invariance on the plant with its finite abstracted model. In this paper, we propose symbolic synthesis achieving both desired behaviors and approximated initial-state opacity. First, a symbolic controller that enforces a desired behavior is synthesized without considering the opacity. Next, we eliminate transitions violating a condition on the opacity, which is induced by the symbolic control barrier function. Finally, it is proved that the controlled plant satisfies the opacity if a deadlock-free sub-transition system is obtained. The proposed method is illustrated with a numerical example on tank volume control.

Keywords: Discrete event systems
Abstract: The analysis of infinite-step opacity in the context of stochastic discrete-event systems has been investigated as almost-infinite-step opacity for quantitative purposes. In this paper, we revisit the verification problem for almost-infinitestep opacity by concentrating on reducing its computational complexity. One of the key steps in the verification of almostinfinite-step opacity is the recognition of the unsafe language for infinite-step opacity. Inspired by recently developed techniques in the literature, we present an improved methodology for the calculation of the unsafe language for infinite-step opacity, which further improves the complexity of the verification of almost-infinite-step opacity.

Keywords: Discrete event systems, Estimation
Abstract: The cyber-layer of a cyber-physical system can be regarded as a discrete-event system that can be fully implemented in a computer, which monitors the working status of physical systems (usually differential equations) or allocates targets to physical systems. A labeled finite-state automaton (LFSA) can be seen as such a monitor and the control properties in LFSAs have been widely studied, where detectability is one of the fundamental inference-based control properties describing whether one could use output sequences to determine the current and subsequent states. In this paper, we study how an attacker affects strong detectability and weak detectability of an LFSA. We formulate an attacker as a K-memory attacker, which can remember and distort K-length output sequences generated by an LFSA. We give a systematic method to verify when this type of attackers affect detectability of an LFSA.

Keywords: Time-varying systems, Human-in-the-loop control, Nonlinear systems identification
Abstract: Control barrier functions (CBFs) have been yielding successful implementations in various control strategies. In this paper, we propose a human assist controller using a zeroing CBF (ZCBF). To ensure the forward-completeness of every solution to a nonautonomous system, we provide the “strict" definition of a ZCBF. Then, we obtain a global condition for a ZCBF from the strict conditions. We simultaneously prove that the proposed controller ensures the forward-completeness and the safety of a nonautonomous system. The effectiveness of the proposed controller using a strict ZCBF is confirmed by a mathematical example.

Keywords: Human-in-the-loop control, Stability of nonlinear systems, Adaptive systems
Abstract: This paper demonstrates the stability limits of a human-in-the-loop closed loop control system, where the plant to be controlled has redundant actuators with uncertain dynamics. The human operator is modeled as a general transfer function, unlike earlier work that consider redundant actuators where specific filters are associated with certain human reactions. This helps with developing a more general stability analysis, and earlier studies can be considered as special cases of the proposed framework in this paper. Adaptive control allocation is employed to distribute control signals among redundant actuators. A sliding mode controller with a time-varying sliding surface provides desired control inputs to the control allocator. A flight control task, where the pilot controls the pitch angle via a pitch rate stick input is simulated to demonstrate the accuracy of the stability analysis. The Aerodata Model in Research Environment is used as the uncertain, over-actuated aircraft model.

Keywords: Control applications, Spacecraft control
Abstract: This article considers the 3D visual coverage problem of an asteroid by establishing a stereo-visual based sensor from monocular cameras. The cameras are individually mounted on two small low-thrust spacecraft, which are maintained in a tight stereo formation using a nonlinear model predictive control scheme to maintain an overlapping field-of-view and a specific baseline distance. The proposed control algorithm adopts a leader-follower architecture to define the relative pose of the spacecraft. However, asteroids provide a challenging environment for such missions due to their rotating nature, irregular shaped bodies with a low (micro) but irregular gravity field. Moreover, the coupling between the orbital and attitude dynamics of the spacecraft need to be accounted. As a result, this article considers a full-state nonlinear control approach to plan correction maneuvers to maintain the desired pose of the spacecraft. The efficiency of the proposed control scheme is demonstrated with a realistic simulation scenario where the results are also visualized in the Gazebo simulation environment.

Keywords: Control applications, Optimal control, Stochastic optimal control
Abstract: Scientific experiments conducted with the help of a particle accelerator rely on the stability of the corresponding particle beam. One way of improving this stability is the application of a control method called beam-based feedback. Following this, the control system of the linear accelerator ELBE is planned to be upgraded by this feedback mechanism. This paper exploits the continuous wave operation mode of ELBE in order to reinterpret the given control problem as a disturbance rejection goal. In this context, the paper studies the influence of stochastic disturbances on electron beam arrival time, proposes and designs a control system that is capable of effectively compensating the arrival time fluctuations. Moreover, the prospective hardware implementation is taken into account when choosing the most suitable control strategy for this specific task. Finally, the simulation of the designed regulator on measured disturbance data indicates an improvement in arrival time performance.

Keywords: Fault detection, Biomedical, Networked control systems
Abstract: The artificial pancreas is an emerging concept of closed-loop insulin delivery that aims to tightly regulate the blood glucose levels in patients with type 1 diabetes. This paper considers bias injection attacks on the glucose sensor deployed in an artificial pancreas. Modern glucose sensors transmit measurements through wireless communication that are vulnerable to cyber-attacks, which must be timely detected and mitigated. To this end, we propose a model-based anomaly detection scheme using a Kalman filter and a χ2 test. One key challenge is to distinguish cyber-attacks from large unknown disturbances arising from meal intake. This challenge is addressed by an online meal estimator and a novel time-varying detection threshold. More precisely, we show that the ordinary least squares is the optimal unbiased estimator of the meal size under certain modelling assumptions. Moreover, we derive a novel time-varying threshold for the χ2 detector to avoid false alarms during meal ingestion. The results are validated by means of numerical simulations.

Keywords: PID control, Process Control
Abstract: The gain crossover frequency and the phase margin are the most common design specifications in robust controller design. An additional flat phase constraint is proposed to guarantee the system robustness under gain variations, where the phase of the desired loop transfer function is flat around the gain crossover frequency. This controller design methodology is successfully implemented on integer-order proportional-integral-derivative (IOPID) controllers and fractional-order proportional-integral (FOPI) controllers in a class of first-order plus time-delay (FOPTD) systems. However, it is not easy to apply the flat phase constraint on general minimum-phase systems since the derivative of the loop transfer function with respect to frequency may be difficult to deduce. A flat phase design approach based on the Bode's integral is proposed for general minimum phase systems which does not need the system model information. An FOPI controller and an IOPID controller are designed in numerical examples to validate the effectiveness of the proposed robust controller design method.