Keywords: Learning, Machine learning, Optimization
Abstract: In this paper, we investigate online convex optimization (OCO) with squared {l}_{2} norm switching cost, which has great applicability but very little work has been done on it. Specifically, we provide a new theoretical analysis in terms of dynamic regret and lower bounds for the case when loss functions are strongly-convex and smooth or only smooth. We show that by applying the advanced Online Multiple Gradient Descent (OMGD) and Online Optimistic Mirror Descent (OOMD) algorithms that are originally proposed for classic OCO, we can achieve state-of-the-art performance bounds for OCO with squared {l}_{2} norm switching cost. Furthermore, we show that these bounds match the lower bound.

Keywords: Learning, Machine learning, Reduced order modeling
Abstract: In this work a complexity reduction methodology is proposed for a data-driven Switched Auto-Regressive eXogenous (SARX) model identification algorithm based on Regression Trees. In particular, we aim at reducing the number of submodels of a SARX dynamical model without compromising (and indeed improving) the model accuracy, and mitigating the overfitting problem. A validation procedure is addressed to compare the performance of the reduced model with respect to the original one. Results show an important reduction in the number of modes of the identified model that ranges between 96% and 99.74%. The accuracy of the reduced model is also tested in terms of closed-loop control performance in a Model Predictive Control (MPC) setup, on a benchmark consisting of a non-linear inverted pendulum on a cart: the comparison is provided with respect to an oracle, i.e. an MPC setup with perfect knowledge of the plant dynamics.

Keywords: Learning, Optimal control, Robust control
Abstract: This paper presents a parameterization of nonlinear controllers for uncertain systems building on a recently developed neural network architecture, called the recurrent equilibrium network (REN), and a nonlinear version of the Youla parameterization. The proposed framework has “built-in” guarantees of stability, i.e., all policies in the search space result in a contracting (globally exponentially stable) closed-loop system. Thus, it requires very mild assumptions on the choice of cost function and the stability property can be generalized to unseen data. Another useful feature of this approach is that policies are parameterized directly without any constraints, which simplifies learning by a broad range of policy-learning methods based on unconstrained optimization (e.g. stochastic gradient descent). We illustrate the proposed approach with a variety of simulation examples.

Keywords: Learning, Optimization, Markov processes
Abstract: This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example.

Keywords: Learning, Subspace methods, Adaptive systems
Abstract: Humans are capable of adjusting to changing environments flexibly and quickly. Empirical evidence has revealed that representation learning plays a crucial role in endowing humans with such a capability. Inspired by this observation, we study representation learning in the sequential decision-making scenario with contextual changes. We propose an online algorithm that is able to learn and transfer context-dependent representations and show that it significantly outperforms the existing ones that do not learn representations adaptively. As a case study, we apply our algorithm to the Wisconsin Card Sorting Task, a well-established test for the mental flexibility of humans in sequential decision-making. By comparing our algorithm with the standard Q-learning and Deep-Q learning algorithms, we demonstrate the benefits of adaptive representation learning.

Keywords: Learning
Abstract: In this work we consider the online control of a known linear dynamic system with adversarial disturbance and adversarial controller cost. The goal in online control is to minimize the regret, defined as the difference between cumulative cost over a period T and the cumulative cost for the best policy from a comparator class. For the setting we consider, we generalize the previously proposed online Disturbance Response Controller (DRC) to the adaptive gradient online Disturbance Response Controller. Using the modified controller, we present novel regret guarantees that improves the established regret guarantees for the same setting. We show that the proposed online learning controller is able to achieve intermediate intermediate regret rates between sqrt{T} and log{T} for intermediate convex conditions, while it recovers the previously established regret results for general convex controller cost and strongly convex controller cost.

Keywords: Networked control systems, Cooperative control, Distributed control
Abstract: When a team of agents, such as unmanned aerial/underwater vehicles, are operating in 3-dimensional space, their coordinated action in pursuit of a cooperative task generally requires all agents to either share a common coordinate frame or know the orientations of their coordinate axes with regard to the global coordinate frame. Given the coordinate axes that are initially unaligned, this work proposes an orientation alignment scheme for multiple agents with a type of leader-following graph typologies using only inter-agent directional vectors, and the direction measurements to one or more landmarks of the first two agents. The directional vectors are expressed in the agents' body-fixed coordinate frames and the proposed alignment protocol works exclusively with the directional vectors without the need of a global coordinate frame common to all agents or the construction of the agents' orientation matrices. Under the proposed alignment scheme, the orientations of the agents converge almost globally and asymptotically to the orientation of the leader agent. Finally, numerical simulations are also given to illustrate the effectiveness of the proposed method.

Keywords: Networked control systems, Cooperative control, Sensor networks
Abstract: This paper presents a coverage controller that allows a team of robots with qualitatively different mobility types to achieve an effective coverage over a domain consisting of multiple terrain types. In this work, each robot has specific types of terrains that it can operate on, and these compatible operating regions of a robot are determined based on its mobility type and current position. With the extended definition of coverage from sensing to servicing of events, the Voronoi cells of the robots are confined to their compatible operating regions. Based on the new Voronoi cells, the mobility constrained locational cost is defined, and a gradient descent controller is used to drive robots to locally optimal coverage positions. In order to keep each robot within its compatible regions while achieving a coverage, control barrier functions are utilized.

Keywords: Traffic control, Cooperative control, Robust control
Abstract: In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection to incorporate uncertainty. Using the possibly noisy observations of actual time trajectories and leveraging Gaussian process regression, we learn the bounded confidence intervals for deviations from the nominal trajectories of CAVs online. Incorporating these confidence intervals, we reformulate the trajectory planning as a robust coordination problem, the solution of which guarantees that constraints in the system are satisfied in the presence of bounded deviations from the nominal trajectories. We demonstrate the effectiveness of our extended framework through a numerical simulation.

Keywords: Networked control systems, Distributed control, Autonomous systems
Abstract: A method is developed for bootstrapping a provided navigation field for a single mobile point agent in a compact obstructed environment to form a distributed controller for a leader-follower task in a multi-agent system (MAS) with distance-limited communications.

Under certain restrictions on the MAS communication radius in relation to the geometry of obstacles, the proposed MAS controller guarantees a chain of agents remains connected throughout time, while following a leader agent as the latter converges to a specified target along an integral curve of the provided navigation field.

In contrast with existing work, the standard radial interactions among agents are replaced with an appropriately rescaled version of the provided navigation field, removing the need for strong simplifying assumptions (e.g., convex or spherical obstacles).

As a result, this approach showcases the trade-offs between communication range, the size of the MAS, and the complexity of navigating in the provided environment.

The reliance of the approach on navigation fields also enables using sensing-based reactive navigation tools designed for settings with incomplete prior knowledge of the environment.

Keywords: Agents-based systems, Cooperative control, Networked control systems
Abstract: Interaction topology through which agents achieve collaboration in multi-agent systems is of fundamental importance. Recently, many efforts have been devoted to the problem of topology inference, e.g., the trajectory information of mobile agents is utilized to regress the topology. In this paper, we develop a distributed topology-preserving collaboration algorithm for multi-agent systems against the topology inference attacks. The novelties lie in that: i) By adding well-designed noises to the system states, the irregularity of the state evolution is largely enhanced, making the underlying topology hard to be inferred accurately from the observations over the system; ii) By dividing the added noises into the random and the disturbing terms with mutual compensation properties, the proposed algorithm guarantees the convergence of the system state, which applies to both undirected and directed topology structures. Specifically, the mean square convergence rate and the non-asymptotic error bound are derived. Extensive simulations are conducted to illustrate the effectiveness of our algorithm.

Keywords: Networked control systems, Network analysis and control, Control of networks
Abstract: We develop a framework to assess the risk of cascading failures when a team of agents aims to rendezvous in time in the presence of exogenous noise and communication time-delay. The notion of value-at-risk (VaR) measure is adopted to quantify the risk of an agent failing to rendezvous with other agents when it is given that another agent has experienced a large fluctuation. It is shown that the risk of cascading failures depends on the Laplacian spectrum of the underlying communication graph, time-delay, and noise statistics. Furthermore, we exploit the structure of several standard graphs to show how the cascading risk behaves differently for various graph topologies. Finally, our theoretical results are supported by several simulation case studies.

Keywords: Identification, Numerical algorithms, Large-scale systems
Abstract: Learning a dynamical system from input/output data is a fundamental task in the control design pipeline. In the partially observed setting there are two components to identification: parameter estimation to learn the Markov parameters, and system realization to obtain a state space model. In both sub-problems it is implicitly assumed that standard numerical algorithms such as the singular value decomposition (SVD) can be easily and reliably computed. When trying to fit a high-dimensional model to data, even computing an SVD may be intractable. In this work we show that an approximate matrix factorization obtained using randomized methods can replace the standard SVD in the realization algorithm while maintaining the finite-sample performance and robustness guarantees of classical methods.

Keywords: Networked control systems, Computational methods, Power systems
Abstract: In this paper, we define and quantify resiliency for a class of nonlinear control systems and propose data-driven algorithms for computing the same. In particular, we consider control-affine nonlinear systems and use the Koopman operator to lift the controlled dynamical system to the space of functions, where the evolution is linear. The linear representation of the nonlinear system allows us to relate controllability and observability of a general control-affine nonlinear system to the controllability and observability of the lifted linear system. Furthermore, it allows us to define controllability and observability gramians for the underlying nonlinear control system on the lifted space. Finally, in terms of these gramians, we define the resiliency of the nonlinear system. We illustrate the efficacy of the proposed approach to compute the resiliency metrics on time-series data obtained from a linear system and this serves as proof of concept. Furthermore, since resiliency is most relevant in the realm of power networks, we also illustrate our method on a data-set obtained from the IEEE 123 bus network microgrid.

Keywords: Smart grid, Machine learning, Optimization
Abstract: The dynamics-aware economic dispatch (DED) problem embeds low-level generator dynamics and operational constraints to enable near real-time scheduling of generation units in a power network. DED produces a more dynamic supervisory control policy than traditional economic dispatch (T-ED) that reduces overall generation costs. However, the incorporation of differential equations that govern the system dynamics makes DED an optimization problem that is computationally prohibitive to solve. In this work, we present a new data-driven approach based on differentiable programming to efficiently obtain offline parametric solutions to the underlying DED problem. In particular, we employ the recently proposed differentiable predictive control (DPC) for offline learning of explicit neural control policies based on identified Koopman operator (KO) model of the system dynamics. We demonstrate the high solution quality and five orders of magnitude computational-time savings of the DPC method over the original optimization-based DED approach on a 9-bus test power grid network.

Keywords: Constrained control, Distributed control, Power systems
Abstract: Modern safety-critical energy infrastructures are operated in a hierarchical and modular control framework allowing only limited data exchange between modules. In this context, to assure system-wide safety each module must synthesize and communicate constraints on the values of exchanged data. To ensure transient safety in inverter-based microgrids, we develop a set invariance-based distributed safety verification algorithm for each inverter module. Applying Nagumo’s invariance condition, we construct a robust polynomial optimization problem to jointly search for safety-admissible set of control set-points and design parameters, under allowable disturbances from neighbors. We solve the verification problem with sum-of-squares programming and we perform numerical simulations using grid-forming inverters to illustrate our method.

Keywords: Control of networks, Stochastic systems, Power systems
Abstract: With the increasing connectedness and interdependence of systems that are stochastic in nature, the issue of how to manage and coordinate them for safe operation has evidently become more important. In many networked system architectures, the system-wide output must be delicately managed, often within a prescribed set of bounds. In this paper, a novel control framework is proposed where the bounds on the outputs are translated into independent bounds on the controllable inputs of each subsystem. The main benefit of this framework is that respecting the individual control bounds suffices to guarantee that the system-wide outputs will remain within safe boundaries. Because the systems are assumed to be stochastic, the bounds on the output are introduced as probabilistic chance constraints. The benefits of this framework are demonstrated by applying it to the control of distributed energy resources in a distribution network where the main goal is to keep the voltage magnitudes within their prescribed bounds. The control bounds are evaluated using real data on an IEEE test system.

Keywords: Power systems, Optimization
Abstract: Critical energy infrastructures are increasingly relying on advanced sensing and control technologies for efficient and optimal utilization of flexible energy resources. Algorithmic procedures are needed to ensure that such systems are designed to be resilient to a wide range of cyber-physical adversarial events. This paper provides a robust optimization framework to quantify the largest adversarial perturbation that a system can accommodate without violating pre-specified resiliency metrics. We formulate the maximal adversarial set characterization as a bi-level optimization problem which is solved via Lagrangian relaxations. We illustrate the proposed algorithm on an islanded microgrid example: a modified IEEE 123-node feeder with distributed energy resources. Simulations are carried out to characterize the tolerable adversarial perturbations for varying levels of available flexibility (energy reserves).

Keywords: Lyapunov methods, Optimal control, Constrained control
Abstract: Control Barrier Functions (CBFs) are becoming popular tools in guaranteeing safety for nonlinear systems and constraints, and they can reduce a constrained optimal control problem into a sequence of Quadratic Programs (QPs) for affine control systems. The recently proposed High Order Control Barrier Functions (HOCBFs) work for arbitrary relative degree constraints. One of the challenges in a HOCBF is to address the relative degree problem when a system has multiple control inputs, i.e., the relative degree could be defined with respect to different components of the control vector. This paper proposes two methods for HOCBFs to deal with systems with multiple control inputs: a general integral control method and a method which is simpler but limited to specific classes of physical systems. When control bounds are involved, the feasibility of the above mentioned QPs can also be significantly improved with the proposed methods. We illustrate our approaches on a unicyle model with two control inputs, and compare the two proposed methods to demonstrate their effectiveness and performance.

Keywords: Lyapunov methods, Nonlinear output feedback, Fluid flow systems
Abstract: In this paper, two boundary controllers are proposed to stabilize the origin of the nonlinear Kuramoto-Sivashinsky equation under intermittent measurements. More precisely, the spatial domain is divided into two sub-domains. The state of the system on the first sub-domain is measured along a given interval of time, and the state on the remaining sub-domain is measured along another interval of time. Under the proposed sensing scenario, we control the considered equation by designing the value of the state at three isolated spatial points, the two extremities of the spatial domain plus one inside point. Furthermore, we impose a null value for the spatial gradient of the state at these three locations. Under such a control loop, we propose two types of controllers and we analyze the stability of the resulting closed-loop system in each case. The paper is concluded with some discussions and future works.

Keywords: Lyapunov methods, Adaptive control, Constrained control
Abstract: This paper studies the problem of utilizing data-driven adaptive control techniques to guarantee stability and safety of uncertain nonlinear systems with high relative degree. We first introduce the notion of a High Order Robust Adaptive Control Barrier Function (HO-RaCBF) as a means to compute control policies guaranteeing satisfaction of high relative degree safety constraints in the face of parametric model uncertainty. The developed approach guarantees safety by initially accounting for all possible parameter realizations but adaptively reduces uncertainty in the parameter estimates leveraging data recorded online. We then introduce the notion of an Exponentially Stabilizing Adaptive Control Lyapunov Function (ES-aCLF) that leverages the same data as the HO-RaCBF controller to guarantee exponential convergence of the system trajectory. The developed HO-RaCBF and ES-aCLF are unified in a quadratic programming framework, whose efficacy is showcased via two numerical examples that, to our knowledge, cannot be addressed by existing adaptive control barrier function techniques.

Keywords: Optimal control, Lyapunov methods, Optimization
Abstract: Developing controllers for obstacle avoidance between polytopes is a challenging and necessary problem for navigation in tight spaces. Traditional approaches can only formulate the obstacle avoidance problem as an offline optimization problem. To address these challenges, we propose a duality-based safety-critical optimal control using nonsmooth control barrier functions for obstacle avoidance between polytopes, which can be solved in real-time with a QP-based optimization problem. A dual optimization problem is introduced to represent the minimum distance between polytopes and the Lagrangian function for the dual form is applied to construct a control barrier function. We validate the obstacle avoidance with the proposed dual formulation for L-shaped (sofa-shaped) controlled robot in a corridor environment. We demonstrate real-time tight obstacle avoidance with non-conservative maneuvers on a moving sofa (piano) problem with nonlinear dynamics.

Keywords: Optimal control, Lyapunov methods, Optimization
Abstract: This paper considers the optimal control problem of nonlinear systems under safety constraints with unknown dynamics. Departing from the standard optimal control framework based on dynamic programming, we study its dual formulation over the space of occupancy measures. For control-affine dynamics, with proper reparametrization, the problem can be formulated as an infinite-dimensional convex optimization over occupancy measures. Moreover, the safety constraints can be naturally captured by linear constraints in this formulation. Furthermore, this dual formulation can still be approximately obtained by utilizing the Koopman theory when the underlying dynamics are unknown. Finally, to develop a practical method to solve the resulting convex optimization, we choose a polynomial basis and then relax the problem into a semi-definite program (SDP) using sum-of-square (SOS) techniques. Simulation results are presented to demonstrate the efficacy of the developed framework.

Keywords: Power electronics, Lyapunov methods, Adaptive control
Abstract: An impedance matching network is mandatory between a source and its load to get the maximum power transfer in any system. These systems can go from antenna adjustment or wireless power transmission to plasma processing technology, widely used in semiconductor wafer processing. Some control strategies have been proposed which aim a robust performance for different impedance matching network configurations; however, they display a slow dynamic or present a power reflection coefficient exhibiting a nonmonotonic decreasing behavior, affecting the overall matching control performance. To solve this problem, in this paper, we propose a control technique based on a Control Lyapunov Function (CLF). A dynamic analysis is presented, proving the asymptotic stability of the system. Numerical simulations on a benchmark setup show the improvement of the proposed control scheme and its robustness under different non-ideal conditions.

Keywords: Hybrid systems, Computational methods, Constrained control
Abstract: Zonotopes are widely used for over-approximating forward reachable sets of uncertain linear systems for verification purposes. In this paper, we use zonotopes to achieve more scalable algorithms that under-approximate backward reachable sets of uncertain linear systems for control design. The main difference is that the backward reachability analysis is a two-player game and involves Minkowski difference operations, but zonotopes are not closed under such operations. We under approximate this Minkowski difference with a zonotope, which can be obtained by solving a linear optimization problem. We further develop an efficient zonotope order reduction technique to bound the complexity of the obtained zonotopic under-approximations. The proposed approach is evaluated against existing approaches using randomly generated instances and illustrated with several examples.

Keywords: Estimation, Observers for nonlinear systems
Abstract: This letter introduces a set-theoretic state estimation approach for bounded-error nonlinear discrete-time systems, subject to nonlinear observations or constraints, when polytope-valued uncertainties are assumed. Our approach relies on finding a polytopic enclosure to the true range of nonlinear mappings via the direct use of hyperplane and vertex representations of polytopes. In particular, we derive a tractable enclosure of the set-product of an interval and a polytope, which is then used in a two-step state estimation approach consisting of (i) state propagation (prediction) using the nonlinear system dynamics and (ii) measurement update (refinement) based on nonlinear observations. Moreover, we analyze the computational complexity of our proposed technique and derive sufficient conditions for stability of the estimation errors. Finally, we compare the effectiveness of our approach with existing polytopic and interval observers in the literature.

Keywords: Observers for Linear systems, Estimation, Uncertain systems
Abstract: This letter proposes a novel robust interval observer for a two-dimensional (treated as a synonym for a double-indexed system) linear time-invariant discrete-time system described by the Fornasini-Marchesini second model. This system is subject to unknown but bounded state disturbances and measurement noise. Built on recent interval estimation strategies designed for one-dimensional systems, the proposed observer is based on the introduction of weighting matrices which provide additional degrees of freedom in comparison with the classical structure relying on a change of coordinates. Linear matrix inequality conditions for the exponential stability and peak-to-peak performance of a two-dimensional system described by the Fornasini-Marchesini second model are then proposed, and applied to the design of a robust interval observer. Numerical simulation results are provided to show the efficiency of the proposed estimation strategy.

Keywords: Networked control systems, Uncertain systems, Nonlinear output feedback
Abstract: A dynamical system is mixed monotone when there exists a related decomposition function that separates the system dynamics into cooperative and competitive state interactions. Such a decomposition enables, e.g., efficient computation of robust reachable sets and forward invariant sets, but obtaining a decomposition function can be challenging. In this paper, we present a method for obtaining a decomposition function for a system that can be represented as an interconnection of subsystems with known decomposition functions. We further extend this approach using tools from interval reachability analysis to accommodate systems with outputs and we provide also conditions for when the system's unique tight decomposition function is obtained via this approach. We demonstrate this methodology for computing decomposition functions with an example of a 3-dimensional unicycle model and with a case study of a 7-dimensional nonlinear spacecraft system defined as an interconnection of subsystems and feedback controllers. Reachable sets for the systems are then computed using their decomposition functions and the standard tools from mixed monotone systems theory.

Keywords: Stochastic systems, Lyapunov methods, Control applications
Abstract: Stability and safety are crucial in safety-critical control of dynamical systems. The reach-avoid-stay objectives for deterministic dynamical systems can be effectively handled by formal methods as well as Lyapunov methods with soundness and approximate completeness guarantees. However, for continuous-time stochastic dynamical systems, probabilistic reach-avoid-stay problems are viewed as challenging tasks. Motivated by the recent surge of applications in characterizing safety-critical properties using Lyapunov-barrier functions, we aim to provide a stochastic version for the probabilistic reach-avoid-stay problems in consideration of robustness. To this end, we first establish a connection between stochastic stability with safety constraints and reach-avoid-stay specifications. We then prove that stochastic Lyapunov-barrier functions provide sufficient conditions for the target objectives. We apply Lyapunov-barrier conditions in control synthesis for reach-avoid-stay specifications, and show its effectiveness in a case study.

Keywords: Formal verification/synthesis, Neural networks, Linear parameter-varying systems
Abstract: A forward reachability analysis method for the safety verification of nonlinear systems controlled by neural networks is presented. The proposed method relies on abstracting the activation functions in the neural networks (NN) by quadratic constraints (QCs) resorting to local sector bounds. To tackle the system nonlinearity, the nonlinear model is embedded into a linear parameter varying (LPV) representation. An outer approximation of the forward reachable set of the closedloop system is obtained using semidefinite programming. A numerical example clearly demonstrates the applicability of the proposed method. Comparison with some available methods reveals that the provided approach may potentially lead to less conservative results.

Keywords: Optimal control, Communication networks, Decentralized control
Abstract: This paper presents two novel factor graph formulations for performing flow control on radio transceivers forming a large Wireless Mesh Network (WMN). Both factor graph formulations are abstractions of a discrete-time finite-horizon Linear Quadratic Regulator (LQR) used to solve an Optimal Control Problem (OCP). The first formulation, WMNLQR, is a centralized controller which demonstrates the superiority of inference-based, graphical solvers over more traditional least-squares and Dynamic Programming (DP) approaches. Specifically, we see linear runtime complexity with respect to both state dimension and trajectory length by leveraging the sparsity inherent to the problem. The second formulation, DWMNLQR, is a decentralized implementation which leverages the physical RF message passing capabilities of the radio transceivers to assist in the factor graph solver. The decentralized algorithm supplements the capabilities of WMNLQR, as it aids in robustness and real-time solving of WMN flow control without compromising global optimality.

Keywords: Optimal control, Constrained control, Computational methods
Abstract: Funnels are widely used in safety-critical applications such as robust motion planning as conservative bounds for possible state deviations due to disturbances. However, the problem of finding a tight funnel has been a challenging topic due to the complexity of the Hamilton-Jacobi partial differential equation that appears when solving the problem. In this paper, we consider the funnel-finding problem for piecewise polynomial systems, with which we can approximate a considerably wider range of real-world dynamic systems, compared to global polynomial approximations. We propose techniques that reduce the computational burden of the problem and improve the tightness of the obtained funnel. Through experimental results using three different example systems, we confirm that the proposed method yields a tighter funnel in a shorter time, compared to competing polynomial-based funnel-computing methods.

Keywords: Optimal control, Constrained control, Traffic control
Abstract: We consider the merging control problem for Connected and Automated Vehicles (CAVs) aiming to jointly minimize travel time and energy consumption while providing speed-dependent safety guarantees and satisfying velocity and acceleration constraints. Applying the joint optimal control and control barrier function (OCBF) method, a controller that optimally tracks the unconstrained optimal control solution while guaranteeing the satisfaction of all constraints is efficiently obtained by transforming the optimal tracking problem into a sequence of quadratic programs (QPs). However, these QPs can become infeasible, especially under tight control bounds, thus failing to guarantee safety constraints. We solve this problem by deriving a control-dependent feasibility constraint corresponding to each CBF constraint which is added to each QP and we show that each such modified QP is guaranteed to be feasible. Extensive simulations of the merging control problem illustrate the effectiveness of this feasibility guaranteed controller.

Keywords: Optimal control, Differential-algebraic systems, Autonomous systems
Abstract: With the growing interest in autonomous surface vessels in the recent years, besides the tasks of driving and positioning, more demanding maneuvers have to be considered, e.g., docking maneuvers. Difficult weather conditions increase the complexity of docking maneuvers because of their proximity to piers or other vessels. As an example, fast sideways winds induce a significant rotational moment on the ship and can prevent it from docking successfully. This contribution proposes a method to perform a simplified rope-assisted docking ma- neuvers for these cases which results in a differential algebraic (DAE) system description. The docking task is embedded in an optimal control problem (OCP), which is solved using a direct (simultaneous) method. Therein, two methods are proposed to integrate the DAE system numerically. To this end, a DAE solver tailored for the solution of index-1 DAE systems is compared to the proposed method which introduces a constraint force to the ODE that serves as an additional decision variable. Additionally, a mathematical description of the vessel’s environment such as static, and possibly dynamic, obstacles is included in the proposed OCP by means of a dual approach. The two integration techniques are compared using a simulated docking scenario.

Keywords: Optimal control, Distributed control, Autonomous systems
Abstract: This paper presents a distance-based formation control of nonlinear agents on a plane. Due to mathematical complexity, the distance-based formation is mainly studied for linear single- and double-integrator models. Here, we introduce a novel control scheme for a distance-based control of a set of nonlinear agents. The formation topology is modeled as directed graphs where just one incident agent controls the corresponding edge (distance constraint). The proposed method is based on state-dependent Riccati equation (SDRE) theory, which can effectively be applied to nonlinear systems. The asymptotic stability of the formation is rigorously proven. Moreover, using the SDRE method and signed area constraints, the proposed controller guarantees collision avoidance and prevents flip ambiguity of the formation. Simulation results are presented that support theoretical results.

Keywords: Optimal control, Distributed control, Control applications
Abstract: The essence of distributed computing systems is how to schedule incoming requests and how to allocate all computing nodes to minimize both time and computation costs. In this paper, we propose a cost-aware optimal scheduling and allocation strategy for distributed computing systems while minimizing the cost function including response time and service cost. First, based on the proposed cost function, we derive the optimal request scheduling policy and the optimal resource allocation policy synchronously. Second, considering the effects of incoming requests on the scheduling policy, the additive increase multiplicative decrease (AIMD) mechanism is implemented to model the relation between the request arrival and scheduling. In particular, the AIMD parameters can be designed such that the derived optimal strategy is still valid.

Keywords: Kalman filtering, Estimation, Differential-algebraic systems
Abstract: State estimators can be a powerful tool in the development of advanced controls and performance monitoring capabilities for vapor compression cycles, but the nonlinear and numerically stiff aspects of these systems pose challenges for the practical implementation of estimators on large physics-based models. We develop smoothing methods in the extended and ensemble Kalman estimation frameworks that satisfy physical constraints and address practical limitations with standard implementations of these estimators. These methods are tested on a model built in the Julia language, and are demonstrated to successfully estimate unmeasured variables with high accuracy.

Keywords: Kalman filtering, Estimation, Nonlinear systems identification
Abstract: This paper is aimed at characterizing the mean square error and probabilistic uncertainty of a popular class of filtering algorithms in nonlinear systems. The state estimation error of the extended Kalman filter and the deterministic-gain Kalman filter are analyzed. We allow a vector state, but assume scalar measurements. A set of conditions for the mean square error to be upper-bounded is derived. Furthermore, the probabilistic bounds for the estimation error are computed via both the moment-based approach and the stochastic comparison analysis approach. The latter provides a formal means determining uncertainty bounds, such as statistical confidence regions.

Keywords: Kalman filtering, Information theory and control, Uncertain systems
Abstract: Maximum correntropy criterion (MCC) has been widely used in Kalman filter to cope with heavy-tailed measurement noises. However, its performance on mitigating non-Gaussian process noises and unknown disturbance is rarely explored. In this paper, we extend the definition of correntropy from a single kernel to multiple kernels. Then, we derive a multi-kernel maximum correntropy Kalman filter (MKMCKF) to cope with multivariate non-Gaussian noises and disturbance. Three examples are provided to show the effectiveness of the proposed methods.

Keywords: Kalman filtering, Model Validation, Automotive systems
Abstract: For further development of assistance systems and autonomous systems for tractor-semitrailers, knowledge of relevant trailer states and parameters is necessary. However, the trailer is sparsely equipped with sensors and the standardized communication between truck and trailer is reduced to braking signals. This paper presents an approach for estimating the lateral and vertical tire forces, roll behavior and articulation angle of the trailer for different loading conditions solely based on trailer signals using an Unscented Kalman Filter (UKF). Due to missing information from the truck, the varying mass and center of gravity (COG) based on different loading conditions must be estimated only on trailer signals. The investigation on structural observability shows that an estimation of all unknown parameters with the UKF is not possible. Thus, the usage of a substitute mass and COG is necessary and calculated based on the trailer’s axle load and domain knowledge of the usual practical loading distribution. The method is validated for different substitute masses and COGs with experimental data from a fully and partially loaded trailer.

Keywords: Networked control systems, Kalman filtering, Sensor networks
Abstract: In this paper we develop a simple, yet effective, secure communication protocol based on linear algebra that allows messages to be transmitted securely, without the need to encrypt them through computationally demanding approaches or to consider integer-valued plain messages, except during an initialization phase. The scheme is composed by several layers of protection and is applied to a scenario involving a networked Kalman filter fed by measurements sent by a sensor through a public network. Specifically, the two actors interact at initialization by exchanging suitably encrypted matrices; then, each message is split in two parts and is altered via a linear non-invertible transformation. Notably, each message is guaranteed to correspond to an unobservable measurement, while the receiver is provided with a strategy to reconstruct the message through simple calculations based on the information exchanged during the initialization phase. To further mask the messages, Gaussian noise with statistics known to both parties is artificially added to the messages. A simulation campaign completes the paper and demonstrates its effectiveness experimentally.

Keywords: Sensor networks, Kalman filtering, Estimation
Abstract: Distributed Kalman filters have been widely studied in vector space and been applied to 2-D target state estimation using sensor networks. In this paper, we introduce a novel distributed invariant extended Kalman filer (DIEKF) that exploits matrix Lie groups and is suitable to track the target’s 6-DOF motion in a 3-D environment. The DIEKF is based on the proposed extended Covariance Intersection (CI) algorithm that guarantees consistency in matrix Lie groups. The DIEKF is fully distributed as each agent only uses the information from itself and the one-hop communication neighbors, and it is robust to a time-varying communication topology and changing blind agents. To evaluate the performance, we apply the algorithm in a camera network to track a target pose. Extensive Monte-Carlo simulations have been performed to analyze the performance. Overall, the proposed algorithm is more accurate and more consistent in comparison with our recent work on the quaternion-based distributed EKF (QDEKF).

Keywords: Optimal control, Machine learning, Stability of nonlinear systems
Abstract: Recent research has shown that supervised learning can be an effective tool for designing optimal feedback controllers for high-dimensional nonlinear dynamic systems. But the behavior of these neural network (NN) controllers is still not well understood. In this paper we use numerical simulations to demonstrate that typical test accuracy metrics do not effectively capture the ability of an NN controller to stabilize a system. In particular, some NNs with high test accuracy can fail to stabilize the dynamics. To address this we propose two NN architectures which locally approximate a linear quadratic regulator (LQR). Numerical simulations confirm our intuition that the proposed architectures reliably produce stabilizing feedback controllers without sacrificing optimality.

Keywords: Distributed parameter systems, Observers for Linear systems
Abstract: This paper presents a new formulation of consensus filters for parabolic PDEs. Using modal decompositions, the information a given distributed filter transmits to and receives from the remaining networked filters depends on the modal information needed. If a given distributed filter can completely reconstruct a specific mode or modes of the PDE, then it does not need any information from any of the networked filters. Similarly, if a distributed filter cannot adequately reconstruct a given mode, then it receives information from the filter that can completely reconstruct that specific mode. This then presents a connectivity which is based on the information needed. This consensus protocol which is dictated by the information a filter does not have but needs, is essentially a projection of information needed onto the unobservable space. This is demonstrated for a diffusion PDE in 1D and subsequently its abstraction is formulated for Riesz-spectral systems. Numerical studies demonstrate the proposed modal consensus filters.

Keywords: Distributed parameter systems, Fluid flow systems, Lyapunov methods
Abstract: The paper studies the feedback stabilization problem of the motion of a tank that contains an incompressible, Newtonian, viscous liquid. The control input is the acceleration of the tank and the overall system consists of two nonlinear Partial Differential Equations and two Ordinary Differential Equations. Moreover, a spill-free condition is required to hold. By applying the Control Lyapunov Functional methodology, a set of initial conditions (state space) is determined for which spill-free motion of the liquid is possible by applying an appropriate control input. Semi-global stabilization of the liquid and the tank by means of a simple feedback law is achieved, in the sense that for every closed subset of the state space, it is possible to find appropriate controller gains, so that every solution of the closed-loop system initiated from the given closed subset satisfies specific stability estimates. The closed-loop system exhibits an exponential convergence rate to the desired equilibrium point. The proposed stabilizing feedback law does not require measurement of the liquid level and velocity profiles inside the tank and simply requires measurements of: (i) the tank position error and tank velocity, (ii) the total momentum of the liquid, and (iii) the liquid levels at the tank walls. The obtained results allow an algorithmic solution of the problem of the spill-free movement and slosh-free settlement of a liquid in a vessel of limited height (such as water in a glass) by a robot to a pre-specified position, no matter how full the vessel is.

Keywords: Distributed parameter systems, Stability of nonlinear systems, Lyapunov methods
Abstract: This work addresses the stabilization problem for a parabolic system governed by semilinear vector reactionconvection-diffusion equation. We suggest a sampled-data switching control design to stabilize the parabolic system under spatially scheduled actuators. The implementation of controller is either by using one moving actuator that can move to the active subdomain in the negligible time or by N actuators placed in each subdomain. Via Lyapunov-Krasovskii approach, sufficient exponential stability conditions are established in the framework of linear matrix inequalities (LMIs). Simulation example is given to demonstrate the effectiveness of the proposed method.

Keywords: Distributed parameter systems, Lyapunov methods
Abstract: In this paper, we consider the boundary control of the 1D heat equation using an irreversible port Hamiltonian systems (IPHS) formulation. This formulation allows to cope with the second principle of Thermodynamics and exhibits the physical properties of the system. We extend the Interconnection and Damping Assignment-Passivity Based Control (IDA-PBC) method developed for port Hamiltonian systems to the available energy-based boundary control of IPHS. This method allows to achieve the desired equilibrium profile without constraints on the boundary conditions or on the initial profile of the plant. The method is illustrated by means of simulations considering the boundary control of the 1D heat diffusion in a copper rod.

Keywords: Distributed parameter systems, Discrete event systems, Lyapunov methods
Abstract: In liquid-solid phase change phenomena, the Stefan problem describes the time evolution of the material's temperature profile and the interface position. This paper presents an event-based boundary control strategy for the one-phase Stefan problem. The proposed control approach consists of a full-state feedback backstepping boundary control law developed to drive the liquid-solid interface position to a desired setpoint and a static event-trigger mechanism which determines the time instants at which the control input needs to be updated. It is shown that the dwell-time between two triggering instances is uniformly bounded from below. Due to the existence of a minimal dwell-time, the closed-loop system is free from the so-called textit{Zeno behavior}. The control input is updated at event times and applied in a textit{Zero-Order-Hold (ZOH)} fashion. The well-posedness of the closed-loop system along with certain model validity conditions is proved. Furthermore, using the Lyapunov approach, it is shown that the proposed control approach globally exponentially stabilizes the temperature profile to the melting temperature of the material in L_2-norm and the moving interface to the desired setpoint. A simulation example is provided to validate the theoretical developments.

Keywords: Iterative learning control, Automotive control, Neural networks
Abstract: With the growing need to reduce energy consumption and greenhouse gas emissions, Eco-driving strategies provide a significant opportunity for additional fuel savings on top of other technological solutions being pursued in the transportation sector. In this paper, a model-free deep reinforcement learning (RL) control agent is proposed for active Eco-driving assistance that trades-off fuel consumption against other driver-accommodation objectives, and learns optimal traction torque and transmission shifting policies from experience. The training scheme for the proposed RL agent uses an off-policy actor-critic architecture that iteratively does policy evaluation with a multi-step return and policy improvement with the maximum posteriori policy optimization algorithm for hybrid action spaces. The proposed Eco-driving RL agent is implemented on a commercial vehicle in car following traffic. It shows superior performance in minimizing fuel consumption compared to a baseline controller that has full knowledge of fuel-efficiency tables.

Keywords: Automotive control, Modeling, Optimization
Abstract: Safety is a driving force behind the continuous improvement and spread of Advanced Driver Assistance Systems (ADAS). For their operation, ADAS rely on sensor information which is available within a limited range, and vehicle-to-everything (V2X) communication is expected to extend virtually this range, at least as far as the traffic interaction is considered. For highway operation, traffic interactions are indeed the most important risk factor for crashes. However, for country roads, more deaths result from loss of control and collisions with static obstacles than from traffic interactions. Against this background, this paper suggests a virtual co-driver (VCD) which can correct actions of the driver if they present a potential risk, e.g., an emergency reaction (brake or escape) would not be possible anymore. The method is tested in simulation over a set of actual runs on a countryside road, and it is shown that the VCD is of relevance, both for human drivers as well as for a controlled vehicle.

Keywords: Optimal control, Learning, Predictive control for linear systems
Abstract: This paper presents a novel strategy for designing the loss function of data-driven vehicle speed predictors to increase the energy efficiency of an ecological adaptive cruise control (eco-ACC) system. A unique eco-ACC simulation is designed to investigate the influence of step-wise accuracy on the performance of eco-ACC. In these simulations, uncertainty is added at only one prediction step of a prediction horizon during an entire trip, and the eco-ACC performance is evaluated in terms of acceleration minimization. A weighted-mean-squared error is chosen as the loss function of vehicle speed predictors, where its weights are tuned based on the quantified influence of uncertainty. The shape of the weight function is formed as a chi^2 distribution with respect to prediction time. The proposed loss function is used for training an example data-driven speed predictor using polynomial regression. Simulation results show that using the proposed loss function allows energy efficiency to be enhanced by up to 5.4% more than using the conventional loss function, namely, mean-squared errors.

Keywords: Automotive control, Automotive systems
Abstract: Merging, e.g. from a ramp to a highway, is a potentially complex operation that frequently requires cooperation between the involved vehicles. Besides safety aspects, it can also lead to significant traffic flow disturbances. While V2X and automated driving might reduce or even solve the problem in the future, mixed traffic, including human-driven vehicles without V2X, will be the norm for a while. This paper proposes a cautious merging assistant based on MPC, which does not only ensures safety but also minimizes the induced disturbance of the traffic flow. As an indicator of the induced traffic flow disturbance, we propose a function related to only readily available measurements of the first disturbed vehicle. The evaluation is done using a catalogue of merging situations extracted from the ”HighD” dataset and confirms that the cautious control does indeed provide a safe merging while affecting less the incoming traffic.

Keywords: Estimation, Kalman filtering, Automotive systems
Abstract: Off-road vehicle movement has to contend with uneven and uncertain terrain which present challenges to path planning and motion control for both manned and unmanned ground vehicles. Knowledge of terrain properties can allow a vehicle to adapt its control and motion planning algorithms. Terrain properties, however, can change on time scales of days or even hours, necessitating their online estimation. The kinematics and, in particular the oscillations experienced by an off-road vehicle carry a signature of the terrain properties. These terrain properties can thus be estimated from proprioceptive sensing of the vehicle dynamics with an appropriate model and estimation algorithm. In this paper, we show that knowledge of the vertical dynamics of a vehicle due to its suspension can enable faster and more accurate estimation of terrain parameters. The paper considers a five degree of freedom model that combines the well known half-car and bicycle models. We show through simulation that the sinkage exponent, a parameter that can significantly influence the wheel forces from the terrain and thus greatly impact the vehicle trajectory, can be estimated from measurements of the vehicle's linear acceleration and rotational velocity, which can be readily obtained from an on board IMU. We show that modelling the vertical vehicle dynamics can lead to significant improvement in both the estimation of terrain parameters and the prediction of the vehicle trajectory.

Keywords: Predictive control for linear systems, Nonlinear systems identification, Automotive control
Abstract: In this paper, we develop a data-driven process for building a model predictive control (MPC) for eco-driving of automated vehicles. The process involves performing system identification in which the non-linear vehicle dynamics model is approximated by the Koopman operator, a linear predictor of higher state-dimension, in a data-driven framework. This approach allows us to formulate the eco-driving problem in a constrained quadratic program that leads to a computationally fast MPC. The MPC is then implemented as a closed-loop control of an electric vehicle in numerical simulations for demonstration.

Keywords: Control applications, Learning
Abstract: The fourth industrial revolution or Industry 4.0 is widely expected to result in fundamental advances in all aspects of manufacturing, including transformation of manufacturing toward sustainability. This paper focuses on the opportunities that Industry 4.0 is expected to provide for improving sustainability and the obstacles in realizing these benefits. We introduce the concept of sustainability and the requirements for a system to be sustainable. This includes economic feasibility, social desirability and ecological viability for current and future generations. We provide a tutorial overview of methods and concepts that are relevant to assessing and designing sustainable systems. These include methods for considering direct and indirect impacts such as carbon footprint analysis and life cycle assessment; approaches for respecting nature’s capacity such as planetary boundaries and ecosystem services; and methods for considering social equity. We summarize the insight that such methods can provide about the sustainability of Industry 4.0. While Industry 4.0 is likely to improve the energy and material efficiency of industrial activities, the wider impact of these innovations is less clear. Some previous innovations have resulted in unintended harm due to the shifting of impacts in space, time, and across disciplines. We describe approaches from sustainable engineering to reduce the chance of such unintended harm from Industry 4.0. One such approach involves the integrated design and control of networks of industrial and ecological systems. Such a system seeks mutually beneficial synergies that can be economically, socially, and ecologically superior to traditional techno-centric designs. We describe how advances in process control, data science, smart sensors, etc. are needed for the practical realization of such win-win benefits.

Keywords: PID control, Process Control
Abstract: The gain crossover frequency and the phase margin are the most common design specifications in robust controller design. An additional flat phase constraint is proposed to guarantee the system robustness under gain variations, where the phase of the desired loop transfer function is flat around the gain crossover frequency. This controller design methodology is successfully implemented on integer-order proportional-integral-derivative (IOPID) controllers and fractional-order proportional-integral (FOPI) controllers in a class of first-order plus time-delay (FOPTD) systems. However, it is not easy to apply the flat phase constraint on general minimum-phase systems since the derivative of the loop transfer function with respect to frequency may be difficult to deduce. A flat phase design approach based on the Bode's integral is proposed for general minimum phase systems which does not need the system model information. An FOPI controller and an IOPID controller are designed in numerical examples to validate the effectiveness of the proposed robust controller design method.

Keywords: Process Control, Chemical process control, Control applications
Abstract: Alarms systems provide important alerts for the safety and efficiency of industrial facilities. However, due to complex plant connectivity and interconnections of process variables, there exist many alarms that are correlated with each other, leading to compromised alarm system performance in indicating the exact abnormality. Therefore, a systematic method to discover correlated alarms from historical Alarm & Event (A&E) logs is proposed in this work. The contributions of this study are twofold: 1) Correlated alarms are captured using a pattern mining approach, such that alarm occurrence orders are preserved to help root cause analysis; 2) network graphs are generated to visualize alarm correlations and their statistical features as indications of potential abnormality propagation paths. The effectiveness of the proposed method is demonstrated via case studies using alarm data from real industrial facilities.

Keywords: Process Control, Fault detection, Networked control systems
Abstract: This work focuses on the problem of enhancing cyberattack detection capabilities in process control systems subject to multiplicative cyberattacks. First, the relationship between closed-loop stability and attack detectability with respect to a class of residual-based detection schemes is rigorously analyzed. The results are used to identify a set of controller parameters (called “attack-sensitive” controller parameters) under which an attack can destabilize the closed-loop system. The selection of attack-sensitive controller parameters can enhance the ability to detect attacks, but can also degrade the performance of the attack-free closed-loop system. To balance this trade-off, a novel active attack detection methodology employing controller parameter switching between the nominal controller parameters (chosen on the basis of standard control design criteria) and the attack-sensitive controller parameters, is developed. The proposed methodology is applied to a chemical process example to demonstrate its ability to detect multiplicative sensor-controller communication link attacks.

Keywords: Process Control, Predictive control for nonlinear systems, Optimal control
Abstract: Next-generation manufacturing involves increasing use of automation and data to enhance process efficiency. An important question for the chemical process industries, as new process systems (e.g., intensified processes) and new data modalities (e.g., images) are integrated with traditional plant automation concepts, will be how to best evaluate alternative strategies for data-driven modeling and synthesizing process data. Two methods which could be used to aid in this are those which aid in testing data-based techniques on-line, and those which enable various data-based techniques to be assessed in simulation. In this work, we discuss two techniques in this domain which can be applied in the context of chemical process control, along with their benefits and limitations. The first is a method for testing data-driven modeling strategies on-line by postulating the experimental conditions which could reveal if a model is correct, and then attempting to collect data which could help to reveal this. The second strategy is a framework for testing image-based control algorithms via simulating both the generation of the images as well as the impacts of control on the resulting systems.

Keywords: Manufacturing systems, Materials processing, Process Control
Abstract: The problem of economically achieving a user specified set of product qualities in an industrial batch process is presented in the current manuscript, demonstrated using a lab-scale uni-axial rotational molding process. To achieve a product with specified qualities, a data driven Economic model predictive control (EMPC) formulation is proposed through constraints on quality variables. A state-space model of the rotational molding process is first identified from previously generated data in the lab. The evolution of the internal mold temperature for a given set of input moves (combination of two heaters and compressed air) is captured by the state space model. Further, this model is augmented with a partial-leastsquares based quality model, which relates the terminal (states) prediction with key quality variables (sinkhole area and impact energy). This augmented model is then integrated within the EMPC scheme that penalizes excessive energy consumption while aiming to achieve on-spec products via constraints on the quality variables. Results obtained from experimental studies illustrates the capability of the proposed EMPC scheme in lowering the process cost (energy requirements) while achieving user specified product.

Keywords: Pulp and Paper Control
Abstract: Lightweight packaging in the pulp and paper industry has received much attention as an effective way to reduce CO2 emissions. One challenge is that the tensile strength of packaging papers, which determines the maximum load, has to be comparable to that of conventional packaging materials. In a pulp digester, wood fibers are chemically separated and physically deformed during pulping, resulting in a dynamic change of tensile strength of end-use papers. In this work, the deformation of fiber during Kraft pulping is described by integrating the multiscale modeling framework of Choi and Kwon [1] and the classical column buckling theory. Then, an approximate model is identified and employed to design a model predictive control system to regulate the fiber deformation during pulping. The proposed control system achieved a lower degree of fiber deformation than a conventional pulping strategy, thereby contributing to the enhanced tensile strength of end-use papers.

Keywords: Variable-structure/sliding-mode control, Flight control
Abstract: The relative degree approach is a powerful tool for obtaining system’s input-output dynamics that are used in output tracking controller designs for minimum phase systems. Designs based on the relative degree approach alone may fail due to restrictions on control authority and an insufficiency of the corresponding control gain. The use of a practical generalized relative degree approach in concert with sliding mode control to compensate for system perturbations is studied in this paper. The efficacy of the proposed approach is demonstrated on a rocket attitude control problem analytically and via simulations

Keywords: Variable-structure/sliding-mode control, Uncertain systems, Control applications
Abstract: Obtaining prescribed accuracy bounds in super-twisting sliding mode control loops often falls short in terms of the applicability of the controller in high-performance systems. This is due to the fact that the selection of the controller gains that are derived from the conditions for finite-time convergence may be too restrictive in connection to actuator limitations and induced chatter. Previous work has shown that in case of periodic perturbations, there can be a systematic selection of much lower controller gains that guarantees boundedness of the closed-loop solutions within predetermined accuracy bounds. This study presents an experimental validation of these findings carried out on a commercial industrial motor system.

Keywords: Flight control, Variable-structure/sliding-mode control, Estimation
Abstract: Inertial Measurement Unit (IMU) subsystems measure accelerations or attitude rates by observing specific forces applied to a mechanical system and/or torques caused by gyroscopic effects. The IMU dynamics yield a dynamically distorted measurement and accuracy deterioration. In this paper, the IMUs accuracy is improved by reconstructing the true acceleration and the true attitude rate input signals by employing the proposed second order sliding mode observer (2-SMO) operating in concert with dynamic inversion, given in real time dynamically distorted measurement and the IMU dynamics. The body bending rate is estimated using reconstructed acceleration. Implementation of the proposed algorithms does not require any modification of the hardware of existing IMUs. The effectiveness of proposed algorithms is validated on a multiple case study for a 3-stage missile interceptor with typical sampling rates of existing IMUs.

Keywords: Spacecraft control, Indirect adaptive control, Uncertain systems
Abstract: An adaptive force control algorithm for small celestial body sampling for a variety of surface properties is developed. The control algorithm consists of an adaptive controller combined with feedback linearization. When a spacecraft makes contact with the surface, it must maintain a desired contact force in order to capture a sample. The properties of the surface are unknown or uncertain before contact with the surface is made. The adaptive controller performs system identification online to create an input-output model of the feedback linearized system. From the input-output model a block observable canonical form is realized and the control input is determined by model predictive control (MPC) to maintain a desired contact force in spite of the unknown surface properties. The approach is applied to a variety of surface properties with linear and nonlinear contact models.

Keywords: Spacecraft control, Predictive control for nonlinear systems, Feedback linearization
Abstract: To date, most of the small satellite formation flying missions have flown with Hill-Clohessy-Wiltshire equations based controllers. Additionally, the vast majority of nonlinear optimal control methods are complex for onboard implementation with the critical need of verification and validation methods. In this context, this paper addresses the relative motion control of formation flying mission with the design and development of an analytical, onboard-compatible optimal controller based upon the Nonlinear Generalized Predictive Control theory. The resulting controller is shown to be suitable to handle perturbed near-circular orbits.

Keywords: Air traffic management, Networked control systems
Abstract: This paper develops a novel physics-based approach for fault-resilient multi-quadcopter coordination in the presence of abrupt quadcopter failure. Our approach consists of two main layers: (i) high-level physics-based guidance to safely plan the desired recovery trajectory for every healthy quadcopter and (ii) low-level trajectory control design by choosing an admissible control for every healthy quadcopter to safely recover from the anomalous situation, arisen from quadcopter failure, as quickly as possible. For the high-level trajectory planning, first, we consider healthy quadcopters as particles of an irrotational fluid flow sliding along streamline paths wrapping failed quadcopters in the shared motion space. We then obtain the desired recovery trajectories by maximizing the sliding speeds along the streamline paths such that the rotor angular speeds of healthy quadcopters do not exceed certain upper bounds at all times during the safety recovery. In the low level, a feedback linearization control is designed for every healthy quadcopter such that quadcopter rotor angular speeds remain bounded and satisfy the corresponding safety constraints. Simulation results are given to illustrate the efficacy of the proposed method.

Keywords: Emerging control applications, Agents-based systems, Traffic control
Abstract: Deriving optimal control strategies for coordination of connected and automated vehicles (CAVs) often requires re-evaluating the strategies in order to respond to unexpected changes in the presence of disturbances and uncertainties. In this paper, we first extend a decentralized framework that we developed earlier for coordination of CAVs at a signal-free intersection to incorporate replanning. Then, we further enhance the framework by introducing a priority-aware resequencing mechanism which designates the order of decision making of CAVs based on theory from the job-shop scheduling problem. Our enhanced framework relaxes the first-come-first-serve decision order which has been used extensively in these problems. We illustrate the effectiveness of our proposed approach through numerical simulations.

Keywords: Control applications, Cooperative control, Distributed control
Abstract: As an increasing number of controllable devices are introduced onto the grid, they can individually provide ancillary services in support of grid stability. However, the goals of each device differ due to their type and individual objectives, causing instances where they may conflict. To reduce the chances of these devices contributing to grid instability, these devices must effectively communicate in a cooperative manner to both meet their own needs while providing services to the grid. Previous work demonstrates that the Network Lasso - ADMM - Limited Communication - DMPC (NALD) algorithm allows coordination between two subsystems that use different control algorithms (building and charging stations) to provide services to the grid and individually optimize their performance in a specific scenario. The ideal NALD algorithm should be generalized to allow plug-and-play capabilities across devices of differing characteristics. This paper takes a step toward achieving this generalization by updating the electric vehicle charging objective and redefining the communication scheme compared to prior work to generalize the coordination and, as a result, improve the performance of the NALD algorithm.

Keywords: Electrical machine control, PID control, Simulation
Abstract: This paper gives a systematic comparison between the digital implementations of two current state-of-the-art control techniques for Brushless DC Motors (BLDC) with trapezoidal back-emf, namely Six-Step-Commutation (6SC) and Modified Field Oriented Control (MFOC). Ideally, both techniques are able to produce ripple-free torque. However, due to several real-world factors, including discrete implementation and requirements on fundamental-to-sampling frequency ratio, this is not possible in reality. Based on continuous PI-controllers and their discrete approximations, including a delayed version, we compare the performance of 6SC and MFOC based on the torque ripple and the mean torque error. Conclusively, with the use of the continuous PI controllers and MFOC it is indeed possible to generate smooth torque also over high dynamic ranges, which so far was only clear for low speeds. Still, for the discrete PI control case, ripple-free torque is not achievable, though it is apparent that the generated ripple is significantly lower than those of 6SC with continuous or discrete implementation. For torque tracking, the error increases disproportionally for 6SC compared to MFOC.

Keywords: Output regulation, Emerging control applications, Stability of nonlinear systems
Abstract: This paper designs and implements a fractional order sliding mode approach (FOSMC) for grid-tied photovoltaic (PV) systems. The proposed FOSMC is derived based on a novel sliding manifold that takes into account the estimates of the unmatched disturbances. Its main objective is to control the dc-link voltage and enhance its dynamic response during unexpected network events. System stability is assessed using the Lyapunov theory. The performance of the proposed FOSMC is assessed using a grid-tied PV system subject to temporary symmetrical grid faults, steep load variations and mismatched disturbances. Comparison analysis with a standard SMC approach (SSMC) is also carried over. The obtained results revealed the superior performance of the proposed approach in improving the transient stability of dc-link voltage during grid faulty conditions. Additionally, the FOSMC was found to have faster time response and stronger robustness against network disturbances than the SSMC.

Keywords: Reduced order modeling, Optimal control, Emerging control applications
Abstract: Closed-loop irrigation can provide a reliable solution for precision irrigation. One challenge in closed-loop irrigation is to estimate the soil moisture information. The very high dimensionality of a typical agricultural field model makes optimization-based state estimators like Moving Horizon Estimation (MHE) very challenging. This work addresses the very high dimensionality issue and proposes a systematic approach for state estimation of large-scale agro-hydrological systems. We first present an algorithm for structure-preserving adaptive model reduction using trajectory-based unsupervised machine learning techniques; and then an adaptive MHE algorithm is developed based on the reduced model. The proposed algorithms are applied to an actual agricultural field to show the effectiveness of the proposed approaches.

Keywords: Emerging control applications, Learning, Identification for control
Abstract: We propose a learning-based model predictive control framework for mitigating the spread of epidemics. We capture the epidemic spreading process using a susceptible-infected-removed (SIR) epidemic model and consider testing for isolation as the control strategy. In the framework, we use a daily testing strategy to remove (isolate) a portion of the infected population. Our goal is to keep the daily infected population below a certain level, while minimizing the total number of tests. Distinct from existing works on leveraging model predictive control in epidemic spreading, we learn the model parameters and compute the feedback control signal simultaneously. We illustrate the results by numerical simulation using COVID-19 data from India.

Keywords: Machine learning, Building and facility automation, Iterative learning control
Abstract: Designing model-based control methods for real building energy optimization using physics-based models is complex and time-consuming. A promising alternative is to use data-driven models, but weather and occupancy changes may cause changes in operating conditions of the building, such that data-driven modeling accuracy and control performance degrade. In this work, we propose a data-driven learning framework for building energy control that addresses the non-stationary nature of building operations caused by factors like weather and occupancy. Our approach relies on a set of data-driven models that characterize the building energy behavior and a deep reinforcement learning (DRL) controller trained on such models. Defining all possible operating modes is intractable for most buildings, therefore, we propose a continual adaptation approach based on non-stationary change detection. We compare our proposal against a DRL state-of-the-art algorithm called proximal policy optimization (PPO), and data-driven model predictive control (MPC) using a five-zone building benchmark. We demonstrate satisfactory results for different non-stationary changes in terms of energy efficiency, thermal comfort, and actuation safety metrics.

Keywords: Machine learning, Estimation, Kalman filtering
Abstract: This paper presents a new method for modelling dynamical systems. The method uses historical data of the outputs to predict the evolution of the system. The proposed method is based on Direct Weight Optimization and the Kriging method. These data-based methods provide predictions as linear combinations of past outputs after solving a quadratic optimization problem. We introduce a novel methodology that we named state-space Kriging, which models the time evolution of the weighting parameters using a state-space formalism. In this way, the potential of Kriging, along with classical estimation methods, as the Kalman filter, can be leveraged to forecast the output of a nonlinear dynamical system. The optimization problems involved are easy to solve, and analytical solutions are provided. Some numerical examples and comparisons are provided to demonstrate the effectiveness of our proposal.

Keywords: Machine learning, Estimation, Reduced order modeling
Abstract: This paper introduces a data-dependent approximation of the forward kinematics map for certain types of motion models. It is assumed that motions are supported on a low-dimensional, unknown configuration manifold Q that is regularly embedded in high dimensional Euclidean space X:=RR^d. This paper introduces a method to estimate forward kinematics from the unknown configuration submanifold Q to an n-dimensional Euclidean space Y:=RR^n of observations. A known reproducing kernel Hilbert space (RKHS) is defined over the ambient space X in terms of a known kernel function, and computations are performed using the known kernel defined on the ambient space X. Estimates are constructed using a certain data-dependent approximation of the Koopman operator defined in terms of the known kernel on X. However, the rate of convergence of approximations is studied in the space of restrictions to the unknown manifold Q. Strong rates of convergence are derived in terms of the fill distance of samples in the unknown configuration manifold, provided that a novel regularity result holds for the Koopman operator. Additionally, we show that the derived rates of convergence can be applied in some cases to estimates generated by the extended dynamic mode decomposition (EDMD) method. We illustrate characteristics of the estimates for simulated data as well as samples collected during motion capture experiments.

Keywords: Machine learning, Manufacturing systems, Materials processing
Abstract: In this paper, we investigate quality classifications of additively manufacturing oxide dispersion strengthened steel parts under varying operating parameters. We perform rational feature engineering and selection through the use of dimension reduction, linear discriminant analysis, and support vector classification. Finally, we identify two robust linearly separable boundaries to help us further understand the physical phenomena that produce good or bad quality parts.

Keywords: Machine learning, Optimal control, Optimization
Abstract: We address the problem of learning from demonstrations when the learner must satisfy safety and/or performance requirements expressed as Stochastic Temporal Logic (StTL) specifications. We extend the maximum causal entropy inverse reinforcement learning framework to account for StTL constraints and show how to encode them via a minimal set of mixed-integer linear constraints. Our method is based on a cut-and-generate algorithm that iterates between two phases: in the cut phase, we use cutting hyperplanes to approximate the feasible region of the non-linear constraint that encodes atomic predicates and in the generate phase, we propagate these hyperplanes through the schematics to generate constraints for arbitrary formulas. Our algorithmic contributions are validated in different environments and specifications.

Keywords: Machine learning, Uncertain systems, Predictive control for linear systems
Abstract: This paper studies the impact of imperfect information in online control with adversarial disturbances. In particular, we consider both delayed state feedback and inexact predictions of future disturbances. We introduce a greedy, myopic policy that yields a constant competitive ratio against the offline optimal policy. We also analyze the fundamental limits of online control with limited information by showing that our competitive ratio bounds for the greedy, myopic policy in the adversarial setting match (up to lower-order terms) lower bounds in the stochastic setting.

Keywords: Machine learning, Power systems, Robotics
Abstract: Reinforcement learning is well-studied under discrete actions. Integer actions setting is popular in the industry yet still challenging due to its high dimensionality. To this end, we study reinforcement learning under integer actions by incorporating the Soft Actor-Critic (SAC) algorithm with an integer reparameterization. Our key observation for integer actions is that their discrete structure can be simplified using their comparability property. Hence, the proposed integer reparameterization does not need one-hot encoding and is of low dimensionality. Experiments show that the proposed SAC under integer actions is as good as the continuous action version on robot control tasks and outperforms Proximal Policy Optimization on power distribution systems control tasks.

Keywords: Machine learning, Statistical learning
Abstract: We introduce tree linear cascades, a class of linear structural equation models for which the error variables are uncorrelated but need not be Gaussian nor independent. We show that, in spite of this weak assumption, the tree structure of this class of models is identifiable. In a similar vein, we introduce a constrained regression problem for fitting a tree-structured linear structural equation model and solve the problem analytically. We connect these results to the classical Chow-Liu approach for Gaussian graphical models. We conclude by giving an empirical-risk form of the regression and illustrating the computationally attractive implications of our theoretical results on a basic example involving stock prices.

Keywords: Multivehicle systems, Machine learning, Autonomous systems
Abstract: Multi-vehicle collision avoidance is a highly crucial problem due to the soaring interests of introducing autonomous vehicles into the real world in recent years. The safety of these vehicles while they complete their objectives is of paramount importance. Hamilton-Jacobi (HJ) reachability is a promising tool for guaranteeing safety for low-dimensional systems. However, due to its exponential complexity in computation time, no reachability-based methods have been able to guarantee safety for more than three vehicles successfully in unstructured scenarios. For systems with four or more vehicles, we can only empirically validate their safety performance. While reachability-based safety methods enjoy a flexible least-restrictive control strategy, it is challenging to reason about long-horizon trajectories online because safety at any given state is determined by looking up its safety value in a pre-computed table that does not exhibit favorable properties that continuous functions have. This motivates the problem of improving the safety performance of unstructured multi-vehicle systems when safety cannot be guaranteed given any least-restrictive safety-aware collision avoidance algorithm while avoiding online trajectory optimization. In this paper, we propose a novel approach using supervised learning to enhance the safety of vehicles by proposing new initial states in very close neighborhood of the original initial states of vehicles. Our experiments demonstrate the effectiveness of our proposed approach and show that vehicles are able to get to their goals with better safety performance with our approach compared to a baseline approach in wide-ranging scenarios.

Keywords: Process Control, Robust control, Machine learning
Abstract: Model predictive control approaches based on hybrid models have received a lot of attention in recent years. Hybrid models combine first principles with machine learning models and can be used to learn unknown dynamics without completely disregarding first-principles information, potentially improving the performance of model-based controllers. However, as for all data-driven models, their accuracy depends on the quality and quantity of data. Often, especially in process engineering, data is scarce, resulting in poor predictions, leading to sub-optimal performance and possible violation of the process constraints. This is particularly true for new processes with unknown or partially unknown dynamics. In this paper, we focus on repetitive finite-time processes, since they are often used for new and innovative processes and are more sensitive to nonlinearities. We propose a learning-supported approach based on a tube-based shrinking-horizon model predictive controller. The method guarantees robust constraint satisfaction and recursive feasibility also in the first process runs, where data is very scarce. Updating the uncertainty description by run-to-run learning allows for increased performance and reduced conservatism. The approach is validated in a simulation.

Keywords: Machine learning, Predictive control for nonlinear systems, Optimization
Abstract: Controlling wind kites requires accurate models, both for safe operation, as well as for thrust maximization. To this end, we present a trajectory tracking model predictive con- trol (MPC) approach in combination with Gaussian processes for model learning. Since perfect prediction models are usually unavailable, we use a hybrid model approach consisting of a physical base model extended by Gaussian processes that learn the model-plant mismatch. This allows for the computation of optimized improved reference trajectories, compared to the nominal model case. We furthermore outline an online-learning trajectory tracking MPC approach, which updates the process model recursively taking new measurements into account if the prediction error becomes too large. In simulations we show that even for large model-plant mismatches correct and safe operation can be achieved using the hybrid model in the MPC.

Keywords: Robotics, Modeling, Machine learning
Abstract: Soft robots have recently drawn extensive attention thanks to their unique ability of adapting to complicated environments. Soft robots are designed in a variety of shapes of aiming for many different applications. However, accurate modelling and control of soft robots is still an open problem due to the complex robot structure and uncertain interaction with the environment. In fact, there is no unified framework for the modeling and control of generic soft robots. In this paper, we present a novel data-driven machine learning method for modeling a cable-driven soft robot. This machine learning algorithm, named deterministic learning (DL), uses soft robot motion data to train a radial basis function neural network (RBFNN). The soft robot motion dynamics are then guaranteed to be accurately identified, represented, and stored as an RBFNN model with converged constant neural network weights. To validate our method, We have built a simulated soft robot almost identical to our real inchworm soft robot, and we have tested the DL algorithm in simulation. Furthermore, a neural network weight combining technique is used which can extract and combine useful dynamics information from multiple robot motion trajectories.

Keywords: Fault detection, Manufacturing systems, Markov processes
Abstract: With the advent of industry 4.0, machine learning (ML) methods have mainly been applied to design condition-based maintenance strategies to improve the detection of failure precursors and forecast degradation. However, in real-world scenarios, relevant features unraveling the actual machine conditions are often unknown, posing new challenges in addressing fault diagnosis problems. Moreover, ML approaches generally need ad-hoc feature extractions, involving the development of customized models for each case study. Finally, the early substitution of key mechanical components to avoid costly breakdowns challenge the collection of sizable significant data sets to train fault detection (FD) systems. To address these issues, this paper proposes a new unsupervised FD method based on double deep-Q network (DDQN) with prioritized experience replay (PER). We validate the effectiveness of the proposed algorithm on real data obtained from a steel plant placed in the south of Italy. Lastly, we compare the performance of our method with other FD methods showing its viability.

Keywords: Fault tolerant systems, Machine learning, Robotics
Abstract: In this paper, we propose an active fault-tolerant control framework for robotic manipulators subjected to joint actuator faults. The proposed active fault-tolerant control scheme includes a neural network-based fault diagnosis module and a reinforcement learning-based fault-tolerant control module. Once the actuator fault is detected and diagnosed, an additive reinforcement learning controller will produce compensation torques to guarantee the system safety and maintain the control performance. Compared to traditional methods, our strategy avoids overly relying on the exact system models, which has potential for wider applications. The scheme is evaluated on a 7-DOF Panda manipulator for tracking control tasks in the MuJoCo simulator. Simulation results demonstrate the effectiveness of the proposed framework in dealing with single and multiple actuator faults.

Keywords: Hybrid systems, Machine learning, Neural networks
Abstract: Reinforcement learning (RL) is a promising approach for deriving control policies for complex systems. As we show in two control problems, the derived policies from using the Proximal Policy Optimization (PPO) and Deep Q-Network (DQN) algorithms may lack robustness guarantees. Motivated by these issues, we propose a new hybrid algorithm, which we call Hysteresis-Based RL (HyRL), augmenting an existing RL algorithm with hysteresis switching and two stages of learning. We illustrate its properties in two examples for which PPO and DQN fail.

Keywords: Learning, Neural networks, Uncertain systems
Abstract: In this work, we investigate the effect of sensor-actuator clock offsets on reinforcement learning (RL) enabled cyber-physical systems. In particular, we consider an off-policy RL algorithm that receives data both from the system's sensors and actuators, and uses them to approximate a desired optimal control policy. Nevertheless, owing to timing mismatches, the control-state data obtained from these system components are inconsistent, hence creating the question of how robust RL will be. After an extensive analysis, we show that RL does retain its robustness, in an epsilon-delta sense; given that the sensor-actuator clock offsets are not arbitrarily large, and that the behavioral control input satisfies a Lipschitz continuity condition, RL converges epsilon-close to the desired optimal control policy. Simulations are carried out on a two-link manipulator, which clarify and verify theoretical findings.

Keywords: Machine learning, Mechanical systems/robotics, Adaptive control
Abstract: Wheeled mobile robots have a great variety of applications both indoors and outdoors. They are often required to work in various environments such as on rough terrains, wet roads, icy roads, and routes with rapid cornering. Therefore, it is important to design a control scheme that delivers robust tracking performance on various terrains, especially the ones that induce skidding and slipping easily. In this work, we focus on the challenge of mobile robot motion under slippery conditions, and propose a hierarchical framework that learns to adapt the control parameters online for robust model predictive tracking control of mobile robots. Concretely, the high level module based on reinforcement learning (RL) actively adjusts the model predictive control (MPC) scheme to be more aggressive or conservative, by adapting key parameters in the MPC optimization formulation. Experiments demonstrate our framework achieves adaptive and robust tracking performance, especially at rejecting slipping and reducing tracking errors when the mobile robot travels through various terrains. Our framework neither relies on knowing specific dynamic parameters nor requires data fitting. The model trained in simulation is validated in a zero-shot manner with completely different real-world terrain conditions to demonstrate its adaptability.

Keywords: Machine learning, Neural networks, Robotics
Abstract: We investigate the effect of including application knowledge about a robotic system states' causal relations when generating explanations of deep neural network policies. To this end, we compare two methods from explainable artificial intelligence, KernelSHAP, and causal SHAP, on a deep neural network trained using deep reinforcement learning on the task of controlling a lever using a robotic manipulator. A primary disadvantage of KernelSHAP is that its explanations represent only the features' direct effects on a model's output, not considering the indirect effects a feature can have on the output by affecting other features. Causal SHAP uses a partial causal ordering to alter KernelSHAP's sampling procedure to incorporate these indirect effects. This partial causal ordering defines the causal relations between the features, and we specify this using application knowledge about the lever control task. We show that enabling an explanation method to account for indirect effects and incorporating some application knowledge can lead to explanations that better agree with human intuition. This is especially favorable for a real-world robotics task, where there is considerable causality at play, and in addition, the required application knowledge is often handily available.

Keywords: Machine learning, PID control, Process Control
Abstract: Many industrial processes utilize proportional-integral-derivative (PID) controllers due to their practicality and often satisfactory performance. The proper controller parameters depend highly on the operational conditions and process uncertainties. This dependence brings the necessity of frequent tuning for real-time control problems due to process drifts and operational condition changes. This study combines the recent developments in computer sciences and control theory to address the tuning problem. It formulates the PID tuning problem as a reinforcement learning task with constraints. The proposed scheme identifies an initial approximate step-response model and lets the agent learn dynamics off-line from the model with minimal effort. After achieving a satisfactory training performance on the model, the agent is fine-tuned on-line on the actual process to adapt to the real dynamics, thereby minimizing the training time on the real process and avoiding unnecessary wear, which can be beneficial for industrial applications. This sample efficient method is applied to a pilot-scale multi-modal tank system. The performance of the method is demonstrated by setpoint tracking and disturbance regulatory experiments.

Keywords: Machine learning, Stochastic optimal control, Optimal control
Abstract: Robustness is a key enabler of real-world applications of Reinforcement Learning (RL). The robustness properties of risk-sensitive controllers have long been established. We investigate risk-sensitive Reinforcement Learning (as a generalization of risk-sensitive stochastic control), by theoretically analyzing the risk-sensitive exponential (exponential of the total reward) criteria, and the benefits and improvements the introduction of risk-sensitivity brings to conventional RL. We provide a probabilistic interpretation of (I) the risk-sensitive exponential, (II) the risk-neutral expected cumulative reward, and (III) the maximum entropy Reinforcement Learning objectives, and explore their connections from a probabilistic perspective. Using Probabilistic Graphical Models (PGM), we establish that in RL setting, maximization of the risk-sensitive exponential criteria is equivalent to maximizing the probability of taking an optimal action at all time-steps during an episode. We show that the maximization of the standard risk-neutral expected cumulative return is equivalent to maximizing a lower bound, particularly the Evidence lower Bound, on the probability of taking an optimal action at all time-steps during an episode. Furthermore, we show that the maximization of the maximum-entropy Reinforcement Learning objective is equivalent to maximizing a lower bound on the probability of taking an optimal action at all time-steps during an episode, where the lower bound corresponding to the maximum entropy objective is tighter and smoother than the lower bound corresponding to the expected cumulative return objective. These equivalences establish the benefits of risk-sensitive exponential objective and shed lights on previously postulated regularized objectives, such as maximum entropy. The utilization of a PGM model, coupled with exponential criteria, offers a number of advantages (e.g. facilitate theoretical analysis and derivation of bounds).

Keywords: Machine learning, Robust control, Optimal control
Abstract: We explore the relation between the risk-sensitive exponential (exponential of total cost) and Distributionally Robust Reinforcement Learning objectives, and in doing so, we unify some of the popular Reinforcement Learning algorithms. Such equivalence (I) allows to understand a number of well-known Reinforcement Learning algorithms from a risk minimization perspective and (II) establishes the robustness properties of risk-sensitive exponential objective in the Reinforcement Learning context, which in turn provides a theoretical justification for the robust performance of risk-sensitive Reinforcement Learning algorithms in the literature. The robustness of exponential criteria motivates risk-sensitizing current risk-neutral Reinforcement Learning algorithms using such criteria.

Keywords: Optimal control, Neural networks, Stability of hybrid systems
Abstract: In this paper, we develop a control-design framework to solve the optimal control problem for unknown linear time-invariant systems using an intermittent approach in the presence of sparse and dropped reward signals. The problem is formulated as an infinite horizon intermittent learning-based problem with sparse and dropped reinforcements. We quantify the sparse reinforcement signals and provide a model-free Q-learning method based on an actor/critic structure with intermittent updates. The framework is provided with guaranteed stability of the closed-loop system and optimality. Finally, simulation results show the efficacy of the proposed framework.

Keywords: Power systems, Machine learning, Stability of nonlinear systems
Abstract: Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems. However, its deployment in real-world power systems has been hindered by a lack of formal stability and safety guarantees. In this paper, we propose a stability constrained reinforcement learning method for real-time voltage control in distribution grids and we prove that the proposed approach provides a formal voltage stability guarantee. The key idea underlying our approach is an explicitly constructed Lyapunov function that certifies stability. We demonstrate the effectiveness of the approach in case studies, where the proposed method can reduce the transient control cost by more than 30% and shorten the response time by a third compared to a widely used linear policy, while always achieving voltage stability. In contrast, standard RL methods often fail to achieve voltage stability.

Keywords: Mechanical systems/robotics, Neural networks, Adaptive control
Abstract: This study proposes an innovative reinforcement learning-based time-delay control (RL-TDC) scheme to provide more intelligent, timely, and aggressive control efforts than the existing simple-structured adaptive time-delay controls (ATDCs) that are well-known for achieving good tracking performances in practical applications. The proposed control scheme adopts a state-of-the-art RL algorithm called soft actor critic (SAC) with which the inertia gain matrix of the timedelay control is adjusted toward maximizing the expected return obtained from tracking errors over all the future time periods. By learning the dynamics of the robot manipulator with a data-driven approach, and capturing its intractable and complicated phenomena, the proposed RL-TDC is trained to effectively suppress the inherent time delay estimation (TDE) errors arising from time delay control, thereby ensuring the best tracking performance within the given control capacity limits. As expected, it is demonstrated through simulation with a robot manipulator that the proposed RL-TDC avoids conservative small control actions when large ones are required, for maximizing the tracking performance. It is observed that the stability condition is fully exploited to provide more effective control actions.

Keywords: Nonlinear output feedback, PID control, Stability of nonlinear systems
Abstract: Ultra-Local Models (ULM) have been applied to perform model-free control of nonlinear systems with unknown or partially known dynamics. Unfortunately, extending these methods to MIMO systems requires designing a dense input influence matrix which is challenging. This paper presents guidelines for designing an input influence matrix for discrete-time, control-affine MIMO systems using an ULM-based controller. This paper analyzes the case that uses ULM and a model-free control scheme: the Hölder-continuous Finite-Time Stable (FTS) control. By comparing the ULM with the actual system dynamics, the paper describes how to extract the input-dependent part from the lumped ULM dynamics and finds that the tracking and state estimation error are coupled. The stability of the ULM-FTS error dynamics is affected by the eigenvalues of the difference (defined by matrix multiplication) between the actual and designed input influence matrix. Finally, the paper shows that a wide range of input influence matrix designs can keep the ULM-FTS error dynamics (at least locally) asymptotically stable. A numerical simulation is included to verify the result. The analysis can also be extended to other ULM-based controllers.

Keywords: Nonlinear systems identification, Machine learning, Neural networks
Abstract: We present a novel data-driven approach for learning linear representations of a class of stable nonlinear systems using Koopman eigenfunctions. Utilizing the spectral equivalence of topologically conjugate systems, we construct Koopman eigenfunctions corresponding to the nonlinear system to form linear predictors of nonlinear systems. The conjugacy map between a nonlinear system and its Jacobian linearization is learned via a diffeomorphic neural network. The latter allows for a well-defined, supervised learning problem formulation. Given the learner is diffeomorphic per construction, our learned model is asymptotically stable regardless of the representation accuracy. The universality of the diffeomorphic learner leads to the universal approximation ability for Koopman eigenfunctions - admitting suitable expressivity. The efficacy of our approach is demonstrated in simulations.

Keywords: Nonlinear systems identification, Machine learning
Abstract: In this paper, we present a new data-driven method for learning stable models of nonlinear systems. Our model lifts the original state space to a higher-dimensional linear manifold using Koopman embeddings. Interestingly, we prove that every discrete-time nonlinear contracting model can be learnt in our framework. Another significant merit of the proposed approach is that it allows for unconstrained optimization over the Koopman embedding and operator jointly while enforcing stability of the model, via a direct parameterization of stable linear systems, greatly simplifying the computations involved. We validate our method on a simulated system and analyze the advantages of our parameterization compared to alternatives.

Keywords: Nonlinear systems identification, Statistical learning
Abstract: Gaussian Process Regression (GPR) has been extensively used to estimate unknown models and quantify model uncertainty in control tasks concerning safety-critical applications. However, one of the drawbacks of GPR is that it does not take into account the function evaluations of the observations in the uncertainty estimation, rendering it unsuitable for applications with observations prone to outliers or to some unassumed noise disturbance. In this work, we introduce the Student-t Process Regression (TPR) as a generalization of GPR for estimating dynamics models in the control literature. The key attribute of TPR is that the estimation variance explicitly depends on the function evaluations rendering it more robust to outliers. We prove uniform error bounds for the estimation based on TPR under certain continuity assumptions. Furthermore, we employ TPR to estimate unknown and nonlinear dynamical systems and we show with control simulations that the resulting estimation uncertainty compensates for the existence of outliers. Such informative variance estimates are of vital importance as they can lead to more informative uncertainty propagation and thus less conservative control policies.

Keywords: Nonlinear systems identification, Variational methods, Filtering
Abstract: We propose a variational Bayesian inference procedure for online nonlinear system identification. For each output observation, a set of parameter posterior distributions is updated, which is then used to form a posterior predictive distribution for future outputs. We focus on the class of polynomial NARMAX models, which we cast into probabilistic form and represent in terms of a Forney-style factor graph. Inference in this graph is efficiently performed by a variational message passing algorithm. We show empirically that our variational Bayesian estimator outperforms an online recursive least-squares estimator, most notably in small sample size settings and low noise regimes, and performs on par with an iterative least-squares estimator trained offline.

Keywords: Learning, Observers for Linear systems, Uncertain systems
Abstract: We study a collaborative multi-agent stochastic linear bandit setting, where N agents that form a network communicate locally to minimize their overall regret. In this setting, each agent has its own linear bandit problem (its own reward parameter) and the goal is to select the best global action w.r.t. the average of their reward parameters. At each round, each agent proposes an action, and one action is randomly selected and played as the network action. All the agents observe the corresponding rewards of the played action, and use an accelerated consensus procedure to compute an estimate of the average of the rewards obtained by all the agents. We propose a distributed upper confidence bound (UCB) algorithm and prove a high probability bound on its T-round regret in which we include a linear growth of regret associated with each communication round. Our regret bound is of order mathcal{O}bigg( sqrt{frac{T}{N log(1/|lambda_2|)}} (log T)^2bigg), where lambda_2 is the eigenvalue of communication matrix with the second-largest absolute value.

Keywords: Communication networks, Machine learning, Optimization
Abstract: Software Defined Networking (SDN) is a recent paradigm in telecommunication networks that disentangles data and control planes and brings more flexibility to the network. The Controller Placement (CP) problem in SDN, which typically has a specific optimality criteria, is one of the primary problems in the SDN systems. Dynamic Controller Placement (DCP) enables a placement that is adaptable to inherent variability in network components. DCP has gained much attention in recent years, yet most solutions proposed in the literature cannot be implemented in real-time, which is a critical concern especially in UAV/drone based SDN networks where mobility is high and real-time updates are necessary. As conventional methods fail to be relevant to such scenarios, we propose a real-time control placement (RCP) algorithm. Namely, we propose a temporal clustering algorithm that provides real-time solutions for DCP, based on a control theoretic framework that is exponentially stable and converges to optimal placement of controllers. RCP has linear O(N) iteration complexity with respect to the underlying network size (N), and also leverages the maximum entropy principle from information theory. This approach results in high quality solutions that are practically immune from getting stuck in poor local optima, which is a serious drawback conventional methods. We compare our work with a frame-by-frame approach and show its superiority, both in terms of speed and incurred cost, via simulations. According to our simulations RCP can be up to 25 times faster than the conventional frame-by-frame methods.

Keywords: Distributed control, Communication networks, Optimization algorithms
Abstract: There has been an approximation-based distributed optimization algorithm that solves univariate non-convex problems to arbitrary precision. The key idea is to construct approximations of local objectives and address a more structured approximate version of the problem. By representing diverse local objectives with compressed coefficients vectors of approximations, such algorithms enjoy gradient-free iterations but face severe security issues when adversaries occur. In this paper, we propose a resilient approximation-based distributed nonconvex optimization algorithm termed R-ADOA to defend attacks from malicious nodes. First, errors caused by adversaries are quantified and unified as the perturbation of coefficient vectors of approximations. Next, we propose a filtering mechanism and resilient stopping mechanism to limit errors arising in consensus-based iterations. Finally, an upper bound of the deviations of the obtained solutions from optimal solutions is given based on the eigenvalue perturbation theory of matrices. Numerical experiments are provided to illustrate the effectiveness of our algorithm. Compared to existing resilient distributed optimization algorithms, R-ADOA addresses non-convex problems, converges exponentially fast, and contains explicit bounds for the deviations of solutions.

Keywords: Distributed control, Cooperative control
Abstract: In this paper, the distributed time-varying optimization problem is addressed for networked Lagrangian systems with parametric uncertainties. Usually, in the literature, to address some distributed control problems for nonlinear systems, a networked virtual system is constructed, and a tracking algorithm is designed such that the agents’ physical states tracks the virtual states. It is worth pointing out that such an idea requires the exchange of the virtual states and hence necessitates communication among the group. In addition, due to the complexities of the Lagrangian dynamics and the distributed time-varying optimization problem, there exist significant challenges. This paper proposes a distributed time-varying optimization algorithm achieving zero optimum tracking error for the networked Lagrangian agents without the communication requirement. The main idea behind the proposed algorithm is to construct a dynamic system for each agent to generate a reference velocity using absolute and relative physical state measurements with no exchange of virtual states needed, and to design adaptive controllers for Lagrangian systems such that the physical states are able to track the reference velocities and hence the optimal trajectory. The algorithm introduces mutual feedback between reference systems and local controllers via physical states/measurements and is amenable to implementation via local onboard sensing in a communication unfriendly environment.

Keywords: Agents-based systems, Distributed control, Optimization
Abstract: This paper considers the distributed convex optimization problem over directed multi-agent networks. We introduce a modified version of the distributed gradient descent method in continuous-time setting. In contrast to the existing literature, we do not assume that agents have any a–priori knowledge about their “out-degrees”. We show that the proposed network flow is guaranteed to converge, on any strongly connected digraph, to the global minimizer of a sum of convex functions provided that the aggregate objective function is strongly convex, the local cost functions have Lipschitzcontinuous gradients.

Keywords: Optimization, Time-varying systems, Distributed control
Abstract: The convergence of an error-feedback algorithm is studied for decentralized stochastic gradient descent (DSGD) algorithm with compressed information sharing over time-varying graphs. It is shown that for both strongly-convex and convex cost functions, despite of imperfect information sharing, the convergence rates match those with perfect information sharing. To do so, we show that for strongly-convex loss functions, with a proper choice of a step-size, the state of textit{each} node converges to the global optimizer at the rate of mathcal{O}left( T^{-1}right). Similarly, for general convex cost functions, with a proper choice of step-size, we show that the value of loss function at a textit{temporal average} of textit{each} node's estimates converges to the optimal value at the rate of mathcal{O}(T^{-1/2+epsilon}) for any epsilon>0.

Keywords: Control of networks, Time-varying systems, Optimization
Abstract: In this paper, we study strong structural controllability of linear time varying network systems that change network topology and edge weights with time. We derive graph based necessary and sufficient conditions for strong structural controllability of temporal networks. We provide a polynomial time algorithm to compute the dimension of the strong structurally controllable subspace of a temporal network. This allows us to verify whether a given set of inputs leads to a controllable system for all choices of numerical realizations. Next, we show that the problem of finding a minimum size input for strong structural controllability of temporal networks is an NP-hard problem. We propose a greedy heuristic algorithm to optimize the size of the input set for strong structural controllability of temporal networks and compare the performance of the greedy algorithm with the optimum solution through simulations.

Keywords: Predictive control for nonlinear systems, Optimal control, Nonlinear systems identification
Abstract: Model predictive control (MPC) is a state-of-the-art control method that can explicitly tackle system constraints. However, its high computational cost still remains an open challenge for embedded systems. To achieve satisfactory performance with manageable computational complexity, a spatial temporal filter (STF)-based data-driven predictive control framework is developed to systematically identify system dynamics and subsequently learn the MPC policy using STF-based function approximations. Specifically, an online nonlinear system identification method that satisfies persistence of excitation (PE) is developed by using a discrete-time concurrent learning technique. An STF-based function approximation is then employed to learn the nonlinear MPC (NMPC) policy based on the identified model. Furthermore, a discrete-time robust control barrier function (RCBF) is introduced to guarantee system safety in the presence of additive disturbances and system identification errors. Finally, simulations on the cart inverted pendulum are performed to demonstrate the efficacy of the proposed control synthesis.

Keywords: Predictive control for nonlinear systems, Chemical process control, Machine learning
Abstract: In this work, we present machine-learning-based predictive control schemes for nonlinear processes subject to disturbances, and investigate system stability properties using statistical machine learning theory. Specifically, we derive a generalization error bound via Rademacher complexity method for the recurrent neural networks (RNN) that are developed to capture the dynamics of the nominal system. Then, the RNN models are incorporated in Lyapunov-based model predictive controllers, under which we study closed-loop stability properties for the nonlinear systems subject to two types of disturbances: bounded disturbances and stochastic disturbances with unbounded variation.

Keywords: Predictive control for nonlinear systems, Constrained control, Biomedical
Abstract: Thermal management of bio-implant and implantable medical devices (IMDs) has gained growing attention to prevent overheating in the surrounding tissue of IMDs in certain applications, such as neural prostheses like deep brain stimulators (DBS). This paper focuses on implementation of nonlinear model predictive control (NMPC) methods for adaptive thermal management of IMDs with multiple heat sources. Thermal dynamics of the IMD is modelled using an identification algorithm introduced in previous papers. Interior point optimization method is implemented with the NMPC to solve for the nonlinear optimization problem for adaptive thermal management of IMDs with multiple heat sources. The NMPC implementation is validated using the COMSOL software simulations.

Keywords: Predictive control for nonlinear systems, Constrained control, Process Control
Abstract: Barrier functions have been utilized in guaranteeing safety for control laws, and their coupling with conditions which guarantee that the closed-loop state can be driven to the origin has also been important for achieving performance objectives. For optimization-based control, approaches have been developed to guarantee safety via barrier functions while simultaneously guaranteeing that a reference or equilibrium can be tracked. One approach has assumed an instantaneous computation of the control action from a quadratic programming-based control law that determines control actions which satisfy a decreasing Lyapunov condition when possible but always satisfy a barrier function-based safety constraint. This approach decouples safety and performance objectives to prefer the safety objective over performance when they conflict. In an alternative approach, a single function is located which, when decreased, guarantees that safety and performance objectives are met simultaneously, and has been implemented within the context of model predictive control. In this work, we discuss connections between barrier function-based and Lyapunov function-based control approaches to maintaining safety.

Keywords: Predictive control for nonlinear systems, Machine learning, Optimal control
Abstract: Employing learned Gaussian process models in nonlinear model predictive control raises the problem of repeat- edly propagating a probability distribution through a nonlinear mapping, which is a challenging task. Existing solutions are either computationally expensive or conservative. We propose to use Gaussian process models that directly yield the entire state sequence without repeated evaluations. As therefrom an exact Gaussian distribution is obtained in each step on the prediction horizon, an increased prediction quality is achieved when compared to employing iterated models. The proposed approach is illustrated in a simulation study, where we show the quality gain in the open-loop state predictions, as well as in the closed-loop performance.

Keywords: Markov processes, Stochastic systems, Large-scale systems
Abstract: A system of multiple agents is considered which at random times change their discrete states on an Ising lattice as a results of their internal interactions and possibly some external control. For certain applications such as directed self-assembly of charged particles, the stochastic dynamics of such interacting agents is represented by a master equation, or equivalently, by a continuous-time Markov chain. The dimension of this master equation is typically large and numerically intractable, since it grows combinatorially with the lattice size. This paper presents two alternative models at significantly lower complexity growing polynomially with the size of Ising lattice. These models describe the interactive dynamics of the agents by two different classes of coupled stochastic differential equations driven by doubly stochastic Poisson processes (Cox processes).

Keywords: Markov processes, Stochastic systems, Stochastic optimal control
Abstract: We investigate the problem of designing optimal stealthy poisoning attacks on the control channel of Markov decision processes (MDPs). This research is motivated by the recent interest of the research community for adversarial and poisoning attacks applied to MDPs, and reinforcement learning (RL) methods. The policies resulting from these methods have been shown to be vulnerable to attacks perturbing the observations of the decision-maker. In such an attack, drawing inspiration from adversarial examples used in supervised learning, the amplitude of the adversarial perturbation is limited according to some norm, with the hope that this constraint will make the attack imperceptible. However, such constraints do not grant any level of undetectability and do not take into account the dynamic nature of the underlying Markov process. In this paper, we propose a new attack formulation, based on information-theoretical quantities, that considers the objective of minimizing the detectability of the attack as well as the performance of the controlled process. We analyze the trade-off between the efficiency of the attack and its detectability. We conclude with examples and numerical simulations illustrating this trade-off.

Keywords: Markov processes, Stochastic systems
Abstract: We study constrained Markov decision processes with finite state and action spaces. The optimal solution of a discounted infinite-horizon optimal control problem is obtained using a Policy Gradient Primal-Dual (PG-PD) method without any policy parametrization. This method updates the primal variable via projected policy gradient ascent and the dual variable via projected sub-gradient descent. Despite the lack of concavity of the constrained maximization problem in policy space, we exploit the underlying structure to provide non-asymptotic global convergence guarantees with sublinear rates in terms of both the optimality gap and the constraint violation. Furthermore, for a sample-based PG-PD algorithm, we quantify sample complexity and offer computational experiments to demonstrate the effectiveness of our results.

Keywords: Optimal control, Switched systems, Markov processes
Abstract: We propose a model for path-planning based on a single performance metric that accurately accounts for the the potential (spatially inhomogeneous) cost of breakdowns and repairs. These random breakdowns (or system faults) happen at a known, spatially inhomogeneous rate. Our model includes breakdowns of two types: total, which halt all movement until an in-place repair is completed, and partial, after which the movement continues in a damaged state toward a repair depot. We use the framework of piecewise-deterministic Markov processes to describe the optimal policy for all starting locations. We also introduce an efficient numerical method that uses hybrid value-policy iterations to solve the resulting system of Hamilton-Jacobi-Bellman PDEs. Our method is illustrated through a series of computational experiments that highlight the dependence of optimal policies on the rate and type of breakdowns, with one of them based on Martian terrain data.

Keywords: Stochastic optimal control, Markov processes
Abstract: We develop a risk-averse safety analysis method for stochastic systems on discrete infinite time horizons. Our method quantifies the notion of risk for a control system in terms of the severity of a harmful random outcome in a fraction of the worst cases. In contrast, classical methods quantify risk in terms of the probability of a harmful event. Our theoretical arguments are based on the analysis of a value iteration algorithm on an augmented state space. We provide conditions to guarantee the existence of an optimal policy on this space. We illustrate the method numerically using an example from the domain of stormwater management.

Keywords: Statistical learning, Switched systems, Markov processes
Abstract: Real-world control applications often involve complex dynamics subject to abrupt changes or variations. Markov jump linear systems (MJS) provide a rich framework for modeling such dynamics. Despite an extensive history, theoretical understanding of parameter sensitivities of MJS control is somewhat lacking. Motivated by this, we investigate robustness aspects of certainty equivalent model-based optimal control for MJS with a quadratic cost function. Given the uncertainty in the system matrices and in the Markov transition matrix is bounded by epsilon and eta respectively, robustness results are established for (i) the solution to coupled Riccati equations and (ii) the optimal cost, by providing explicit perturbation bounds that decay as mathcal{O}(epsilon + eta) and mathcal{O}((epsilon + eta)^2) respectively.

Keywords: Manufacturing systems, Iterative learning control, Optimization algorithms
Abstract: This paper presents a model-free iterative learning control (ILC) scheme for multi-objective temperature control in Selective Laser Melting (SLM). The goal is to ensure that while temperature distribution in the selected region is sufficient to cause melting and fusion, the meltpool is not overheated. We first formulate this goal as an optimization problem with the power profile as the decision variable and the cost function to be minimized being the sum of two unidirectional error terms (for upper and lower temperature bounds, respectively). Given the difficulty in analytically modeling the temperature-laser power relationship in SLM for gradient computations as in standard ILC, we solve the minimization problem using a model-free ILC scheme. In this scheme, the control input that minimizes the cost function is learned through a data-driven gradient descent update that uses the process itself to compute the gradient direction. The gradient descent algorithm proposed here accounts for the time-varying behavior of the SLM thermal dynamics because of the scan path. This is accomplished by feeding the temperature output error, reversed in time, through the process itself with a reversed scan path direction. For validation, this multi-objective gradient-based ILC algorithm is implemented on a three-phase high-fidelity simulation of the SLM process. The results demonstrate the algorithm's ability to drive the temperature distribution to within a prescribed range in scenarios where standard (single-objective constant gain) ILC fails.

Keywords: Manufacturing systems, Iterative learning control
Abstract: Selective Laser Melting (SLM) is an Additive Manufacturing (AM) technique with challenges in its complexity of process parameters and lack of control schemes. Traditionally, people tried time-domain or frequency-domain control methods, but the complexity of the process goes beyond these methods. In this paper, a novel spatial transformation of SLM models is proposed, which transforms the time-domain process into a spatial domain model and, thus, allows for state-space layer-to-layer control methods. In a space domain, this also provides the convenience of modelling laser path changes. Finally, a layer-to-layer Iterative Learning Control (ILC) method is designed and demonstrates the methodology of spatial control for SLM. A simulation demonstrates its application and performance.

Keywords: Machine learning, Autonomous systems, Manufacturing systems
Abstract: Here we propose an algorithm that combines two classic ideas, transfer learning and temporal abstraction, to accelerate learning in high variable cost environments (HVC-envs). In an HVC-env, each sampling of the environment incurs a high cost, thus methods to accelerate learning are sought to reduce the incurred cost. Transfer learning can be useful for such environments by using prior knowledge from a source environment. As only a small number of samples can be collected from an HVC-env due to high sampling cost, learning becomes challenging when the source environment provides inaccurate rewards. To overcome this challenge we propose a simple but effective way of creating useful temporally extended actions from an inaccurate physics guided model (PGM) that acts as the source task. At first we address this issue theoretically by providing performance bounds between two semi-Markov Decision Processes (SMDPs) with different reward functions. Later we develop two benchmark HVC-envs where learning must happen using a small number of real samples (often on the order of ∼ 10 2 or 10 3 ). Finally we show that it is possible to obtain sequential high rewards in both of these environments using ∼ 10 3 real samples by leveraging knowledge from PGMs with inaccurate reward models.

Keywords: Robotics, Neural networks, Control applications
Abstract: In this paper, a data-driven modeling and control framework is developed for task space control of a soft robot gripper which consists of four individual soft fingers. Each of the four fingers is modeled as a manipulator with high degrees of freedom. The corresponding task space dynamics of the manipulator are derived using a rigid-link approximation of the continuum manipulator. A neural network approach is used to learn the derived dynamics in State Dependent Coefficient (SDC) form. Using the learned SDC matrices, an asymptotically stable optimal closed-loop tracking controller which is based on solving the State Dependent Riccati Equation (SDRE) is derived. The model learning and trajectory tracking controller is implemented on an open source Soft Motion (SoMo) platform simulating the soft gripper motion and corresponding tracking results are presented.

Keywords: Estimation, Reduced order modeling, Optimization
Abstract: In recent years, there has been significant interest in use of soft and continuum manipulators in diverse fields including surgical and agricultural robotics. Consequently, researchers have designed open-loop and feedback control algorithms for such systems. Here, the knowledge of the manipulator shape is critical for effective control. The estimation of the manipulator shape is challenging due to their highly deformable and non-linear nature. Researchers have explored inductive, magnetic and optical sensing techniques to deduce the manipulator shape. However, they are intrusive and economically expensive. Alternate non-contact sensing approaches may involve use of vision or inertial measurement units (IMUs) that are placed at known intervals along the manipulator. Here, the camera provides position of the marker, while the slope (rotation matrix or direction cosines) can be determined using IMUs. In this paper, we mathematically model the manipulator shape using multiple piecewise continuous quintic Pythogorean-Hodograph (PH) curves. A PH-curve has continuous slope and is a convenient parametric model for curves with constant length. We investigate the use of multiple piecewise continuous-curvature PH curves to estimate the shape of a soft continuum manipulator. The curves model manipulator segments of constant lengths while the slopes at the knots are assumed to be known. For N curve segments with (4N+8) unknowns, the shape estimation is formulated as a constrained optimization problem that minimizes the curve bending energy. The algorithm imposes (4N+3) nonlinear constraints corresponding to continuity, slope and segment length. Unlike traditional cubic splines, the optimization problem is nonlinear and sensitive to initial guesses and has potential to provide incorrect estimates. We investigate the robustness of the algorithm by adding variation to the direction cosines, and compare the output shapes. The simulation results on a five-segment manipulator illustrate the robustness of the algorithm. While the experimental results on a soft tensegrity-spine manipulator validate the effectiveness of the approach. Here max estimation error of the end-effector position normalized to manipulator length is 6.53%.

Keywords: Modeling, Simulation, Mechanical systems/robotics
Abstract: Soft robots have been intensively investigated for manipulation and locomotion in recent years. However, the current state of soft robotics has significant design and development work but lags in modeling and control due to the difficulty in modeling them. In this paper, we present a physics-based analytical framework to model soft robots driven by Twisted-and-Coiled Actuators (TCAs), an artificial muscle that can be arranged in arbitrary shapes in the soft body of a soft robot to achieve programmable motions. The framework can model 1) the complicated routes of multiple TCAs in a soft body and 2) the coupling effect between the soft body and the TCAs during their actuation process. When not actuated, a TCA in the soft body is an antagonistic elastic element that restrains the magnitude of the motion and increases the stiffness of the robot. By stacking several modules together, we simulate the sequential motion of a soft robotics arm with three-dimensional bending, twisting, and grasping motion. The presented modeling and simulation approach will facilitate the design, optimization, and control of soft robots driven by TCAs or other types of artificial muscles.

Keywords: Optimal control, Learning
Abstract: The paper develops a feature learning-based method to solve optimal control problems using B-splines to represent the optimal solutions. The feature learning-based optimal control method can quickly generate near-optimal trajectories for general optimal control problems subject to system dynamics and constraints. The steps in the proposed method are as follows: Firstly, by representing the state and control variables with B-spline functions, the optimal control problem is equivalently converted into a nonlinear programming (NLP) problem, where parameters of the B-splines are identified as features of the optimal solution. Secondly, for a specific problem with designated inputs, a dataset of the optimal trajectories under varying inputs is generated by solving the corresponding NLP problem offline. Finally, the neural network is applied to map the relationship between the designated inputs and identified features, represented by the parameters of B-splines and time variables. To show the effectiveness of the proposed method for solving the optimal control problem, simulation results are presented and analyzed in the simulation section.

Keywords: Optimal control, Linear systems, Kalman filtering
Abstract: The paper provides a data integrity cyber-attack detection framework based on minimum robust positively invariant sets. A general linear control system with a Kalman filter is considered. The set localization of the state estimator error is taken into account for developing the attack detector. An intelligent attacker algorithm is developed that has access to a subset of signals from the sensor and actuator channel of the control system. It is assumed that the attacker possesses the entire control system model to perform the most proficient attack for a certain set-up of data availability and compromisation. The attacker compromises a set of measurement data under the constraint of remaining non-discovered by the detector. The presented methodology allows assessing the effectiveness of the control system defense achievable in various data integrity attack scenarios. The developed detector and attacker algorithm were implemented on an illustrative example of a power system with two control areas and automatic generation control.

Keywords: Optimal control, Linear systems, Learning
Abstract: In this paper, we analyze the optimization landscape of gradient descent methods for static output feedback (SOF) control of discrete-time linear time-invariant systems with quadratic cost. The SOF setting can be quite common, for example, when there are unmodeled hidden states in the underlying process. We first establish several important properties of the SOF cost function, including coercivity, L-smoothness, and M-Lipschitz continuous Hessian. We then utilize these properties to show that the gradient descent is able to converge to a stationary point at a dimension-free rate. Furthermore, we prove that under some mild conditions, gradient descent converges linearly to a local minimum if the starting point is close to one. These results not only characterize the performance of gradient descent in optimizing the SOF problem, but also shed light on the efficiency of general policy gradient methods in reinforcement learning.

Keywords: Optimal control, Multivehicle systems, Spacecraft control
Abstract: Indirect formalism of optimal control theory is used to generate minimum-time and minimum-fuel trajectories for formation of two spacecraft (deputies) relative to a chief satellite. For minimum-fuel problems, a hyperbolic tangent smoothing method is used to facilitate numerical solution of the resulting boundary-value problems by constructing a one-parameter family of smooth control profiles that asymptotically approach the theoretically optimal, but non-smooth bang-bang thrust profile. Impact of the continuation parameter on the solution of minimum-fuel trajectories is analyzed. The fidelity of the dynamical model is improved beyond the two-body dynamics by including the perturbation due to the Earth's second zonal harmonic, J2. In addition, a particular formation is investigated, where the deputies are constrained to lie diametrically opposite on a three-dimensional sphere centered at the chief.

Keywords: Optimal control, Nonholonomic systems
Abstract: We consider the problem of time-optimal path planning for simple nonholonomic vehicles. In previous similar work, the vehicle has been simplified to a point mass and the obstacles have been stationary. Our formulation accounts for a rectangular vehicle, and involves the dynamic programming principle and a time-dependent Hamilton-Jacobi-Bellman (HJB) formulation which allows for moving obstacles. To our knowledge, this is the first HJB formulation of the problem which allows for moving obstacles. We design an upwind finite difference scheme to approximate the equation and demonstrate the efficacy of our model with a few synthetic examples.

Keywords: Optimal control, Numerical algorithms, Linear systems
Abstract: Interpolating or smoothing splines both require the inversion of a matrix of a size proportional to the total number of data points. This becomes a prohibitively costly proposition for large data sets and constitutes an inappropriate approach when the data is incremental or the underlying distribution from which the data is drawn is time-varying. In this paper, an algorithm is developed for the construction of recursive smoothing splines based on Calculus of Variations that can handle incremental and intermediate-sized batches of data sets, as well as data sets that are modified over time. The algorithm uses the previous data in the construction of the updated curve and a numerically efficient algorithm is developed to solve the problem.

Keywords: Observers for Linear systems, Robust control, Time-varying systems
Abstract: This paper proposes a computationally efficient and traceable way to synthesize finite horizon linear time-varying (LTV) output feedback controllers. It is based on a separate observer and state feedback synthesis with guaranteed performance in a mixed sensitivity setting. The approach avoids a grid-wise evaluation of coupled synthesis conditions that limits existing output feedback syntheses and instead uses two subsequent steps. However, it guarantees the same performance as the original output feedback problem. A trajectory tracking controller for an ascending space launcher in the earth's atmosphere demonstrates the feasibility of the approach.

Keywords: Filtering, Information theory and control, Networked control systems
Abstract: In this paper, we investigate distributed filtering in continuous-time scenarios building on an information-theoretic view of Kalman-Bucy filtering. We consider a two-node system where each node is associated with a time-varying state and obtains noisy observations of both nodal states at each time. In addition, one of the two nodes receives encoded messages from the other node via a Gaussian channel with feedback and infers its current state based on available observations and received messages. We design a real-time encoding strategy for generating the transmitted messages and show under which conditions such a strategy is optimal. Moreover, we present a relation between information dissipation rate and Fisher information for distributed filtering. Our finding is an extension of the connection between Shannon and Fisher information for Kalman-Bucy filtering, which was established by Mitter and Newton.

Keywords: Observers for nonlinear systems, Estimation
Abstract: This paper proposes a novel unified interval-valued observer synthesis approach for locally Lipschitz nonlinear continuous-time (CT) and discrete-time (DT) systems with nonlinear observations. A key feature of our proposed observer, which is derived using mixed-monotone decompositions, is that it is correct by construction (i.e., the true state trajectory of the system is framed by the states of the observer) without the need for imposing additional constraints and assumptions such as global Lipschitz continuity or contraction, as is done in existing approaches in the literature. Furthermore, we derive sufficient conditions for designing stabilizing observer gains in the form of Linear Matrix Inequalities (LMIs). Finally, we compare the performance of our observer design with some benchmark CT and DT observers in the literature.

Keywords: Observers for nonlinear systems, Stability of nonlinear systems, Estimation
Abstract: Controller confidentiality under sensor attacks refers to whether the internal states of the controller can be estimated when the adversary knows the model of the plant and controller, while only having access to sensors, but not the actuators. We show that the controller's state can be estimated accurately when the nonlinear closed-loop system is detectable. In the absence of detectability, controller confidentiality can still be breached with a periodic probing scheme via the sensors under a robust observability assumption, which allows for the controller's state to be estimated with arbitrary accuracy during the probing period, and with bounded error during the non-probing period. Further, stealth can be maintained by choosing an appropriate probing duration. This study shows that the controller confidentiality for nonlinear systems can be breached by balancing the estimation precision and the stealthiness of the adversary.

Keywords: Observers for nonlinear systems
Abstract: This paper shows the existence of robust functional observers for autonomous dynamical systems that verify a backward-distinguishability condition with respect to the functional to be estimated. The proof leverages on the theory of Kazanzis-Kravaris/Luenberger (KKL) observer design, which is adapted to the functional context. Then, we show how those results can be exploited to show existence of asymptotic observers for controlled systems under appropriate distinguishability conditions, when the input is known to be generated by some finite-dimensional autonomous dynamical system. This is done by considering an extended system made of the plant and the input generator. Applications include state estimation in presence of known/unknown inputs, unknown input observers and input reconstruction/estimation. On the other hand, when the input can only be approximated by such signals, practical functional estimation may be achieved by exploiting the observer robustness.

Keywords: Stability of nonlinear systems, Observers for nonlinear systems, Lyapunov methods
Abstract: This paper addresses stability of systems under state estimate feedback where the estimate error dynamics exhibit uniform, semi-global, practically asymptotic stability. That is, the state estimate error converges to an arbitrarily small positively invariant within an arbitrarily large stability basin. The analysis uses the notion of set stability, which is especially well suited to scenarios where the estimate error is only ultimately bounded. We show that, if a certain timing condition holds, the observer parameters can be chosen such that trajectories of the closed-loop system converge to an asymptotically stable set. A further corollary indicates a control design strategy that ensures the set's asymptotic stability is uniform in time, provided the observer's convergence rate can be made arbitrarily fast.

Keywords: Biomedical, Delay systems, Markov processes
Abstract: Case isolation, that is, detection and isolation of infected individuals in order to prevent spread, is a strategy to curb infectious disease epidemics. Here, we study the efficiency of a case isolation strategy subject to time delays in terms of its ability to stabilize the epidemic spread in heterogeneous contact networks. For an SIR epidemic model, we characterize the stability boundary analytically and show how it depends on the time delay between infection and isolation as well as the heterogeneity of the inter-individual contact network, quantified by the variance in contact rates. We show that network heterogeneity accounts for a restricting correction factor to previously derived stability results for homogeneous SIR models (with uniform contact rates), which are therefore too optimistic on the relevant time scales. We illustrate the results and the underlying mechanisms through insightful numerical examples.

Keywords: Biological systems, Biomedical, Estimation
Abstract: This paper identifies a state-space model of the impact of the peritoneal perfusion of an oxygenated perfluorocarbon (PFC) on the dynamics of carbon dioxide (CO_2) transport in a large laboratory animal. Previous research shows that such perfusion has the potential to enable the peritoneal cavity to serve as a "third lung" that supplements oxygenation during hypoxia. However, the effect of this potential treatment modality on CO_2 transport dynamics remains relatively unexplored. The paper addresses this gap by: (i) proposing a three-compartment model of CO_2 transport dynamics; (ii) utilizing time scale separation to simplify it into a residualized single-compartment model; and (iii) parameterizing the model using experimental data. Two experimental datasets are used for parameterization, involving the use of reduced minute ventilation to induce hypercarbia both (i) with and (ii) without PFC perfusion. Fisher analysis is used for quantifying the resulting model parameter uncertainties. The outcomes of this analysis strongly suggest a positive impact of perfusion on CO_2 clearance, with further validation experiments planned as future work.

Keywords: Biomedical, Differential-algebraic systems, Modeling
Abstract: The lack of methods to evaluate mechanical function of donated hearts in the context of transplantation imposes large precautionary margins, translating into a low utilization rate of donor organs. This has spawned research into cyber-physical models constituting artificial afterloads (arterial trees), that can serve to evaluate the contractile capacity of the donor heart.

The Windkessel model is an established linear time-invariant afterload model, that researchers committed to creating a cyber-physical afterload have used as a template. With aortic volumetric flow as input and aortic pressure as output, it is not directly obvious how a Windkessel model will respond to changes in heart contractility.

We transform the classic Windkessel model to relate power, rather than flow, to pressure. This alters the model into a differential-algebraic equation, albeit one that is straightforward to simulate. We then propose a power signal model, that is based on pressure and flow measurements and optimal in a Bayesian sense within the class of C2 signals. Finally, we show how the proposed signal model can be used to create relevant simulation scenarios, and use this to illustrate why it is problematic to use the Windkessel model as a basis for designing a clinically relevant artificial afterload.

Keywords: Biomedical, Kalman filtering, Mechatronics
Abstract: This paper introduces a new invariant extended Kalman filter design that produces real-time state estimates and rapid error convergence for the estimation of the human body movement even in the presence of sensor misalignment and initial state estimation errors. The filter fuses the data returned by an inertial measurement unit (IMU) attached to the body (e.g., pelvis or chest) and a virtual measurement of zero stance-foot velocity (i.e., leg odometry). The key novelty of the proposed filter lies in that its process model meets the group affine property while the filter explicitly addresses the IMU placement error by formulating its stochastic process model as Brownian motions and incorporating the error in the leg odometry. Although the measurement model is imperfect (i.e., it does not possess an invariant observation form) and thus its linearization relies on the state estimate, experimental results demonstrate fast convergence of the proposed filter (within 0.2 seconds) during squatting motions even under significant IMU placement inaccuracy and initial estimation errors.

Keywords: Switched systems, Adaptive control, Human-in-the-loop control
Abstract: Hybrid exoskeletons integrate powered exoskeletons and functional electrical stimulation (FES) to restore limb function and improve muscle capacity. However, technical challenges exist to customize the control of hybrid devices due to the nonlinear, uncertain gait and muscle dynamics of the human-machine system. Different from optimization techniques for gait control that leverage extensive model knowledge, this paper exploits a learning-based adaptive strategy to provide torque assistance about the hip and knee joints using a cable-driven exoskeleton with FES for treadmill walking. The human-machine system is modeled with phase-dependent switched pendular dynamics to capture gait phase transitions. A concurrent learning adaptive controller is designed to estimate a subset of the uncertain leg parameters during the swing phase to improve gait control. A sliding-mode controller provides robust leg support during stance. Stability of the overall switched system is proven using a multiple Lyapunov approach and dwell time analysis to guarantee exponential tracking and parameter estimation convergence across gait phase transitions.

Keywords: Hybrid systems, Lyapunov methods, Healthcare and medical systems
Abstract: Motorized functional electrical stimulation (FES) cycling can serve as a physical rehabilitation strategy for individuals whose lower limbs are affected by neurological injuries. Motorized FES cycling is unique among human-robot interaction tasks because the cycle’s motor and rider’s leg muscles must be simultaneously controlled. In this paper, two tracking objectives are proposed for the combined cycle-rider system. First, the rider’s leg muscles are stimulated to pedal the cycle at a desired cadence, and second, the cycle’s motor is used to regulate the interaction torque between the cycle and rider using an admittance controller. A novel hybrid systems analysis using Lyapunov- and passivity-based techniques is conducted to prove that the cycle’s motor can globally exponentially regulate the admittance error system and prove that the cadence error system is output feedback passive. Experiments conducted on five participants illustrate the efficacy of the controllers with an average admittance error of -0.01±0.70 RPM at an average cadence of 45.51±1.77 RPM for a desired cadence of 50 RPM.

Keywords: Energy systems, Automotive systems, Machine learning
Abstract: Diesel air handling systems are becoming increasingly complex and commonly feature technologies such as exhaust gas recirculation (EGR) and variable geometry turbochargers (VGTs) in order to meet stringent emissions and fuel economy requirements. The control of such systems is challenging because they are nonlinear and have coupled dynamics. In this paper, artificial neural networks (ANNs) and recurrent neural networks (RNNs) are leveraged to control the low pressure (LP) EGR valve position and VGT vane position simultaneously on a light-duty multi-cylinder diesel engine. Intake manifold pressure (IMP) and the air–fuel equivalence ratio (lambda) are selected as control objectives, since they are directly relevant to engine emissions and power output. In addition, both signals are available on production engines so no additional hardware costs will be introduced. Steady-state experimental data is used to train the neural networks (NNs). The ANNs and RNNs with the minimum mean square error (MSE) for training data sets are further compared to conventional proportional-integral (PI) control with the validation data sets. Both of the selected NN controllers show almost no overshoot during the transient process and have steady-state errors of less than 4.0% for IMP or lambda, thus showing potential for further use in engine air handling control.

Keywords: Automotive control, Automotive systems, Control applications
Abstract: To optimize combustion efficiency, it is often desired to operate the engine as close to its borderline knock as possible. However, detecting borderline knock is a time- consuming process through an engine mapping process. This paper applies a machine learning algorithm, namely the stochastic Bayesian optimization, to efficiently detect borderline knock based on a tradeoff relationship between knock intensity and fuel economy, considering both system and measurement noises. A dual-surrogate model structure, along with a dis- tribution mapping process, is proposed for implementing the Bayesian iterative optimization with two competing objectives (knock intensity and indicated specific fuel consumption) and two control inputs (spark timing and intake valve timing). The proposed algorithm is validated by running the engine bench test with a pre-defined test budget. Finally, the optimized control parameters are found based on trained surrogate models and guarantee that the engine runs right below the borderline knock with the best fuel economy possible.

Keywords: Automotive control, Optimal control
Abstract: In this paper, a computationally efficient two-path non-linear optimal control allocation method is proposed to improve the yaw stability of four-wheel-independent-steering, four-wheel-drive vehicles. The virtual controller output is allocated using an optimization problem to compute each wheel's steering and traction commands at every controller time step. The optimization problem is solved by running a sequential quadratic programming (SQP) procedure, which may take some time to obtain satisfactory results. The proposed two-path control structure is derived from a more complex single-path allocation problem where torque allocation and steering correction optimal solutions are calculated concurrently. In this separated two-path control structure, computational load due to the complexity of the single block problem is reduced. In real applications, each problem can be run in parallel on different controllers of the vehicle controller network, which decreases the execution time with near-optimal results. The performance and speed comparisons of both approaches are studied using detailed vehicle simulations.

Keywords: Automotive control, Constrained control
Abstract: In this paper, an internal model control (IMC) control scheme combined with reference governor (RG) for constraint management is proposed to regulate a nonlinear multivariable air-path system for a proton exchange membrane fuel cell (PEMFC) system. The control objectives are to avoid oxygen starvation, run at the maximum net efficiency, achieve fast tracking of air flow and pressure set-points, and be easy to calibrate. To operate at maximum efficiency, a set-point map is generated for air pressure at the cathode inlet. Considering that the conventional PEMFC system cannot independently control the inlet pressure using only the compressor motor, we formulated a new multivariable analysis and control scheme by considering an electronic throttle body (ETB) valve downstream of the cathode as a new degree of freedom in the control problem. Based on this new configuration of the fuel cell model, we designed an IMC controller with intuitive tuning parameters to simultaneously control airflow and pressure and achieve a fast and smooth response despite strongly coupled plant dynamics. Further, a RG using a computationally tractable linear prediction model is included with IMC-based Multi-Input Multi-Output (MIMO) controller to satisfy the constraint on oxygen level. Compared with a Single-Input Single-Output (SISO) air-flow control approach, the proposed MIMO control approach demonstrated up to 7.36 percent lower hydrogen fuel consumption.

Keywords: Automotive control, Automotive systems, Predictive control for linear systems
Abstract: This paper presents the results of a model predictive controller (MPC) development for diesel engine air-path regulation. The control objective is to track the intake manifold pressure and exhaust gas recirculation (EGR) rate targets by manipulating the EGR valve and variable geometry turbine (VGT) while satisfying state and control constraints. The MPC controller is designed and verified using a high-fidelity engine model in GT-Power. The controller exploits a low-order rate-based linear parameter-varying (LPV) model for prediction which is identified from transient response data generated by the GT-Power model. It is shown that transient engine thermal dynamics influence the airpath dynamics, specifically the intake manifold pressure response, however, MPC demonstrates robustness against inaccuracies in modeling these thermal dynamics. In particular, we show that MPC can be successfully implemented using a rate-based prediction model with two inputs (EGR and VGT positions) identified from data with steady-state wall temperature dynamics, however, closed-loop performance can be improved if a prediction model (i) is identified from data with transient thermal dynamics, and (ii) has the fuel injection rate as extra model input. Further, the MPC calibration process across the engine operating range to achieve improved performance is addressed. As the MPC calibration is shown to be sensitive to the operating conditions, a fast calibration process is proposed.

Keywords: Supervisory control, Control applications, Switched systems
Abstract: This paper presents a drive mode control method for plug-in hybrid electric vehicles (PHEVs) to reduce fuel consumption and emissions. In the PHEVs, a management strategy for efficiently allocating energy resources is drive mode control according to driving routes. The drive mode control practically requires a catalyst converter to have a high temperature before starting the engine for emissions reduction. To incorporate the temperature condition into the energy management, we propose an optimization framework of the drive mode control embracing catalyst models. The newly developed temperature model employs a discrete mixed logical dynamical system representation to lessen the optimization’s computational burden. The proposed method is verified by an advanced simulator with real-world datasets. The simulation results exhibit that the proposed method achieved a 1.1 % reduction in fuel consumption than that of a commercial strategy even under a severe but realistic requirement. The comparative studies of various catalyst conditions reveal how the temperature requirement influences the performance of the drive mode strategy.

Keywords: Human-in-the-loop control, Control applications
Abstract: The principles and methodology of control science and engineering are relevant beyond engineered systems to all dynamical systems. In this paper we discuss how control terminology and concepts can be interpreted in the context of decision-making by human managers, and the benefits that can accrue from the analogy. Examples of managerial decision making are presented in a control systems context. A selected review of earlier literature in the area is included. The connections of some popular business practices with control principles are reviewed. Points of differentiation, especially human-in-the-loop aspects of managerial control, are highlighted. Two examples, on return-on-investment dynamics and short interval control in process plant operation, are discussed. A number of control concepts are mapped to the management domain, revealing what we hope are useful insights for decision makers. This is the main paper for a tutorial session at the 2022 American Control Conference. An appendix is included that contains abstracts of the other papers and presentations in the session.

Keywords: Human-in-the-loop control, Control applications
Abstract: Many categories of manager decision-making (human organisations, environmental systems, supply chains) are in some way or another related to systems. However, system dynamics do not always lend themselves well to superficial, or intuitive, interpretation. This can inadvertently result in sub-optimal managerial decision-making. The application of control science concepts for guiding managerial decision-making has the potential to improve results. The contemporary manager is typically resource-constrained and time-stressed. Control-science-based decision-making guidance that does not accommodate the reality of the manager's time constraints may have limited affect in practice. A manager's attention-scheduling behaviour is more analogous to that of a networked control system (NCS) than to that of a singularly focused control loop.

There is an opportunity to apply NCS aspects of control science to identify the minimum attention/frequency requirements of key decision-making realms. This paper acknowledges the time-poor reality of the contemporary manager. It considers how learnings from NCS theory can be applied to add resilience and efficiency to control-science-inspired improvements to manager decision-making. Just as regulatory control systems don't perform well when subjected to unexpected network or input/output delays, the application of control theory concepts to manager decision-making will be challenged if the time-poor aspects of the manager are not catered for.

Keywords: Human-in-the-loop control, Control applications
Abstract: This paper asks: how do we apply the fundamental principles of feedback in physical systems to business processes? This is a tempting idea because feedback is clearly present in business/decision processes, but as in the case of feedback of biological systems, getting beyond the qualitative and phenomenological descriptions to models with structure for which parameters can be determined from measurements is difficult. In this context, what can feedback principles, so often based on rigid mathematical analysis, provide to such systems for which any mathematical rigor is hard to find? Our approach in this section will be inspired by the words of Captain Barbosa in Pirates of the Caribbean, as to think of fundamental feedback principles as guidelines, rather than actual rules. That being said, we believe those guidelines provide a rich source of correction for business processes. In the end our feedback-fundamentals inspired guidelines may not guarantee us only correct decisions, but they can keep us away from practices we would never try in engineering systems.

Keywords: Flexible structures, Control applications, Modeling
Abstract: The study of variable-shape wave energy converters has been receiving more attention recently. In this work, a dynamic model for axisymmetric Variable-Shape Buoy Wave Energy Converters is derived within the context of Lagrangian mechanics. The assumed modes method is used to approximate the flexible modes of the buoy. Passive control is adopted that implements a bang-bang control algorithm. A numerical simulation case study is presented that demonstrates the utility of the developed model in studying the behavior of the flexible shell buoy and the overall performance of the spherical Variable-Shape buoy Wave Energy Converter.

Keywords: Maritime control, Energy systems, Fluid power control
Abstract: This paper presents the development of the optimal constrained control for arrays of wave energy converters (WECs). Most current WEC optimal control methods require reactive power; that is, a power flow from the WEC to the ocean, at times, in order to increase the overall harvested energy over a period of time. The power take off (PTO) unit that has the capability of providing reactive power is usually expensive and complex. In this work, an optimal control is derived analytically in which the objective is to maximize the harvested energy from an array of WEC devices with periodic excitation, while constraining the power flow direction to eliminate the need for reactive power. Low fidelity numerical simulations are presented comparing the proposed control to the known Optimal Reactive Loading (ORL) control.

Keywords: Control applications, Energy systems, Hierarchical control
Abstract: This paper presents an integrated path planning and tracking control of marine hydrokinetic energy harvesting devices. To address the highly nonlinear and uncertain oceanic environment, the path planner is designed based on a reinforcement learning (RL) approach by fully exploring the historical ocean current profiles. The planner will search for a path to optimize a chosen cost criterion, such as maximizing the total harvested energy for a given time. Model predictive control (MPC) is then utilized to design the tracking control for the optimal path command from the planner subject to problem constraints. The planner and the tracking control are accommodated in an integrated framework to optimize these two parts in a real-time manner. The proposed approach is validated on a marine current turbine (MCT) that executes vertical waypoint path searching to maximize the net power due to spatiotemporal uncertainties in the ocean environment, as well as the path following via an MPC tracking controller to navigate the MCT to the optimal path. Results demonstrate that the path planning increases harvested power compared to the baseline (i.e., maintaining MCT at an equilibrium depth), and the tracking controller can successfully follow the reference path under different shear profiles.

Keywords: Sensor fusion, Estimation, Flight control
Abstract: Underwater energy-harvesting kites possess the capacity, under appropriate control algorithms, to deliver more than an order of magnitude additional power per unit mass than stationary devices. However, in order to perform path-following control, it is essential to maintain accurate estimates of the kite's position and velocity. Obtaining these measurements is complicated by: (i) operation in GPS-denied environments, and (ii) tether curvature, which renders straight-line approximated, line angle sensor-based measurements inaccurate. In this paper, we present a nonlinear closed-loop observer that fuses line angle sensor and IMU measurements using a dynamic tether model to maintain an estimate of an underwater kite's states. The observer design has been experimentally validated on a 1/10-scale kite model, using a customized tow testing system deployed in the NC State Aquatic Center.

Keywords: Energy systems, Optimal control
Abstract: This paper formulates and solves a periodic trajectory optimization problem for a tethered underwater kite. The goal is to maximize the average mechanical power harvested by the kite. The type of kite considered in this work extracts electricity from ocean currents by moving cross-current as its reel away from its base station, and consumes electricity to reel back. The problem of optimizing this kite’s trajectory is challenging due to the high dimensionality and nonlinearity of its dynamics. To tackle this challenge, the literature often separates the problem into two subproblems focusing on optimizing

the cross-current and the reel-in/reel-out components of the trajectory, respectively, which may be sub-optimal. In contrast, this work solves for the combined cross-current and reel-in/reelout trajectory by linearizing the dynamics of the kite around a zero-power reference equilibrium trajectory in spherical coordinates. This allows the trajectory optimization problem to be solved analytically for simple sinusoidal input perturbations from equilibrium. We use linear quadratic regulation to enable the nonlinear kite model to track the optimized trajectory. The result is a computationally efficient approach that achieves an attractive Loyd factor of 19.9%, while providing important insights into the nature of the optimal trajectory.

Keywords: Kalman filtering, Estimation, Filtering
Abstract: In the case of a vehicle descending to the deep sea floor, DVL (Doppler velocity log) usually loses the bottom lock for quite a long time. Therefore, various initial alignment methods available on the surface cannot guarantee its accuracy down to the sea floor and therefore cannot be directly applicable to this deep sea case. Besides, near the seafloor the vehicle's USBL (Ultra-short baseline) aiding position usually includes range-related time delay which can be up to several seconds. This paper proposes an simple but effective initial alignment scheme for deep sea underwater vehicles combining with position aiding time delay compensation method. Some practical issues in the construction of SDINS (Strapdown inertial navigation system), such as lever arm effects compensation, are also discussed. Experimental tests are also carried out in the water tank environment to verify the effectiveness of proposed schemes.

Keywords: Automotive control, Predictive control for nonlinear systems, Simulation
Abstract: With the advent of autonomous vehicles on public roads imminent in the near future, special emphasis needs to be placed on addressing scenarios pertaining to mixed-traffic settings, comprised of human-driven and autonomous vehicles. In this paper, we address the problem of autonomous vehicle overtaking in a bidirectional mixed-traffic setting. We design a mixed-integer model predictive controller that maximizes the ego vehicle’s velocity while prioritizing safety and accounting for driver comfort. The proposed approach: (i) operates in a limited sensing range while accounting for occlusion; (ii) is able to retract the overtake decision through a receding horizon approach; (iii) is robust to the variations in sensory input and driving behaviors of external agents due to behavior-dependent safety margins; and (iv) reduces to a mixed-integer optimization problem with linear constraints, yielding low computational complexity. We demonstrate the behavior of the proposed approach in a realistic traffic simulation environment.

Keywords: Automotive control, Stochastic systems
Abstract: Diesel engines equipped with ignition assisting actuators may improve combustion in aerial powertrains operating at low temperatures and with fuels of varying ignition behavior. Fuel injection timing can be aided by the additional energy from ignition assist (IA) devices such as glow plugs. This work presents a closed-loop control scheme that coordinates start of injection (SOI) and IA power to regulate both mean and cycle-to-cycle variability (CV) in diesel combustion phasing. Engine experiments at a chilled condition demonstrated the IA’s slow dynamics, high sensitivity to engine thermal state, and large nonlinearities in control authority that justify its limited use as a secondary actuator. A low-order model informed by experiments facilitated closed-loop system simulation. Control of mean combustion phasing was achieved using a PI controller for SOI. Reduction and control of CV within acceptable bounds was achieved with mean Kalman filtering of feedback, adaptation of mean phasing using windowed variance estimation from feedback, and conditional IA activation driven by saturation of the mean phasing or SOI values. Simulation results show the controller can track a desired mean setpoint of the start of combustion within 4-6 engine cycles without amplifying closed-loop CV, and can conditionally actuate IA while coordinating with SOI to manage unplanned disturbances in system CV behavior.

Keywords: Automotive systems, Automotive control
Abstract: Lean-burn gasoline engines have demonstrated significant fuel saving benefits due to the reduced pumping loss under part-load operation, while NOx emission control in lean environment has become one of the major barriers. To reduce NOx emissions from lean-burn gasoline engine, passive selective catalytic reduction (p-SCR) systems have been introduced and the efficacy of p-SCR systems has been validated in various studies. One of the major challenges in a lean-burn gasoline engine with p-SCR system is the periodic mode switching from highly efficient lean operation to inefficient rich operation for ammonia generation. This study focuses on design and optimization of a micro hybrid electric powertrain which integrates the lean-burn gasoline engine with p-SCR systems (with and without NOx storage component on a three-way catalyst) into hybrid electric powertrain to enable more efficient engine operation. A novel active torque control-based ammonia generation strategy is proposed to reduce the fuel penalty associated with ammonia generation. Simulation results demonstrated that, by implementing the novel control strategy on the micro-hybrid powertrain system, the proposed hybrid electric powertrain can reduce the fuel penalty associated with p-SCR operation by up to 71%, in comparison to the conventional powertrain with the same lean-burn engine and aftertreatment system.

Keywords: Automotive systems, Transportation networks, Optimization
Abstract: In this paper, we investigate the problem of routing, rebalancing, and charging for electric autonomous mobility-on-demand systems concerning traffic congestion. We analyze the problem at the macroscopical level and use a volume-delay function to capture traffic congestion. To address this problem, we first formulate an optimization problem for routing and rebalancing. Then, we present heuristic algorithms to find the loop of the traffic flow and examine the energy constraints within the resulting loop. We impose charging constraints on the re-routing problem so that the new solution satisfies the energy constraint. Finally, we verify the effectiveness of our method through simulation.

Keywords: Automotive control, Autonomous systems, Control applications
Abstract: This paper presents a new lateral control structure for path following control in an autonomous vehicle. The proposed method transforms a conventional feedback path following controller to a control architecture that relies exclusively on feedforward control of the curvature at a lookahead point of a blend path that connects the current vehicle kinematics with the desired target trajectory at some merge point. This approach provides a practical method for ensuring tracking performance and passenger comfort. Mathematical proofs and experiment results are provided to demonstrate that the proposed controller guarantees feedback stability of the lateral error dynamics, provides a systematic way of setting the corresponding feedback gains for accurate path tracking, and delivers desirable transient characteristics and smooth steering actuation effort.

Keywords: Automotive control, Autonomous systems, Control applications
Abstract: Abstract—Singular perturbation approach is an advanced method for nonlinear system stability analysis and control system design. In recent years, the notion of singular perturbation margin has been introduced which, when specialized to Single-Input, Single-Output (SISO) Linear Time-Invariant (LTI) systems, has been found to have a bijective relationship with the phase margin for all-pole systems. A simple yet effective method for assessing the singular perturbation margin in terms of phase margin of the (unperturbed) nominal system using time-scale or bandwidth-scale separation has also been reported for all-pole LTI systems, which can be used as a design tool for Multiple-Timescale-Nested-Loops (MTNL) systems. In this paper, we will extend the earlier results to LTI systems with stable zero-dynamics (non-minimum phase zeros). First, a bijective relationship between the Singular Perturbation Margin (SPM) emax the PM of the nominal system is established with an approximation error on the order of O(e2). To prove this result, relationships between the singular perturbation parameter e, PM of the perturbed system, PM of the nominal system with the net phase change due to phase lead by zeros, and phase lag by poles of the fast system is revealed. Finally, an easy yet practically significant method for estimating the nominal systems PM loss is obtained. Two academic examples are offered to demonstrate the correctness and applications of the theoretical results. The theoretical results have been applied to 3-DOF trajectory tracking control design of wheeled ground vehicles and validated on a full-scale Kia Soul EV, which is also presented.

Keywords: Mechatronics, Numerical algorithms, Variational methods
Abstract: Extraction of quantitative nanomechanical data in AFM Resonance modes, such as Amplitude and Frequency Modulation, is a challenging task. It requires either restoration of the force curve for analysis or using experimental data from the Resonant modes directly to estimate parameters of the unknown force. In many situations, force restoration is preferable, because direct parameter estimation methods work only if adequate parametric force model is available (which is rarely the case). At the same time, the force curve provides the most valuable source for material characterization, even when not parameterized. This paper describes a novel approach to force curve restoration from AFM Resonant mode experimental data. The approach is based on Krylov-Bogoliubov-Mitropolsky (KBM) asymptotic dynamics of AFM. Tikhonov regularization is used in case of noisy measurements, when the problem of force restoration becomes ill-posed.

Keywords: Mechatronics
Abstract: Increasingly stringent throughput requirements in the industry necessitate the need for lightweight design of high-precision motion systems to allow for high accelerations, while still achieving accurate positioning of the moving-body. The presence of position dependent dynamics in such motion systems severely limits achievable position tracking performance using conventional sequential loop closing (SLC) control design strategies. This paper presents a novel extension of the conventional SLC design framework towards linear-parameter-varying systems, which allows to circumvent limitations that are introduced by position dependent effects in high-precision motion systems. Advantages of the proposed control design approach are demonstrated in simulation using a high-fidelity model of a moving-magnet planar actuator system, which exhibits position dependency in both actuation and sensing.

Keywords: Hierarchical control, Energy systems, Optimization
Abstract: This paper presents a design framework for the optimization of hierarchical model predictive control (MPC) architectures and parameters. Hierarchical MPC is a powerful tool to balance objective trade-offs when controlling large dynamic systems composed of heterogeneous components. However, the multi-layer structure of the controller requires decomposition of the system and tuning of a large number of control parameters. The literature presents methods to identify optimal distributed and hierarchical control architectures, but these studies do not optimize MPC parameters concurrently with the architecture. This paper addresses this gap by embedding hierarchical MPC into a control co-design (CCD) optimization problem. Facilitated by a novel hierarchical control strategy for passing information between layers, the architecture and MPC parameters (e.g., timesteps, horizons) are simultaneously optimized. The design approach is tested on a model of an electric vehicle (EV) powertrain with cooling. The hierarchical control strategy is compared against a baseline control approach, and the behavior of different hierarchical architecture options are compared using performance- and energy-based metrics.

Keywords: Vision-based control, Autonomous systems, Switched systems
Abstract: Small-target motion detector (STMD) neurons found in the visuomotor system of some insects enable pursuit of a single agent but may play a role in facilitating group or swarm motions, and an understanding of the relationship of STMD to these motions is informed by simplified mathematical models of STMD neurons. In this work, STMD neurons are modeled as mathematical maxima (max) and argument of maxima (arg max) operators which operate on optic flow signal calculated by elementary motion detectors. Six contemporary models of insect feedback are considered as guidance laws operating on the output of the STMD model receiving optic flow in a multiagent environment. This paper studies the resulting closed loop system, which is a switched continuous system having multiple equilibria. Stability analysis of the switched system indicates bounded trajectories within a compact set under average dwell time constraints for the switching signal, and simulation results are used to compare the performance of the idealized control law against one imposing a dwell time constraint on stability. The results suggest that feedback from STMD neurons can help insects participate in swarm motion, and that selective attention property of STMD neurons to ‘target’ a single individual in a multi-agent heterospecific environment may be tied to the underlying stability and robustness properties of the system.

Keywords: Indirect adaptive control, Control applications, Predictive control for linear systems
Abstract: Indirect adaptive control of flexible structures is considered under harmonic and broadband disturbances. Limited prior modeling information is assumed, and system identification with an input-output model structure is performed online in the presence of the exogenous disturbance. By realizing the input-output model structure in block observable canonical form, the full state is available, which facilitates output-feedback control without the need for an observer. The control input is determined by model predictive control (MPC) using quadratic programming for receding horizon optimization. The resulting sampled-data controller is implemented at a fixed sample rate, where the frequencies of some of the modes may lie above the Nyquist rate, thus emulating spillover. The approach is applied to a truss structure with 16 lightly damped modes.

Keywords: Estimation, Neural networks, Biological systems
Abstract: It has been recognized that the initialization of fractional-order systems requires time-varying functions. This factor is very intricate and affects the convergence properties of the parameters and fractional differentiation order estimation. For this reason, we propose a novel technique to simplify the pre-initialization process of fractional differential system by designing an appropriate initialization function that ensures the fast and precise convergence to the exact states of the systems. Subsequently, we present a joint estimation approach of the parameters and the fractional differentiation order for initialized fractional-order systems. The performance of the proposed method is illustrated through different numerical examples. Furthermore, a potential application of the algorithm is presented, which consists of joint estimation of parameters and fractional differentiation order of a fractional-order arterial Windkessel model.

Keywords: Aerospace, Neural networks, Learning
Abstract: This paper designs a path planning algorithm for a group of unmanned aerial vehicles (UAVs) to track multiple spreading wildfire zones. Due to limited observable information, the fire evolution is hard to model. A regression neural network is online trained with real-time UAV observation data and applied for fire front prediction. To track fire fronts effectively, a UAV path planning algorithm is proposed by Q-learning. Various practical factors are taken into account by cost function designs such as moving target tracking, field of view of UAVs, spreading speed of fire zones, collision/obstacle avoidance, and maximum information collection. Simulation results validate the fire prediction accuracy and UAV tracking performance.

Keywords: Flight control, Neural networks, Machine learning
Abstract: In recent years artificial intelligence (AI) and machine learning techniques have found immense success in the fields of pattern recognition, classification, and data analytics. These techniques also have shown to provide viable means of controlling and modeling of uncertain, nonlinear dynamic systems. However, such techniques have not yet found widespread adoption in controls due to concerns in reliability, interpretability, and stability. In the past, much of the work in the field of dynamics has been based on well understood physical principles (i.e., Newtonian, Lagrangian, and Hamiltonian mechanics), while control has been model-based, as they adequately address the aforementioned concerns. The presented work attempts to retain the benefits of both AI and physics-based control, by using recently developed neural networks that incorporate Lagrangian mechanics into the learning scheme to create an inverse dynamic model of a quadcopter. The inverse dynamic model is utilized in developing a control scheme that is shown to learn the changes in system parameters effectively in an online fashion. The proposed control scheme is validated with the help of extensive simulation studies performed on a quadcopter, and the performance is compared to simple adaptive control for cases where mass and inertia change in flight for complex trajectories.

Keywords: Neural networks, Supervisory control, Nonholonomic systems
Abstract: The paper proposes a novel training approach for a neural network to perform switching among an array of computationally generated stochastic optimal feedback controllers. The training is based on the outputs of off-line computed lookup-table metric (LTM) values that store information about individual controller performances. Our study is based on a problem of bicycle kinematic model navigation through a sequence of gates and a more traditional approach to the training is based on kinematic variables (KVs) describing the bicycle-gate relative position. We compare the LTM and KV based training approaches to the navigation problem and find that the LTM training has a faster convergence with less variations than the KV based training. Our results include numerical simulations illustrating the work of the LTM trained neural network switching policy.

Keywords: Aerospace, Neural networks, Optimal control
Abstract: This letter considers assignment problems consisting of n pursuers attempting to intercept n targets. We consider stationary targets as well as targets maneuvering toward an asset. The assignment algorithm relies on an n x n cost matrix where entry (i,j) is the minimum time for pursuer i to intercept target j. Each entry of this matrix requires the solution of a nonlinear optimal control problem. This subproblem is computationally intensive and hence the computational cost of the assignment is dominated by the construction of the cost matrix. We propose to use neural networks for function approximation of the minimum time until intercept. The neural networks are trained offline, thus allowing for real-time online construction of cost matrices. Moreover, the function approximators have sufficient accuracy to obtain reasonable solutions to the assignment problem. In most cases, the approximators achieve assignments with optimal worst case intercept time. The proposed approach is demonstrated on several examples with increasing numbers of pursuers and targets.

Keywords: Fault diagnosis, Neural networks, Pattern recognition and classification
Abstract: The deep learning methods for bearing fault diagnosis have achieved some success under certain circumstances. However, the distribution discrepancy across the data collected under different working conditions limits the application of these methods. Transfer learning can break the limits by reducing the distribution discrepancy of data across different domains, which expands the application scope of the deep learning methods. The traditional transfer learning methods for bearing fault diagnosis only focus on matching the marginal distribution, which might lead to the incorrect alignment of the same class between two domains and thus affect the performance of models. To address this problem, we developed a new transfer learning-based deep learning method for fault diagnosis, called Deep Joint Transfer Network (DJTN). Compared with the traditional transfer learning methods, DJTN can accurately align the same class between two domains by jointly matching the marginal and the conditional distributions of the two domains through the proposed domain adaptation module. The Case Western Reserve University bearing dataset was used to verify the proposed model. In twenty-four transfer tasks, the DJTN outperformed the other widely used methods and could handle bearing fault diagnosis tasks under different working conditions with high accuracy. Additionally, our proposed method was more robust than the other methods in transfer tasks when there are more fault categories.

Keywords: Neural networks, Machine learning, Pattern recognition and classification
Abstract: Extracting spatial-temporal knowledge from data is useful in many applications. It is important that the obtained knowledge is human-interpretable and amenable to formal analysis. In this letter, we propose a method that trains neural networks to learn spatial-temporal properties in the form of weighted graph-based signal temporal logic (w-GSTL) formulas. For learning w-GSTL formulas, we introduce a flexible w-GSTL formula structure in which the user’s preference can be applied in the inferred w-GSTL formulas. In the proposed framework, each neuron of the neural networks corresponds to a subformula in a flexible w-GSTL formula structure. We initially train a neural network to learn the w-GSTL operators, and then train a second neural network to learn the parameters in a flexible w-GSTL formula structure. We use a COVID-19 dataset and a rain prediction dataset to evaluate the performance of the proposed framework and algorithms. We compare the performance of the proposed framework with three baseline classification methods including K-nearest neighbors, decision trees, support vector machine, and artificial neural networks. The classification accuracy obtained by the proposed framework is comparable with the baseline classification methods.

Keywords: Neural networks, Model/Controller reduction, Machine learning
Abstract: In this paper, we propose a concept of approximate bisimulation relation for feedforward neural networks. In the framework of approximate bisimulation relation, a novel neural network merging method is developed to compute the approximate bisimulation error between two neural networks based on reachability analysis of neural networks. The developed method is able to quantitatively measure the distance between the outputs of two neural networks with the same inputs. Then, we apply the approximate bisimulation relation results to perform neural networks model reduction and compute the compression precision, i.e., assured neural networks compression. At last, using the assured neural network compression, we accelerate the verification processes of ACAS Xu neural networks to illustrate the effectiveness and advantages of our proposed approximate bisimulation approach.

Keywords: Output regulation, Neural networks, Control applications
Abstract: This paper investigates an application of feedforward neural networks (FNN) to a tracking-control technique in order to render it model-free. The controller, proposed elsewhere by an author of this paper, is based on the Newton-Raphson fluid-flow dynamics for matching a system’s predicted output to a target-reference signal. Most of the extant results require that the predictor be based on a knowledge of the input-output system’s model. In order to overcome this limitation, we construct the predictor using an FNN slated to provide adequate approximations to future outputs. We test by simulation the efficacy of the resulting controller in a model-free environment, and compare it to results obtained from a model-based approach. The respective results are not far apart, suggesting that the FNN-based model-free controller may have a scope in future applications.

Keywords: Predictive control for nonlinear systems, Machine learning, Stochastic optimal control
Abstract: This paper proposes a learning-based adaptive-scenario-tree model predictive control (MPC) approach with probabilistic safety guarantees using Bayesian neural networks (BNNs) for nonlinear systems. First, a data-driven description of the model uncertainty (i.e., plant-model mismatch) is learned using a BNN. Then, the learned description is employed to generate adaptive scenarios online for scenario-based MPC (sMPC). To accurately represent the evolution of uncertainties, we use a moment-matching method to compute the probabilities of the generated time-varying scenarios. Moreover, probabilistic safety guarantees are provided by ensuring that the trajectories of the scenarios contain the real trajectory of the system and all the generated scenarios satisfy the constraints with a high probability. By realizing a less conservative estimation of the model uncertainty, the proposed approach can improve robust control performance with respect to sMPC with a fixed scenario tree. Closed-loop simulations on a cold atmospheric plasma system with prototypical applications in (bio)materials processing demonstrate that the proposed approach results in an improved control performance compared to sMPC with a fixed scenario tree.

Keywords: Robotics, Machine learning, Statistical learning
Abstract: We synthesize controllers for underactuated robotic systems using data-driven approaches. Inspired by techniques from classical passivity theory, the control law is parametrized by the gradient of an energy-like (Lyapunov) function, which is represented by a neural network. With the control task encoded as the objective of the optimization, we systematically identify the optimal neural net parameters using gradient-based techniques. The proposed method is validated on the cart-pole swing-up task, both in simulation and on a real system. Additionally, we address questions about controller's robustness against model uncertainties and measurement noise, using a Bayesian approach to infer a probability distribution over the parameters of the controller. The proposed robustness improvement technique is demonstrated on the simple pendulum system.

Keywords: Learning, Distributed control, Control of networks
Abstract: Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with directed coordination graphs, and propose a distributed RL algorithm where the local policy evaluations are based on local value functions. The local value function of each agent is obtained by local communication with its neighbors through a directed learning-induced communication graph, without using any consensus algorithm. A zeroth-order optimization (ZOO) approach based on parameter perturbation is employed to achieve gradient estimation. By comparing with existing ZOO-based RL algorithms, we show that our proposed distributed RL algorithm guarantees high scalability. A distributed resource allocation example is shown to illustrate the effectiveness of our algorithm.