Systems of high variability can benefit from probabilistic dual control. To illustrate this, 50 different parameter sets (horizontal) are simulated for 5 different controllers (vertical). The colour indicates overall cost, from dark blue (low) to dark red (high).
Uncertainties have long been recognized as a key difficulty for control, deteriorating performance or even putting system safety at risk. This issue has been classically addressed by robust controller design, making use of a deterministic bound on the uncertainty and designing the controller for all possible uncertainty realizations. Tight uncertainty bounds are, however, difficult to obtain, limiting performance in practice, and robust techniques generally suffer from significant computational complexity.
The scope of this research is to develop new methods and tools for high performance and computationally efficient control of uncertain complex systems. We take a probabilistic approach, considering that the uncertainty affecting the system has a probabilistic nature. In contrast to common stochastic control methods, a key feature of the developed techniques is that they enable an online identification of the uncertainty distribution to maximize system performance, while providing (probabilistic) guarantees on the closed-loop system characteristics. The methods thereby leverage an interplay of probability, robustness, and adaption.
While uncertainties can enter controllers in various forms, we focus on model-based control techniques with model uncertainties due to high system complexity or due to the environment. The challenge is that learning happens in closed loop, i.e.,\ 1) the resulting model is directly critical for both performance and safety at every time step and, 2) control and system identification have to be performed simultaneously.
Learning-based model predictive control Model predictive control (MPC) makes use of predictions of the system and the environment in order to optimally choose a sequence of control inputs that provides safety in the form of stability and constraint satisfaction and minimizes a performance objective.
We investigate the use of learning-based identification methods for integration in an MPC controller, offering an ideal framework to incorporate probabilistic predictions based on online data. Due to their flexibility, Gaussian processes (GP) are used to model complex system behaviors and, most importantly, characterize the corresponding model uncertainty. The work [ ] proposed a modeling and control approach using Gaussian processes to predict time-periodic errors that are then compensated by a predictive controller, which was demonstrated for high precision control of a telescope mount.
Dual control A key challenge of simultaneous control and system identification is that the control action influences not only the performance, but also the uncertainty in the dynamics. This is also called the exploration-exploitation trade-off. Dual control offers one way to address this trade-off by considering the control of the dynamical system augmented by the unknown parameters. While the exact dual controller is intractable, available approximations generally cannot maintain all core features: caution, exploration and the future value of information.
We have derived a tractable approximation of the dual control formulation that aims at maintaining these features, in particular the value of information, and have extended the technique to general nonlinear systems. The developed dual control framework has been applied in simulation to the problem of building control, providing simultaneous identification and control guided by electricity prices, weather, and occupancy.