Staying Afloat
Safe recommendations for adaptive control algorithms - by Marcell Bartos
Imagine that you are rowing down the river. The river is flowing fast and takes quick turns, it is wide at parts but dangerously narrow at others, and its depth is unknown. The weather is not on your side either: strong wind gushes shove and tug at you from different directions and cause waves, and the heavy rain limits your visibility. And as if that weren't enough, you are not particularly good at rowing.
It sounds like a nightmare, and it will become your reality soon. You will have no other choice but to learn by doing, and continuously adapt to the changing and hostile environment: you want to stay afloat after all.
There are multiple strategies that you can follow to learn the proper technique, and you likely must keep monitoring and adapting your technique to the current conditions. Similarly, there is a variety of boats, oars, paddles and other equipment that you can choose from.
You can observe the river and the weather first and decide on the learning strategy and rowing technique that you plan to follow, and then select your boat and other equipment that you will take with yourself. But be careful: not every learning strategy, rowing technique, and equipment will be compatible with each other and with the environment. For example, you cannot use kayak paddles if the boat is too wide, and it is better to avoid backward-facing rowing in narrow and violent rivers. Moreover, your initial observation will only give you a rough estimate of what you can expect, and you only get to know what you are dealing with once you are in the river.
But don't worry! Before you get on board, we are here to help you and give you our expert advice. We have prepared a list of compatible strategies, techniques, and equipment, for different environmental conditions. You are free to choose from the list, and we can guarantee that you will stay afloat. It is now up to you pick your preferred combination, jump in the river, keep our recommendations in mind, and safely reach the lake at the end of the river. By then, you will be an expert in rowing in difficult conditions.
The same story holds for control algorithms that utilize online learning to continuously adapt to changing conditions. The unknown river and the violent weather correspond to an unknown and changing dynamical system. The equipment represents the controller, and adapting the rowing technique and swapping the equipment symbolizes online adaptation of the control policy. Finally, the learning strategy is the analogue to online model learning.
Our research focuses on finding lists of compatible building blocks for online-learning-based controllers for different classes of dynamical systems.
Text by Marcell Bartos; picture by Connor Scott McManus (pexels.com) 
Safe Recommendations for Adaptive Control Algorithms - by Marcell Bartos