No model, no problem
But Stay in Control - by Thierry Schwaller
Imagine driving an unfamiliar car for the first time. You do not yet know exactly how strongly it will react when you turn the steering wheel or press the pedals. Making sudden movements could cause you to lose control. Instead, you begin carefully, observe how the car responds, and gradually adjust your driving. A similar idea can be used to make machines learn safely.
Robots, drones, and industrial machines rely on a controller, which acts like a set of instructions telling them how to act. Traditionally, these instructions are designed using a detailed mathematical description of how the machine works. In practice, however, this description may be incomplete, inaccurate, or unavailable.
This creates an important challenge: how can a machine adapt to unfamiliar conditions and improve its behavior without losing control in the process?
Suppose a computer is tasked with learning how to control the unfamiliar car. Rather than requiring a perfect description of the system, we allow the controller to learn directly from experience. While the machine is operating, it observes how the car responds to its actions and uses these observations to adjust the rules it uses to control the car.
To ensure the rule adjustment is safe, the controller does not adopt a completely new strategy all at once. Instead, it improves gradually, much like a careful driver testing a slightly different steering technique before fully relying on it. When there is still a lot of uncertainty about how the car behaves, only small changes are allowed. As more experience is collected and the computer becomes more confident, larger improvements can be made.
To achieve this, we combine learning from data with established ideas from control engineering. The learning process searches for better ways to control the system, while control theory restricts these changes so that stability is preserved.
Returning to the car example, the goal is for the driver to become better at handling the unfamiliar car without crashing it while learning. The same principle can be applied far beyond cars, allowing a wide range of machines to adapt from experience while remaining safe during the learning process.
Text by Thierry Schwaller; illustration generated with ChatGPT 5.5
But Stay in Control - by Thierry Schwaller