Madupite
Solve bigger Markov Decision Processes in less time - by Matilde Gargiani
When trying to solve stochastic optimal control problems, the lack of distributed and user friendly implementations of dynamic programming methods represent a strong limitation. Especially when dealing with large-scale systems, the memory storage requirements go beyond the memory capacity of a single-computer. Distributing the storage also requires the design of communication-efficient and distributed methods to handle the computation of the solution.
Our mission is to design fast, distributed and communication efficient variants of dynamic programming methods and code them in the context of a solver. Ultimately, we want to provide an easy-to-deploy and efficient solver for large-scale Markov Decision Processes (MDPs). For that we have developed Madupite, a C++ solver which relies on PETSc library for the distributed operations.
Our solver allows to efficiently solve an MDP whose data are stored across multiple-computers in a totally distributed fashion. Madupite comes with a very detailed documentation, toy-examples and with a Python interface. Any basic Python-user should be able through the documentation to install and deploy Madupite to store and solve her/his MDPs on a high-performance computing cluster.
Our benchmarks show that standard single-core MDP solver can store and solve instances with at most around 10’000 states. With Madupite we have solved instances with millions of states at the same computational costs of instances with few hundreds of states.
Guaranteed safety even in uncertain environments - by Jair Reyes Gutierrez