A Multi-Agent RL Algorithm for Single-Day Operating Room Scheduling

Surgery scheduling is a challenging combinatorial optimization problem that allows determining the operations start time of every surgery to be performed, as well as the resources to be assigned to each surgery over a predetermined period. It has been demonstrated that Reinforcement Learning approaches are highly promising to handle complex scheduling scenarios. In this paper, we formulated the problem of daily scheduling of elective patient surgeries as a Cooperative Markov Game. Then, we developed a Q-learning algorithm with multiple agents, each controlling one surgery. In so doing, we exploit the advantages coming from learning in cooperation with other agents to reach the common goal to obtain an optimal daily schedule of surgeries. Preliminary results highlight the improvements obtained when combining multi agent cooperation with reinforcement learning in surgery scheduling against a traditional approach.