Petaflop biofluidics simulations on a two million-core system

We present a computational framework for multi-scale simulations of real-life biofluidic problems. The framework allows to simulate suspensions composed by hundreds of millions of bodies interacting with each other and with a surrounding fluid in complex geometries. We apply the methodology to the simulation of blood flow through the human coronary arteries with a spatial resolution comparable with the size of red blood cells, and physiological levels of hematocrit (the red blood cell volume fraction). The simulation exhibits excellent scalability on a cluster of 4000 M2050 Nvidia GPUs and achieves close to 1 Petaflop aggregate performance, which demonstrates the capability to predicting the evolution of biofluidic phenomena of clinical significance. The combination of novel mathematical models, computational algorithms, hardware technology, code tuning and optimization required to achieve these results are presented. Copyright 2011 ACM.