Replica exchange nested sampling

In my ongoing effort to accelerate nested sampling (NS) simulations, I developed a GPU-based NS implementation for JAX-based machine learning potentials called JAXNEST. While internal parallelization of NS is inherently limited, modern GPUs can handle significantly higher loads. To make full use of this, I implemented a scheme for running multiple independent NS simulations in parallel on a single GPU. This approach enabled the computation of entire phase diagrams within a single run, by executing multiple NS simulations simultaneously. At some point, I began to wonder whether these independent NS simulations could be made to "talk" to each other. This idea eventually led to the development of replica exchange nested sampling (RENS).

RENS provides a framework for the exchange of information between multiple NS runs. It not only resolves several long-standing issues in atomistic NS simulations but also substantially reduces computational cost. While the implementation of the RENS algorithm itself is relatively straightforward, the bulk of the work involved demonstrating its benefits through carefully chosen example systems. We spent countless hours preparing detailed comparisons between independent NS and RENS simulations. Among other examples, we also used two simple interatomic potentials for demonstration purposes. Since JAXNEST was not ideally suited for these models, I implemented a fully parallelized CPU based NS code in C++ specifically for this purpose.

Although this project took much longer than expected, I’m extremely happy with the outcome. Not only did the thorough analysis of test cases strengthen my confidence in the RENS method, but applying my (at the time) beginner-level C++ skills to a real HPC project also helped me grow significantly as a coder. You can find a set of slides discussing this work here, which provide a basic introduction and highlight key results from the paper. Furthermore, there is now also a more detailed article on RENS in the Nested sampling theory section.

Unglert, N.; Pártay, L. B.; Madsen, G. K. H. Replica Exchange Nested Sampling. J. Chem. Theory Comput. 2025. https://doi.org/10.1021/acs.jctc.5c00588.