Community Collections: A Framework for Openly Sharing Software Stacks Across Research Computing Centers Using Singularity and Lmod
Event Type
TimeTuesday, July 3011am - 11:30am
LocationCrystal C
DescriptionAutomatic software build infrastructure has advanced significantly with software build tools such as EasyBuild and Spack. Similarly, scientific container tools such as Singularity have greatly eased the burden of maintaining individual software components at research computing centers. Typically, most research computing sites use tools such as Environment Modules or Lmod to present entire software hierarchies to users. These tools are designed to better serve individual sites or specific toolchains. Our aim is to complement these preexisting build tools by inverting the concept of site-specific software collections to serve multiple institutions. This shared framework relies on both Lmod and Singularity to allow individual sites to seamlessly deploy predefined collections of software components which can be further tailored to their needs.

Johns Hopkins University, Iowa State University, Ohio Supercomputer Center, and Georgia Tech have deployed an extendable framework based on Singularity and Lmod to enable "Community Collections", a software framework that allows multiple institutions to customize the deployment of Singularity software images as a module. This deployment allows for multiple institutions to trivially extend their existing Lmod based software stacks and participate in a greater effort to reduce software deployment redundancy and to allow for re-duplication of software stacks in cloud environments. These efforts are centered around deployment that allows users and systems administrators to leverage shared Singularity containers. The tool facilitates the sharing of module files and their interface to corresponding containers consisting of single-node, shared memory, GPU/CPU-based software.