scverse Conference 2024

Containerization has fueled a movement around virtualization that has impacted nearly the entire landscape of software development ecosystems. The ability to enable simple isolation of both environment and tool within a portable, lightweight package has completely reshaped many aspects of the software management paradigm. This overhaul is not limited to production workflows as low-friction virtualization has created established use cases for containerized management across development, execution, testing, interactive compute, distribution, and deployment. Even with these advancements, utilization of these conventions remains limited in some areas of research computing and bioinformatics, especially single cell biology.

Here, we present SCleeStacks - an effort to expand the accessibility of containerized workflows in the scverse. The goals of this project are to: 1) standardize and simplify the containerization and dependency landscape across the scverse ecosystem; 2) lower barriers to scverse environment sharing across teams and communities; 3) promote improvements to research software methods, publishing, and reproducibility; 4) foster upstream compatibility for development and execution of interdependent tool combinations; and 5) expand opportunities for use of scverse tools in higher-order execution frameworks such as Nextflow and Snakemake.

The central outcome in achieving these goals is the establishment of a repository of maintained, tool-level images across scverse ecosystem packages. As many uses of scverse tools focus on multi-modal data that require an intersection of package functions, a class of these images attempt to offer multi-layered tool stacks built around common workflows or data type modalities. Image structure follows common best practices in provisioning for security, performance, and compatibility through the Docker framework. Singularity is embraced for use in non-root or restricted HPC environments. A critical element of this implementation is a common design language to foster both contributions to the creation of images as well as their use. We expect a common application to be interactive computing during analysis and pipeline development. To address this, a specific emphasis is placed on documentation around workflows like IDE dev containers in tools such as VS Code, JetBrains, or Jupyter Notebooks. Where possible, authors hope to sustain an open dialogue with upstream tool maintainers and the wider community, striving for collaboration that can elevate tool harmony and integration. Much like any community development effort, we expect the evolution of this resource to be guided by the needs of users and the field. With that, we hope the efforts behind this project offer something of value and can serve the growing needs of the scverse community.