09-11, 17:00–18:00 (Europe/Berlin), Workshop room II - MW 0234
The "Benchmarking Open Problems in Single Cell Analysis" workshop aims to address critical challenges in the field by fostering community engagement in the development of robust benchmarks. This 90-minute session will introduce participants to the core mission of the Open Problems in Single-Cell Analysis, followed by an interactive session where we will build a new benchmark from scratch. As part of this tutorial, participants not only learn about the technical aspects of setting up a benchmark within the Open Problems framework, but will also learn about essential best practices in benchmarking computational methods.
The workshop will commence with a 20-minute introduction, outlining the goals and importance of the Open Problems initiative in single-cell analysis. This segment will focus on the role of benchmarking in improving reproducibility and reliability in computational methods.
Following the introduction, the 70-minute interactive session will guide participants through the process of designing and developing a benchmark.
- Problem identification
- Experimental design
- Dataset selection and processing
- Metric selection and implementation
- Implement methods
- Implement control methods
- Essential best practices for benchmarking computational methods
By the end of the workshop, participants will have practical experience in creating benchmarks and an understanding of best practices that can be applied to their research projects.
Notes:
- Target audience: Researchers, bioinformaticians, and data scientists involved in single-cell analysis.
- Prerequisites: Basic understanding of single-cell biology and computational analysis.
- Benchmarking topic: For the sake of being able to cover a lot of material and implement a benchmark from scratch, presenters will identify a benchmarking topic prior to this workshop.
- Materials provided: All slides, source code and other materials will be made available via GitHub.
- Outcome: Attendees will acquire practical skills in benchmark development, contributing to the Open Problems initiative and enhancing their research methodologies in single-cell analysis.
No previous knowledge expected
Malte leads the integrative genomics lab at Helmholtz Munich. His lab uses machine learning and single-cell technologies to build reference atlases of human tissues and organs with the goal of using these resources for clinical applications, with a specific focus on lung disease. His lab is strongly involved in the Human Cell Atlas, where Malte leads the integration team. This team is spread across 3 continents and focuses on building integrated single-cell reference atlases for HCA Bionetwork organs and tissues.
Malte is also a co-founder of the Open Problems in Single-cell Analysis consortium, which hosts a living benchmarking platform and runs competitions in single-cell data science.
Robrecht, CTO at Data Intuitive (data-intuitive.com), is passionate about making single-cell analysis more accessible and reliable. As the lead developer of Viash, he's all about creating workflows that are easy to reproduce and reuse. He's also the lead developer of OpenProblems, a platform dedicated to establishing the best practices for single-cell analysis by rigorously testing different computational methods. These efforts drive the creation of new living best practices (sc-best-practices.org) and the development of modular single-cell workflows (openpipelines.bio).