A Systematic Comparison of Single-Cell Perturbation Response Prediction Models

[szalata]

add https://www.biorxiv.org/content/10.1101/2024.12.23.630036v1.full

[Lanxiang-Li]

Hi, thanks for your interest in our research! Below is a summary of our study:

In this study, we established a standardized evaluation framework and conducted a systematic analysis focused on three key tasks: Unseen Perturbation Prediction, Combinatorial Perturbation Prediction, and Cell State Transfer. This framework integrates 24 evaluation metrics for multidimensional assessment and provides a comprehensive comparison of the performance of nine single-cell perturbation prediction models across 17 diverse datasets, offering valuable insights into their applicability in complex biological perturbation scenarios.

We found that performance is highly context-dependent, with no single method excelling across all scenarios. While some models perform well in capturing global expression profiles, they often struggle to predict the nuanced effects of perturbed genes, and vice versa. Additionally, although foundation models often outperform simpler methods, they tend to converge on population averages, making it difficult to capture heterogeneous cellular responses to perturbations.

Our study highlights both the potential and limitations of single-cell foundation models, identifies opportunities for future improvements, and provides a roadmap for advancing single-cell predictive biology.

Please let me know if you need any additional information related to this study.