How Stable Are Single-Cell Embeddings? A Comprehensive Analysis of Dimensionality Reduction Under Input Shuffling

¹ Department of Computer Science, National Chengchi University, Taipei, Taiwan
² Institute of Neuroscience, National Chengchi University, Taipei, Taiwan

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Single-cell RNA sequencing (scRNA-seq) offers precise quantification of the transcriptome at an individual cell level, surpassing traditional bulk RNA-seq. Despite its advancements, the high-dimensional nature of scRNA-seq data complicates the extraction and visualization of underlying biological information. Various dimensionality reduction algorithms have been developed to aid biologists in uncovering cellular relationships, especially through clustering. However, the stability of single-cell dimensionality reduction has been largely overlooked, particularly the variation of neighbor relations or cluster quality in the reduced space.

In this study, we generate alternative datasets by randomly shuffling rows or columns of the single-cell expression matrix. These alternative datasets are processed similarly to the original data. Stability is measured by comparing results from the original and alternative data using multiple metrics, including knn-preservation for neighbor relations, the Calinski-Harabasz, Davies-Bouldin, and Xie-Beni indexes for cluster internal evaluation, and the Jaccard Index for clustering consistency. Additionally, the RF-hierarchical metric evaluated the preservation of global metainformation.

We employed Monocle 3, an R toolkit, to assess the stability of scRNA-seq visualizations using two popular methods: t-SNE and UMAP. Our evaluation involved six datasets, including two from the C. elegans Monocle 3 tutorial, two from the Single Cell Portal (PBMC ID 345 and islet ID 1526), and two comprising Mouse Retinal and Brain samples. Internal cluster scores varied after data shuffling, with the Jaccard Index of C. elegans-con dropping below 0.6, indicating instability, and the RF distance of SCP1526 reaching the maximum value. Our findings suggest that dimensionality reduction techniques vary in stability with shuffled inputs, indicating that claims of their robustness may be premature without considering input variations.