BABEL Architecture and Design.

BABEL consists of four modular neural networks as subcomponents (Fig. 1C). Two encoder networks are trained to project either RNA or ATAC profiles into a single, shared 16-dimensional latent representation. Similarly, two decoder neural networks are trained to take points in this shared latent representation and infer their corresponding RNA or ATAC profiles. This shared latent space simultaneously summarizes the epigenetic and transcriptomic profiles of a cell and can be thought of as an abstract, integrated representation of cellular state. This latent space also serves as an intentionally low-dimensional “information bottleneck” to encourage the model to capture major cellular variation, rather than potentially spurious deviations (SI Appendix, Supplementary Note and Table S1). Our goal is then to learn encoder models that project single-cell profiles into this cellular state latent space, along with decoder models that can infer observed phenotypes from this same latent cellular representation.

While a standard autoencoder maps one data modality onto itself (e.g., scATAC-seq to scATAC-seq), BABEL maps across multiple modalities (e.g., it uses the same ATAC encoder to map ATAC to ATAC and ATAC to RNA). This enables the model to be more flexible and efficient. The networks responsible for RNA encoding and decoding consist of fully connected layers that project a continuous, transcriptome-wide expression vector to and from the latent cellular representation (Fig. 1C). The networks responsible for encoding and decoding a binarized genome-wide ATAC chromatin accessibility signal are also composed of fully connected layers, but we leverage the insight that most chromatin accessibility interactions occur at an intrachromosomal level (31) to prune the majority of interchromosomal connections (Fig. 1C). This approach substantially reduces the parameter space and helps the model avoid spurious correlations (Methods and SI Appendix, Supplementary Note). We binarize all ATAC measurements throughout this work—any peak with nonzero signal is set to “1” and is otherwise “0.” BABEL then predicts the probability that a peak is active (i.e., 1). This approach for learning on scATAC-seq data has been used in prior scATAC-seq models (14) and improves the quality of BABEL’s ATAC predictions.

BABEL is trained using a loss function that requires both encoders to be interoperable with either decoder. This is expressed by enumerating all four possible compositions of our two encoders and two decoders and simultaneously training all four of these “paths” through our model to produce correct outputs using four corresponding loss terms (Fig. 1D). Under this formulation, the ATAC encoder’s latent output must be consumable by both the ATAC decoder and RNA decoder while producing correct outputs for both modalities, and the same goes for RNA encoder. Similarly, the ATAC and RNA decoders must understand the latent representation generated by either ATAC or RNA encoder. This interoperability constraint leverages paired data to learn a single, unified latent space that embeds multimodal cell state without explicit latent space alignment, while improving BABEL’s ability to generalize to unseen cell types (SI Appendix, Supplementary Notes and Table S2). To evaluate the correctness of inferred RNA values (whether these inferences were generated from an ATAC or RNA input), we use a negative binomial (NB) loss, which has seen success in prior works imputing and denoising single-cell expression (11, 13). Notably, this loss encourages BABEL to estimate the denoised single-cell expression, rather than the noisy experimental counts (11). To evaluate the correctness of inferred ATAC values, we use a binary cross entropy (BCE) loss—a natural cost function for binary predictions used in prior deep learning models for scATAC-seq data (14). After training, BABEL can translate between a continuous vector representing an individual cell’s (denoised) transcriptome-wide gene expression spanning 34,861 genes, and a probability vector describing genome-wide ATAC accessibility profiles spanning 223,897 high-resolution peaks with mean and median widths of 796 and 573 base pairs.

BABEL Performs Cross-Domain Translation with High Accuracy.

We train BABEL using single-cell multiomic data jointly profiling ATAC chromatin accessibility and RNA gene expression generated on 10x Genomics’ multiomic platform, which starts from single nuclei (Methods). These data span cells collected from several human primary cell types and transformed cell lines: peripheral blood mononuclear cells (PBMCs), colon adenocarcinoma COLO-320DM (DM) cells, colorectal adenocarcinoma COLO-320HSR (HSR) cells, and lymphoblastoid GM12878 cells. We pool and cluster the PBMC, DM, and HSR cells together, reserving one cluster for validation (n = 2,881 cells) and one cluster for test (n = 1,979 cells), with the remaining cells (n = 28,408) being used for training (Fig. 2A and SI Appendix, Table S3). Although BABEL itself is agnostic of cluster identity, cluster-based data splits reduce similarity between data splits, challenging the model to generalize to new cell populations. Jointly profiled GM12878 cells are held out from any training purposes, serving as a measure of generalization even more challenging than the test cluster.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2.

Summary of train and test data, and BABEL’s performance. (A) We train BABEL using PBMC, DM, and HSR cells with jointly profiled ATAC and RNA measurements. Data from these three are pooled and clustered (“Training Data” box), with one cluster reserved as a test cluster. Splitting by cluster reduces similarity between train and test data. Moreover, we exclude a set of jointly profiled GM12878 cells from all training purposes, used only for model evaluation (“External cell type” box). (B) BABEL’s performance on the test cluster and external GM12878 evaluation set, with cross-validation performance ranges in brackets (SI Appendix, Tables S3 and S4). To contextualize BABEL’s ATAC to RNA performance, we evaluate comparable gene activity score estimation methods from ArchR and MAESTRO. (C) Density scatterplot showing the expression of each gene in each cell within the test cluster (x axis represents empirical expression; y axis represents BABEL’s inferred expression). BABEL exhibits a strong correlation of 0.62 on the test cluster. (D) Similarly shows BABEL’s inferences on the external GM12878 evaluation set. (E) ArchR gene activity scores for these GM12878 cells, which are less accurate than BABEL’s inferences.

BABEL achieves strong performance for cross-domain inference on all test data. Inferring RNA expression from ATAC accessibility, it achieves a Pearson correlation of 0.62 and a Spearman’s correlation of 0.35 (Fig. 2 B and C). Inferring ATAC from RNA on this same test cluster, BABEL achieves an area under the receiver operating characteristic (AUROC) of 0.92. We use different metrics for evaluating ATAC and RNA predictions due to the binary versus continuous nature of these modalities. BABEL’s performance is consistent across cluster-based cross-validation as well, exhibiting ATAC to RNA Pearson’s correlations ranging from 0.62 to 0.82 with a median of 0.80, and RNA to ATAC AUROCs ranging from 0.87 to 0.92 with a median of 0.87 (Fig. 2B and SI Appendix, Table S3).

We evaluated k-nearest-neighbor (KNN) models to establish performance baselines for these translation tasks, using the same cluster-based data splits. A KNN model takes each query point, identifies the (k = 10) most similar cells in the training set, and computes an average of those cells (see Methods for more details). KNN achieves a Pearson correlation of 0.27 when inferring RNA from ATAC on the aforementioned test set, and an AUROC of 0.84 when inferring ATAC from RNA. BABEL significantly outperforms both of these baselines.

To quantify BABEL’s ability to generalize to a different cell type, we applied BABEL trained on PBMC, DM, and HSR data to paired single-cell ATAC/gene expression data profiling GM12878, without any tuning or modification. This constitutes a challenging test since the GM12878 lymphoblastoid cell line exhibits substantial differences from the three cell types used for training. BABEL generalizes well to the GM12878 external evaluation set with an ATAC to RNA Pearson correlation of 0.59, Spearman correlation of 0.38, and an RNA to ATAC AUROC of 0.65 (Fig. 2 B and D). BABEL’s performance on GM12878 is also robust across cross-validation, where the training and validation clusters shift, and the five resultant models are evaluated on the same GM12878 evaluation set (Fig. 2B and SI Appendix, Table S4). Performance is similar for intradomain translations (i.e., inferring RNA output from RNA input and ATAC from ATAC input), which further validates BABEL’s subcomponent networks (SI Appendix, Tables S3 and S4).

Since one key objective of BABEL is to infer RNA gene expression from ATAC accessibility, existing tools that infer gene activity scores from ATAC data offer natural benchmarks. Like BABEL, these tools attempt to estimate the gene expression corresponding to an ATAC profile, drawing from observations and patterns present in prior data. Critically, while BABEL uses machine learning techniques to train a model, these existing tools use expert knowledge to hand-craft formulas based on prior observations that accessibility near a gene correlates with its expression. We specifically evaluated the approaches implemented by state-of-the-art scATAC-seq analysis suites ArchR (24) and MAESTRO (25). Evaluating the ArchR and MAESTRO gene activity scores on the GM12878 paired data yields Pearson’s correlations of 0.15 and 0.19, respectively, compared to BABEL’s 0.59 (Fig. 2 B and E). Since GM12878 was never used for training BABEL, nor was it the singular benchmark for developing gene activity scores, this represents a fair comparison where all methods are given similar input and are asked to generate similar outputs. For benchmarking the opposite RNA to ATAC translation, we are not aware of any prior methods that perform single-cell prediction. BIRD is a recently developed relevant method, although it is trained to make cluster-aggregated predictions of ATAC signals instead (32, 33), and consequently may be less flexible when applied to cell types it has not seen before. BABEL compares favorably to BIRD as well (SI Appendix, Fig. S1).

We additionally investigated how BABEL could perform on nonhuman data. We trained a separate version of BABEL on paired single-cell ATAC/gene expression data from the adult mouse cerebral cortex, generated via the SNARE-seq joint profiling protocol, which like the 10x multiomic data, profiles individual nuclei (3). On the held-out test cluster, BABEL achieves an ATAC to RNA Pearson correlation of 0.55, and an RNA to ATAC AUROC of 0.80 (SI Appendix, Fig. S2). As a second, independent mouse experiment, we trained BABEL on paired single-cell ATAC/gene expression data profiling mouse skin, generated via the SHARE-seq protocol (34). Unlike previous datasets, this SHARE-seq dataset is generated from single cells, rather than nuclei. On the held-out test cluster, BABEL exhibits an ATAC to RNA Pearson correlation of 0.53, and an RNA to ATAC AUROC of 0.80 (SI Appendix, Fig. S3). All of these values are similar to those we observed on the human dataset. Together, these results demonstrate that BABEL is applicable across different species and can be successfully trained using data generated by a variety of experimental protocols.

BABEL Cross-Modality Inference Captures Empirically Validated Cell States.

It is especially interesting to apply BABEL in settings where we do not have paired measurements, but where BABEL has been trained on reasonably similar cell types. This can enable exploratory analyses and hypothesis generation on new data via computationally imputed paired measurements. As a case study for this application, we reconstruct expression profiles for a set of healthy PBMC cells profiled using scATAC-seq. These unpaired data were generated using a different experimental protocol than was used to generate BABEL’s paired training data, and thus exhibit markedly different noise patterns that we do not explicitly adjust for (SI Appendix, Fig. S4). We use the pretrained BABEL to impute transcriptome-wide RNA expression signatures for each cell. We then apply standard preprocessing (size normalization and log-transformation) to BABEL’s inferred single-cell expression, and visualize the cells using the uniform manifold approximation and projection (UMAP) algorithm (35, 36) applied to their imputed expression signatures. We color each cell by its original ATAC-based cell type (SI Appendix, Fig. S5A) to visualize how well these are retained in BABEL’s imputed RNA profiles (Fig. 3A).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 3.

BABEL’s ATAC to RNA translation closely matches empirical results on unpaired PBMCs. (A) UMAP visualization of single-cell expression profiles imputed by BABEL from scATAC-seq, colored by ATAC-derived cell type identities. (B) UMAP visualization and cell types in empirical PBMC scRNA-seq for comparison. These two plots exhibit highly concordant global structure (i.e., both show three main cell groups) as well as very similar inter-cell type relationships. C highlights the expression of CD14 (a marker for CD14⁺ monocytes) within BABEL’s inferred gene expression (Left) and the empirical measurements (Right). In both cases, CD14 expression is highly correlated with CD14⁺ monocytes, as expected. (D) Similarly illustrates expression of LYZ, which is also a marker for CD14⁺ monocytes. E and F highlight expression of GNLY and MS4A1, which are markers for NK and B cells, respectively.

For comparison, we include a parallel clustering and analysis of empirical single-cell expression in healthy PBMCs (Fig. 3B). We find many similarities between our imputed visualization and this empirical visualization. There are three major clusters in both the imputed and empirical data. One major cluster corresponds to B cells (Fig. 3 A and B, Top), one consists of CD14⁺/CD16⁺ monocytes and dendritic cells (Left), and one primarily contains CD4, CD8, and natural killer (NK) cells (Bottom Right). These results suggest that BABEL’s imputed single-cell gene expression retains much of the empirical global gene expression patterns and relationships. Further analysis also reveals that this overall concordance is consistent across variants of BABEL trained on different cross-validation folds (SI Appendix, Table S5) and is also recapitulated in BABEL’s latent representation (SI Appendix, Fig. S5). This consistency suggests that BABEL can recognize complex relationships between cells in its input—despite variable noise patterns—and leverages these biological relationships when generating latent representations and predicting transcriptomic profiles. This property helps BABEL generalize to more contexts (SI Appendix, Supplementary Note) and suggests that BABEL’s latent representation could be a potentially interesting basis for downstream analyses like clustering or lineage tracing.

In addition to examining cell cluster concordance, we evaluate how well BABEL imputes expression of well-known marker genes for specific cell types. CD14 is a canonical marker for CD14⁺ monocytes. Coloring each cell in BABEL’s expression UMAP by its imputed CD14 expression, we find that CD14 expression coincides very well with CD14⁺ monocyte cells (as identified via standard scATAC-seq analysis methods; Fig. 3C, Left). As expected, experimentally measured CD14 expression overlaps nearly perfectly with CD14⁺ cells as well (Fig. 3 C, Right). Performing a similar comparison for LYZ, another well-known marker for CD14⁺ cells, we see that BABEL likewise reproduces experimentally validated expression distributions (Fig. 3D). Such concordance extends to other cell types as well. Examining the expression of GNLY, a marker for NK cells (Fig. 3E), and MS4A1, which corresponds to B cells (Fig. 3F), we consistently see that BABEL’s imputed marker gene expression matches the correct cell types just as the empirically measured expression does. This shows that BABEL is not just predicting average expression of genes across all cells regardless of ATAC profile but is performing highly specific expression inference for individual cells. These results suggest that BABEL can facilitate downstream cell type analysis by computationally generating missing data modalities. Although BABEL’s strong performance here benefits from having seen similar PBMC cells in its training set (SI Appendix, Supplementary Note and Fig. S6), generalizing to this external PBMC dataset is nonetheless challenging due to aforementioned differences in experimental protocols used to generate the data.

We also performed the opposite study, taking a set of PBMC cells profiled using unpaired scRNA-seq and inferring their genome-wide chromatin accessibility signatures. Having previously explored the performance of BABEL’s RNA to ATAC predictions, we now assess the usefulness of these predictions by investigating whether they recover meaningful biological signals. We focus on a cluster of NK cells (overexpressing GNLY), as these cells appear to have the most unambiguous transcriptomic signature. Among BABEL’s accessibility predictions, we identified the 10 most predictive peaks for these NK cells and determined the nearest annotated transcribed region for each resultant peak (see Methods for more details). This highlights known NK pathways and proposes potential unstudied regulatory interactions. BABEL’s predictions implicate a region proximal to CD96, a well-characterized regulator of NK cell function (37, 38). BABEL’s predictions also highlight LINC01550, a long noncoding RNA identified as an immune-related oncogenic biomarker (39), and FOXP1, a transcriptional regulator in T cell development (40). This case study shows that BABEL can make reasonable expression-to-accessibility predictions that may be useful for hypothesis generation.

BABEL Can Generate New Insights for Patient Samples.

We next apply BABEL to scATAC-seq data acquired from BCC tumors to investigate its application on challenging patient samples without any additional fine-tuning or training. These samples, acquired from seven BCC patients, represent malignant, stromal, and immune cells present within the tumor microenvironment both before and after anti-programmed cell death protein 1 (PD-1) immunotherapy, and were profiled using scATAC-seq to generate chromatin accessibility profiles for 37,818 cells (10). These cells were originally analyzed using standard scATAC-seq methods by calculating gene activity scores with Cicero (41) and using these scores to label cell types and infer lineages.

A superset of these BCC samples has also been studied in a separate, unpaired experiment using scRNA-seq profiling (42), which enables evaluation of aggregate concordance metrics. Specifically, we leverage the intuition that by averaging across all cells in a single-cell experiment, the resulting per-gene pseudobulk expression values should be comparable across related experiments. We compared pseudobulk RNA expression profiles derived from BABEL’s inferred single-cell expression against empirical pseudobulk from the corresponding patients’ tissue-matched scRNA-seq, and found good agreement (Pearson’s r = 0.70; Fig. 4A). This concordance also holds when we examine specific cell types, rather than the global population (SI Appendix, Fig. S7). Furthermore, the mismatch between BABEL prediction and empirical scRNA-seq tended to manifest as increased predicted expression for genes with low observed expression, which could have been experimentally underreported due to drop-outs. For context, we also calculated the pseudobulk correlation of Cicero’s gene activity scores against the empirical pseudobulk and observed a weaker correlation of 0.27 (Fig. 4B). Overall, BABEL’s predicted RNA expression profiles based on BCC scATAC-seq are robust. More generally, we propose that aggregate concordance metrics like those described here can serve as simple checks when applying BABEL to a new sample. Comparing BABEL’s aggregated predictions to known bulk signatures and observing a high correlation would indicate that BABEL is likely making reasonable predictions, whereas low correlation might suggest BABEL may not generalize well on a particular sample.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 4.

BABEL makes accurate predictions on clinical basal cell carcinoma (BCC) scATAC-seq samples and generates new interpretations. (A) Correlation of transcriptome-wide pseudobulk expression: x axis represents expression within a tissue-matched scRNA-seq study averaged across cells; y axis represents BABEL’s predicted single-cell expression from scATAC-seq data, also in pseudobulk. The strong correlation, especially compared to a similar plot for gene activity scores (B), suggests that BABEL generalizes well to patient cancer samples. (C) ATAC-based UMAP visualization of these BCC cells. Produced using data from ref. 10. We use this projection as a scaffold to visualize BABEL’s imputed single-cell RNA expression. D highlights BABEL’s imputed expression of CD3E (Left), a T cell marker, compared to gene activity scores (Right). BABEL recapitulates expected expression of CD3E here, as well as for the myeloid marker CD86 (E). For NK marker KLRC1 (F) and CD8⁺ activated T cell marker IFNG (G), BABEL predicts more localized, specific expression than gene activity scores. However, for genes that have little to no presence in the training data such as KRT14, BABEL shows weaker performance (H). BABEL also lets us expand on prior conclusions, predicting more distinct overexpression of immunosuppressive genes like CTLA4 (I) in the Tfh cluster (SI Appendix, Fig. S8), strengthening these cells’ reported similarity to exhausted CD8⁺ cells.

We overlay BABEL’s predicted expression on the ATAC-based UMAP projection for these cells (Fig. 4C), which consist of several populations of CD4⁺ and CD8⁺ T cells, B cells, NK cells, malignant tumor cells, among others. We explored whether BABEL could accurately predict cell type specific expression for marker genes corresponding to each cell type. CD3E is a marker gene for T cells, and we see that BABEL recovers this relationship (Fig. 4 D, Left), much as gene activity scores do (Fig. 4 D, Right). CD86, a marker for myeloid cells, exhibits high specificity and concordance with gene activity scores as well (Fig. 4E). BABEL’s predicted single-cell expression of KLRC1 (killer cell lectin-like receptor C1), a marker for NK cells, is also highly concordant with both annotated cell types and gene activity scores (Fig. 4F). In fact, the distribution of BABEL’s imputed KLRC1 expression appears to be even more specific to the NK cell clusters than gene activity scores—Cicero also predicts KLRC1 activity in epigenetically similar cytotoxic CD8⁺ T cell clusters (43, 44). We observe similar specificity improvements for IFNG, which encodes the interferon gamma protein and is specifically expressed in CD8⁺ activated T cells in scRNA-seq experiments measuring these BCC samples (42). BABEL’s predicted IFNG expression is much more localized than the corresponding gene activity scores (Fig. 4G) and better approximates empirical scRNA-seq. These two examples highlight concrete cases where BABEL offers clear improvements over traditional gene activity scores. However, BABEL is less accurate if the cell type corresponding to a marker gene is not present in its training data. For example, KRT14 is a prominent marker for malignant BCC cells, but since there are no basal epithelial cells in BABEL’s training data, BABEL predicts a weak signal for KRT14 (Fig. 4H).

BABEL can also help develop more nuanced understandings for certain cell types in this study. The original scATAC-seq BCC study identified a set of T follicular helper (Tfh) cells that exhibited striking epigenomic similarity to exhausted CD8⁺ T cells (TEx), suggesting that Tfh and TEx differentiation may be driven by a shared regulatory program (10). Using BABEL’s predictions, we find that this Tfh cell cluster may also overexpress common immunosuppressive genes such as CTLA4 (Fig. 4I), PAK2, and FAS (45⇓–47). While CTLA4 is also overexpressed in Tfh based on gene activity scores, PAK2 and FAS gene activity scores show weak to no overexpression, despite all three of these genes being overexpressed in Tfh cells in the tissue-matched scRNA-seq study (SI Appendix, Fig. S8). Here, BABEL’s ATAC-based inferences help recover an additional parallel between TEx and Tfh cells that is confirmed by scRNA-seq, whereby these cell types not only share epigenetic similarity, but are also similar in their overexpression of immunosuppressive factors.

BABEL Can Be Extended with Additional Data Modalities.

We finally demonstrate that BABEL can be easily extended to predict additional data modalities, such as protein epitope profiles. We trained an auxiliary protein epitope decoder network using 33,287 jointly profiled RNA and protein epitope measurements of human bone marrow cells, profiled using CITE-seq (23). This protein decoder network leverages BABEL’s pretrained latent representation to infer protein epitope profiles (see Methods for more details). The resulting protein decoder network can accurately impute epitope profiles from expression profiles on the test cluster and on novel PBMC cells (SI Appendix, Fig. S9 A and B). As a proof-of-concept, we show that BABEL’s interoperable structure also enables us to generate epitope profiles of a cell from its scATAC-seq profile (SI Appendix, Fig. S9 C–E). BABEL’s ability to make these predictions without explicitly training the ATAC-protein modality pair further illustrates the potential power and versatility of its computational cross-modality translation approach. As more single-cell modalities become available, BABEL can flexibly incorporate these data and link increasingly diverse layers of molecular information, even across modalities yet to be jointly profiled.