Common Issues in Genome Wide Analysis of Translational Control.

Two types of data are produced from each sample when studying translational activity: actively translated mRNAs (“translational activity data”) and cytosolic mRNAs (“cytosolic mRNA data”) obtained in parallel. Typically, log (translational activity data/cytosolic mRNA data) ratios are calculated (i.e., logged cytosolic mRNA data is subtracted from its associated logged translational activity data) with the idea of obtaining a cytosolic mRNA data-corrected estimate of translation activity and compared between classes (Fig. 1A). Because log ratios are differences between logged values, class comparison effects are estimated from difference scores. Log difference scores can be correlated with cytosolic mRNA data, however, leading to incorrect biological conclusions. Such is the case when translational activity data and cytosolic mRNA data are uncorrelated for technical or biological reasons (Fig. 1 B and C). For example, false positives can arise from log ratios when mRNAs fail to reach the predetermined threshold for the number of ribosomes necessary to join the pool of actively translated mRNAs or when short poorly translated mRNAs, but not their paired cytosolic mRNAs, fail to achieve counts for protected mRNA pieces above the noise level. Log ratios for such mRNAs tend to produce false positives as they appear to be under translational control (Fig. 1B). False negatives can be produced by log ratios when translational activity is regulated independently of the cytosolic mRNA level (Fig. 1C). The phenomenon of a difference score (Y - Z) correlating with each of its terms (Y and Z) was first described by Pearson in 1897 (7) who labeled it “spurious correlation” because of the frequent practice of interpreting such correlations as substantive rather than artifactual. We will also refer to these correlations as spurious, although our focus will be on the inadequacy of difference scores to control for cytosolic mRNA data and as a consequence fail to correctly estimate class effects (e.g., genotype, treatment, disease). The following equation illustrates the mathematical necessity underlying the problem (8): [1]where Y is a vector of translational activity data for a specific mRNA, Z is a vector of paired cytosolic mRNA data for the same mRNA, r is the Pearson correlation coefficient, and s is the sample standard deviation. When translational and cytosolic levels are uncorrelated, Eq. 1 simplifies to: [2]Eq. 2 makes the origin of these false negatives and false positives clear; under this circumstance the correlation between the log ratios and their corresponding log cytosolic mRNA data is a function of the standard deviations of translational activity data and cytosolic mRNA data replicates. When the standard deviations are also equal, Eq. 2 yields a correlation of -0.71 . That is, half (-0.71²) of the variance associated with the log ratio would be due to cytosolic mRNA levels. A similar difficulty with log ratios arises in the more typical situation in which the correlation between translational activity data and cytosolic mRNA is nonzero. When standard deviations are also equal, Eq. 1 then simplifies to: [3]When the correlation between translational activity and paired cytosolic mRNA is 0.60, Eq. 3 yields a correlation of -0.45. Thus, under various realistic scenarios, the correlation between log ratios and cytosolic mRNA is nontrivial and gives rise to biological false positives and negatives.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1.

(A–D) Common problems when analyzing translational activity data. Data were simulated to generate two sample classes, each with five measurements of paired translational activity data (indicated as translation on each y-axis) and cytosolic mRNA data (indicated as transcription on each x-axis). The examples were generated to illustrate different scenarios of translational control analysis. For each example, a two-tailed t-test was performed comparing the sample classes using the translational activity data only or the log-ratio data (p-values as indicated). anota was also performed on each example (p-value as indicated). Lines represent the regression lines estimated by anota for the two sample categories. (E) Common spurious correlations in published datasets of translational control. Shown are boxplots of the spurious correlations that emerge between the log (translational activity data/cytosolic mRNA data) ratios and log cytosolic mRNA data. (F) Small overlap between the log-ratio approach and anota. The top 5% genes ranked by significance from the log-ratio approach and anota were collected. The overlap between the log-ratio approach and anota is visualized using Venn diagrams for the three example studies.

To examine the practical implications of problems with log ratios, we calculated the per gene correlations between logged translational activity data/cytosolic mRNA data ratios and log cytosolic mRNA data per gene across all samples in a set of published studies (9–28). The median of the per dataset medians for the spurious correlations was -0.61 with values covering almost the entire range between +1 and -1 (Fig. 1E). Although these results do not imply low data quality, they demonstrate that log ratios are inappropriate for controlling the confounding effects of cytosolic mRNA levels. It is noteworthy that using the union from the translational activity data only analysis and the log ratio approach resolves the false negative example (Fig. 1C), but leads to increased false positives (Fig. 1 A and B). Using the intersection of the translational activity data only analysis and the log ratio approach limits identification of differential translation to situations that are shown in Fig. 1D and may fail to identify instances of translational regulation that are independent of cytosolic mRNA levels (Fig. 1C). One might argue that a set of heuristic thresholds for differences at the translational activity data and cytosolic mRNA data could be used either alone or in combination with the translational activity data only or the log ratio approach. Such heuristics suffer from numerous failings, however, including the absence of objective statistical thresholds and that any choice of subjective threshold generates unknown rates of false positives and false negatives. Thus, from both theoretical and practical standpoints, current approaches are inadequate for analysis of differential translation.

Anota Outperforms Current Approaches for Analysis of Differential Translation.

APV (29) provides a more suitable analysis of translational activity data because APV-corrected translation measurements do not, by definition, show spurious correlations. We have implemented this approach in the anota (analysis of translational activity) R-package. In anota, a common slope for all sample categories is identified for each gene from the least squares linear regression of translational activity data on cytosolic mRNA data. Class comparison effects are estimated by calculating differences between sample category intercepts; the sum of squares error for these comparisons is reduced by the sum of squares associated with the covariance between the translational activity data and the cytosolic mRNA data. We found that anota potentially correctly analyzes the different scenarios outlined in Fig. 1 A–D. Anota corrects for confounding cytosolic mRNA levels (Fig. 1A), handles false positives arising from technical issues (Fig. 1B), identifies translational regulation that is independent of cytosolic mRNA levels (Fig. 1C), and identifies translational control when there is some confounding cytosolic mRNA difference (Fig. 1D). In summary, anota handles different problematic scenarios by automatically adapting the analysis to the situation at hand.

To further assess anota’s performance, we analyzed genome wide data from both the polysome microarray approach and the sequencing-based approach in three studies [Ingolia et al. (24), Kitamura et al. (21), and Otulakowski et al.(9); see Fig. 1E]. Two sample categories were identified in each study. We compared results obtained from anota with those obtained from a t-test between the log ratios of the two sample classes. Scatter plots of effects and of p-values reveal large discrepancies between the two approaches for many genes (Fig. S1). We investigated this further by classifying mRNAs as either translationally activated or inactivated and examining the overlap among the top 5% of differentially translated mRNAs (ranked by nominal p-values) between the two analytic approaches. As shown in Fig. 1F, the number of genes that were identified as translationally regulated by one method but not the other was substantially higher than the number of shared genes. Genes were then classified into one of three sets: identified by anota only, log ratio only, or by both log ratio and anota. The top two genes (ranked by p-value) from each set from the Ingolia et al. study (24) illustrate the differences between the methods (Fig. 2). Of the top two genes identified by the log ratio approach, one was of the type exemplified in Fig. 1B with cytosolic mRNA data differences combined with smaller translational activity data differences (Fig. 2, rank 2) whereas the other showed a larger difference at the cytosolic mRNA level compared to the translational activity level, leading to the conclusion that a gene with lower translational activity data is more translationally active. This could be a case of “overcorrection” and hence represents a false positive or possibly a biologically interesting finding. However, it is commonly assumed that a gene that is translationally activated would also show higher protein levels and this gene would therefore not be a primary target for follow-up studies. The top two genes that were identified by anota only showed translational regulation that was independent of cytosolic mRNA levels and accordingly belonged to the type exemplified in Fig. 1C. The genes that were identified by both anota and the log ratio approach showed translational regulation independent of cytosolic mRNA level (as exemplified in Fig. 1C) or a small difference in cytosolic mRNA level associated with a larger difference in translational activity data (as exemplified in Fig. 1D). The two other studies analyzed in Fig. 1F showed similar results (Figs. S2 and S3).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2.

A comparison of the top two mRNAs identified as differentially translated by the log-ratio approach and by anota. Translational activity data (indicated as translation on each y-axis) and cytosolic mRNA data (indicated as transcription on each x-axis) for the top two ranked genes (by p-value) for each set (log-ratio only; anota only; and both) are plotted using a common scale for all graphs. The effects obtained from anota and from the log-ratio approach are indicated together with their corresponding p-values. The differences in means between the sample classes for the translational activity data only and the cytosolic mRNA data only are indicated as delta translation and delta transcription, respectively. Lines represent the regression lines estimated by anota for the two sample categories.

To extend the observations from Fig. 2, we compared all genes from the three sets (identified by anota only, log ratios only, or both log ratios and anota). Within each set, genes were classified further into one of three modes of regulation based on between-group changes in translational activity data and cytosolic mRNA data (shown as delta translation and delta transcription respectively, in Fig. 2). The first mode, labeled “translation > transcription,” identifies those genes for which the translational activity data difference was larger than the opposing cytosolic mRNA difference; i.e., for a gene that is translationally activated, the activation originated primarily from an activation at the translational activity level and not from a decrease at the cytosolic mRNA level (all genes identified by anota only or by both anota and log ratios in Fig. 2 fit into this group). The second mode, labeled “no translation,” identifies those genes for which there was no difference in the translational activity data; i.e., for a gene defined as activated, no increase in the translational activity data (delta translation ≤ 0) was associated with a decrease in cytosolic mRNA data (the log ratio only genes from Fig. 2 would belong to this mode). The third mode, labeled “transcription > translation,” identifies those genes that show a translational activity data difference but for which the cytosolic mRNA data difference in the opposite direction is of larger magnitude. Of these three modes, the translation > transcription mode corresponds to the examples shown in Fig. 1 C and D and represents cases when translational regulation is plausible whereas the no translation and the transcription > translation modes are variants of the example in Fig. 1B when it is unclear if there is any translational regulation. Anota primarily identified genes belonging to the preferred translation > transcription mode whereas most of the genes identified by the log ratio approach belong to the undesired no translation or transcription > translation modes (Fig. 3). Interestingly, those genes that were identified by both anota and the log ratio approach showed a distribution among the three modes similar to that of anota and thus represent a subset of genes for which the log ratio approach obtained a result similar to anota. In summary, anota outperforms the log ratio approach and can be used both for microarray-based as well as sequencing-based datasets.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 3.

A comparison of the mode of translational regulation of genes identified as differentially translated by the log-ratio approach and by anota. The top 5% of genes ranked by p-value from the three examined datasets (from Fig. 1F) were separated into translationally inactivated and translationally activated (i.e., negative and positive effects) and further divided into three sets each: those identified by log ratio only, those identified by anota only, and those identified by both log ratio and anota. The genes within each of these six sets were categorized based on their mode of regulation (the modes are further discussed in the main text): Translation > transcription (similar to Fig. 1 C and D); no translation (similar to Fig. 1B); Transcription > translation (similar to Fig. 1B). Shown is the percent of genes among the translationally activated or translationally inactivated belonging to each mode.

Improving the Performance of Anota Using Variance Shrinkage.

In genome wide studies, gene-specific variances of many genes will be greatly under or overestimated due to chance factors. It is therefore possible that some of the genes shown in Fig. 2 have been identified as a result of unrealistically low variance estimates. This characteristic of high dimensional data has produced a consensus among methodologists to favor variance shrinkage methods that adjust error variances for statistical tests by weighting gene-specific variance estimates and a variance estimate obtained from all genes in the dataset (30). We have generalized one such method, the random variance model (RVM) (31), for APV within anota (referred to as “anota RVM”). We performed an analysis identical to that shown in Fig. 2 but with the RVM-adjusted error term. The top three genes that were selected by anota RVM but not by the log ratio approach in the Ingolia et al. study (24) are shown in Fig. 4A. These genes show larger effects than those genes identified without RVM (compare Fig. 4A to Fig. 2). To illustrate the effect across all genes, we generated “volcano” plots comparing anota and anota RVM. The volcano plots from the Ingolia et al. dataset show that smaller effects are less likely to be associated with high significance when anota RVM is used (Fig. 4B). Finally, we compared the cumulative distribution function of the raw p-values and p-values that had been adjusted for multiple testing (32) from anota and anota RVM (Fig. 4C). This analysis indicated that anota RVM generated smaller p-values than anota without RVM adjustment [similar results were obtained for the other two datasets from Fig. 1F (Figs. S4 and S5)].

• Download figure
• Open in new tab
• Download powerpoint

Fig. 4.

RVM improves the performance of APV within anota. (A). Translational activity data (indicated as translation on each y-axis) and cytosolic mRNA data (indicated as transcription on each x-axis) from the top two genes (ranked by p-value) that were identified among the top 5% differentially translated (ranked by p-value) by anota RVM but not by the log-ratio approach are shown. The two sample categories are indicated by squares and triangles. The same scale as in Fig. 2 is used. Effects calculated by anota RVM and by the log-ratio approach are indicated together with their corresponding p-values. The differences in means between the sample classes for the translational activity data only and the cytosolic mRNA data only are indicated as delta translation and delta transcription, respectively. Lines represent the regression lines estimated by anota for the two sample categories. (B). Volcano plots for anota and anota RVM. The - log 10 p-value is plotted as a function of the effect. (C). A comparison of statistical power between anota and anota RVM. Shown is the cumulative distribution function (CDF) of all p-values from anota and anota RVM. CDFs for both the raw p-values and p-values adjusted for multiple testing as described in the text are shown.

We searched for enrichment of genes with shared biological functions among genes identified by anota RVM and the log-ratio approach to assess whether the former enabled new biological insights. Genes identified as differentially translated by anota RVM shared functions to a larger extent than genes identified as translationally regulated by the log-ratio approach in all three datasets shown in Fig. 1F (Figs. S6–S8). This suggests that anota RVM is associated with fewer biological false positive and false negative findings inasmuch as a completely random set of genes would not enable identification of any enriched biological functions. Thus, anota identifies differential translation that is more biologically coherent compared to log ratios and thereby offers greater promise for biological discoveries.