Standard workflows combining multiple tools are widely used in RNA-Seq analysis. As these workflows and the associated training resources have become more accessible, researchers without specialized bioinformatics expertise have gained the ability to perform complex analyses. However, this progress has also created a problem: following the prescribed steps and completing the workflow can become ends in themselves. Indeed, analysis results of questionable reliability can be published in scientific papers and pass peer review even when underlying problems in the data have been overlooked.
The underlying problem may be that procedure-focused training encourages researchers to regard following a standard workflow as inherently correct, leaving too little room for critically evaluating the results.
Understanding Statistical Models Is an Important Step Beyond Following Procedures
One important response to this problem is to emphasize understanding the statistical models behind analytical tools, rather than simply learning how to operate them. For example, when using edgeR or DESeq2, understanding why negative binomial models are used, why dispersion estimation matters, and what assumptions these models rely on provides an essential foundation for evaluating the results. StatQuest with Josh Starmer is an excellent educational resource that explains these statistical concepts through accessible videos. Understanding the reasoning behind statistical models and mathematical formulations allows researchers to move beyond simply operating analysis tools.
However, there is another concern.
If education intended to promote an understanding of statistical models turns into a collection of rigid rules, such as “You must always use this method for this type of data” or “You must never use that method,” it risks becoming a new form of dogmatism.
Consider the statement that “t-tests should not be used for RNA-Seq analysis.” There are clear problems with applying an ordinary t-test directly to raw count data. However, that does not mean the potential applicability of t-tests to appropriately processed data can be categorically dismissed.
The shift toward understanding statistical models is an important development. But if that understanding is reduced to another set of inflexible rules, we will have failed to fully learn from the limitations of procedure-driven education.
The Real Goal Is to Understand Biological Questions
Ultimately, what we want from RNA-Seq analysis is insight into the biological questions we are investigating. We may be searching for genes involved in the causes of a disease, trying to understand how cells respond to a drug, or investigating the mechanisms behind an observed phenomenon. Given these different research objectives, it makes little sense to assume that the same analytical procedure will always be optimal. Statistics and biology do not necessarily approach data analysis with the same priorities, but perhaps what connects them is how we formulate questions, examine data, and evaluate results.
Using statistical methods correctly is important, but it is not the ultimate goal of research. What matters is developing the ability to decide which methods to use, and how to use them, to understand biological questions.
In exploratory data analysis, we may not initially know what is happening in the data, or even what we should be investigating. We therefore begin by examining the data, developing hypotheses about possible explanations, and considering what expression patterns we would expect if those hypotheses were correct. We then select appropriate analytical methods to investigate those patterns. By repeating this process, we gradually move toward a better understanding of the underlying biology.
The important point is that following the conventional use of statistical methods is not necessarily the only valid approach. Exploratory data analysis can also involve understanding the principles and limitations of existing methods and creatively exploring ways to use different tools according to the research objective, without being constrained by standard workflows or conventional decision thresholds.
Before Seeking Precise Answers, Consider Whether We Are Asking the Right Questions
John W. Tukey, a statistician who made major contributions to the development of exploratory data analysis (EDA), wrote in his 1962 paper The Future of Data Analysis :
Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.
This statement does not reject statistical rigor. Rather, it reminds us that no matter how precisely we calculate an answer, we may fail to advance our research if we have not asked an appropriate question in the first place. In RNA-Seq analysis, perhaps we should reconsider not only whether our analytical methods are technically correct, but also what we are actually trying to discover.
Rigor at the Individual Step Does Not Guarantee the Validity of the Overall Analysis
As RNA-Seq analysis methods have advanced, individual analytical tools have become increasingly rigorous and internally consistent. However, improving the statistical rigor of individual steps is not the same as ensuring that the overall analysis is appropriate for the biological question, or that its different results are mutually consistent.
For example, a statistical tool performing differential gene expression analysis does not necessarily determine whether the input data have been severely distorted by batch effects or other experimental factors. Even if false positives are appropriately controlled in DEG analysis, the resulting gene list may not provide an answer to the research question. Nor is that gene list necessarily the most appropriate input for subsequent enrichment analysis.
Similarly, even when a method performs exceptionally well in simulations, that evaluation depends on the assumed data-generating model and how the ground truth is defined. The importance of carefully designing and interpreting benchmarking studies has also been discussed in the literature. Whether a method contributes to meaningful discoveries in real biological data is a separate question that requires further investigation.
Using Statistical Tools According to Biological Objectives
In RNA-Seq analysis, it is common to select DEGs using both P-value and fold-change criteria. This approach reflects a practical need in biological research: to consider not only statistical significance but also the magnitude of expression changes.
In case studies comparing edgeR, DESeq2, and t-tests using low-variance, small-sample in vitro data and moderately sized biopsy-derived datasets, many genes identified as significant only by the t-test showed extremely low within-group variance but relatively small expression changes. However, when a fold-change criterion was added, many of these genes were excluded, and the overlap among the gene lists produced by the three methods increased.
In other words, a practical selection criterion long used in biological research may, in these cases, have approximated the gene selection produced by modern statistical models. Of course, these observations cannot be generalized to every dataset. Nevertheless, it is interesting that practical approaches developed in biological research can sometimes reduce differences between the results of statistical methods.
As another example, a case study exploring expression patterns predicted by biological hypotheses uses a thought experiment involving combinations of causal factors and inhibitors to propose an unconventional use of t-test P-values. Rather than collecting genes with the smallest P-values, it suggests deliberately selecting genes with P-values near the conventional significance threshold and using clustering to identify gene groups whose expression patterns are consistent with biological hypotheses.
Whether this approach would actually enrich for the desired genes remains unknown. Nevertheless, selecting candidate genes based on their consistency with biological hypotheses, rather than relying solely on statistical significance, is an approach worth considering in exploratory analysis.
Many published studies follow a workflow in which DEGs are selected before clustering. However, this approach cannot explore the expression patterns of genes that were not identified as DEGs. In fact, the Bioconductor RNA-Seq workflow also illustrates clustering based on genes with high variation across samples, rather than restricting the analysis to DEGs. To make full use of clustering as an exploratory method, we need to think much more carefully about which genes should be included, depending on the research objective.
What these examples have in common is that statistical measures are used as tools for addressing biological questions, rather than being treated as ends in themselves.
From Exploration to Validation, and Toward New Questions
In exploratory data analysis, the relative importance of avoiding missed candidates and controlling false positives depends on the research objective.
For example, selecting statistically significant DEGs for reporting and identifying a broad range of candidates for functional validation in subsequent experiments do not necessarily call for the same selection criteria. Indeed, some genes that were not identified as statistically significant DEGs have subsequently been shown experimentally to have biological importance.
Of course, candidates identified through exploratory analysis require careful biological validation. Nevertheless, the objectives and methodologies of exploratory RNA-Seq analysis deserve to be discussed with the same seriousness and depth that we devote to statistical rigor.
Exploratory data analysis is not simply a process of answering the question we started with. By examining the data, we can also develop and refine the questions themselves. Its value lies in generating hypotheses, comparing them with observations, narrowing down possibilities, and identifying new questions that deserve further investigation.
Changing hypotheses or analytical methods during exploration may raise concerns about cherry-picking. However, the boundary between legitimate exploratory analysis and cherry-picking is not always clear, and the statistical implications of data-dependent analytical decisions have already been widely discussed. In this context, the important principle is not to avoid trial and error, but to make the exploratory process transparent, as emphasized in the American Statistical Association's statement on P-values.
Rather than reporting only successful attempts, researchers should document the hypotheses they considered, the approaches they tried, and what did not work, while clearly distinguishing insights generated through exploration from conclusions established through validation.
As AI increasingly draws on scientific knowledge, this principle is likely to become even more important.
