1. Support
  2. Help - Installing & Using Software
  3. Setting Up the RNA-Seq FASTQ Processing Pipeline on macOS

Setting Up the RNA-Seq FASTQ Processing Pipeline on macOS

Published: Updated:

This article explains how to set up the RNA-Seq FASTQ processing pipeline in Subio Platform for macOS. Subio Platform uses the following tools to obtain gene expression data from FASTQ files. For the Windows setup instructions, see Setting Up the RNA-Seq FASTQ Processing Pipeline on Windows.

Tool Function Tested Version
fastp Adapter trimming and low-quality read filtering 1.1.0
Kallisto Rapid expression quantification using a reference transcriptome 0.52.0 (official release)
HISAT2 Read alignment to a reference genome 2.2.3
StringTie Transcript expression quantification using HISAT2 alignment results 3.0.3

The versions listed above have been tested on Macs with Apple Silicon. Other versions may also work.

This article first explains how to set up the Kallisto pipeline, followed by the setup procedure for HISAT2 + StringTie. If you do not plan to use HISAT2 + StringTie, you can skip that part of the setup.

If an error occurs while processing FASTQ files, check the temporary folder created inside the folder containing your FASTQ files. The run.log file in that folder records information about pipeline execution and errors. If the log has been compressed as run.log.gz, decompress it before viewing its contents. You may be able to identify the cause of the error and find possible solutions by asking an AI assistant such as ChatGPT to examine the run.log file.

1. Installing Conda

Conda, a package manager, is used to install fastp, HISAT2, and StringTie. If Conda is not installed, we recommend using Miniforge. On Macs with Apple Silicon, select the arm64 installer. If you already have a working Conda environment, you do not need to install Miniforge separately.

After installation, open Terminal. If the conda command is not recognized, load etc/profile.d/conda.sh from your Miniforge installation directory. The installation path may vary depending on your environment.

2. Installing fastp

Run the following command in Terminal to install fastp from Bioconda. 

conda install -n base bioconda::fastp

Once installation is complete, run the following command to find the path to the executable. 

which fastp

Enter the displayed path in the fastp executable field in the Subio Platform pipeline settings.

3. Downloading and Configuring Kallisto

Download the macOS version of Kallisto from its official GitHub repository. Kallisto can also be installed through Bioconda, but in our test environment, the Bioconda-installed executable could not properly read the index file. Therefore, we recommend using the official Kallisto release that we have verified to work with Subio Platform.

  1. Open the official Kallisto releases page.
  2. Download kallisto_mac-v0.52.0.tar.gz.
  3. Extract the downloaded archive.
  4. In the Subio Platform pipeline settings, select the extracted Kallisto executable.

If macOS Security Blocks Kallisto

Files downloaded through browsers such as Safari may be assigned a quarantine attribute by macOS. As a result, Gatekeeper, a macOS security feature, may block Kallisto when you try to run it for the first time. If this happens, first verify that the file was downloaded from the official Kallisto releases page, then follow Apple's instructions to allow it to run.
Apple Support: Safely open apps on your Mac

4. Configuring the Kallisto Index

To process FASTQ files with Kallisto, you need a transcriptome index for the species you are analyzing. Click the "Download reference data..." button in the Subio Platform pipeline settings to open the species selection window. After you select a species, the required reference data will be downloaded and configured automatically. If you have already downloaded the reference data, you can specify the path to the corresponding file directly.

5. Testing the Kallisto Pipeline

Because processing FASTQ files can take considerable time, we recommend first limiting the number of reads processed to confirm that the pipeline works correctly.

  1. Open the pipeline settings in Subio Platform.
  2. In the fastp settings, select the "Process 10000 reads" preset and click "Load".
  3. Import FASTQ files for one or two samples and confirm that processing completes successfully.
  4. After the test is complete, remove the samples imported for testing.
  5. In the fastp settings, select "Default" and click "Load" to restore the normal processing settings.

Be sure to restore the "Default" preset before starting your full analysis. If "Process 10000 reads" remains selected, only up to 10,000 reads from each FASTQ file will be processed.

6. Installing HISAT2 and StringTie

This section explains how to set up the HISAT2 + StringTie pipeline. If you only plan to use Kallisto, you can skip the remaining setup steps. You can continue using the Conda environment and fastp installation you have already set up.

Open Terminal and, if necessary, initialize your Conda environment. Then run the following command. 

conda install -n base bioconda::hisat2 bioconda::stringtie

Once installation is complete, run the following commands to find the paths to the executables. 

which hisat2
which stringtie

Enter the displayed paths in the HISAT2 and StringTie executable fields in the Subio Platform pipeline settings. The versions installed from Bioconda may vary depending on the installation date and your system environment.

7. Configuring the HISAT2 Index and GTF File

The HISAT2 + StringTie pipeline requires a reference genome index and a GTF file containing gene annotations.

  1. Download the HISAT2 Index for the species you are analyzing.
  2. Prepare a GTF file that corresponds to the same reference genome version used to build the index.
  3. In the Subio Platform pipeline settings, specify the paths to the HISAT2 Index and GTF file.

Make sure the HISAT2 Index and GTF file correspond to the same reference genome. Using different genome versions may affect the accuracy of expression quantification.

8. Testing the HISAT2 + StringTie Pipeline

As with Kallisto, first limit the number of reads processed to confirm that the HISAT2 + StringTie pipeline works correctly.

  1. In the Subio Platform pipeline settings, select the "Process 10000 reads" preset for fastp and click "Load".
  2. Import FASTQ files using the HISAT2 + StringTie pipeline.
  3. Confirm that processing completes successfully.
  4. Remove the samples imported for testing.
  5. Restore the "Default" fastp preset and click "Load".

Use the "Default" preset for your full analysis.

9. Starting Normal FASTQ Processing

The RNA-Seq FASTQ processing pipeline is now configured. After confirming that the settings used for testing have been restored, import the FASTQ files you want to analyze. Note that the tests described above only confirm that the pipeline can run successfully from start to finish. They do not establish the reliability of the resulting expression data. To assess data quality and the reliability of the results, it is important to examine read quality, alignment results, and expression distributions across samples. For more information, see our case study on identifying problems in analysis results through data visualization.

Subio Platform Pipeline Setup and Execution Workflow

The diagram below illustrates the sequence of screens from configuring the pipeline in Subio Platform to executing FASTQ file processing. The basic workflow is the same for both Kallisto and HISAT2 + StringTie.

FASTQ Processing Settings Subio Platform