This page describes how authorized customers and partners can quickly and efficiently send large datasets, such as BCL, FASTQ, and BAM files, to 10x Genomics. This approach makes it easy to use multi-stream transfers to ensure optimal and secure utilization of your Internet connection. Redstone uploads to Amazon Web Services S3 which is secured using SSL during transport.
Illumina BCL data
- Please archive the BCL folder into a single file, but do not include thumbnail images.
In the following example, replace the strings with the specifics of your run. /mnt/runs should be replaced with the parent folder that contains your Illumina run folders.
tar -czvf H3NVFBBXX.tar.gz -C /mnt/runs --exclude=Thumbnail_Images 150929_ST-K00126_0004_AH3NVFBBXX
FASTQ or BAM data
- If you are sending multiple files, please archive them into a single ZIP or GZipped TAR (
.tgz) file.
In the following example for a FASTQ directory, replace the strings with the specifics of your run. /fastq/directory should be replaced with actual directory that contains your FASTQ files.
tar -czvf compressed-fastqs.tar.gz /fastq/directory
- If your FASTQ files are already individually compressed (
.gzextension) then you do not need to compress the entire archive. Similarly, BAM files are already inherently compressed. In this case, simply tar the files together and omit thezoption to compress.
In the following example for a FASTQ directory, replace the strings with the specifics of your run. /fastq/directory should be replaced with actual directory that contains your FASTQ files. The difference here is omitting a 'z' from the tar command.
tar -cvf allfastqs.tar /fastq/directory
Imaging data
- If you are sending multiple files, please archive them into a single ZIP or GZipped TAR (
.tgz) file. - Please transfer the bright-field H&E TIFF imaging files for analysis. Images should be one per sample, not slide, and should be named in such a way as to make it clear what sample is in a given image.
- Please send images with the following convention:
<slide_serial_number>-<capture_area>.tif
Example Slide: Lot# V19L22, ID# 001.
The four capture areas (from top to bottom) would be submitted as images named:
V19L22-001-A1.tif
V19L22-001-B1.tif
V19L22-001-C1.tif
V19L22-001-D1.tif
The remaining steps are operating system dependent. Please select the link (right) for your operating system to continue.
10x Genomics provides an easy-to-use multi-stream upload tool called Redstone. If you installed Cell Ranger v4 or higher, the Redstone executable is already located in the Cell Ranger install path cellranger-x.y.z/lib/bin/redstone and need not be installed again. You may add this path to your .bashrc using export PATH=/path/to/cellranger-x.y.z/lib/bin:$PATH or type in the full path on your command prompt to call Redstone.
The download link below is for the Linux version, compatible with RedHat/CentOS 5.2 or later, and Ubuntu 8.04 or later:
Redstone is a single 9MB executable that can be run directly and requires no compilation or installation. Place the executable file in a directory that is on your PATH. For this Linux version please make sure to chmod 700 to make it executable.
Make sure to perform this step from a computer with an unencumbered path to your Internet connection. That means hard-wired 1GbE or 10GbE connections, and not Wi-Fi.
Please also ensure the time/date are set correctly on your system, which is required for security reasons. If your clock is mis-set, you will encounter a RequestTimeTooSkewed error.
To run Redstone, just pass it your email address, the email address of the recipient at 10x Genomics, and the name of the file to be uploaded. The entries for email addresses and file name must be customized before running.
redstone [email protected] [email protected] H3NVFBBXX.tar.gz
Uploading file...
Size: 10,773,244,733 bytes
50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
If you are transferring files to Customer Support, please use [email protected] as the recipient e-mail address.
10x Genomics provides an easy-to-use multi-stream upload tool called Redstone. The link below is for the macOS version.
Download redstone-mac and open a terminal window
After you click the link to download redstone, the executable will download to your Downloads folder. Right click on the file and select Open. You will likely see this message on a pop-up window:
macOS cannot verify the developer of “redstone-mac”. Are you sure you want to open it?
Click Open. It is not a .dmg file so the installer will try to open the application in your default text editor. This file is not human-readable so you can close it. However, this step is necessary because it changes how the macOS gatekeeper views the file, from unsafe to safe.
Next, use the terminal program on your Mac to open a terminal window. A window will open with a prompt that looks something like this:
user.name@computer:~$
Make redstone-mac executable
Use the chmod command to make the redstone-mac file executable.
user.name@computer:~$ chmod 700 Downloads/redstone-mac
Move redstone-mac to Applications
Move the redstone program to the Applications folder on your computer using the mv command.
user.name@computer:~$ mv Downloads/redstone-mac /Applications
Add redstone-mac to the PATH
Next we need to add redstone to your PATH using the export command. This will allow you to type redstone-mac on the command line to run the program.
user.name@computer:~$ export PATH=/Applications:$PATH
Run redstone-mac to get the usage
Now if you type redstone-mac on the command line and push enter it will print the usage statement. The usage statement will contain a pattern for building the command you will use to transfer your data.
user.name@computer:~$ redstone-mac
Usage:
redstone --from=EMAIL --to=EMAIL [options]
redstone -h | --help | --version
Transfer a single file
If you are sending a single file, such as a tissue optimization cDNA footrpint tiff, you would replace file in the usage statement above with a "path" to the file. A "path" is the series of folders that you would go through on your computer to get to the file. A trick that works on macOS is to find the folder you want to send in finder and drag and drop it into the terminal window. If I wanted to send an image downloaded from a shared drive into the Downloads folder, the command would look like this:
user.name@computer:~$ redstone-mac [email protected] [email protected] /Users/username/Downloads/V19L22-001-A1.tif
Uploading file...
Size: 10,773,244,733 bytes
50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
Transfer a compressed directory of files
If you are sending a compressed directory of files, like you would have if you compressed multiple images using the tar command described in "Prepare Your Data" above, the final command will have the same structure as sending a single file and may look like this:
user.name@computer:~$ redstone-mac [email protected] [email protected] /Users/username/Downloads/H3NVFBBXX.tar.gz
Uploading file...
Size: 10,773,244,733 bytes
50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
If you are transferring files to Customer Support, please use [email protected] as the recipient e-mail address.
10x Genomics provides an easy-to-use multi-stream upload tool called Redstone. The link below is for the windows version:
Download redstone.exe and open a command prompt window
After you click the link to download redstone, the executable will download to your Downloads folder. Do NOT click on the downloaded file. Installation does not require the windows installer, it will give an error. Instead do the following:
Use the Command Prompt app to open a Command prompt. You can find Command Prompt by searching for it in the start menu search bar. A window will open with a prompt that looks something like this:
C:\Users\username>
Run redstone-exe from the command prompt to get the usage statement
Now if you type downloads\redstone.exe on the command line and push enter it will print the usage statement. The usage statement will contain a pattern for building the command you will use to transfer your data.
C:\Users\username> Downloads\redstone.exe
Usage:
redstone --from=EMAIL --to=EMAIL [options]
redstone -h | --help | --version
Transfer a Single File
To transfer a file the first step is to change directories to the directory that contains the file you want to transfer using the cd command. If the file is in the Downloads directory the cd command would look like this:
C:\Users\username> cd \Users\username\Downloads
The redstone.exe command would then start with a path to the Redstone executable followed by the --to= and --from= email addresses and ending with the name of the file that you want to transfer. The command would look like this:
C:\Users\username> \Users\username\Downloads\redstone.exe [email protected] [email protected] V19L22-001-A1.tif
Uploading file...
Size: 10,773,244,733 bytes
50% [=============> ] 5,386,622,366 376.79Mb/s eta 1m 54s
If you are transferring files to Customer Support, please use [email protected] as the recipient e-mail address.
Error Message: 'redstone.exe' is not recognized as an internal or external command, operable program or batch file.
This error indicates that redstone.exe program did not download properly, the full path to redstone.exe was not provided at the command prompt, or the name of the executable was misspelled. Please try one of these solutions:
- Check the spelling of the executable name
- Run redstone.exe using a full path (Downloads\redstone.exe)
- Click on the download link again to download the executable again