thoa run
The thoa run command submits jobs to Thoa's compute infrastructure.
Jobs run arbitrary code in isolated, reproducible environments, any size, no infrastructure to manage.
Every thoa run invocation requires:
- A command (
--cmd) - Exactly one environment spec (
--tools,--env-source, or--env-id) - Optionally, input data (
--inputor--input-dataset)
Specifying Input Data
There are several ways to provide input data to a job, including combining them in the same run.
Upload local files with --input
Pass one or more local paths. Files and directories are both supported.
# Single directory
thoa run --input ./reads/ --cmd "..."
# Multiple specific files
thoa run --input sample1.fastq.gz --input sample2.fastq.gz --cmd "..."
# A mix of files and directories
thoa run --input ./config.yml --input ./data/ --cmd "..."
Files are uploaded before the job starts and mounted inside the container at the same relative path they have on your machine (relative to your current working directory). For example, if you run from /home/user/project/ and pass --input ./reads/sample.fastq.gz, the file will be available at reads/sample.fastq.gz inside the job.
Limit: Up to 1,000 files per job. For larger datasets, consider using --input-dataset with a dataset you have already uploaded.
Import from Google Drive with --input
Pass a Google Drive URL instead of a local path, in the form <url>::<mount_path>.
Both folder links and single-file share links work:
# A whole Drive folder, mounted at ./data inside the job
thoa run --input "https://drive.google.com/drive/folders/<folder-id>::data" --cmd "..."
# A single shared file
thoa run --input "https://drive.google.com/file/d/<file-id>/view::sample.fastq.gz" --cmd "..."
The first time you use a Google Drive source, thoa run opens your browser to
authorize access. Running over SSH or somewhere without a browser? It still works:
paste the redirect URL when prompted instead.
You can mix local and Google Drive sources in the same run by passing --input
multiple times with different kinds of sources. Files already uploaded, or already
imported from Drive in a previous run, are automatically skipped instead of being
re-transferred. Unsupported Drive file types (e.g. native Google Docs/Sheets) are
skipped with a message rather than failing the whole import.
thoa run \
--input ./local_reads/ \
--input "https://drive.google.com/drive/folders/<folder-id>::remote_reads" \
--cmd "..."
Amazon S3 URLs (s3://...) are recognized but not yet supported.
Export output to Google Drive with --export-to
Pass a Google Drive folder URL to have the job's output dataset exported there automatically once the job finishes. Uses the same browser authorization flow as Google Drive inputs.
thoa run --cmd "..." --tools python --export-to "https://drive.google.com/drive/folders/<folder-id>"
Reuse an existing dataset with --input-dataset
If you have previously run a job or uploaded a dataset, you can reuse it without re-uploading. Pass the dataset UUID (visible in the UI or via thoa dataset list):
thoa run \
--input-dataset 157d2823-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
--cmd "python analyse.py" \
--env-source environment.yml
This skips the upload step entirely, saving time and bandwidth. The files will be staged inside the container at the same paths they had in the original dataset.
The dataset must be fully uploaded and not in the process of being deleted, or the run is rejected immediately. Its total size must also fit within --storage; if it doesn't, the error tells you the minimum --storage value to use instead.
--input and --input-dataset are mutually exclusive. Use one or the other.
No input files
If your job doesn't need input data (e.g. a data generation script), simply omit both flags:
thoa run --cmd "python generate_data.py" --tools python
Specifying an Environment
Every job needs exactly one environment source. The three options are mutually exclusive.
Option 1: --tools (quick tool list)
The simplest option. Pass a comma-separated list of tool names from Bioconda or conda-forge. Thoa builds a conda environment with those tools before running your job.
thoa run --tools "bwa,samtools=1.9,python" --cmd "..."
Pin specific versions with =:
thoa run --tools "fastqc=0.12.1,multiqc,trimmomatic=0.39" --cmd "..."
--tools is best for quick, ad-hoc jobs where you just need a few packages. For reproducibility and complex dependency trees, prefer --env-source.
Option 2: --env-source (YAML, requirements.txt, or your current environment)
Point to a local environment file, or capture your active environment automatically. Three input formats are accepted:
A conda environment YAML (.yml / .yaml): the full spec (channels, packages, pinned versions) is sent to Thoa and built before the job runs:
thoa run --env-source environment.yml --cmd "bash run_pipeline.sh"
A typical environment.yml looks like this:
name: my-analysis
channels:
- bioconda
- conda-forge
- defaults
dependencies:
- python=3.11
- bwa=0.7.17
- samtools=1.18
- fastqc=0.12.1
- multiqc
- pandas
- numpy
The name: field in the YAML is ignored by Thoa; the environment is identified by its UUID.
A pip requirements file (.txt): automatically converted into a conda spec (packages placed under a pip: section):
thoa run --env-source requirements.txt --cmd "python analyse.py"
Your currently active environment (use-current): captures whatever conda
environment or virtualenv you have active right now, no file needed:
thoa run --env-source use-current --cmd "python analyse.py"
This runs conda env export --no-builds if you're in an active (non-base) conda
environment, or pip freeze otherwise, with your current Python version pinned
automatically so the right interpreter gets used.
Once built, the environment is stored and can be reused in future jobs with --env-id (see below), which avoids the rebuild step.
Option 3: --env-id (reuse a built environment)
If you have a previously validated environment, pass its UUID directly. The job skips the environment build step and starts immediately.
thoa run \
--env-id b0f9fefe-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
--input ./reads/ \
--cmd "bash run_pipeline.sh"
To find your environment UUIDs, use:
thoa envs list
Only environments with status validated are usable. If thoa envs list shows validation_failed for an environment, inspect it with thoa envs show <uuid> -v to see the build logs.
Downloading Output
By default, thoa run does not copy anything to your machine: output files
stay in your Thoa workspace once the job completes. You can always pull them down
later with thoa dataset download, or from
the job's page in the Thoa UI.
To have thoa run download the results automatically as soon as the job finishes,
pass --download-dir with a local directory that already exists:
thoa run --cmd "..." --tools python --download-dir ./results
--output and --download-dir are not the same flag. --output (default ./) tells Thoa the path inside the container where your job writes its output; it's used to resolve each output file's relative path. --download-dir is the local folder those files are written into. If you omit --download-dir, the job still runs and completes normally, but nothing is downloaded to your machine.
Examples
Running a Python script
thoa run \
--cmd "python script.py" \
--input ./inputdata \
--tools "python" \
--n-cores 16 \
--ram 64 \
--storage 10 \
--download-dir ./outputs
Running an R script
thoa run \
--cmd "Rscript analysis.R" \
--input ./inputdata \
--tools "r-base" \
--n-cores 16 \
--ram 64 \
--storage 10 \
--download-dir ./outputs
Running a Bash script
thoa run \
--cmd "bash pipeline.sh" \
--input ./inputdata \
--tools "bash,coreutils" \
--n-cores 8 \
--ram 32 \
--storage 20 \
--download-dir ./outputs
Using a full conda YAML environment
For pipelines with complex or pinned dependencies, define a environment.yml and pass it with --env-source:
# environment.yml
name: wgs-pipeline
channels:
- bioconda
- conda-forge
- defaults
dependencies:
- python=3.11
- bwa=0.7.17
- samtools=1.18
- picard=3.1.1
- gatk4=4.4.0.0
- fastqc=0.12.1
- multiqc
thoa run \
--cmd "bash wgs_pipeline.sh" \
--input ./fastq_files/ \
--env-source environment.yml \
--n-cores 32 \
--ram 128 \
--storage 500 \
--download-dir ./outputs
Reusing an existing environment
After running a job once, the environment is saved in Thoa. Reuse it by ID to skip the build step:
# Find the UUID of your validated environment
thoa envs list
# Submit a new job reusing it
thoa run \
--cmd "python analyse.py" \
--input ./new_data/ \
--env-id b0f9fefe-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
--n-cores 16 \
--ram 64 \
--storage 100 \
--download-dir ./outputs
Running a Nextflow pipeline
You can run Nextflow workflows as a single Thoa job today:
thoa run \
--cmd "nextflow run pipeline.nf -profile standard" \
--input ./pipeline/ \
--tools "nextflow,openjdk" \
--n-cores 32 \
--ram 128 \
--storage 500 \
--download-dir ./outputs
Current status: Nextflow pipelines run as a single Thoa job: all steps execute sequentially within that job's allocated resources. We are actively working on native Nextflow integration that will split individual workflow steps into separate Thoa jobs, with each step visible, linkable, and independently re-runnable in the Thoa interface.
Running a Snakemake workflow
thoa run \
--cmd "snakemake --cores 16 --snakefile Snakefile" \
--input ./workflow/ \
--tools "snakemake,python" \
--n-cores 16 \
--ram 64 \
--storage 200 \
--download-dir ./outputs
Current status: Snakemake workflows run as a single Thoa job. All rules execute within the resources allocated to that job. Native Thoa–Snakemake integration (where each rule becomes an individually tracked Thoa job) is in development.
Automatic AI Debugging
If a job's environment fails to build, or the job itself fails during execution, Thoa automatically diagnoses the problem and retries: you don't need to manually inspect logs and resubmit for common issues.
- Preflight check: before the job starts, your script and environment spec are reviewed for likely failure causes (missing packages, hardcoded paths, obvious syntax issues) and corrected automatically where possible.
- Environment debugging: if the environment build fails, the root cause is diagnosed against a knowledge base of previously successful builds, and a corrected environment spec is generated and retried.
- Execution debugging: if the job itself fails after starting, the failure is diagnosed and, where a fix is confident, the job is retried with the correction applied.
Control this behavior with three flags:
# Disable AI auto-retry entirely — job fails as-is, no automatic diagnosis or retry
thoa run --cmd "..." --tools python --strict
# Cap the number of AI retry attempts (default: 3)
thoa run --cmd "..." --tools python --max-attempts 1
# Skip only the preflight check, but keep env/execution debugging on failure
thoa run --cmd "..." --tools python --disable-preflight
Jobs going through an AI-driven retry show status retrying in thoa jobs list / thoa jobs get. The job detail page in the Thoa UI shows exactly what was changed (attempted vs. used script/environment).
Arguments
| Flag | Description |
|---|---|
--cmd (req) | The shell command to run inside the compute environment. |
--input, -i | Local path(s) to upload, or a Google Drive URL as <url>::<mount_path>. Use multiple flags to combine several, including mixing local and Drive sources in one run. Supports directories. |
--input-dataset | UUID of an existing dataset to reuse (skips upload). Mutually exclusive with --input. |
--export-to | A Google Drive folder URL. The finished output dataset is exported there after the job completes. |
--output, -o | Path inside the container where output files will be found. Defaults to ./. |
--tools | Comma-separated tool names from Bioconda / conda-forge (e.g. "bwa,samtools=1.9"). |
--env-source | Path to a conda .yml/.yaml file, a .txt requirements file, or use-current to capture your active environment. See Specifying an Environment. |
--env-id | UUID of an existing validated environment to reuse. |
--n-cores | Number of CPU cores to allocate. Default: 16. |
--ram | GB of RAM to allocate. Default: 64. |
--storage | GB of free disk space for outputs (after inputs are mounted). Default: 200. |
--download-dir | Local directory to download output files into after the job finishes. If omitted, output files are not downloaded automatically. See Downloading Output. |
--run-async | Submit the job and exit immediately after upload, without streaming logs. Monitor progress at thoa.io instead. Default: false. |
--job-name | Optional custom name for the job. |
--job-description | Optional description for tracking purposes. |
--dry-run | Validate inputs and print a cost estimate without submitting the job. |
--verbose | Print additional detail (e.g. a full "Verbose" line) in the job configuration summary shown before submission. Default: false. |
--strict | Disable AI auto-retries: the job will not be automatically diagnosed or retried if it fails. Default: false. See Automatic AI Debugging. |
--max-attempts | Maximum number of AI retry attempts before giving up. Default: 3. |
--disable-preflight | Skip the AI preflight check that reviews your script/environment before the job starts. Default: false. |