thoa dataset

The thoa dataset command group lets you manage and interact with datasets stored in Thoa.
Datasets are collections of uploaded input files used when launching jobs.
Each dataset is uniquely versioned and stored in the cloud, so you can reuse them across multiple jobs without re-uploading the same data.

Examples

List your top 10 datasets by size

thoa dataset list --sort-by size --number 10

Download specific files from a dataset

thoa dataset download <DATASET_UUID> ./localdir --include "reads/*.fastq.gz"

Find the largest dataset by number of files

thoa dataset list --sort-by files --number 1

Commands

thoa dataset list

Lists all datasets available to your workspace.

thoa dataset list

thoa dataset download <DATASET_UUID> <DESTINATION_PATH>

Downloads a dataset (or selected files from it) to a local folder.

• DATASET_UUID: the dataset's ID, shown by thoa dataset list or thoa dataset ls. The easiest way to get it, though, is from the dataset in the UI: it has a Download button that generates the full command (ID and syntax included) for you.
• DESTINATION_PATH: a local folder to download into. It's created automatically if it doesn't exist.

thoa dataset download <DATASET_UUID> <DESTINATION_PATH>

By default this downloads every file in the dataset. Use --include/--exclude to grab only part of it: each value can be a file's public ID (from thoa dataset ls) or a glob pattern matched against file paths, e.g. --include "reads/*.fastq.gz".

Each dataset has a limited number of downloads (shown in the CLI output as "downloads remaining"). A download only counts against that limit once files actually transfer successfully; a failed attempt (e.g. insufficient disk space) doesn't consume it. Very soon, we'll be implementing unlimited downloads, removing this limit entirely.

thoa dataset ls <DATASET_UUID>

Lists all files in a dataset by its UUID.

thoa dataset ls 157d2823-xxxx-xxxx-xxxx-xxxxxxxxxxxx