Data

The data resource publishes robot data to Qualia and downloads exact published trees. Uploaded bytes go straight to storage; the Qualia API brokers verified manifests and paired publication, never the bulk data itself.

Install

pip install qualia-sdk

The native transfer engine that powers upload/download is bundled in the wheel — there is no separate step.

Upload a dataset

Stage a local dataset directory and atomically publish its validated source, Git history, and lakehouse rows in one call. Episodes and frames become visible only after the paired publication succeeds.

from qualia import Qualia

client = Qualia()

result = client.data.upload("/data/gongsta", name="gongsta")
print(result["dataset_id"], result["episode_count"], result["total_frames"])

Response

{
  "repo_path": "acct-01234567-89ab-4cde-8fab-0123456789ab/gongsta",
  "name": "gongsta",
  "file_count": 128,
  "committed_files": 128,
  "dataset_id": "…",
  "dataset_version_id": "…",
  "episode_count": 42,
  "total_frames": 51840
}

Parameters

Parameter	Default	Description
local_dir	—	Directory of dataset files (or an MCAP session) to upload.
name	—	Dataset name within your account (a single path segment).
source_format	"lerobot_v3"	Structured source format. One of lerobot_v2_1, lerobot_v3, or mcap_ros2; the two LeRobot layouts are distinct.
max_workers	4	Files read + chunked concurrently. Bounds CPU/RAM/disk so a large upload can’t saturate your machine.
num_threads	~half your cores	Native chunk/hash thread-pool size.

Re-uploading a dataset only sends the chunks that changed — unchanged files are skipped client-side, and edited files upload just their changed regions.

Import an exact Hugging Face dataset

Install the optional official Hub client:

pip install 'qualia-sdk[huggingface]'

The import runs on your SDK client, not as an API-side URL fetch. It accepts only an exact lowercase 40-hex commit and the fixed official Hub endpoint. First resolve the bounded file/byte summary, then import that same commit:

from qualia import Qualia

client = Qualia()
repo = "qualia-robotics/openarm-cube-in-box-v1"
revision = "17c29c487d886878ddcb3c5649101b2860fbc047"

summary = client.data.preflight_huggingface(
    repo,
    revision,
    source_format="lerobot_v3",
    max_bytes=5 * 1024**3,
)
print(summary["file_count"], summary["total_bytes"], summary["tree_sha256"])

result = client.data.import_huggingface(
    repo,
    revision,
    "openarm-cube-in-box",
    source_format="lerobot_v3",
    max_bytes=5 * 1024**3,
)
print(result["dataset_version_id"], result["git_oid"])

For private or gated repositories, pass hf_token=os.environ["HF_TOKEN"]. The token is sent as an authorization value to the official Hub client; it is never placed in a URL, provenance file, result, or log. The downloaded files are streamed through their remote identity (LFS SHA-256 or canonical Git blob OID), and those per-file identities are committed into the source provenance sidecar before the normal staged, direct-to-storage QDS publisher runs. Success always includes a Source, an immutable DatasetVersion, and its paired Git/snapshot receipt. A repository containing a zero-byte file is rejected before download because canonical QDS source manifests cannot represent one without changing the remote tree.

QDS intentionally rejects dot-prefixed Git-control paths. The importer permits only the exact root names .gitattributes and .gitignore: it downloads and verifies them, commits their path, size, Git blob OID, and content identity under excluded_repository_metadata, and removes them before QDS publication. Any other hidden path, .lock path, or meta/qds/** attempt fails preflight. The provenance sidecar itself is published at the non-authoritative, QDS-valid meta/qualia/imports/huggingface-dataset.json path.

The pinned medium validation fixture TobiBrtnr/openarm_shirt_folding_new_converted@c34057e9cb7760f939dbd5759679c12818fa896b is LeRobot 2.1 and about 2.08 GB. Its pinned media inventory reports AV1 even though legacy metadata may label it avc1; QDS treats validated media bytes as authoritative. The public metadata does not establish collection hardware, so do not describe it as real-robot data without separate provenance evidence.

Upload a robot MCAP session

Recorded robot sessions upload through the same call with source_format="mcap_ros2". Lay the session out as one subdirectory per episode, each holding exactly one .mcap bag. An episode.json sidecar may provide normalized task and mode fallbacks when the bag does not:

session-2026-07-20/
  episode-000/
    episode-000.mcap
    episode.json        # { "task": "fold the towel", "mode": "teleop" }
  episode-001/
    episode-001.mcap
    episode.json

result = client.data.upload(
    "/data/session-2026-07-20",
    name="dk1-kitchen",
    source_format="mcap_ros2",
)
print(result["dataset_id"], result["episode_count"], result["total_frames"])

The upload validates the session first and fails fast if a bag is missing or not a real MCAP file. A missing sidecar is valid; QDS derives task and recording purpose from authenticated bag contents when available.

Download a dataset

Download a dataset into a local directory, preserving relative paths. Returns the list of written file paths. This is repository-tree download; for a pinned training dataset, use immutable version materialization below.

paths = client.data.download("gongsta", dest="/data/gongsta-copy")

Materialize an immutable DatasetVersion

Render one pinned DatasetVersion as an exact LeRobot v3 directory and download it directly from object storage. The API returns a durable operation and bounded pages of immutable artifact metadata; bulk bytes never pass through the Gateway. The SDK authenticates the complete closure, verifies every artifact, and atomically publishes dest only after the whole tree succeeds. Camera streams use QDS’s versioned canonical AV1 profile; an already canonical, full-range AV1 source can be preserved byte-for-byte, while trims and other codecs are frame-exactly encoded and fully decoded for verification. The destination must not already exist.

result = client.data.materialize_version(
    "44444444-4444-4444-8444-444444444444",
    "/data/training-set-v7",
)
print(result.bundle.bundle_id, result.written_file_count, result.destination)

Interrupted rendering is recoverable: repeating the call addresses the same deterministic materialization identity. A failed download never exposes a partially published destination.

Explore the episode lake

Filter episodes across every source dataset in the authenticated account with one bounded query. The result includes one offset page, totals over the complete filtered set, and task, tier, and embodiment facets. Each facet axis excludes its own filter while retaining every other filter.

from qualia import LakeExploreQuery, LakeNumericPredicate

query = LakeExploreQuery(
    tasks=["fold shirt"],
    tiers=["retrainable"],
    tags=["reviewed"],
    numeric_predicates=[
        LakeNumericPredicate(field="jerk_mean", op="lt", value=0.4)
    ],
    limit=50,
)
page = client.data.explore(query)
print(page.total, page.total_frames, page.facets.by_task)
for episode in page.episodes:
    print(episode.dataset_name, episode.episode_index, episode.metrics)

Numeric fields are the lake columns duration_s, frame_count, and quality_score, plus QDS’s typed computed metrics. Channel predicates support action, state, and telemetry streams with min, max, mean, or std. Every matching episode carries channel_predicate_evidence: the exact native channel, index, aggregate observation, operator, and threshold that satisfied each predicate. The SDK binds that evidence back to the ordered request and rejects missing, changed, non-finite, or non-satisfying observations.

Search is a literal, case-insensitive substring match over task and dataset name; % and _ have no wildcard meaning. A facet axis returns at most 512 values, while by_task_total_values, by_tier_total_values, and by_interface_hash_total_values report the complete cardinality. Active selections are retained when an axis is truncated.

Each response’s page, totals, and all facet axes come from one database statement and therefore one live MVCC snapshot. page_limit and page_offset bind the response to the request. Offset pagination is not mutation-stable between calls: after publishing data or changing review state, restart at offset zero. Use exploration for analysis and review, then publish a pinned DatasetVersion for reproducible training. The SDK rejects malformed rows, duplicate identities, inconsistent totals or facets, altered evidence, and truncated or mismatched pages.

Analyze an immutable DatasetVersion

Run native action/state health analysis against one exact published version:

report = client.data.health_for_version(
    "44444444-4444-4444-8444-444444444444"
)
print(report.health_score, report.grade)
for flagged in report.flagged_episodes:
    print(flagged.member_ordinal, flagged.episode_revision_id, flagged.reasons)

QDS re-proves the complete ordered membership and dataset-snapshot identity. It then fully proves and reads exact Parquet ranges for an evenly spaced, deterministic sample of at most 120 members. Every flag is bound to an immutable episode revision and is a review recommendation, not an automatic deletion.

The sample is explicit in report.sampling. This synchronous analysis is not a byte scrub of unsampled artifacts or video, and it does not claim model-based visual or behavioral metrics without a pinned model revision and output artifact; those fields are marked unavailable in evidence_completeness.

Review and apply bad-data suggestions

Analyze one or more immutable DatasetVersions without changing them. Every recommendation includes a risk score, machine-readable reasons, and supporting evidence. The response binds the complete recommendation set to an evidence digest and the exact paired Git/lakehouse head.

evidence = client.data.suggest_curation(
    ["44444444-4444-4444-8444-444444444444"]
)
for suggestion in evidence.suggestions:
    print(suggestion.episode_index, suggestion.risk_score, suggestion.reasons)

After review, publish the analysis’s sealed recommendation set as a new immutable DatasetVersion. Reuse the same idempotency key after a lost response. The source version, source bytes, and Git history remain available.

from uuid import uuid4

receipt = client.data.apply_curation(
    evidence,
    idempotency_key=uuid4(),
    definition_id="55555555-5555-4555-8555-555555555555",
    note="Reviewed automated bad-data evidence",
)
print(receipt.version_id, receipt.membership_hash, receipt.git_oid)

Application fails closed if the analysis selection, evidence digest, repository head, target definition, membership counts, snapshot, or Git receipt changes between review and publication.

List datasets

page = client.data.list()
for d in page.items:
    print(d.artifact_id, d.path)

Get immutable training features

Training accepts a pinned DatasetVersion, never a mutable repository name or external dataset identifier. Retrieve its adapter-pinned camera descriptors and use their key values in camera_mappings.

features = client.data.get_training_features(
    "44444444-4444-4444-8444-444444444444"
)
for camera in features.camera_features:
    print(camera.key, camera.stream, camera.timebase_numerator,
          camera.timebase_denominator, camera.frame_timestamp_mapping)

Response

{
  "dataset_version_id": "44444444-4444-4444-8444-444444444444",
  "training_adapter_revision_id": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
  "camera_features": [
    {
      "key": "observation.images.wrist",
      "stream": "video/wrist",
      "timebase_numerator": 1,
      "timebase_denominator": 1000000,
      "frame_timestamp_mapping": "time_start_plus_frame_over_fps"
    }
  ]
}

Next steps

• Finetune — train a VLA on a dataset you’ve uploaded.