Checks_RunCheck
const url = 'https://dev-api.qualiastudios.dev/v1/accounts/example/checks/example:run';const options = { method: 'POST', headers: {'X-API-Key': '<X-API-Key>', 'Content-Type': 'application/json'}, body: '{"name":"example","dataset":"example","selection":{"names":{"names":["example"]},"filter":"example"},"model":"example","validateOnly":true,"maxEpisodes":1,"maxCost":"example","requestId":"example"}'};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request POST \ --url https://dev-api.qualiastudios.dev/v1/accounts/example/checks/example:run \ --header 'Content-Type: application/json' \ --header 'X-API-Key: <X-API-Key>' \ --data '{ "name": "example", "dataset": "example", "selection": { "names": { "names": [ "example" ] }, "filter": "example" }, "model": "example", "validateOnly": true, "maxEpisodes": 1, "maxCost": "example", "requestId": "example" }'Runs a check over a dataset again; checks/- runs the dataset’s default
set (an upload runs episode-rules, a mixture runs mixture; reach is never
in it, name checks/reach). Answers as soon as the run is accepted,
naming the checks started;
they run in the background. Their findings land as issues, and a
finding no longer raised is dismissed as RESOLVED. INVALID_ARGUMENT,
before anything starts, when the check does not apply to the dataset
(MIXTURE on an upload, EPISODE_RULES on a mixture, REACH on a dataset
with no target robot, or one whose target robot is in another workspace,
VLM when no selected episode has a published task) or the
selection matches no episode. VLM is FAILED_PRECONDITION on a
deployment with no VLM check pipeline. On a merged dataset VLM and REACH
are one run over its own episodes (QUA-3463, QUA-3511), and the other
checks run in each upload it holds, one run per upload, the selection
split by upload; either way the findings are issues of the upload that
holds each episode, named under the merged dataset. model chooses the
VLM check’s model; with validate_only nothing starts and the response
carries the estimate.
Each run is a CheckRun (check_runs): GetCheckRun says where it stands.
A person’s checks/vlm (estimate included) is FAILED_PRECONDITION while
a VLM run of it is still running on the dataset, naming that run: a
second would reserve and pay for the same episodes again. A retry with
the first call’s request_id is answered with its receipt, not refused.
checks/reach is the Run solver press, and it is recorded before the
answer: the dataset’s settings (reach_settings, else the target robot’s
defaults) and the robot’s config make the run’s key, every episode that
already has a succeeded result under that key is answered from it (the
other episodes, failed ones included, are queued), and the run, counted
in ReachRun, is what GetReachStatus reports. The RunCheck operation ends
when the work is queued; the run’s own progress is the ReachRun. On a
merged dataset the press is one run over all its episodes, every
upload’s, on its own settings. Succeeded results under its current full
reach key answer it from storage; separate pre-merge TCP solves can
have different calibration identities.
FAILED_PRECONDITION, before anything is queued, on a server where the
reach check is switched off (nothing would run the work).
Authorizations
Section titled “Authorizations ”Parameters
Section titled “ Parameters ”Path Parameters
Section titled “Path Parameters ”The account id.
The check id.
Request Body required
Section titled “Request Body required ”Request for Checks.RunCheck.
object
Accounts/{account}/checks/{check}. checks/- runs the dataset’s
default set: an upload runs episode-rules, a mixture runs mixture.
Reach is never in it; name checks/reach.
Accounts/{account}/datasets/{dataset}
Only these episodes (names under dataset); unset means all. Ignored by
MIXTURE, which always reads the whole mixture.
object
Episode names under the request’s parent.
object
Accounts/{account}/datasets/{dataset}/episodes/{episode}, at most 10000.
A ListEpisodes filter over the parent (e.g. what the filter bar leaves).
checks/vlm only: the model that reads the episodes, one of
CheckEstimate.models (what this deployment’s VLM check template offers).
Empty: the template’s default. INVALID_ARGUMENT, before anything starts,
for a model the template does not offer, or on any other check.
Check the request and answer what it would do, starting nothing and
reserving nothing: every refusal a run would meet, then estimate (VLM
only). checks and check_runs are empty.
checks/vlm only: sample at most this many linked episodes across the
scope. Must be positive; unset checks all eligible episodes.
checks/vlm only: maximum credits reserved and spent. Must be positive.
Optional UUID4 idempotency key. Matching retries return the original admission; a different caller, scope or normalized input is refused. An intentional rerun needs a new key. validate_only never reserves it.
Example generated
{ "name": "example", "dataset": "example", "selection": { "names": { "names": [ "example" ] }, "filter": "example" }, "model": "example", "validateOnly": true, "maxEpisodes": 1, "maxCost": "example", "requestId": "example"}Responses
Section titled “ Responses ”OK
Response for Checks.RunCheck: what was started.
object
Accounts/{account}/checks/{check} of the checks started, in the order they run.
Accounts/{account}/operations/{operation}: the run RunCheck recorded (the
first upload’s, for a merged dataset’s per-upload checks). For
checks/reach it is also ReachRun.operation, the run GetReachStatus
reports.
Every run started, in the order they run: one, or one per upload of a merged dataset for the checks run per upload (VLM and REACH are one run). Read each with GetCheckRun.
checks/vlm with validate_only: what the run would read and reserve.
object
Credits the run would reserve: on a merged dataset, the sum over its uploads’ runs.
Episodes the model would read: linked to a published task, in a published version, with a camera.
Episodes of the selection it would leave out: not linked to a task, linked to one with no published revision, not in a published version, or without a camera.
The model it would read with: the request’s, else the template’s default.
Every model this deployment’s VLM check offers, the default first.
The durable request event, when correlated Activity writing is enabled.
Example generated
{ "checks": [ "example" ], "operation": "example", "checkRuns": [ "example" ], "estimate": {}, "activityEvent": "example"}default
Section titled “default ”Default error response
The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the API Design Guide.
object
The status code, which should be an enum value of [google.rpc.Code][google.rpc.Code].
A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the [google.rpc.Status.details][google.rpc.Status.details] field, or localized by the client.
A list of messages that carry the error details. There is a common set of message types for APIs to use.
Contains an arbitrary serialized message along with a @type that describes the type of the serialized message.
object
The type of the serialized message.
Example generated
{ "code": 1, "message": "example", "details": [ { "@type": "example" } ]}