Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Run, monitor and rerun jobs

run() in Python and CyteTypeR() in R submit an annotation job and wait for its results. This page explains each step, how to follow a job while it runs, what happens when the client stops waiting, and how to fetch the results later or annotate the same data again.

What a run does

Python
R
adata = annotator.run(
    study_context="Peripheral blood mononuclear cells from a healthy human donor, 10x Genomics 3' scRNA-seq.",
    metadata={"title": "PBMC 3k"},
)

  1. Checks. The client looks for results already stored under the same results_prefix, finds your API key, and validates the study context, the metadata tags, the visibility and any llm_configs. A problem here stops the run before anything is uploaded.

  2. Upload. It uploads vars.h5 and obs.duckdb in chunks, several at a time (see Large datasets).

  3. Copy of the request. With save_query=True (the default), it writes the request to query.json.

  4. Submission. The server checks your API key, your usage limits and the requested visibility, and tests your own models if you passed any. If everything passes, it creates the job and returns its ID.

  5. Job details. The client records the job ID and the report link. Python stores them in adata.uns["cytetype_jobDetails"] straight away; CyteTypeR stores them in the object it returns at the end.

  6. Report link. The client prints the link to the job’s report.

  7. Waiting. It checks the job’s status every poll_interval_seconds (10 by default) and shows the progress.

  8. Results. When the job completes, it downloads the results, adds them to your object and returns the object.

Always keep the returned object: adata = annotator.run(...) in Python, obj <- CyteTypeR(obj, ...) in R (see Keep the returned object).

Follow a job

The job runs on CyteType’s servers. You can follow it in several places:

In your session, the client shows its progress while it waits (show_progress=True, the default):

Python
R
CyteType job submitted.
If your session disconnects, results can still be fetched later with:
`results = annotator.get_results()`

[TRACK PROGRESS]
https://cytetype.nygen.io/report/<job_id>
⠹ CyteType job running... 04:12 elapsed

In a terminal, the last line updates in place. In a Jupyter notebook, a single output line updates. When the output goes to a file, as in a batch job, a new line is printed only when the status changes. At the end, the client prints [DONE] CyteType job completed. (or [FAILED] CyteType job failed.).

show_progress=False turns this display off; messages such as the report link are still printed. In R, it also hides the upload progress bars.

Waiting and timeouts

The job does not depend on your session. The client only waits for it, and the job keeps running when the client stops waiting for any reason: a timeout, an interrupted call, a restarted notebook kernel, a computer that went to sleep or a lost connection.

Python
R
from cytetype import CyteTypeTimeoutError

try:
    adata = annotator.run(study_context="...", timeout_seconds=3600)
except CyteTypeTimeoutError:
    # The job is still running. Its ID is already stored in annotator.adata.uns.
    results = annotator.get_results()  # None until the job has completed

If the job fails

Python raises JobFailedError and prints the report link; CyteTypeR stops with Server error: Job failed. Failed jobs do not count toward your usage limits. Submit the job again (in Python, from the same object, with override_existing_results=True or another results_prefix, because the failed job’s details are already stored); if it keeps failing, contact support with the job ID, which is in the report link and the email.

A job can also complete with some clusters that have no result. Both clients still add the results to your object, mark the cells of those clusters as Unknown and log a warning that names the clusters. In R, these clusters have no row in the results table.

Fetch results later

Each job’s results stay on the server, so you can fetch them after the client stopped waiting, in the same session or a new one. Results in AnnData and Seurat describes what is returned.

Python
R

get_results() first returns any results stored in the object. If there are none but the object holds job details, it checks the job once: when the job has completed, it downloads the results, adds the columns as run() would (to annotator.adata) and returns them; while the job is still running, it returns None.

Same session. Call it on the same annotator:

results = annotator.get_results()
if results is None:
    print("The job is still running; try again later.")

New session. The job details are part of the AnnData, so when the client stops waiting, save the object with annotator.adata.write_h5ad(...). Later, reload it and create a CyteType object with the same group_key (and label_na, if you used it) so that the cluster IDs line up:

import anndata as ad
from cytetype import CyteType

adata = ad.read_h5ad("pbmc_submitted.h5ad")
annotator = CyteType(adata, group_key="leiden", rank_key="rank_genes_leiden")
results = annotator.get_results()
annotator.cleanup()

Creating CyteType prepares the data again, including the upload files, which cleanup() then deletes.

Only the job ID. If you lost the object, take the job ID from the report link, the dashboard or the email, and add it to the AnnData before you create CyteType:

adata.uns["cytetype_jobDetails"] = {"job_id": "<job_id>"}
annotator = CyteType(adata, group_key="leiden", rank_key="rank_genes_leiden")
results = annotator.get_results()

With another results_prefix, use the key f"{results_prefix}_jobDetails" and call get_results(results_prefix=...).

Rerun a job

You may want to annotate the same data again, for example with a more detailed study context or with your own LLM. Runs are not deterministic, so a rerun can also differ in wording or, for ambiguous clusters, in the call itself.

Python
R
adata = annotator.run(study_context="Human PBMC", results_prefix="cytetype")
adata = annotator.run(
    study_context="Peripheral blood mononuclear cells from a healthy human donor, 10x Genomics 3' scRNA-seq.",
    results_prefix="cytetype_v2",
)
annotator.cleanup()  # after the last run

Other run options

Every argument is described in the Python API reference, and in R with ?CyteTypeR. These are the ones that change how a run behaves.

require_artifacts

By default (True), a failure to build or upload vars.h5 or obs.duckdb stops the run. With require_artifacts=False, the job runs without the missing files: CyteType still annotates from the marker genes and cluster summaries, but evidence computed from your cells is missing, Chat has less data to work with, and target analysis is not available. In Python, a file that was uploaded is still used; CyteTypeR uses the files only when both uploads succeed. Errors with your API key or account (authentication and authorization errors) always stop the run.

Use the default unless an upload keeps failing for a reason you cannot fix, and report the error (see Troubleshooting).

save_query and query_filename

With save_query=True (the default), each run writes the request to query_filename (default query.json in the working directory), overwriting an earlier file of that name. The Python client never writes LLM API keys to it. In CyteTypeR the file also contains your llm_configs, including their keys, so pass save_query = FALSE whenever you use your own models (see Use your own LLM).

visibility

visibility sets who can open the report: "private" (the default), "org" or "public". Organization visibility needs an organization API key, issued under your organization’s license. The personal key that cytetype setup or SetupCyteTypeR() saves (cyt_p_...) is refused, even if your account belongs to an organization. Both clients warn before the upload when the key looks like a personal key, but the server checks the key only at submission, after the upload. You can also change the visibility in the dashboard at any time. See Visibility and sharing.

n_parallel_clusters

The number of clusters annotated at the same time, from 1 to 50 (default 2). It applies only when you use your own models; see Use your own LLM.

When a submission is refused

SituationPythonR
No API key, or a rejected keyAuthenticationErrorCyteType sign-in is required. Run SetupCyteTypeR() first., or the server’s message
Personal usage limit reachedRateLimitErrorError with the server’s message
Organization quota used upQuotaExceededErrorError with the server’s message
One of your own models failed the testLLMValidationErrorJob submission failed: [LLM_VALIDATION_FAILED] ...
visibility="org" with a key without an organization license, such as a personal API keyAPIError (code HTTP_ERROR)Job submission failed: [HTTP_ERROR] Org visibility requires an organization license

A missing or rejected API key is caught before the files are uploaded. The other checks run at submission, so when a job is refused for them, the upload time has already been spent, and the files are uploaded again when you resubmit. For visibility="org" with a key that looks like a personal key (cyt_p_...), both clients warn before the upload, so you can interrupt the run and change the key or the visibility. The message for a personal limit says which limit you reached and when it resets. See Usage limits and quotas for the limits, and Errors and error codes for every error.

Next steps

Prepare and run
Study context and metadata
Prepare and run
Use your own LLM