Import and export

CyteArc reads common single-cell count formats, writes a Zarr store for analysis, and exports counts or metadata to interoperable formats. Most imports follow the same pattern: inspect when the source layout can vary, open a reader, then call a matching *ToZarr writer.

Export paths write Matrix Market or H5AD. CyteArc does not write Seurat .rds or .h5seurat files. See CyteArc for Scanpy users or CyteArc for Seurat users for ecosystem-specific workflow mapping.

Start with the Matrix Market example below. The later sections cover other input formats and show how to keep supplied analysis results or handle a larger import.

Download example datasets¶

CyteArc hosts example datasets in the public CyteBase bucket in formats such as MTX, 10x HDF5, and H5AD. Connect to the cytearc_docs repository to download them:

Naming format: <author>_<number of cells>_<cell/tissue type or species>_<single-cell method>

Each download returns the directory it wrote, which the readers below use as their input path.

This tutorial writes downloads and converted stores below one temporary directory. Replace output_dir with a persistent project directory in your own workflow.

Import Matrix Market¶

A Matrix Market import needs three matching files: the count matrix, feature identifiers, and cell barcodes. inspect_mtx finds complete triplets and reports their dimensions. If the source contains both raw and filtered matrices, compare the candidates and choose which cells to import; inspection does not make that choice for you.

A source can be an .mtx or .mtx.gz file, a directory, or a direct MEX ZIP. Inspection recognizes canonical 10x names, common prefixed triplets, and Parse DGE directories.

This directory contains one complete matrix. Open its reader, choose an output path, and call dump() to write the CyteArc store. If your inspection finds several candidates, choose the one whose cells and counts you want to analyse:

Open the store and inspect its cells and features:

The store is ready for analysis. Continue with Identify PBMC populations with scRNA-seq for an RNA workflow, or read on if you need another input format.

Parse DGE directories¶

Parse DGE matrices use cells by genes orientation and require cell_metadata.csv. CyteArc imports its non-ID columns. It recognizes bc_wells and bc_index; pass cell_id_key when both are present:

# Select the Matrix Market triplet in the Parse directory.
candidate = cytearc.inspect_mtx("/path/to/parse_dge")[0]

# Open a reader for the selected source format.
reader = cytearc.MtxReader(candidate, cell_id_key="bc_index")

# Write the converted count store.
cytearc.MtxToZarr(reader, zarr_loc="parse.zarr").dump()

Import H5AD¶

H5AD files vary in where they store counts, feature names, metadata, and layers. Inspect the file before conversion rather than assuming X, obs, and var contain the intended values.

H5adReader.from_inspect uses the discovered matrix and metadata keys. Override the inspection only after confirming that another layer contains the raw count matrix required by the analysis. This file also contains a UMAP and cell-type labels; other H5AD files may lack these results. Here we ask the reader to keep them with embedding_roles and cluster_keys, so we can plot and export them later. Omit these two arguments for a counts-and-metadata import.

embedding_roles and cluster_keys select analytical values for artifact import. The result maps their source names to exact refs in embeddingArtifacts and clusterArtifacts. These values are not flattened into live cell metadata. Load them through the datastore or pass their refs directly to consumers. Sparse, non-numeric, or row-mismatched selected embeddings are rejected.

For a file with several assay types, pass assay_split_key to the writer to split them; from_inspect alone writes one assay. The Import and export API reference covers multi-assay imports and metadata handling.

Import a larger 10x HDF5 dataset¶

CyteArc stores data as dense, compressed chunks in Zarr. CrH5Reader and CrToZarr convert Cell Ranger HDF5 into that layout. Assay type is inferred from the H5 feature types (RNA, ATAC, or multimodal). mem_budget bounds the memory the conversion plans for. The default count shards of this 89,796-peak assay do not fit mem_budget="8G", so the import passes the smaller policy that its MemoryError names. Non-RNA assays write no countsT copy, so the smaller shards do not slow later countsT reads.

Open the written store. The summary lists an ATAC assay, which confirms that inference from the H5 feature types survived the dump:

Import Seurat RDS¶

CyteArc can import a serialized Seurat object from an .rds file through inspect_seurat, SeuratReader, and SeuratToZarr. This path reads the on-disk RDS document. It does not attach to a live R session, and it does not read .h5seurat.

Inspect first. The result reports which assays and reductions are importable, their dimensions, and any blocking diagnostics or notices:

# Open count stores and run CyteArc analyses.
import cytearc

# Inspect the available count matrices and metadata.
inspection = cytearc.inspect_seurat("pbmc.rds")

# Compare the active assay with the importable assays and reductions.
{
    "active assay": inspection.activeAssay,
    "importable assays": [assay.name for assay in inspection.assays if assay.importable],
    "importable reductions": [r.name for r in inspection.reductions if r.importable],
}

Open a reader for the assays and reductions you want, then write the Zarr store. Omitting reductions selects every available reduction; SeuratToZarr raises if any selected reduction is not importable. Pass an empty sequence to skip reductions, or pass only importable names to import a subset. SeuratToZarr prepares and reads each selected count layer once when it is created, so the integral counts that Seurat holds as R doubles are stored unsigned, and each assay gets its own count dtype and layout:

# Open the requested Seurat assays and reductions.
with cytearc.SeuratReader(
    "pbmc.rds",
    assays=["RNA"],
    reductions=["pca"],
) as reader:

    # Import the selected Seurat assays and reductions.
    result = cytearc.SeuratToZarr(reader, zarr_loc="pbmc_from_seurat.zarr").dump()

# Open the count store for this analysis.
ds = cytearc.DataStore("pbmc_from_seurat.zarr")

# Inspect the imported assays, identities, and PCA result.
{
    "assays": result.assayNames,
    "default assay": result.defaultAssay,
    "notices": result.notices,
    "identities": result.activeIdentity,
    "PCA": result.reductionArtifacts["pca"],
}

activeIdentity is an exact cluster-label artifact. reductionArtifacts maps each requested Seurat reduction name to its exact imported-coordinate ref. Neither result is installed as a live analysis column. Pass the reduction ref to graph construction or load either payload explicitly.

What this import covers:

• Legacy Assay, Assay5, and ChromatinAssay count layers when their matrix layout is supported

• Literal cell metadata, plus artifact refs for active.ident and selected reductions such as PCA or LSI

• Partial Assay5 cell membership as per-assay boolean columns (<assay>_I) when the selected count layers hold only some cells

What it does not import as analysis artifacts:

• Neighbour graphs, Seurat neighbors objects, images, commands, and most tools slots

• Normalized layers when the selected count layer is used for the CyteArc assay

• Transposed Assay5 storage (Assay5T)

• A return path to .rds or .h5seurat (export H5AD or MTX instead)

Pass assay_layers when an assay stores several count layers and you need a non-default choice. A cell that only an unselected layer holds is recorded as not measured by that assay. Pass sidecar_path_remaps when a SaveSeuratRds sidecar cache points at moved on-disk matrices. Prefer original 10x HDF5 or Matrix Market counts when they are available and you only need raw matrices.

Large Assay5 assays can exceed the reader’s index memory budget (256 MiB by default). inspect_seurat then reports the assay as not importable with the metadata_index_limit diagnostic, and SeuratToZarr raises the matching SeuratImportError. Raise maxMetadataBytes through matrix_limits or select fewer count layers with assay_layers.

Export to Matrix Market¶

Open the H5AD-derived store written in the “Import H5AD” section and load the selected analytical artifacts:

Plot the imported layout colored by the imported cluster labels:

Export to H5AD and AnnData¶

DataStore.to_anndata builds an in-memory AnnData object from the selected counts and metadata. In this example, add the imported UMAP to obsm and cluster labels to obs, then write that object as H5AD. Those assignments affect the AnnData object and exported file; they do not change the CyteArc store.

For a completed pipeline, prefer ds.to_anndata(run=run). That form exports the run’s frozen assay, cell selection, feature selection, and metadata. It rejects live cell_key, feature_indexes, and feature_names overrides so the exported axes cannot drift from the run.

Attach exact imported artifacts¶

Reload the H5AD and confirm that the explicitly attached layout is in obsm:

Export a feature panel with to_anndata¶

Full-assay export can require enough memory and disk for the selected cell by feature matrix. When only a marker panel is needed, select features before materializing AnnData. Resolve the requested display names against the store once, then export only those columns:

Use feature_indexes instead when stable feature rows are already available. feature_names and feature_indexes are mutually exclusive, preserve the requested order, and reject duplicate or unknown selections.

This export contains the selected genes’ raw counts and live metadata for cells selected by I. Omitting the feature selection exports every gene in the assay, while the cell selection still applies. Saved analytical artifacts are not attached automatically in this form; add the needed values to the AnnData object as above.

Import CSV¶

CSVReader and CSVtoZarr provide small-data compatibility for dense CSV count matrices. The toy matrix below is synthesized in-notebook so the conversion does not depend on a catalog file. Rows are cells and columns are features; cell_data_cols moves selected columns into cell metadata.

Convert the CSV, keeping quality as cell metadata:

quality is cell metadata rather than a count column, which is what cell_data_cols is for.

Import sparse matrices¶

SparseToZarr accepts a SciPy CSR matrix with matching cell and feature IDs.

Write this small matrix with matching cell and feature identifiers:

For a complete DataStoreMerge example, continue to Integrating datasets by merging.

Other import paths¶

Chunked arrays¶

chunked_to_zarr writes from a CyteArc ChunkedArray when lazy out-of-core conversion is needed.

Remote Zarr destinations¶

Writers also accept remote Zarr locations. Choose the cloud profile for an object-store destination and pass credentials through the environment or runtime configuration:

# Read object-store credentials from the environment.
import os

# Inspect the available count matrices and metadata.
inspection = cytearc.inspect_h5ad("counts.h5ad")

# Open a reader for the selected source format.
reader = cytearc.H5adReader.from_inspect(inspection)

# Configure a writer for the remote destination.
writer = cytearc.H5adToZarr(
    reader,
    zarr_loc="s3://my-bucket/project/data.zarr",
    storage_options={
        "access_key_id": os.environ["AWS_ACCESS_KEY_ID"],
        "secret_access_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    profile="cloud",
)

# Write the converted count store.
writer.dump()

Count storage and memory¶

Writers choose the count dtype from the values, and RNA stores keep both cell-major counts and gene-major countsT arrays. Start with the default storage layout: if an import cannot fit its mem_budget, the error names a smaller CountMatrixPolicy that fits, as in the ATAC example above. Prefer a larger memory budget when your machine allows it, because smaller RNA shards slow later gene-major reads. See Memory and execution for memory planning and the Import and export API reference for reader buffering and storage options.

Common mistakes and limitations¶

• Give every store its own fresh directory outside other stores: Writers raise FileExistsError for a path holding data and ValueError for nested paths like data.zarr/RNA; overwrite=True replaces only an earlier conversion no DataStore has opened, so delete a prepared store yourself or choose another path.

• Inspect the source instead of assuming what it holds or that CyteArc can read it: Confirm which H5AD layer contains raw counts before conversion, re-import older RNA stores that lack countsT, and remember Seurat import covers counts, metadata, and reductions - not neighbour graphs, images, normalized layers, or .h5seurat.

• CyteArc moves exactly what you select. Stored count dtype follows the values rather than a reader argument; analytical H5AD values become artifacts only when you name embedding_roles or cluster_keys; embeddings must be dense, numeric, row-aligned obsm arrays; keep matrix="raw" unless the receiving method calls for normalized expression; and export a feature panel rather than materializing a full AnnData when a panel answers the question.