DataStore API reference#
DataStore is the primary analyst-facing object.
It inherits graph, mapping, and assay helpers from the classes below.
Use this page for analyst-facing methods and consult the inheritance appendix when extending CyteArc.
A DataStore is not designed for concurrent use from several threads; call its methods from one
thread at a time. Write a store from one process at a time.
Graph-construction methods are documented on Graph construction API reference.
Artifact and run inspection, including the metadata-only DataStore.summary(), is documented on Artifacts, lineage, and summaries API reference.
Mapping methods are on Mapping API reference, and integration metrics are on Integration and metrics API reference.
Those methods are excluded here rather than repeated.
summary is reserved for DataStore.summary() and cannot be used as an assay name.
Writers and DataStore opening reject that name before mutating store-level state.
Assays are also available as attributes such as ds.RNA; an assay named like a DataStore
attribute, such as cells, or starting with an underscore is available only through get_assay(name).
Opening also rejects the removed {assay}/state group. Rebuild such a store with the current
release; CyteArc does not read or migrate its former analysis state.
Analysis namespaces#
For a standard RNA analysis, start with ds.pipeline.run(label=...). Use the namespaces below
when selecting individual analysis steps. Each method keeps its scientific defaults, return
type, and saved-result identity. Cell metadata stays on ds.cells, and ds.get_assay(name)
provides access to every assay, including one whose name matches a namespace.
ds.artifacts#
find resolves one complete saved result. It searches computed results by default; use
ds.artifacts.find("embedding", imported="X_umap", from_assay="RNA") for an imported embedding.
See Artifacts, lineage, and summaries API reference for supported filters and ambiguity handling.
ds.qc#
- class cytearc.datastore.namespaces.QcAccessor(store)#
- auto_filter(attrs=None, min_p=0.01, max_p=0.99, *, cell_selection=None, artifact_metrics=None, invalidate_cache=False, method='mad', sample_column=None, sample_artifact=None, n_mads=3.0, min_cells_per_sample=20)#
Create an immutable automatically filtered cell selection.
Defaults to median absolute deviation (MAD) bounds. Counts and feature counts use log1p and two-sided bounds; mitochondrial and ribosomal percentages use upper bounds. Other metrics use two-sided raw bounds.
Requested columns are read from current cell metadata when this method is called. Exact quality-metric artifacts can be supplied alongside metadata metrics. Metadata values are fingerprinted and artifact references are stored in provenance.
MAD thresholds are pooled unless a sample source is supplied. Use
method="gaussian"for pooled median/std Gaussian quantiles instead.These are the pipeline filtering rules. A cell whose metric is recorded as missing never passes and does not inform the bounds; the fingerprints of metadata-column missing-value masks are recorded in provenance. Non-finite metrics among the remaining active cells, missing sample labels among active cells, and an empty result raise
ValueError.- Parameters:
attrs (
Iterable[str] |None) – Column names to be used for filtering.min_p (
float) – Quantile used for the lower threshold (Gaussian path only).max_p (
float) – Quantile used for the upper threshold (Gaussian path only).cell_selection (
ArtifactRef|None) – Optional prior cell-selection artifact.artifact_metrics (
Iterable[NamedCellArtifact] |None) – Named exactquality_metricartifact vectors.method (
Literal['mad','gaussian']) – MAD bounds by default, or explicit pooled Gaussian bounds.sample_column (
str|None) – Optional cell-metadata column with sample labels. When set, MAD bounds are calculated within each sample.sample_artifact (
NamedCellArtifact|None) – Optional named exacthto_identitysample vector.n_mads (
float) – Number of scaled MADs used for bounds.min_cells_per_sample (
int) – Groups with fewer active cells than this are retained without MAD filtering and emit a warning.
- Return type:
- Returns:
A complete datastore-scoped
cell_selectionartifact.- Raises:
UnmeasuredCellsError – If a metric’s assay did not measure a selected cell.
- doublets(clusters, graph, *, from_assay=None, cluster_sample_fraction=0.05, max_cells_per_cluster=100, simulation_ratio=1.0, heterotypic_fraction=0.8, save_k=5, smoothing_t=2, normalize_scores=True, random_seed=4444, invalidate_cache=False)#
Compute doublet scores from explicit cluster and graph artifacts.
- Raises:
ValueError – If
heterotypic_fractionis above 0 with one cluster.UnmeasuredCellsError – If the assay did not measure a cell of the graph.
- Return type:
- feature_percentage(cell_selection, features, *, invalidate_cache=False)#
Persist the percentage of counts assigned to selected features.
- Parameters:
cell_selection (
ArtifactRef) – Exact datastore-scoped cell-selection artifact.features (
ArtifactRef) – Exact assay-scoped feature-selection artifact.invalidate_cache (
bool) – Recompute instead of reusing an exact artifact.
- Return type:
- Returns:
A complete assay-scoped
quality_metricartifact.- Raises:
UnmeasuredCellsError – If the assay did not measure a selected cell.
- filter(attrs, lows, highs, *, cell_selection=None, keep_bounds=False, invalidate_cache=False)#
Create an immutable selection from metadata thresholds.
The requested columns are read from current cell metadata when this method is called. Their values are fingerprinted in provenance, and the resulting immutable selection is stored. The live
Icolumn is snapshotted whencell_selectionis omitted. Pass a prior selection explicitly to compose multiple filtering steps.Cells whose value is recorded as missing in a nullable column never pass its bounds. The fingerprint of such a column’s missing-value mask is also recorded in provenance.
- Parameters:
attrs (
Iterable[str]) – Names of distinct columns to be used for filteringlows (
Iterable[float|None]) – Lower bounds, in the same order asattrs. Each bound is a finite number, text for a text column, or None.highs (
Iterable[float|None]) – Upper bounds, in the same order asattrs. A lower bound cannot exceed its upper bound.cell_selection (
ArtifactRef|None) – Optional prior cell-selection artifact.keep_bounds (
bool) – Retain values exactly equal to a bound.
- Return type:
- Returns:
A complete datastore-scoped
cell_selectionartifact.- Raises:
TypeError – If a bound or
keep_boundshas an invalid type.ValueError – If a bound is invalid, the input selection is empty, or no cell passes the filter.
UnmeasuredCellsError – If a column’s assay did not measure a selected cell.
- hto_demultiplexing(cell_selection, *, from_assay=None, random_seed=0, invalidate_cache=False)#
Demultiplex HTO counts into an immutable identity artifact.
- Parameters:
from_assay (
str|None) – HTO assay name (default:'HTO').cell_selection (
ArtifactRef) – Explicit cell-selection artifact.random_seed (
int) – Seed used for HTO demultiplexing.invalidate_cache (
bool) – Recompute even when matching provenance exists.
- Return type:
- Returns:
A complete
hto_identityartifact.- Raises:
UnmeasuredCellsError – If the HTO assay did not measure a selected cell.
ds.features#
- class cytearc.datastore.namespaces.FeaturesAccessor(store)#
- detected(cell_selection, *, from_assay=None, min_cells=20, invalidate_cache=False)#
Select features detected in at least
min_cellsselected cells.- Raises:
UnmeasuredCellsError – If the assay did not measure a selected cell.
- Return type:
- hvgs(cell_selection, *, from_assay=None, min_cells=20, top_n=1000, min_var=-inf, max_var=inf, min_mean=-inf, max_mean=inf, n_bins=200, lowess_frac=0.1, blacklist='^MT-|^RPS|^RPL|^MRPS|^MRPL|^CCN|^HLA-|^H2-|^HIST|^XIST$|^DDX3Y$|^USP9Y$|^EIF1AY$|^KDM5D$|^SRY$|^ZFY$|^UTY$|^TMSB4Y$|^NLGN4Y$', keep_bounds=False, show_plot=True, max_cells=None, bin_strategy='adaptive', invalidate_cache=False, **plot_kwargs)#
Persist highly variable genes as an immutable feature selection.
Extra keyword arguments are options of
cytearc.plotting.highly_variable_features(), used whenshow_plotis True. Any other keyword raisesTypeErrorbefore anything is computed or saved.- Raises:
UnmeasuredCellsError – If the assay did not measure a selected cell.
- Return type:
- prevalent_peaks(cell_selection, *, from_assay=None, top_n=10000, invalidate_cache=False)#
Feature selection method for ATACassay type assays.
This method first calculates prevalence of each peak by computing sum of TF-IDF normalized values for each peak and then marks top_n peaks with the highest prevalence as prevalent peaks.
- Parameters:
from_assay (
str|None) – Assay to use for peak selection. If no value is provided then defaultAssay will be usedcell_selection (
ArtifactRef) – Explicit cells used to calculate peak prevalence.top_n (
int) – Number of top prevalent peaks to be selected, from 1 to one fewer than the number of peaks. (Default: 10000)
- Return type:
- Returns:
The persisted prevalent-peak feature-selection artifact.
- Raises:
UnmeasuredCellsError – If the assay did not measure a selected cell.
- snapshot(*, from_assay=None, mask=None, feature_indexes=None, invalidate_cache=False)#
Persist an explicit feature mask as an immutable artifact.
- Return type:
- universe(*, from_assay=None)#
Create or reuse the canonical all-true feature universe.
The universe is an artifact only. It is never mirrored into feature metadata or registered under a mutable label.
- Return type:
ds.reduction#
- class cytearc.datastore.namespaces.ReductionAccessor(store)#
ds.graph#
- class cytearc.datastore.namespaces.GraphAccessor(store)#
ds.embeddings#
- class cytearc.datastore.namespaces.EmbeddingsAccessor(store)#
- tsne(graph, initialization, *, symmetric_graph=False, graph_upper_only=False, tsne_dims=2, lambda_scale=1.0, max_iter=500, early_iter=200, alpha=10, box_h=0.7, verbose=True, invalidate_cache=False)#
Run SGtSNE-pi (Read more here: fcdimitr/sgtsnepi). This is an implementation of tSNE that runs directly on graph structures. We use connectivity graphs generated by graph-construction methods to create a layout of cells using tSNE algorithm. This function uses the
sgtsnepipackage. To get a better understanding of how the parameters affect the embedding, check this out: http://t-sne-pi.cs.duke.edu/- Parameters:
graph (
ArtifactRef) – Explicit connectivity map or integrated graph to embed.initialization (
ArtifactRef|ndarray) – Explicit initialization artifact or coordinate array. An array must hold finite real values.symmetric_graph (
bool) – This parameter is forwarded to graph.load and is same as there. (Default value: False)graph_upper_only (
bool) – This parameter is forwarded to graph.load and is same as there. (Default value: False)tsne_dims (
int) – Number of tSNE dimensions to compute (Default value: 2)lambda_scale (
float) – λ rescaling parameter (Default value: 1.0)max_iter (
int) – Maximum number of iterations (Default value: 500)early_iter (
int) – Number of early exaggeration iterations (Default value: 200)alpha (
int) – Early exaggeration multiplier (Default value: 10)box_h (
float) – Grid side length (accuracy control). Lower values might drastically slow down the algorithm (Default value: 0.7)verbose (
bool) – If True (default) then the full log from SGtSNEpi algorithm is shown. If False, the log is discarded and the process’s standard output and error are restored afterwards.
- Return type:
- Returns:
Reference to the immutable embedding artifact.
- Raises:
ImportError – If
sgtsnepicannot be imported for a new embedding.
- umap(graph, initialization, *, symmetric_graph=False, graph_upper_only=False, umap_dims=2, spread=2.0, min_dist=1, n_epochs=300, repulsion_strength=1.0, initial_alpha=1.0, negative_sample_rate=5, use_density_map=False, dens_lambda=2.0, dens_frac=0.3, dens_var_shift=0.1, random_seed=4444, parallel=False, nthreads=None, invalidate_cache=False)#
Build and return an immutable UMAP embedding artifact.
- Parameters:
graph (
ArtifactRef) – Explicit connectivity map or integrated graph to embed.initialization (
ArtifactRef|ndarray) – Explicit initialization artifact or coordinate array. An array must hold finite real values; UMAP optimizes a C-ordered float32 copy and leaves the caller’s array unchanged.symmetric_graph (
bool) – This parameter is forwarded to graph.load and is same as there. (Default value: False)graph_upper_only (
bool) – This parameter is forwarded to graph.load and is same as there. (Default value: False)umap_dims (
int) – Number of dimensions of UMAP embedding (Default value: 2)spread (
float) – Same as spread in UMAP package. The effective scale of embedded points. In combination withmin_distthis determines how clustered/clumped the embedded points are.min_dist (
float) – Same as min_dist in UMAP package. The effective minimum distance between embedded points. Smaller values will result in a more clustered/clumped embedding where nearby points on the manifold are drawn closer together, while larger values will result on a more even dispersal of points. The value should be set relative to thespreadvalue, which determines the scale at which embedded points will be spread out. (Default value: 1)n_epochs (
int) – Same as n_epochs in UMAP package. The number of epochs to be used in optimizing the low dimensional embedding. Larger values may result in more accurate embeddings. (Default value: 300)repulsion_strength (
float) – Same as repulsion_strength in UMAP package. Weighting applied to negative samples in low dimensional embedding optimization. Values higher than one will result in greater weight being given to negative samples. (Default value: 1.0)initial_alpha (
float) – Same as learning_rate in UMAP package. The initial learning rate for the embedding optimization. (Default value: 1.0)negative_sample_rate (
float) – Same as negative_sample_rate in UMAP package. The number of negative samples to select per positive sample in the optimization process. Increasing this value will result in greater repulsive force being applied, greater optimization cost, but slightly more accuracy. (Default value: 5)use_density_map (
bool) – If True, run densMAP (density-preserving UMAP) instead of standard UMAP.dens_lambda (
float) – densMAP density preservation strength (Default value: 2.0).dens_frac (
float) – Fraction of nearest neighbors used for local density estimation (Default value: 0.3).dens_var_shift (
float) – Variance shift for density correction (Default value: 0.1).random_seed (
int) – (Default value: 4444)parallel (
bool) – Whether to run UMAP in parallel mode. Setting value to True will use nthreads threads. The results are not reproducible in parallel mode. (Default value: False)nthreads (
int|None) – If parallel=True then this number of threads will be used to run UMAP. By default, the nthreads attribute of the class is used. (Default value: None)
- Return type:
- Returns:
Reference to the immutable embedding artifact.
- Raises:
ValueError – If densMAP finds a cell whose local radius is not finite.
ds.integration#
- class cytearc.datastore.namespaces.IntegrationAccessor(store)#
ds.clusters#
- class cytearc.datastore.namespaces.ClustersAccessor(store)#
- leiden(graph, *, resolution=1.0, backend='igraph', symmetric_graph=False, graph_upper_only=False, random_seed=4444, invalidate_cache=False)#
Build and return immutable Leiden cluster labels.
resolutionmust be a finite positive number and is recorded as a float, so1and1.0identify the same artifact.random_seedmust be a non-negative integer because stored labels are reused.- Return type:
- load_paris(ref)#
Load a Paris result from an explicit completed cut artifact.
- Return type:
- membership_strength(clusters, graph, *, invalidate_cache=False)#
Store per-cell cluster membership strength as an artifact.
For each cell, computes the fraction of KNN neighbors sharing its own cluster label.
- Parameters:
clusters (
ArtifactRef) – Explicit axis-aligned cluster-label artifact over the cell selection ofgraph, with a label for every cell. Labels that its linked missing mask flags raiseValueError.graph (
ArtifactRef) – Explicit connectivity-map or integrated-graph artifact.
- Return type:
- Returns:
Reference to the immutable membership-strength artifact.
- Raises:
ValueError – If
graphis not a connectivity map or integrated graph.PermissionError – If no matching result exists and the store is not opened with
zarr_mode='r+'.
- paris(graph, *, n_clusters='auto', min_cluster_size=None, invalidate_cache=False)#
Build and return an immutable Paris cut artifact.
A fixed integer
n_clustersmust be 1 or at least the number of connected components ingraph, and the hierarchy must split into exactly that many clusters at one height. Otherwise aValueErrornames the alternatives, such asn_clusters='auto'.- Return type:
- relabel_by_overlap(to_relabel, base_label, *, invalidate_cache=False)#
Relabel one cell-label artifact using another label artifact.
Values in artifact A are relabeled from their overlap with artifact B. For each unique value in A, the most frequently occurring value in B is found. If two or more values in A have maximum overlap with the same value in B, then they all get the same label as B along with different suffixes: ‘a’ to ‘z’, then ‘aa’, ‘ab’, and so on. The suffixes are ordered based on where the largest fraction of the B label lies. If one label from A takes up multiple labels from B then all the labels from B are included, and they are delimited by hyphens. Both artifacts need one cell selection and a label for every cell; labels that a linked missing mask flags raise
ValueError.- Parameters:
to_relabel (
ArtifactRef) – Explicit axis-aligned label artifact to relabel.base_label (
ArtifactRef) – Explicit axis-aligned base-label artifact.
- Return type:
- Returns:
Reference to the immutable relabeled-values artifact.
- Raises:
ValueError – If two labels of A would receive the same new name, which hyphen-joined base labels can cause.
PermissionError – If no matching result exists and the store is not opened with
zarr_mode='r+'.
ds.markers#
- class cytearc.datastore.namespaces.MarkersAccessor(store)#
- export_csv(marker, csv_filename, *, min_score=0.25, min_frac_exp=0.2)#
Export markers of each cluster/group to a CSV file where each column contains the marker names sorted by score (descending order, highest first). Columns follow the natural order of the group labels, as in
markers.load. This function does not export the scores of markers as they can be obtained using markers.load function.- Parameters:
marker (
ArtifactRef) – Exact marker-table artifact returned bymarkers.search.csv_filename (
str) – Required parameter. Name, with path, of CSV file where the marker table is to be saved.min_score (
float) – This value dictates how specific the feature value has to be in a group before it is considered a marker for that group. The value has to be greater than 0 but less than or equal to 1 (Default value: 0.25)min_frac_exp (
float) – Minimum fraction of cells in a group that must have a non-zero value for a gene to be considered a marker for that group.
- Raises:
ValueError – If the marker table records no
fold_change_policy.- Return type:
None
- load(marker, *, group_id=None, min_score=0.25, min_frac_exp=0.2)#
Return marker features from markers.search.
When
group_idisNone(default), markers for every group under the artifact are returned in one long table with agroup_idcolumn. Groups follow the natural order of their labels, as in plots: numeric labels come first by value, so'2'precedes'10', and other labels follow in natural text order. Pass a specificgroup_idto return markers for that group only. For a wide export of marker names only, usemarkers.export_csv.- Parameters:
marker (
ArtifactRef) – Exact marker-table artifact returned bymarkers.search.group_id (
str|int|None) – One stored group label, orNonefor all groups. An integer is matched by its string form.min_score (
float) – This value dictates how specific the feature value has to be in a group before it is considered a marker for that group. The value has to be greater than 0 but less than or equal to 1 (Default value: 0.25)min_frac_exp (
float) – Minimum fraction of cells in a group that must have a non-zero value for a gene to be considered a marker for that group.
- Return type:
DataFrame- Returns:
Pandas dataframe with marker statistics and a string
group_idcolumn.fold_changemay be +inf or NaN.- Raises:
ValueError – If
group_idis not a group of the marker artifact, or the table records nofold_change_policy.
- search(clusters, *, from_assay=None, features, nthreads=None, invalidate_cache=False, **norm_params)#
Persist marker tables for an explicit clustering artifact.
Raw counts of any storage dtype are ranked. Library-size markers require finite non-negative counts and cell totals.
- Parameters:
from_assay (
str|None) – Name of the assay to be used. If no value is provided then the default assay will be used.clusters (
ArtifactRef) – Completecluster_labelsorcluster_cutartifact with a label for every cell. Labels that its linked missing mask flags raiseValueError, as do labels that are blank,'.'or'..', or contain'/'or'\', because each label names its stored marker group. To compare the groups of a cell metadata column, or only some cells or clusters, freeze the labels over those cells withsnapshot_cluster_labels.features (
ArtifactRef) – Explicit feature-selection artifact.nthreads (
int|None) – Threads for marker search.**norm_params (
Any) – Extra keyword arguments forwarded tonormed.
- Return type:
- Returns:
A complete immutable marker-table artifact.
- Raises:
ValueError – If a library-size normalized value of a selected cell and feature, or the total of a selected cell (its
<assay>_nCountsvalue, or withrenormalize_subset=Trueits sum over the selected features), is negative or not finite. Nothing is written.PermissionError – If no matching result exists and the store is not opened with
zarr_mode='r+'. The check runs before the search.UnmeasuredCellsError – If the assay did not measure a labelled cell.
ds.scores#
- class cytearc.datastore.namespaces.ScoresAccessor(store)#
- aucell(net, cell_selection, *, from_assay=None, features, tmin=5, n_up=None, tie_seed=0, ambiguous_targets='drop', invalidate_cache=False)#
Score gene sets by recovery among each cell’s top-ranked RNA features.
AUCell ranks every feature selected by
featuresfrom raw counts. Network weights are ignored. Targets are matched without case sensitivity, then sources with fewer thantminmatched targets are removed.- Parameters:
net (
DataFrame) – Network withsourceandtargetcolumns.from_assay (
str|None) – RNA assay to score. The default assay is used when omitted.cell_selection (
ArtifactRef) – Explicit cells to score.features (
ArtifactRef) – Explicit feature-selection artifact.tmin (
int) – Minimum number of matched targets required per source.n_up (
int|None) – Number of top-ranked features used for recovery. When omitted, five percent of the ranking universe is used, clipped to its valid range.tie_seed (
int) – Seed for the global feature permutation used to resolve ties.ambiguous_targets (
Literal['drop','error']) – Handling of a target that matches several active features, such as a gene symbol shared by two feature ids."drop"removes its edges beforetminpruning, logs a warning, and records the target in the artifact’sdropped_ambiguous_targetsattribute."error"raises.
- Return type:
- Returns:
A complete
enrichment_scoresartifact.- Raises:
PermissionError – If the store is not opened with
zarr_mode='r+'.UnmeasuredCellsError – If the assay did not measure a selected cell.
Note
Cache identity covers selections, method parameters, and the prepared network, whose digest hashes only the retained edges. It assumes the stored count matrix is immutable.
- cell_cycle(cell_selection, *, from_assay=None, s_genes=None, g2m_genes=None, ctrl_size=None, log_transform=None, n_bins=50, rand_seed=4466, invalidate_cache=False)#
Computes S and G2M phase scores by taking into account the average expression of S and G2M phase genes respectively. Following steps are taken for each phase:
Normalized expression is log1p transformed when log_transform is True.
Genes are ranked into n_bins bins by mean expression across selected cells.
A control set of genes is identified by sampling genes from same expression bins where phase’s genes are present.
The average expression of phase genes (Ep) and control genes (Ec) is calculated per cell.
A phase score is calculated as
Ep - Ec.G1 is assigned when both scores are negative.
G2M is assigned when the G2M score exceeds the S score.
S is assigned otherwise, including tied non-negative scores.
- Parameters:
from_assay (
str|None) – Name of assay to be used. If no value is provided then the default assay will be used.cell_selection (
ArtifactRef) – Explicit cells to score.s_genes (
list[str] |None) – A list of S phase genes. If not provided then CyteArc loads pre-saved genes accessible at cytearc.quality_control.s_phase_genesg2m_genes (
list[str] |None) – A list of G2M phase genes. If not provided then CyteArc loads pre-saved genes accessible at cytearc.quality_control.g2m_phase_genesctrl_size (
int|None) – Controls sampled per bin. None uses the shorter input gene list.log_transform (
bool|None) – Apply log1p before binning and scoring. None applies it only fornorm_lib_size.n_bins (
int) – Number of bins into which average expression of genes is divided.rand_seed (
int) – A random seed for sampling control genes. (Default value: 4466)
- Return type:
- Returns:
A complete
cell_cycleartifact containing S, G2M, and phase values.- Raises:
UnmeasuredCellsError – If the assay did not measure a selected cell.
- load(enrichment, *, sources=None)#
Load an explicit enrichment artifact without materializing scores.
- Parameters:
enrichment (
ArtifactRef) – Artifact returned byscores.waggrorscores.aucell.sources (
Sequence[str] |None) – Optional source names to select and order.
- Return type:
- Returns:
The stored metadata and a lazy cells-by-sources score matrix.
- waggr(net, cell_selection, *, from_assay=None, features, mode='wmean', tmin=5, log_transform=False, ambiguous_targets='drop', invalidate_cache=False)#
Score weighted gene sets from streamed normalized RNA counts.
Targets are matched to active feature names without case sensitivity. Sources with fewer than
tminmatched non-zero edges are removed. Results are written to the assay’s enrichment group and returned lazily.- Parameters:
net (
DataFrame) – Network withsourceandtargetcolumns. An optionalweightcolumn supplies signed numeric edge weights. Missing weights default to one.from_assay (
str|None) – RNA assay to score. The default assay is used when omitted.cell_selection (
ArtifactRef) – Explicit cells to score.features (
ArtifactRef) – Explicit feature-selection artifact.mode (
Literal['wmean','wsum']) –"wmean"divides each weighted sum by the sum of absolute source weights."wsum"returns the weighted sum.tmin (
int) – Minimum number of matched targets required per source.log_transform (
bool) – Applylog1pafter library-size normalization.ambiguous_targets (
Literal['drop','error']) – Handling of a target that matches several active features, such as a gene symbol shared by two feature ids."drop"removes its edges beforetminpruning, logs a warning, and records the target in the artifact’sdropped_ambiguous_targetsattribute."error"raises.
- Return type:
- Returns:
A complete
enrichment_scoresartifact.- Raises:
PermissionError – If the store is not opened with
zarr_mode='r+'.UnmeasuredCellsError – If the assay did not measure a selected cell.
Note
Cache identity covers selections, method parameters, normalization, and the prepared network, whose digest hashes only the retained edges. It assumes the stored count matrix is immutable.
ds.imputation#
- class cytearc.datastore.namespaces.ImputationAccessor(store)#
- compute_imputed(feature_name, diffusion, *, from_assay=None)#
Impute feature values by diffusing along the KNN graph (MAGIC-style).
Each name is resolved in two steps. A name that exactly matches a live cell metadata column (case-sensitive) diffuses that column’s values. Any other name matches assay feature names case-insensitively, and the mean of all matching features is diffused when a name is not unique. A metadata column whose linked missing mask flags a cell in the operator’s cell selection raises
ValueError, because diffusion would spread its stored placeholder to neighbouring cells.- Parameters:
feature_name (
str|Sequence[str] |ndarray|Series) – One name, or a sequence, one-dimensional string array, or Series of names in the desired output order.diffusion (
ArtifactRef) – Explicit diffusion-operator artifact returned byimputation.diffusion.from_assay (
str|None) – Assay whose features are imputed. A native graph’s operator uses the graph’s own assay, which a given value must match. An integrated graph’s operator requires it unless every name is a cell metadata column.
- Return type:
ndarray- Returns:
A vector for one name, or a cells-by-features array for several names.
- Raises:
UnmeasuredCellsError – If the assay did not measure a cell of the operator.
- diffusion(graph, *, t=2, invalidate_cache=False)#
Compute and persist a sparse MAGIC diffusion operator.
The returned artifact is aligned to the graph’s exact stored cell selection. Repeating the same graph and diffusion power reuses the complete artifact unless
invalidate_cacheis true.- Raises:
MemoryError – A diffusion step, or loading the finished operator, would exceed the datastore memory budget. The check runs before anything is persisted.
- Return type:
- load_diffusion(diffusion)#
Load a validated diffusion-operator artifact as a COO matrix.
- Return type:
coo_matrix
ds.trajectory#
- class cytearc.datastore.namespaces.TrajectoryAccessor(store)#
- aggregation(pseudotime, *, features, min_exp=0.001, window_size=200, chunk_size=50, smoothen=True, z_scale=True, n_neighbours=11, n_clusters=10, batch_size=None, ann_params=None, nan_cluster_value=-1, invalidate_cache=False, **norm_params)#
Cluster features by pseudotime and store one immutable artifact.
- Parameters:
pseudotime (
ArtifactRef) – Explicit pseudotime artifact defining cells and ordering.features (
ArtifactRef) – Explicit assay-scoped feature-selection artifact.min_exp (
float) – Minimum mean normalized expression required for clustering.window_size (
int) – Rolling window size used to smooth feature values.chunk_size (
int) – Number of pseudotime bins to create.smoothen (
bool) – Whether to smooth expression along pseudotime.z_scale (
bool) – Whether to standardize each retained feature.n_neighbours (
int) – Number of neighbors in the feature graph.n_clusters (
int) – Number of feature modules to create.batch_size (
int|None) – Number of features processed per batch. When None, selected features are grouped into chunk-aligned blocks that fit the operation memory budget.ann_params (
dict|None) – Parameters forwarded to the HNSW index.nan_cluster_value (
int) – Value assigned to features excluded from clustering.**norm_params (
Any) – Extra keyword arguments forwarded to normalized expression.
- Return type:
- Returns:
Reference to the immutable pseudotime-aggregation artifact.
- Raises:
UnmeasuredCellsError – If the assay did not measure a valid pseudotime cell.
- fate(pseudotime, sink_labels, *, sinks=None, beta=10.0, solver_tol=1e-06, max_iterations=1000, invalidate_cache=False)#
Compute fate probabilities from explicit pseudotime and labels.
- Parameters:
pseudotime (
ArtifactRef) – Explicit pseudotime artifact. Its graph lineage is used.sink_labels (
ArtifactRef) – Explicit axis-aligned cell-label artifact.sinks (
list[Any] |None) – Ordered sink labels. Every matching selected cell becomes a fate boundary.beta (
float) – Strength of the penalty applied to backward graph edges.solver_tol (
float) – Absolute tolerance on the Bellman residual of each solved sink column. A solve stops once its largest residual over all cells is at mostsolver_tol, or half of its share of the validation limit when that is smaller, independent of sink size. Results are rejected when any residual exceeds10 * solver_tol * (len(sinks) - 1), bounded below by float32 precision and above by1e-3.max_iterations (
int) – Maximum number of preconditioned GMRES inner iterations per sink. GMRES restart is fixed at 20.
- Return type:
- Returns:
Reference to an artifact containing
probabilitiesandvalid.
- load_aggregation(ref)#
Load a lazy pseudotime aggregation from an explicit artifact.
- Return type:
- load_fate(ref)#
Load fate probabilities from an explicit completed artifact.
- Return type:
- load_markers(ref)#
Load a pseudotime-marker table from an explicit artifact.
- Return type:
- load_pseudotime(ref)#
Load pseudotime values from an explicit completed artifact.
- Return type:
- markers(pseudotime, *, features, min_cells=10, gene_batch_size=None, invalidate_cache=False, **norm_params)#
Store feature correlations with an explicit pseudotime artifact.
- Parameters:
pseudotime (
ArtifactRef) – Explicit pseudotime artifact defining cells and ordering.features (
ArtifactRef) – Explicit assay-scoped feature-selection artifact.min_cells (
int) – Minimum number of expressing cells required for a feature.gene_batch_size (
int|None) – Number of features loaded per batch. When None, selected features are grouped into chunk-aligned blocks that fit the operation memory budget.**norm_params (
Any) – Extra keyword arguments forwarded to normalized expression.
- Return type:
- Returns:
Reference to the immutable pseudotime-marker artifact.
- Raises:
UnmeasuredCellsError – If the assay did not measure a valid pseudotime cell.
- pseudotime(graph, *, n_singular_vals=30, source_sink=None, sources=None, sinks=None, ss_vec=None, min_max_norm_ptime=True, random_seed=4444, component_policy='largest', invalidate_cache=False)#
Calculate pseudotime and return an immutable cell artifact.
This is a reimplementation of the population balance analysis approach from Weinreb et al. 2018. The graph’s stored cell selection defines the output axis. Disconnected cells excluded by
component_policyremain represented with NaN values and a falsevalidentry.- Parameters:
graph (
ArtifactRef) – Explicit connectivity-map or integrated-graph artifact.n_singular_vals (
int) – Number of smallest singular modes requested, including the null mode.source_sink (
ArtifactRef|None) – Explicit axis-aligned label artifact. Required withsourcesorsinksand omitted whenss_vecis supplied.sources (
list[Any] |None) – A list of group/cluster ids fromsource_sinkto be treated as sources. Sources are usually progenitor/precursor or other actively dividing cell states.sinks (
list[Any] |None) – A list of group/cluster ids fromsource_sinkto be treated as sinks. Sinks are usually more differentiated (or terminally differentiated) cell states.ss_vec (
ndarray|None) – Custom source-sink value for each selected cell. It must sum to zero, with negative values for sources and positive values for sinks. This is mutually exclusive with label-based arguments.min_max_norm_ptime (
bool) – Whether to perform min-max normalization on the final pseudotime values so that values are in 0 to 1 range. (Default: True)random_seed (
int) – A random seed for svds (Default: 4444)component_policy (
Literal['largest','error']) – How to handle a disconnected selected graph.'largest'scores the largest connected component and marks other selected cells as unscored.'error'raises instead.
- Return type:
- Returns:
Reference to an artifact containing
pseudotimeandvalid.
ds.mapping#
- class cytearc.datastore.namespaces.MappingAccessor(store)#
ds.compare#
- class cytearc.datastore.namespaces.CompareAccessor(store)#
- compute_bulk(groups, *, from_assay=None, cell_selection=None, secondary_groups=None, aggr_type='mean', return_fraction=False, feature_label='index', remove_empty_features=True, pseudo_reps=1, null_vals=None, secondary_null_vals=None, random_seed=4466)#
Merge data from cells to create a bulk profile.
With
aggr_type='mean', cells are normalized with the assay’s normalization fitted once over every selected cell, then averaged within each group. RNA library-size normalization is per cell; ATAC document frequency and ADT CLR geometric means are therefore shared by all groups rather than learned separately for each group.Cells whose group or sub-group label is missing join no group, like cells whose value is in
null_vals. Missing labels areNaN,None, blank text, and values flagged by a linked missing mask.- Parameters:
groups (
ArtifactRef|str) – Explicit clustering artifact or user-owned metadata column used to group cells.from_assay (
str|None) – Name of assay to be used. If None andgroupsis an artifact, the artifact’s assay is used; otherwise the default assay is used.cell_selection (
ArtifactRef|None) – Explicit selection. Artifact grouping derives its selection from lineage, and this argument may only narrow it. For metadata-column grouping, None uses the liveIcolumn as the selection.secondary_groups (
ArtifactRef|str|None) – Optional clustering artifact or user-owned metadata column used to sub-group cells.aggr_type (
Literal['mean','sum']) – Type of aggregation to be used. Can be either ‘mean’ or ‘sum’. (Default value: ‘mean’)return_fraction (
bool) – Return the fraction of cells expressing a gene in each group. (Default value: False)feature_label (
Literal['index','id','name']) – The column in feature metadata table to use as row labels. (Default value: ‘index’)pseudo_reps (
int) – Within each group, randomly split cells into this many pseudo-replicates. Values greater than 1 produce descriptive resamples of the same cells, not independent biological replicates. (Default value: 1)remove_empty_features (
bool) – Remove features that are not expressed in any cell. (Default value: True)null_vals (
list[Any] |None) – Primary group values to skip.secondary_null_vals (
list[Any] |None) – Secondary group values to skip. These values will be skipped.random_seed (
int) – Seed used when assigning cells to pseudo-replicates.
- Return type:
DataFrame|tuple[DataFrame,DataFrame]- Returns:
A pandas dataframe containing the bulk profile. If return_fraction is True, then a tuple of two dataframes is returned. The second dataframe contains the fraction of cells expressing each feature in each group.
- Raises:
ValueError – If two groups produce the same column name, for example groups
'a_b'with sub-group'c'and'a'with sub-group'b_c'.UnmeasuredCellsError – If the assay did not measure a cell that is read.
- load_tests(artifact)#
Return one exact immutable statistical-test result.
- Return type:
- test(keys, grouping, *, cell_selection=None, groups=None, comparisons=None, test='auto', posthoc=None, adjustment='fdr_bh', alternative='two-sided', sample_by=None, study_design=None, pair_by=None, sample_stat='mean', expression_cutoff=0.0, subset_by=None, from_assay=None, normalization=None, skip_save=False, invalidate_cache=False)#
Run statistical tests on values grouped by an explicit source.
This mirrors the inputs of
distributionso results can be compared directly against violin or box plots.keysmay be cell-metadata columns or feature names. The chosen test follows the single-cell conventions for zero-inflated, non-normal values:"mann_whitney": two independent groups, two-sided. When the two groups can be formed in at most 100,000 ways, as in typical sample-level designs, p-values come from the exact permutation null with ties handled exactly. Larger designs use the tie- and continuity-corrected normal approximation of the marker search.result.p_value_methodrecords"exact"or"asymptotic"."kruskal_wallis": three or more groups, with optionalposthoc="dunn"for pairwise significance."wilcoxon": paired samples on aggregated (pseudobulk) data. Requiressample_byandpair_by."welch"(alias"t_test"): cell-level Welch’s t-test on raw normalized values for exactly two groups, honouringalternative. Descriptive only; no sample aggregation."one_way_anova": cell-level one-way ANOVA omnibus test on raw normalized values. Descriptive only; no post-hoc yet.
With
test="auto"the test is chosen from the design: paired data uses Wilcoxon, two groups use Mann-Whitney, and three or more use Kruskal-Wallis. Auto never picks a parametric method.groupsrestricts the group set and fixes its order (which sets the contrast direction);comparisonsrestricts pairwise rows to the listed group pairs. Withposthoc="dunn"both the omnibus Kruskal-Wallis and the pairwise Dunn’s results are preserved. When multiple keys are tested,adjustmentcorrects p-values across keys in one pooled pass (default"fdr_bh"); post-hoc p-values are corrected separately.A
study_designwithsubject_by(orpair_by) pairs samples only for the Wilcoxon test: explicitly, or withtest="auto"when two conditions remain. Repeated measures across three or more conditions, subjects nested within conditions, and independent tests raise an error. For nested subjects, pass the subject column assample_byinstead.Results are persisted as immutable artifacts unless
skip_saveisTrue. Writing requireszarr_mode='r+'; a matching saved result is still reused from a read-only store. Passresult.artifacttocompare.load_testsfor exact retrieval.- Parameters:
keys (
str|CellField|FeatureRef|Sequence[str|CellField|FeatureRef]) – Feature names or cell-metadata columns to test.grouping (
ArtifactRef|CellField) – Exact categorical artifact or explicit cell metadata field.cell_selection (
ArtifactRef|None) – Optional frozen selection for a metadata field, or a subset of an artifact grouping’s stored selection.groups (
Sequence[Any] |None) – Keep and order only these grouping categories.comparisons (
Sequence[tuple[Any,Any]] |None) – Restrict pairwise comparisons to these group pairs.test (
Literal['auto','mann_whitney','kruskal_wallis','wilcoxon','welch','t_test','one_way_anova']) – Statistical test, or"auto"to pick from the design.posthoc (
Optional[Literal['dunn']]) – Pairwise test to run after Kruskal-Wallis ("dunn").adjustment (
Literal['fdr_bh','bonferroni','holm','none']) – Multiple-testing correction across keys and rows.alternative (
Literal['two-sided','less','greater']) – Direction of the alternative hypothesis. Only the Welch t-test honours it; other tests remain two-sided.sample_by (
str|None) – Cell metadata column identifying biological samples.study_design (
StudyDesign|None) – Study design supplyingsample_byand, for the Wilcoxon test only, the pairing column.pair_by (
str|None) – Cell metadata column identifying subjects or donors for paired tests.sample_stat (
Literal['mean','median','fraction']) – Aggregation across cells within a sample.expression_cutoff (
float) – Detection cutoff forsample_stat="fraction".subset_by (
str|None) – Boolean metadata column keeping onlyTruecells.from_assay (
str|None) – Assay to read feature values from.normalization (
NormalizationSpec|None) – How feature values are read.skip_save (
bool) – Return results without writing to Zarr.invalidate_cache (
bool) – Recompute even when a matching artifact exists.
- Return type:
- Returns:
- Raises:
PermissionError – If a result must be written to a store that is not opened with
zarr_mode='r+'. The check runs before any value is computed.UnmeasuredCellsError – If a feature’s assay did not measure a tested cell.
Updating existing calls#
Replace the method name using this table. Arguments, defaults, return values, and saved
artifacts stay the same. The previous names are no longer methods on DataStore. Saved
provenance retains its operation names, such as run_umap.
Previous call |
Current call |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Store-bound plotting#
DataStore.plots binds the datastore argument for canonical store-first functions in cytearc.plotting.
Array and DataFrame diagnostics such as elbow, qc, graph_qc, and highly_variable_features remain standalone.
- class cytearc.datastore.plot_accessor.DataStorePlotAccessor(store)#
Datastore-bound facade for store-first plotting functions.
- cluster_connectivity(*, group_by=None, layout_key=None, groups=None, layout=None, graph, cell_key='I', position='median', positions=None, categorical_scale=None, size_scale=None, minimum_edge_weight=0.02, max_edges_per_node=None, show_cells=False, cell_size=None, cell_alpha=0.3, cell_color=None, node_edgecolor=None, node_linewidth=0.8, edge_color=None, edge_alpha=0.45, edge_width_range=(0.4, 5.0), labels=True, target=None, figsize=None, theme='notebook', show=True)#
Summarize cell graph connectivity between embedding clusters.
- Return type:
- cluster_tree(*, graph, clusters, from_assay=None, fill_by_value=None, force_ints_as_cats=True, width=1, lvr_factor=0.5, vert_gap=0.2, min_node_size=10, node_size_multiplier=10000.0, node_power=1.2, root_size=100, non_leaf_size=10, show_labels=True, fontsize=10, root_color='#C0C0C0', non_leaf_color='k', cmap='tab20', color_key=None, edgecolors='k', edgewidth=1, alpha=0.7, figsize=(5, 5), ax=None, theme='notebook', show=True)#
Plot a stored hierarchical clustering tree.
- Return type:
- composition(*, category_by=None, categories=None, cell_key='I', sample_by=None, grouping=None, subject_by=None, pair_by=None, condition_by=None, study_design=None, kind='stacked', show_summary=True, uncertainty=None, categorical_scale=None, bar_width=0.82, bar_gap=0.12, segment_edgecolor=None, segment_linewidth=0.5, show_percent_labels=False, label_min_fraction=0.08, percent_format='{:.0%}', target=None, figsize=None, max_figure_width=7.5, theme='notebook', show_legend=True, show=True)#
Plot category composition for the selected cells.
- Return type:
- distribution(keys, *, grouping=None, cell_selection=None, groups=None, split_by=None, sample_by=None, study_design=None, sample_stat='mean', expression_cutoff=0.0, subset_by=None, from_assay=None, normalization=None, categorical_scale=None, split_scale=None, kind='violin', bins=40, max_points=10000, point_size=0.8, point_alpha=0.28, seed=0, color='steelblue', color_by='group', color_scale=None, orientation='vertical', row_standardize=False, share_y=None, violin_inner='quartile', violin_linewidth=0.8, violin_alpha=0.9, italicize_features=False, target=None, figsize=None, max_figure_width=7.5, title=None, theme='notebook', show_legend=True, stats_results=None, stats_keys=None, stats_bracket_height=None, stats_show_p=True, show=True)#
Plot distributions of cell metadata or feature values.
stats_resultsoverlays significance brackets fromcompare.testresults onto the drawn violins or boxes; seecytearc.plotting.distribution()for the full behaviour.max_pointsdefaults to10000; explicitNonedisables the point overlay for stacked violins and otherwise uses10000.- Return type:
- dotplot(*, features, group_by=None, groups=None, cell_key='I', from_assay=None, sample_by=None, study_design=None, normalization=None, expression_cutoff=0.0, standardize='none', color_scale=None, size_scale=None, categorical_scale=None, group_order=None, feature_order=None, swap_axes=False, marker_edgecolor=None, marker_linewidth=0.3, label_wrap=None, italicize_features=False, target=None, figsize=None, max_figure_width=7.5, theme='notebook', show_legend=True, show=True)#
Summarize feature expression as a dot plot.
- Return type:
- embedding(*, layout_key=None, layout=None, run=None, color_by=None, facet_by=None, facet_order=None, cell_key='I', from_assay=None, normalization=None, point_size=None, point_sizes=None, point_size_range=(1.0, 28.0), point_edgecolor=None, point_edgewidth=None, point_alpha=1.0, sort_values=False, color_scale=None, categorical_scale=None, default_color='steelblue', missing_color=None, clip_fraction=0.0, subset_by=None, groups=None, n_columns=None, target=None, figsize=None, theme='notebook', legend_loc='auto', max_on_data_labels=40, show_legend=True, show_titles=True, frame='minimal', density_overlay=None, highlight=None, seed=None, rasterize_threshold=50000, show=True)#
Plot cells in a stored two-dimensional embedding.
- Return type:
- embedding_raster(*, layout_key=None, layout=None, run=None, color_by=None, cell_key='I', pixels=400, block_rows=None, color_scale=None, missing_color=None, subset_by=None, target=None, figsize=None, theme='notebook', seed=0, show=True)#
Rasterize continuous cell metadata over a stored embedding.
- Return type:
- mapping_calibration(transfer, *, known_labels, metric='voteFraction', direction='auto', thresholds=None, n_thresholds=50, chosen_threshold=None, target=None, figsize=None, theme='notebook', show=True)#
Plot held-out label accuracy against retained coverage per threshold.
- Return type:
- mapping_confusion(transfer, *, known_labels, normalize='true', known_order=None, predicted_order=None, abstention_label='Abstained', color_scale=None, target=None, figsize=None, theme='notebook', show_legend=True, show=True)#
Plot known query labels against the labels of one saved transfer.
- Return type:
- mapping_evidence(transfer, *, target_groups=None, metrics=('voteFraction', 'topTwoMargin', 'voteEntropy', 'referenceDistancePercentile'), kind='histogram', bins=30, categorical_scale=None, target=None, figsize=None, theme='notebook', show_legend=True, show=True)#
Plot the saved evidence of one label transfer for query groups.
- Return type:
- mapping_score(result, *, reference, target_groups=None, layout=None, kind='embedding', reference_labels=None, size_by_score=False, log_transform=True, multiplier=1000, weighted=True, fixed_weight=0.1, bins=40, point_size=None, color_scale=None, categorical_scale=None, target=None, figsize=None, theme='notebook', show_legend=True, show=True)#
Plot reference-cell mapping scores for one or more query groups.
- Return type:
- marker_heatmap(*, marker, topn=5, log_transform=None, vmin=-1, vmax=2, figsize=None, fontsize=10, width_factor=0.03, height_factor=0.02, cmap='magma_r', color_scale=None, row_order=None, column_order=None, cluster_rows=True, cluster_columns=True, cluster_method='ward', cluster_metric='euclidean', row_annotations=None, column_annotations=None, annotation_scales=None, target=None, theme='notebook', show_legend=True, show=True, **heatmap_kwargs)#
Plot the stored marker table as a heatmap.
- Return type:
- matrixplot(*, features, group_by=None, groups=None, cell_key='I', from_assay=None, sample_by=None, study_design=None, normalization=None, expression_cutoff=0.0, value='mean', standardize='none', color_scale=None, feature_order=None, group_order=None, cluster_features=False, cluster_groups=False, cluster_method='average', cluster_metric='euclidean', row_annotations=None, column_annotations=None, annotation_scales=None, target=None, figsize=None, theme='notebook', show_legend=True, show=True)#
Summarize feature expression as a matrix plot.
- Return type:
- modality_weights(*, graph, layout, point_size=None, point_alpha=1.0, cmap='viridis', n_columns=None, target=None, figsize=None, theme='notebook', frame='minimal', rasterize_threshold=50000, show=True)#
Plot each assay’s WNN contribution over an explicit embedding.
- Return type:
- pseudotime_heatmap(*, aggregation, show_features=None, feature_order=None, feature_cluster_order=None, figsize=(5, 10), vmin=-2.0, vmax=2.0, heatmap_cmap=None, pseudotime_cmap=None, clusterbar_cmap=None, color_scale=None, feature_cluster_scale=None, pseudotime_scale=None, tick_fontsize=10, axis_fontsize=12, feature_label_fontsize=12, target=None, theme='notebook', show_legend=True, show=True)#
Plot feature profiles ordered by stored pseudotime.
- Return type:
- run_recipe(recipe, *, artifacts=None, targets=None, output_dir=None, show=False, continue_on_error=False)#
Run a declarative plotting recipe against this datastore.
- Return type:
Selected analysis contracts#
Feature producers return immutable ArtifactRef values. Use
features.resolve(assay, ref) for strict read-only validation.
features.universe(from_assay=...) creates or reuses the canonical immutable all-feature
selection for granular workflows. It does not write a live feature metadata column.
snapshot_cell_selection(cell_key) captures the explicit immutable cell input for granular graph
construction. features.normalize(cell_selection, features) requires both exact refs. It saves the
values of the assay’s normed with its configured normMethod. A log_transform or
renormalize_subset left as None is True for the RNA library-size normalizers that apply it, and
False for every other normalizer, whose scale CyteArc cannot know; a flag that the normalizer cannot
apply raises ValueError when True (Assays and metadata API reference).
Direct marker, WAGGR, AUCell, and pseudotime feature analyses likewise require exact refs.
Graph-derived methods require a graph ref and project feature selections through its named lineage edges.
They do not accept a separate feature selection.
cells = ds.snapshot_cell_selection("I")
features = ds.features.universe(from_assay="RNA")
normalized = ds.features.normalize(cells, features)
diffusion = ds.imputation.diffusion(graph_ref, t=2)
imputed = ds.imputation.compute_imputed("CD4", diffusion)
membership = ds.clusters.membership_strength(cluster_ref, graph_ref)
doublets = ds.qc.doublets(cluster_ref, graph_ref)
imputation.compute_imputed starts with the feature name and aligns its returned array to the cell selection in
the explicit diffusion-operator lineage. It accepts one name, a sequence of names, a
one-dimensional string array, or a Series. A name that exactly matches a live cell metadata
column, including case, diffuses that column; it takes precedence over an assay feature of the same
name. Other names match assay feature names case-insensitively, and duplicates are averaged.
Metadata columns need no assay, so from_assay is required only for feature names from an
integrated graph’s operator; a native graph’s operator uses its own assay. A
metadata column with a missing value in the operator’s cell selection raises ValueError, because
diffusion would spread its stored placeholder to neighbouring cells.
imputation.diffusion raises MemoryError before persisting an operator that could not be
formed or loaded again within the datastore memory budget. imputation.load_diffusion(ref) exposes
the validated sparse operator for direct matrix work. Membership strength and doublet detection consume exact cluster
and graph refs and return artifacts.
compare.compute_bulk(groups, ...) accepts a categorical cell artifact or a user-owned metadata column name.
Artifact inputs derive their cell selection from lineage; cell_selection= can restrict that
selection explicitly. A metadata column without cell_selection= snapshots the live I column.
secondary_groups= provides an optional nested grouping without writing artifact labels to a cell
column. Mean profiles fit the assay normalization once over every selected cell, so ATAC document
frequency and ADT CLR geometric means are shared by all groups. Each mean averages the group’s rows
of the assay’s normed values, so a cell without counts adds zeros and still counts toward its
group’s size. A mean or floating-point sum that is not finite, such as one from a custom
normalization that returns NaN, raises ValueError naming the group and the feature index. RNA
means also raise ValueError when a selected cell’s <assay>_nCounts total is negative or not
finite. Cells whose group or sub-group label is recorded as missing join no group, as values in
null_vals do. Group values that would produce the same column name raise an error.
add_grouped_assay(groups, assay_label=...) similarly accepts a
pseudotime-aggregation ref or an explicit feature metadata column when constructing a new assay.
An aggregation ref groups only the features it clustered, and missing metadata values never form a
group. The source normalization is fitted on the cells that the source assay measured, and other
cells get zero means.
add_grouped_assay and add_melded_assay make the new assay visible only after its counts are
complete. A write that fails with an error removes the partial assay. A pending assay stays when
the write is interrupted, as by KeyboardInterrupt, when the process is killed, or when the final
write that publishes the assay fails, because that write is never undone. Opening ignores a pending
assay, repacking refuses it, and it blocks its name in every workspace; after confirming that no
process is writing it, remove it with discard_interrupted_assay(assay_label).
The new assay has a row for every cell and measures the cells that its source assay measured. When
the source records them in its membership column <source>_I, the new assay’s <assay_label>_I is
a copy, written with the assay and removed with a failed or discarded one; a source without the
column gives an assay without one, which measures every cell (Assay membership). A cell
column that already has the name <assay_label>_I makes both methods raise ValueError before
they compute anything.
cytearc.datastore.namespaces.QcAccessor.auto_filter() defaults to method="mad"
over the pooled selection. Supplying sample_column estimates MAD bounds separately per sample.
n_mads controls the bounds; groups with fewer than min_cells_per_sample=20 active cells
are retained with a warning, including small pooled selections. Pass method="gaussian"
explicitly to use the former pooled Gaussian policy and configure its min_p and max_p
quantiles. Changing these probabilities with either MAD path raises an error.
qc.filter and qc.auto_filter apply the same rules as pipeline filtering. A cell whose
metric is recorded as missing in a nullable column or artifact never passes a filter and does not
inform automatic bounds. MAD filtering rejects missing sample labels among active cells. Automatic
bounds reject non-finite metric values, such as the undefined percentages of zero-count cells, and
every filter raises when no cell remains.
cytearc.datastore.datastore.DataStore.select_cells() thresholds the numeric values of an
exact cell-aligned artifact, or retains categorical values with include=[...], and composes the
result with its stored source selection. It reads the kind’s canonical array
(Read per-cell values), such as values of a quality metric, phase of a cell-cycle
artifact, labels of a Paris cut, or pseudotime of a pseudotime artifact, and refuses a kind
that is not cell-aligned. An explicit
cell_selection= may narrow, but never widen, that source selection. Cells whose value the
artifact records as missing are never selected, and a selection that retains no cell raises.
Doublet detection, markers.search, clusters.membership_strength, and clusters.relabel_by_overlap reject label
artifacts with missing labels before reusing or writing a result. For markers.search, select
the labelled cells with select_cells(labels, include=[...]) and freeze their labels with
snapshot_cluster_labels(labels, cell_selection=...). For clusters.relabel_by_overlap, freeze both label
artifacts over one selection of the cells labelled in both. Doublet detection and
clusters.membership_strength need a label for every cell of the graph. Build the graph over labelled
cells only when some of its cells have no label, then freeze the labels over the cell selection
that the graph was built from, such as run["analysis_cell_selection"] of a pipeline run.
cytearc.datastore.datastore.DataStore.snapshot_cluster_labels() freezes one label for each
cell of an explicit cell_selection into a datastore-scoped cluster_labels artifact. Label
consumers accept it like a clustering, among them markers.search, compare.compute_bulk, select_cells,
compare.test, clusters.relabel_by_overlap, and clusters.compute_concordance. The labels come from a
cell metadata column, such as an annotation or a condition, or from a cell-label artifact, such as
a clustering, a Paris cut, or imported clusters. An artifact is read for a subset of its own cell
selection, so its labels can be narrowed to the labelled cells or to some of its clusters.
Floating-point labels that are whole numbers within the int64 range, such as the float64 ids that
pandas writes for integer ids with missing values, are stored as int64, and other floating-point
labels raise TypeError. Every selected cell needs a label: a missing label, including a row that
a linked missing mask flags, or a blank label raises ValueError. Text is stored at the width of
the selected labels, and integer and boolean labels keep their dtype. The identity holds the source
column name or source artifact, the cell selection, and a fingerprint of the stored labels, so the
same labels reuse one artifact, also from a read-only store, and changed labels create a new one
while earlier snapshots keep their values. On a mounted store the
artifact is written to the target. The labels are not checked against the marker-group naming
rule; markers.search rejects a label that cannot name a stored marker group before it writes
anything.
For example, rank the genes that separate disease from normal cells within one annotated cell type of a mounted store:
ds = cytearc.mount_datastore(source_path, at=target_path, default_assay="RNA")
cell_type = np.asarray(ds.cells.fetch_all("cell_type"), dtype=object)
ds.cells.insert("is_t_cell", cell_type == "T cell", overwrite=True)
t_cells = ds.snapshot_cell_selection("is_t_cell")
disease = ds.snapshot_cluster_labels("disease", cell_selection=t_cells)
screen = ds.markers.search(disease, features=ds.features.universe(from_assay="RNA"))
ds.markers.load(screen, group_id="COVID-19")
This screen is descriptive: it treats every cell as an independent observation, although cells of
one donor are not. To test a condition, aggregate cells to donors, for example with
compare.test(genes, CellField("disease"), cell_selection=t_cells, sample_by="donor_id"),
as in Compare biological conditions across donors.
Nullable metadata columns keep a stored placeholder in each row that their linked missing mask
flags. cells.fetch and cells.fetch_all return stored values, placeholders included.
cells.to_pandas_dataframe, cells.head, get_cell_vals, plots, and exports show those rows as
missing: numeric columns become float64 with NaN, and other columns hold a missing value.
sift and multi_sift never select them. imputation.compute_imputed and cytearc.metrics.silhouette_scoring
reject a live column with a missing value among the cells they use.
cytearc.metadata.MetaData.insert() writes such a column. Values given only for the rows that
key selects ("I" by default) leave the other rows missing, holding empty text, NaN, 0,
False, or NaT; values for every row keep every row. None, NaN, pd.NA, and NaT among
object, categorical, nullable, or string values are missing too, and those values are typed as
imports type them: bool, int64, float64, complex128, or else text; object integers outside the int64
range raise ValueError. An explicit fill_value is stored as a real value without a mask and must
fit the values: text for text, which widens the column, a bool for a Boolean column, an integer
within range for integers, a real number that is not a bool for floating-point values, and a
datetime without a time zone or a timedelta that the unit of datetime or timedelta values holds
exactly; anything else raises ValueError before the store changes. I, ids, and names never
hold a missing value. A boolean column with masked rows selects the same cells as one filled with
False, so selections snapshotted from it keep their identity, while metadata snapshots and
filters over it record its mask. Replacing a column reads the previous column, its attributes, and
its mask first; if writing the new column fails or is interrupted, the previous one is written back
before the error propagates. When that also fails, a note on the error names the column.
The cell table reserves the membership column <assay>_I of every assay of the store, which only
imports, merges, and derived assays write (Assay membership). insert, with or without
overwrite, update_key, and reset_key raise ValueError for that name whether or not the
column exists, and drop raises for a column that carries the membership role, because a store
without it counts every cell as measured. A plain column of that name without the role can be
dropped.
A cell that an assay did not measure holds zero counts of the assay, which are no measurement, so
an operation that reads the assay’s values refuses it. Normalization, integration.modalities, HVG and
detected-feature selection, WAGGR, AUCell, marker search, compare.compute_bulk, the feature keys of
compare.test, prevalent peaks, cell-cycle scoring, feature percentages, HTO
demultiplexing, doublet detection, pseudotime markers and aggregation, the feature names of
imputation.compute_imputed, mapping.run of the query assay, to_anndata(matrix="normed") without a run, and
pipeline.run check their cells against the membership column after their own argument and
assay-type checks and before they reuse or write anything, on a read-only store too, and raise
UnmeasuredCellsError, a ValueError that names the
operation, the assay, the column, and how many of the selected cells the assay did not measure.
compare.compute_bulk checks the cells whose counts it reads: the cells of its bulk columns, or every selected
cell for a mean that fits the assay’s normalization over them, as ATAC and ADT means do; for a
metadata grouping without cell_selection, it finds them from the live I column before it
snapshots I. Quality control reads the assay of each metric that it filters on:
qc.auto_filter, qc.filter on such a column, and the filtering stage of pipeline.run
refuse unmeasured cells too. A cell
column is a metric of the assay whose preparation wrote it, <assay>_nCounts,
<assay>_nFeatures, or a percentage column that the assay records in its percentFeatures
attribute, such as RNA_percentMito; a quality_metric artifact of an assay is a metric of that
assay; other columns read no assay. Operations that read only other results or metadata (such as
PCA, graphs, clustering, and snapshot_cluster_labels) and derived assays do not check. Exports
that declare membership, to_h5ad and to_anndata without layers, keep unmeasured cells; to_mtx
and a to_anndata layer of another assay cannot declare it and refuse them.
select_measured_cells() keeps the measured cells of
a selection: pass its result as cell_selection, which compare.compute_bulk, compare.test,
and the QC filters also take directly, build graphs over it, or freeze labels over it with
snapshot_cluster_labels(labels, cell_selection=...). A pipeline run and normalized export take a
cell_key column instead: one True only for the cells of I that the assay measured, such as
ds.cells.insert("RNA_measured", ds.cells.fetch_all("I") & ds.cells.fetch_all("RNA_I")); the
membership column alone also holds cells that a filter removed from I. select_measured_cells
returns the input selection itself when the assay measured every selected cell, so fully measured
stores keep their identities. Plots and get_cell_vals show the feature values of unmeasured cells
as missing, NaN, instead of zero, and normalize only the measured cells, so that a normalizer
fitted over the cells it reads, such as ATAC TF-IDF, gives a measured cell the value of a read of
the measured cells alone. Opening saved results does not rerun membership checks; a pipeline
configuration that selects unmeasured cells raises before any stage executes.
- exception cytearc.metadata.membership.UnmeasuredCellsError(operation, assay, column, unmeasured, selected, remedy=None, cell_key=None)#
An operation would read an assay over cells that the assay did not measure.
- operation#
The refusing operation.
- assay#
The assay that the operation reads.
- column#
The assay’s membership column,
<assay>_I.
- unmeasured#
How many of the selected cells the assay did not measure.
- selected#
How many cells the operation would read.
- remedy#
The input that the message tells the caller to narrow.
- cell_key#
The boolean cell column whose cells the operation reads, or None.
Saved cytearc.datastore.namespaces.MarkersAccessor.search() calls return the exact immutable marker-table reference.
Pass that reference as markers.load(marker=ref) to select the exact feature-specific result.
Marker tables report group_id as a string and list groups in the same order as plot
categories: decimal labels first in numeric order, so "2" precedes "10", then other labels in
natural order, so "1_2" precedes "1_10" and "2_T" precedes "B cell".
markers.export_csv uses the same column order. An unknown group_id raises an error.
Fresh marker results include score, expression fractions, fold change, AUC, two-sided Mann-Whitney p-values, and Benjamini-Hochberg values adjusted within each one-versus-rest group over tested features.
These are cell-level marker statistics, not replicate-aware differential expression.
fold_change is the group’s mean divided by mean_rest, the means of the values that the
search ranks. It is inf for a feature that no other cell expresses, so such features sort above
every finite ratio, and NaN when both means are 0 or either is negative, as signed counts can
give. Every other statistic is finite. The ranked values are the assay’s normalized values, such
as library-size normalized RNA counts, log-scale with log_transform=True, CLR values of ADT
counts, which are log-scale too, or TF-IDF values of ATAC counts. No pseudocount is added, and
fold changes compare only within one assay and normalization. Marker tables record this policy in
their fold_change_policy metadata. markers.load, markers.export_csv, and marker_heatmap
raise ValueError for tables missing that policy; recompute them with markers.search.
Marker search reads raw counts of any storage dtype. Library-size markers require finite
non-negative counts and cell totals: a negative or non-finite normalized value of a tested feature,
or total of a selected cell (its <assay>_nCounts value, or with renormalize_subset=True its sum
over the tested features), raises ValueError before anything is written.
Every marker, pseudotime-marker, and pseudotime-aggregation search ranks or correlates the values
of the assay’s normed with its configured normMethod. Their log_transform and
renormalize_subset default to False; log_transform=True takes log1p of the normalizer’s own
output, and a flag that the normalizer cannot apply, such as log_transform=True with ADT CLR or
ATAC TF-IDF, raises ValueError (Assays and metadata API reference). The artifact records the resolved flags.
cytearc.datastore.namespaces.TrajectoryAccessor.markers() leaves untested features with r_value 0.0 and NaN for p_value and p_value_adjusted, and adjusts p-values over tested features only.
cytearc.datastore.namespaces.CompareAccessor.test() computes Mann-Whitney
p-values from the exact permutation null, ties included, when the two groups can be formed in at
most 100,000 ways, as in typical sample-level designs. Larger designs keep the marker-search normal
approximation. StatisticalTestResult.p_value_method records "exact" or "asymptotic", and a
saved Mann-Whitney result without that record must be recomputed. A StudyDesign pairing column
applies only to the paired Wilcoxon test; for subjects nested within conditions, pass the subject
column as sample_by. Writing a result requires zarr_mode='r+' and is checked before any value
is computed, while a matching saved result is still reused from a read-only store.
cytearc.datastore.namespaces.ScoresAccessor.waggr() and
cytearc.datastore.namespaces.ScoresAccessor.aucell() match network targets to active feature
names without case sensitivity. A target that matches several active features, such as a gene
symbol shared by two feature ids, is ambiguous. The default ambiguous_targets="drop" removes its
edges before tmin pruning, logs a warning, and records the target in the artifact’s
dropped_ambiguous_targets attribute; ambiguous_targets="error" raises instead. Both producers
require zarr_mode='r+' and an intact prepared dataset identity.
cytearc.datastore.namespaces.TrajectoryAccessor.aggregation() aggregates and clusters
features before it starts an artifact. It raises ValueError, and leaves no incomplete artifact,
when too few features pass min_exp or when n_clusters is greater than one but smaller than the
number of disconnected components of the feature neighbour graph. Modules tied at the cut height are merged so that exactly
n_clusters modules are returned.
cytearc.datastore.namespaces.TrajectoryAccessor.fate() treats solver_tol as an absolute
bound on the largest Bellman residual of each solved sink column, so large sink groups do not
loosen the solve.
modalities() is the public SNN/WNN graph integration entry point.
WNN requires two or more cell-aligned assays and stores one per-cell weight for each assay.
SNN also requires two or more explicit connectivity-map refs.
The recommended workflow is in Read RNA and protein together with CITE-seq; method comparison and diagnostics are in Diagnose multimodal integration.
Analysis results#
- class cytearc.FateMappingResult(ref, graph, pseudotime, sink_labels_artifact, cell_selection, sink_labels, values, valid)#
Fate probabilities loaded from an immutable artifact.
- class cytearc.PseudotimeScoreResult(ref, graph, cell_selection, values, valid)#
Pseudotime values loaded from an immutable artifact.
- class cytearc.PseudotimeMarkerResult(ref, table, assay, cell_selection, feature_selection, pseudotime)#
Pseudotime correlation table loaded from an immutable artifact.
- class cytearc.PseudotimeAggregationResult(ref, data, feature_indices, feature_clusters, assay, cell_selection, feature_selection, pseudotime)#
Lazy pseudotime aggregation loaded from an immutable artifact.
- property feature_ids: ndarray#
Frozen feature IDs aligned to the aggregation rows.
- property feature_names: ndarray#
Frozen feature names aligned to the aggregation rows.
- class cytearc.EnrichmentResult(data, source_names, source_sizes, cell_index, artifact, storage_path, assay, cell_selection, feature_selection, method)#
A persisted gene-set enrichment result.
- data#
Lazy score matrix with cells as rows and sources as columns.
- source_names#
Source names aligned to the score columns.
- source_sizes#
Matched target counts aligned to
source_names.
- cell_index#
Assay cell indices aligned to the score rows.
- artifact#
Immutable enrichment artifact backing the result.
- storage_path#
Zarr path that owns the result.
- assay#
Name of the RNA assay used for scoring.
- cell_selection#
Immutable cell selection used for scoring.
- feature_selection#
Feature-selection artifact used for scoring.
- method#
Enrichment method, either
"waggr"or"aucell".
- class cytearc.features.statistical.StatisticalTestResult(method, posthoc, adjustment_method, grouping, group_field, sample_by=None, pair_by=None, sample_stat='mean', expression_cutoff=0.0, alternative='two-sided', equal_var=None, n_groups=0, n_cells=0, tested_features=(), summary_scope='cell', artifact=None, cell_selection=None, cell_selection_fingerprint=None, group_fingerprint=None, group_order=(), normalization=<factory>, normalization_method=None, size_factor=None, source_assays=(), source_dataset_fingerprint=None, value_fingerprints=(), sample_fingerprint=None, pair_fingerprint=None, tables=<factory>, posthoc_tables=<factory>, p_value_method=None)#
Statistical tests for one grouping across multiple value keys.
p_value_methodis"exact"or"asymptotic"for Mann-Whitney, chosen by design size as the module docstring describes, andNonefor other tests.
- class cytearc.clustering.ParisClusteringResult(labels, mode, n_clusters, diagnostics=(), min_cluster_size=None, hierarchy_artifact_id=None, ref=None)#
Labels and diagnostics returned by a Paris cut.
refis set when the cut was written to a store and staysNonefor cuts produced directly from a hierarchy.
- class cytearc.clustering.ParisClusterDiagnostic(label, selected_node, parent_event, component, size, resolution_lower, resolution_upper, persistence, forced, blocking_child_count, folded_cell_count, decision_margin)#
Structural diagnostic for one selected Paris cluster.
Inheritance appendix#
These implementation classes are not intended for analysts to construct directly.
- class cytearc.datastore.base_datastore.BaseDataStore(zarr_loc, assay_types, default_assay, min_features_per_cell, mito_pattern, ribo_pattern, zarr_mode, workspace, resources, storage_profile, storage_options=None, storageIo=None, source_storage_options=None)
This is the base datastore class that deals with loading of assays from Zarr files and generating basic cell statistics like nCounts and nFeatures. Superclass of the other DataStores.
- Parameters:
zarr_loc (
str|Store) – Path to Zarr file created using one of writer functions of CyteArcassay_types (
dict[str,str] |None) – A dictionary with keys as assay names present in the Zarr file and values as either one of: ‘RNA’, ‘ATAC’, ‘ADT’, ‘HTO’, ‘CRISPR’, ‘ANTIGEN’, ‘CUSTOM’, ‘GeneActivity’, ‘GeneScores’, ‘URNA’ or ‘Assay’. A read-only open cannot change an assay’s type.default_assay (
str|None) – Name of assay that should be considered as default. It is mandatory to provide this value when DataStore loads a Zarr file for the first timemin_features_per_cell (
int) – Writable opens remove fromIevery cell whose default-assay feature count is not greater than this value, unless that would remove at least half of the active cells.mito_pattern (
str|None) – Feature-name pattern for the{assay}_percentMitocolumn of each RNA assay. The first writable open replaces any existing column with values computed from this pattern, or^MT-when None. Later opens keep the stored values when None and reject a pattern that differs from the recorded one.ribo_pattern (
str|None) – The same for{assay}_percentRibo, using^RPS|^RPL|^MRPS|^MRPLwhen None.zarr_mode (
Literal['r','r+','a','w','w-']) – For read-write mode user+or for read-only user. (Default value:r+)workspace (
str|None) – Workspace name within the Zarr store (None for legacy single-workspace layout).resources (
ResourceBudget) – Resolved memory and worker budget for this datastore.storage_profile (
Literal['fast_local','cloud']) – Zarr encoding profile used for new arrays written through this datastore.storage_options (
dict[str,Any] |None) – Backend options passed when opening the Zarr store.storageIo (
Any|None) – Optional explicit read, compute, and write widths for storage work. Unset values stay under automatic planning.
- cells
MetaData object with cells and info about each cell (e. g. RNA_nCounts ids).
- nthreads
Number of threads to use for this datastore instance.
- z
The Zarr file (directory) used for this datastore instance.
- class cytearc.datastore.graph_datastore.GraphDataStore(zarr_loc, assay_types, default_assay, min_features_per_cell, mito_pattern, ribo_pattern, zarr_mode, workspace, resources, storage_profile, storage_options=None, storageIo=None, source_storage_options=None)
This class extends BaseDataStore by providing methods required to generate a cell-cell neighbourhood graph.
It also contains all the methods that use the KNN graphs as primary input like UMAP/tSNE embedding calculation, clustering and trajectory analysis.
- cells
List of cell barcodes.
- nthreads
Number of threads to use for this datastore instance.
- z
The Zarr file (directory) used for this datastore instance.
- class cytearc.datastore.mapping_datastore.MappingDatastore(zarr_loc, assay_types, default_assay, min_features_per_cell, mito_pattern, ribo_pattern, zarr_mode, workspace, resources, storage_profile, storage_options=None, storageIo=None, source_storage_options=None)
This class extends GraphDataStore by providing methods for mapping/ projection of cells from one DataStore onto another. It also contains the methods required for label transfer, mapping score generation and co-embedding.