Source: https://docs.nygen.io/CyteArc/api/integration.html Revision: fccee7b6e0c167e47ce605c8d292fba5b567ebfe Integration and metrics API reference ************************************* Prefer the "DataStore" methods below in analysis code. "integration.modalities" accepts two or more exact source refs. WNN is the default; it consumes neighbour artifacts and follows each one’s named native reduction or batch-correction coordinates. SNN is explicit and consumes connectivity-map artifacts. The returned integrated-graph reference is the exact downstream "graph" argument. WNN does not accept imported-coordinate ancestry. "coordinates" is a provenance input on neighbour and ANN artifacts, not a separate artifact kind. wnn = ds.integration.modalities([rna_neighbors, adt_neighbors]) snn = ds.integration.modalities([rna_graph, adt_graph], method="snn") WNN also rejects neighbours built on normalized values, as a pipeline run with "pca_dims=0" builds them, because it weighs each assay by its reduced coordinates; SNN integrates such graphs. Integration holds its inputs in memory: SNN every source graph with a float64 matrix of shared-neighbour fractions per graph, about "16 * n_graphs + 20" bytes per edge, and WNN every assay’s neighbours and coordinates with the integrated graph and its modality weights. Both compare that estimate with the datastore’s memory budget before they load a graph or coordinate and raise "MemoryError" over it, naming the bytes, the cells, the neighbours, the dimensions, and the limit. "reduction.harmony" does the same before it reads coordinates: it holds the float64 coordinates several times over and float64 matrices of soft cluster assignments, about 5 KB per cell with 30 dimensions and 100 clusters, so "nclust" in "harmony_params" and the dimensions set its memory. Its admission also counts the batch labels, which it reads before it admits the fit and holds until the fit ends, at the size of their text objects: about 60 bytes per cell for each batch column of short labels. These estimates are near-exact counts of what each step allocates, as traced while it runs, plus stated allowances; the budget compares them with its limit, and they are not a cap on the memory of the process (see Memory and execution). The default WNN artifact stores one per-cell weight for each input assay. Plot those values with "cytearc.plotting.modality_weights()" or the bound "ds.plots.modality_weights(graph=wnn, layout=wnn_layout)" accessor. An explicit SNN artifact does not contain modality weights. Neighbour-based metrics require "neighbors"; graph metrics require "graph"; reduction metrics require their exact coordinate artifact. Batch-mixing methods and biological-annotation cLISI or graph connectivity intentionally read imported metadata columns over the rows selected by artifact lineage. They are not an indirect path for scoring CyteArc-produced clusterings. Concordance instead accepts two exact clustering refs with the same frozen cell selection. None accepts a storage path or an omitted artifact input. mixing = ds.integration.compute_ilisi("batch", rna_neighbors) connectivity = ds.integration.compute_graph_connectivity("cell_type", snn) agreement = ds.clusters.compute_concordance(rna_clusters, adt_clusters, metric="ari") The metric methods return dataset-level scalar summaries such as iLISI, cLISI, and graph connectivity. "cytearc.metrics.compute_lisi()" returns per-cell LISI values from KNN arrays. Metric label columns must label every scored cell: masked, "NaN", "None", and blank labels raise "ValueError". All LISI entry points resolve "perplexity=None" to "floor(k / 3)", where "k" is the number of neighbours. Explicit values are capped at "k / 3" with a warning when the graph is too small. With three to five neighbours, the default is 1, so scores can approach 1 when the nearest distance is unique. Tied distances can produce larger values. Cluster separability summaries expose "macro_f1_fold_sd", the sample standard deviation of macro F1 across validation folds. Interpret it as fold variability. DataStore methods ================= +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Integrate explicit graph or neighbor artifacts across assays. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | modalities | | | " | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Compute scIB integration LISI on a persisted KNN graph. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | compute_il | | | isi" | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Compute scIB cell-type LISI on a persisted KNN graph. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | compute_cl | | | isi" | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Summarize batch LISI as a normalized neighborhood-mixing score. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | compute_ba | | | tch_mixing | | | " | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Score label connectivity on a persisted, symmetrized assay graph. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | compute_gr | | | aph_connec | | | tivity" | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Calculate modified silhouette scores for evaluating cluster separation. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | compute_gr | | | aph_silhou | | | ette" | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Compare two immutable clustering artifacts using ARI or NMI. | | atastore.n | | | amespaces. | | | ClustersAc | | | cessor.com | | | pute_conco | | | rdance" | | +------------+--------------------------------------------------------------------------------------------+ | "cytearc.d | Evaluate cluster-label separability in PCA coordinates. | | atastore.n | | | amespaces. | | | Integratio | | | nAccessor. | | | compute_cl | | | uster_sepa | | | rability" | | +------------+--------------------------------------------------------------------------------------------+ IntegrationAccessor.modalities(sources, method='wnn', chunk_size=10000, invalidate_cache=False, l2_normalize=True) Integrate explicit graph or neighbor artifacts across assays. SNN combines shared edge support across two or more assays. WNN accepts two or more assays and uses Hao-inspired per-cell modality weights. CyteArc WNN scores only the union of the existing self- free KNN rows and uses the distance span from the nearest to the k-th neighbour as its bandwidth, so it is not bit-identical to Seurat’s default wider search and SNN-far bandwidth. Parameters: * **sources** ("list"["ArtifactRef"]) – Connectivity-map refs for SNN or neighbor refs for WNN. * **method** ("str") – Choose a method for modality integration. Available options: ‘wnn’: Hao-inspired weighted nearest neighbor integration and ‘snn’: shared nearest neighbour integration. * **chunk_size** ("int") – Number of cells per stored chunk of the integrated edge, weight, and modality-weight arrays. It does not bound memory: integration holds every source graph, and for WNN every source’s coordinates, in memory. * **invalidate_cache** ("bool") – Force a new integrated-graph artifact. * **l2_normalize** ("bool") – L2-normalize modality coordinates during WNN scoring. This algorithmic setting is stored in artifact provenance. Return type: "ArtifactRef" Returns: Reference to the integrated-graph artifact. Pass it to *embeddings.umap*, *embeddings.tsne*, or the clustering methods as their "graph" argument. Raises: * **MemoryError** – If the integration exceeds the memory budget. * **UnmeasuredCellsError** – If a source assay did not measure a selected cell. WNN modality weights remain in the returned immutable artifact. IntegrationAccessor.compute_ilisi(batch_colname, neighbors, *, perplexity=None, scale=True) Compute scIB integration LISI on a persisted KNN graph. Parameters: * **batch_colname** ("str") – Cell metadata column containing batch labels. * **neighbors** ("ArtifactRef") – Explicit neighbor artifact to score. * **perplexity** ("float" | "None") – Effective neighborhood size. "None" uses "floor(k / 3)". * **scale** ("bool") – Scale the median LISI by the number of observed batches. Return type: "float" Returns: Median iLISI, scaled so higher values indicate better batch mixing when "scale" is true. -[ Notes ]- CyteArc persisted KNN graphs exclude self-neighbors, as required by this metric. The batch column is intentionally an imported metadata input. IntegrationAccessor.compute_clisi(annotation_column, neighbors, *, perplexity=None, scale=True) Compute scIB cell-type LISI on a persisted KNN graph. Parameters: * **annotation_column** ("str") – Cell metadata column containing imported biological annotations. * **neighbors** ("ArtifactRef") – Explicit neighbor artifact to score. * **perplexity** ("float" | "None") – Effective neighborhood size. "None" uses "floor(k / 3)". * **scale** ("bool") – Invert and scale the median LISI by the number of observed labels. Return type: "float" Returns: Median cLISI, scaled so higher values indicate better label conservation when "scale" is true. -[ Notes ]- CyteArc persisted KNN graphs exclude self-neighbors, as required by this metric. The annotation column is intentionally an imported metadata input, not a CyteArc-produced clustering. IntegrationAccessor.compute_batch_mixing(label_colname, neighbors, *, perplexity=None) Summarize batch LISI as a normalized neighborhood-mixing score. This computes batch LISI on the supplied KNN graph and rescales its mean against the mixing that perfectly integrated data would reach given the dataset’s batch sizes. Unlike raw LISI, the result is bounded in "[0, 1]", which makes it easier to compare across graphs and datasets. Parameters: * **label_colname** ("str") – Cell metadata column holding the batch assignment. * **neighbors** ("ArtifactRef") – Explicit neighbor artifact to score. * **perplexity** ("float" | "None") – Effective neighborhood size; None uses floor(k / 3). Return type: "float" Returns: A value in "[0, 1]". Scores near 1 indicate that neighborhoods mix batches as well as the global composition allows, and scores near 0 indicate poorly mixed batches. Raises: **ValueError** – If KNN inputs are invalid or the column has fewer than two batches. IntegrationAccessor.compute_graph_connectivity(annotation_column, graph) Score label connectivity on a persisted, symmetrized assay graph. Parameters: * **annotation_column** ("str") – Cell metadata column containing imported biological annotations. * **graph** ("ArtifactRef") – Explicit connectivity-map or integrated-graph artifact. Return type: "float" Returns: Mean fraction of cells retained in the largest connected component for each label. -[ Notes ]- Persisted directed edges are treated as undirected. This follows the original scIB symmetrized-graph definition and intentionally differs from the directed strong-component calculation currently used by YosefLab "scib-metrics". The annotation column is an imported biological label, not a CyteArc-produced clustering. IntegrationAccessor.compute_graph_silhouette(neighbors, clusters, *, random_seed=4444, sample_size=11) Calculate modified silhouette scores for evaluating cluster separation. This implements a graph-based silhouette score that measures how similar cells are to their own cluster compared to the nearest neighboring cluster. Parameters: * **neighbors** ("ArtifactRef") – Explicit neighbor artifact to score. * **clusters** ("ArtifactRef") – Explicit cluster-label or Paris cluster-cut artifact. * **random_seed** ("int") – Seed used for cluster sampling. * **sample_size** ("int") – Maximum size of each sampled cluster group. Return type: "ndarray" | "None" Returns: numpy array of silhouette scores for each cluster, or None if computation fails Raises: **ValueError** – If graph, labels, sampling, or embedding data are invalid -[ Notes ]- Scores range from -1 to 1: - Near 1: Cluster is well-separated from neighboring clusters - Near 0: Cluster overlaps with neighboring clusters - Near -1: Cluster may be incorrectly assigned Implementation uses sampling for efficiency with large datasets. NaN values indicate clusters that couldn’t be scored due to size constraints. ClustersAccessor.compute_concordance(first, second, metric='ari') Compare two immutable clustering artifacts using ARI or NMI. Both clusterings must carry the exact same cell-selection reference. This allows clusterings from different assays to be compared without consulting mutable cell metadata. Parameters: * **first** ("ArtifactRef") – First cluster-label or Paris cluster-cut artifact. * **second** ("ArtifactRef") – Second cluster-label or Paris cluster-cut artifact. * **metric** ("Literal"["'ari'", "'nmi'"]) – ""ari"" for the adjusted Rand index or ""nmi"" for normalized mutual information. Return type: "float" Returns: Agreement between the two partitions. ARI ranges from -1 to 1 and NMI from 0 to 1, with higher values meaning stronger agreement. Raises: **ValueError** – If the clusterings do not use the same frozen cell selection or the metric name is invalid. IntegrationAccessor.compute_cluster_separability(pca, clusters, *, n_folds=5, max_sample_cells=50000, max_silhouette_cells=10000, random_seed=4444, svm_c=1.0, svm_max_iter=10000) Evaluate cluster-label separability in PCA coordinates. Return type: "ClusterSeparabilityResult" Harmony ======= cytearc.embeddings.fit_harmony(data_mat, meta_data, theta=None, lamb=None, sigma=0.1, nclust=None, tau=0, block_size=0.05, max_iter_harmony=50, max_iter_kmeans=20, epsilon_cluster=1e-05, epsilon_harmony=0.0001, random_state=0, cluster_fn='kmeans') Fit Harmony and return corrected coordinates with portable state. Return type: "HarmonyResult" class cytearc.embeddings.HarmonyResult(original, corrected, assignments, centroids, sigma, ridge, batch_columns, batch_levels, parameters) Converged Harmony state required for portable reference mapping. Metrics ======= Methods and classes for evaluation class cytearc.metrics.ClusterSeparabilityResult(clustering_scores, cluster_scores, confusion, sample_indices) cytearc.metrics.clisi_knn(distances, indices, cell_labels, perplexity=None, scale=True) Compute the median cell-type LISI score from a self-free KNN graph. When scaled, higher values indicate better conservation of cell labels. This follows the cLISI aggregation and scaling used by scIB. The KNN arrays must exclude each cell from its own neighbor row. CyteArc’s persisted KNN graphs satisfy this requirement. Self- including KNN arrays produce a different statistic and are not adjusted automatically. Return type: "float" -[ References ]- Luecken et al. 2022 doi: 10.1038/s41592-021-01336-8 cytearc.metrics.compute_lisi(distances, indices, metadata, label_colnames, perplexity=None) Compute the Local Inverse Simpson Index (LISI) for each column in metadata. LISI measures how well mixed different groups of cells are in the neighborhood of each cell. Higher values indicate better mixing of different groups. Parameters: * **distances** ("ndarray" | "Array") – Pre-computed distances between cells, stored in zarr array format * **indices** ("ndarray" | "Array") – Pre-computed nearest neighbor indices, stored in zarr array format * **metadata** ("DataFrame") – DataFrame containing categorical labels for each cell * **label_colnames** ("Iterable"["str"]) – Column names in metadata to compute LISI for * **perplexity** ("float" | "None") – Effective neighborhood size; None uses floor(neighbor count / 3). Returns: Matrix of LISI scores with shape (n_cells, n_labels) Each column corresponds to LISI scores for one label column in metadata Return type: "ndarray" -[ Example ]- For metadata with a ‘batch’ column having 3 categories: - LISI ≈ 3: Cell has neighbors from all 3 batches (well mixed) - LISI ≈ 1: Cell has neighbors from only 1 batch (poorly mixed) -[ References ]- Korsunsky et al. 2019 doi: 10.1038/s41592-019-0619-0 cytearc.metrics.graph_connectivity(edges, labels, batch_rows=100000, *, weights=None) Score label connectivity on a persisted, implicitly undirected graph. Each directed edge is treated as an undirected connection. The result is the mean, across labels, of the fraction of cells in the largest connected component for that label. When weights are supplied, zero-weight edges do not connect cells. This follows the original scIB symmetrized-graph definition. It does not match the directed strong-component calculation currently implemented by YosefLab "scib-metrics". Return type: "float" -[ References ]- Luecken et al. 2022 doi: 10.1038/s41592-021-01336-8 cytearc.metrics.ilisi_knn(distances, indices, batch_labels, perplexity=None, scale=True) Compute the median integration LISI score from a self-free KNN graph. When scaled, higher values indicate better batch mixing. This follows the iLISI aggregation and scaling used by scIB. The KNN arrays must exclude each cell from its own neighbor row. CyteArc’s persisted KNN graphs satisfy this requirement. Self- including KNN arrays produce a different statistic and are not adjusted automatically. Return type: "float" -[ References ]- Luecken et al. 2022 doi: 10.1038/s41592-021-01336-8 cytearc.metrics.label_concordance_score(label_sets, metric='ari') Compare two label partitions using ARI or NMI. Parameters: * **label_sets** ("Sequence"["ndarray"]) – Two arrays of labels to compare. * **metric** ("Literal"["'ari'", "'nmi'"]) – Either ""ari"" or ""nmi"". Return type: "float" Returns: Label agreement. ARI ranges from -1 to 1 and NMI from 0 to 1. cytearc.metrics.lisi_batch_mixing_score(lisi_scores, batch_labels) Normalize mean batch LISI against the dataset’s batch proportions. Raw batch LISI depends on how many batches are present and how unevenly cells are split between them, so its values are hard to compare across datasets. This score rescales the mean batch LISI onto a fixed range by dividing the observed neighborhood mixing by the mixing that perfectly integrated data would reach. The reference point is the inverse Simpson index of the global batch proportions, which is the LISI a neighborhood would show if it mirrored the whole dataset. Parameters: * **lisi_scores** ("ndarray") – Per-cell batch LISI values, typically the array returned by "compute_lisi()" for a batch label. * **batch_labels** ("Sequence"["object"] | "ndarray") – Batch assignment for each cell, aligned with "lisi_scores". Return type: "float" Returns: A value in "[0, 1]". Scores near 1 mean neighborhoods mix batches as well as the global composition allows, and scores near 0 mean batches stay separated. Raises: **ValueError** – If the inputs are misaligned, contain non- finite scores or missing labels, or assign the cells to fewer than two batches. cytearc.metrics.silhouette_scoring(ds, graph, hvg_data, assay_type, res_label, *, cell_key='I', random_seed=4444, sample_size=11, distance_metric, neighbor_indices=None, neighbor_distances=None) Compute modified silhouette scores for clusters in single-cell data. This implementation differs from the standard silhouette score by using a graph-based approach and comparing clusters to their nearest neighbors. Parameters: * **ds** (cytearc.datastore.datastore.DataStore) – DataStore object containing cell metadata * **graph** ("csr_matrix" | "None") – Optional CSR matrix representing the weighted KNN graph * **hvg_data** ("ndarray" | "Array" | "ChunkedArray") – Cell-by- dimension matrix whose rows are compared, such as the coordinates behind the KNN graph * **assay_type** ("str") – Type of assay (e.g., ‘RNA’, ‘ATAC’) * **res_label** ("str") – Label for clustering resolution * **distance_metric** ("Literal"["'l2'", "'cosine'", "'ip'"]) – Neighbor metric used to compare sampled rows: ‘l2’, ‘cosine’, or ‘ip’. Inner-product distances are one minus the inner product, clipped at zero, so rows whose inner products exceed one compare as identical. Returns: Array of silhouette scores for each cluster, or None if cluster labels are not found Return type: "ndarray" | "None" Raises: **ValueError** – If a selected cell has a missing cluster label, either a null value or a row that the column’s linked missing mask flags. -[ Notes ]- Scores are calculated using a sampling approach for efficiency. NaN values indicate clusters that couldn’t be scored due to size constraints. CyteArc’s iLISI and cLISI use the scIB median and scaling definitions over CyteArc’s self-free persisted KNN arrays. Graph connectivity follows the original scIB symmetrized-graph definition. YosefLab "scib-metrics" currently uses directed strong components for graph connectivity, so those values need not match. See Luecken et al. 2022, doi: 10.1038/s41592-021-01336-8.