Build a neighborhood graph stage by stage

A cell graph describes which cells resemble one another. Each node is a cell, and an edge connects it to a nearby cell in the chosen representation. CyteArc searches for k nearest neighbors per cell, then converts their distances into weighted connections used by clustering, embeddings, imputation, and trajectory methods.

For RNA, similarity is usually measured in PCA space; for ATAC, it is measured in LSI space. Reducing dimensionality limits the influence of noisy features and makes neighbor search more efficient. Nearby cells share patterns in the selected representation, but this does not by itself establish a common cell identity. Batch effects and other unwanted variation can also bring cells together.

CyteArc first builds an approximate neighbor index, then queries neighbors and calculates connectivity. Calling these stages directly helps when you need to change one parameter, insert batch correction, or reuse an expensive reduction. Start with Identify PBMC populations with scRNA-seq if you have not yet run an analysis. This page explains the individual steps behind that workflow.

Feature selection is covered in Feature selection: choosing informative features. This guide begins once a feature-selection artifact exists.

The standard graph workflow¶

For RNA, the graph stages are:

1. Normalize the selected genes.

2. Reduce them with PCA.

3. Optionally correct the reduced coordinates with Harmony.

4. Build an approximate-neighbour index.

5. Query k neighbours per cell.

6. Convert neighbour distances into weighted connectivity.

7. Build a separate initialization for UMAP and t-SNE.

ATAC follows the same shape but uses TF-IDF normalization and LSI. The initialization does not define graph edges; it only supplies starting coordinates for a layout.

ds.pipeline.run() orchestrates the standard RNA path and can continue through UMAP, clustering, doublet scoring, and markers. Use it when the defaults match the analysis. The stage methods below expose the same persisted results with more control.

Build an RNA graph explicitly¶

The rebuilt store carries a completed docs_default pipeline run. This page starts from that run’s frozen cell and feature selections, then calls every graph stage explicitly. Identical calls reuse the run’s matching artifacts. Later sections create the branches they discuss.

Retrieve the cell and gene selections used by the saved analysis:

Each method returns a reference to its saved result. Pass it to the next method to keep the steps connected. We use 15 PCs to match the prepared example; reduction.pca defaults to 21.

Build the neighbor index from PCA, then turn neighbor distances into connectivity:

The default neighbour count is 11. We will change only that value in the comparison below.

graph.load returns a sparse cell-by-cell connectivity matrix. Its default keeps the directed neighbour edges. For the diagnostics below, use symmetric=True to include a connection when either cell selects the other. This lets us count each cell’s neighbours in either direction. A cell can then have more than k neighbours because other cells may select it. In this symmetric matrix, nnz counts both directions of each connection.

Check isolation and whether degree tracks QC metrics before treating the graph as ready for clustering:

The graph should contain a row for every selected cell, with finite positive weights on its stored connections. Zero entries mean there is no edge between those cells. The degree summary detects isolated cells, but a graph with no isolated cells can still contain separate components.

Separate components can represent distinct populations. Investigate unexpected fragmentation or a strong relationship between degree and QC metrics using markers, feature selection, PCA dimensions, and k. Increasing k just to join every component can connect unrelated cells.

Use the graph for a layout and clustering¶

embeddings.umap and embeddings.tsne require both the graph and its matching initialization. Leiden and Paris require the graph. Each returns an immutable artifact without adding cell-metadata columns. Use clusters.load_paris(ref) only when hierarchy diagnostics are needed. Resolution 0.5 matches the prepared PBMC analysis; Leiden’s default is 1.0.

Branch by retaining both references¶

Suppose PCA and the approximate nearest-neighbor (ANN) index are expensive to recompute, but you want to compare two neighbor counts. Query the same index with a different k: only the neighbor and connectivity stages need new results. Keep both graph references so either analysis can continue independently.

Both branches remain complete, addressable artifacts. Downstream calls must receive one of them explicitly, so a parameter experiment cannot silently replace another branch.

Degree and edge weight both shift when every cell sees more neighbours:

To analyse the side branch, pass its exact graph reference. Retain both returned refs so neither branch replaces the other.

Place both partitions on the shared k=11 UMAP so changes in group boundaries are visible, then compare their assignments with a crosstab:

Cluster numbers can change even when the groups stay the same. Look for a row spread across several columns, or a column collecting several rows, to find splits or merges that depend on k. Review marker evidence before accepting those boundaries.

Recompute only what changed¶

Artifact identity includes the operation, scientific parameters, upstream inputs, and the operation’s revision. Calling an identical stage reuses its completed result. Changing k reuses normalization, PCA, and the ANN index but creates new neighbour and connectivity artifacts. Changing the cell or feature selection requires new downstream results. The saved results of the earlier analysis remain available.

Harmony changes the coordinates used to measure similarity. It reuses the existing PCA result, but its corrected coordinates need a new ANN index, neighbor search, and connectivity graph. Use it only when the study design supports technical batch correction:

# Correct PCA coordinates using a technical batch column.
corrected = ds.reduction.harmony(pca, ["technical_batch"])

# Build the neighbor index from corrected coordinates.
corrected_index = ds.graph.ann_index(corrected)

# Find neighbors in the corrected space.
corrected_neighbors = ds.graph.neighbors(corrected_index, k=21)

# Build connectivity from the corrected neighbors.
corrected_graph = ds.graph.connectivity(corrected_neighbors)

Use Why CyteArc records provenance to inspect complete lineage and Reuse results and compare analysis settings for reuse and invalidation patterns.