Clustering and cluster evidence

Clustering groups cells that share neighbours on the graph. For this RNA example, those neighbourhoods reflect similarity in expression. A cluster can represent a cell type, a temporary state, or technical variation. Marker evidence and stability checks help distinguish these possibilities.

Leiden seeks groups with stronger connections within groups than expected from its graph model. The resolution controls the scale at which it forms those groups. Lower resolutions tend to produce broader groups, while higher resolutions tend to split them into smaller groups. A coarse partition can hide a rare population, and a fine partition can divide a homogeneous population. Paris instead builds a hierarchy of nested merges, which you can cut adaptively or at a fixed cluster count.

We will inspect a saved Leiden clustering, vary its resolution on the same graph, and compare it with Paris. Keeping the graph and layout fixed helps us see what the clustering choice changes.

Open the existing graph¶

Open the downloaded store and its saved analysis.

The store carries an analysis saved as docs_default. Inspect its selected clusters first:

This prepared example used Leiden resolution 0.5. The standalone method defaults to 1.0, and the standard pipeline can compare several resolutions. Cluster numbers are labels, not cell types. We use the same graph and UMAP below so only the clustering changes.

Sweep Leiden resolution¶

Higher resolution usually produces more and smaller groups, but the partitions need not be nested. Investigate a split when it tracks a technical covariate, has weak marker evidence, or disappears under a modest parameter change. A rare or transitional population may need more evidence than a well-separated abundant group.

The adjusted Rand index (ARI) compares partitions without requiring the cluster numbers to match. A value of one means the partitions agree. It does not tell us which partition is better:

Inspect membership strength¶

Membership strength measures how strongly a cell connects to its assigned cluster: it is the fraction of the cell’s graph neighbours that carry the cell’s own cluster label. A value of 1 means that every neighbour shares the cell’s cluster. A cell whose neighbours sit mostly in other clusters scores low, even when those neighbours all share one other label. We use resolution 0.5 for this walkthrough.

Low values throughout one cluster suggest a weak boundary: many of its cells have most of their neighbours in other clusters. A narrow band of low values between otherwise coherent groups may represent continuous biology.

Review marker evidence¶

Marker search compares each cluster with the remaining cells. Pass the cluster result and the feature selection explicitly so the marker table refers to the partition being reviewed. We inspect the largest and smallest clusters here to check whether both have coherent marker support.

Markers for the largest cluster:

Markers for the smallest cluster:

The p-values are cell-level one-versus-rest marker tests with within-group adjustment. They are not replicate-aware differential expression. A defensible partition combines marker evidence, graph support, technical covariates, replicate coverage, and the study question.

Pipeline cluster selection¶

The full pipeline can compare several Leiden resolutions for you. It uses a silhouette score, which compares a cell’s distances within its assigned group with distances to the nearest other group. Each candidate is scored on the same deterministic sample of at most 10,000 cells, using the graph’s PCA or Harmony coordinates or, with pca_dims=0, its normalized values. This keeps the comparison consistent across candidates. Paris can still run as clustering_run["paris"], but it is not an automatic winner. The cluster_selection artifact saves the scores, sampling policy, reasons a candidate was invalid, tie order, and selected key:

# Keep the saved cluster-selection diagnostics.
decision_ref = clustering_run["cluster_selection"]

# Keep the clustering chosen by the pipeline.
selected_cluster_ref = clustering_run["clusters"]

This automatic choice is a reproducible baseline, not proof that the selected resolution is best for every biological question. Retain alternative refs when the decision needs domain-specific evidence.

Optional: compare Paris cuts¶

Paris offers another way to divide the same graph. clusters.paris builds a hierarchy of nested merges and returns a cluster_cut reference. Load the result explicitly to inspect the hierarchy diagnostics alongside the labels.

Persistence combines group size with the range of hierarchy merge scales over which a branch survives. It is not elapsed biological time, and larger groups can have higher scores because of their size. The decision margin measures the preference for retaining the branch rather than splitting it, divided by the group size. A forced group satisfies a structural constraint and is not, by itself, strong biological evidence. Neither the hierarchy nor its cut establishes a developmental lineage.

Check what a cluster means¶

• Separate marker discovery from donor-level inference. Cell-level marker tests help describe the groups you found. Cells from the same donor are not independent biological replicates, so small marker p-values do not establish a reproducible difference between conditions. Use Pseudobulk and differential expression (DE) primer for replicate-aware comparisons.

• Look for continuous transitions. Development and cell-cycle progression can cross the discrete boundaries imposed by clustering. Inspect membership strength and cluster connectivity before treating transitional cells as a distinct terminal type.

• Check technical explanations for splits. Compare groups against sequencing depth, mitochondrial fraction, and batch or donor metadata. Automatic selection and a high resolution do not establish biological meaning. A Paris group marked forced also needs independent marker and stability evidence.