Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Dose response

Composition tests asked whether a condition changes which cells a sample holds. A dose series asks of each cell type which genes respond to a dose and how, and two things come before any test or curve: where each dose sits on the axis of a curve, which DoseAxis records, and whether the samples of each cell type can support tests at all, which check_dose_design reports. dose_response then tests every cell type that supports tests, with a fitted curve and its band for each gene, tests of trends, of each dose against the vehicle and of candidate shapes of response, a gate that decides which genes count as responding, which permutation_check checks against permuted doses, and the lowest dose from which each gene responds, and series simulated from known curves with simulate_dose_series show how well a dose method recovers the responses at a design before it is run. Dose response per cell type runs the steps on a simulated series.

Dose axes and the vehicle

Where does the vehicle, the dose of 0, go on a dose axis? A curve of dose usually follows log10 of the dose, on which a half-log series takes equal steps, and 0 has no logarithm. DoseAxis records the dose column of the sample table, its unit, what the dose measures and the axis, and places the vehicle in one of two ways:

vehicleWhere the vehicle goesWhat the design holds
"own_group" (default)In a group of its own, off the axisA column dose_treated, 1 for a dose above 0; the spline of the treated doses, 0 for the vehicle
"on_axis"At log10(offset) on a log10 axis, with offset added to every dose, or at 0 on a linear oneThe spline of every sample’s dose

An offset decides where the vehicle sits and so the slope at the low end. DoseFinding’s linear-in-log model adds one, log(dose + off) Bornkamp et al., 2026, and sci-Plex-GxE added 0.001 to every dose, a value chosen from experiments whose data are not shown McFaline-Figueroa et al., 2024; DoseAxis takes no default, and records the offset given. DoseAxis.positions places doses on the axis, and DoseAxis.doses reads positions back as doses, which prints knots in the dose’s unit.

metric says what the dose measures, because doses of different metrics are not comparable without a conversion:

DoseAxis.to_dict records the axis, with its unit and metric, as plain values.

DoseAxis.spline_basis places the knots on the doses of the samples, in the restricted cubic form by default, and DoseAxis.design_columns gives the columns of each sample. Its default rule, "levels", places the knots at quantiles of the distinct doses, each dose counted once, so a knot can fall between two doses. For a cube of hepatocytes whose sample table holds each sample’s dose in dose, the vehicle and eight doses from 0.01 to 30 ug/kg:

axis = cp.DoseAxis("dose", unit="ug/kg", metric="administered")
basis = axis.spline_basis(cube.samples, df=3)
print(axis.doses(basis.knots), axis.doses(basis.boundary_knots))
# [0.14422496 2.08008382] [1.e-02 3.e+01]
samples = cube.samples.with_columns(axis.design_columns(cube.samples, basis))
columns = ["dose_treated", "dose_s1", "dose_s2", "dose_s3"]
design = cp.Design(samples, "~ " + " + ".join(columns))
result = cp.limma_voom(
    cube,
    design,
    contrasts=columns,
    f_tests={
        "spline": columns,  # any dose against the vehicle
        "among_doses": columns[1:],  # any change among the treated doses
        "nonlinearity": columns[2:],  # departure from a line in log10 dose
    },
)

The two interior knots sit at 0.14 and 2.1 ug/kg, between doses of the ladder, and the boundary knots at the lowest and highest doses. Each F-test has its table, such as result.table("hepatocyte", "spline"), whose f_statistic, p_value and adjusted_p_value columns test each gene: spline whether it responds at all, among_doses whether it changes along the treated doses, monotone or not, and nonlinearity whether that change departs from a line in log10 dose. The tables of the spline columns, and the log2_fold_change_dose_s1 to log2_fold_change_dose_s3 columns of the F-tests’ tables, hold coefficients that mean nothing one by one and change with the basis. log2_fold_change_dose_treated is the jump from the vehicle to the curve at the lowest dose, because every spline column is 0 at the lower boundary knot.

The fitted curve of each gene comes from the columns at a grid of doses and the coefficients in the table:

import numpy as np

table = result.table("hepatocyte", "spline")
at = axis.design_columns({"dose": np.geomspace(0.01, 30.0, 50)}, basis)
curves = sum(np.outer(table[f"log2_fold_change_{name}"], at[name]) for name in columns)

Row i of curves is the fitted log2 fold change of gene i of the table against the vehicle, at each of the 50 doses.

How many spline columns? A spline of df columns needs at least df + 1 treated doses, so df is at most the number of treated doses minus one; at that maximum the curve passes through the mean of every dose group, and spline is the F-test of a factor of dose groups against the vehicle. Fewer columns give a smoother curve, with fewer coefficients to estimate from 2 or 3 replicates per dose, as df=3 on the eight doses above; for a time course of many ordered levels, the limma User’s Guide recommends 3 to 5 (section 9.6.2). With vehicle="on_axis", the design holds the spline columns alone, the vehicle counts among the doses, and df can be one more. The vehicle’s rows of the spline columns change neither the three F-tests nor any fitted curve; they decide only what dose_treated and the intercept mean.

dose_response builds these tests itself, in every cell type at once, as contrasts of a model of the dose groups whose variance the four tests share, beside the other terms of the design (Which genes respond to dose); the columns here serve models it does not build, such as a spline that differs by another term, dose by sex, or a spline on knots of your own.

On a ladder of decades, such as 0.1, 1 and 10, the first restricted cubic column holds the whole numbers 0, 1 and 2, which Design.check warns may be codes of groups (numeric_codes). The warning does not apply to the columns of a dose axis, and check_dose_design takes the design of the other terms, so its report does not carry it.

What a number of dose groups supports

How much can a dose series say? The number of dose groups K, the vehicle included, sets it. A test of any dose against the vehicle has K - 1 degrees of freedom. A spline of dose with the vehicle in a group of its own runs along the K - 1 treated doses, so it has at most K - 2 columns, and K - 1 with the vehicle on the axis (Dose axes and the vehicle). A curve of p parameters fitted to the means of K groups leaves K - p groups to test its fit, so a curve with a spare group needs p at most K - 1. Two documents set numbers of doses: EMA’s qualification of MCP-Mod (multiple comparison procedure and modelling), a method for dose finding in phase 2 trials that tests candidate curve shapes and then fits one European Medicines Agency, 2014, and the standard methods of EPA’s Transcriptomic Assessment Products (ETAP), for transcriptomic studies of chemicals in rats US Environmental Protection Agency, 2024.

KTestsA curve with a spare groupWhat documents ask
2The dose against the vehicleNone
3Any dose, on 2 degrees of freedom; each dose against the vehicle; a spline of 1 columnA lineMCP-Mod’s test step, placebo included
4A spline of up to 2 columnsAn Emax curve, which rises to a maximum effect, of 3 parametersMCP-Mod’s modelling step
5A spline of up to 3 columnsA Hill curve, of 4 parameters
9A spline of up to 7 columnsAny curve of up to 8 parametersETAP: 8 doses and a vehicle

Replicates count too: n samples in each of K groups leave K(n - 1) residual degrees of freedom before moderation, 3 for a vehicle and two doses in duplicate, so the prior of the moderation (Empirical Bayes moderation) carries much of each gene’s variance estimate.

Checking a dose design

Which cell types of a dose series can be tested, and which can only be described? Filtering can leave a dose group without a sample in a depleted cell type Hoffman et al., 2026, a dose processed in a lane or plate of its own cannot be told apart from that batch, which breaks the control of false positives Lun & Marioni, 2017, and a dose given to a single animal has no replicate. check_dose_design reports these per cell type before anything is fitted:

  1. Samples. Each cell type uses the samples that its fit in limma_voom would use, with the same min_cells and min_counts: at least 10 cells there by default, some counts, and at least min_counts counts summed over all features. The expression filter does not run.

  2. Dose groups. The distinct doses of the design’s samples, the vehicle first. A dose group that keeps no sample in a cell type is dropped there; one that keeps a single sample stays, and the doses table shows it. The replicates of a dose group are its samples or, when the design has random intercepts, the levels of each grouping among them: two samples of one animal at a dose, under random="(1 | animal)", are one replicate.

  3. The dose-group model. The design holds the terms other than dose, such as "~ 1" or "~ sex", and the check adds an intercept and an indicator of each treated dose group that keeps a sample. The rank, by R’s qr() rule, gives the residual degrees of freedom. A dose group is estimable when the design can compare it with the vehicle apart from its other terms: when its column lies outside the span of the other columns Searle, 1971, which limma’s nonEstimable reports with the column placed last. Random intercepts are checked on the samples used, as Design.check checks them.

  4. The batch. With batch, a column such as the lane or plate of each sample, a dose group is confounded with the batch when it is estimable and its difference from the vehicle is also a difference between batches. Batches of a single sample are pooled first, since such a batch cannot be told apart from its sample’s own deviation, and a sample alone in its batch, when it is the only one, joins no batch, so a lane per sample confounds nothing. The rule finds exact confounding: a batch unbalanced across the doses without confounding them, as when samples are loaded into lanes in dose order, also biases the comparisons when the model leaves it out, so a known batch belongs in the design whenever it is unbalanced.

Each cell type gets one of three statuses:

StatusWhenWhat it supports
"inferential"An estimable treated dose group has two replicates or more, the model leaves residual degrees of freedom, and every dose group confounded with the batch is compared through a random intercept of itTests with p-values, and curves with bounds
"descriptive"No estimable treated dose group has a second replicate; or no residual degrees of freedom are left; or a dose group is confounded with the batch, and the design does not have the batch as a random intercept over two or more batches of the vehicle and of each such groupCurves that describe the means, without p-values or bounds
"refused"Every sample was left out; the design or the cell type has no vehicle sample; no treated sample is left; no treated dose group is estimable; or the random intercepts cannot be estimated beside the dose groupsNo dose analysis

The replicates that count are those of the treated doses: a replicated vehicle with a single sample at each dose leaves residual degrees of freedom, but no dose shows the variation between replicates given it. A batch that the design holds as a fixed term absorbs the dose groups it confounds, which then are not estimable. One that it holds as a random intercept compares them against the variance between batches when the vehicle and each such dose span two or more batches; with the vehicle alone in one lane, that variance rests on too few lanes, and the cell type stays descriptive.

For a cube of a series modelled on studies of TCDD (dioxin) in mouse liver, 24 mice given the vehicle or one of eight doses from 0.01 to 30 ug/kg, in lanes of two mice, with hepatocytes, stellate cells, which become scarce at the top dose, rare cholangiocytes and macrophages, which the vehicle mice lack:

axis = cp.DoseAxis("dose", unit="ug/kg", metric="administered")
other_terms = cp.Design(cube.samples, "~ 1")
report = cp.check_dose_design(
    cube, other_terms, dose=axis, batch="lane", rules="etap_2024", sex="sex"
)
print(report)
flagged = report.doses.filter(group_id="stellate", n_samples_used=1)
missed = report.minimums.filter(met=False)

batch, rules and sex are optional: cp.check_dose_design(cube, other_terms, dose=axis) checks the samples, dose groups and replicates alone, as an in vitro or clinical series needs.

DoseDesignReport: 4 cell types, 9 dose groups from 0 to 30 ug/kg, batch 'lane',
                  rules etap_2024
  hepatocyte: inferential; 9 of 9 dose groups keep a sample, 2 to 4 each; 15
              residual df; rules: deviation
  stellate: inferential; 9 of 9 dose groups keep a sample, 1 to 4 each; 14
            residual df; rules: deviation, ETAP excludes doses short of 2
            passing samples (M: 1 at 30 ug/kg)
  cholangiocyte: descriptive; 3 of 9 dose groups keep a sample, 1 each; 0
                 residual df; no treated dose group that can be compared with
                 the vehicle keeps a second sample, so no dose has a replicate;
                 rules: deviation, ETAP excludes doses short of 2 passing
                 samples (M: 1 at 0.01 ug/kg, 1 at 0.03 ug/kg, 0 at 0.1 ug/kg, 0
                 at 0.3 ug/kg, 0 at 1 ug/kg, 0 at 3 ug/kg, 0 at 10 ug/kg, 0 at
                 30 ug/kg)
  macrophage: refused; 8 of 9 dose groups keep a sample, 2 to 3 each; 12
              residual df; no vehicle sample is left (4 vehicle samples left
              out); compare the doses with each other with limma_voom; rules:
              deviation

The hepatocytes keep every mouse and 15 residual degrees of freedom. The stellate cells stay inferential though one of the two mice at 30 ug/kg has fewer than 10 of them: eight replicated groups estimate the variance, and flagged shows the group that keeps one mouse at 30 ug/kg. The cholangiocytes keep three mice, one per dose group, so their curves can only describe; the macrophages have no vehicle sample to compare with. The lanes, assigned at random, confound no dose.

dose_response runs this check itself, with the same batch, min_cells and min_counts, and tests the cell types it calls inferential (Which genes respond to dose).

The DoseDesignReport holds four tables:

TableOne row perWhat to look at
summaryCell typestatus and reason; n_dose_groups, the dose groups that keep a sample, and n_dose_groups_estimable, those the design can compare with the vehicle; df_residual
dosesCell type and dose groupGroups with n_samples_used 1, or a low n_cells_min or total_counts_min; estimable False beside a kept group; confounded_with_batch
samplesCell type and sampleused, and reason for a sample left out, as limma_voom words it
minimumsMinimum of the rule setRows with met False, their observed text and when_unmet; group_id None marks a minimum of the design

DoseDesignReport.status reads the status of a cell type, and report.design_check holds Design.check of the design given.

ETAP’s design minimums

Which of EPA’s design requirements does a dose series miss? The standard methods of EPA’s Transcriptomic Assessment Products (ETAP) set minimums for the design and for the samples that pass quality control US Environmental Protection Agency, 2024. With rules="etap_2024" and sex, the column of each sample’s sex, minimums lists those that samples show, each met or not, with what ETAP does when it is unmet in when_unmet:

minimumSectionRequiredWhen unmet, ETAP
dose_levels3.4.1.3At least 8 doses and a vehicledeparts from its design
dose_spacing3.4.1.3Half a log10 between doses, down from the highest, and the lowest dose a full log10 below the second lowestdeparts from its design
sexes3.4.2Male and female animalsdeparts from its design
animals_per_sex_and_dose3.4.2At least 4 animals of each sex in each dose group, the vehicle’s includeddeparts from its design
passing_samples_per_dose3.4.5.2At least 2 passing samples at each dose levelexcludes the dose level and goes on
passing_vehicle_samples3.4.5.2At least 3 passing vehicle samplesdoes not model the dose response

A step counts as half a log10 when its log10 lies within 0.05 of 0.5, and as a full log10 within 0.05 of 1: one decimal place of log10, the precision of “half-log10”, which holds both steps of a 1-3-10 series, 0.48 and 0.52. Animals are counted among the design’s samples, one per animal, and passing samples are those the cell type uses. ETAP defines a dataset by tissue, sex and exposure regimen, so the minimums of passing samples apply to each cell type and sex. DoseDesignReport.rules_status is "deviation" when a minimum of the design or the cell type’s vehicle minimum is unmet, and "met" otherwise: a dose level that ETAP excludes is listed, in minimums and in the printout, without changing it, since ETAP’s procedure goes on without the level. The TCDD series above misses three minimums of the design: one sex, 2 or 3 mice per dose, and steps of 0.48 log10 at its low end, where ETAP asks for a full log10. The other four rows of missed belong to cell types: ETAP excludes the stellate cells’ 30 ug/kg, where one mouse passes, and every dose level of the cholangiocytes, and the cholangiocytes and macrophages keep 1 and 0 of the 3 passing vehicle samples ETAP needs to model a dose response.

Which genes respond to dose

Which genes respond to the dose in each cell type, and how? dose_response answers for every cell type of a cube in one call. It runs check_dose_design and keeps its report as result.design_report, so the check needs no call of its own; fits one moderated model of the dose groups in each cell type that supports tests, by limma_voom (the default), limma_trend or dream_voom; and runs four tests as contrasts of that one fit, so that a gene’s tests share its one moderated variance:

TestQuestionWhat is testedNumerator df
splineDoes the gene respond at all, along a smooth curve of dose?A natural cubic spline of dose fitted to the dose groups’ means, with dose_treated when the vehicle is a group of its ownThe spline’s columns, plus 1 with the vehicle in a group of its own
nonlinearityDoes the response bend away from a straight line?The spline’s restricted cubic columns after the firstThe spline’s columns minus 1
linear_trendDoes expression rise or fall along the axis, and how fast?The slope of a straight line fitted to the dose groups’ means1, a moderated t-test
dose_groupsDoes any dose differ from the vehicle?Every treated dose against the vehicleThe treated doses compared

Read spline first, the default gate (Which genes count as responding): it asks whether a gene responds at all. The linear trend and the non-linearity test describe the responses it finds, and dose_groups asks the same question as spline without a curve. The dose-group test is the one-way analysis of variance of the dose groups with moderated variances, the moderated counterpart of the ANOVA that the genomic dose-response approach of the National Toxicology Program runs as a gate: at least one gene must pass it at a Benjamini-Hochberg FDR below 0.05 for a data set to be analysed further National Toxicology Program, 2018.

The spline is fitted by least squares to the means of the dose groups that a cell type compares, each group weighted by its samples, so that each of its coefficients is a contrast of the dose groups’ differences from the vehicle. Without observation weights or other terms in the design, these are the coefficients of the spline fitted to the samples themselves, as the limma User’s Guide fits a spline of time (section 9.6.2); under voom’s weights the means are those of the weighted fit Law et al., 2014. The knots are placed once, on the doses of the design’s samples, by DoseAxis.spline_basis with its rule "levels", so that the knots and curves of the cell types compare. By default the spline has 2 columns, 3 knots, when the axis holds 4 doses or more, and 1 otherwise: on simulated series of the vehicle and 8 doses, 2 columns found as many responsive genes as 3, and more than 1 column or a spline through every dose group (Validation). spline_df sets another size. A spline with as many coefficients as the dose groups a cell type compares passes through every group’s mean, and its F-test is the dose-group F-test.

The linear trend’s slope, the log2_fold_change of its table, is the change in log2 expression per log10 unit of dose, or per unit of dose on a linear axis. With the vehicle in a group of its own, the line runs through the treated doses only: a gene that jumps at the lowest dose and then stays flat has a slope near 0, and its jump from the vehicle is what spline and dose_groups test. With the vehicle on the axis, the line runs through every dose, the vehicle at its offset, as the regression on log dose of Tukey’s trend test does Tukey et al., 1985, and the slope is per log10 unit of the dose plus the offset.

For a series simulated from known curves (Simulated dose series with known curves), the vehicle and eight doses from 0.01 to 30 ug/kg in three replicates, with hepatocytes, stellate cells that fall to a tenth at the top dose and rare cholangiocytes:

cube, truth = cp.simulate_dose_series(
    [0, 0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30],
    replicates=3,
    cell_types=["hepatocyte", "stellate", "cholangiocyte"],
    cells=[300, 60, 6],
    composition={"stellate": [1, 1, 1, 1, 1, 1, 1, 1, 0.1]},
    rng=7,
)
axis = cp.DoseAxis("dose", unit="ug/kg", metric="administered")
result = cp.dose_response(cube, cp.Design(cube.samples, "~ 1"), dose=axis)
print(result)
spline = result.table("hepatocyte", "spline")
responding = spline.filter(spline["adjusted_p_value"] <= 0.05)
DoseResponseResult (limma_voom): 3 cell types, 2 tested; tests: spline,
    nonlinearity, linear_trend, dose_groups
  dose: ug/kg (administered) on a log10 axis, the vehicle in a group of its own;
        spline of 2 columns, knots at 0.548 ug/kg, boundary knots at 0.01 and 30
        ug/kg; linear_trend slopes per log10 unit of dose
  gate: spline test, Benjamini-Hochberg within each cell type, at 5% FDR
  hepatocyte: 9 dose groups, 27 samples, 817 genes; 88 pass the gate; calls at
              5% FDR: spline 88, nonlinearity 31, linear_trend 81, dose_groups
              77
  stellate: 8 dose groups, 24 samples, 494 genes; 43 pass the gate; calls at 5%
            FDR: spline 43, nonlinearity 9, linear_trend 38, dose_groups 34
  cholangiocyte: not tested, refused: no vehicle sample is left (3 vehicle
                 samples left out); compare the doses with each other with
                 limma_voom

The DoseResponseResult prints the axis, the spline’s knots in doses, the unit of the linear trend’s slope and the gate, which decides which genes count as responding (Which genes count as responding), and for each cell type the dose groups it compares, its samples, the genes it tests, the genes that pass the gate and the genes each test calls at 5% FDR. The hepatocytes compare all 9 dose groups and the spline calls 88 genes, 85 of which respond in truth. The stellate cells keep no sample at 30 ug/kg, so they compare 8 dose groups, and their dose-group test has one degree of freedom fewer. The cholangiocytes keep no vehicle sample of 10 cells or more, so the check refuses them, and the summary gives its reason.

The result’s tables, which DoseResponseResult.table gives by cell type and name, and its summary:

TableOne row perWhat to look at
table(g, "spline"), "nonlinearity", "dose_groups"Genef_statistic, p_value, adjusted_p_value and adjusted_p_value_across_groups; df, the denominator degrees of freedom
table(g, "linear_trend")Genelog2_fold_change, the slope; t, p_value and the adjusted p-values
table(g, "each_dose")Gene and treated doseEach dose against the vehicle: log2_fold_change, standard_error, t, p_value and adjusted_p_value, adjusted within the dose
table(g, "gate")Genepasses_gate, the gate’s call; p_value and adjusted_p_value of the gate’s test
result.summaryCell typestatus, the check’s; tested and reason; n_dose_groups_tested; tests_not_run; n_genes_passing_gate

DoseResponseResult.fit computes a cell type’s fit again, with the values, weights and moderated statistics behind its tables. adjusted_p_value adjusts the p-values of a cell type and test by Benjamini-Hochberg, as limma’s topTable does, and adjusted_p_value_across_groups adjusts each test over every cell type that ran it, one family per test, for a report of calls across cell types. The tests of each dose are adjusted within the cell type and dose only. For each dose against the vehicle with control over a gene’s several doses, as Dunnett’s test gives it, pass the cell type’s moderated fit and its dose contrasts to multiple_contrast_test (Trends, doses against the vehicle and candidate shapes shows how); to call the doses across genes, use DoseResponseResult.lowest_effective_doses (From which dose a gene responds).

A cell type compares the dose groups that keep a sample there and that the design can compare with the vehicle; the provenance lists them with their samples. A test whose contrasts a cell type’s doses cannot estimate is not run there, such as a spline of 2 columns on the vehicle and two treated doses, and tests_not_run names it, with the reason in the record and in the error of result.table. When the dose groups are few, two tests can ask the same question: with the vehicle and two treated doses, a spline of one column and dose_treated pass through the three groups’ means, so the spline’s F-test is the dose-group test; with the vehicle on the axis and three doses, a spline of one column is the straight line. By default a cell type then runs the gate’s test, then the first of spline, dose_groups and linear_trend, and lists the others as not run; a test named in tests runs whatever it repeats.

Curves with a band

What shape does a response take? DoseResponseResult.curve gives the fitted spline of chosen genes at any dose within those a cell type compares, as a change in log2 expression from the vehicle’s mean, with a pointwise 95% band, and plot_dose_response draws it beside each dose’s estimate:

from cytepool.plotting import plot_dose_response

genes = responding.sort("p_value")["feature_id"][:4].tolist()
curve = result.curve("hepatocyte", genes)
ax = plot_dose_response(result, "hepatocyte", genes=genes)

curve has one row per gene and dose, with log2_fold_change, standard_error, ci_lower and ci_upper, by default at 100 doses evenly spaced along the axis over the doses the cell type compares; at takes doses of your own, and genes may be named by symbol. At dose x, the curve is a combination a(x)‘theta of the dose groups’ differences from the vehicle, theta; its standard error is (a(x)’ V a(x))^(1/2) times the gene’s posterior standard deviation, with V the unscaled covariance of theta, each gene’s own under voom’s weights, and the band is the estimate plus and minus the 0.975 quantile of t on the degrees of freedom of the gene’s moderated t-tests, as limma’s topTable(confint = TRUE) forms intervals. The band is pointwise and covers the spline: at each dose, 95% of such bands contain the spline that least squares fits to the true means of the dose groups, which is the true curve where the response has the spline’s shape. Where the spline cannot follow a response, as at a step or the turn of a bell, the band can miss the true curve, so read it beside each dose’s estimate. On series of the vehicle and 8 half-log doses with 3 replicates, the bands held the true change at the doses for 95% of the doses of flat genes but 87% of those of responsive genes, 80% for bells and 81% for steps, while each dose’s own 95% interval held it for 95% (Validation). Over the whole range at once, the curve stays inside the band less often. With the vehicle in a group of its own the curve is 0 at the vehicle, exactly; with the vehicle on the axis the spline need not pass through the vehicle’s mean, and the curve’s value at dose 0 shows how far it misses. curve refuses doses outside those the cell type compares, so the stellate cells above have no curve beyond 10 ug/kg, and it gives none for results of dream_voom, whose fits do not keep the degrees of freedom of each point of a curve.

Trends, doses against the vehicle and candidate shapes

Does a gene’s response grow with dose, which doses differ from the vehicle, and does the response follow one of a few expected shapes? The spline test says that a gene responds, not how. Three multiple contrast tests of the same fit answer these questions, each from the largest of several correlated moderated t-statistics of the gene, whose p-value comes from the multivariate t distribution (Several contrasts of one gene). They cost far more than the F-tests, so they run only when named in tests: on 20,000 genes of the vehicle and 7 doses in three replicates, on two threads, Dunnett’s test and the MCP step each added about 10 s to the 0.9 s of the default tests and the Williams-type test 23 s; with 3 doses the MCP step took 0.6 ms per gene and the other two under 0.1 ms (Timings).

TestQuestionContrasts of the dose groupsRead it when
dunnettDoes any dose differ from the vehicle, and which differs most?Each dose against the vehicle Dunnett, 1955The doses themselves matter; each dose’s own p-value comes from multiple_contrast_test (below)
williamsDoes the response grow with dose?The vehicle against the mean of the top one, two or more doses, each dose weighted by its samples Bretz, 2006Responses are expected to rise or fall in dose order
mcpDoes the response follow one of a few expected shapes?The contrast that best detects each candidate shape: the MCP step, the multiple comparison procedure of MCP-Mod Bretz et al., 2005Pinheiro et al., 2014A few shapes are expected, from earlier studies or the pharmacology

Each test is two-sided, so it finds rises and falls. A gene’s p_value is adjusted for its own contrasts, the single-step max-t test: the chance that the largest of the gene’s t-statistics is at least as extreme as its own Hothorn et al., 2008. adjusted_p_value adjusts these p-values over the cell type’s genes by Benjamini-Hochberg, and adjusted_p_value_across_groups over every cell type that ran the test. Each contrast’s standard error and the contrasts’ correlation come from the gene’s own covariance of its doses’ estimates, voom’s weights included, so the three tests refuse limma’s approximate standard errors (contrast_standard_errors="limma"), which give no such covariance. Under dream_voom each dose has its own degrees of freedom, and the multivariate t takes the smallest of the gene’s (Limits and differences). The Williams-type test is not Williams’ trend test of the rule sets of the National Toxicology Program and EPA’s Transcriptomic Assessment Products, which compares each dose with the vehicle after fitting means that change in dose order (Dunnett’s and Williams-type contrasts).

CandidateShapes holds the guesses of the MCP step, each a curve of the change from the vehicle along dose in one of DoseFinding’s model families Bornkamp et al., 2026: Emax, d / (ED50 + d); sigmoid Emax, the Hill equation d^h / (ED50^h + d^h); exponential, exp(d / delta) - 1, BMDS’s exponential 2 model, a rise that grows with dose; and beta, an umbrella shape that rises to a peak and falls. The candidate exponential shape rises late, as the simulator’s power curves do; the simulator’s exponential curves, BMDS’s exponential 4, level off as Emax shapes do (Simulated dose series with known curves). DoseFinding’s advice for MCP-Mod is that 3 to 7 shapes from 2 to 4 families usually suffice, with no linear shape, which an Emax shape of large ED50 approximates, and that designs have three treated doses or more, with 4 to 7 recommended Bornkamp et al., 2026. EMA’s qualification asks for the vehicle and two doses for the test step, which "mcp" runs, and three for the modelling step, not implemented yet European Medicines Agency, 2014. A shape becomes the contrast c of the dose groups that maximises the non-centrality of its t-statistic, c’mu / sqrt(c’Sc), for the shape’s means mu at the doses and the covariance S of the groups’ means: for groups of n_k samples, c_k is proportional to n_k (mu_k - the size-weighted mean of mu), which optimal_contrasts computes, as DoseFinding’s optContr does. A shape scaled or shifted gives the same contrast.

By default, dose_response spreads five shapes over the design’s doses with CandidateShapes.spread: between the lowest and highest treated dose, along the axis’s positions, two Emax shapes with ED50s a third and two thirds of the way, a sigmoid Emax shape of Hill slope 3 halfway, an exponential shape that reaches half its change at the top dose D at 0.75 D, and a beta shape that peaks halfway, is half its peak halfway between the lowest dose and the peak, and has DoseFinding’s scale of 1.2 D. The beta shape and the slope come from the share of a true response’s squared non-centrality, which sets its power, that the best of the shapes’ contrasts keeps: on simulated bells at 9 dose groups, a median 0.60 with the beta shape, against 0.36 with DoseFinding’s quadratic umbrella or with none, and Hill slopes of 2 to 4 moved the medians by under 0.01. CandidateShapes.means gives each shape’s means at any doses, scaled as DoseFinding’s getResp scales them, CandidateShapes.names names them, and CandidateShapes.to_dict records them. On the series of Which genes respond to dose:

result = cp.dose_response(
    cube,
    cp.Design(cube.samples, "~ 1"),
    dose=axis,
    tests=("spline", "dunnett", "williams", "mcp"),
    rng=1,
)
print(result)
mcp = result.table("hepatocyte", "mcp")
DoseResponseResult (limma_voom): 3 cell types, 2 tested; tests: spline, dunnett,
    williams, mcp
  dose: ug/kg (administered) on a log10 axis, the vehicle in a group of its own;
        spline of 2 columns, knots at 0.548 ug/kg, boundary knots at 0.01 and 30
        ug/kg
  gate: spline test, Benjamini-Hochberg within each cell type, at 5% FDR
  hepatocyte: 9 dose groups, 27 samples, 817 genes; 88 pass the gate; calls at
              5% FDR: spline 88, dunnett 69, williams 73, mcp 92
  stellate: 8 dose groups, 24 samples, 494 genes; 43 pass the gate; calls at 5%
            FDR: spline 43, dunnett 24, williams 26, mcp 49
  cholangiocyte: not tested, refused: no vehicle sample is left (3 vehicle
                 samples left out); compare the doses with each other with
                 limma_voom

In the hepatocytes, the MCP step calls 92 genes, 89 of which respond in truth, the spline 88 (85), the Williams-type test 73 (69) and Dunnett’s test 69 (67). Of the 33 responsive genes with bell-shaped responses, the spline calls 21, the MCP step 19, Dunnett’s test 17 and the Williams-type test 12: pooling the top doses misses responses that turn back down. Over simulated series of the vehicle and 8 doses, the MCP step found 42% and 49% of the responsive genes with 2 and 3 replicates, the spline 39% and 47%, the Williams-type test 27% and 35% and Dunnett’s test 23% and 33% (Validation). The table of each test has one row per gene: contrast, the contrast of the gene’s largest |t|, such as "10 - vehicle", "3 to 30 - vehicle" or a shape’s name, "emax_1"; its t; df, the degrees of freedom of the multivariate t; p_value and p_value_error, its Monte Carlo standard error, at most 1% of it; and the adjusted p-values. For the MCP step, contrast names the candidate whose contrast best matches the observed changes, not a fitted shape: the shapes’ contrasts are strongly correlated, and fitting and choosing a shape is MCP-Mod’s modelling step, not implemented yet. The weights of every contrast on the treated doses are in the cell type’s entry of result.provenance["groups"], under contrast_tests, and DoseResponseResult.candidates gives the shapes. rng sets the Monte Carlo error of the p-values: dose_response records one seed in result.settings["seed"], and each cell type and test draws from a generator seeded by it, so that rng= the recorded seed gives the same p-values whatever the other cell types, the other tests named and the threads; tests of two contrasts or fewer use no random numbers.

The tables give each gene’s p-value over all its doses or shapes. Each dose’s own p-value, adjusted for the gene’s doses as Dunnett’s test adjusts it, comes from multiple_contrast_test on the cell type’s fit, whose estimates include each dose against the vehicle:

import numpy as np

from cytepool.stats.multiple_contrasts import multiple_contrast_test

fit = result.fit("hepatocyte")
names = list(fit.moderated.fit.coefficient_names)
# The fit's estimates: each dose against the vehicle, then the spline's contrasts.
doses = {n: np.eye(len(names))[j] for j, n in enumerate(names) if n.startswith("dose_")}
per_dose = multiple_contrast_test(fit.moderated, contrasts=doses, rng=1)
at_ten = per_dose.table(contrast="dose_10", feature_ids=fit.features["feature_id"])

at_ten has each gene’s estimate, standard_error, t, p_value and p_value_error at 10 ug/kg, and the smallest of a gene’s p-values over its doses is its p-value in the "dunnett" table, within their Monte Carlo errors. To call the doses across genes, use DoseResponseResult.lowest_effective_doses (From which dose a gene responds).

To test shapes of your own, give their parameters in the dose’s unit, such as ED50s from an earlier study:

# The beta shape peaks at 36 x 0.5 / (0.5 + 2) = 7.2 ug/kg.
shapes = cp.CandidateShapes(
    emax=[0.3, 3.0], sigmoid_emax=[(1.0, 2.0)], beta=[(0.5, 2.0)], beta_scale=36.0
)
guessed = cp.dose_response(
    cube,
    cp.Design(cube.samples, "~ 1"),
    dose=axis,
    tests="mcp",
    candidates=shapes,
    rng=1,
)

With the vehicle and K - 1 treated doses, the contrasts of the dose groups point in at most K - 1 directions, so more shapes than treated doses cannot all differ: five shapes on two treated doses span two dimensions, and the printout says so. The MCP step still tests every shape, at the cost of the multiplicity of shapes that duplicate one another; at three dose groups, EMA’s minimum for the MCP step European Medicines Agency, 2014, Dunnett’s test asks nearly the same. A shape that does not change at a cell type’s doses, such as a narrow peak at a dose that the cell type does not keep, is left out there, and the record says why. MCP-Mod applied gene by gene to expression data at eight concentrations found that the sigmoid Emax model fitted the less noisy genes best and simpler models the noisy ones Duda et al., 2022; here the shapes give only contrasts, and fitting them, MCP-Mod’s modelling step, is not implemented yet.

Which genes count as responding

Which genes count as responding to the dose, and what does that call rest on? A dose series runs several tests of each gene, and the genes that respond go on to curves, gene-set roll-ups and points of departure. One test decides: the gate, a test of any response at a stated false discovery rate, run after the expression filter. The filter reads each gene’s counts and library sizes, never the doses, the kind of filter that leaves the error rate of the test after it in place Bourgon et al., 2010. dose_response takes the gate as gate, records it and prints it:

gateWhat it asks of a gene
{"test": "spline"}, the defaultDoes it respond along a smooth curve of dose?
{"test": "dose_groups"}Does any dose group differ from the vehicle? The default when fewer than two doses are on the axis, where no spline is placed
{"test": "dunnett"}, "williams" or "mcp"Does any dose differ from the vehicle, does the response grow in dose order, or does it follow a candidate shape?
{"fold_change": 1.5}Does it change by more than 1.5-fold, up or down, at a dose?

"fdr" sets the level, 0.05 by default: a gene passes when the Benjamini-Hochberg adjusted p-value of the gate’s test, over the cell type’s genes, is at most the level. The gate’s test runs whether tests names it or not. "linear_trend" and "nonlinearity" cannot gate, since they test a straight line or a departure from one, not any response.

gate = result.table("hepatocyte", "gate")
large = cp.dose_response(
    cube, cp.Design(cube.samples, "~ 1"), dose=axis, gate={"fold_change": 1.5}
)
passed = large.table("hepatocyte", "gate")["passes_gate"]
print(f"default gate: {gate['passes_gate'].sum()} genes; 1.5-fold gate: {passed.sum()}")
default gate: 88 genes; 1.5-fold gate: 24

The table "gate" has one row per gene tested: p_value, the p-value of the gate’s test, adjusted_p_value and passes_gate. result.summary counts the genes that pass in n_genes_passing_gate, NaN where the gate’s test did not run, which says nothing of whether the cell type responds. DoseResponseResult.gate gives the gate as dose_response(..., gate=result.gate) takes it back, and the provenance records its adjustment for multiple testing (gate_adjustment) beside its test, level and fold change.

A fold change enters each dose’s test against the vehicle as its null hypothesis, by TREAT McCarthy & Smyth, 2009, instead of a cut on the estimated changes after the test, as the rules of NTP and EPA’s Transcriptomic Assessment Products apply it: a cut controls no error rate for the size of the change. Each dose’s TREAT p-value tests whether its change exceeds the fold change; a gene’s p-value is the smallest of its doses’ times the doses, a Bonferroni adjustment that holds whatever the correlation between a gene’s doses, which the gate records as the test "each_dose"; Benjamini-Hochberg then runs across genes. In the hepatocytes above, 93 of the genes tested change by more than 1.5-fold at one of the doses, in truth. The 1.5-fold gate passes 24 of them and no other gene; the default gate with a cut at 1.5-fold on the estimated changes keeps 86 genes, 77 of those 93, 7 whose true change never exceeds 1.5-fold and 2 flat genes. TREAT asks for evidence that a change exceeds the threshold, so a change must lie well above it to pass, and thresholds of 1.2 to 1.5 are usual (Testing against a fold-change threshold). A gate with a fold change needs limma_voom or limma_trend (Limits and differences).

The gate takes the place of BMDExpress’s prefilter, the step that picks the genes that go on to curves. BMDExpress’s default prefilter takes an unadjusted Williams p-value and cuts the estimated fold change; the gate adjusts its p-values and puts any fold change inside the test. It is not the ANOVA gate of NTP’s and EPA’s rules, a test of the data set that at least one gene must pass before the set is analysed further National Toxicology Program, 2018. DoseResponseResult.lowest_effective_doses calls the doses of the genes the gate passes (From which dose a gene responds); curves keep every gene that the expression filter keeps, and gene-set roll-ups and points of departure, which will admit only the genes the gate passes, are not implemented yet.

Could the calls come from noise?

Could the genes that a gate passes come from noise at this design? In zebrafish embryos exposed to the vehicle alone and placed at random in 4 mock concentration groups of 6 samples, BMDExpress’s default prefilter, a Williams trend test with a 1.5-fold cut, gave a transcriptomic point of departure, the benchmark dose of the 25th most sensitive gene, in 93% of runs Lee et al., 2024. With the concentration labels of a real series permuted 100 times, a workflow whose prefilter took an unadjusted Williams p-value found responsive genes in every permutation, where one with a false-discovery-rate-adjusted Williams test found none in nearly all Bruns et al., 2026. permutation_check asks the question of a result, at its own design:

check = cp.permutation_check(result, n_permutations=100, rng=1)
print(check)
PermutationCheck of the gate (spline test, Benjamini-Hochberg within each cell
    type, at 5% FDR): 3 cell types, 2 checked; seed 1
  hepatocyte: 88 genes pass the gate; over 100 random runs of 1.1e21
              assignments, the gate passed genes in 3 runs (0.03), 0.04 genes
              per run; p = 0.0099, the smallest these runs can give
  stellate: 43 genes pass the gate; over 100 random runs of 3.7e17 assignments,
            the gate passed genes in 3 runs (0.03), 0.09 genes per run; p =
            0.0099, the smallest these runs can give
  cholangiocyte: not checked: cell type 'cholangiocyte' was not tested
                 (refused): no vehicle sample is left (3 vehicle samples left
                 out); compare the doses with each other with limma_voom

For each cell type, the dose labels of the samples it uses are permuted among the samples that the design treats alike, dose_response runs again on that cell type with each permuted labelling, the result’s settings and its gate, and the genes that pass the gate are counted. The check returns a PermutationCheck: PermutationCheck.summary gives for each cell type the genes that the true doses pass, the share of runs that pass genes, the mean genes per run, and the permutation p-value (1 + b) / (1 + B) for the b of B runs that pass at least as many genes as the true doses Phipson & Smyth, 2010. PermutationCheck.table gives each run’s count, and the provenance each run’s assignment, so that any run can be made again with the result’s seed.

Read the p-value first. Under the null hypothesis that no gene responds to dose, every assignment of the labels is as likely as the true one, so a small p-value says that the true doses pass more genes than permuted ones: not all of the gate’s calls are noise, though the p-value does not say which calls are false. Above, the gate passed genes in 3 of 100 runs in each cell type, and no run passed as many genes as the true doses, so p is 1/101, the smallest that 100 runs can give. The share of runs that pass genes says how often the gate passes genes with these samples’ doses shuffled. It estimates how often noise alone passes genes only where no gene responds, as in vehicle-only data or a cell type whose p-value is large, and then only for these samples: on 100 simulated series of the vehicle and three doses in duplicate in which no gene responds, the share averaged 0.16 and varied between series with a standard deviation of 0.08 (Validation). How often a gate passes noise at a design takes many series, as the Validation page runs them, or a vehicle-only series.

Which samples count as alike follows the design. Without random effects the units are the samples, in blocks with the same row of the design’s matrix apart from dose, so that "~ sex" permutes the doses within each sex, and a numeric covariate leaves each sample in a block of its own, which the check lists as not checked. With a random intercept such as (1 | animal), animals that each received one dose are permuted whole, in blocks of animals with the same rows, so that each run keeps the samples and animals of every dose group; donors whose cells received every dose keep their samples, and the doses are permuted within each donor. Each run keeps the doses of the design, so the spline’s knots, the sizes of the dose groups and the genes that the expression filter keeps stay those of the result. Before the runs, the true doses run once through the same path and must give the result’s gate, the same genes with p-values equal up to a relative 1e-9 and the same calls, so that the runs repeat the analysis that the result records; another CPU’s arithmetic moves the p-values by about 1e-13.

A design with few samples allows few assignments: the vehicle and two doses in duplicate allow 6! / (2! 2! 2!) = 90. When the assignments other than the true one number at most n_permutations, the check runs each of them once, which makes its p-value exact. Some runs pass the true doses’ genes whatever the data, because the gate cannot tell their labels from the true ones:

The check counts these ties among its runs, as the runs whose gate gives the true doses’ p-values up to rounding, and the summary gives the smallest p-value the runs can give as smallest_possible_p_value, (1 + t) / (1 + B) for the t of B runs that tie. The Williams-type test and the MCP step tell the doses apart. Dunnett’s test of three or more treated doses computes its p-values by Monte Carlo integration, which leaves its renamings apart by that error, so they do not count. For the dose-group gate:

DesignAssignmentsSmallest possible p-value of the dose-group gate, every assignment run
The vehicle and 1 dose x 262/6, 0.33
The vehicle and 1 dose x 3202/20, 0.10
The vehicle and 1 dose x 4702/70, 0.029
The vehicle and 2 doses x 2906/90, 0.067
The vehicle and 2 doses x 31,6806/1,680, 0.0036
The vehicle and 3 doses x 22,52024/2,520, 0.0095

At the vehicle and two doses in duplicate, with the vehicle in a group of its own as the default axis places it, no p-value of the default gate falls below 0.05, however strongly the genes respond. With random runs, B is n_permutations: the smallest possible p-value is 0.0099 at the default 100 runs when no run ties, and the last two designs reach theirs only with n_permutations of 1,679 and 2,519, which run every assignment.

On series in which no gene responds, simulated at 4 and 9 dose groups with 2 and 3 replicates, the check’s runs passed genes as often as the series themselves did, the excess at 2 replicates included (0.162 of runs against 0.154 of the other series at 4 dose groups in duplicate), and its p-value was at most 0.05 in 1.5% to 6.0% of cell types (Validation).

A series of vehicle-only samples that a design places in mock dose groups, as Lee et al. (2024) studied, is a null series by construction: dose_response and this check run it as they run any series, and every gene the gate passes is a false call. On 200 simulated series of their design, 4 mock dose groups of 6 samples, the default gate passed a gene in 17 of 400 cell types, 4.3%, and a prefilter like the one they studied, a Williams-type p-value below 0.05 with no adjustment across genes and a 1.5-fold change at a dose, passed a gene in every one (Validation).

Each run fits the gate’s test alone, and takes the time of the cell type’s analysis and of that test. On 20,000 genes and 24 samples, on two threads, a run took 0.70 s with the spline gate, so 100 runs take about 70 s per cell type; 1.4 s with a fold change, which fits the cell type once more for TREAT; and 24 s with the Williams-type gate, whose multivariate t probabilities take about a millisecond per gene. Under dream_voom, each run takes the time of dream’s fit (Timings).

From which dose a gene responds

From which dose does each gene respond? ICH E4 asks that the lowest dose recommended for use itself show an effect International Council for Harmonisation, 1994, and toxicology reads the lowest observed effect level (LOEL) of each endpoint. Calling each dose against the vehicle in every gene would multiply the false calls by the doses and the genes, so DoseResponseResult.lowest_effective_doses calls them in two stages, as stageR does Berge et al., 2017:

  1. The gate, the spline by default, passes R of the G genes it tested at its false discovery rate, alpha (Which genes count as responding).

  2. In each gene it passes, the doses’ moderated t-tests against the vehicle, those of the table each_dose, are called by Holm’s step-down procedure at the level R alpha / G that the gate leaves, with Shaffer’s first step: the first multiplier is the gene’s doses minus one, not the doses, since the gate has rejected that every dose equals the vehicle Holm, 1979Shaffer, 1986. Under a gate with a fold change, each dose’s TREAT p-value against that fold change takes the place of its t-test.

The lowest effective dose is the lowest dose called. The calls control the overall false discovery rate (OFDR) at alpha: the expected share of the genes passed with at least one false call, the gate’s or a dose’s, proved under independence between genes Heller et al., 2009 and expected wherever Benjamini-Hochberg controls the false discovery rate Berge et al., 2017. They do not control the share of false calls among all gene-dose calls, which counts a gene once for each dose called. Calling contrasts across genes gives the formula.

lowest = result.lowest_effective_doses("hepatocyte")
called = lowest.filter(lowest["passes_gate"])
top = called.sort("stage_wise_adjusted_p_value").take(range(5))
for gene, dose, change, p, n in zip(
    top["feature_id"],
    top["lowest_effective_dose"],
    top["log2_fold_change"],
    top["stage_wise_adjusted_p_value"],
    top["n_doses_called"],
):
    print(f"{gene}  {dose:>4g} ug/kg  {change:+.1f} log2  p {p:.2g}  {n} of 8 doses")
gene_0631    30 ug/kg  +1.9 log2  p 6.3e-10  1 of 8 doses
gene_0172    30 ug/kg  +2.4 log2  p 1.6e-08  1 of 8 doses
gene_0556    10 ug/kg  +2.1 log2  p 3.6e-08  2 of 8 doses
gene_0412     3 ug/kg  -2.0 log2  p 5.5e-07  3 of 8 doses
gene_0040    10 ug/kg  +1.7 log2  p 7.1e-07  2 of 8 doses

The table has one row per gene tested: passes_gate is the gate’s call; lowest_effective_dose gives the lowest dose called, in the axis’s unit, with that dose’s log2_fold_change and stage_wise_adjusted_p_value; and n_doses_called counts the doses called. The lines above show the five genes of smallest stage-wise adjusted p-value. Of the 88 genes the gate passes in the hepatocytes, 69 have a dose called. A dose is called when its stage-wise adjusted p-value is at most the gate’s level, as stageR counts, and the values hold only at that level, since R depends on it: another level, or another test of any response, is another gate, such as gate={"test": "williams", "fdr": 0.1} in dose_response, rather than the values read against another threshold.

An effective dose here is a dose whose change is detectable at the design, with no threshold on its size but the gate’s fold change: the clinical minimum effective dose usually adds a clinically relevant change, as MCP-Mod’s target dose does Bornkamp et al., 2026. It is a tested dose, not a potency between doses, and it moves with the replicates, the spacing of the doses and the gate’s level; a gene can pass the gate with no dose called, when its response spreads over several doses that each fall short. Under the 1.5-fold gate above, the lowest effective dose is the lowest dose whose change beyond 1.5-fold is called. On simulated series, the overall false discovery proportion of these calls was 0.033 to 0.057 (standard errors 0.006 to 0.011), and the lowest effective dose lay a median of 8.9 to 14 times above the gene’s true benchmark dose of a 1.1-fold change (Validation).

Every dose called, not only the lowest, comes from stage_wise_adjusted_p_values on the same tables, here 167 gene-dose calls:

import numpy as np

from cytepool.stats.multiple_contrasts import stage_wise_adjusted_p_values

each = result.table("hepatocyte", "each_dose")
doses = np.unique(each["dose"])
p = np.asarray(each["p_value"]).reshape(doses.size, -1).T
spline = result.table("hepatocyte", "spline")
adjusted = stage_wise_adjusted_p_values(spline["p_value"], p, alpha=0.05)
calls = adjusted <= 0.05  # genes x doses

Simulated dose series with known curves

How well does a dose method find the genes that respond, the shape of each response and the dose at which it starts, at a given design? Only data whose truth is known can say, and the one single-cell dose benchmark compared tests on cells, without curves Nault et al., 2022. simulate_dose_series draws a cube of pseudobulk counts from known curves of each gene’s expression along dose, and DoseTruth holds the truth. Here the spline F-test of Which genes respond to dose runs on a simulated series, and the truth says which genes it should call:

import numpy as np

cube, truth = cp.simulate_dose_series([0, 0.01, 0.1, 1, 10, 30], replicates=3, rng=1)
axis = cp.DoseAxis("dose", unit=truth.unit, metric=truth.metric)
result = cp.dose_response(cube, cp.Design(cube.samples, "~ 1"), dose=axis)

tested = result.table("cell_type_1", "spline")
called = tested.filter(tested["adjusted_p_value"] < 0.05)["feature_id"]
genes = truth.table("cell_type_1")
genes = genes.filter(np.isin(genes["feature_id"], tested["feature_id"]))
hit = np.isin(genes["feature_id"], called)
size = np.abs(genes["max_log2_change"])
responsive, flat = size > 0, size == 0
print(
    f"{hit[responsive].sum()} of {responsive.sum()} responsive genes called, "
    f"{hit[flat].sum()} of {flat.sum()} flat"
)
for change in (0.5, 1, 2):
    chosen = size == change
    print(f"maximal change {change}: {hit[chosen].sum()} of {chosen.sum()} called")
# 62 of 113 responsive genes called, 3 of 461 flat
# maximal change 0.5: 4 of 32 called
# maximal change 1: 24 of 38 called
# maximal change 2: 34 of 43 called

Of the 574 genes that pass the expression filter, the spline’s F-test calls 62 of the 113 that respond and 3 of the 461 flat ones at 5% FDR: with 3 replicates per dose and 100 cells per profile, nearly half of the responses go unseen while false calls stay rare. 28 of the 51 misses are changes of 0.5 log2 units, of which 4 in 32 are called, against 34 in 43 changes of 2. The same lines with replicates=4 or cells=30 show what another design gains or loses before it is run.

The cube has one group per cell type and one sample per replicate, in dose order. Its sample table holds sample_id, such as d2_r1 for the first replicate of the third dose; dose, with 0 for the vehicle, which DoseAxis reads as it stands; dose_group, d0 to d5 here, which sorts in dose order with the vehicle first, for designs with a factor of dose groups; and donor. The layer counts and n_cells hold each profile’s counts and cells, and the provenance records every argument. Each profile sums the counts of a random number of cells:

LevelModelDefault spread
A gene’s expression at the vehicleExpected counts per cell, log-normal, partly shared by the cell typesMedian 0.1 counts per cell; SD 1.5 (natural log) shared and 0.5 per cell type
A gene’s variabilityA log-normal factor on biological_sd and donor_sdSD 0.3 (natural log)
Cells per profilePoisson, with a log-normal mean of cells times the composition factor of the doseThe Poisson mean’s median 4.4% below cells, its coefficient of variation 0.31; counting adds 1 / cells to the squared coefficient of variation
Sequencing depthA log-normal factor with mean 1SD 0.2 (natural log)
Biological effectNormal in log2 units, per replicate (biological_sd), plus a donor effect shared by the doses when donors are crossed with dose (donor_sd)biological_sd 0.25
CountsNegative binomial, the sum of the profile’s cellsDispersion 0.5 per cell, so 0.5 / n for n cells

Profiles of fewer cells vary more beyond their counts, so precision follows the cells per profile along dose, as it does for a cell type that a dose depletes. Replicates differ by effects on every gene, as in the simulations of muscat and scDesign3 Crowell et al., 2020Song et al., 2024. A share responsive of the genes respond, with these curves of the change in log2 expression from the vehicle, where M is the maximal change (max_log2_change), E the EC50 (ec50), the dose of half the maximal change, n the slope (slope), P the peak dose (peak_dose), w the bell’s SD along log10 dose (peak_width) and T the top dose:

shapeChange at dose dFitted families that contain itDefault share
hillM / (1 + (E/d)^n)BMDS’s Hill, drc’s LL.4, DRomics’ Hill30%
exponentialM (1 - 2^(-d/E))BMDS’s exponential 4, DRomics’ exponential15%
linearM d / TBMDS’s linear and power, DRomics’ linear5%
powerM (d/T)^nBMDS’s power10%
log_gauss_probit (a symmetric bell)M exp(-(log10(d/P) / w)^2 / 2)DRomics’ log-Gauss-probit20%
frechetM 2^(-(E/d)^n)none10%
stepM from d = E on, 0 belownone10%

The fitted families are the curves that BMDS US Environmental Protection Agency, 2026, drc’s four-parameter log-logistic model LL.4 Ritz et al., 2015 and DRomics Delignette-Muller et al., 2023 fit; Hill, sigmoid Emax and four-parameter log-logistic curves are one family Kappenberg et al., 2023. The Fréchet curve, drc’s Weibull type 1 with a negative slope parameter, is the Weibull curve of BMDS’s exponential 5 model turned end for end on a log-dose axis: flat at low doses, then a steep rise and a slow approach to its plateau. Steps and Fréchet curves lie outside every family, so a method is not checked only on curves of its own form. A truth inside a family can still lie outside a tool’s bounds: on doses scaled to the top dose, BMDS bounds the Hill EC50 at 5, which 9% to 11% of Hill truths exceed, and the rate of its exponential 4 model below 100, an EC50 above 0.007 times the top dose.

The maximal changes max_log2_changes, 0.5, 1 and 2 log2 units by default, a 1.4-, 2- and 4-fold change, come in equal shares, half rises and half falls.

DoseTruth.table gives each gene’s shape, parameters, expected counts per cell at the vehicle, expression_sd and two benchmark doses, the doses at which the true curve first departs from the vehicle by a stated change, the benchmark response (BMR) Crump, 1984: bmd_expression_sd at one expression_sd and bmd_fold_change at a fold change of 1.1 in either direction, BMDS’s two defaults for continuous data, one SD of the control and a change of 10% US Environmental Protection Agency, 2026, with the fold change taken on the log2 scale. DoseTruth.curve gives the true change at any dose, and DoseTruth.benchmark_doses the benchmark dose at any change in log2 expression, such as the 1.349 SDs of EPA’s Transcriptomic Assessment Products (ETAP); a rise of 10% is log2_change=np.log2(1.1), not 0.1. A method’s estimates can then be compared with the truth gene by gene: their bias, and the coverage of their intervals, the share of intervals that contain the true value over simulated datasets.

The benchmark dose of a 1.1-fold change of Hill, exponential, Fréchet and step curves is log-uniform from an eighth of the log10 span of the positive doses below the lowest dose to an eighth above the top dose, so that a tenth lies below the doses and a tenth above. The EC50 follows from it, so about a quarter of Hill and exponential EC50s lie above the top dose, where the doses never reach half the maximal change, and over a third at 4 dose groups. Slopes are log-uniform from 0.5 to 4, and from 1 to 4 for power curves, which reach their maximal change at the top dose, as linear curves do.

Bells peak within the doses, with an SD along log10 dose of 0.75 to 1.5 times the gap between the doses on either side of the peak, so that 98% to 99% reach half their peak at two or more doses, at six designs of 3 to 9 dose groups, evenly spaced or not. Their benchmark dose, the lower crossing, lies below the lowest dose more often in sparse designs: in 31% of bells at nine dose groups (the vehicle and eight half-log doses) and 94% at four (the vehicle and three doses a decade apart) (Validation).

expression_sd is the SD of true expression between vehicle replicates, from biological and donor variation, before counting, and every truth is a change in true expression. Counting widens the SD of observed log-CPM values most for genes with few counts: at 100 cells per profile, 3.6 times for genes with 1 to 5 expected counts at the vehicle and 1.2 times above 100. The prior count of log-CPM values compresses the observed changes of the same genes, falls more than rises: at the top dose and 100 cells, to 0.40 of the true change below 1 expected count, 0.80 from 1 to 2 and 0.95 from 2 to 5, and to 0.99 or more from 5 counts on (Validation).

For SDs on the observed scale, simulate the same seed with many vehicle replicates, such as replicates=[2000, 3, 3, 3, 3, 3], which keeps the truth, and pass a multiple of each gene’s observed SD to DoseTruth.benchmark_doses. That pairs an observed SD with the true curve, so compare such benchmark doses only for genes with at least 5 expected counts at the vehicle and the top dose, or draw the observed curve, the mean log-CPM at each dose, from a series with many replicates of every dose.

The random draws come in independent streams, so a cell type’s curves and expression stay the same when replicates, cells, composition, biological_sd or donor_sd change, or another cell type is added: designs compare on the same genes. Doses multiplied by a factor give EC50s, peak doses and benchmark doses multiplied by it, and the same counts.

Defaults and choices

Limits and differences

Checks and further reading

Dose axes, dose designs, dose tests and simulated dose series are compared with R, limma, scanpy, mpmath and published data, run as separate programs on the same inputs:

Further reading:

Gene-set tests begins the part that turns the results for genes into results for pathways and regulators.

References
  1. Bornkamp, B., Pinheiro, J., Bretz, F., Sandig, L., Thomas, M., & Sabanés Bové, D. (2026). DoseFinding: Planning and Analyzing Dose Finding Experiments. R package version 1.4-2, CRAN. https://cran.r-project.org/package=DoseFinding
  2. McFaline-Figueroa, J. L., Srivatsan, S., Hill, A. J., Gasperini, M., Jackson, D. L., Saunders, L., Domcke, S., Regalado, S. G., Lazarchuck, P., Alvarez, S., Monnat, R. J., Shendure, J., & Trapnell, C. (2024). Multiplex single-cell chemical genomics reveals the kinase dependence of the response to targeted therapy. Cell Genomics, 4(2), 100487. 10.1016/j.xgen.2023.100487
  3. European Medicines Agency. (2014). Qualification Opinion of MCP-Mod as an efficient statistical methodology for model-based design and analysis of Phase II dose finding studies under model uncertainty (Qualification Opinion EMA/CHMP/SAWP/757052/2013). European Medicines Agency, Committee for Medicinal Products for Human Use. https://www.ema.europa.eu/documents/regulatory-procedural-guideline/qualification-opinion-mcp-mod-efficient-statistical-methodology-model-based-design-analysis-phase-ii_en.pdf
  4. US Environmental Protection Agency. (2024). Standard Methods for Development of EPA Transcriptomic Assessment Products (ETAPs) (EPA Report EPA/600/X-23/083). US Environmental Protection Agency, Office of Research. 10.23645/epacomptox.25365496
  5. Hoffman, G. E., Lee, D., Bendl, J., Prashant, N. M., Hong, A., Casey, C., Alvia, M., Shao, Z., Argyriou, S., Therrien, K., Venkatesh, S., Voloudakis, G., Haroutunian, V., Fullard, J. F., & Roussos, P. (2026). Efficient differential expression analysis of large-scale single-cell transcriptomics data using Dreamlet. Nature Communications, 17(1), 9597. 10.1038/s41467-026-75680-8
  6. Lun, A. T. L., & Marioni, J. C. (2017). Overcoming confounding plate effects in differential expression analyses of single-cell RNA-seq data. Biostatistics, 18(3), 451–464. 10.1093/biostatistics/kxw055
  7. Searle, S. R. (1971). Linear Models. Wiley.
  8. National Toxicology Program. (2018). NTP Research Report on National Toxicology Program Approach to Genomic Dose-Response Modeling (NTP Research Report No. 5). National Institute of Environmental Health Sciences. 10.22427/ntp-rr-5
  9. Law, C. W., Chen, Y., Shi, W., & Smyth, G. K. (2014). voom: precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biology, 15(2), R29. 10.1186/gb-2014-15-2-r29
  10. Tukey, J. W., Ciminera, J. L., & Heyse, J. F. (1985). Testing the statistical certainty of a response to increasing doses of a drug. Biometrics, 41(1), 295–301. 10.2307/2530666
  11. Dunnett, C. W. (1955). A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association, 50(272), 1096–1121. 10.1080/01621459.1955.10501294
  12. Bretz, F. (2006). An extension of the Williams trend test to general unbalanced linear models. Computational Statistics & Data Analysis, 50(7), 1735–1748. 10.1016/j.csda.2005.02.005
  13. Bretz, F., Pinheiro, J. C., & Branson, M. (2005). Combining multiple comparisons and modeling techniques in dose-response studies. Biometrics, 61(3), 738–748. 10.1111/j.1541-0420.2005.00344.x
  14. Pinheiro, J., Bornkamp, B., Glimm, E., & Bretz, F. (2014). Model-based dose finding under model uncertainty using general parametric models. Statistics in Medicine, 33(10), 1646–1661. 10.1002/sim.6052
  15. Hothorn, T., Bretz, F., & Westfall, P. H. (2008). Simultaneous inference in general parametric models. Biometrical Journal, 50(3), 346–363. 10.1002/bimj.200810425