DESeq2

DESeq2#

fit_deseq2() estimates DESeq2’s dispersions and fits every gene as DESeq() does; deseq2_wald_test() tests a coefficient or a contrast and deseq2_likelihood_ratio_test() a reduced design against the full one, such as all the levels of a factor at once, each with the columns of results(). deseq2_dispersions() gives the dispersion estimates of every stage, and takes given values for each stage, as DESeq2’s lower-level functions do.

DESeq2’s dispersion estimates and its Wald and likelihood-ratio tests.

DESeq2 (Love, Huber and Anders 2014) models the count y_gi of gene g in sample i as

y_gi ~ NB(mu_gi, alpha_g), var(y_gi) = mu_gi + alpha_g mu_gi^2, log2(mu_gi) = log2(s_i) + x_i’ beta_g,

with s_i the size factor of sample i (by default the median-of-ratios factors of Anders and Huber (2010), median_ratio_size_factors()), x_i its row of the design and beta_g the coefficients of gene g on the log2 scale. fit_deseq2() estimates the dispersions and fits every gene, as DESeq() does; deseq2_wald_test() tests a coefficient or contrast and deseq2_likelihood_ratio_test() a reduced design against the full one, with the columns of results(). They follow DESeq2’s documentation and paper, with DESeq2’s defaults; conventions the documentation leaves open were found by running DESeq2 1.46.0 as a black box and are marked “found by probing”. Genes without counts are left out of every step and get NaN.

Dispersions (deseq2_dispersions(), as estimateDispersions()):

  1. The means of the likelihood. For a one-way layout, whose design rows take as many distinct values as it has columns, the linear model of the normalized counts y / s: s_i times the mean normalized count of the samples with sample i’s row. For other designs the fit described below, at a rough dispersion: the smaller of a moment estimate from that linear model’s residuals, its fitted values raised to at least 1, and one from the mean and variance of the normalized counts (found by probing, as PyDESeq2 computes them). Means below min_mean (0.5) are raised to it.

  2. Gene-wise estimates: the maximum of the Cox-Reid adjusted profile log-likelihood (Cox and Reid 1987; McCarthy, Chen and Smyth 2012), conditional on those means,

    l(alpha) - 1/2 log det(X’ W X), W = diag(mu / (1 + alpha mu)),

    over [1e-8, max(10, n_samples)], the bounds of DESeq2 (found by probing). A gene whose rough dispersion is at the lower bound keeps the bound as its estimate, as in DESeq2, even where the maximum lies above it (found by probing).

  3. The trend of the dispersions on the mean normalized count m, alpha_tr(m) = a0 + a1 / m: a gamma-family generalized linear model with identity link, fitted to the genes whose gene-wise estimate is above 1e-6 (found by probing) and lies within (1e-4, 15) times the current trend, from a0 = 0.1 and a1 = 1, and fitted again until the squared logarithmic changes of a0 and a1 add up to less than 1e-6 after a fit that converged, in at most 11 fits. Each fit is that of R’s glm() with its default control, as DESeq2 makes it: iteratively reweighted least squares from the coefficients of the fit before, stopped when the deviance changes by less than 1e-8 of it or after 25 steps, with a step that makes a mean non-positive halved towards the coefficients before it (found by probing; with these, the coefficients agree with DESeq2’s to rounding). The trend is then evaluated at every gene. With trend="mean", the trend is the mean of the gene-wise estimates above 1e-7, trimmed by 0.1% at each end, instead.

  4. The prior log(alpha_g) ~ N(log(alpha_tr(m_g)), sigma_d^2). With s_lr the median absolute deviation, scaled as R’s mad, of the log residuals log(alpha_gw / alpha_tr) over the genes whose gene-wise estimate is at least 1e-6 (found by probing), and d the residual degrees of freedom, sigma_d^2 = max(s_lr^2 - trigamma(d / 2), 0.25). For d of 3 or fewer the paper matches the density of the log residuals to that of simulated ones, log(chi^2_d / d) + N(0, sigma_d^2); here sigma_d^2 is the maximum likelihood estimate in that model instead, over [0, 8] (the range of DESeq2’s grid), and at least 0.25.

  5. Maximum a posteriori estimates: the maximum of the adjusted log-likelihood plus the logarithm of the prior density, over the same interval and with the same means.

  6. A gene whose log gene-wise estimate exceeds the log trend by more than 2 s_lr is a dispersion outlier: it keeps its gene-wise estimate, and the others take their maximum a posteriori estimate.

Each maximum is found from the sign changes of the derivative on a grid of 30 log dispersions, refined to 1e-12 by the Illinois variant of false position (Dowell and Jarratt 1971), the largest of several being kept. The derivative is computed without cancellation where the dispersion is small: as series in the dispersion and from logarithms of ratios.

Fits (fit_deseq2(), as nbinomWaldTest()): DESeq2’s iteratively reweighted least squares raises fitted values below min_mean to it in the weights and working values, and adds a ridge penalty of 1e-6 beta^2 / 2 on the log2 scale (found by probing). Its fixed point maximizes a concave function: the log-likelihood in which the term of each observation is continued, below a fitted value of min_mean, by its quadratic expansion there in the linear predictor, with the expected information as curvature, less the penalty. Each gene is fitted by Newton’s method on that function, to steps of 1e-10 in the log fitted values; a design that is an intercept alone is fitted by the log of the mean normalized count, as DESeq2 does (found by probing). The covariance of the coefficients is (I + L)^-1 I (I + L)^-1, with I = X’ W X, W = mu / (1 + alpha mu) at the raised fitted values and L the penalty; the hat values are the diagonal of W^(1/2) X (I + L)^-1 X’ W^(1/2). The fitted values reported are not raised.

Cook’s distances are r^2 / p h / (1 - h)^2, with h the hat value, p the number of coefficients and r the Pearson residual (y - mu) / sqrt(mu + a mu^2) at a robust moment estimate a of the dispersion (found by probing, with the constants of PyDESeq2): a = (v - m) / m^2, at least 0.04, with m the mean normalized count and v a robust variance of the normalized counts, the largest over the groups of three or more samples that share a design row of a trimmed mean of squared deviations from the group’s trimmed mean (at each end 1/3 of the samples of a group of 3, 1/4 below 24 and 1/8 from 24, scaled by 2.04, 1.86 or 1.51), or, without such groups, the same over all samples with 1/8 and 1.51.

Tests and results():

  • Wald test: the log2 fold change c’ beta of a contrast c, its standard error sqrt(c’ Sigma c), and the statistic c’ beta / sqrt(c’ Sigma c) with two-sided normal p-values. When a contrast has weights of both signs and every count is zero in the samples with a nonzero entry in a column of positive weight, and in those with one in a column of negative weight, the log2 fold change and the statistic are 0 and the p-value 1 (found by probing; results() documents this for a comparison of two groups). The standard error is kept.

  • Likelihood ratio test: the difference of the deviances of the reduced and the full fit, against the chi-squared distribution with as many degrees of freedom as the full design has more columns. Its log2 fold change and standard error are those of a coefficient of the full fit.

  • A gene’s p-value is removed (NaN) when its largest Cook’s distance over the samples whose design row occurs three or more times exceeds the 0.99 quantile of F(p, n - p); in a comparison of two groups (one factor of two levels, with or without an intercept) not when three or more samples have larger counts than the sample with the largest distance over all samples (found by probing). A numeric covariate of two values is not such a comparison; a design matrix that is not a Design is, when its rows take two distinct values of zeros and ones.

  • Adjusted p-values are Benjamini-Hochberg’s (Benjamini and Hochberg 1995), after independent filtering on the mean normalized count (Bourgon, Gentleman and Huber 2010): at 50 quantiles of the mean counts, from the fraction of genes without counts to 0.95, the adjusted p-values of the genes at or above the quantile, and the number of them below alpha. The quantile kept is the lowest whose number exceeds the maximum of a LOWESS fit (span 1/5) less the root mean square of its residuals, or failing that 90% and then 80% of that maximum (found by probing), or the lowest quantile; and the lowest quantile when no number exceeds 10. Genes below it get NaN.

Deliberate differences from DESeq2, found by probing:

  • DESeq2 maximizes the dispersions by a line search that stops when an iteration gains less than 1e-6, and that can stall where it starts, returning its starting value: its gene-wise estimates are often about 1e-4 from the maximum on the log scale and a few up to 1 or more, and its maximum a posteriori estimates up to about 1e-2 from theirs, relative. Here each is the maximum, but for the genes of step 2 that keep the lower bound. DESeq2’s own maximum a posteriori routine, given a nearly flat prior, finds the same gene-wise maxima.

  • DESeq2’s fits stop when the deviance changes by less than 1e-8 of it: its coefficients up to about 3e-3 from the fixed point, relative to the larger of the coefficient and 1, and the means of its first fit, at the rough dispersion, up to about 1e-3. Here every fit converges.

  • For three or fewer residual degrees of freedom DESeq2 chooses sigma_d^2 from a grid of 1,000 values on [0, 8] by comparing densities of simulated residuals; its choice moves by several percent with the sample of residuals, and is high for small variances (0.55 to 0.58 for 0.5). The maximum likelihood estimate used here is the limit of its criterion as those densities become exact.

  • Where the parametric trend fails, a coefficient not positive or no convergence in 11 fits, DESeq2 substitutes a local regression, which CytePool does not have; the mean trend is used instead, with a warning, as PyDESeq2 does.

Not implemented: the replacement of outlier counts that DESeq() makes for design rows that occur seven times or more (the results here are those of DESeq(..., minReplicatesForReplace = Inf)), shrunken fold changes (lfcShrink), the local trend and glmGamPoi, observation weights, normalization factors that differ between genes, thresholds on the fold change (lfcThreshold) and t-distributed Wald tests (useT).

The constants and the structure taken from PyDESeq2 (Muzellec et al. 2023), in the robust dispersions of Cook’s distances, the rough dispersions and the fallback to the mean trend, are recorded, with PyDESeq2’s license, in THIRD_PARTY_NOTICES.md.

References

Love MI, Huber W, Anders S (2014). Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biology 15, 550. doi:10.1186/s13059-014-0550-8

McCarthy DJ, Chen Y, Smyth GK (2012). Differential expression analysis of multifactor RNA-Seq experiments with respect to biological variation. Nucleic Acids Research 40, 4288-4297. doi:10.1093/nar/gks042

Bourgon R, Gentleman R, Huber W (2010). Independent filtering increases detection power for high-throughput experiments. Proceedings of the National Academy of Sciences 107, 9546-9551. doi:10.1073/pnas.0914005107

Cook RD (1977). Detection of influential observation in linear regression. Technometrics 19, 15-18. doi:10.1080/00401706.1977.10489493

Muzellec B, Telenczuk M, Cabeli V, Andreux M (2023). PyDESeq2: a python package for bulk RNA-seq differential expression analysis. Bioinformatics 39, btad547. doi:10.1093/bioinformatics/btad547

Anders S, Huber W (2010). Differential expression analysis for sequence count data. Genome Biology 11, R106. doi:10.1186/gb-2010-11-10-r106

Cox DR, Reid N (1987). Parameter orthogonality and approximate conditional inference. Journal of the Royal Statistical Society B 49, 1-18. doi:10.1111/j.2517-6161.1987.tb01422.x

Benjamini Y, Hochberg Y (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B 57, 289-300. doi:10.1111/j.2517-6161.1995.tb02031.x

Dowell M, Jarratt P (1971). A modified regula falsi method for computing the root of an equation. BIT 11, 168-174. doi:10.1007/BF01934364

Olver FWJ, Olde Daalhuis AB, Lozier DW, Schneider BI, Boisvert RF, Clark CW, Miller BR, Saunders BV, Cohl HS, McClain MA, eds (2026). NIST Digital Library of Mathematical Functions, release 1.2.8 of 2026-09-15. https://dlmf.nist.gov/

class cytepool.stats.deseq2.DESeq2Dispersions(dispersion, genewise, trended, map_estimate, outlier, all_zero, base_mean, base_variance, means, size_factors, trend, asymptotic_dispersion, extra_poisson, log_residual_variance, prior_variance, df_residual, min_mean)#

DESeq2’s dispersion estimates of many genes, from deseq2_dispersions().

Arrays with a gene axis have one value per gene, in the order of the columns of the counts, and NaN (or False) for genes without counts; arrays with a sample axis are samples x genes. DESeq2’s names are given in parentheses. The arrays are read-only.

Variables:

  • dispersion – (n_features,) final estimates (dispersion): the maximum a posteriori estimate, or the gene-wise estimate of an outlier.

  • genewise – (n_features,) gene-wise estimates (dispGeneEst): the maxima of the adjusted profile likelihood, or 1e-8 for genes whose rough dispersion is at that lower bound.

  • trended – (n_features,) the trend at each gene’s mean normalized count (dispFit).

  • map_estimate – (n_features,) maximum a posteriori estimates (dispMAP).

  • outlier – (n_features,) dispersion outliers (dispOutlier), which keep their gene-wise estimate.

  • all_zero – (n_features,) genes without counts (allZero).

  • base_mean – (n_features,) mean normalized counts (baseMean).

  • base_variance – (n_features,) variances of the normalized counts (baseVar).

  • means – (n_samples, n_features) the fitted means on which the gene-wise and maximum a posteriori estimates are conditional, raised to min_mean (DESeq2’s mu after estimateDispersionsGeneEst).

  • size_factors – (n_samples,) size factors.

  • trend – "parametric", "mean", or "given" for values of the trend given.

  • asymptotic_dispersion – a0 of the trend a0 + a1 / mean (asymptDisp); the mean for the mean trend; NaN for a given trend.

  • extra_poisson – a1 (extraPois); 0 for the mean trend, NaN for a given one.

  • log_residual_variance – s_lr^2, the squared scaled median absolute deviation of the log residuals (varLogDispEsts); NaN when no gene has a gene-wise estimate of at least 1e-6, and then no gene is a dispersion outlier.

  • prior_variance – sigma_d^2, the variance of the prior of the log dispersions (dispPriorVar).

  • df_residual – Residual degrees of freedom, the number of samples less the number of coefficients.

  • min_mean – The lower bound on the means.

Explained in: Count models

class cytepool.stats.deseq2.DESeq2Fit(coefficients, standard_errors, covariance, fitted_values, minus_twice_log_likelihood, hat_values, cooks_distances, max_cooks, converged, iterations, dispersion, dispersion_estimates, size_factors, base_mean, counts, design, coefficient_names, min_mean, replicated, two_groups)#

DESeq2’s negative binomial fits of many genes, from fit_deseq2().

Coefficients, their standard errors and covariances are on the log2 scale, as DESeq2 reports them. Arrays with a gene axis have one row per gene, in the order of the columns of the counts, and NaN for genes without counts; arrays with a sample axis are samples x genes. DESeq2’s names are given in parentheses. The arrays are read-only.

Variables:

  • coefficients – (n_features, n_coefficients) log2 coefficients, one column per design column.

  • standard_errors – (n_features, n_coefficients) their standard errors (SE_...).

  • covariance – (n_features, n_coefficients, n_coefficients) covariance matrices of the coefficients.

  • fitted_values – (n_samples, n_features) fitted means (the mu assay).

  • minus_twice_log_likelihood – (n_features,) -2 times the log-likelihood at the fitted means (deviance).

  • hat_values – (n_samples, n_features) diagonals of the hat matrix (the H assay).

  • cooks_distances – (n_samples, n_features) Cook’s distances (the cooks assay).

  • max_cooks – (n_features,) the largest Cook’s distance over the samples whose design row occurs three or more times (maxCooks); NaN when there are none.

  • converged – (n_features,) whether each fit converged; True for genes without counts.

  • iterations – (n_features,) Newton steps of each fit.

  • dispersion – (n_features,) the dispersions of the fits.

  • dispersion_estimates – The dispersion estimates of every stage, or None when the dispersions were given.

  • size_factors – (n_samples,) size factors.

  • base_mean – (n_features,) mean normalized counts (baseMean).

  • counts – (n_samples, n_features) the counts.

  • design – (n_samples, n_coefficients) design matrix.

  • coefficient_names – Names of the coefficients, the design’s columns.

  • min_mean – The lower bound on the fitted values in the fits.

  • replicated – (n_samples,) samples whose design row occurs three or more times, which count for max_cooks.

  • two_groups – Whether the design compares two groups of one factor, where a gene is not flagged by Cook’s distance when three or more samples have larger counts than the sample with the largest distance.

Explained in: Count models

class cytepool.stats.deseq2.DESeq2Results(base_mean, log2_fold_change, standard_error, statistic, p_value, adjusted_p_value, test, name, contrast, df, cooks_outlier, converged, reduced_converged, cooks_cutoff, alpha, filter_threshold, filter_quantile, filter_quantiles, rejections, rejections_fit)#

One test of every gene, with the columns of DESeq2’s results().

Per-gene arrays have one value per gene, in the order of the columns of the counts; genes without counts have NaN in all of them. The arrays are read-only.

Variables:

  • base_mean – (n_features,) mean normalized counts (baseMean).

  • log2_fold_change – (n_features,) the contrast’s log2 fold change (log2FoldChange); for the likelihood ratio test that of name, a coefficient of the full design.

  • standard_error – (n_features,) its standard error (lfcSE).

  • statistic – (n_features,) the Wald statistic, or the likelihood ratio statistic (stat).

  • p_value – (n_features,) p-values (pvalue); NaN for genes flagged by Cook’s distance.

  • adjusted_p_value – (n_features,) Benjamini-Hochberg adjusted p-values after independent filtering (padj); NaN for genes filtered out.

  • test – "Wald" or "likelihood ratio".

  • name – The name of the contrast or coefficient of log2_fold_change.

  • contrast – (n_coefficients,) its weights over the coefficients.

  • df – Degrees of freedom of the likelihood ratio test; None for a Wald test.

  • cooks_outlier – (n_features,) genes whose p-value Cook’s distance removed.

  • converged – (n_features,) whether the fit of each gene converged (betaConv, or fullBetaConv for the likelihood ratio test); True for genes without counts.

  • reduced_converged – (n_features,) for the likelihood ratio test, whether the fit of the reduced design converged (reducedBetaConv); None for a Wald test.

  • cooks_cutoff – The cutoff on Cook’s distance; infinite when none is applied.

  • alpha – The false discovery rate for which the filter was chosen.

  • filter_threshold – The mean normalized count below which genes are filtered out (filterThreshold); NaN without independent filtering.

  • filter_quantile – Its quantile among the genes (filterTheta); NaN without independent filtering.

  • filter_quantiles – (50,) the quantiles tried; empty without independent filtering.

  • rejections – (50,) the number of adjusted p-values below alpha at each (filterNumRej).

  • rejections_fit – (50,) their LOWESS fit (lo.fit).

Explained in: Count models

cytepool.stats.deseq2.deseq2_dispersions(counts, design, *, size_factors=None, trend='parametric', genewise_dispersions=None, trended_dispersions=None, prior_variance=None, min_mean=0.5, n_threads=None)#

Estimate every gene’s dispersion as DESeq2’s estimateDispersions().

The module description gives the steps: gene-wise estimates, a trend, a prior on the log dispersions around it, and maximum a posteriori estimates. Gene-wise estimates, values of the trend and the prior’s variance may be given in place of their estimates, as DESeq2’s lower-level functions allow (estimateDispersionsGeneEst, a custom dispersionFunction and the dispPriorVar of estimateDispersionsMAP).

Parameters:

  • counts – (n_samples, n_features) counts: whole numbers, finite and non-negative, as DESeq2 requires.

  • design – (n_samples, n_coefficients) design matrix of full column rank with fewer columns than samples, an object with matrix and columns attributes such as a Design, or a table with columns. A Design with random effects is refused.

  • size_factors – (n_samples,) positive size factors. Defaults to the median-of-ratios factors of median_ratio_size_factors(), as estimateSizeFactors() gives them.

  • trend – "parametric" (DESeq2’s default) or "mean".

  • genewise_dispersions – (n_features,) gene-wise estimates to use instead of estimating them: positive and finite for genes with counts, any value or NaN for the others.

  • trended_dispersions – (n_features,) values of the trend to use instead of fitting one, positive and finite for genes with counts.

  • prior_variance – The variance of the prior of the log dispersions, positive and finite, instead of its estimate.

  • min_mean – The lower bound on the means of the likelihood, positive (DESeq2’s minmu).

  • n_threads – The most threads for estimating genes in parallel. None uses numba’s setting for the calling thread. The result does not depend on it, and with one thread no parallel region runs.

Returns:

The estimates of every stage, with the trend and the prior.

Raises:

  • TypeError – If an argument has the wrong type, or is or holds a numpy masked array.

  • ValueError – If shapes disagree; counts are negative, not finite or not whole numbers; no gene has counts; the design is not finite, has random effects, is not of full column rank or leaves no residual degrees of freedom; a size factor is not positive and finite, or every gene has a zero count when size factors are to be estimated; a given dispersion or prior variance is not positive and finite; trend is unknown; or, without prior_variance, no gene has a gene-wise estimate of at least 1e-6 to estimate the prior from.

Explained in: Count models

cytepool.stats.deseq2.deseq2_likelihood_ratio_test(fit, reduced, *, coefficient=None, alpha=0.1, independent_filtering=True, cooks_cutoff=None, n_threads=None)#

Likelihood ratio tests of a reduced design, as nbinomLRT() and results().

The reduced design is fitted with the dispersions and size factors of the full fit, as nbinomLRT() fits it. The statistic, the difference of the deviances, tests all the coefficients the reduced design leaves out at once, such as every level of a factor or every interaction.

Parameters:

  • fit – The fit of the full design, from fit_deseq2().

  • reduced – The reduced design: a matrix of full column rank whose columns lie in the column space of the full design, with fewer columns, or an object with matrix and columns attributes such as a Design built from the same samples (Design(samples, "~ batch")). A formula alone is refused, as the fit keeps the design matrix and not the samples.

  • coefficient – The coefficient of the full design whose log2 fold change is reported; defaults to the last.

  • alpha – The false discovery rate for which the independent filter is chosen, in (0, 1).

  • independent_filtering – Whether to filter genes on their mean normalized count before adjusting the p-values.

  • cooks_cutoff – The cutoff on Cook’s distance, as for deseq2_wald_test().

  • n_threads – The most threads for fitting the reduced design. None uses numba’s setting for the calling thread.

Returns:

The test of every gene.

Raises:

  • TypeError – If fit is not a DESeq2Fit, reduced is a formula, an argument has the wrong type, or is or holds a numpy masked array.

  • ValueError – If the reduced design has the wrong number of rows, is not finite, not of full column rank, not nested in the full design or not smaller than it, has random effects, or coefficient is not a coefficient name.

Used in: Bulk RNA-seq

Explained in: Choosing a method, Count models, Multiple testing

cytepool.stats.deseq2.deseq2_wald_test(fit, contrast=None, *, alpha=0.1, independent_filtering=True, cooks_cutoff=None)#

Wald tests of a coefficient or contrast, as DESeq2’s results() gives them.

Parameters:

  • fit – The fit, from fit_deseq2().

  • contrast – What to test: a coefficient name (DESeq2’s name), an expression in coefficient names such as "groupC - groupB", weights over the coefficients (a vector in their order, or a mapping of one contrast name to such weights or an expression), or a Contrasts holding one contrast, such as design.contrasts.select("stim_vs_ctrl"). Defaults to the last coefficient, which results() tests by default when the design ends with a factor or a covariate.

  • alpha – The false discovery rate for which the independent filter is chosen, in (0, 1).

  • independent_filtering – Whether to filter genes on their mean normalized count before adjusting the p-values.

  • cooks_cutoff – The cutoff on Cook’s distance above which a gene’s p-value is removed. None uses the 0.99 quantile of F(p, n - p); False or infinity applies none.

Returns:

The test of every gene.

Raises:

  • TypeError – If fit is not a DESeq2Fit, an argument has the wrong type, or is or holds a numpy masked array.

  • ValueError – If contrast names no coefficient, does not hold exactly one contrast, has the wrong length or is zero on every coefficient, or an option is out of range.

Used in: Bulk RNA-seq

Explained in: Choosing a method, Count models, Multiple testing

cytepool.stats.deseq2.fit_deseq2(counts, design, *, size_factors=None, dispersion=None, trend='parametric', min_mean=0.5, n_threads=None)#

Fit DESeq2’s negative binomial model to every gene, as DESeq() does.

Estimates the size factors and dispersions unless they are given, fits every gene as nbinomWaldTest() does, and computes the standard errors, hat values and Cook’s distances; the module description gives the methods. Test the fit with deseq2_wald_test() or deseq2_likelihood_ratio_test(). Outlier counts are not replaced: the fit is that of DESeq(..., minReplicatesForReplace = Inf), which is DESeq2’s default where no design row occurs seven or more times.

Parameters:

  • counts – (n_samples, n_features) counts: whole numbers, finite and non-negative.

  • design – (n_samples, n_coefficients) design matrix of full column rank with fewer columns than samples, an object with matrix and columns attributes such as a Design, or a table with columns. A Design with random effects is refused.

  • size_factors – (n_samples,) positive size factors. Defaults to those of dispersion when it is a DESeq2Dispersions, else to the median-of-ratios factors.

  • dispersion – The dispersions of the fits: those of deseq2_dispersions(), or (n_features,) values, positive and finite for genes with counts (NaN or any value for the others). Estimated with deseq2_dispersions() by default.

  • trend – The trend of the dispersions when they are estimated, "parametric" or "mean".

  • min_mean – The lower bound on the fitted values in the weights of the fits and on the means of the dispersion estimates, positive (minmu).

  • n_threads – The most threads for fitting genes in parallel. None uses numba’s setting for the calling thread. The result does not depend on it.

Returns:

The fit of every gene, with the dispersions it used.

Raises:

  • TypeError – If an argument has the wrong type, or is or holds a numpy masked array.

  • ValueError – As deseq2_dispersions(), and if given dispersions do not have one value per gene or are not positive and finite for genes with counts.

Used in: Bulk RNA-seq

Explained in: Choosing a method, Count models