PertEMA

PertEMA

A machine-learning reliability layer for single-cell perturbation predictors
Wrap any perturbation predictor and get a per-perturbation reliability score, a calibrated error estimate, and a 90% conformal interval, so you know which predictions to act on.
The scorer is a gradient-boosted-tree estimator with isotonic calibration and split-conformal intervals, trained only on prediction-time features (leakage-safe). It is a reliability layer, not a predictor, and it does not rank biological importance.
Gains are modest on noisy data. The honest out-of-fold reliability and the measured accuracy ceiling are shown on every result.
runs in your browser gradient-boosted trees isotonic calibration split-conformal 90% SHAP attribution leakage-safe features
Runs entirely in your browser. Your predictions never leave your machine — no upload, no server. The model and all scoring run locally, so you can score unpublished predictions privately.
Loading the reliability model (one-time, ~4 MB, then cached)…
Score your own predictions

Upload, paste, or drag a long-format CSV of your predictor's outputs (perturbed_gene, gene, predicted_lfc, optional src_context/dst_context). Download template. Contexts must be Rest, Stim8hr, or Stim48hr.

CSV format help
Columns: perturbed_gene, gene, predicted_lfc (required); src_context, dst_context (optional). One row per (perturbed_gene, gene). predicted_lfc is your model's predicted delta log-fold-change; the perturbation's input magnitude is the mean of |predicted_lfc| over its genes. Contexts must be Rest, Stim8hr, or Stim48hr. Genes may be HGNC symbols or Ensembl IDs. Drag a .csv onto this card to load it.
default contexts if not in CSV: to

or try the built-in example: (training-derived, so its scores are in-sample and optimistic, a demo of the flow, not a quality claim)

How it works (prediction-time only, nothing about the true error reaches the scorer)
Model integrity
Reliability bands (labeled, never color alone)
high
moderate
low
very-low
Risk-coverage (mean predicted error of the kept subset, most reliable first)
Calibration (reliability diagram)
Context-transfer difficulty (the primary contribution: which context transfers are hard)
Triage (keep the most reliable predictions, drop the rest)
100%
Worked example: accept or abstain on a validation shortlist (retrospective, held-out donor truth, from the bundled Gladstone CD4 data)

A wet lab validates a fixed shortlist (top 10 percent of predictions). Which shortlist is more enriched for perturbations whose true destination-context effect is reproducible across donors? We compare the default effect-magnitude shortlist (accept the largest predicted effects) against the PertEMA-reliability shortlist (abstain on the least-reliable predictions), scoring each against cross-donor reproducibility of the true effect as a held-out validation target. This is retrospective and uses held-out donor truth, not a prospective screen.

Before: effect-magnitude default
accept the largest predicted effects
-0.014
reproducibility uplift vs no selection. Magnitude anti-selects reproducibility.
After: PertEMA reliability
abstain on the least-reliable predictions
+0.035
reproducibility uplift vs no selection. 95 percent CI [+0.009, +0.067], positive in 99 percent of bootstraps.

Payoff: on this destination (Stim48hr, donor transfer) the reliability shortlist is +0.049 more reproducible than the effect-magnitude default (positive in 99.5 percent of bootstraps), and +0.049 to +0.063 across the two donor-transfer destinations.

Honest scope: the defensible claim is relative. Magnitude actively anti-selects reproducibility, so reliability yields the safer shortlist. Reliability's absolute enrichment is marginal (threshold-free Spearman +0.014, p 0.59 on Stim48hr, at chance), and the gain is small and bounded by the accuracy ceiling. This does not identify biologically important genes.

model v0.1.0 | calibration coverage 0.900 (target 0.90) | accuracy ceiling Pearson r 0.73 on hit genes, 0.11 on all genes | source: results/utility/reproducibility_uplift.csv (bootstrap B=2000, top 10 percent, n=1563)
Per-perturbation results click a row to see why the model scored it

PertEMA is a reliability estimator, not a prediction-accuracy leaderboard. See the open reliability benchmark at the benchmark page and the methods for the honest scope.