Frequently asked questions
What is PertEMA?
PertEMA is a post-hoc, model-agnostic reliability layer for single-cell perturbation predictors. It wraps any predictor and estimates the per-perturbation error of that predictor's outputs from prediction-time features only, so it can tell you which predictions to trust. It is not itself a predictor, and it does not identify biologically important genes. The primary deliverable is an open reliability benchmark plus this self-hostable tool. See the methods page for the full method and the measured results.
Does it need my predictor's ground truth?
No. To score your predictions PertEMA uses prediction-time features only, so it needs no true effects at scoring time. The shipped estimator was trained once, on the developers' own data, and it is loaded read-only. If you do supply ground truth, it is used only to report your realized error back to you in an evaluate mode. It is never used to refit the shipped estimator. This is a hard no-leakage rule: a test perturbation's true error never reaches the estimator during training.
What does a low band mean?
The low band means the calibrated error estimate for that prediction falls in the
0.88 to 0.94 range on the 1 - Pearson scale, which reads as treat with
skepticism. It is worse than high and moderate and better than
very-low. Practically, do not rely on a low-band prediction on its own. A band is a relative
ranking on noisy data, not an absolute verdict, so see interpret your results for
the full band table and the caveats.
Why are the gains small?
Because the underlying predictions are noisy and the error is noise-dominated. The measured accuracy
ceiling of the perturbation predictions is modest (the best achievable cross-donor Pearson r is about 0.73 on hit
genes and 0.11 on all genes, higher is better), which leaves little headroom for any reliability layer to exploit. We report
this plainly. A routing procedure that picks the most-reliable predictor per perturbation does not beat the
best single predictor on the noisy primary data, and on single-context Norman the PRESCRIBE method's
intrinsic uncertainty beats PertEMA. The clearest positive result is external and cross-cell-line. Honesty
outranks winning here, so the honest out-of-fold reliability and the accuracy ceiling are shown on every
result.
What about foundation-model pretraining overlap?
A foundation model may have seen a benchmark dataset during its pretraining, in which case strong
apparent generalization can really be memorization. To keep that from being mistaken for a clean result,
every entry in the benchmark carries a pretraining-provenance flag. The shipped estimator here is flagged
CLEAN. When foundation-model results are added, their provenance flag travels with them so a
possible train-test overlap is always visible next to the number.
Accessibility
The interface uses the Okabe-Ito colorblind-safe palette, and every reliability band is labelled with text (high, moderate, low, very-low), never encoded by color alone. Controls are native buttons, inputs, and selects, so they are keyboard operable, the sortable table headers respond to Enter and Space, and keyboard focus is shown with a visible ring. The charts carry text alternatives. A formal automated WCAG AA audit is not yet part of the build and is the documented remaining step.
More detail and the full reference list: the methods and honest scope page and the open benchmark.