Interpret your results

PertEMA scores which of a predictor's per-perturbation outputs to trust. This page explains what each number on the results table means, so you can act on it without over-reading it. Read the honest scope at the bottom before you draw conclusions.

The reliability score and the calibrated error

Each perturbation gets a calibrated error, an estimate of that prediction's error measured as 1 - Pearson between the predicted and the true effect (so lower is better, 0 is perfect and 1 is uncorrelated). The reliability score is just the negative of the calibrated error, so a higher reliability means a lower expected error. Sort the table by reliability to put the predictions most worth acting on at the top. The estimate is produced from prediction-time features only, so it never sees the true effect of the perturbation it is scoring.

The four reliability bands

The calibrated error is bucketed into four fixed, documented bands. The thresholds are constant across runs so a band means the same thing every time. Each band is always shown with its text label, never by color alone. The colors are from the Okabe-Ito colorblind-safe palette.

Band	Calibrated error (1 - Pearson)	What it means	What to do
high	below 0.80	Among the more trustworthy predictions on this noisy data.	Reasonable to prioritize, still not a guarantee.
moderate	0.80 to 0.88	Usable with caution.	Keep, but corroborate before acting.
low	0.88 to 0.94	Treat with skepticism.	Do not rely on it alone.
very-low	0.94 and above	Likely unreliable.	Deprioritize.

Note the scale: even a high-band prediction here carries a calibrated error below 0.80 in 1 - Pearson, which is far from perfect. The bands rank predictions relative to each other on noisy data. They do not promise that a high-band prediction is accurate in an absolute sense.

Calibration coverage and what 0.90 means

Calibration is checked on held-out data the shipped estimator did not train on. The reported coverage is the fraction of held-out perturbations whose true error actually landed inside the reported interval. The target is 0.90 and the measured empirical coverage is 0.900. So 0.90 is a frequency statement about the intervals as a set, roughly 90 out of every 100 intervals contain the true error, not a probability attached to any single prediction. The isotonic recalibration that produces the calibrated error has a measured expected calibration error of about 0.003, meaning the predicted error levels line up closely with realized error on held-out data.

The conformal interval

Every prediction carries a conformal interval [lo, hi] around its calibrated error. It is a split-conformal band of fixed width (about 0.227 in 1 - Pearson units, with the lower bound clipped at 0) whose width was chosen on held-out data to reach the 0.90 coverage described above. Read it as a plausible range for the realized error of that prediction, not as a tight point estimate. A wide interval that sits high on the error scale is another signal to deprioritize.

Honest scope: read before concluding

Reliability tells you which predictions to trust, not which genes are important. A high band says the predictor's output for that perturbation is more likely to be right, not that the perturbation is biologically meaningful. Those are different axes.
Gains are modest on noisy data. The measured accuracy ceiling of the underlying predictions is modest (the best achievable cross-donor Pearson r is about 0.73 on hit genes and only 0.11 on all genes, higher is better), so the error is noise-dominated and the room for any reliability layer to help is limited.
The bundled example is in-sample and optimistic. It is a slice of the model's own training data, so its scores flatter the method. Your own unseen predictions get honest out-of-sample reliability, which is the number that matters.
PertEMA never retrains on your ground truth. If you supply true effects, they are used only to report realized error back to you, never to refit the shipped estimator.

For the method, the measured comparisons, and the negative results, see the methods and honest scope page and the open benchmark.