How to measure security control effectiveness.

Key findings

Why common control metrics mislead

Each of these is easy to collect, which is exactly why they end up in reports.

The four dimensions of effectiveness

Each dimension answers a different question, so each needs its own number. The path weight underneath keeps all four focused on where attackers would go.

Then the method is short:

Worked example: a path-weighted scorecard in Python

Let's compute the four dimensions for an EDR layer from a week of validation results. Each row in results.csv is one assertion run on one asset.

import pandas as pd

r = pd.read_csv("results.csv")
# columns: asset, path_weight, agent_healthy, expected, observed,
#          drift, drift_detect_h, restore_h

w = r["path_weight"]

coverage   = (r["agent_healthy"] * w).sum() / w.sum()
efficacy   = ((r["observed"] == r["expected"]) * w).sum() / w.sum()
drift_rate = r["drift"].sum() / r["asset"].nunique()
mttd_drift = r.loc[r["drift"], "drift_detect_h"].median()
mttr       = r.loc[r["observed"] != r["expected"], "restore_h"].median()

print(f"coverage   {coverage:.0%}")
print(f"efficacy   {efficacy:.0%}")
print(f"drift      {drift_rate:.2f} events per asset, detected in {mttd_drift:.0f}h")
print(f"restore    {mttr:.0f}h median")

A typical output for an EDR layer looks like this:

Three rules that keep the numbers honest

How to report it

Show each control layer with its four numbers and a 12-week trend. Call out the single worst dimension per layer in plain words, then pair it with an owner and a target date.

For the evidence behind those numbers, follow How to Prove Security Controls Are Working.

How Hayrok approaches control effectiveness

Hayrok computes the four dimensions from evidence, not surveys.

FAQ