Design Partner Program
EXPLAINER · SAFE PRODUCTION TESTING

Safe production testing: prove real exposure without risking real systems.

Attackers don't test staging. Production has the real configuration, the real identities and the real controls, and it's where results matter. Testing it safely comes down to three things: what each test is allowed to do, what stops it automatically, and who approved it.

SAFETY ENVELOPE · RUN 2291env: production
Scope lock 14 paths · 2 identities
Enforced
Safety level L1 read-only · writes blocked
Enforced
Rate cap 20 req/s per service
Enforced
Circuit breaker p95 > 800 ms for 3 s
Armed
Kill switch Last tested 2 days ago
Armed
Approval service owner · change ticket
Recorded
Every limit is set before the run starts. None of them depend on someone watching.
01 · WHY PRODUCTION

Why Test Security in Production? Staging Results Drift

Staging is the right place to test new code. It's the wrong place to answer "are we exposed right now?" Because the things that decide exposure are different in production.

Configuration

STAGINGDefaults, debug flags, permissive CORS.
PRODHardened in some places, drifted in others.

Identity & access

STAGINGA handful of test users and broad roles.
PRODReal role sprawl, stale grants, federated access.

Controls

STAGINGWAF and EDR often in monitor mode or absent.
PRODThe controls that actually stand in an attacker's way.

Network paths

STAGINGFlat, simplified topology.
PRODReal ingress, peering and third-party connections.
02 · THE RISK MODEL

What Can Go Wrong When You Test in Production, and the Control for Each

Name the failure modes up front. Each one needs a control that works without a human in the loop.

RISK HOW IT HAPPENS CONTROL

Service degradation

Test traffic adds load to a service that's already near capacity. Rate cap per service, plus a latency and error-rate circuit breaker that pauses automatically.

Data modification

A test request changes or deletes real records. Safety levels: read-only by default; writes only on tagged test objects with verified rollback.

Data exposure

A successful test retrieves sensitive data into the tool. Capture the minimum, redact before storage, and set retention on evidence.

Scope creep

A test follows a link or redirect to a system nobody approved. Scope lock on hosts, paths and identities; anything outside is denied, not warned.

Alert fatigue

The SOC treats test traffic as a live incident. Tagged traffic and test identities, with the run schedule shared ahead of time.

Third-party impact

Tests reach a vendor or partner API you don't own. Third-party endpoints excluded by default; included only with written permission.
03 · SAFETY LEVELS

Four Safety Levels for Production Security Testing

Classify every test by what it can change. The level decides where it may run, which guardrails apply and who signs off.

EXAMPLESCross-tenant read attempts (BOLA)Auth bypass on GET endpointsControl checks with inert payloads
GUARDRAILS✓Scope lock to approved hosts and paths✓Rate cap and latency circuit breaker✓Responses redacted before storage
RUNS IN PRODUCTION Yes, within limits
APPROVAL Service owner approves the plan once; reruns inherit it.
WHAT IT PROVES That a weakness is exploitable, with request-and-response evidence.

Levels are a recommended model. Map them to your own change-management categories.

04 · A GUARDED RUN

Production Testing Guardrails in Action: A Circuit Breaker Trips

A read-only authorization run against a production API. Midway, the target's latency climbs past the threshold set in the plan. The run pauses itself, waits for recovery, and resumes at a lower rate.

Illustrative run · synthetic data · 20 s replay

billing-api · L1 read-only · 2 identities breaker: p95 > 800 ms for 3 s · cap 20 req/s
Ready
REQ/S 0
P95 LATENCY 180 ms
WRITES 0
00:00 Plan approved by service owner · scope: 14 GET paths · 2 identities
00:01 Run started at 20 req/s
00:06 Finding: tenant_acme can read tenant_globex invoices (API1:2023)
00:09 p95 latency rising on billing-api
00:12 Circuit breaker tripped · p95 > 800 ms for 3 s · run paused
00:15 Latency recovered · resumed at 8 req/s
00:19 Run complete · 0 writes · evidence sealed
00:00 / 00:20
05 · PRE-FLIGHT

Pre-Flight Checklist for Your First Production Security Test

Tick each item off as you confirm it. Anything left open is a reason to start at L0 or L1.

READINESS 3 / 10 HIGHEST LEVEL READYL0 · Observe

Start with passive observation while you close the open items for L1.

06 · WHO SIGNS OFF

Approval and Ownership for Production Security Testing

Safe testing is as much a coordination problem as a technical one. Four roles, each with a clear decision.

01

Security owner

Owns the testing program and the safety policy behind it.

DECIDES · Which levels are allowed where.
02

Service owner

Accountable for the service under test and its availability.

DECIDES · Whether this plan runs on this service.
03

SOC / detection

Sees test traffic and uses it to verify detections.

DECIDES · How test activity is recognized and triaged.
04

Change management

Keeps writes aligned with release and maintenance windows.

DECIDES · When L2 runs are scheduled.
07 · WHERE HAYROK FITS

Governed Validation: Safety Built Into Every Run

Hayrok treats the safety envelope as part of the test, not a policy document beside it. Every scenario carries its level, its limits and its approvals, and the HIVE Agents enforce them at run time.

01

Safety level on every scenario

Each scenario in the library is classified L0–L3; L3 is blocked from production by policy.

02

Enforced scope

Hosts, paths and identities are locked in the plan; agents deny anything outside it.

03

Automatic circuit breakers

Rate caps and latency and error thresholds pause runs without waiting for a person.

04

Approvals in the record

Who approved what, when and for which service is stored with the run.

05

Redacted, replayable evidence

Evidence Fabric stores masked request pairs and every guardrail event.

08 · FAQ

Safe Production Testing FAQ

It can be, if each test is classified by what it can change and enforced limits apply: scope locks, rate caps, automatic circuit breakers, a tested kill switch and recorded approval. Destructive tests should stay out of production entirely.

START AT L1

Validate one production service: read-only, rate-limited, fully recorded.

Pick one service and set the limits together with its owner. We'll run an approved L1 plan and walk you through every request, guardrail event and finding.