Fail Open or Fail Closed?

When a permission, quota, fraud, or policy check times out, your code still has to choose. Allowing the action is fail-open. Denying it is fail-closed.

There is no safe global answer. Choose per dependency and per protected action.

Build the decision matrix

For each check, record:

• what action it protects;
• the harm of allowing it incorrectly;
• the harm of denying it incorrectly;
• the timeout and retry budget;
• the default on timeout, error, and stale data;
• whether a cached decision is allowed;
• the owner and recovery procedure.

Authentication, permissions, spend caps, data deletion, and other destructive actions normally fail closed. A recommendation widget or non-sensitive enrichment may fail open by skipping the enhancement. A checkout tax dependency may need a queued/manual state rather than a fake yes or no.

Do not let an SDK exception silently choose the policy. Catch the failure, map it to the documented state, show callers that the system is degraded, and alert the owner.

Timeouts, retries, and break glass

Every dependency call needs a bounded timeout. Retrying inside several service layers can multiply one outage into a retry storm, so give one layer ownership of the retry budget.

A break-glass path is an explicit, temporary exception for a named operator and action. It needs approval, a reason, an expiry, and an audit record. It is not a hidden environment variable that permanently converts a security check to fail-open.

Recovery matters too. Decide whether queued actions resume automatically, require revalidation, or remain blocked after the dependency returns.

Verification checklist

1. Force each dependency to time out. Assert the exact allow, deny, or queue result.
2. Return an error and stale response. Confirm those states do not accidentally share a default.
3. Verify the user receives an honest degraded-state message.
4. Confirm one alert fires with the dependency, action, and chosen default.
5. Exercise the break-glass path, then prove it expires and leaves an audit record.
6. Restore the dependency and prove queued work follows the documented recovery path.
7. Load-test the outage and confirm retries remain bounded.

The dry-run starter returns a failure-default matrix, timeout policy, degraded behaviour, alert plan, break-glass procedure, and outage tests. It never changes permissions or performs a protected action.

Download the runnable pack

• Complete workflow pack ZIP
• Importable n8n workflow
• GitHub validation workflow
• GitHub manual run workflow
• Docker Compose file
• Fictional sample payload
• Expected safe dry-run result

Comment FAILCLOSED for the pack. The important part is making the failure decision before the failure arrives.