Steer IT through a crisis without losing direction

Three good intentions can produce an unsafe recovery

In this teaching scenario set in 2020, invoicing becomes unstable at 09:06. Paul, the infrastructure lead, increases capacity. Sonia, the application lead, prepares to restart pending processing. The integrator plans to roll back yesterday's release. Each action makes sense within one scope; together they alter three variables and obscure their effects. Amel, the programme director, first stops competing changes rather than requesting a better dashboard.

Written in 2026, this retrospective examines a 2020 context in which continuity, team availability and urgent decisions could override transformation schedules. The people, times and volumes are illustrative, not a CYTIZEN case study. The subject is operational command across business, IT and suppliers: recover safely, then protect worthwhile programmes without continuing infeasible commitments through inertia.

09:10: regain control of interventions

Amel asks what has started, in which environment and with which rollback. Paul has already increased capacity; Sonia has not restarted processing; the integrator awaits approval. The incident recorder retains these distinct states instead of merging them into actions in progress. Sonia pauses. The integrator keeps its rollback ready but does not execute it. Paul confirms the completed change and before-and-after measures. This coordination pause prevents the team losing its last interpretable evidence.

A technical lead sequences tests, a business decision maker accepts recovery risk, and a communication owner prepares updates. Amel coordinates dependencies and cross-team decisions without replacing the infrastructure specialist or Finance. Their mandates define who can stop processing, authorise limited recovery and accept further delay. Otherwise, people wait for senior approval on technical details and then make business risk decisions alone to save time.

09:20: describe business impact rather than an application incident

Finance distinguishes issued invoices, calculated but unsent invoices and uncertain-status cases. A total amount is insufficient: some can wait while others meet contractual deadlines. Sonia extracts identifiers and timestamps without replaying transactions. Business owners define authorised categories. Support must not advise duplicate entry as a workaround. Tickets retain attempted operations and original references rather than becoming evidence in an argument about whose system remained available.

The illustrative backlog contains 240 cases, including 30 with uncertain status. These are decision examples, not estimates of real activity. The first recovery excludes the uncertain cases. Amel explains why processing fewer cases can be safer than announcing complete restoration while creating duplicates. A signed exclusion list provides stronger decision evidence than a recovery percentage; residual risk remains visible.

09:35: one intervention, one hypothesis, one check

First assess the completed capacity change. If a test case still fails, insufficient capacity is not declared the only cause. Sonia then tests limited processing on the current release under authorised conditions. A rollback requires compatibility checks with data already produced. Yesterday's software does not automatically restore today's transactions to their previous state. Separating configuration, code and data prevents an accessible application with inconsistent postings.

Every intervention records an identifier, start, finish, expected result and stop condition. The technical lead explicitly retires disproved hypotheses. Multiple contributing causes do not justify changing everything simultaneously. An urgent action may precede full diagnosis when its risk is understood and scope limited. The log preserves this uncertainty so later analysis does not pretend the team already knew the answer.

In practice: the competing-change log

10:00: communicate usable uncertainty

Finance is told which categories can resume, which remain suspended and who handles uncertain cases. Management receives the principal risk, selected alternative and next evidence. The supplier receives its test window, authorised access and intervention approver. Amel does not send everybody the same technical note. Information must enable each audience to act without inventing a workaround policy.

The next update has a time even when recovery does not. An inconclusive test is reported plainly: added capacity was insufficient; application testing continues; suspended categories remain unchanged. An invented restoration time shifts disruption to teams organising their work around it. Disciplined communication does not remove uncertainty; it establishes what remains valid until the next checkpoint.

That afternoon: protect capacity for useful programmes

After limited recovery is secured, Amel reviews transformation work drawing on the same specialists. Paul was preparing a migration, Sonia an acceptance test, and the integrator a nonurgent fix. The original schedule no longer establishes feasibility. The sponsor postpones the migration and retains stability work, allocating a different owner where possible. Suspension records retained assets, committed cost, restart conditions and the next review. A paused project must remain restartable.

Every delay must not become another emergency. The team protects recovery time and arranges a handover able to understand the log. When the same specialists cover crisis, operations and projects continuously, human error becomes a dependency of the plan. Management therefore receives actual available capacity, not a named-resource list. Supplier commitments are reconsidered when they depend on people or access no longer available.

Sector differences: recovery criteria cannot be identical

In manufacturing and pharma, IT recovery is assessed against process state, batch status and applicable quality procedures. An application owner may confirm availability without authority to release product. The log separates these acceptances. An IT emergency does not justify bypassing industrial safety boundaries.

In banking and insurance, establishing a financial transaction's status may precede any replay. Evidence includes external reference and reconciliation, not merely a vanished error. In services and public administration, restoring a channel does not clear its backlog. Honest updates identify handling times, priorities and access to assistance or review. Business owners accept the degraded service and duration with the appropriate control functions.

Close the crisis without erasing its debt

Closure separates usable service, reconciled transactions and temporary measures removed or converted to enduring work. Emergency privileged access, extra capacity and manual procedures each have an owner and expiry. Later analysis reconstructs events with the uncertainty participants actually faced. It seeks decisions that could have reduced impact rather than a convenient culprit. A useful conclusion changes a specific mandate, alert, test or recovery mechanism whose effect can be demonstrated in another exercise.

Sources and method

Primary sources checked on 4 October 2026. The year identifies the period being examined; this retrospective was written in 2026. Later documents provide present-day comparisons, not knowledge attributed to that period. People, scenarios and numerical examples are illustrative teaching material, not results of a CYTIZEN engagement.

• NIST SP 800-34 Rev.1 — system contingency planning, 2010
• NIST SP 800-46 Rev.2 — remote access security, 2016