incident responseを検証可能な学習へ変える

到達目標

1. 複数signalを時刻順のevidence timelineへ揃え、影響と意思決定を根拠へ結べる

   • 影響、意思決定、証拠、寄与要因、検証可能な対策を含むレビュー
   • NIST incident responseとGoogle SREのpostmortem cultureを比較する5分発表

2. 個人の性質ではなくsystem conditionとして複数のcontributing factorを分析できる

   • 影響、意思決定、証拠、寄与要因、検証可能な対策を含むレビュー
   • 証拠不足、個人非難、検証不能な対策を診断する回答

3. 検知条件が変わる時に影響を再計算し、対策のowner、期限、検証を更新できる

   • 証拠不足、個人非難、検証不能な対策を診断する回答
   • 検知時刻だけを変えて影響時間と学習行動を再評価した記録

能力の進行

1. recognize

   観測事実、解釈、意思決定、contributing factor、actionを区別できる

   証拠: 影響、意思決定、証拠、寄与要因、検証可能な対策を含むレビュー

2. explain

   non-blaming reviewが責任をなくすのではなくsystem改善を可能にする理由を説明できる

   証拠: NIST incident responseとGoogle SREのpostmortem cultureを比較する5分発表

3. apply

   evidence timelineから影響と意思決定を再構成し対策へ追跡できる

   証拠: 影響、意思決定、証拠、寄与要因、検証可能な対策を含むレビュー

4. diagnose

   後知恵、単一原因、個人非難、action itemの曖昧さを反証できる

   証拠: 証拠不足、個人非難、検証不能な対策を診断する回答

5. lead

   incident response、学習review、対策検証、再発時の更新を職能横断で主導できる

   証拠: 検知時刻だけを変えて影響時間と学習行動を再評価した記録

なぜ重要か

incident responseの第一目的は利用者影響を止め安全に回復することであり、学習reviewの目的は次の対応能力をsystemとして改善することである。記憶だけの物語、唯一のroot cause、担当者への非難では、意思決定時に利用できたsignalと再発を許したsystem conditionを検証できない。

NIST SP 800-61 Rev. 3（April 2025）はincident responseをCybersecurity Risk Managementへ統合する。Google SRE WorkbookのIncident Responseは組織的な応答を、Postmortem Culture — Learning from Failureは学習文化を扱う。ここでは両者をevidence timeline、利用者影響、意思決定、contributing factor、verifiable actionへ落とす。

メンタルモデル

最初にclockとevidence IDを揃え、何がいつ観測されたかだけを並べる。次にincident startからdetectionまでを未緩和のpre-detection impact、incident startからrecoveryまでをtotal durationとして別々に算出し、当時のdecisionをその時点までに利用可能だったevidenceへ結ぶ。contributing factorは個人の性質ではなく、alert、権限、interface、容量、変更境界など変更可能なsystem conditionとして記述する。

non-blamingはaccountabilityを消す言い換えではない。個人の注意力を再発防止策にせず、誰が改善を所有し、どのfixtureとsignalで効果を確認し、失敗時に何を再設計するかを明確にする。

動く例で考える

checkout latency incidentを証拠から再構成する

前提

lesson-defined simulated incidentであり、実利用者、実組織、production logを観測したものではない。限られたevidenceでreview構造とimpact導出を検査するfixtureである。

入力

順序を崩した四つのevidence event、incident start minute、baseline detection minute、recovery minute、毎分のaffected request数、二つのdecision、system condition、action定義と観測済みverification resultを与える。

操作

eventをminute順に並べ、detectionからincident startを引いて未緩和のpre-detection impactを算出する。decisionとactionのevidence IDがtimeline内に存在し、actionのverificationが実際にobserved system outcomeを持つことをfail closedで検査する。

観測

baselineのdetection minuteは10、transferは18であり、検知が8分遅れると同じincident evidenceでも未緩和のimpact minutesが増える。impact式をrecovery minuteだけへ差し替えるとincident-causal-invariantが検知する。

結論

command successだけではreviewは完了しない。impact、evidence-linked decision、non-blamingなsystem condition、verifiable actionが揃い、その後のsystem outcomeで対策を検証できることが学習証拠になる。

python3.13 - <<'PY'
import json

HARNESS = "incident_learning_review_lab_v1"
TRANSFER_TASK = (
    "検知遅延だけを変え、同じincident evidenceから影響時間と"
    "学習行動を再評価する"
)
INCIDENT_EVIDENCE = [
    {
        "evidence_id": "ev-recovery",
        "minute": 48,
        "source": "checkout-sli",
        "observation": "latency and success returned inside SLO",
    },
    {
        "evidence_id": "ev-change",
        "minute": 0,
        "source": "deployment-record",
        "observation": "connection pool configuration activated",
    },
    {
        "evidence_id": "ev-mitigation",
        "minute": 34,
        "source": "incident-command-log",
        "observation": "previous connection pool setting restored",
    },
    {
        "evidence_id": "ev-saturation",
        "minute": 9,
        "source": "database-metric",
        "observation": "connection waiters exceeded alert threshold",
    },
]
BASELINE_ASSUMPTION = {
    "detection_minute": 10,
}
TRANSFER_ASSUMPTION = {
    "detection_minute": 18,
}
IMPACT_INPUT = {
    "incident_start_minute": 0,
    "recovery_minute": 48,
    "affected_requests_per_minute": 24,
}
DECISION_INPUTS = [
    {
        "decision": "declare incident and assign incident commander",
        "minute": 18,
        "evidence_ids": ["ev-saturation"],
    },
    {
        "decision": "restore previous connection pool setting",
        "minute": 31,
        "evidence_ids": ["ev-change", "ev-saturation"],
    },
]
FACTOR_INPUTS = [
    {
        "factor_id": "factor-alert-routing",
        "system_condition": (
            "database saturation alert was not routed to checkout on-call"
        ),
        "individual_blame": False,
        "evidence_ids": ["ev-saturation"],
    },
    {
        "factor_id": "factor-change-guard",
        "system_condition": (
            "connection pool change lacked a saturation stop condition"
        ),
        "individual_blame": False,
        "evidence_ids": ["ev-change", "ev-saturation"],
    },
]
ACTION_INPUTS = [
    {
        "action_id": "action-route-alert",
        "owner": "observability-owner",
        "due": "2026-08-14",
        "verification": (
            "inject saturation signal and observe checkout on-call receipt"
        ),
        "verification_observation": {
            "observed": True,
            "expected_outcome": "checkout-on-call-received",
            "actual_outcome": "checkout-on-call-received",
        },
        "evidence_ids": ["ev-saturation"],
    },
    {
        "action_id": "action-canary-pool",
        "owner": "database-reliability-owner",
        "due": "2026-08-21",
        "verification": (
            "replay pool-change fixture and require saturation rollback"
        ),
        "verification_observation": {
            "observed": True,
            "expected_outcome": "rollback-triggered",
            "actual_outcome": "rollback-triggered",
        },
        "evidence_ids": ["ev-change", "ev-mitigation"],
    },
]
EVENT_FIELDS = {"evidence_id", "minute", "source", "observation"}
DECISION_FIELDS = {"decision", "minute", "evidence_ids"}
FACTOR_FIELDS = {
    "factor_id",
    "system_condition",
    "individual_blame",
    "evidence_ids",
}
ACTION_FIELDS = {
    "action_id",
    "owner",
    "due",
    "verification",
    "verification_observation",
    "evidence_ids",
}
VERIFICATION_FIELDS = {
    "observed",
    "expected_outcome",
    "actual_outcome",
}
IMPACT_FIELDS = {
    "incident_start_minute",
    "recovery_minute",
    "affected_requests_per_minute",
}

def valid_string_list(value):
    return (
        type(value) is list
        and bool(value)
        and all(type(item) is str and item for item in value)
    )

def validate_impact_input(impact):
    if (
        type(impact) is not dict
        or set(impact) != IMPACT_FIELDS
        or not all(
            type(impact[field]) is int
            for field in IMPACT_FIELDS
        )
        or impact["incident_start_minute"] < 0
        or impact["recovery_minute"]
        <= impact["incident_start_minute"]
        or impact["affected_requests_per_minute"] <= 0
    ):
        raise AssertionError(
            "incident-causal-invariant: invalid impact schema"
        )

def validate_incident_source_inputs():
    valid_events = (
        type(INCIDENT_EVIDENCE) is list
        and bool(INCIDENT_EVIDENCE)
        and all(
            type(event) is dict
            and set(event) == EVENT_FIELDS
            and type(event["evidence_id"]) is str
            and bool(event["evidence_id"])
            and type(event["minute"]) is int
            and event["minute"] >= 0
            and type(event["source"]) is str
            and bool(event["source"])
            and type(event["observation"]) is str
            and bool(event["observation"])
            for event in INCIDENT_EVIDENCE
        )
    )
    valid_decisions = (
        type(DECISION_INPUTS) is list
        and bool(DECISION_INPUTS)
        and all(
            type(decision) is dict
            and set(decision) == DECISION_FIELDS
            and type(decision["decision"]) is str
            and bool(decision["decision"])
            and type(decision["minute"]) is int
            and decision["minute"] >= 0
            and valid_string_list(decision["evidence_ids"])
            for decision in DECISION_INPUTS
        )
    )
    valid_factors = (
        type(FACTOR_INPUTS) is list
        and len(FACTOR_INPUTS) >= 2
        and all(
            type(factor) is dict
            and set(factor) == FACTOR_FIELDS
            and type(factor["factor_id"]) is str
            and bool(factor["factor_id"])
            and type(factor["system_condition"]) is str
            and bool(factor["system_condition"])
            and type(factor["individual_blame"]) is bool
            and valid_string_list(factor["evidence_ids"])
            for factor in FACTOR_INPUTS
        )
    )
    valid_actions = (
        type(ACTION_INPUTS) is list
        and bool(ACTION_INPUTS)
        and all(
            type(action) is dict
            and set(action) == ACTION_FIELDS
            and all(
                type(action[field]) is str and action[field]
                for field in (
                    "action_id",
                    "owner",
                    "due",
                    "verification",
                )
            )
            and valid_string_list(action["evidence_ids"])
            and type(action["verification_observation"]) is dict
            and set(action["verification_observation"])
            == VERIFICATION_FIELDS
            and type(
                action["verification_observation"]["observed"]
            )
            is bool
            and type(
                action["verification_observation"]["expected_outcome"]
            )
            is str
            and bool(
                action["verification_observation"]["expected_outcome"]
            )
            and type(
                action["verification_observation"]["actual_outcome"]
            )
            is str
            and bool(
                action["verification_observation"]["actual_outcome"]
            )
            for action in ACTION_INPUTS
        )
    )
    if not valid_events:
        raise AssertionError(
            "incident-evidence-invariant: invalid timeline schema"
        )
    if not valid_decisions:
        raise AssertionError(
            "incident-decision-evidence-invariant: invalid decision schema"
        )
    if not valid_factors:
        raise AssertionError(
            "incident-causal-invariant: invalid factor schema"
        )
    if not valid_actions:
        raise AssertionError(
            "incident-action-outcome-invariant: invalid action schema"
        )
    evidence_ids = [
        event["evidence_id"]
        for event in INCIDENT_EVIDENCE
    ]
    if len(evidence_ids) != len(set(evidence_ids)):
        raise AssertionError(
            "incident-evidence-invariant: duplicate evidence id"
        )
    decision_ids = [
        decision["decision"]
        for decision in DECISION_INPUTS
    ]
    if (
        len(decision_ids) != len(set(decision_ids))
        or any(
            len(decision["evidence_ids"])
            != len(set(decision["evidence_ids"]))
            for decision in DECISION_INPUTS
        )
    ):
        raise AssertionError(
            "incident-decision-evidence-invariant: "
            "duplicate decision evidence"
        )
    factor_ids = [
        factor["factor_id"]
        for factor in FACTOR_INPUTS
    ]
    if (
        len(factor_ids) != len(set(factor_ids))
        or any(
            len(factor["evidence_ids"])
            != len(set(factor["evidence_ids"]))
            for factor in FACTOR_INPUTS
        )
    ):
        raise AssertionError(
            "incident-causal-invariant: duplicate factor evidence"
        )
    action_ids = [
        action["action_id"]
        for action in ACTION_INPUTS
    ]
    if (
        len(action_ids) != len(set(action_ids))
        or any(
            len(action["evidence_ids"])
            != len(set(action["evidence_ids"]))
            for action in ACTION_INPUTS
        )
    ):
        raise AssertionError(
            "incident-action-outcome-invariant: "
            "duplicate action evidence"
        )

def validate_detection_transfer(baseline, transferred):
    for assumption in (baseline, transferred):
        if (
            type(assumption) is not dict
            or set(assumption) != {"detection_minute"}
            or type(assumption["detection_minute"]) is not int
            or assumption["detection_minute"] < 0
        ):
            raise AssertionError(
                "incident-transfer-invariant: invalid assumption schema"
            )
    changed = [
        field
        for field in sorted(set(baseline) | set(transferred))
        if (
            field not in baseline
            or field not in transferred
            or baseline[field] != transferred[field]
        )
    ]
    if changed != ["detection_minute"]:
        raise AssertionError(
            "incident-transfer-invariant: transfer changed wrong input"
        )
    return changed

def materialize_action(source):
    observation = source["verification_observation"]
    system_outcome = (
        observation["observed"]
        and observation["actual_outcome"]
        == observation["expected_outcome"]
    )
    return {
        "action_id": source["action_id"],
        "owner": source["owner"],
        "due": source["due"],
        "verification": source["verification"],
        "evidence_ids": list(source["evidence_ids"]),
        "verification_result": {
            "observed": observation["observed"],
            "expected_outcome": observation["expected_outcome"],
            "actual_outcome": observation["actual_outcome"],
            "system_outcome": system_outcome,
        },
    }

def fixed_incident_evidence():
    # baselineとtransferで同じ証拠を別objectとして構成し、同一参照を
    # 比較してfalse greenになることを防ぐ。
    return {
        "timeline": sorted(
            [
                {
                    "evidence_id": event["evidence_id"],
                    "minute": event["minute"],
                    "source": event["source"],
                    "observation": event["observation"],
                }
                for event in INCIDENT_EVIDENCE
            ],
            key=lambda event: event["minute"],
        ),
        "decisions": [
            {
                "decision": decision["decision"],
                "minute": decision["minute"],
                "evidence_ids": list(decision["evidence_ids"]),
            }
            for decision in DECISION_INPUTS
        ],
        "factors": [
            {
                "factor_id": factor["factor_id"],
                "system_condition": factor["system_condition"],
                "individual_blame": factor["individual_blame"],
                "evidence_ids": list(factor["evidence_ids"]),
            }
            for factor in FACTOR_INPUTS
        ],
        "actions": [
            materialize_action(action)
            for action in ACTION_INPUTS
        ],
    }

def derive_impact(
    detection_minute,
    incident_start_minute,
    recovery_minute,
    affected_rate,
):
    impact_minutes = detection_minute - incident_start_minute
    total_incident_minutes = recovery_minute - incident_start_minute
    # 未緩和時間を元入力へ戻せる加法不変条件で検査し、recovery値を
    # impactへ直接流す置換を検知する。
    if (
        impact_minutes <= 0
        or impact_minutes + incident_start_minute != detection_minute
        or total_incident_minutes < impact_minutes
    ):
        raise AssertionError(
            "incident-causal-invariant: impact derivation bypassed"
        )
    return {
        "duration_minutes": impact_minutes,
        "affected_requests": impact_minutes * affected_rate,
        "total_incident_duration_minutes": total_incident_minutes,
        "semantics": "unmitigated-pre-detection-impact",
    }

def validate_evidence_links(items, evidence_ids, item_kind):
    for item in items:
        if not set(item["evidence_ids"]).issubset(evidence_ids):
            raise AssertionError(
                "incident-causal-invariant: "
                + item_kind
                + " has unknown evidence"
            )

def validate_decision_evidence_time(decisions, timeline):
    evidence_minute = {
        event["evidence_id"]: event["minute"]
        for event in timeline
    }
    for decision in decisions:
        # decision時点より後の証拠を根拠へ混ぜると後知恵になるため、
        # evidence ownershipを時刻境界でもfail closedにする。
        if any(
            evidence_minute[evidence_id] > decision["minute"]
            for evidence_id in decision["evidence_ids"]
        ):
            raise AssertionError(
                "incident-decision-evidence-invariant: "
                "decision references future evidence"
            )

def main():
    validate_impact_input(IMPACT_INPUT)
    validate_incident_source_inputs()
    changed_fields = validate_detection_transfer(
        BASELINE_ASSUMPTION,
        TRANSFER_ASSUMPTION,
    )
    baseline_evidence = fixed_incident_evidence()
    transferred_evidence = fixed_incident_evidence()
    if baseline_evidence != transferred_evidence:
        raise AssertionError(
            "incident-transfer-invariant: fixed evidence drifted"
        )
    timeline = baseline_evidence["timeline"]
    decisions = baseline_evidence["decisions"]
    factors = baseline_evidence["factors"]
    actions = baseline_evidence["actions"]
    evidence_ids = {
        event["evidence_id"]
        for event in timeline
    }
    validate_evidence_links(decisions, evidence_ids, "decision")
    validate_decision_evidence_time(decisions, timeline)
    validate_evidence_links(factors, evidence_ids, "factor")
    validate_evidence_links(actions, evidence_ids, "action")
    if (
        len(factors) < 2
        or not all(
            factor["system_condition"]
            and not factor["individual_blame"]
            for factor in factors
        )
    ):
        raise AssertionError(
            "incident-causal-invariant: non-blaming factors missing"
        )
    if not all(
        action["verification_result"]["observed"]
        and action["verification_result"]["system_outcome"]
        for action in actions
    ):
        raise AssertionError(
            "incident-action-outcome-invariant: outcome not observed"
        )
    baseline = derive_impact(
        BASELINE_ASSUMPTION["detection_minute"],
        IMPACT_INPUT["incident_start_minute"],
        IMPACT_INPUT["recovery_minute"],
        IMPACT_INPUT["affected_requests_per_minute"],
    )
    transferred = derive_impact(
        TRANSFER_ASSUMPTION["detection_minute"],
        IMPACT_INPUT["incident_start_minute"],
        IMPACT_INPUT["recovery_minute"],
        IMPACT_INPUT["affected_requests_per_minute"],
    )
    if (
        transferred["duration_minutes"]
        <= baseline["duration_minutes"]
    ):
        raise AssertionError(
            "incident-causal-invariant: transfer changed wrong cause"
        )
    report = {
        "harness": HARNESS,
        "fixture_metadata": {
            "kind": "simulated",
            "provenance": "lesson-defined checkout incident evidence",
            "limitations": (
                "simulated eventsでありproductionの因果、全signal、"
                "利用者影響を網羅しない"
            ),
            "synthetic_or_observed_explicit": (
                "値はすべてsimulated入力またはその計算結果"
            ),
        },
        "runtime_bound": {
            "records": (
                len(INCIDENT_EVIDENCE)
                + len(decisions)
                + len(factors)
                + len(actions)
            ),
            "subprocesses": 0,
            "maximum_iterations": (
                len(INCIDENT_EVIDENCE)
                + len(decisions)
                + len(factors)
                + len(actions)
            ),
        },
        "external_network_used": False,
        "evidence_timeline": timeline,
        "impact": baseline,
        "decisions": decisions,
        "contributing_factors": factors,
        "verifiable_actions": actions,
        "detection_delay_transfer": {
            "changed_assumption": "detection-delay",
            "changed_fields": changed_fields,
            "same_incident_evidence": (
                baseline_evidence == transferred_evidence
            ),
            "baseline_incident_evidence": baseline_evidence,
            "transferred_incident_evidence": transferred_evidence,
            "baseline_detection_minute": (
                BASELINE_ASSUMPTION["detection_minute"]
            ),
            "transferred_detection_minute": (
                TRANSFER_ASSUMPTION["detection_minute"]
            ),
            "baseline_impact_minutes": baseline["duration_minutes"],
            "transferred_impact_minutes": (
                transferred["duration_minutes"]
            ),
        },
        "command_success_distinction": {
            "command_completed": True,
            "system_outcome_checked": (
                baseline["affected_requests"] > 0
                and transferred["affected_requests"]
                > baseline["affected_requests"]
                and all(
                    action["verification_result"]["observed"]
                    and action["verification_result"]["system_outcome"]
                    for action in actions
                )
            ),
            "command_success_equals_system_outcome": False,
            "outcome_evidence": (
                "impact、evidence links、system factors、action verification"
            ),
        },
        "mastery_evidence": {
            "lab_steps": [
                {"step": 1, "evidence": timeline},
                {"step": 2, "evidence": factors + actions},
                {"step": 3, "evidence": transferred},
            ],
            "assessments": [
                {"assessment": 1, "evidence": factors},
                {"assessment": 2, "evidence": actions},
            ],
            "rubric_dimensions": [
                "technical-correctness",
                "judgment",
                "evidence",
                "communication",
            ],
            "transfer": {
                "task": TRANSFER_TASK,
                "changed_assumption": "detection-delay",
                "evidence": transferred,
            },
        },
    }
    print(json.dumps(report, ensure_ascii=False, sort_keys=True))

try:
    main()
except AssertionError as error:
    raise SystemExit(str(error)) from None
PY

トレードオフと失敗モード

• 誤診: 最初に異常を見た担当者の確認不足がroot causeである。反証: evidenceはalert routingと変更時のstop conditionというsystem conditionを示し、別の担当者でも同じ失敗を許す。
• 誤診: timelineが完成したので因果は確定した。反証: timelineは観測順序を示すが、欠測signalやcounterfactualを含まず、contributing factorは反証可能な仮説として残す必要がある。
• 誤診: 「監視を強化する」をactionにすれば再発防止になる。反証: owner、due、対象signal、検証fixture、成功条件がなければ完了もsystem outcomeも確認できない。

知識チェック

1. evidence timelineで観測事実、解釈、decisionをどう分離するか。
2. non-blamingなcontributing factorと曖昧な一般論をどう区別するか。
3. verifiable actionに必要な四つのfieldとevidenceへの接続を説明せよ。
4. detection minuteだけを変える時、固定すべきincident evidenceは何か。

出典と次の学習

NIST SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management（April 2025）をrisk managementとの接続に使う。Google SRE Workbook: Incident Responseを応答構造に、Postmortem Culture — Learning from Failureをnon-blamingな学習運用に使う。完全な書誌情報と参照先はlesson metadataに分離している。

次はcore-24で、incidentから得た停止条件をfail-closed CI、staged delivery、provenance verification、rollback outcomeへ接続する。

実践ラボ

checkout latency incidentの学習reviewを作る

提出成果物: 影響、意思決定、証拠、寄与要因、検証可能な対策を含むレビュー

1. 固定simulated evidenceを時刻順に並べ、検知から回復までの影響と意思決定を算出する
2. 個人非難を除き、証拠へ結び付くsystem conditionと検証可能なactionを構成する
3. incident evidenceを固定したままdetection minuteだけを変え、影響時間と学習行動を再評価する

説明して理解を確かめる

5分で、NIST SP 800-61 Rev. 3（April 2025）のCybersecurity Risk Managementとの統合と、Google SRE WorkbookのIncident Response、Postmortem Culture — Learning from Failureを、responseとlearningの異なる目的へ結ぶ。

アセスメント

1. 問い: 担当者が確認を忘れたことをroot causeとしてreviewを閉じる提案を評価せよ。

   期待する証拠: 利用可能だったsignal、alert設計、権限、手順、workload、decision context、system condition

2. 問い: 監視を強化するというaction itemを検証可能に書き直せ。

   期待する証拠: owner、due、対象signal、閾値、test fixture、成功条件、evidence ID

別問題へ転用する

検知遅延だけを変え、同じincident evidenceから影響時間と学習行動を再評価する

復習スケジュール

1. 1日後

   evidence timelineで観測事実と解釈を分ける方法は何か

2. 7日後

   non-blamingとaccountabilityを両立するsystem conditionの書き方は何か

3. 30日後

   verifiable actionにowner、due、verificationが必要な理由は何か

4. 90日後

   evidence timelineで観測事実と解釈を分ける方法は何か

評価ルーブリック

出典

以下の外部資料は利用者が選択したときだけ開きます。

• NIST SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management (April 2025) (standard)
• Google SRE Workbook: Incident Response (primary)
• Google SRE Workbook: Postmortem Culture — Learning from Failure (primary)