diff --git a/docs/TTX/Tabletop-Exercise-Framework.md b/docs/TTX/Tabletop-Exercise-Framework.md index b42b781..d7c055b 100644 --- a/docs/TTX/Tabletop-Exercise-Framework.md +++ b/docs/TTX/Tabletop-Exercise-Framework.md @@ -23,31 +23,31 @@ Tabletops simulate real threats that could manifest against the organization. T 8. Post-Debrief Actions ## Define Objectives & Scenarios ## -Before the TTX is announced and scheduled, it is important to set the objectives the organization seeks to achieve through the exercise’s activities. This can include such things as “improve our cyber incident preparedness”, “seek areas of automation”, or “demonstrate to our regulators we have an IR plan and regularly test it”. Key objectives, strategies and goals for the exercise should be documented so that as participants are engaged in the process, they all have a clear baseline understanding about the desired outcomes of the TTX. At this time the required subject matter experts are identified to ensure the appropriate people/teams/functions are participating in the exercise. Exercise logistics like length, systems in scope, and after-action steps to be taken are laid out. +Before the TTX is announced and scheduled, it is important to set the objectives the organization seeks to achieve through the exercise’s activities. This can include such things as “improve our cyber incident preparedness”, “seek areas of automation”, or “demonstrate to our regulators we have an IR plan and regularly test it”. Key objectives, strategies and goals for the exercise should be documented so that as participants are engaged in the process, they all have a clear baseline understanding about the desired outcomes of the TTX. At this time the required subject matter experts are identified to ensure the appropriate people/teams/functions are participating in the exercise. Exercise logistics like length, systems in scope, and after-action steps to be taken are laid out. The threat environment of the organization should be reviewed to understand the most likely threat actors and attack scenarios that could impact the organization so that a plausible attack scenario can later be developed with relevant incident injects that test out all areas of the organization’s preparedness. ## Assemble the team ## -Next, the subject matter experts that need to be involved are contacted and briefed on the upcoming TTX. This initial meeting helps lay out all party’s Roles and Responsibilities within the organization and the systems/application/process that is being tested. It is critical to have experiences and knowledgeable participants that have not only have the knowledge of operations and processes, but also can influence changes to those elements after the exercise. +Next, the subject matter experts that need to be involved are contacted and briefed on the upcoming TTX. This initial meeting helps lay out all party’s Roles and Responsibilities within the organization and the system/application/process that is being tested. It is critical to have experienced and knowledgeable participants that have not only have the knowledge of operations and processes, but also can influence changes to those elements after the exercise. As the team is assembled and briefed, documentation on the participant’s Roles & Responsibilities should be shared with all participants. The initial meeting allows the subject matter experts to ask questions about the logistics of the exercise (but not details about the scenario or injects). Everyone should come away with a clear understanding of their role in the exercise, the bounds of what is in or out of scope, and have clear expectations around their participation before, during, and after the exercise. The Team’s participants will respond to the situations presented in the TTX based on their knowledge of the current Incident Response plan. They will draw on existing policies, procedures, and technology. Typical participants asked to be part of the TTX team include network/server administrators, help desk, IT support, CSIRT, PSIRT, managers, directors, legal, HR, business operations, corporate officers and executives. There are five main types of participants: -- Subject Matter Experts (SME) - have an active role in discussing or performing their regular roles and responsibilities during the exercise. SMEs participate in the incident’s execution and discuss the scenario, and recommend best steps to take in response to the simulated emergency. +- Subject Matter Experts (SME) - have an active role in discussing or performing their regular roles and responsibilities during the exercise. SMEs participate in the incident’s execution and discuss the scenario, and recommend best steps to take in response to the simulated emergency. - Observers – these members will not directly participate in the exercise, but provide support to SMEs. They typically are actors outside the operations/incident response team that may be called in for expert opinions for relevant questions that arise during the mock incident. -- Facilitators – act as conductor of the incident, providing situation updates and moderating discussions. They also may provide additional information about the scenario, injects, or possible external outcomes and help resolve questions as required. +- Facilitators – act as conductor of the incident, providing situation updates and moderating discussions. They also may provide additional information about the scenario, injects, possible external outcomes, and help resolve questions as required. - Scribe – It is vital to have a 3rd party not participating in the event witness the TTX and take detailed notes. The Scribe helps track any Action Items that are assigned throughout the course of the exercise, capture discussions, and assist the Facilitator during the After-Action reviews. - Sponsor – It is important that the TTX has a sponsor that helps set the organizational goals the TTX is seeking to explore and resolve. The Sponsor should assist in selecting a relevant scenario, participate as an Observer of the exercise, and ultimately is the party that receives the after-action and debriefing along with the TTX’s documentation. The Sponsor is the party that ultimately determines the effectiveness of the exercise and helps prioritize any improvement actions that follow. The TTX will typically begin with a kick-off call with the Sponsor to set the objectives and timelines for the event. ## Develop the scenario & Establish ground rules ## To have a productive exercise that resonates with the participants it is important to design the incident scenario so that they can see themselves in it. Having plausible scenarios and injects helps with the verisimilitude of the TTX and should as closely mimic real-world circumstances as possible. While it is an interesting thought exercise to “gamify” a meteor crashing into a building, the real-world benefits that could be derived from that are debatable. Walking through a scenario where the SMEs can envision as something that is part of their daily activities helps make the exercise more successful. -Thinking about historic events, or current events out of the news allows the participants to make better connections (not having to suspend disbelief and “fight” the scenario) and ideally helps get a better map to how people would perform during an actual incident. The type of adversary/attacker and their capabilities should be determined to assist in planning out useful injects to deliver through out the event. -As the scenario is developed it is also important to establish ground rules of how the exercise will be conducted and how participants are asked to react during it. Documenting the communication protocols (especially to distinguish between “production” alerts and “exercise” ones) help ensure both continued uninterrupted production environments, but makes the TTX flow more smoothly. Also properly time-boing the event and the injects throughout help simulate the real-word pressures or “working on the clock” during an actual incident. +Thinking about historic events, or current events out of the news, allows the participants to make better connections (not having to suspend disbelief and “fight” the scenario) and ideally helps get a better map to how people would perform during an actual incident. The type of adversary/attacker and their capabilities should be determined to assist in planning out useful injects to deliver through out the event. +As the scenario is developed it is also important to establish ground rules of how the exercise will be conducted and how participants are asked to react during it. Documenting the communication protocols (especially to distinguish between “production” alerts and “exercise” ones) not only helps ensure both continued uninterrupted production environments, but makes the TTX flow more smoothly. Also properly time-boxing the event and the injects throughout help simulate the real-word pressures or “working on the clock” during an actual incident. A good TTX scenario should have the following attributes: - Be timely – be related to current threats that exist -- Be relevant – be related to threats the organization is concerned about of that have occurred to peers +- Be relevant – be related to threats the organization is concerned about or that have occurred to peers - Be inclusive – the threat should span across multiple functions/organizations - Be defined – have a clear beginning, middle, and end that all participants understand -The scenario situation should be customized to support the exercise objectives. It should provide generic and qualitative descriptions of situations relevant to the overall exercise goal(s). The opening situation may be used as the context or starting point for Participants to identify major concerns and formulate their responses. Participants are welcome to share their internal concerns they hope to test out as part of the exercise and the Facilitator and Scribe should pay careful attention to those as the TTX is documented and the after-action report is created. +The scenario situation should be customized to support the exercise objectives. It should provide generic and qualitative descriptions of situations relevant to the overall exercise goal(s). The opening situation may be used as the context or starting point for Participants to identify major concerns and formulate their responses. Participants are welcome to share their internal concerns they hope to test out as part of the exercise, and the Facilitator and Scribe should pay careful attention to those as the TTX is documented and the after-action report is created. Arrangements should be made for a shared physical/virtual space for all participants to join in. It is recommended that participants arrange to have their regular duties and meetings covered during the TTX so they can wholly be in the meeting and contribute to the session. As the scenario is being crafted and the meeting time arranged, now is a good time to create or collect any additional materials for the exercise. Artifacts like policies, procedures, presentation slides or a cheat sheet on terms or processes to help the participants understand their roles and the exercises as a whole. Additional context around the designated scenario may also be helpful and encourage a productive session together. As the exercise is being created, the following areas should be created and/or considered: - Create Pre-Exercise Briefing: @@ -61,7 +61,7 @@ As the exercise is being created, the following areas should be created and/or c ## Create injects ## -After the outline of the scenario is defined, just like a real incident, the situation will change as it runs its course. These alterations are an integral part of the inputs representing a new piece of information delivered at key times by the facilitator to expand the discussion These changes to the scenario are referred to as “injects” that will periodically be released and shared with the TTX team that will provide new scenario parameters and information for them to respond to. The injects will describe some alteration of the parameters of the TTX that the participants need to analyze, discuss, react to, and collect additional evidence for. The Facilitator may inject different tactics or scenarios into a simulation to take the discussion in a different direction or unveil new simulated threats. Having access to internal processes, policies, and playbooks will structure how the team responds and what next steps they would take. +After the outline of the scenario is defined, just like a real incident, the situation will change as it runs its course. These alterations are an integral part of the inputs representing a new piece of information delivered at key times by the facilitator to expand the discussion. These changes to the scenario are referred to as “injects” that will periodically be released and shared with the TTX team that will provide new scenario parameters and information for them to respond to. The injects will describe some alteration of the parameters of the TTX that the participants need to analyze, discuss, react to, and collect additional evidence for. The Facilitator may inject different tactics or scenarios into a simulation to take the discussion in a different direction or unveil new simulated threats. Having access to internal processes, policies, and playbooks will structure how the team responds and what next steps they would take. Ideally, the injects should be plausible and relevant to the event or are outcomes that logically would follow from the preceding stages. These injects attempt to simulate the “randomness”, stress, and pressures that will arise during an actual incident. Good injects will follow real world attacker’s Tactics, Tools, and Procedures (TTPs) and may not initially seem to have connection to the main event and be perceived as “red herrings” to some. Magical “McGuffins” or “deus ex machina” should be avoided. During the preparation for the exercise, the complete attack should be mapped out to best present the injects to the team that provoke thought, conversation, and analysis of available capabilities and procedures. Defender’s reactions should be anticipated and countered/avoided, with alternate attack paths or tactics to be employed as the TTX progresses. @@ -128,7 +128,7 @@ As each inject is revealed to the participants, the following steps will be take 8. Repeat with each Inject until the exercise has resolved and is concluded. There will be times when participants will not have answers or solutions at each stage. This is perfectly acceptable, as it highlights gaps (in training, documentation, expertise) that can be addressed as improvement outcomes of the TTX. Having the discussions around the scenario should help bring unclear, undocumented, or areas where there is clear lack of ownership or experiences to the forefront so they can be developed before an actual incident occurs. -The Facilitator should keep the exercise on track by regularly checking in with each team/participant and making sure that everyone understands and is comfortable with their roles. During the exercise, ensure notes are being taken for key decisions that are being made so they can be discussed later. Ensure that the team is assessing the strengths and weaknesses of everyone’s responses throughout the TTX in order to learn from mistakes and identify processes that went well. +The Facilitator should keep the exercise on track by regularly checking in with each team/participant and making sure that everyone understands and is comfortable with their roles. During the exercise, ensure notes are being taken for key decisions that are being made so they can be discussed later. Ensure that the team is assessing the strengths and weaknesses of everyone’s responses throughout the TTX in order to learn from mistakes and identify processes that went well. ## Outputs ## Throughout the course of the TTX artifacts and other deliverables will be used as reference to guide how the exercises activities would progress. These along with any documentation created as part of the exercise forming should be collected as part of the exercise’s deliverables to the sponsors. These artifacts ultimately will help document any process improvements that are needed as a result of the findings of the TTX. Ideally these also help shape or improve playbooks the organization will use in the event of an actual cybersecurity incident. @@ -145,8 +145,8 @@ The scope of the Debriefing meeting may include more individuals or teams than o ## Debrief and evaluate ## Conducting a thorough debriefing is a critical component of a cybersecurity tabletop exercise. The debriefing provides an opportunity to discuss the exercise, identify strengths and weaknesses, and gather insights for improvement. -After the AAR has been assembled and is ready to be shared with the Sponsor and TTX participants a Debrief session should be scheduled. This helps ensure all stakeholders are on the same page about what happened during the TTX and allow the team to think about how best ro address future cybersecurity threats. The AAR and Debrief also allow the organization the opportunity to track progress over time and the Output documentation collected can serve as a resource for the future when respond to any incidents. -It is important for the team to review key decisions that were made and to discuss what improvements might be made to improve organizational readiness. The Debrief meeting also allows the opportunity to highlight and recognize team members and their efforts and also collect feedback on how future tabletops can be improved. +After the AAR has been assembled and is ready to be shared with the Sponsor and TTX participants, a Debrief session should be scheduled. This helps ensure all stakeholders are on the same page about what happened during the TTX and allow the team to think about how best to address future cybersecurity threats. The AAR and Debrief also allow the organization the opportunity to track progress over time, and the Output documentation collected can serve as a resource for the future when respond to any incidents. +It is important for the team to review key decisions that were made and to discuss what improvements might be made to improve organizational readiness. The Debrief meeting also allows the opportunity to highlight and recognize team members, and their efforts and also collect feedback on how future tabletops can be improved. ## Conducting the Debriefing: ## 1. Introduce the Debriefing: @@ -158,7 +158,7 @@ It is important for the team to review key decisions that were made and to discu 4. Explore Decision-Making: - Examine the decision-making process during the exercise. Discuss how participants communicated, collaborated, and made critical choices. Identify areas where decision-making could be improved or where there were notable successes. 5. Assess Communication Effectiveness: -6. Evaluate the effectiveness of communication both within the organization and with external parties. +6. Evaluate the effectiveness of communication, both within the organization and with external parties. - Discuss how well information was shared, whether there were delays or misunderstandings, and how communication strategies could be enhanced. 7. Identify Strengths and Weaknesses: 8. Encourage participants to share their perspectives on what worked well and where there were challenges. @@ -175,7 +175,7 @@ It is important for the team to review key decisions that were made and to discu - Discuss the accuracy and completeness of the information gathered and recorded. 15. Generate Improvement Recommendations: - Based on the discussion, collaboratively generate recommendations for improvement. -- These recommendations may include updates to the incident response plan, additional training, changes in communication protocols, or enhancements to technical capabilities. +- These recommendations may include updates to the incident response plan, additional training, changes in communication protocols, or enhancements to technical capabilities. 16. Share the After-Action Report (AAR) with stakeholders: 17. Summarize the key findings, lessons learned, and improvement recommendations in an after-action report. - Distribute the AAR to relevant stakeholders and leadership for further review and action. @@ -183,7 +183,7 @@ It is important for the team to review key decisions that were made and to discu If multiple improvements are discovered, asking the participants and Sponsor for feedback on how they might be prioritized and scheduled for completion helps keep the collaboration and teamwork going after the exercise and the Debriefing have concluded. ## Post-Debrief Actions ## -The Debriefing should yield a list of observations and actions the organization desires to implement. It is critical that all the work of the TTX not be lost and that an action plan is developed and follow-up on periodically to ensure that the organization learns from the lessons garnered from the exercise. The Facilitator and Scribe should assemble the desired improvements and work with leadership to schedule and prioritize any findings/learnings. The following four areas should be considered as the organization follow-through after the exercise has concluded: +The Debriefing should yield a list of observations and actions the organization desires to implement. It is critical that all the work of the TTX not be lost and that an action plan is developed, and follow-up on periodically to ensure that the organization learns from the lessons garnered from the exercise. The Facilitator and Scribe should assemble the desired improvements and work with leadership to schedule and prioritize any findings/learnings. The following four areas should be considered as the organization follow-through after the exercise has concluded: 1. Implement Changes: - Act on the improvement recommendations by updating policies, procedures, and plans. Implement changes to address identified weaknesses and enhance the organization's cybersecurity posture. 2. Training and Awareness: @@ -207,7 +207,7 @@ By following this debriefing process, organizations can maximize the value of cy - Regular Candence - External expertise - Legal & ethical consideration -- confidentiality +- Confidentiality ## References ## [CISA Exercise Overview template](https://www.cisa.gov/resources-tools/resources/cybersecurity-scenarios)