Once a live outtage (defined as a serious issue that is impacting a large number of our users) is reported:
-
A call out should occur in Techy Chat to ensure that the largest number of people possible are aware.
-
If there are specific services struggling use (insert application / service ownership spreadsheet) to determine ownership and call in the right people.
-
Always try to involve an operations team member via the Platform Operations room
-
A First Responder is nominated/steps up in techy chat
-
The First Responder is responsible for the following tasks:
- Open a zoom room for the outtage
- all joiners to the room should turn off video
- Update the War Room topic
-
Move conversation into the War Room
-
Find the 2 - 3 people who could most effectively address the issue and get them into the War Room
-
Make sure a member of the operations team is in the room.
-
If the live outage is going to last longer than 30 minutes update the status page
-
Once the issue is resolved (does not need to be First Responder):
- create a post-mortem following the directions here