The DR Manager wears many hats throughout the year. We are asked to put together Disaster Recovery (DR) Plans, Technical Recovery Plans (TRPs), Data Center Recovery Plans (DRPs), Business Impact Analysis (BIA) documents, as well as countless other documents. We need to know and have a good understanding of the technology employed in our data centers, the networking and connectivity bringing them together, the number of Critical Applications in the landscape, the various infrastructure both in the primary and DR data centers, as well as the capacity in the DR colocation. There are also a large number of other items that we are required to know, fill out, report on, and perform to ensure that we have put our organizations in a position to succeed in the event of a disaster. But do we all know what the DR Manager’s job is during an exercise?
The DR exercise is the culmination of months’ worth of planning, depending on whether you perform quarterly, semi-annual, or annual exercises. All the hard work has been done, or at least should have been done, to get you to this point. But, since every DR Manager and every exercise is different, are there certain things that we should all do to ensure not only a successful test, but also include team members who want to assist with the exercise year over year?
The day of the exercise comes, and you are sequestered at a remote venue getting ready to call the exercise to a start. There are many things that should have been taken care of in the days and hours prior to the start, when you will say we are beginning the annual DR exercise. You have verified that the internet capacities and speed are sufficient for the number of people participating from this remote location. You have brought in a couple monitors per person as well as the docking station for each of the various laptop models your company employs, and you have a keyboard and mouse for each person. Do not forget those power cables, extension cords, and any other pieces of technology you may need. Did you bring a router so that everyone can access a more secure environment?
Some of the other things that I always like to make sure that I have accounted for are food and drink. Not the items that you will order for lunch and dinner, but rather snacks that people can have access to as they hit their Snickers moment. I have always reached out to every person who will be in our remote site to ensure that I have covered each of their wants and requests. Whether that be diet or regular sodas, some with and without caffeine, sugared items, salty items, even the keto friendly items. If you make sure that you are taking care of the resources you are counting on to make YOUR annual exercise a success, you will have individuals who are always willing to go the extra mile.
So, you start the exercise. You tell everyone that as of a certain time, the DR exercise has commenced. Now what do you do? The most important thing that a DR Manager can do during the exercise is remain calm.
More than likely, there are several issues that will come up during the exercise. We have had exercises that should have been completed within 24 hours that went several days. The last thing that the people, busily working on fixing issues, need is a DR Manager who seems to panic or becomes concerned. You are their rock. You need to make sure that you do not get emotional and raise your voice. You should NEVER yell at anyone performing restorations, recoveries, builds, validations, etc. They need you to be the calming presence that helps get them through. Show that you have faith in them. They know their jobs. The do not need a DR Manager who thinks she/he knows more than actual engineers performing the work they do every day. You are all there to learn. Let mistakes occur, but never, ever show that you think they are not doing their best to accomplish the goal.
There are several other functions that the DR Manager must accomplish during the exercise. Reporting on a regular schedule, ordering food, keeping track of what has been completed and who is responsible for the next steps, as well as opening lines of communication and bringing people together to mitigate issues are some of them. But, as I stressed in the previous paragraphs, the most important thing you will ever do during an exercise is to remain calm and have faith in the people entrusted to bring everything back and make you successful.
Remember that. You will never look good if they fail. And it is even worse if you are part of the reason that they failed. Or worse yet if they intentionally failed because of the way you treated them. Show them the respect they deserve. They define your success; the DR Manager’s Job During an Exercise.
