Have you given any thought to BCP / DRP?by Steve Ragan - Jun 30 2009, 20:00
Say the worst happens, the main office was directly in the path of a hurricane, the building suffers a huge hit, it’s a mess, and all of the servers, equipment, and data are presently sitting under a collapsed ceiling and covered by about three feet of water. What do you do?
Officially, the Atlantic hurricane season started on June 1, the Pacific hurricane season started on May 15, both will end November 30. If your business is in an area that would be affected by these devastating storms, does it have a business continuity plan (BCP) and a disaster recovery plan (DRP) in place to deal with a worse case scenario? If so, how long has it been since it was tested or updated?
Not too long ago, I was under a contract to help a company recover from horrific flood due to a burst water pipe. The contract was a last minute deal, actually a call in the middle of the night, because a friend needed some help moving file cabinets from under few feet of water. The cabinets had old software contracts and receipts, install disks and some old tapes from backups made in the years past. I asked, what about the servers in the basement, were they ok?
As it turns out, some of the servers were a mess. The ones on the lower part of the rack – the Exchange server, one of the dev servers, as well as a backup web server, were swimming. It wasn’t an email alert that caused my friend to discover the flood; it was the water company. When the water main burst, they called the property owners, who in turn called the name listed as the emergency contact for the company leasing the building.
It was a messy recovery, the servers needed replaced, and the drives were recovered after an expensive recovery company worked their magic. Yet, this job led me to ask about BCP/DRP operations. As it turns out, the company had a policy for BCP and DRP, but it was never maintained.
Contact names, phone numbers, and other information in the BCP/DRP were simply outdated. After all, calling a datacenter and telling them to switch on a redundant web server does little good when the company no longer does business with them. (Yep, you guessed it. The backup web server that took a swim was the same one previously hosted at the datacenter.)
Since water caused all of the damage that my friend’s employer had to deal with, it got me thinking about floods. Then I remembered that not too long ago a PR friend sent me a story about a Digital DRP program offered by a company called AppRiver. As it was storming in Indianapolis when I wrote this, her email and the story of one AppRiver client came to mind.
AppRiver offers a ton of various security services to SMBs, much like the company with all the water issues, and their DRP offering is free. However, when my friend sent me the story about them, she mentioned hurricanes.
Her story centered on Hyde Shipping Corp, a small family owned business located on the Miami River. Small, meaning they only have about 100 employees. They have been personally affected by hurricanes and flooding that crippled communication between their ports and shipping vessels at sea.
The company transports cargo to and from Florida and Belize, Mexico, Honduras and the Cayman Islands. The ability to communicate internationally via email from ship to ship and from ship to port is of utmost importance to them. So why email? Simple, email communication throughout international voyages is the most cost effective and efficient method of correspondence according to Hyde’s owners.
Yet, that form of communication is obviously vulnerable to disasters. Such was the case in 2005, when Hurricane Wilma completely disabled the company's e-mail servers, leaving the ships in a communication blackout. "It was a crazy time," recalled Robbie Brantley, MIS Director at Hyde Shipping. In addition, a 2006 flood again disabled servers and interrupted critical communications.
So, Hyde turned to AppRiver’s DRP offering, officially called Digital Disaster Recovery Program. In short, the DDRP offering monitors a company’s e-mail server activity. When AppRiver detects a loss in connectivity with the e-mail server, the DDRP begins queuing the business's incoming e-mail messages in one of its own securely located data centers until that company’s e-mail servers are up and running again.
The kicker, this service is free. (Just contact AppRiver to set it up, it takes about ten minutes.) So it wouldn’t hurt to add that to a checklist as an option for email protection and recovery. The catch is that it only protects email after the fact. As was the case of my friend’s Exchange server, all of the data stored on it was a loss until disk recovery people restored the data. However, email continuity is a huge part of BCP/DRP policies.
Hurricanes are extreme examples of disaster, but they happen. However, severe storms, floods, fire, and other disasters are more likely. So back to the original question, does your company have a business continuity plan (BCP) and a disaster recovery plan (DRP) in place and if so, how long has it been since it was tested or updated? What if your business doesn’t have a BCP/DRP policy in place, what then?
If you don’t have a BCP/DRP in place, it’s not the end of the world and you shouldn’t panic, but you should look into developing one. We won’t get into all the things a good BCP/DRP policy should consist of. A great source of information for this is here. (This is Domain 8 for those who recognize it.) Just remember that it can take months to develop solid policy, and each aspect of the business needs to be involved in this process.
Now, if there is a BCP/DRP policy in place, it needs to be updated and maintained. It does no good to research assets and develop plans only to forget that businesses grow and change, so using the same BCP/DRP is counterproductive. Also remember, as most experts will point out, BCP/DRP policies should remain flexible. Sometimes, the best plans fall apart when needed because of unforeseen circumstances.
One example of this would be the use of backup generators in a building, an obvious policy win as it keeps power on and systems functioning, until the fire department orders all power to the building cut, rendering the generators useless. This happened during the outage at The Planet.
BCP/DRP is often the forgotten aspect to security and business as a whole. It takes some effort to develop, but for SMBs the time spent is worth its weight in gold. Thanks to the explosion of SaaS (Software-as-a-Service) and MSPs (Managed Services Providers), the process of planning and any costs associated can be easy to manage. Just remember to completely vet the vendor and know who you are outsourcing with.