Business Strategies for the Worst-Case Scenario
January 01, 2002

By Staff
Appeared in Area Development

Concerns about protecting business operations and functions have risen exponentially since the events of Sept. 11, 2001. While chief executives across the United States are increasingly eyeing remote sites to house operations in case primary facilities are damaged or destroyed, they may not be asking all the right questions to ensure that sensitive data and customer connections are protected. Preparedness lies not just in securing a facility at a remote site, but also in knowing precisely the people, infrastructure, and systems necessary to restore critical functions. No CEO wants to lead the organization through the unthinkable, but the time has come to confront the hard facts and prepare for the worst by identifying the key processes and plans that go into being fully prepared for any emergency. The following six questions should be addressed before any capital is invested in backup facilities.

1. Have you rated each facility within your organization by how critical it is to supporting business operations?
Look carefully at how your buildings are tied to business functions such as revenue production, revenue collection, customer service, information systems, accounting, marketing, and human resources. For example, a maker of kitchen appliances might have a single site where stoves and refrigerators are manufactured and several warehouse sites where they are stored before being shipped out. If one warehouse burns down, it would delay revenue collection because goods have to be shipped from other locations to fulfill retail orders. But the firm would not be out of business. If the manufacturing site burns down, revenue production is slowed and customer service can suffer since the goods would not be readily available. If the company headquarters loses power, payroll may be delayed, but manufacturing and shipping can continue to keep the company in business. Each facility needs to be assigned a value to indicate its importance to bottom-line results.

2. Within each facility or data center, have your identified the components that are most essential to operations?
By grouping all your critical equipment in the same location, you can maximize the benefit of building and operating a highly reliable infrastructure that protects your data. For example, a small data center that processes invoices can be moved in with computer equipment that hosts the company website and maintains its information systems. One large cable company realized major costs savings when it moved a small data center supporting back-office functions into a larger space that held servers storing the program content sent to subscribers. In the event of a major power failure, the uninterruptible power supplies support the subscriber content first, while the less-crucial accounting and internal e-mail systems can go offline until the regular utility-based power is restored.

3. How long can these different operations be down before essential services must be moved to a new site?
If a mail-order furniture maker located in South Carolina hears about an impending hurricane, management may be able to batten down the hatches at the plant and send folks home to wait out the storm without suffering a significant financial impact. However, the call center that handles the company’s catalog ordering and customer complaints needs to be up and running 24 hours a day. The firm would have to make arrangements for all calls to be instantly routed to a backup location staffed by a knowledgeable work force.

If the storm only causes minor damage, the manufacturing operations can resume in a day or two and there is no need to set up operations elsewhere. If the hurricane tears the roof off the plant, an alternate location must be secured until repairs are completed in a few months. An excellent example of a business getting back on its feet after a disaster emerged after the World Trade Center attack. Within days of the event, a savvy global accounting firm had secured a hotel in Manhattan with hundreds of rooms for temporary office space. The hotel replaced the beds with desks and boosted its occupancy while the new “guests” stayed in business.

4. Are documented plans and processes in place to transition from the damaged facility to the backup site?
If personnel cannot easily transfer to the new remote location, skilled labor should be available nearby to get business back on track. Unfortunately for all of us, the events of September 11 highlighted the fact that the majority of firms never included additional employees in their disaster-recovery plans. If you lost half of your staff — or they could not get to work because roads were flooded or their homes were burned — how would you stay in business? One way to ensure a smooth transition with temporary staff is by storing well-defined business processes in a data system — not in someone’s memory. Make sure all key business functions are written down in a procedures manual. If the CEO is busy trying to reassure customers that their orders will be filled, he or she doesn’t have time to reconstruct the inner-workings of the firm for a temporary work force.

5. Have time limits been set for recreating essential support services?
To determine what kind of backup systems are needed, list all your crucial functions and the amount of time they can afford to be down before you suffer a major hit to the bottom line (e.g., a bank that goes down for a minute may face the loss of millions of dollars from interest on failed transactions). The graphic above demonstrates how recovery time determines the priority you assign to getting a facility back online.

When no amount of time can be lost (e.g., insurance-claims processing during a major disaster), companies need to build and equip two separate call centers in two separate locations with the telecommunications equipment to reroute customer calls if one center goes down. When both facilities are operational at the same time, it’s known as an Operational Hot Site. In comparison, a call center that handles magazine subscriptions and simply needs a new location that staffers can report to the next day is an example of an Operational Cold Site.
If time is not of the essence, a firm can maintain a list of nearby locations where office space is readily available. Disaster-recovery firms rent furnished space for any length of time, but before signing a contract, verify that they are not oversold and that the space will be available when you need it. This is a critical issue that was not considered by many firms before September 11. The next day they were left with no backup sites because they never envisioned so many companies in the same area needing space simultaneously.

6. Are regular drills performed to keep the staff prepared for any disaster?
A well-crafted disaster-recovery plan is worthless if never practiced. Indeed, the recovery will be slow and the transition rocky. Performing regular drills uncovers the weak spots and continually improves the plan.

Consider the lesson learned by a major trading floor in New York that built a mirrored site in New Jersey outfitted with new computers, software, and an infrastructure allowing seamless transfer of data in case a disaster shut down the Manhattan office. Transportation logistics were expertly detailed, complete with offsite employee parking and shuttle service to the new location. But the plan was never distributed and no drills ever took place. No one foresaw the need to make maps of the New Jersey site or to communicate the move to all employees in a timely fashion. Since the plan had been on a back shelf for some time, its list of employee phone numbers and management contacts was out of date.

Luckily for this firm, a consultant looked over the dusty disaster plan and updated all the content in addition to securing a new contact for a shuttle service that guaranteed the right amount of buses on very short notice. The additional cost of that contract was factored into the overall cost of recovery so that adequate insurance coverage could be drawn up.
When planning for the worst, the leaders of today’s businesses must set realistic goals that balance the need for continuing operations against the cost of maintaining the ability to recover from a disaster. If you have a disaster-recovery plan, revisit it annually. And for those businesses that don’t have a strategy for recovery, now is the time to define your needs, develop a plan, and then review it annually.