Hurricane Irma: A Case Study in Readiness
Posted by Margaret Carnohan on May 29, 2018 12:05 pm
As a data center company, business continuity and disaster recovery are central to our operations and to our mission. We invest time, energy, and resources into building a secure, redundant, and resilient nationwide infrastructure that ensures our customers—and theirs—can continue with business as usual, even under extreme circumstances.
But what actually goes into fortifying a data center facility about to be hit by a major hurricane? While families and local businesses are boarding up and heading out of town, a data center team must stay put and prepare for the onslaught of torrential rains, 100+ mph winds, and other threats of a storm of that magnitude.
Here’s how our Miami facilities and operations team prepared to endure Hurricane Irma:
In the week leading up to the hurricane making landfall in South Florida, our facilities and security leadership conferred to discuss storm preparations and mobilize the full force of our staff and partners. Of the greatest concern was making sure the facility in Miami would have enough fuel, supplies, and reserves to remain operational throughout a potentially devastating event and that everyone staying in place would be safe.
As the storm approached the mainland, we watched as the weather forecasters and government officials updated the intensity and projected impact of the storm. Based on the worst-case forecasts, we made plans to send in additional personnel and other resources as reinforcement.
About 50% of our mission-critical staff at CoreSite are U.S. Military Veterans, so they are familiar with being asked to go into situations like this. Former Marine and Facilities Engineer, John Rufino, stepped up to head out from New York to Miami. He flew to the site and went around to stores, stocking up on water, non-perishable food, batteries, radios, rain slickers—everything we’d need to keep people safe on-site. He also provided some respite for our Miami-local employees to make preparations for their own homes.
Hurricanes are categorized by the strength and intensity of their winds, and with Irma packing sustained winds of greater than 185 mph, we needed to increase our normal wind-proofing to make sure that nothing on our grounds would inadvertently become a projectile or danger to the surrounding area. Our employees walked the entirety of the property, flipping and anchoring trash receptacles, and even heading to the roof to bolster the mounts for the rooftop A/C units, in addition to ensuring that normal debris removal was current.
With storms as large as Irma, widespread power outages are a near certainty and without power, all of the infrastructure that supports our customers’ businesses would fail. It was imperative that we prevent that from happening, so we made sure that all of our on-site backup generators were in working order. All of our critical equipment maintenance was up to date and in accordance with our robust maintenance program. We topped off our generators’ fuel and ensured they were warmed up and ready to go in case of emergency.
We also had to prepare for a potentially extended, and unpredictable period of keeping the generators in service. To protect against running out of fuel, we connected with our national fuel vendor and arranged for two fuel trucks to be positioned with ready access to the site after the storm.
Once the storm hit, staff, led by Facility Manager and Navy Veteran David Adam and Telecom Manager and Army Veteran Lee McDaniel, was in execution mode. We’d prepared heavily for the event, and it paid off. As utility power started to fail, we switched over to generator power for what turned out to be five days. Our prearranged fuel trucks made way to the site and kept our generators fed throughout the event. The team of David, Lee, and John responded to all issues that arose, from generator and fuel system checks to surveillance for possible leaks (and containment if necessary). Once the storm passed, Navy Veteran and Facilities Technician Dennis Banks, flew in from New York to relieve John and assist at the site.
The on-site team was able to share minute-to-minute updates with those of us watching from across the country. Throughout the week of this storm, they worked diligently to make sure everything at the facility stayed safe and customers wouldn’t be impacted. The employees were able to keep in contact with their families weathering the storm, and check in at home once the storm passed. These employees are a great example of the preparedness and dedication of our Field Operations team putting into practice our principles of Operational Excellence. I am very proud of the CoreSite team and of their performance and commitment during the storm. For the sake of our customers, they stepped up and ensured 100% uptime through a major natural disaster. The team’s constant communication and excellent teamwork is what made that possible.
Many companies profess a deep commitment to their customers and end users. CoreSite is different only in that our commitment is the core of our culture; we’re committed to excellence and providing customers with an unrivaled experience and support—especially during times of disaster.