Pentagon’s protection agency crippled by a “catastrophic network tech outage”

The Pentagon Force Protection Agency (PFPA) describes itself as a civilian defense agency within the U.S. Department of Defense (DoD) charged with protecting and safeguarding the occupants, visitors, and infrastructure of 100 military buildings of the Pentagon, Navy Annex and other Pentagon facilities.

PFPA does all that with law enforcement officers (U.S. Pentagon Police); criminal investigative and protective services agents; threat management agents; chemical, biological, radiological, nuclear and explosives technicians; anti-terrorism/force protection and physical security personnel.

What PFPA’s self-description leaves out is its reliance on a computer network system called the Pentagon Life Safety System Network (LSSN) and Life Safety Backbone (LSB), about which I did an Internet search but found nothing informative.

Four months ago on January 3, 2014, this all-important Pentagon Force Protection Agency experienced “a catastrophic network technological outage” that left “personnel and facilities” across the “National Capital Region” at risk. The National Capital Region refers to the Washington metropolitan area that includes all of the federal district and parts of the U.S. states of Maryland and Virginia, along with a small portion of West Virginia.

The Pentagon expects that repairs will be completed in 6 to 12 months, which means that the entire Washington metropolitan area will remain unprotected until this July or until January of next year!

But this astonishing “catastrophic outage” happened without the media reporting or the American people knowing, and was discovered only when Bob Brewin of Nextgov got hold of a document titled “Justification for an Exception to Fair Opportunity,” from the DoD’s Acquisition Directorate, which was posted on the Federal Business Opportunities website on March 28, 2014. (Read the document here.)

The purpose of the DoD Acquisitions document is to “justify” the Pentagon’s extension of its contract with SRA International, the incumbent contractor. The incumbent contract expired on April 30, 2014. The memo proposed to establish a six (6) month sole-source bridge contract with SRA International from May 1, 2014 through October 31, 2014 (with an estimated value of approximately $7.3M), with an option to extend the contract by no more than 4 months, if required, from November 1, 2014 through February 28, 2015, bringing the total possible award to an estimated $11.4M.

________________

Note: A sole-source contract is a non-competitive or no-bid contract, which means that there is only one person or company that can provide the contractual services needed, so any attempt to obtain bids would only result in that person or company bidding on it.

________________

The DoD document’s justification for “why the Government must issue a sole-source firm fixed price task-order” is “to maintain continuity of critical support for the Pentagon Life Safety System Network (LSSN) and Life Safety Backbone (LSB) services, pending repair and recovery of data resulting from a catastrophic network technological outage of the LSB on 03 January 2014. This outage left PFPA without access to the mission-critical systems needed to properly safeguard personnel and facilities, rendering the agency blind across the National Capital Region (NCR). PFPA anticipates it will take six (6) to twelve (12) months to effect repairs and to upgrade the network core to mitigate future outage risks.”

The DoD document continues:

“Due to the unexpected January 2014 LSB network outage, the Government (in conjunction with SRA), under the incumbent contract, promptly developed a four-phased approach to mitigate the risks posed by potential future outages. The four phases are:

• Phase #1 – Immediate recovery of the LSB;
• Phase #2 – Update the existing LSB Network Core devices;
• Phase #3 – Replace existing LSB Network Core devices;
• Phase #4 – Replace existing LSB outer edge devices.

Phases 1 and 2 are complete. Phases 3 and 4 cannot be completed before the incumbent contract expires on April 30, 2014.”

Thus the need for a no-bid extension of the Pentagon’s contract with SRA International.

Brewin writes that Defense Department spokesman Lt. Col. Damien Pickart attributed the January outage to the failure of a “legacy” or older network component that he did not identify. 

Pickart said the Life Safety System Network was designed to network security capabilities at the Pentagon and various Defense facilities across the national capital region to provide real-time situational awareness and contingency response, and it is not connected to any other Defense networks. He said within days of the outage, the entire system had been inspected, the component replaced (bypassed) and the Life Safety System Network and Life Safety Backbone brought back online, Pickart said. Both are currently fully operational. At no point during the outage were Pentagon employees or Defense assets vulnerable, he added.

Pickart said the Pentagon Force Protection Agency was in the process of replacing the network at the time of the outage with the objective of establishing a more reliable, redundant and resilient system to protect personnel and facilities and awarded the sole source contract to SRA to speed up necessary upgrades. Pickart also said the Force Protection Agency is prepared for any contingency that might jeopardize the safety and security of the Pentagon. The knowledge that system failures can occur is one reason why the Pentagon Force Protection Agency exercises well-established alternate security procedures to ensure Pentagon facilities and employees are protected during any type of safety or security issue.

Repairs to the Pentagon Force Protection Agency include recovery of data after the catastrophic network technological outage and upgrade and replacement of switches and routers. The sole source contract extension with SRA called for refreshed hardware and software for the Life System Safety Network, a new network design that minimizes single point failure, including dual homing, which reduces the risk of failure.

SRA International, Inc. is an information technology services and solutions consulting company incorporated as Systems Research and Applications Corporation in 1976 and beginning operations in 1978. Founded by Dr. Ernst Volgenau, it is headquartered in Fair Lakes, Virginia, has offices in 21 states in the U.S. and in four countries, and employs more than 5,700 people worldwide.

According to Washington Business Journal of January 24, 2014, SRA International is deep in debt.

~StMA