Do the Dept. of Energy and ESnet have a problem on their hands?

by Steve Ragan - Jul 6 2011, 07:00

ESnet is a network that connects most of the major labs under the U.S. Department of Energy (DOE) to other research facilities across the globe. It’s a wealth of knowledge, considering who has access to it. So what would happen if ESnet was breached? How can one be sure that it hasn’t already happened?

If you consider the labs connected to ESnet, at least the ones that spring to mind where the DOE is concerned, it’s an impressive mix of research and development. ESnet connects the Oak Ridge National Laboratory (ORNL), the Pacific Northwest National Laboratory (PNNL), the Y12 National Security Complex, and FermiLab, just to name a few.

Last week, the PNNL closed their public website and rejected all incoming email traffic after they discovered that someone - somewhere - was targeting them. PR people called the attack sophisticated, but media coverage makes it look less so. The idea that access was restricted to email and Internet suggests that someone was Phishing, and someone at PNNL fell for it.

According to a PNNL spokesperson, the lab sees nearly 4 million attacks per day on its external network. However, if Phishing was at the root of the problem, and someone clicked on the malicious links or attachments, then the PNNL employee wouldn’t be alone. The same thing happened recently at ORNL, and once before years ago. This is in addition to another set of security issues. 

In 2006, an ORNL contractor brought an unclassified laptop into a restricted area at Y12. Later it was learned that as many as 37 other laptops - owned by ORNL employees - were brought into the same restricted area. According to a DOE Inspector General report from 2008, nine of those laptops were taken on foreign travel to sensitive countries, a serious policy violation, and all 38 laptops were infected with Malware. In addition, 26 of them had wireless communications ability.

In 2007, ORNL suffered another breach after several emails with malicious attachments allowed outsiders to access a database with personal information stored on it. The database contained information on all of the visitors to the lab, spanning nearly 14 years. The emails were targeted to specific people, using topics that would uniquely hold their attention and entice them to open the attachments.

After more than 1,100 attempts, the attackers managed to get 11 people to fall for the scam. Moreover, ORNL said that the Los Alamos National Laboratory in New Mexico and California's Lawrence Livermore National Laboratory were also targeted by the same attack.

In April, ORNL blocked external internet and email access after Malware - linked to Phishing attacks - was discovered on their networks. According to the lab, 570 people were sent malicious emails, of which 50 people accessed the links contained within.

An ORNL spokesperson confirmed to The Tech Herald that the attack led to the loss of less than 1GB of data before the network was taken offline. At the same time, the spokesperson also confirmed that other national labs and government agencies were reporting an uptick in Phishing attacks. PNNL said the same thing last week, when they reported that at least one other national lab and Battelle were also under attack.

FermiLab was breached in 2002, and an investigation showed that 17 computers were compromised to house movies and other Warez for Web distribution. Nearly a year later, police arrested a 17-year-old in the U.K. for the breach.

Along with the incident including ORNL, Y12 was breached in June as part of AntiSec. Details, including usernames and passwords, as well as other database information, were leaked to the Web. The database is used to manage the CMS part of the Y12 domain. During the cleanup, Y12 was forced to take the domain offline.

In these examples, a larger picture starts to take shape. National research labs, working on classified and unclassified projects, each one with a connection to ESnet, have been singled out by cyberattacks. On several occasions, these targeted attacks led to security incidents. Internally, policy failures allowed other security issues to arise. Still, is this a problem or a string of coincidences?

According to an email sent to The Tech Herald, each of these labs are managed by a GoCo - or a Government owned/Contractor operated type of relationship. Battelle is the company that manages PNNL and ORNL, while Babcock and Wilcox manage Y12. FermiLab is managed by a 501(c)(3), the Fermi Research Alliance (FRA).

Each lab has their own policies and rules to manage thousands of employees and researchers, if not more in some cases. It’s important to note, commented the person who wished to remain anonymous, that the scientists and support staff at these labs and management companies are separate entities.

The research team will be staffed with brilliant post-doctorate students and senior PhD researchers, while the blue team is comprised of ‘regular’ - albeit smart - people.

“That is to say, the cybersecurity research program at ORNL has nothing to do at all with the blue team that is tasked to defend the infrastructure. The ESnet connection may not even touch the commodity Internet connection in many cases, but that doesn't mean someone didn't want access to that, and we know the air gap can be overcome with a simple USB stick,” the email explained.

In fact, ORNL has 2 or 3 really great people on their blue team, the email noted. Yet, “…it's always important to point out that people are the greatest strength and biggest weakness of any organization.”

The air gap itself can be as simple as a Phishing email or an incorrectly configured network setting, to something as complicated as custom Malware spread through network resources or USB drives leveraging any number of attack vectors.

According to Thomas Zacharia, the deputy director of ORNL, one of their core competencies is cybersecurity research. Yet, during the April attack, the lab had to contend with dormant Malware.

After cleaning up one server known to be compromised, Zacharia told Wired that several others suddenly went active, causing ORNL to block all outbound traffic. These systems were previously thought to be clean.

The person who sent the email is familiar with how things work when it comes to ESnet and the research labs. They made an interesting observation while expressing their thoughts.

“…the labs have Internet access, and thousands of users. We know the Internet is full of badness and people click links, open files, and make mistakes. Combine that with a blue team that is regularly compromised with high-level events and does not change…” and the bigger picture arrives.

It’s something far more than a consistent series of internal policy violations, and external breaches leading to compromised systems within one lab. It’s a systematic dismantling of security, targeting the most valuable prize in the world - information. The attackers are hitting all the labs at once, and the goal is to use the gained access to move inward. It’s a complicated approach to a simple stepping-stone attack.

So tinfoil hats aside, what can be done?

“I think it's clear that ESnet needs a serious amount of threat intelligence on that network. Connecting the various labs and factoring in the Battelle corporate network, analyzing ESnet network activity and making sense of the terabytes of raw data undoubtedly being collected right now by all the blinky boxes labeled ‘security’ would paint a more complete picture,” commented Rafal Los, Security Evangelist for HP.

Los wrote about the potential ESnet problems on Monday. It was his post which led us to start asking questions, eventually resulting in our email out of the blue.

“The case for a more complete threat intelligence setup is compelling ...if not painfully obvious. When the network is this complex, the assets this critical - how else do you have any faith in your security? Just as a point of clarification - I'm not saying that Battelle or the ESnet folks don't have a good handle on what's going on in their network - but clearly once you start drawing lines and connecting dots the picture gets very interesting…,” Los added.

As Los points out, more research is needed on this. Considering the patterns, and established cybersecurity profiles from scores of high-value attacks, it’s not hard to imagine that there are infected systems somehow connected to ESnet that no one is aware of.

“I think there are some very serious questions those folks need to be asking themselves,” Los concluded, “Where are the threats right now? Can we trust who's already on that network? Can we stop a multi-phase, multi-point, stepping-stone type of attack?”

We’ve reached out to ESnet for more information.

Around the Web