Are We Leaving Ourselves Open For Attack?

By now, if you’re in the information security industry you’ve read more about the Solarwinds attack than any other incident since WannaCry. Make no mistake, there’s still a lot to learn about this incident and we’ll probably learn more about it in the coming weeks. It hit the one area we were all concerned about, but sort of hoped nothing would ever happen – the software supply chain. By injecting malware into a widely used network monitoring tool, attackers were able to embed themselves into the heart of the networks of many companies. At last count, over 30,000 companies were affected but only a few of those were really considered high value targets by the attackers. But after it’s all said and done, how can we protect ourselves from future attacks like this?

Some may not be as caught up on what the attack really was, so let me go through the details that are known now. Malware most likely created by a threat group known as APT29, otherwise known as Cozy Bear, was inserted into the build process of network monitoring and management software from Solarwinds, an Austin based company. APT29, also known as SVR, is one of the most advanced threat groups in the cybercriminal space. The group is essential a part of the Russian equivalent of the CIA and has been behind some of the more highly publicized attacks. They were behind the attack on the DNC during the 2016 presidential election as well as other nation-state attacks. They are primarily focused on information exfiltration and intelligence gathering.

Infiltration Through Supply Chain

The attackers were able to gain access to Solarwinds and compromise the build process for some of the Orion network monitoring tools. While it is unclear how the attackers were able to gain access to Solarwinds environment, once the malware was inserted and published for update, once a company updated to the corrupted software they were potentially vulnerable to being attacked. Two weeks after installation, the software was designed to reach out to a set of command and control (C2) servers based in the US to avoid detection. From there, the attackers used the foothold to move laterally within the network, taking advantage of the network permissions given to the Orion software.

The attackers were careful to not be too noisy to avoid detection. In many cases, they placed their C2 servers within cloud environments, hiding in the large address blocks those servers use for their customers. It is a very good tactic and probably took many months prior to attack to set up the infrastructure to support the attack. But I think this is a point everyone should pay attention to in looking at this attack. While these servers hid in cloud infrastructure, the attacks could possibly have been prevented with certain network security infrastructure, particularly layer 7 firewalls.

Command and Control Access

Let me explain. The malware dropped by APT29 requires that it reach out and register with the C2 servers to allow for the attack to continue. If the malware doesn’t reach a C2 server, it is essentially dormant, living on the server but incapable of doing anything. It is only when it reaches a C2 server and the threat actor instructing it to take additional actions can it do any more harm. If a server is given unrestricted access to the Internet or possibly access to cloud address blocks, then the malware has a path to connect. But if the server is restricted in what it can reach by fully qualified domain names (FQDNs) or specific addresses, the malware has no path to connect.

In network analysis of impacted servers, only about 18,000 of them actually reached out to the C2 servers. The others were unable to reach the servers and never were a threat. However, of those servers that did, the threat actors were mostly concerned with federal government servers or those of security vendors. There could be more and as the investigation continues, we may find out how widespread the attack actually was. In fact, had it not been for FireEye identifying the exfiltration of some of their red team tools, the attack would have gone unnoticed, possibly for a few years. It’s frightening to think of the damage that could have been done.

Prevention For the Future

What is disturbing is that these federal government servers were allowed such access to the Internet. Had their security and network teams employed a policy of least privilege, the servers would only reach out to the network resources necessary for them to function. In the cases of the Solarwinds servers, that would probably only be to their site for updating software. Even restricting access to known update sites would have prevented the attack.

So, putting aside the supply chain protections that are needing, probably the best thing companies can do is consider the external access they give their servers, particularly critical infrastructure servers. Companies also need to be investing in better firewall protection when they can, particularly layer 7 protection. Employ least privilege principles on server access. Consider better segmentation of server farms to help protect against and detect any unusual activity.

We were extremely lucky this time because the attackers got a little greedy and careless. Their attack on FireEye tripped wires and led to deeper investigation. If FireEye had not discovered this, we would never have known for a while.