Network Stability Through Resilience Engineering
In Part 1 of this series, we discussed NSA’s recommendations for three foundational steps that reduce the number of cyber security attacks. They include:
Building upon the NSA foundation, NetCraftsmen suggests three additional steps to limit the spread of malware.
Network segmentation is like role-based authentication for network devices and applications. Like watertight compartments that keep a ship from sinking, segmentation keeps systems separate, making it more difficult for malware to propagate horizontally. The company accounting system should be on a separate subnet from other business functions. A poor design would have servers for multiple functions on the same subnet.
You should even consider segmentation within an application suite. Let’s say you have an application architecture that uses Web, application, and database tiers. Each tier should be on a separate subnet, with either firewalls or ACLs that permit only the required communications between tiers and denies all other traffic.
Also, consider distributing each tier’s servers on more than one subnet to increase the application’s resilience. A problem that affects one subnet shouldn’t impact the servers on another subnet, implying a network – and its servers – should be based on a redundant design. In an ideal world, you would have a fully redundant application where you can control the data flows from clients to the web servers and between each application tier. If there is a problem with one part of the application infrastructure, simply shift traffic to the unaffected parts of the infrastructure.
Patching, patching, and more patching. Malware likes to find old, unpatched systems. The WannaCry ransomware propagated to over 250,000 systems that hadn’t installed a patch that Microsoft made available a month prior. Weeks after WannaCry, the Petya ransomware used the same attack mechanism – demonstrating that timely patching is critical for good security.
Patch everything, including network equipment, application software, database software, operating systems, and IoT systems. Don’t forget systems like UC controllers and building facilities controllers, the latter of which are notorious for running old software. Nothing should be excluded. Automation tools make patching easier and faster.
Don’t be surprised if some application architectures aren’t easily patched. For example, an electronic health records application in a hospital that needs to be continuously available or a manufacturing line that runs 24×7, both difficult to patch. Regardless of the application, you need to determine how and when to patch it.
Another dilemma, how do you handle a system where no patches are available, perhaps a vendor no longer supports it?. We recommend protecting this system as much as possible by firewalling it from other systems and using application traffic whitelisting (one of NSA’s recommendations and a component of network/application segmentation).
What do you do if ransomware managed to find its way in? Do you pay for the encryption key? Hackers provide no assurances that they will hand over the encryption key, though they generally do to encourage payment. What’s the cost? In a CSO Online article, the author discusses the risks of paying hackers and how they typically demand one Bitcoin per server. A widespread attack can cost hundreds of thousands of dollars, just for the encryption key.
Even if you pay for the encryption key, you still have to eliminate the vulnerability or risk another attack. Fortunately, there is a strategy to avoid paying for the ransomware key: making frequent backups of all important files. You must verify that the backups can be used to quickly restore services. Don’t be one of those organizations that attempted to restore a backup in the middle of a crisis, only to find that it wasn’t usable.
I recommend verifying the time that it takes to restore a complete system (OS, application, and data). Consider the amount of effort and time required if you had to restore ten, twenty, or fifty servers.
If you’re thinking that you don’t have to reinstall everything from scratch, consider that ransomware often waits days or weeks to spread within an organization before it encrypts the data files. Loading a complete system backup may re-install the ransomware along with your unencrypted data, with predictable consequences.
Other common practices can increase your risk to ransomware. For example, sharing documents on network drives also exposes those files to ransomware. A good alternative is a cloud storage vendor that keeps copies of data files for easy retrieval. You should test the file recovery mechanism and understand how much effort it requires and look for ways to automate the recovery.
Finally, where are the backups stored? You don’t want your backups on the same storage system that the ransomware can access. Instead, use a process that stores backups where ransomware can’t encrypt them.
Unfortunately, there isn’t a simple solution to network security. A combination of tools, people, and process are needed. Your network security system should be based on defense in depth. Note that the above recommendations, combined with the NSA recommendations, begin to cover many of the attacks that we see today.
Are you thinking that this level of security is too expensive? Consider the consequences. Addressing a successful attack requires reloading all the affected systems so that you know that you’re not reinstalling hibernating ransomware. It’s always better and less costly to implement preventive functions than to use remedial functions.
There’s one more article coming. We’ll cover a few additional steps that help avoid malware and introduce the Cyber Defense Matrix, which allows you to judge the coverage of your IT security systems.
To read the original blog post, view No Jitter’s post here.
Network Stability Through Resilience Engineering
Cloud Security 101
BGP Traffic Engineering
Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.
Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.
John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services. Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.
He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.