And are attackers using it to breach your network?
Artificial Intelligence is ruling the stock market and may be on the verge of ruling the world if you believe the business influencers. If it’s as powerful as some say, surely AI can protect your Windows systems from hackers?
The products our new EPS test almost certainly rely on AI-related technologies to detect and protect against attacks. These technologies have been running in the background for about 20 years. We can argue that not only does anti-virus/ endpoint protection use AI, but it’s been doing so for many years, and certainly before Cylance claimed to be the first.
But I did something sneaky there. I slid in the word ‘-related’. Because when people talk about ChatGPT and other popular ‘AI’ tools, they are usually talking about something else. They are amazed by the utility of Machine Learning (ML) systems, which appear to be able to mimic human thought in a rather magical way.
ML is a subset of AI, so it’s related to AI but it isn’t capable of thought. It cannot reason, in the way that we hope future AI systems will. It is great at recognising patterns, but it can make mistakes and it’s not very good at understanding why it makes those mistakes.
As I wrote this introduction, I asked ChatGPT for a fun fact about SE Labs. It claimed we had run a cyber security ‘bake-off’ that involved employees baking “virus-shaped cupcakes [and] firewall-layered cakes. That sounds fun, and maybe we should do it, but we haven’t, so it’s not a fact. Fun or otherwise.
(I corrected ChatGPT, which responded, “You’re right, I made that up in an attempt to be fun and creative.” Maybe tomorrow’s robot overlords will be “fun and creative” and it won’t be so bad if they take over.)
Being able to match patterns is incredibly useful for cyber security tools, because attackers behave in largely similar ways, with small variations. ML can often detect new variations. Attackers can use ML, as indeed does SE Labs when creating some new threats, to try to evade detection. It’s a cat-and-mouse game, with both sides using computer brainpower to detect or escape detection.
As we launch our first XDR security testing program, we’d like to explain how SE Labs tests XDR solutions. But first, what is XDR? The industry has various opinions!
Extended Detection and Response (XDR) is a combination of security products working together. Its goal is to provide defenders with a coherent response to attacks. This joined-up approach can help defenders identify different stages of each attack without scrambling around using many different tools.
XDR is supposed to make things simpler for defenders, providing a dashboard (a ‘single pane of glass’) that provides complete insight into a network’s security situation.
SE Labs has produced the first comprehensive method of testing XDR solutions. The components of an XDR solution under test can be sold by the same company or different security vendors.
For example, we can test a solution that combines a Cisco email security gateway with endpoint security from CrowdStrike. And we can test a Cisco email security gateway alongside Cisco’s own endpoint security.
An SE Labs XDR test can assess combinations of cloud services such as email and identity alongside on-site firewalls, endpoint protection and Internet of Things (IoT) security products. If there is an XDR integration available, we can test it.
XDR in detail
There are plenty of definitions of XDR in the market. At SE Labs we define an XDR solution as a combination of at least two products, each of different types.
The products deployed do not need to be from the same vendor.
They must either talk to each other or a third management system, which provides the overall dashboard for detection and response.
Here is a list of products that can make up an XDR solution. They can be variously installed on-site or in-cloud:
The SE Labs testing team behaves like a real customer, allowing security vendors to provide and configure their products exactly as they would in a production environment. The testing team then change roles, behaving as attackers. It runs attacks from the beginning to the end of the attack chain, while also monitoring the security system for detections and other behaviour.
As the testers know every stage of the attack in detail, they assess how completely the products (and, more importantly, the combination of products) detect the different parts of the attacks as well as the entire attack episode.
In this way, the SE Labs testing team tests like hackers and analyses like defenders. The results are useful and realistic.
Who should care?
The results are useful, but for whom?
There are two main groups that benefit from SE Labs XDR testing.
Security sellers
The first group comprises the security vendors themselves. They can identify areas where detection is weaker and needs improvement. They may also discover areas where integration between different products could be better. The SE Labs test provides a good opportunity to make changes and strengthen the products, which means stronger protection for their users.
When things work well, security vendors can use SE Labs’ test results to highlight their successes in the market.
Security buyers
Secondly, but no less importantly, security buyers can use either public or bespoke test results to help choose the most appropriate products for their own organisations. Having real test data, showing how products handle threats in the real world, reduces risk and improves value for money.
Not every product works equally well, and that applies not only to general security effectiveness but also integration with other products. Security testing results are always an important resource before investing in a product. They are doubly so when looking to buy or build an XDR solution.
XDR Examples
Here are some examples of XDR implementations. We’ve chosen vendors and products randomly, but sensibly. For example, it makes sense to combine endpoint and email security solutions with a central data repository like a SIEM. It also makes sense to combine products from different market-leading providers, or to use all of the products from a single one.
Infer no judgement about the suitability of specific vendors from this example list:
Microsoft Defender (endpoint); Microsoft Defender (email); Splunk Cloud Platform (SIEM)
In the first example we have combined the detections and other data from two Microsoft products (endpoint and email) and sent them to a cloud-based platform that claims to provide insight into all activity.
In the second example, a simple setup combines the detection capabilities of endpoint and email threat detection products from different vendors.
Thirdly, some security companies are able to provide products for many different areas, such as firewalls, endpoint, email and web security. In this example, one vendor provides and manages all the components of the detection system.
Ransomware is the most visible, most easily understood cyber threat affecting businesses today. Paralysed computer systems mean stalled business and loss of earnings. We tested CrowdStrike Falcon’s endpoint security vs. ransomware.
SE Labs launches first public Network Detection and Response test
SE Labs tested VMware NSX Network Detection and Response against a range of hacking attacks designed to compromise systems and penetrate target networks in the same way as criminals and other attackers breach systems and networks.
Full attack chain test in the datacentre
By running the most realistic set of attacks possible we put NDR products to a significant challenge. Can they detect real attacks in real-time, often using unique scripts and malware? If you want to know more about advanced persistent threats on the network please read past the initial graphs in this report and dig into the detail.
How SE Labs tests and scores email security services
Email security services can do a lot to protect users from online threats. This also means a lot can go wrong too. Testing and scoring these services requires a lot of attention to detail and a scoring method that takes into account all of the possible outcomes, including unexpected ones.
Technology gives us tools to achieve sometimes amazing things. But no matter how advanced, it usually requires humans to make an effort.
We’ve all heard the terms, “fire and forget” or “plug and play” but these are usually marketing dreams. In security, as with any other area, you can buy tools, but you need to understand how to use them if you are going to succeed.
All posts
