Is Microsoft’s anti-virus good enough? Are the ‘next-gen’ endpoint products as good as they claim? Is our combination of anti-malware and whitelisting giving us full threat coverage? Enterprises are asking themselves, and SE Labs, these questions all the time. The good news is, we can help provide an answer.
The reports below contain security testing results. You can compare the performance of a variety of products that claim to protect you against online threats. This, in theory, will help individuals and businesses choose the best security product.
Rules of engagement
But these are free reports. How can you trust that the high-scoring vendors didn’t just pay for their ranking? Do you suspect that some low-scoring vendors dropped out of the report? Or asked to be retested until they scored better?
What are the rules behind the scenes in security testing?
SE Labs works with security companies to help develop and validate their products. When a security company works with SE Labs it gains two main benefits. If the product performs well it gains a much sought-after award. If it encountered problems the testing team will provide valuable information to help fix the issues.
See our presentation video from the AMTSO Town Hall meeting.
Network detection and response products monitor networks for attacks. They look for related information too, to help detect and recover from breaches.
Network detection and response: TESTED!
SE Labs has applied its Enterprise Advanced Security testing methods to network detection and response products. In this video we explain how and why we test the way we do.
The Enterprise Advanced Security testing programme includes new attack groups.
Our Enterprise Advanced Security (EAS) tests can assess any security software, hardware appliance, cloud service or combination thereof. Always evolving, these tests have expanded to include new attacks.
(These tests were originally called the Breach Response test. We renamed them for a number of reasons.)
Hackers and way they hack
Research on real attacker behaviour is a fundamental element of our EAS testing. Our team looks at the real-world behaviour of advanced threat groups, known as Advanced Persistent Threats (APTs).
Real-world security reports don’t always reflect your real world.
What makes a real-world security test useful? Does it need to provide a full assessment of a product or service? An assessment that is directly relevant for all potential customers? Or does it need to give just a taste of how effective a product can be?
The perfect security test
Tests can vary in how they are run and the level of information that they provide. Not all tests are equally reliable or even useful. But one thing they all have in common is that they aren’t perfect. Let’s look at how tests are limited, how you can interpret them and what the future holds.
False positives are not all equal. Or always real false positives!
Security tests ought to test for ‘false positives’. It’s important to see if a security product stops something good on a customer’s system, as well as the bad stuff.
Measuring the balance in security
Almost nothing in this world can be reduced to ‘good’ or ‘bad’ accurately. There is too much subtlety: what’s good for one person is bad for another. Someone else might feel neutral about it, or slightly positive or negative. The same applies when testing security products. It’s rare to get a straightforward good/ bad result.
An anti-malware product might block all threats but also all useful programs. It might ask the user frequent and unhelpful questions like, “Do you want to run this ‘unknown’ file?” Alternatively, it might let everything run quietly. Or prevent some things from running without warning or explanation. Maybe you want to see alerts, but maybe you don’t.
We look at how to put the nuance back into security testing.
Endpoint protection against different types of attacker tested.
Our reports are based on ‘testing security from zero to Neo’. Their wide scope helps you choose the best anti-malware product, that can protect you from ransomware and other types of attack.
Targeted attacks come in all levels of sophistication
There seems to be no limit to the powers of cyber criminals. In 2021 the public became aware of the advanced capabilities of the NSO group, now infamous for helping governments spy on dissidents and others.
Security testing needs to be more realistic and subtle than just running malware
Your own network can provide everything that an attacker needs to achieve its goal. In many ways it’s impossible to tell the difference between an effective attacker and a good systems administrator.