Technology: Email Security Services (ESS)
Put your focus on Business Email Compromise (BEC) scenarios
Don’t ignore Business Email Compromise test cases
Good security testing is realistic, using the kinds of threats customers see in real life. This is why we put a lot of focus on Business Email Compromise (BEC) scenarios, rather than just more conventional threat types (like generic phishing and malware).
Put focus on Business Email Compromise (BEC) scenarios
Many organisations focus on blocking spam and detecting malware, but BEC attacks present a different kind of threat. BEC targets the human element of email communication. Attackers craft convincing, fraudulent emails that appear to come from legitimate sources, tricking recipients into transferring money, sharing sensitive information or performing other actions that compromise the organisation. BEC cases are not about malware detection or basic spam filtering. Instead, they exploit trust and authority.
These attacks may bypass traditional security mechanisms because they often don’t contain malicious links or attachments. Instead, they rely on social engineering, making them incredibly dangerous and quite hard to spot by either people or technology.
The cyber security industry refers to this sequence of steps as the ‘attack chain.’ The MITRE organization has documented these stages in its ATT&CK framework. While this framework doesn’t provide an exact blueprint for real-world attacks, it offers a structured guide that testers, security vendors, and customers (like you!) can use to conduct tests and interpret the results.
How we test
SE LABS Ⓡ tested three email security services, one that is commercial, the other open-source. We also tested a commercial email platform. Each service was exposed to the same threats, which were a mixture of targeted attacks using well-established techniques and public attacks that were found to be live on the internet at the time of the test.
The results indicate how effectively the services were at detecting and/or protecting against those threats in real-time and shortly after the attacks took place.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
All reports
05/2024 - 05/2024
Email Security Services (ESS): Microsoft Defender for Office 365 2024 Q2
Email Security Is Essential
Email is one of the most common ways that threats will hit an organisation. It’s the first stage in a series of unpleasant, expensive events that leads to data theft, data destruction and business cessation. Email is one of the very few standard ways that hackers start their attacks, this is why email security is so essential.
Classic examples of email threats include phishing emails, designed to steal important information that aid deeper attacks. Emails can contain links to dangerous websites that can trick users into handing over critical information or may even directly attack the user’s computer. Attached documents may contain nasty surprises, such as backdoors that give attackers access to the business’ network. Access means theft and destruction (e.g. ransomware).
Loading…
Reload document 
Open in new tab If the email security service you use can stop most of that, it massively reduces the risk from hacking. Not using one is, frankly, irresponsible.
You cannot just plug in email security or rely on the security features provided by your email platform, though. Configuration is king. Given that most businesses in the UK and USA don’t have a cyber security plan, it’s likely that many Office 365 users have not changed their email security settings from the default. In this report we used Microsoft’s best practice configuration, rather than the default. Even then, after the test, Microsoft recommended changes.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. This report has gone through the AMTSO certification process to ensure that we say what we’re going to do; do it; and can prove it. Our results help vendors improve their products and buyers choose the best for their own needs.
Does it matter if your company is hacked?
And why are some businesses overconfident that they are secure?
A true story: There was a team manager, a head of IT and a chief financial officer. I asked each if they considered their network to be secure, hacked or in some other state. The ex-military team manager was supremely confident that the secure network was, as its optimistic name suggested, secure. The IT manager said, “I don’t know,” and the CFO said, “I don’t know, and does it matter?” Does it matter if your company is hacked?!
It does matter, because when businesses are compromised it affects their ability to perform their main function: to make money.
Our reports help you choose the best enterprise and SMB security products that can protect your organisation from ransomware and other types of attacks.
Email Security Services (ESS) test: Enterprise and SMB test explained
This test examined the effectiveness of five email security solutions. Microsoft Defender for Office 365 and Google Workspace Enterprise are commercial email platforms. Trellix Email Security, WithSecure Email Security and Mailcow Open Source solution are third-party ‘add-on’ services designed to provide additional security. Of the ‘add-ons’, the services from Trellix and WithSecure are commercial, while Mailcow’s is open-source.
Loading…
Does it matter if your company is hacked?
There are a couple of common reasons why people don’t think their organisations will be hacked. Firstly, they think that their security is the best. Secondly, they don’t think they are a worthy target. But all businesses are targets because they are designed to make money. And if they cannot operate, they can’t perform their main function – making money.
Hackers know this and extort money from victims by stealing their data and threatening to release it to the public, exposing victims to large regulatory fines and litigation. And, of course, there’s the embarrassment factor of looking amateur. Hackers can also encrypt data on business systems, paralysing companies until they pay up.
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Does it actually matter if your company is hacked?
And why are some businesses overconfident that they are secure?
Does it matter if your company is hacked? A true story: There was a team manager, a head of IT and a chief financial officer. I asked each if they considered their network to be secure, hacked or in some other state. The ex-military team manager was supremely confident that the secure network was, as its optimistic name suggested, secure. The IT manager said, “I don’t know,” and the CFO said, “I don’t know, and does it matter?” Does it matter if your company is hacked?!
It does matter, because when businesses are compromised it affects their ability to perform their main function: to make money.
Oue reports help you choose the best enterprise and SMB security products that can protect your organisation from ransomware and other types of attacks.
Email Security Services (ESS) test: Enterprise and SMB test explained
This test examined the effectiveness of five email security solutions. Microsoft Defender for Office 365 and Google Workspace Enterprise are commercial email platforms. Trellix Email Security, WithSecure Email Security and Mailcow Open Source solution are third-party ‘add-on’ services designed to provide additional security. Of the ‘add-ons’, the services from Trellix and WithSecure are commercial, while Mailcow’s is open-source.
Loading…
Product factsheets:
Does it matter if your company is hacked?
There are a couple of common reasons why people don’t think their organisations will be hacked. Firstly, they think that their security is the best. Secondly, they don’t think they are a worthy target. But all businesses are targets because they are designed to make money. And if they cannot operate, they can’t perform their main function – making money.
Hackers know this and extort money from victims by stealing their data and threatening to release it to the public, exposing victims to large regulatory fines and litigation. And, of course, there’s the embarrassment factor of looking amateur. Hackers can also encrypt data on business systems, paralysing companies until they pay up.
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Read this SE Labs assessment of world-leading email security products and discover how they handle well-known threats and targeted attacks.
Cyber Threat Intelligence
Annual Report 2023: Threat Intelligence for 2023
Welcome to the fourth annual report from SE Labs. This edition focuses on cyber threat intelligence.
Understanding threats is crucial when trying to defend against them. Knowing your enemy’s tactics helps clarify security planning.
We use threat intelligence when testing security products, to ensure our results are useful to companies facing real threats in the real world.
We’re sharing our insights here to help you build a strategy for success in the face of the global cyber threat.
What are the Threats?
We explore the current threats and explain why so many organisations remain vulnerable. There’s good news and bad news…
Loading…
Ransomware
Learn about the very latest innovations in testing anti-ransomware security approaches.
Annual Security Awards
Our Annual Security Awards recognises security vendors that not only do well in our tests, but perform well in the real world with real customers. These awards are the only in the industry that recognise strong lab work combined with practical success.
How we work (and could work with you!)
Discover which types of tests we run and how we can work with you to improve your product or your choice of products.
DIY Email Security
Can you defend against email threats better than the security companies?
How well do the main email platforms handle threats? Is it worth paying for additional email security from a third-party specialist? Or could you create your own secure email server (DIY email security) and get top-grade protection for free?
Our reports help you choose the best enterprise security products and services.
Compare a major email platform with a third-party service and an open-source solution.
In this special, one-of-a-kind report we investigate how well one of the world’s largest email providers performs when trying to filter out harmful security threats from your email. We also assess the benefits of a well-known email security service that you can bolt onto any other email solution. And finally, we built an open-source email server running a combination of security and management tools to see how well it compared.
We wanted to answer the questions:
- Is there value to be had from specialist email security services?
- Should you run your own server?
- Can you combine your own server with a specialist service?
Loading…
DIY Email Security
In this report we compare a major platform with a third-party email security service to see if it’s worth spending extra on security. We worked with both companies but neither wished to be identified in this report. We reported back to them all of the threats that they identified (and missed) and provided them with an opportunity to dispute any mistakes that they identified. This report is the result of that engagement.
Choose your reports and reviews carefully
We pride ourselves on a level of transparency that elevates our work above the less open reports available. But don’t just take our word for it. Our results help vendors improve their products and buyers choose the best for their own needs.
Email security services tested
Cyber criminals often use email as a way to start an attack
A primary vector for cyber attacks, email services need to be secure. Email platforms provide one level of security while third-party companies offer additional services, claiming to increase protection.
Choose the best email security solution
Email security services don’t handle all threats in the same way. Some will be stopped dead, while others can infiltrate fully. Somewhere in the middle we see email quarantine systems, Junk folders and edited messages – emails that have their links, attachments and even the words in the message tampered with.
This tampering may effectively remove a threat, or it may not. There is a lot to assessing an email security solution!
Loading…
How seriously do you take the email threat?
The approach that we take is to measure everything and then judge how important each result is. Our view is that keeping threats as far away from the user as possible is best. But sometimes security personnel need to see what’s coming in, so quarantines can be useful investigation tools. We have devised a scoring method that credits or penalises services according to our view on best outcomes.
SE Labs Security Awards
Find out which products won in our annual awards
In this report, we reveal the SE Labs Security Awards for 2020-2021 and provide the latest security testing updates.
The third annual report from SE Labs charts the successes and failures of security companies, their customers and the criminals who keep relentless pressure on us all. Working from home is at its highest level in human history, which emphasises the need to secure all devices, everywhere.
Our annual awards recognise great performance in tests and in the real world.
Focus on endpoint protection results – 6 years of testing
After six solid years of testing endpoint protection we’ve produced a review that examines some of the trends and data points we’ve identified. How did your favourite anti-virus behave over the last few years?
What can we test, and how do we do it?
Meet the team behind SE Labs and find out which security solutions we test, and how we do it more realistically than anyone else.
Loading…
In our second annual report we review the unprecedented year of 2020, announce our annual awards winners and discuss testing like hackers.
You can find out all about the exciting new Breach Response and Email Security Services Protection tests. And learn that even security testers can use machine learning to make testing better, while still testing like hackers.
Loading…
We will explain who we work with, to try and improve everyone’s security in this time of uncertainty. We’ll also explore how security testing has improved (or not) over the last 12 months and suggest ways in which you can use us better to help you personally or your organisation.
Awards winners and testing like hackers
Our annual award winners announcement shows who has impressed us the most since our last annual report.
And if you are new to security testing, we explain what the full attack chain is, and why you should use it when assessing security products.
Loading…
