Email Security Blog

Why You Should Care About an Advanced Threat Catch Rate

Paul Chavez February 7, 2019 Email Security
Advanced Threat Capture Rate

Artificial intelligence (AI), machine learning, and deep learning analysis have become common buzzwords synonymous with cybersecurity. Particularly in the email security space, deploying a solution that leverages these technologies only makes sense given the hundreds of thousands of new threats being discovered every day.

Without the application of an artificial intelligence-based system like ensemble models, genetic algorithms, intelligent agents, or a knowledge graph, it would be impossible to keep up with sophisticated threat actors and the ever-changing threat landscape. According to AV-TEST, over 350,000 new malicious programs (malware) and potentially unwanted applications (PUA) are discovered every single day. So, how do you know if the solution you choose is going to protect you from the latest advanced attacks?

The Importance of the Catch Rate

Unfortunately, there is no advanced email security solution vendor today that can confidently tell you that they will stop 99% or more of the advanced email threats they see. This is actually quite surprising given the state of maturity for the advanced threat protection market, which is going on seven years. Given that vendors should now have enough data to confidently benchmark their own performance, along with the fact that most vendors do provide an effectiveness rating for other features of their product lines such as antispam and antivirus, providing an advanced threat protection catch rate should be standard.

Today’s Secure Email Gateway market leaders such as Proofpoint, Microsoft, and Symantec document service level agreements that indicate better than a 98% effectiveness against spam and 100% effectiveness against known viruses. They’ll even go so far as to provide credits for dropping below these ratings. Yet they do not publish rates for their Targeted Attack Protection or Advanced Threat Protection add-ons. Something does not add up. In fact, if a true artificial intelligence based system is being deployed, it should be much easier to predict effectiveness.

Predictive modeling should have a well-defined repetitive process that combines a well-curated dataset with model testing, training, and deployment, where at each stage the system has the ability to iterate and make improvements millions of times a day. Even before deployment, vendors should be able to predict with high confidence how well the model should perform. At a minimum, this insight should be exposed to set expectations. If the solution you are evaluating does not meet these basic requirements and the vendor cannot at least provide the expected benchmark, then it cannot be a true artificial intelligence-based system.

Vendors may argue that the only way to appropriately benchmark a solution is to conduct a Proof-of-Value whereby you can test the solution itself. However, if the environment is not targeted with varying advanced attacks during the testing, if you are only routing a subset of your email traffic, or if you are only testing against certain types of attack techniques, the Advanced Threat Catch Rate (ATCR) will not be accurate. It could take several months before you can collect the necessary data to accurately define the solution’s effectiveness and very few organizations have the resources or the time to wait before making a decision. Before even considering a POV, you should ask that the vendor provide an Advanced Threat Catch Rate that you can benchmark against for comparison.

Calculating the Catch Rate

The ATCR should be a simple calculation that allows organizations to measure the true effectiveness of an advanced threat protection solution. The ATCR calculation Agari uses is as follows:

Advanced Threat Catch Rate

Based on the above calculation, the Agari Advanced Threat Catch Rate is currently 99.9%.

As part of the calculation, the ATCR should account for all threats missed by existing email security defenses. In nearly all cases, a Secure Email Gateway will have been deployed for several years and appropriately tuned to stop the majority of email attacks. This would include, but not be limited to, the following: spam, known viruses, malware, ransomware, spear phishing, graymail (unwanted marketing emails or newsletters), and explicit content.

Even if the technologies designed to stop these attacks are not 100% effective, it would be safe to assume that the attacks that are missed are the most evasive, as the threat actor employed a technique that was designed to evade your existing defenses. The spam attack that did not get detected likely used a legitimate and reputable email sending service to evade the domain and IP reputation-based filtering system enabled on the gateway. The malware used in the attack likely evaded detection by leveraging multi-stage evasion techniques where the payload did not exist in the initial email. And finally, even the commodity phishing attack evaded your defenses because it used an identity deception technique like domain spoofing to convince the recipient that the email was from a trusted sender. In all of these cases, the attacks place a high risk to the organization in potential resources, time, and dollars lost due to harmful messages reaching the inbox.

The goal of the advanced threat protection solution is to help eliminate or replace any unnecessary security layers. If a solution is only effective against phishing, but not effective against business email compromise, you end up complicating your environment by having to deploy multiple solutions. Any solution designed to supplement your existing email infrastructure should stop all missed attacks and therefore must incorporate all threats into their Advanced Threat Catch Rate.

Setting the Catch Rate Benchmark

Today, Agari Advanced Threat Protection maintains an Advanced Threat Catch Rate of 99.9%. Agari is the first advanced threat protection solution to track and publish a capture rate that incorporates all threat types across the entire customer base. While no organization can at this point claim to be 100% effective, Agari is the most effective advanced threat protection solution on the market today. As a commitment to our customers, we continually monitor, calculate, and expose our ATCR to ensure we maintain the high standards they expect.

As your organization starts the journey toward evaluating new advanced threat protection solutions for email, we encourage you to challenge each potential vendor to expose their ATCR and to “show their math.” Remind them that there is a solution that exists that has a published 99.9% Advanced Threat Catch Rate across a customer ecosystem that sees 100s of millions of messages daily. Anything less than that level of effectiveness should be unacceptable.

At the end of the day, Agari Advanced Threat Protection can help save your organization time, resources, and money by stopping the attacks that all other solutions miss. Now is the time to check it out.

Discover more about how Agari can help your employees trust their inboxes in the Advanced Threat Protection solution brief.

Leave a Reply

Your email will not be published. All fields are required.

Agari Blog Image

August 27, 2019 Taylor La

Expanding Email Security One Post at a Time — Experiences of a Digital Marketing Intern

Agari is more than an email security company that detects cyberattacks. It is a community…

Agari Blog Image

August 20, 2019 Armen Najarian

Brand Impersonation and Look-alike Domains: How Cybercriminals are Hurting Tech Brands

Here’s some earned media you don’t want for your brand—headlines announcing that your customers are…

Agari Blog Image

August 15, 2019 Paul Lorence

Building a Career at Agari: One Designer’s Experience

Being a user experience (UX) designer and managing design teams is a rewarding job. I…

Agari Blog Image

August 1, 2019 John Wilson

Email @ttack: Why Identity Deception is Your #1 Threat

Want to know how email became the number one attack vector for cybercriminals? Look no…

Agari Blog Image

July 31, 2019 Armen Najarian

2020 Presidential Candidates Remain Vulnerable to Email Fraud Despite Increased Warnings

Only three months ago, the Agari team published our first in-depth analysis on how the…

mobile image