Microsoft Advanced Threat Protection (ATP) serves a real need when it comes to fortifying the basic anti-phishing, anti-virus, and anti-malware that make up EOP, which hackers bypass with ease. ATP does anti-spam and anti-malware, while also scanning links and attachments. But as you’ll soon find out, there are major speed bumps slowing the flow of policy implementation, threat management, and the end user experience in ATP.
There are many options in ATP, and they can be overwhelming. Understanding the form and function of ATP is a crucial step before you can decide to use it or try a third party like Avanan for advanced email and collaboration security. In this document, we’ll drill into a phishing campaign with ATP and Avanan, showing the experience of the end user and admin. (We performed these tests on an E5 account with ATP.)
There are several considerations for using ATP or Avanan for anti-phishing, each being a section in this post, that you can see below in the chart. We provide ample detail with screenshots, paragraphs, and video clips where Avanan and ATP are connected to the same O365 account.
| Category | ATP | Avanan |
| Overall detection |
|
|
| Ease of deployment and policy configuration |
|
|
| End-user experience |
|
|
| Threat management reporting |
|
|
|
Extension of security to OneDrive and SharePoint |
|
|
|
Search and Destroy |
|
|
Overall Detection
It’s important to remember that Microsoft isn’t a security company. They’re a maker of productivity tools, and they’ve had to adapt in order to provide security for those tools. Because of this, Microsoft’s AI is built from their spam filtering, which is suited for their large customer base of 100 million inboxes. Their anti-phishing AI provides a generic analysis in an effort to avoid the false positives that would disrupt the productivity tools which are Microsoft’s main business. Spearphishing gets around that mass analysis.
>
This chart was made from a sample of 44,000 emails over a 14-day period.
The graph above summarizes trial results for a customer who tested Avanan against ATP over a period of 14 days. Out of the 44,000 total emails scanned, there were a total of 3,574 phishing emails, shown in the graph above. 3,033 of those (84.9%) were delivered to the user’s inbox when Microsoft was used without Avanan's additional layer of protection. In another study we conducted, we found that ATP classified 64.9% of phishing emails caught by Avanan as clean.
Avanan's machine-learning algorithm was built specifically to identify and quarantine the threats that Microsoft misses. Furthermore, Avanan’s processing power is more specific to the organization at which it is deployed, creating a more customizable experience for the admin. Essentially, Avanan can think on a micro level where Microsoft must do mass analysis.
For protection beyond the industry's best anti-phishing, Avanan's unified multi-vendor security platform leverages cloudified technology from best of breed vendors like Check Point for whom, unlike Microsoft, security is their main business. Microsoft uses well, Microsoft.
If you use ATP, you can only rely on Microsoft's intel. With Avanan, you can draw upon the expertise of security industry leaders whose tools are recognized by the Gartner Magic Quardrant and are available for instant deployment on the Avanan platform.
Ease of Implementation
Admin need to block phishing attacks quickly and easily.
The implementation and configuration processes are good metrics for what the ongoing experience of a security platform will be. The more cumbersome the setup, the more likely it is that the rest of the admin and end-user experience will reflect that.
ATP
Configuring ATP to your environment can feel a lot like configuring your environment to ATP.
ATP policy configuration happens on the Security & Compliance platform. From the left hand navigation, choose “Threat management,” then select “Policy” from the dropdown. You’ll see policy engines for anti-phishing, anti-spam, anti-malware, attachment scanning, link scanning, and DKIM. We created an anti-phishing policy called “Spoof of CXO.”
You must edit the policy in separate windows rather than a single window. Instead of simply breaking down who the policy affects, what the workflow will be, and what remediation actions will be taken, Microsoft chooses a different approach. They break the policy creation process down into "Policy setting," “Impersonation,” “Spoof,” and “Advanced Settings.” But what, exactly, is the difference between an impersonation and spoof?
This is what creating a policy workflow looks like in ATP.
Beyond this, you need to speak their language when it comes to policy creation. For example, in the image above where the policy “Spoof of CXO” is being created, there are no platform-native descriptions of what “Safety tips” or “Mailbox intelligence” means. (To clarify, Safety tips are supposed to teach the end user about phishing, and Mailbox intelligence is how ATP monitors user behavior.)
With ATP, you have to make the awkward choice of how "aggressive" you want the policy to be at the expense of false positives.
Avanan
To simplify email security, Avanan's SmartPhish has a one-click deployment for anti-phishing, as you can see below. This is as out-of-box as a policy can get. Avanan's SmartPhish detects the threats targeting ATP's vulnerabilities, such as baseStriker and ZeroFont, which ATP failed to identify as threats.
Deploy an anti-phishing policy in a single click with Avanan's SmartPhish algorithm.
Configuring custom policies in Avanan is easy.
Avanan's custom policy wizard has clearly defined parameters and is easy to use.
End User Experience
How the end users are notified (or not notified) of detected phishing is critical in rolling out new email security features to the enterprise. Let's look at how end-users are notified with Avanan vs. ATP.
ATP end-user alerts
With ATP, the end-user experience can be disruptive. If an end-user receives a phishing document with a malware trojan, Microsoft will send an alert to them in the form of a .txt attachment.
This is the malware alert a user would get in ATP. There is no body in the alert email—only a text file.
When the user opens the alert to learn more, they are greeted with this unfriendly, 4-line text summary that really just tells them there’s trojan malware.
This is how the end-user experiences threat management in ATP.
Avanan end-user alerts
Avanan gives end-users context when a security event occurs.
Avavnan describes the situation to the end-user and provides a link to request document removal from quarantine.
The end-user can request that the quarantined document be released in a message to the admin.
The end-user can provide as much or as little detail as they please in their restore request.
Threat Management Reporting
Avanan malware reporting
Now that we’ve seen how the end-user would request a document to be released from quarantine with Avanan, let’s see how the admin would respond to that request.
In the screenshot below, you can see the message sent by the end-user to the admin. (It’s the second restore request. The message matches up with the image shown above.) This window is under “Restore requests” from QUARANTINE in the main navigation for the Avanan platform.
The admin can click the sender, mailbox, subject, attachment, and message for more detail about the restore request.
When you click on the subject “Another Test,” this is what you’ll see: a comprehensive report showing the email profile, header information, and the security that found the threat.
The admin can quickly investigate the email header, body, and attachment. They can conveniently choose to remove from quarantine or decline the request. Or, they can look into the email even more.
When you click on the email attachment “Malicious Demo Malware.xlsm,” you are taken to another window that shows you the attachment profile, allowing you to drill even further into the threat report.
For even more detailed reporting, the admin can view security event intel from Avanan's security partners, like Check Point and LastLine.
Here’s what you’d see after clicking on the Check Point’s Malware Sandboxing report. This threat report is not segregated into the Reporting page on the main dashboard, as it would be in the ATP Security & Compliance platform. You can see helpful insights, such as the algorithm's verdict, risk level of the attack, and malware activities.
The admin can see threat details, view quick summaries, and see how the malware affects different operating systems.
But this is the most exciting feature: you can watch a threat emulation video showing exactly what actions the malware would take in your environment.
Emulation videos give the admin a clear view of the malware's actions and affects.
With Avanan’s advanced forensics feature, you can see a timeline that shows the steps malware took as it detonated in the sandbox. This visualization comes in the form of a bar chart, with insights into process, registry, and network events.
The admin can export a report of the advanced forensics shown above.
ATP malware reporting
In contrast with Avanan’s rich reporting above, all of which stemmed from a single document restore request, this is the insight that ATP provides.
The admin can't click into any object for further details.
ATPs threat dashboards are segregated under the Reports tab. ATP has many data visualizations that give the admin an unfiltered list of 24 reports which provide surface-level context on user-facing threats.
ATP's "Top Malware" report has very limited insights. The admin can only see the total count of malware.
Extension of Security to OneDrive & SharePoint
Collaboration isn’t confined to just email, which is why it’s important that anti-phishing tools extend to the cloud environments in which links and attachments are shared, like OneDrive.
ATP doesn't fully extend to OneDrive & SharePoint
Microsoft claims that ATP secures OneDrive, SharePoint, and Teams. But if you read the fine print, Microsoft documentation states that "ATP will not scan every single file in SharePoint Online, OneDrive for Business, or Microsoft Teams." The importance of this is demonstrated in the video clip below, where we are able to clearly share and download a known-to-be malicious file from OneDrive.
This is from "Office 365 ATP for SharePoint, OneDrive, and Microsoft Teams."
ATP doesn't extend to other 3rd party SaaS
ATP's threat detection and remediation capabilities are confined to the Office 365 environment. That means you can't scan content in your corporate Box account with ATP.
The more connected apps become, it will become increasingly important that security solutions can straddle those interfaces. For example, it was recently announced that OneDrive files can be imported into Slack. This is good news for productivity, but not security. We predict that Slack will inherit some of OneDrive's vulnerabilities, and we know that ATP won't be able to intervene.
Avanan protects email, the O365 environment, and collaborative SaaS apps
Avanan secures OneDrive, SharePoint, and Teams. But we see every file and link distributed within and between those environments. (We can also extend DLP and anti-malware to other cloud based applications, like Slack.)
In this video clip, we tested the same malicious file, giving ATP and Avanan a chance to identify the threat. ATP failed to classify the attachment as malicious, while Avanan succeeded.
Watch how ATP and Avanan report a malicious file in OneDrive.
Search and Destroy
Security teams need to quickly figure out where the smoking gun lies in a phishing event that may have resulted in a breach. Getting a list of all the emails from a particular sender/domain with a specific subject is critical. Equally as critical is getting those emails out of user's inboxes (or spam folders) with a single click.
On the right, you can see the limits of ATPs mailbox search feature.
ATP Search & investigation feature
In ATP, search and investigation is broken into four categories. (The "Productivity app discovery" leads to an entirely different platform that's a lot like a CASB.)
To start using ATP's "Content search," click the dropdown under "Search & investigation."
You will be taken to a summary page showing existing content searches made in ATP. There are two options to create a new search: guided search and new search. The only difference between the two searches is that "guided" requires one step to be completed before another, whereas "new" lets you search within the report view.
Here's ATP's quick view of existing Content Searches.
There are a total of 19 keywords, or "conditions," that shape the search experience in ATP.
The admin can use any combination of these conditions to guide their email search in ATP.
Each condition corresponds with a group, with the most common conditions listed at the top.
ATP returns the results of the query into a report with four columns: date, event, details, and user. You can't quarantine all the returned results in this view.
The admin can't click into the "Details" column, giving them only a surface-level summary in their search rather than an in-depth investigation.
Avanan's Search and Destroy feature
Conclusion
Avanan’s role is to fill the security gaps created by Microsoft EOP and ATP. The main differentiator between Avanan and ATP is the level of detail with which the admin and end-users are provided, without compromising usability. With Avanan, you can drill into any phishing campaign and take mass actions like quarantine all at once.
Contact Sales 
