Eskenzi PR ad banner Eskenzi PR ad banner
  • About Us
Wednesday, 1 February, 2023
IT Security Guru
Eskenzi PR banner
  • Home
  • Features
  • Insight
  • Events
    • Most Inspiring Women in Cyber 2022
  • Topics
    • Cloud Security
    • Cyber Crime
    • Cyber Warfare
    • Data Protection
    • DDoS
    • Hacking
    • Malware, Phishing and Ransomware
    • Mobile Security
    • Network Security
    • Regulation
    • Skills Gap
    • The Internet of Things
    • Threat Detection
    • AI and Machine Learning
    • Industrial Internet of Things
  • Multimedia
  • Product Reviews
  • About Us
No Result
View All Result
  • Home
  • Features
  • Insight
  • Events
    • Most Inspiring Women in Cyber 2022
  • Topics
    • Cloud Security
    • Cyber Crime
    • Cyber Warfare
    • Data Protection
    • DDoS
    • Hacking
    • Malware, Phishing and Ransomware
    • Mobile Security
    • Network Security
    • Regulation
    • Skills Gap
    • The Internet of Things
    • Threat Detection
    • AI and Machine Learning
    • Industrial Internet of Things
  • Multimedia
  • Product Reviews
  • About Us
No Result
View All Result
IT Security Guru
No Result
View All Result

Testing the Un-testable: How to Assess Advanced Threat Products

by The Gurus
September 28, 2015
in News, This Week's Gurus
Share on FacebookShare on Twitter

Anti-virus is dead and it’s all about the next-generation products that can detect, track and gather intelligence on sophisticated, targeted attacks. Or so Wall Street seems to think. If you also buy the hype then you might at least be interested to learn exactly how effective these new products and services are. Which of the extraordinary marketing claims contain a grain of truth and which are pure snake oil? It’s hard to know without independent test results but sadly there is currently a dearth of such reports. Why is that? I can give you a few reasons…
The APT myth
If you don’t clearly define the threats a product is supposed to manage then it’s very hard to test the vendor’s claims. There are many different definitions for an Advanced Persistent Threat (APT) and similar terms and acronyms used to describe anything from a specific entity of nation-state cyber-spies to a vaguely targeted phishing email. The minute a vendor clearly defines what it protects against the testers have something to test – so a vendor may not provide a clear definition. Vendors that rely heavily on marketing over engineering see no benefit and only potential problems with independent tests.
One way to infer a definition for ‘APT’ is to read the detailed blog posts that some vendors publish when they uncover an ‘APT’ campaign. If that campaign involves sending a spear-phishing email with an attached Word document (that contains an exploit that’s been public for two years) then you have the basis of a test methodology right there. If the attack involves primarily a well-managed infrastructure to push out zero days then things look harder for the tester.
A vendor might claim an interest only in the latter, hard-to-test areas, even if in reality it (and its customers) care a lot about more mundane issues. That way its product becomes untestable and its marketing claims unassailable.
Can you test the untestable?
Is it possible to test an anti-APT product? Yes, of course you can. If you can hack then you can attack test systems and see how an endpoint or appliance product responds. That is one answer, but another more thoughtfully considered one is, “probably, as long as you define your scope.” Are you testing the product’s abilities in handling basic attacks; attacks that require at least some knowledge of public hacking tools; or genuinely sophisticated attacks that use novel ways to evade detection? You need to decide which areas, or combinations of areas, to cover to create a defensible methodology.
Another equally valid answer is, “no, you cannot test.” This has nothing to do with a tester’s competence and everything to do with lawyers. Some products have end user license agreements that forbid comparative testing. One should ask why such restrictions are imposed on customers of equipment and software that costs thousands, or tens of thousands, in any first world currency. So, why are vendors shy of being tested? The answer may lie in history.
Over the years, and even within the last 12 months, there have been public spats between security vendors and third-party testers. The private arguments are even more frequent. Some vendors believe that testers are dishonest, incompetent or both. As such, their results are deeply suspicious and are of minimal value when marketing their products or seeking to improve them. I’ve heard one vendor refer to a certain testing organisation as running a coin-operated testing service, which provides the results you pay for.
Anti-APT products are often developed and managed by people who used to work in the ‘anti-virus’ industry. Some of these will have encountered testers who proved to be unimpressive. It’s not hard to imagine that their experiences will prejudice their view of any tester that now claims to offer APT testing. Experienced testers are tarred with the same brush, while new testers are inexperienced and most likely lack the necessary contacts to even get their operation off the ground.
Advice to testers
If you want to test one or more anti-APT products or services be clear about the purpose of your test. A basic test has worth as long as you don’t stretch its conclusions to breaking point. Decide which type of attacker(s) you want to replicate and then research the known tactics and tools that they apparently use.
Test all layers of a product’s capabilities, though, even in a basic test. For example, if you want to introduce a Word document  infected with a certain exploit create this file, attach it to an email and send it to the target over the internet. If you just drop the file onto the Desktop from a virtual machine or USB drive you’re potentially ignoring a whole range of detections and protection layers.
Don’t have your exploit perform a harmless action like running the Windows calculator application. Get remote control of the system, issue commands and potentially escalate privileges to System. Or at least exfiltrate some data.
Nothing is untestable, but neither is a test ever perfect. Tests are useful simulations, which is why we strap crash test dummies into cars, rather than real people, before ramming them at speed into concrete blocks.
Simon P.G. Edwards will be talking about Effectively Testing APT Defences on Friday at the Virus Bulletin conference in Prague.

FacebookTweetLinkedIn
Tags: advanced persistent threatAnti-APTAPTattackBreachCloudCyberCyber Securitydata breachdeenisdennis groupdennis publishingexpertFlawGovernmentGuruhackedinfosecurityit securityMalwareMicrosoftmobileNSApepen testingproduct testingRansomwaresimon edwardsTestingthreat actorvendorVulnerability
ShareTweetShare
Previous Post

Advertisements Hiding Layer 7 HTTP Floods

Next Post

Trump confirms carders raided Las Vegas hotel sales tills

Recent News

JD Sports admits data breach

JD Sports admits data breach

January 31, 2023
Acronis seals cyber protection partnership with Fulham FC

Acronis seals cyber protection partnership with Fulham FC

January 30, 2023
Data Privacy Day: Securing your data with a password manager

Data Privacy Day: Securing your data with a password manager

January 27, 2023
#MIWIC2022: Carole Embling, Metro Bank

#MIWIC2022: Carole Embling, Metro Bank

January 26, 2023

The IT Security Guru offers a daily news digest of all the best breaking IT security news stories first thing in the morning! Rather than you having to trawl through all the news feeds to find out what’s cooking, you can quickly get everything you need from this site!

Our Address: 10 London Mews, London, W2 1HY

Follow Us

© 2015 - 2019 IT Security Guru - Website Managed by Calm Logic

  • About Us
No Result
View All Result
  • Home
  • Features
  • Insight
  • Events
    • Most Inspiring Women in Cyber 2022
  • Topics
    • Cloud Security
    • Cyber Crime
    • Cyber Warfare
    • Data Protection
    • DDoS
    • Hacking
    • Malware, Phishing and Ransomware
    • Mobile Security
    • Network Security
    • Regulation
    • Skills Gap
    • The Internet of Things
    • Threat Detection
    • AI and Machine Learning
    • Industrial Internet of Things
  • Multimedia
  • Product Reviews
  • About Us

© 2015 - 2019 IT Security Guru - Website Managed by Calm Logic

This site uses functional cookies and external scripts to improve your experience.

Privacy settings

Privacy Settings / PENDING

This site uses functional cookies and external scripts to improve your experience. Which cookies and scripts are used and how they impact your visit is specified on the left. You may change your settings at any time. Your choices will not impact your visit.

NOTE: These settings will only apply to the browser and device you are currently using.

GDPR Compliance

Powered by Cookie Information