Businesses can make better decisions if they have a good idea of who is using their website. Are they young or old? Male or female? It turns out they are very likely not even people. The majority of web traffic is made up of automated bots, and a great deal of these are malicious.
The popular narrative around bots is that they are spreading misinformation on social media, but most aren’t involved in nation state-level propaganda. They’re trying to make money.
There’s a number of ways these malicious bots can attack a site. Credential stuffing takes advantage of the way people will reuse passwords and try to takeover accounts using usernames and passwords that have been stolen elsewhere. Other bots do similar things with huge lists of stolen credit card numbers, while more sophisticated bots will buy up limited edition items for sale at a markup elsewhere.
Businesses are, luckily, aware of this problem. Or they think they are—they know that bots exist and need to be stopped. But they are unaware of the sheer scale of the problem, a miscalculation that puts them at risk of successful bot attacks, with all the financial and reputational fallout that will entail.
What are businesses getting wrong?
Our research into how aware businesses are of the dangers of bots was partly reassuring. The majority of businesses have bot solutions in place, they have budget to deal with the problem, they understand what bots can do, and they know what the implications for the business are if things go wrong.
However, there’s one glaring problem. Asked how much of their website traffic is taken up by bot activity, most businesses said between 10%-19%. This is far too low. Only 1% said that bots were consuming over 50% of their web application resources—a far more realistic estimate based on both our own experience of bots and wider research.
This means that for every bot that a business is detecting, there is at least one other, if not more that it is failing to detect.
The designers of bots go to great lengths to disguise their activities. For example, there are bots that will test the rate-limiting capabilities of a site, then, once it has found this, it will operate in a way that means it can stay just below this limit. Other bots hide behind randomized activity that looks less “bot-like” or try to emulate human behaviour. The more sophisticated the bot, the more likely it is to fly under the radar—and the more likely it is to cause serious damage.
The inevitable conclusion is that there are bots out there taking over accounts, stealing data, and disrupting businesses, while remaining completely undetected.
How can businesses fix this?
Fixing the problem is not as simple as better bot detection. These businesses are fully aware of the problem and have solutions in place, but this doesn’t seem to be making the difference when it comes to visibility.
One major hurdle is the diffusion of responsibility. For most businesses, no one department is responsible for bots—most report that four or more departments have some say in bot management strategy, from CIO and CISO to CMO and even head of customer services. Diffuse responsibility makes it easy for problems to go unnoticed; they’re part of someone else’s remit. A ship sailing in icy waters may be captained by someone who knows how dangerous icebergs can be, but if no one person is given responsibility for looking out, disaster is inevitable.
There is also a general lack of awareness of the bot ecosystem. Bots make use of stolen credentials that are traded on dark web marketplaces and increasingly in less obscure places. Intelligence on these marketplaces would be invaluable to businesses looking to understand attacks on their site. What credentials are available that could be used against us? Have we been breached without knowing?
High awareness and low visibility make for a dangerous cocktail. It’s critical that businesses better understand the problem of sophisticated bots. Without a deep understanding of how bots operate, and the wider bot ecosystem, businesses are risking taking big financial hits and losing their customers’ loyalty.
Contributed by Andy Still, CTO, Netacea
