By Alex White, Head of ALM Research at Redington
Figure 1 Source: https://www.imperva.com/resources/resource-library/reports/2024-bad-bot-report/
There is significant variation, especially around 2014-2015. The explanation for 2014 is changes to search engines, using more data scraping- while for 2015 it’s a huge influx of new users, largely from China, India, and Indonesia. That’s important, because it’s not a repeatable event - there’s not the same number of “fresh” humans who can start using the internet.
All of this motivates some tests of statistical significance - is there a trend down or is it just noise? We looked at human traffic percentages and ran a simple t-test, which gave a p-value of 80%. Even starting in 2015 (to account for the 2015 surge of new users), the p-value is still 75%. So while it looks bad, there’s no significant evidence of a trend- it could all just be noise. So should we celebrate?
Perhaps not yet. We also ran the same tests on bad bot traffic. Bad bots are, as the name suggests, bots designed with malicious intent; they cover a huge range of sins, from scalping, to credential stuffing, to outright fraud (‘good’ bots do things like indexing web pages).
When we look here, again we see no statistical significance, with a p-value of 28%. However, starting in 2015, the p-value is a far more poignant 1%. Even with so few data points, the rise is statistically significant if the data is taken from anytime between 2016 and 2020, and even with 2 data points the p-value from 202 is still 7%. Bad bots are on the rise.
This is perhaps more concerning in the context of how quickly the internet is growing. About 40% of the data in the world was generated in the last 2 years. As an aside, 90% is often quoted, but that seems to have been true around 2012, but not since, not least due to the enormous volume of videos, which make up around half of all data.
Figure 2- Source: https://explodingtopics.com/blog/data-generated-per-day
So we are producing more and more data, while traffic is more and more driven by malicious bots. It’s hard to see AI helping rather than hindering this problem too. From one perspective, we are making the internet a larger and larger Darwinian environment and are being outcompeted by bad bots.
So what are the takeaways? When Willie Sutton was asked why he robbed banks, he gave the eminently sensible answer that “that’s where the money is”. In a similar vein. 37% of all ATO attacks target the financial services industry . So don’t scrimp on cyber security.
|
