FIs Suffer Highest Proportion of Bad Bot Traffic: Distil Report

Financial services companies have the highest percentage of bad bots with 42.2%, according to Arlington, Va./San Francisco-based Distil Networks, in its report, “Bad Bot Report 2019: The Bot Arms Race Continues.”

The bot mitigation firm’s annual report investigated hundreds of billions of bad bot requests from 2018 over thousands of domains to provide deeper insight into the daily automated attacks wreaking havoc on websites, mobile apps and APIs. The findings, produced by the Distil Research Lab, suggested bot attack sophistication continues to evolve, as advanced hackers adapt techniques to nullify current defense tactics.

The report explained bad bots interact with applications in the same way a legitimate user would, making them harder to prevent. “They enable high-speed abuse, misuse, and attacks on your websites and APIs. They enable attackers, unsavory competitors, and fraudsters to perform a wide array of malicious activities.” Such activities include web scraping, competitive data mining, personal and financial data harvesting, brute-force login and digital ad fraud, spam, transaction fraud, data theft and more.

“Bot operators and bot defenders are playing an incessant game of cat and mouse, and techniques used today, such as mimicking mouse movements, are more human-like than ever before,” Tiffany Olson Kleemann, CEO of Distil Networks, said. She added as sophistication strengthens, so too does the extensiveness of industries affected by bad bots. “While bot activity on industries like airlines and ticketing are well-documented, no organization – large or small, public or private – is immune.”

Broken down by industry:

Financial services companies had the highest percentage of bad bots (42.2%) and typically suffer from attempts to access user accounts.

Ticketing had the second highest percentage with 39.3%. Bots prevalent here: scalping bots, seat inventory checkers, and credential stuffing bots accessing user accounts.

Education had 37.9% bad bot traffic, which look for research papers, class availability, and user accounts access.

Government, with 29.9% of bad bots, tries to protect business registration listings from scraping bots, and to stop election bots from interfering with voter registration accounts.

Gambling and gaming, with 25.9% of bad bot traffic, suffers from aggregators relentlessly scraping for ever-changing betting lines, and trying to take over accounts to access money or loyalty points that, if compromised, are transferable to another user.

Airline bots at 25.9%, scrape for pricing information, and account takeover access to empty airline mile balances.

E-commerce companies. (18.0% of the bad bot traffic). A wide range of bad bot attacks include price and content scraping, account takeovers, credit card fraud, and gift card abuse.

“To add insult to injury, the financial investment sector also deploys bots to scrape for information such as inventory levels and pricing data,” the Distil study reported. This information, sometimes known as alternative data, is used by hedge funds to make investment decisions.

Distil acknowledged it’s well known that bots were used to exploit social media sites in an attempt to influence political dialogue and elections, but held the real motivation behind the majority of bad bots is simpler, money. While the goal of each bot operator might be different depending on their industry, the report also noted, bots are the tool of choice and are vital to their success.

The bot report also revealed not only do businesses have to deal with the competitive pricing pressure resulting from the scraping bots, but have to maintain infrastructure uptime and redundancy so that real customers aren’t inconvenienced. In addition, they also suffer from skewed decision-making metrics because their web traffic has been polluted by bad bots.

Other Key findings from the report:

• In 2018, bad bots accounted for about 1 in 5 website requests. Good bots decreased slightly to make up 17.5% of traffic.
• Almost 74% of bad bots are classified as advanced persistent bots, which are characterized by their ability to cycle through random IP addresses, enter through anonymous proxies, change their identities, and mimic human behavior.
• Nearly 50% of bad bots report their user agent as Chrome. Mobile browsers, such as Safari Mobile, Android and Opera increased from 10.4% last year to 13.9%.
• Amazon is the leading ISP for originating bad bot traffic. In 2018, 18% of bad bot traffic originated from Amazon compared with 10.62% the previous year.
• Despite 53.4% of bot traffic originating from the United States, Russia and Ukraine combined made up nearly half (48.2%) of country-specific IP block requests.