Oxylabs, the proxy and public web data acquisition solution provider enabling companies of all sizes to utilise the power of big data, has published a whitepaper, Alternative Data Unlocks Key Decisions in the UK & US Finance Industries, finding almost three-quarters (71 per cent) of financial services organisations are now using web scraping to gather external data to guide their decision-making.
In cooperation with Censuswide, Oxylabs surveyed 500 UK-based and 501 US-based senior data decision-makers from UK and US financial services companies, gathering insights on attitudes towards data collection in the sector.
Internal data (74 per cent) remains the most popular source for businesses, but there are clear signs web scraping is set to overtake it in the near future. Over two-thirds of respondents (68 per cent) plan to increase their data budgets in the coming year, while 80 per cent believe they will focus more strongly on web scraping in the same period.
Other methods, such as third-party data aggregators (used by 56 per cent) remain an important part of the data-gathering ecosystem, but are now a long way behind web scraping and internal data.
Gediminas Rickevičius, VP of global partnerships at Oxylabs, said: “Internal data is relatively cheap, efficient and company-unique, so it’s not a surprise that it’s the dominant acquisition method. Alternative data and web scraping being so close, however, is something completely new. There’s a clear interest and even necessity to uncover new ways of unearthing data-driven insights from untapped sources. It’s clear – financial services companies are expecting to get their edge through web-scraped alternative data.”
Despite web scraping’s rise, there remain some concerns among decision-makers about how to implement it in a safe, cost-effective, and compliant manner. Almost four in ten (38 per cent) state risk and legal complications as the main factors discouraging them from using web scraping. Alongside this, 36 per cent are concerned about budget constraints or the need to introduce complicated new technology, and 34 per cent believe they lack the technical know-how to make web scraping a success.
Vaidotas Šedys, head of risk management at Oxylabs, added: “Risk is inherent to all new technologies, as pioneering a new field means there are no footsteps to follow. Any web scraping risks, however, can be greatly mitigated by applying data collection and management best practices and employing legal and risk professionals to ensure web scraping is done in a compliant, safe and ethical way.”
He concluded: “These practices may take some time to get used to, but done in the right way, there’s every chance that financial services businesses can gain valuable new insights, improve customer experiences and achieve strong ROI through web scraping.”
