Why small data is the next big thing.

SMALL data is set to be the next big thing in 2023, a leading Yorkshire-based tech expert has predicted. 

For the past two decades, businesses have been obsessed with utilising the opportunities which “big data” can provide. 

But data-specialist Dash Tabor says the paradigm is shifting – and the next 12 months will be crucial in this process. 

Ms Tabor, the co-founder of TUBR, said: “The Big Data era has resulted in 96% of companies collecting data in the hope they will be able to gather insight which will improve their business metrics. 

“However, the lack of understanding around how data works, what it can be used for, and how it can be used in a successful way, hasn’t completely translated through the data collection era. 

“The tools available for data today require big data, which means lots and lots of data points. However the simple fact is most companies don’t have the massive amounts of data needed to use the tools. In fact, only 1% of companies can really benefit from these tools and 85% of companies will never collect enough data to use traditional ML/AI products. I believe we will see more and more investment in this area over the next 12 month and that small data will be a big thing moving forward.”

On the challenges that lie ahead, Ms Tabor continued: “Many business leaders are still asking themselves “how can data impact my business?” and the fact is they won’t know without a significant change in how we view data.  Many reports predict 70% of companies to move from big data to small and wide data by 2025. This means utilising smaller amounts of data to find relevant insights.  Companies are changing how they market and operate with a look to become more local in their planning but there’s a long way to go.” 

Outlining the benefits of small data, Dash said: “Using big data at scale is a massive effort, and one that requires tremendous computer power for analysis purposes. Small data is easier. Analysing small chunks of data can be done very efficiently, without much investment of time and effort. This means that small data is more actionable than big data.  Also, it’s worth noting that small data is everywhere: Small data is already widely-available for many industries. For example, social media provides a lot of actionable bytes of data that can be utilised for a wide variety of purposes, marketing or otherwise. 

“Small data also focuses on the end user: With small data, researchers can target the end user and their needs first. Small data provides the why behind end user behavior. In many use cases, small data is a fast, efficient approach to analysis and can help inform powerful insights about customers across industries.”

Very few tools are on the market for small data inputs and even less for ML/AI. But Ms Tabor and her team have developed solutions via TUBR, which is a machine learning service that can take small data inputs and provide back predictions that help you anticipate your customers needs so you can plan your operations. 

Explaining the issues which can cause small data Dash added: “The main one is that 

companies don’t generate enough data and probably never will. Covid has broken the trend and made any previous history invalid meaning trends are changing. The historical data available now is likely not going to provide the right learnings. It’s vital we look to change the way we collect data. Data with gaps often happens because collection devices break or don’t send data quickly enough creating small data inputs. These are problems which can then take time, and money, to fix.”