Why Are The Right Insights So Hard to Find?

We are drowning in data. By 2025, we will be swimming in at least 175 Zettabytes of data. 90% of the world’s data was generated in the past 2 years!

At the same time, we are also trying to be more data-driven. Organisations are transforming themselves to figure out how best to utilise these treasure troves of data to help them analyse their processes, improve their performance, learn more about their customers, and even predict the future. However, finding the right insights is not easy. Here are some reasons why that is so:

1. There’s simply too much information

While the amount of information available to us has been increasing, our ability to process this information has not changed significantly. Drinking from a firehose is a common saying to describe how vast amount of information can easily drown and overwhelm us. We are living in a super-connected world where we are processing information from so many different sources: emails, Twitter, Facebook, Instagram, Youtube, Discord, Whatsapp, text messages, etc.
Let’s just use email as an example. On average, knowledge workers spend 20 hours a week reading and responding to emails. They check their email apps up to 100 times per day, and yet only 40% is considered crucial. As the internet empowers everyone to have a voice, the amount of content that is generated increases exponentially. As Internet of Things (IoT) devices become more prevalent, we will be generating data at an even higher rate.

2. There’s too much noise

In addition to having to be the recipient of so much information, we are also inundated by noise. These are irrelevant information that slowly whittles away our cognitive energy. As we process and filter these information, our ability to make good decisions depletes too.

In the context of insights generation, that means filtering out irrelevant information and downright incorrect information. The ability to identify the most relevant information to use and assess their credibility still lie on human cognition. As people have more access to generate and distribute data and information, it will become even more difficult to separate the chaff from the wheat.

3. Fragmentation

Data comes in all shapes and sizes, and formats and languages. Having access to a broad range of data helps avoid sample size bias and helps you generate a more objective and comprehensive view of the world. At the same time, data is often collected, processed, and presented differently depending on their use cases, sources, and time during which they are collected. Market size data collected for industry growth analysis may differ from market size data collected for production growth analysis. Unemployment data collected by the government may differ from unemployment data collected by academic institutions. The same type of data that is collected across two different periods may not be 100% comparable due to changes in data collection mechanism, sampling errors, and other external factors.

Often times, to generate truly meaningful insights, one may need to combine fragmented data from different sources. This process takes significant time and effort, requiring various steps to vet the data, standardise them before analysis can start.

4. Context

Once you are ready to generate your insights, you need context. You need to have a clear goal of what you are trying to achieve with the data that you have collected and cleaned. The same set of data can yield different insights depending on what you are using them for.

For instance, mobility data can be used to determine footfall traffic in a shopping mall if you are trying to understand the best location to open a new retail store. The same data can also be used to determine crowd clustering if say you are analysing for hotspots for a virus transmission. Having a clear goal at the beginning will help you generate insights that are relevant and contextualised to your business needs.

Are you looking for knowledge workers who can help you with gathering insights or doing specific research? 

Ravenry curates a network of expert researchers, analysts, and writers who have worked for the likes of Deloitte, PwC, Oliver Wyman, and MIT Technology Review. 

Our clients leverage Ravenry to quickly scale their knowledge and content team on-demand. Expand your team with experts today.

More Articles

The importance of time management to freelancers

Everyone has the same number of hours in a day, but how they utilize each and every minute is entirely up to them. As such, time management is vital to those who want to get the most out of each

How technology is changing the way that freelancers work

The term ‘freelance’ actually dates back to the early 19th century and referred to independent individuals who would sell their skills to others. While the meaning of the word has not changed much even two centuries later, the ways in

Why hiring a consultant for your business is beneficial

As employers grow their businesses, they may come across some roadblocks that they have difficulty overcoming. This is where a consultant comes in. Consultants are people who excel at a particular field; they hence have the skills and expertise necessary

Subscribe to our newsletter for more interesting insights!

Send us your feedback!