User data is critical in product decision making, but figuring out the approach for data collection and synthesis can be a challenge; knowing which questions to ask, which methods to use, and how much data is needed. At Ostmodern we think it makes sense for user data to grow incrementally, and balancing quantitative with qualitative methods is key.
Clients often approach us with some sense of what might be missing in their user data. While this understanding may not always be fully developed, they recognise the potential for deeper insights. Some clients have highly sophisticated traffic data; others incorporate personas into their decision making process, but need them re-validated and refreshed.
Regardless of whether clients possess qualitative or quantitative data - or none at all - we help clients identify the specific data needed, then distil the insights that help address key challenges.
Why do we rely on data when designing products?
Data is, at its core, just information - it helps us understand who our users are, their pain points and motivations, and how they interact with products. It gives us the tools to make informed decisions about how to resolve user challenges and demands.
When user insights are excluded from design and strategy, decisions are made based on assumptions (essentially hunches and guesswork), which jeopardises the service’s chance of success.
What is the difference between qualitative and quantitative user research?
There are many different activities and methods we can use to gather data, and data collection methods fall under either: Qualitative or Quantitative research.
Qualitative research delves into the ‘why?’ and ‘how?’ users interact with digital experiences in the manner that they do, offering insight into user motivations, behaviours and mindsets. It’s broadly about gathering non-numerical data. It allows us to uncover knowledge gaps within products and services, and provides context around problems and successes.
Quantitative research is often associated with collecting numerical data, predominantly to answer questions such as ‘what?’, ‘how many?’ and ‘who?’. This approach helps us to measure, compare, cluster information and spot patterns. Due to its numerical nature, quantitative data allows us to review large and broad data sets such as subscriber demographics and Google Analytics.
Traditionally, quantitative data has been favoured in business contexts for its measurable nature, whereas qualitative data has been preferred in experience design for its rich insight into user behaviour and motivation.
At Ostmodern we believe that both types of data are valuable in their own right, but they deliver the greatest value when used to compliment each other.
‘At Ostmodern we believe that both types of data are valuable in their own right, but they deliver the greatest value when used to compliment each other.’
How do we decide what to research, and which method to use?
When deciding on a suitable research method, it's important to identify the key research questions that need answering. These questions should come from the core team and reflect the priorities of the project and wider business, to ensure that research is meaningful and based on service needs. Prioritisation is also key to understanding what is critical to find out in what order.
Usually, the process goes like this:
- What do I want to know? (the research priorities questions)
- What information do I need to collect to answer these questions? (what type of data and information needs to be collected)
- How can I get this information? (what research methods need to be used)
The quality of research outcomes hinges on how well planned and crafted research is. Being proactive about research needs, rather than reactive to user behaviours and issues, helps clients to be more strategic in their decision making.
‘The quality of research outcomes hinges on how well planned and crafted research is.’
How can we tell if we have gathered enough information?
Good questioning is critical. An example from World War II helps to personify the importance of this.
Researchers studied fighter planes returning from battle to see where they got hit by bullets most, in order to reinforce and protect future planes. However, researchers failed to consider that these were the planes that had returned home safely - the most vulnerable parts of the plane were probably in the places where there were no bullet holes - planes hit in those areas obviously never made it back from battle.
Digital product design is less often about life and death, but the example speaks volumes. We find that when clients rely too heavily on analytics and traffic data, they can miss out on critical information like ‘who isn’t coming to the site?’, or ‘why are some customers choosing to go elsewhere?’ It just reinforces the remit of what they already understand - a lack of qualitative understanding, makes it harder to think critically around problems. We must correctly interpret data and be aware of its limits.
After each round of research - collection and synthesis, it's important to capture new questions that emerge and reassess research objectives, to see if a change of direction is needed. Iterating the research plan and method as we go helps us keep insights valuable and plug holes in our understanding.
Can research be used for innovation?
One common misconception about research is that the practice can only look at what is, not what will be; simply testing and reporting what users are currently doing, not looking to the future. However, good research practice uncovers way more, it unearths problems we may not be able to see or describe yet.
“If I’d asked people what they wanted they would have said faster horses.” As the ever-famous Henry Ford quote illustrates, people are very good at describing the problems and challenges they have, but less good at proposing useful solutions.
Innovation is inherently risky, as with anything we need to grow, solutions are not always comfortable nor a guaranteed success on the first try. User research helps us to progress with less risk, by proving and disproving, and validating concepts and ideas.
‘Innovation is inherently risky, as with anything we need to grow, solutions are not always comfortable nor a guaranteed success on the first try. User research helps us to progress with less risk…’
At Ostmodern, we believe in continuous research throughout our process; consistently validating hypotheses and assumptions as we go, and making informed decisions based on real life references. Methods can vary from generative user interviews during discovery, to usability testing with prototypes during the concept phases depending on the stage of development and type of data we need.
Summary
Leveraging both qualitative and quantitative methods is crucial for conducting valuable research. We recommend using a combination of these approaches whenever possible, and urge clients to remember the importance of interpreting data with nuance. Research is scalable; it can be light and quick or broad and in-depth.
If clients find themselves in limited situations, it’s best to prioritise answers which will have the greatest impact on the product, then create opportunities for continuous learning. Qualitative methods can provide explanations about why something is happening and quantitative methods can help size up the issue.
