A small amount of bias is part of being a human with past experiences. But have you ever thought about how much of it you are bringing to your research projects?
In this workshop, Emma Howell explored the role of bias in our research and ways to reduce its effect.
Emma kicked off by admitting that every single bit of research she has ever done has been biased, even though she started her career within academia, where they had standard operating procedures setting out exactly how each test should be carried out, and data had to be scrapped if a variable was thrown by an external factor – such as a zoo visitor throwing a ham sandwich at the baboons she was studying.
She showed how bias can impact on user research, using Coca Cola as a case study. In the 1980s Coca Cola lost market position to Pepsi and conducted product development research to develop a new, sweeter recipe. This backfired massively, and after $4m in development costs, the company was left with $30m of product that their customers hated. Looking back, Emma observed that it is easy to see how bias had affected their research:
The conclusion? If you are bias, you can design the wrong stuff, which no one wants.
Emma asked participants to think about the last research session they were involved in to identify bias. Participants were encouraged to think about bias in the questioning, bias caused by environmental factors, and bias caused by participant choice, then share their experiences.
The way we act within a test session can have a massive impact on our participants and can change the findings of our research.
Emma used the example of Wilhelm, who tried to teach animals to count. He found a horse that he believed could count by tapping its foot, so he trained it to carry out calculations and took it round the country to show it off. However, when someone watch Wilhelm, rather than the horse, they noticed that he was giving very subtle signals that told the horse when to stop tapping its hoof. The horse was still very clever to pick up on those signals, but it wasn’t doing what people thought it was doing.
Next, Emma asked participants to think not just about individual research sessions, but a whole research project, and how bias can creep in at different stages. She broke the project down into several stages, including the kick off, designing the research, planning the research, set up, running the research session, analysing the data and delivering the results. The exercise was described as being like therapy!
Emma then asked participants to pick one or two of their examples and identify ways they could minimise the effects. She focussed on one example: when someone already has a solution to the problem before the research is even designed. She suggested that listing all assumptions at the beginning of the project, directly challenging assumptions in the discussion guide, and educating stakeholders could all help to reduce the impact of this bias.
She concluded by sharing some of the things she tries to do to reduce bias in her own research.
Before the session:
During a session:
After the session:
Emma has 12 years of research experience, starting in academia before finding her way to crafting lovely digital stuff. She has a mixed bag of research experience. Her fave bits of research include:
She’s currently a Senior User Experience Consultant and Research Lead at cxpartners.
