For months the polls showed the Labour and Conservative parties neck and neck right up until the day before the election. So how did they get it so wrong?
Well, the first past the post electoral system rewards small differences in opinion with large numbers of seats but this is known and understood. So do polls pass the tests of validity and reliability?
Reliability, or whether those polled can be applied to a wider group, is accurate by research standards. However, validity, or whether the research is really measuring what it claims to be measuring, is more dubious.
As House M.D. says, everybody lies. Or they might be uncertain. Or they might not be able to express themselves accurately. This is why it’s not enough to just listen to your customers or ask them to complete a survey but to really understand how they interact with your product or service.
This is why our user research includes usability testing to discover the errors that people make rather than what they will tell you and ethnographic research such as contextual inquiries to see how people really use your apps, sites or products in the wild.