Research validity is incredibly important, without it you risk biasing and even invalidating your research efforts.
Good validity = good research
What is research validity?
Validity refers to the quality of research. In short, the rigor or trustworthiness. Joppe (2000) provides the following explanation of validity:
“Validity determines whether the research truly measures that which it was intended to measure or how truthful the research results are.”
How can it go wrong?
The main cause of poor research validity is caused by the interviewer and can be due to inexperience, poor interview skills or subjective bias that they’re unaware of. Asking leading questions ‘Would you say design A is the best then?’ is especially common as it can take years to perfect this art but leading questions give misleading and false answers, therefore biasing and even invalidating your data.
Can you imagine if you recommend going with design A when actually if the research had been conducted differently the answer might have been to go with design B.
It’s important to remember that users won’t just sit there and tell you their honest opinions, they want to be helpful, they want to please you, so they are highly influenced by the interviewer. It’s the interviewer’s job to be as objective and open minded as possible, not transferring any bias through poorly worded leading questions, tone of voice, facial expressions or body language to the user.
How to improve your validity
Unbias your discussion guide
I’ve heard stories from clients of receiving discussion guides full of leading questions. Fail at this stage and your bound to fail at the interview stage. The first step is to ensure all the questions in the discussion guide are fully thought through, flow well and all leading questions re-worded – Try using open questions that begin with the 5 Ws: What, Why, Who, When, Where and How.
Practice like an actor
The more familiar you are with what you’re testing and the questions in the discussion guide, the greater ability you’ll have to stray away from the discussion guide when you need to. This preparation will also free your mind to give you greater ability to construct questions on the spot that aren’t leading. The more your cognitive resources are taken up the less likely you’ll be able to construct a good question, so by reading through the discussion guide multiple times in the days leading up to the research you’ll hopefully be so familiar and comfortable that you’ll be able to conduct some great research!
Watch your performance critically
When you re-watch the research interviews, watch out for moments where you may have influenced the participant. Think about how you could ask the same question but in a different way to get a better response. Depending on the amount of influence you may have had, you may need to remove that user’s response from your data analysis. However, being critical of your own performance and learning from your mistakes is the best thing you can do to improve for next time.
Don’t worry about the odd slip
It’s bound to happen. More at the start and less the more experienced you get. But even with experience, the odd slip will happen. Don’t worry about it too much. With experience you’ll recognise your mistake the moment the words leave your mouth. Don’t let the frustration of your error show to the user, let them answer, then later on re-word the question in a different way to double check their response. If it differs from their previous response then you know you biased their previous response, just take their new answer as their true opinion.
A note from me:
This is a post I recently published on Linkedin Pulse. If you like it, make sure you follow my future posts or connect with me Linkedin
One thought on “Improving your research validity”