Tips for conducting international user research

February 17, 2017February 17, 2017Leave a comment

This week I’ve once again been conducting and overseeing an international user research project, so I thought I’d share with you my top tips for finding trusted research partners, managing and conducting international research projects.

Finding a trusted partner

When it comes to conducting user research in another country, your biggest challenge will be finding an agency you can trust to:

Be there for you when you need them.
Communicate well enough in your language that nothing is lost in translation.
Have good project management in place.
Find good quality participants/users.
Provide a good quality technical setup.
Conduct user testing well (many aren’t used to this).
Not rip you off.

Beware of time zone differences

With any agency you use, it’s important they are there for you when you need them. If you send them a message about a project, you will need to take into consideration the differences in time zones as it can cause delay to your projects.

I’ve recently been working with an agency in New York and on another project with Mexico, so have had to be flexible with my hours, working into the late evenings for calls and emails. I also have to bear in mind that when I want something the next morning, I’ll have to wait until 2pm at the earliest to receive a reply.

Good communication is crucial

You’ll need to check both their written and verbal ability to speak your language. It’s no point them conducting great research, if they can’t communicate the results well in the report or presentation. For example, I find that people in some Asian countries write English very well but it’s more difficult to understand their spoken English especially as we’re often communicating over the phone/internet so the line isn’t always very clear.

Project management needs to be awesome

Leading on from communication, their project management needs to be really tight. How are they getting on with their recruitment? When are they seeing their first user? How did the first interview go? Do they have any concerns? You shouldn’t need to chase them too much, a good partner will be pro-active in letting you know where they’re up to and any potential issues in the project.

Finding good quality participants

It can be difficult enough finding a good user recruitment agency in your own country, nevermind one in a different country where you don’t speak the same native language or know the culture. You’ll need to feel certain the agency can recruit your user types and do it in the time they state. Trust your gut feeling and beware of costs that appear too low as they may be using methods that are easier for them but attract lower quality participants.

Technical setup

Always ask for a sample of a previous research video, to check the moderator or agency’s previous recording quality. Check the video and audio quality, especially where devices and screens are involved. You’ll also need to check that they have the equipment and software to test mobile devices.

Can the agency conduct user testing?

This is something you’ll need to check. If they say they do user testing, you’ll also need to check to confirm they’re telling the truth – many market research agencies have added this on to their list of services without any experience. It doesn’t necessarily stop me from using them – it depends on the project and I may decide to spend a little time training them over Skype if, for example, the website testing is a smaller part of the research hour.

Are they ripping you off?

I’ve certainly experienced receiving a quote from a research agency or a recruitment agency in another country that I knew was far too high. Watch out for agencies who see you’re from (the UK in my case) and hike their prices up. If I feel they’re way too high and they’re taking the mick, I’ll simply go to another agency. If I feel they’re slightly too high I’ll address it directly with them to see if they’ll come down. Remember, for some countries, negotiating and haggling prices is expected.

Add additional translation time

Translation is one of those activities that takes far longer than you expect. If you’re getting a translator in your own country to translate research conducted in another language, make sure you give yourself plenty of time, then add some more as a buffer!

Need help?

Need help with international research? keepitusable.com

Need help with recruiting users in other countries? iNeedUsers.com

Smyths: Increasing conversion through user research

January 27, 2017January 27, 2017Leave a comment

(Scroll down to watch the video)

I’ve had a fantastic time this week working with Smyths Toys to improve their future e-commerce website.

As part of improving your customer experience and online conversion, it’s crucial to involve user research / user testing. If you’re job title includes the two letters ‘U’ and ‘X’ but you’re not including real life users in the design process, then you’re kidding yourself, you are not a UX designer, you’re just a designer. But don’t worry, you don’t need to do the research yourself, it might be another member of your team or even an external agency, but it’s so crucial to your understanding of your customers, how they think and how they behave that without research, it’s unlikely the experiences you design will be as successful because the more you get to know your users, the better fit your designs have to their needs and the more you’ll be able to influence their behaviour.

Smyths Toys have been working to improve their e-commerce website, particularly their mobile experience. They recognise the importance of listening to their customers, watching how they interact with their website and making design changes to be more in line with customer needs in the online environment. Of course, they’re aware that the more they do this, the more customers will enjoy using their site, they’ll be more likely to transact with them as opposed to a competitor, they’re likely to increase their basket size, return to buy again in the future, recommend the website to friends and family and shop in-store.

Here’s what the team had to say:

It’s important for us to get feedback from test candidates to see if we’re on the right track and we certainly spotted some things that we need to work on in the future and the team here helped us to identify those little issues and gave good recommendations on how to improve those for the future. And that’s what we’re going to do now… go back home, work on it and come back to do another session to evaluate that we’ve done the right changes.

Companies used in this research:

UX agency for UK research: Keep It Usable

User recruitment: INeedUsers.com

UX Lab: HomeUXLab.com

Mobile UX and ‘thick users’

April 1, 2016October 28, 2016Leave a comment

7 years ago today my time with Sony Ericsson came to an end.

I used to work for them as a software and hardware usability specialist. Our part of the company was dedicated to smartphones. Back then, they weren’t mainstream. They were incredibly expensive, a status symbol, mostly owned by business people. Perhaps you had one? (leave a comment if you did!)

At that time, there were no guidelines in the public domain for smartphone UI design. When it came to things like navigation, fundamentals, hit areas, button sizes, tactile feedback, hardware ergonomics, we had to design and test everything from scratch. And this was much more difficult than it is today.

Design and prototyping

Believe it or not those lovely prototyping tools you’re used to using didn’t exist back in the early smartphone days. Seriously, count yourselves lucky you have these! The UI design team used to use Adobe products for design. For prototyping, Macromedia Director and Flash were firm favourites.

For the early flip style smartphones, we had to design not just for one style of interaction (full touch) but there were actually three interaction paradigms!

1. Full touch. This is like what you have with your current smartphone – the full UI has touch interaction.

2. Full flip keypad. With the flip closed, the UI could be fully navigated and interacted with just using the hardware keys on the flip.

3. Combined: Touch and keypad. With the flip in the closed position, the touchscreen shrunk to the smaller size but it could still be pressed using touch. The UI could also be fully navigated and interacted with using the hardware keys on the flip. This meant for a complex interaction style. Everything that was designed had to be tested with three interaction paradigms – complex stuff!

Old skool mobile user research

User testing meant looking over the user’s shoulder to see what they were doing. This was coupled with note taking at the speed of lightning to miss as little as possible!

Conducting research with mobile users years ago was fun to say the least. We didn’t have the means to record mobile UIs, so it meant looking over the user’s shoulder to see what they were doing. This was coupled with note taking at the speed of lightning to miss as little as possible and get everything down before you forgot it, being mindful that as you were scribbling, you were missing further user interaction. As the researcher, you then had to also follow your discussion guide and focus on maintaining the flow of the interview. I developed the knack of note taking without looking at the paper in the end – it wasn’t pretty but it worked a treat!

HIPPOs and developers

Developers were particularly problematic and I remember seeing red once when one said to me I must have asked ‘thick users’

At the end of the research there was no video evidence so then began the battle of convincing stakeholders. Developers were particularly problematic and I remember seeing red once when one said to me I must have asked ‘thick users’ because my research findings didn’t agree with their personal opinion. Seriously… I’ve heard it all! Patience is a definite requirement of any UX person and fortunately I have bags of it – queue a big friendly smile and a simple explanation of why users aren’t thick.

HIPPOs were also a huge problem. This is when the highest paid persons opinion overrules everyone else (in our case this was made worse by the fact the top decision makers were based in another country). It’s still incredibly common in companies and the only way to overcome it is to get the HIPPO on your side. Befriend them, educate them, show them evidence, let them think they’re making the decision.

Get the HIPPO on your side. Befriend them, educate them, show them evidence, let them think they’re making the decision

Running around corridors after users…

I remember a time when I wanted to replicate more natural usage of mobile, so I tasked users with walking down the corridor whilst carrying out tasks. Of course, this meant I had to scurry along behind them, trying to see what they were doing whilst making notes, remembering my guide, asking questions, etc, etc. It won’t come as a surprise to you to hear I didn’t do this again in a hurry! There’s only so much multi tasking one person is capable of.

Mobile research is so much easier now, thankfully!

Twinkeys and no keys… dealing with poor hardware usability

Our industrial designers were based over in Sweden, silo’d from the UI team. One day the hardware would just turn up and there’d be crucial functions missing that had been specified in the software. This then meant a long battle to make changes. I’m a qualified ergonomist so I adapted my role to include focus on hardware usability and worked on building relationships with the ID team. This worked really well and in the end they genuinely appreciated having someone to review their early design mockups and be the intermediary between them and the UI team.

Everyone benefits from capturing potential issues as early as possible

What happened to mobile innovation?

I was fortunate to make my way into several future concept groups and to help define some incredible future technology for mobile devices. There were some amazing things in the pipeline that I still haven’t seen on any devices. It feels as though mobile innovation has come to a bit of a standstill since the iPhone. I’m really looking forward to the day when the next big tech change in mobile happens.

7 years on…mobile is bigger than ever!

So, 7 years have passed… how did time go so fast!? Mobile is now bigger than ever and smartphones are mainstream. Most of my work in mobiles now involves helping companies to improve their mobile website conversion or their mobile app user experience.

Despite the fact that mobile is now huge, it remains the most difficult platform to design for

So many well known brands still make obvious mistakes in their mobile experiences.

There’s a real opportunity to stand out if your brand offers the best mobile experience

Need help with mobile?

Then you’re looking in the right place! At this point I should probably point out that at Keep It Usable, we also have the UX designer of the first ever smartphone.

Our mobile expertise is unrivalled

We know mobile design and user behaviour on mobile inside out, we know what works.

PS If you used to own a Sony Ericsson smartphone let me know! 🙂

Addicted to user research

July 10, 2015July 10, 2015Leave a comment

I was recently invited to a business meal and as I found my place and settled down, I waited with happy anticipation for the others to arrive on my table. A quick look at the name cards told me that my direct table mates to my right and left were both males and judging from a quick look round the room they were going to be middle aged or older.

The gentleman to my right was the first to arrive, let’s imagine he was called John. John funnily enough did turn out to be middle aged but he was very clued up on technology, having run a social media agency in his past. He was a very interesting character, having previously worked in PR for celebrities and lived a rather extravagant life.

David

To my left was an older gentleman, let’s call him David. The curious researcher inside me lit up when I clocked David. Most of our clients want research with millennials, so although we do research with older people, it’s not that often. Yet I find talking to older people quite fascinating. They’re generally quite good at reflecting on their behaviour, on why they do what they do and it’s so enjoyable to listen to.

David was an intelligent gentleman. He’d had an incredibly successful career and worked in top positions in very high profile high street brands. As I asked him about the technology he used and how he shops, I found myself entering research mode, engaging in a very interesting conversation about his shopping preferences and how they change depending on the type of product.

David and technology

I was curious as to the devices David owned. I was fairly surprised to hear that he owned a Macbook. Apple is a brand we generally associate with the younger audience, however, David was incredibly enthusiastic about his experience so far. When I questioned his choice, he immediately stated ‘ease of use’ as the key reason and that ‘it just works’. He told me all about the problems he used to have with Windows computers and how in comparison, his Mac was just so simple to use.

Do you think David had a tablet? Well, yes he did have a tablet. Knowing that many of the older generation are given hand me downs from sons/daughters, especially to communicate, when he told me that he used his tablet to communicate with his son and grandchildren in another country, I was quick to enquire how he had become the owner of an iPad – was it his love of Apple having an influence or was it indeed a hand me down? It turned out to be a hand me down from his son so they could keep in touch.

When it came to his mobile device, David was, I’d say, very typical of his generation. At this point he pulled out a mobile from his pocket that most young people would probably not even recognise and think belonged in a museum. It was an old, very worn, Nokia phone, with just a 0-9 keypad and a non-touch screen. Having a long mobile history myself (I used to work at Sony Ericsson on smartphones and turned down a job at Nokia) I just had to take a photo! I was quite overjoyed to see this relic still in use. He clearly still cared for it too, as he’d kept the plastic cover on the screen (see pic below).

David explained that he had absolutely no need for one of these new types of phones. Everything he felt he needed a phone for he could do using his old Nokia. It was interesting but not surprising that although David had the latest computer and tablet tech, he had no interest in updating his phone. Ease of use was very important to him. For David, his Macbook made his life easier. His tablet made communication with his family easier. They had clear benefits. However, he saw his old Nokia, with it’s limited features as the simplest mobile for him. A smartphone with it’s array of features was perceived as a hindrance.

How does David shop?

When it came to shopping, David was more than happy to shop online using his Macbook. He was very satisfied with the convenience of shopping from the comfort of his home. However, I suspected there would be exceptions to this generalisation and when I explored more deeply, it was clear that David had different rules for different types of products and services that he purchased. There were some physical products that David insisted you needed to shop in person for. There was a clear theme throughout the examples he gave and that was products that have strong sensory qualities, particularly tactile qualities. One example David talked about was shoes because ‘you need to try them on to see what they’ll feel like’.

I can’t tell you how much I enjoyed speaking with David about his use of technology and how he shops. The older generation are often under-represented within businesses yet they’re an important consumer base to consider. It’s important to remember that as time progresses they are changing as consumers. They’re becoming more comfortable with technology, they’re owning the latest devices thanks to influence and hand me downs from their children, they’re seeing the benefits that technology can give them and with their children all grown up, they have plenty of disposable cash. But they’re clever shoppers. They want to know what they’re buying is the best for them and that it’s easy for them to use.

Improving your research validity

April 10, 2015April 10, 20151 Comment

Research validity is incredibly important, without it you risk biasing and even invalidating your research efforts.

Good validity = good research

What is research validity?

Validity refers to the quality of research. In short, the rigor or trustworthiness. Joppe (2000) provides the following explanation of validity:

“Validity determines whether the research truly measures that which it was intended to measure or how truthful the research results are.”

How can it go wrong?

The main cause of poor research validity is caused by the interviewer and can be due to inexperience, poor interview skills or subjective bias that they’re unaware of. Asking leading questions ‘Would you say design A is the best then?’ is especially common as it can take years to perfect this art but leading questions give misleading and false answers, therefore biasing and even invalidating your data.

Can you imagine if you recommend going with design A when actually if the research had been conducted differently the answer might have been to go with design B.

It’s important to remember that users won’t just sit there and tell you their honest opinions, they want to be helpful, they want to please you, so they are highly influenced by the interviewer. It’s the interviewer’s job to be as objective and open minded as possible, not transferring any bias through poorly worded leading questions, tone of voice, facial expressions or body language to the user.

How to improve your validity

Unbias your discussion guide

I’ve heard stories from clients of receiving discussion guides full of leading questions. Fail at this stage and your bound to fail at the interview stage. The first step is to ensure all the questions in the discussion guide are fully thought through, flow well and all leading questions re-worded – Try using open questions that begin with the 5 Ws: What, Why, Who, When, Where and How.

Practice like an actor

The more familiar you are with what you’re testing and the questions in the discussion guide, the greater ability you’ll have to stray away from the discussion guide when you need to. This preparation will also free your mind to give you greater ability to construct questions on the spot that aren’t leading. The more your cognitive resources are taken up the less likely you’ll be able to construct a good question, so by reading through the discussion guide multiple times in the days leading up to the research you’ll hopefully be so familiar and comfortable that you’ll be able to conduct some great research!

Watch your performance critically

When you re-watch the research interviews, watch out for moments where you may have influenced the participant. Think about how you could ask the same question but in a different way to get a better response. Depending on the amount of influence you may have had, you may need to remove that user’s response from your data analysis. However, being critical of your own performance and learning from your mistakes is the best thing you can do to improve for next time.

Don’t worry about the odd slip

It’s bound to happen. More at the start and less the more experienced you get. But even with experience, the odd slip will happen. Don’t worry about it too much. With experience you’ll recognise your mistake the moment the words leave your mouth. Don’t let the frustration of your error show to the user, let them answer, then later on re-word the question in a different way to double check their response. If it differs from their previous response then you know you biased their previous response, just take their new answer as their true opinion.

A note from me:

This is a post I recently published on Linkedin Pulse. If you like it, make sure you follow my future posts or connect with me Linkedin

Infographic: 2015 E-commerce Opportunities

October 24, 2014October 24, 2014Leave a comment

Never use the word ‘Test’

June 4, 2014July 4, 20145 Comments

When I first started life as a user researcher, it was commonplace (and it it still is) to refer to research as user testing or usability testing. I soon observed that when you use the word ‘test’ it:

a) Implies that you’re testing the end user (which is wrong, you’re testing the interface, you’re understanding of the customer, your user journeys, etc).

b) As soon as you mention the word ‘test’ to a participant they instantly tense up and worry. I used to say something along the lines of ‘please don’t be concerned, we’re not testing you, we’re testing the software’ and even this was too much. It’s a bit like if I say to you, don’t think of a pink elephant, the first thing you think of is a pink elephant – you just can’t help it, it’s how the human brain works.

I also noticed that when I used the word ‘test’ sometimes participants would ask me during the research session how they were doing or ask whether they’d got something right. In effect, they were treating it like a test. I haven’t experienced this since I stopped all use of the word ‘test’.

Now, when speaking with participants I always use the word ‘research’ which has a much more positive connotation. Of course, clients still use terms like user testing, and that’s absolutely fine, let’s not undo all the hard work ux professionals have done over the years to gain awareness of what we do, but let’s keep in mind that we’re always researching and aiming to understand things from the perspective of your target audience.

Have fun researching! 🙂

Information Architecture (IA)

October 26, 2012October 27, 20124 Comments

Have you ever been to a website specifically to look for something and no matter how hard you look you just can’t find it? Most people will give up within a few seconds, hit the back button and go to a competitor. This is why your Information architecture is incredibly important – get it right and you will keep more people within your site, lowering your bounce rate and improving your conversion.

What is information architecture?

In simple terms, it’s about structuring your content to feel intuitive and logical to the end user.

An example of how not to do it

Tesco Direct have placed Halloween items within the heading ‘Christmas’ on the navigation bar. Users will struggle to find this as it makes no logical sense – halloween and christmas are completely separate occasions.

If a visitor to your website has the intention of browsing halloween things, they will already have expectations of where halloween things will be. Your aim is to try to understand their expectations of where they’ll find halloween related products. Only when you understand this, can you position it in the optimal place.

Card Sorting to create intuitive IA

One of the methods I employ to help create intuitive Information Architecture is Card Sorting. It’s an activity carried out with users (i.e. your target audience) using labelled cards to group and organise pages of content. Users categorise the pages in the way that makes sense to them and they can use existing grouping or create their own. What this enables us to do is to see the structure of your site or software from the user’s point of view – we can see and understand their mental model.

Card sorting exercise in action:

If you’d like to understand more about how reviewing your IA can help your business or if you’re curious about card sorting please don’t hesitate to get in touch.

UX Booth guest post: 5 Useful Lies to Tell User Research Participants

June 19, 2012June 21, 2012Leave a comment

Do you read UX Booth? If you’re interested in User Experience then bookmark it now! It’s been my favourite site for reading interesting and useful UX articles for a number of years now. As such, when I decided to write a guest post, they were my first choice. I decided to write about 5 little (white) lies that can be told during user research interviews to gain higher validity data.

I’ve conducted hundreds of research interviews and I’ve picked up a few useful techniques along the way to encourage the best out of the people I interview. This includes making them feel more at ease, increasing rappor, gaining trust and encouraging an open dialogue where it is ok to be 100% honest.

Active and passive deception has been used in research for a long time. In the past it was unfortunately used unethically and there are a lot of examples out there of how not to use deception. The Milgram experiment is one of the most known for the psychological and physical damage it caused.

Of course, all the lies I use and recommend are incredibly nice. They’re white lies and many UX researchers use some or even all of them. You don’t have to use any but they are a useful tool to have in your UX toolbox. Enjoy! 🙂

Read the article: 5 Useful Lies to Tell User Research Participants

BBC Home page research and redesign

May 27, 2012June 4, 20123 Comments

I conducted a quick piece of research on the BBC Home page as part of an event called BBC Connected. Did you know it’s the third most visited home page? Yet only a very small percent of visitors actually use it? As our research discovered, most people bypass the page completely, preferring to use the navigation bar or a direct url (usually saved as a bookmark). Here, we share our findings and a few of our design proposals to improve the user experience of the BBC Home page, in particular under-served audience(s).

How do you use the BBC website? Do you ever look at or click on items on the home page? Have you used it more or less since the last redesign? I’d love to hear about your experience. Share your story using the comment box below.

Research by Keepitusable.com

Lisa Marie Ortega

Digital psychology, online behaviour and ux

Tag: user research