M_Kirk_Rodgers--User Experience Researcher

User Research Recommendations: Risk vs. Reward

Steve Bromley (a Games User Researcher) that I know online, have a great deal of respect for, but have not yet met in person did a blog post recently on “What’s in a Games User Research Report.

There are a lot of great insights in this blog post, especially for junior user researchers that want insight about how to structure a report in a way that can be consumed easily by stakeholders.  I am going to present another perspective on recommendations, however.

Steve says:

There is no recommendation for how to resolve the issue...In a typical debrief, we’d pair sharing the usability issues with a session about fixing the issues. This is in place of providing recommendations ourselves – often with something as complicated as games, many ‘obvious’ solutions would not be possible. So, to reduce the time taken suggesting things that cannot be done, instead we prefer development teams to generate solutions themselves. A workshop format means that the researcher can input their usability best practise and knowledge, while still making sure that developers lead the discussion on what fixes are appropriate.

Emphasis added by me.  I take Steve’s point, but I disagree with the concept that games are too complicated to make helpful recommendations for.  While making poorly focused or scoped recommendations will damage your credibility, I see the following advantages for making recommendations in a report:

  • Developers often know issues exist, and are unhappy that they are in the game in the first place.  It’s very rare to present a negative finding that is a complete surprise to a team.  

  • GURs are hired for insight about games as much as they are for their ability to find data.  Having an opinion about what to do about the data makes the GUR a more active participant in the development process.

  • Developers turn to user researchers in part to help provide data and direction to areas of contention.  Once a GUR has built a relationship with a team, the data/direction ration tilts more and more towards direction.  Without making recommendations, you lose the opportunity to build that credibility.

Steve (and others in the GUR discord chat) see disadvantages:

  • Games are complicated, and it’s easy to make a recommendation that’s either impossible to implement or impractical for other reasons.

  • Making the wrong recommendation damages your credibility.

  • GURs are usually not game designers, and may not think of the optimal solution.

Here are my strategies for approaching recommendations in a way that preserves your credibility as a GUR while still allowing you a stronger position within your team:

  • Understand your game, game genre, and competitors:  The biggest risk that Steve and others identified is looking stupid in front of your team.  The most important thing to do is make sure that you’re as much an expert about your game genre and game as any of the designers or engineers.  If someone brings up a point about a mode in a competitor’s game, it’s important to be able to have an answer to that.

  • Involve stakeholders in data collection and analysis:  There is a great risk if the first time the team was exposed to the general trends of your findings is at the report.  One way to mitigate this is to involve members of the game team in the research process.  Invite them into moderation rooms and jam with them between sessions.  Train them to be playtest moderators (you will be surprised how excited designers will be to moderate a playtest session.  Let them observe issues first-hand; they’ll be much more prepared to deal with them.

  • Preview results to key decision-makers (especially the ones that will have to execute recommended changes:  One technique I started using at Zoosk that I continue to do to this day is give stakeholders a preview of the results and recommendations 1-on-1, usually at their desk or somewhere similarly informal.  This gives you the opportunity to gauge their reactions, get their input on your recommendations, and workshop better, more relevant recommendations.  More than once I looked like a wizard in front of the CEO at Zoosk because I came into a meeting with results buttoned up, recommendations vetted, and presenting the plan for addressing the issues with my stakeholders, rather than simply to them.

  • Know when to push (and when not to):  Even in the best case, there will still sometimes be conflict and/or pushback about recommendations.  As a researcher, you should know ahead of time which recommendations can flex and which ones represent a clear danger to the game or the user experience.  Being flexible about solutions in most cases will give you the standing to push back when its really important.

Note: all these recommendations come from the perspective of an embedded user researcher.  Consulting or centralized user researchers may need to approach the problem differently.

Hopefully these recommendations about recommendations were helpful!

Tinder FTUX analysis

I recently bought a new phone, and this came along with the re-installation of many apps., triggering the first time user experience (FTUX) for many of them.  In the next few weeks I'll be doing microposts looking at the FTUX of some of these apps discussing what they do right and what they do wrong.  Let's start with Tinder.


After a (short) on-boarding experience that primarily is meant to connect your facebook profile to Tinder, you're faced with the above view.  No instructions, no explanations, nothing except good affordances.  Three inviting buttons anchor the bottom of the page, providing even neophytes to swiping attractive targets for interacting with the app.  In addition to that, the user card is clearly the focus of the page.  If you try to interact with the user card by tapping it, it takes you into the more information screen, which is appears to be strictly optional from Tinder's perspective. Stacked cards that suggest a stack of papers to be sorted through invite those familiar with swiping to give it a try.  When they do, they get the following results:


Sliding the card one way or another gives immediate feedback about the results of the user's actions with short, clear labels.  You can also slide it back and forth before committing and "Like" and "Nope will fade in and out appropriately.

When you let go, it commits the choice and this popover appears:


While the other feedback is omnipresent in the app, this particular pop-up is designed to make sure that users don't take an irreversible action unintentionally, provides context for what the action does, and provides a chance to undo it.  This happens the first time you swipe left, swipe right, tap on the heart, and tap on the X for each install.

Tinder does a lot right here:

  • Light-weight FTUX doesn't get in the way of the user starting to use the app immediately.
  • The visual design of the app provides clear affordances to help the user understand what actions are possible.
  • In-progress feedback for user actions help users understand what they can do without being forced to commit to an action.
  • Popovers explain the function and consequences action the first time they perform it.  They also give users a one-time escape hatch in case they didn't understand these consequences.

There are a few things that I think could use improving:

  • Moments are entirely unexplained in the context of the app.  As they are outside of the core loops of swiping and chatting, this is forgivable.
  • The onboarding slideshow includes some tutorial information, but I would wager a large percentage of users skip that, and it's never accessible again.
  • In case someone does not know what to do at all (unlikely, but possible) they should consider adding a swipe gesture or put a glowing circle on the heart after a short amount of time.

All in all, Tinder does a good job of keeping their FTUX lean and effective.

No effort + big benefit: Google Now is the future of UX

I recently was trying to soak in the last bit of fun at a street fair a few weeks ago.  My flight was in a couple of hours, and I was heading towards the train when I felt my phone buzz quietly in my pocket.  I took it out, and raised an eyebrow.

An example from google how a user might get context sensitive information about a flight cancellation

An example from google how a user might get context sensitive information about a flight cancellation

Google Now had sent me a push notification that my flight had been delayed for an hour.  I happily put my phone away and immersed myself in another hour of fun.  It wasn’t until long after I would have gotten on the train that the airline text alert that I had signed up for arrived.

That interaction seems small, but it’s a small sign of things to come; Technology will become a source of seamless assistance in our lives, quietly finding the things that need doing and providing a gentle prompt at just the right time, at just the right place.

Google Now is extraordinary in part because it requires so little setup and maintenance.  It just appeared on my phone one day with a system upgrade.  At first it seemed like a replacement search box, but slowly I began to notice it was finding articles I might be interested in, and was surprisingly accurate in its choices.  It started taking note when I went to my favorite coffee shop the same day every week, despite not creating a calendar update, and providing reminders of when I had to leave to get there on time.  It tells me--anytime I want-- how far I am from home and the most efficient way to get there.

Google Inbox is the next step in this evolution.  Others have exhaustively detailed the features, and it looks remarkably clean and distraction free, but that’s not what catches my attention about it.  I now have the ability to snooze emails until I arrive at a certain location, an ability Google Now already had for tasks.  That way, I get the right notifications at the right time and place.  A notification you simply dismiss is a wasted notification, and by delaying the notification until I am where I can properly deal with it I am less likely to forget it.  I anticipate that, over time, google will start anticipating when and where I should receive notifications for new emails based on my input, making email notifications suddenly much more salient to my life and useful for my productivity. 

I’m a UX Research expert, so I’d be remiss if I didn’t go into some detail about why I think Google Now works so well as a User Experience.  User experience can be described as the ratio between user benefit and user effort.  What google is doing is increasing user benefit while simultaneously reducing user effort through automation and predicting behavior.

Of course, there are downsides to that equation, too.  When google gets it wrong, it seems more jarring because of how natural it feels the rest of the time, as if it were a normally socially fluent person who suddenly starts spouting nonsense in the middle of a meeting.  For example, sometimes I get notifications to go to meetings that I’ve been invited to but ignored because I was too busy to decline them.  While I admit that perhaps I should have declined them to make the service work better for me, the fact that I have to change my behavior to enhance the service is somewhat irritating.

This will resolve itself in time.  Right now we're only seeing the edges of what this kind of technology will do for us in the future.  Someday I'll be wandering down a road in a city I've never been in before, and I'll get a notification about a coffee shop or art gallery that google thinks I'll like and have an experience I'd never have discovered without them.

And that will make my life better.

But people are looking!

I've been reading the rather lovely book Dataclysm: Who We Are (When We Think No One Is Looking) by Christian Rudder.  It makes some excellent points, and purports to take data from multiple sources and use it to make inferences about who we are when we think we're not being observed on dating sites.

Of particular interest to me is his graph comparing male perception of female attractiveness to female perception of male attractiveness:

If we took the graph at face value, we could believe his conclusion that "men are far more generous than women" when it comes to rating attractiveness.  he goes on to discuss how he's validated this against other dating sites and against random pictures on social networks (I'm very curious how he did this, because if he presented the pictures outside of a dating site context, many of my criticisms below might be moot).

There is a very real problem with these data, and that comes from the fact that these ratings don't live in a vacuum.  On OKC in particular (and dating sites in general) rating someone as very attractive usually triggers some kind of in-app or email notification like this:

Keep in mind that men and women experience dating sites in very different ways.  Many women are already getting more attention than they can handle, and creating an event that sends out a signal to some guy that "she likes you" complicates the question that the site is supposedly asking, which is "how attractive is this person." 

A woman not only needs to calculate how hot a person is, but how willing she is to express interest in that person, along with the likelihood that it will result in aggressive pursuit by the recipient of the notification.  ("What?" I can hear the man say, "a woman rated me highly!  I'm practically already in, just have to be super, ridiculously, and annoyingly persistent."    The "Oh, and when she doesn't respond I'll insult her and say I wasn't interested anyway." may come later).

In contrast, males on average get much fewer messages, so the possibility of a girl responding to them liking them isn't as potentially overwhelming, and is therefore less likely to bias their responses. 

The problem with making general inferences about human behavior from data drawn from complex systems like dating sites is that people understand that their behavior has social consequences.  As such, it does not represent some untapped font of truth about what people really do or say if the context isn't carefully considered and taken into account.

Big Data only reveals truths of human behaviors insofar as we understand the context in which the behaviors occur.