A method for quantifying user experience

05.10.2009 | Author: Eric Reiss
Back in January, 2009, I published my definition of user experience. UX, as user experience is popularly called, is a difficult subject to discuss with business clients. To them, “UX” is just more expensive hot air from the folks who brought us the dot bomb.

The basic problem is that discussing an experience – any experience – is highly subjective. And although others have attempted to set up metrics (notably Robert Rubinoff’s User Experience Audit, and Livia Labate's User Experience Health Check), we don’t always end up with particularly useful data. Here at FatDUX, we were looking for a simple tool that could help us turn observations and subjective conclusions into useful dialog with our clients.

Our UX quantification model will undoubtedly be criticized by the scientific hardliners. But it does help us uncover many problems and communicate these to the client. And it works better than beating them over the head with statistics.

Please note, we take a very broad view of “user experience,” incorporating both online and offline interactions of three types:
 
- active
- passive
- secondary

Please refer to the original user-experience blogpost for details regarding these types of encounter.

Avoiding complicated algorithms
There are lots of complicated ways to work numbers, particularly when dealing with the subjective data that invariably lies at the heart of any discussion of user experience. But rather than putting together confusing formulae to present our research, we work directly with our clients to quantify empirical observations in a very simple model.

The model in brief
We start by consolidating our research findings in a single first-person narrative – an X-log (experience log). This is somewhat related to phenomenology. Once we’ve assembled this story, we work together with the client to:

1. mark each individual interaction – we call these “snapshots”
2. assign a value from 1 to 3 to each snapshot in relation to its contribution to the overall experience
3. grade the experience on a scale from -3 to +3
4. multiply the value by the grade to get a score (this is the really useful number)
5. note any events that are recurring, unique, or may be influenced by chronology (cause and effect relationships).

Plugging in the numbers
We mark each interaction, but some may later be thrown out if they are sufficiently trivial or so unique in character that they are deemed irrelevant in the broader, generic sense of the project. Although no individual snapshot can be assigned a value of 0, if you really think it deserves a value of 0, this is probably an interaction you'll want to ignore.

When we grade the individual snapshots, we use the following scale:

+3 = fantastic
+2 = good
+1 = better than expected
0 = no effect on the ultimate user experience
-1 = poor
-2 = awful
-3 = mission critical

Unique or chronological events won’t always influence the score, but in the case of repeating events, the interaction clearly needs to be looked at carefully.

A sample narrative
Here’s a simple story based on a trip to the movies. It represents an amalgam of several user interviews, onsite research, review of user-satisfaction surveys, etc.

My family (my wife, myself, and our two kids) decided to go to the movies. We checked the internet and found the website for our local cinema complex after a quick search on Google. But we had to click three times to get to the program page and wait through a silly animated ad for a movie that hadn’t even been released yet. Worse still, we were forced to download a pdf to find out the specific movie names and playing times. And after all that, we couldn’t even order tickets online, much less purchase them, so we couldn’t avoid waiting in line when we arrived. You’d think a big four-screen complex would have a more sophisticated website. But we did find out what was showing, decided to see the latest Harry Potter movie, and piled into the car.

Finding a parking place was easy. The theater has a big lot, which is important since driving to this particular theater is really our only option. Just as we were leaving the car, it really started to rain, but happily, the entrance wasn’t far away.

There were three ticket windows open, so the lines were short. The girl behind the counter was noisily chewing gum and barely looked up during the entire transaction. In fact, she didn’t say a single word to me except to ask for the money. Wow, prices have really increased this past year. I was surprised at how expensive it was.

The lobby was inviting and quite clean. We bought popcorn and soda at the concession and found our way to our particular auditorium. It was easy to spot the signs pointing the way. As we approached, we noticed overflowing trashcans with popcorn and other garbage from previous audiences.

The seats were well-marked and easy to find. The seating was comfortable but there was old popcorn underfoot. The temperature in the room was pleasant, although all of the wet people made it get a little steamy. The sound was great and really enhanced the special effects, so we really enjoyed the movie. When we left, there was a nice usher, who opened the exits and wished us a pleasant evening as we went out. And it had stopped raining. A nice end to a nice family outing.

Defining the interactions
Reading through the narrative, we mark the individual interactive events – the snapshots. This gives us the following list:

1. Find website on internet
2. Click three times to find relevant page on site
3. Reaction to irrelevant animation
4. Find schedule (download PDF)
5. Reaction to lack of purchasing options
5a. Opinion of website
6. Park car
7. Reaction to parking lot
8. Reaction to rain
9. Reaction to proximity of parking to entrance
10. Reaction to short line
11. Reaction to rude ticketseller
12. Buy tickets
13. Reaction to ticket prices
14. Reaction to lobby
15. Buy popcorn and soda
16. Find auditorium
17. React to overfilled trashcans
18. Find seats
19. Reaction to seats
20. Reaction to popcorn on floor
21. Reaction to temperature
22. Reaction to steaminess
23. Reaction to sound
24. Reaction to movie
25. Reaction to nice usher
26. Reaction to dry weather
26a. Opinion of evening

Note that opinions are not really interactions, hence we have 5a and 26a.

Assigning values and grades
Ask your clients to help you fill out the values and grades. This is a great way to get clients emotionally involved in the design project without having to show them pretty layouts.

x_log 

Conclusions
Having made this chart, there are several things that become painfully apparent. First, the lack of purchasing options is really a major problem. The need to watch irrelevant animations and resort to PDFs for information was also pretty bad. Snapshots 11, 15, and 25 suggest that additional emphasis should be placed on customer-service training for front-line personnel. Snapshots 17 and 20 illustrate that cleaning is a problem. Snapshot 22 revealed that the climate-control system was out of whack, which proved to be an easy repair.

The most important point of the exercise, though, was that the client suddenly understood how all of these events ultimately contributed to the total perception of the movie-going experience. The X-log narrative started a productive dialog about user experience and not about the color of the links.

We hope you’ll find it useful.

59 Comments »

  1. Kensington Square is also near to Nex Shopping Centre as well as the buzzling Toa Payoh area. Entertainment for your loved ones and friends is therefore at your fingertips with the full condo facilities as well as the amenities in Bartley

    Comment by custom jerseys - 00:00 01.01.0001
  2. Wow!!! Your post was really interesting. It's full of knowledge and information.
    <a href="http://thecoralsatkeppelbay.org">Corals At Keppel Bay</a>
    <a href="http://www.youtube.com/watch?v=Ht0qPpyfGX8">D'pristine</a>

    Comment by d'pristine - 00:00 01.01.0001
  3. Gaudi’s work is truly a must see for anyone traveling to this area. It’s very unique and unlike anything I’ve seen before. And yes, of course enjoy the beaches when you travel to Barcelona.

    Comment by the free camping - 00:00 01.01.0001

  4. This is a really nice blog and it gives a great way to know the things which is necessary to know. Wish your more amazing posts.

    Comment by replica watches - 00:00 01.01.0001
  5. The Skywoods is here......

    Comment by The Skywoods - 00:00 01.01.0001
  6. Beware early repayment penalties on fixed rate and other discounted deals but most mortgages allow up to 10pc of the outstanding balance to be repaid each year without penalty.

    Comment by financial adviser tips - 00:00 01.01.0001
  7. I want to voice my appreciation for your kindness supporting those people that require help on this particular concept. Your personal commitment to getting the solution all over became amazingly productive and has frequently empowered many people much like me to attain their desired goals. Your personal helpful publication implies a great deal to me and much more to my peers. Best wishes; from all of us.

    Comment by feestjurken - 00:00 01.01.0001
  8. I was working and suddenly I visits your site frequently and recommended it to me to read also. The writing style is superior and the content is relevant. Thanks for the insight you provide the readers!

    Comment by http://www.stocksmarketreport.com - 00:00 01.01.0001
  9. ECOTECH @ SUNVIEW is a proposed multiple-user light industrial (Business-2) development and it is set to be the envy of many with the beautiful architecture standing at 9-storey high with lush greenery landscaping and water features on the roof-top apart from other exciting facilities.

    Comment by Ecotech Sunview - 00:00 01.01.0001
  10. e them both, but I'm more of an Android fan.

    Comment by sohbet odalari - 00:00 01.01.0001
  11. Mon Jervois @ District 10

    Comment by Mon Jervois - 00:00 01.01.0001
  12. This is my new i check out here. I discovered so many exciting things in your website especially its conversation. From the plenty of feedback on your content

    Comment by www.connect2health.co.uk - 00:00 01.01.0001
  13. Gucci Outlet, http://www.guccishoesfactories.net/
    Polo Outlet Online, http://www.polo-outletstore.com/
    Ralph Lauren UK, http://www.ralphlaurenuker.com/
    Polo Lauren Ralph, http://www.ralphwlaurenpolo.com/
    Pas Cher Longchamp Sacs, http://www.saclongchamppairs.com/
    Canada Goose Outlet, http://www.canadagooser.com/
    Monster Headphones Outlet Online, http://www.discountbeatsbydresale.com/
    Burberry Outlet, http://www.burbagssale2013.com/
    North Face Jackets Outlet, http://www.north-faceoutletonlines.net/
    Michael Kors Outlet, http://www.michaelkors.so/
    Marc Jacobs Outlet Online, http://www.marcjacobsonsale.com/
    Michael Kors Outlet Online, http://www.michaelkorstates.com/
    Cheap Hollister UK Online, http://www.cheaphollistersale.co.uk/
    North Clearance, http://www.northsclearance.com/
    Coach Purses Outlet Online, http://www.coach-purseonsale.com/
    Coach Factory Online,http://www.coach-factoryy.com/
    Coach Outlet Online, http://www.coach-outletusa.com/

    Comment by polo - 00:00 01.01.0001
  14. Another option is to write your own blog and use your travel stories to drive traffic to it. You’d be surprised at the number of bored, lifeless people who want to live vicariously through someone traveling the world.

    Comment by christmasre sort packages - 00:00 01.01.0001
  15. A Christian couple who marry without a state marriage license are contracting a valid and legal marriage that even the state recognizes if certain conditions are met.

    Comment by common law rights - 00:00 01.01.0001
  16. UK based Beauty Blog. A place where i can obsess about the best things in life, make up and fashion!!

    Comment by cheap online shopping guide - 00:00 01.01.0001
  17. Interesting approach to break up the un-grasp-able value of a holistic experience into smaller parts that can be quantified.
    Could you explain the concept of "events that [..] have a chronological (linear) element."? The number of snapshots that are marked with this quality is too low to help me understand...

    Comment by Peter Boersma - 20:24 05.10.2009
  18. Sure, Peter. The fact that it rained was directly responsible for the high humidity in the theater. Hence, snapshot 22 is chronologically related to snapshot 8. The reactions to the ticket prices (snapshot 13) could be an indirect reaction to the nasty girl at the ticket office, who put folks in a bad mood. The popcorn on the floor (snapshot 20) could be related to the overfilled rubbish bins (snapshot 17). Hence the question-marks, which suggest there may be something worth investigating further.

    The question-mark for snapshot 11 (rude ticket seller) could be a unique occurance - is this an isolated incident, a single bitchy employee, or a chronic staff problem? We don't know - but we now know that something is amiss.

    Comment by Eric Reiss - 20:44 05.10.2009
  19. Very interesting, I would love to learn more about this -- examples, reports.

    One question, I may be missing:
    On the chart, what do the letter columns mean and the Y and ?
    GVSRUC

    Are they different "Users"? The narative seemed to have 4 users but there are 6 letters. Why do they have different ratings? It is each of their take on things? Sorry If I just totally missed something here.

    I am really interested in this, so please clarify! Thanks!

    Comment by amber - 21:08 09.10.2009
  20. Hi Amber,

    GVSRUC refer to the points made in the text:
    Grade
    Value
    Score
    Recurring
    Unique
    Chronological

    Y stand for "yes"

    Score = Grade x Value

    OK?

    Comment by Eric Reiss - 16:26 12.10.2009
  21. Eric, your analysis blows me away! I've been dreaming of this moment - the breakdown and scoring of various interactions - literally for years! Some people told me that this couldn't be analyzed in any sort of quantifiable way, and I disagreed, and now here it is!!!!

    Comment by Sue Fensore - 01:18 23.10.2009
  22. Thanks, Sue. Spread the word. I think this article takes the prize as the LEAST read article on this blog (closely followed by the length of an internet year).

    Comment by Eric Reiss - 17:08 22.11.2009
  23. Hi Eric,
    Amazing post,nice case study and analysis,I Really enjoyed reading it and also appreciate the information you have analyzed and offered in this post.
    Thanks for sharing:)
    cheers and happiness

    Comment by Study8Home - 10:11 09.02.2010
  24. Hi eric, I think that you have a very interesting research. I will also appriciate youfor giving the better result as I hoped it should be.

    Comment by Penny Bid Auctions - 12:25 09.02.2010
  25. well nice post, and thanks for sharing your ideas.

    thanks

    Comment by mark - 14:02 09.02.2010
  26. Wow! You are really providing interesting method.I enjoying reading your post and it is a really good read for me.

    Comment by Penny Auction Script - 13:11 10.02.2010
  27. Great tips, as always!

    Comment by Missouri satellite internet - 22:42 30.08.2012
  28. Pour participer maintiennent numéro, vous aurez peut avoir comptes driver (code RIO ) numero rio. Vous obtiendrez est certain d'obtenir gratuitement pour gratuit par appelant tonus de la voix du serveur ou du service à la clientèle satisfaction client partir de votre fournisseur de services numero rio bouygues. Vous ne CAN get un SMS avec votre numero rio orange rio b and you rio sosh. Avec votre propre code RIO, alors vous êtes capable d' sur le offre de de en orange orange .

    Comment by numéro rio - 17:36 07.01.2013
  29. industry-best replica oakley sunglasses protection from the fake oakley sunglasses sun or flying objects replica oakleys.Get a clear view with fake oakleys High Definition Optics?oakley sunglasses outlet in Oakley Men’s Sunglasses — wherever oakley outlet the world takes you.

    Comment by fake oakley - 13:51 22.03.2013
  30. Several buses are available near along Ang Mo Kio Ave 5 and Yio Chu Kang Road along with shopping centers and restaurants. Belgravia Villas is also near to Nex Shopping Centre as well as the buzzling Toa Payoh area. Entertainment for your loved ones and friends is therefore at your fingertips with the full condo facilities as well as the amenities around Ang Mo Kio.

    Comment by Belgravia Villas - 03:47 30.03.2013
  31. Ecopolitan EC will be accessible with Punggol MRT Station as well as Punggol Bus Interchange. It is also right beside Tampines Expressway(TPE). Ecopolitan EC is also near to Marina Country Club, Sengkang Riverside Park and Sheng Siong hypermart in Punggol Central.

    Comment by Ecopolitan - 10:22 25.04.2013
  32. The mystery to winning Baccarat play is to get a run on either side of the shoe. Thus, most players stay informed regarding the winning involved a scorecard gave by the house. A run of four or five wins on one side or the alternate is not exceptional.

    Comment by www.theonlinecasinos4u.com - 12:20 03.05.2013
  33. The condo’s facilities provide full family entertainment needs for your family and loved ones. Indulge in a serene and tranquil lifestyle right in the heart of Jurong East.

    Comment by J Gateway - 04:27 16.05.2013
  34. Coral Edge Residences will be accessible with Punggol MRT Station as well as Punggol Bus Interchange. It is also right beside Tampines Expressway(TPE). Coral Edge Residences is also near to Marina Country Club, Sengkang Riverside Park and Sheng Siong hypermart in Punggol Central.
    Thx

    Comment by Coral Edge Residences - 14:06 02.06.2013
  35. DUO Residences is also near to Bugis Junction as well as the buzzling city area. Entertainment for your loved ones and friends is therefore at your fingertips with the full condo facilities as well as the amenities in Bugis.

    Comment by DUO Residences - 09:26 06.06.2013
  36. Several buses are available near Lush Acres EC along with shopping centers and restaurants. Lush Acres EC is also near Waterway Point, the shopping, dining and entertainment hub which is scheduled to open in 2 years time. Also, it is right beside Punggol Waterfront. Entertainment for your loved ones and friends are therefore at your fingertips with the full condo facilities as well as the amenities near Lush Acres EC.

    Comment by Lush Acres - 06:35 18.06.2013
  37. Your blog provided us with valuable information to work with. Each & every tips of your post are awesome.

    Comment by thomas sabo - 06:58 25.06.2013
  38. Kensington Square is also near to Nex Shopping Centre as well as the buzzling Toa Payoh area. Entertainment for your loved ones and friends is therefore at your fingertips with the full condo facilities as well as the amenities in Bartley

    Comment by http://kensingtonsq.org - 03:29 04.07.2013
  39. very nice article.. thanks for contributing

    Comment by http://www.theskywoodscondo.sg - 12:17 10.07.2013
  40. The Tembusu

    Comment by http://www.thetembusucondo.org - 12:18 10.07.2013
  41. Skypark Residences will be accessible with Sembawang MRT station & Sembawang Bus Interchange. It is also near to Vista Point Shopping Mall, Causeway Point Shopping Mall, Cold Storage, Shop N Save, and many more.

    Comment by http://skyparkresidence.org - 16:53 10.07.2013
  42. Thanks, Sue. Spread the word. I think this article takes the prize as the LEAST read article on this

    Comment by ateş ve su buz devri - 13:26 24.07.2013
  43. Several buses are available near Lush Acres EC along with shopping centers and restaurants. Lush Acres EC is also near Waterway Point, the shopping, dining and entertainment hub which is

    Comment by kafa topu - 13:28 24.07.2013
  44. The Skywoods, a brand new luxurious residential development along Dairy Farm road, in the Upper Bukit Timah neighbourhood, will be adorned with lush greenery within and set with a picturesque backdrop of forested area with the likes of Dairy Farm Nature Park and Bukit Timah Nature Reserve catering to the affluent lifestyle of the locality.

    Comment by The Skywoods - 16:27 26.07.2013
  45. Residents of The Skywoods are blessed with having Dairy Farm Nature Park, Zhenghua Park and Bukit Timah Nature Reserve for their family getaway from the hustle and bustle of city life with just a short stroll away.

    Comment by The Skywoods - 16:30 26.07.2013
  46. Heard of Jurong Lake District? Yes? No? If you have not heard of the Jurong Lake District Masterplan yet, you probably do not know that there will be an exponential growth with many ongoing & future exciting developments in our western region.

    Comment by Jurong New Launch - 16:32 26.07.2013
  47. J GATEWAY, the 1st brand new condo development right in the heart of the biggest commercial hub outside city centre. A unique lakeside destination for business & leisure with a stunning new life in a spectacular new city.

    Comment by J Gateway - 16:34 26.07.2013
  48. Mon Jervois is a very rare District 10 new launch development nestled in a prestigious and highly sought after residential neighbourhood at Jervois road, an exclusive prime location within close proximity of Good Class Bungalows and various embassy such as High Commission of Brunei Darussalam, High Commission of Malaysia and Embassy of Republic of Indonesia.

    Comment by Mon Jervois - 16:36 26.07.2013
  49. great eco site. it is always good to check it out

    Comment by ecopolitan ec - 11:51 10.09.2013
  50. great home by the sea. do not miss

    Comment by vue 8 residence - 09:41 11.09.2013
  51. great flora site at ec price. worth bookmarking

    Comment by inflora - 06:56 09.10.2013
  52. best blog. worth bookmarking.

    Comment by sea horizon - 13:14 21.10.2013
  53. A method for quantifying user experience is the great article.I didn;t get the exactly meaning of this,please share more info.

    Comment by convocco - 22:01 27.10.2013
  54. I liked to know more about the a user method.thanks for share this great info.

    Comment by Ignite My Butt - 22:07 27.10.2013
  55. As always your advice is practical and helpful, thank you for sharing the Eric and helps me to live a better life.

    Comment by Best Christmas gifts for kids - 22:16 27.10.2013
  56. Love this step when you want to find a link and get a nice one.

    Comment by adult video - 19:36 10.11.2013
  57. Classes on your dog can be overwhelming, but that’s just genuine if you’re unsure just how to do so perfectly. A lot love anything else, puppy education calls for that bring best procedures then educate your self about the subject point. The ways to train their dog are wide ranging

    Comment by adolfocamarillobirthday - 08:06 26.02.2014
  58. this is extremely mathematical.

    Comment by puteri cove residences - 10:06 29.03.2014
  59. quantitative methods are the best methods to figure this out.

    Comment by sky habitat floor plan - 10:08 29.03.2014

Leave a comment