Unity usability testing results

[Copied from Gobby document: other-design-n-usability-testing-results]

Charline took 15 users through a set of tasks

- 15 participants
- Mix of men and women
- Range of ages
- All technologically savvy
- Overall liked the look of Unity
- Thought Unity was between a phone and a computer
- Like the "immediacy" of getting into applications
- Encountered a number of problems

Problems encountered:
- Difficult to do several tasks at one time
  * e.g. Open Facebook, then go to OpenOffice.org Impress
  * System performed very slowly
  * Tried to use the launcher icon to open a new document
  * Decided to go to Files Place - took a long time to start
  * Opened another doc from the Files Place
  * Minimized the OO.o to see Facebook, but now she didn't know how to get back to her document
  * Users often assumed that Files Place was the only place to manage documents
  * Tried to delete the document from Files Place - tried DnD to trash
  * User had to give up on deleting the document

- Working between two documents
  * User asked to cut-and-paste between two docs
  * Couldn't figure out how to switch between the two docs
  * Expected to be able to work on docs side-by-side in Expose mode
  * Switching between two documents was not intuitive
  * Users find switching cumbersome

- Interact with the launcher
  * Asked the user to put the Calendar in the launcher
  * Users tried to DnD to launcher
  * When that failed they tried to select the icon, but single click opens the app
  * Once Calendar starts she cannot figure out how to pin it to the launcher because the Calendar icon is no longer visible
  * Launcher used the Evolution Mail icon instead
  * Even users that noticed the launcher icon didn't realise they needed to do anything to make it stay there
  * As soon as the user closes Calendar the icon goes away

- Reorder the icons in the launcher
  * Users tried to click-and-drag straight up
  * Result was the icons all move as one
  * Tried to remove the icon by dragging to trash icon

- Change the background
  * Users were able to find the Appearance applet
  * Assumed they had to click [ Add ]
  * File -> Open dialog was confusing and users didn't know if they were successful
  * Background is covered up by applications, so you cannot see the background

- Bringing up the Dash
  * Was difficlt to find how to start the Dash
  * When clicking on Music the user expected to see the music player
  * Confused when the Media filter was opened (and took a long time to appear)

- Find an application to install
  * Users are not used to having a local application for finding new applications
  * Started by going to the Applications Place - expected some link for finding software
  * Eventually found Software Center by finding it in the Dash
  * Took over 60 seconds to start Software Center
  * While waiting for each Place to start the users thought the grey icons were "inactive"
  * Once the user found a game to install they could figure out how to install it
  * Received sufficient feedback to indicate the application was installing
  * Once the app was installed they didn't know how to find the application to start it
  * Tried [ More Information ] and tried to play the game from the screenshot
  * Some users thought they had to go to the website to play the game

General feedback
- Users didn't figure out the keyboard shortcuts
- Approx 1/3 of participants did learn they could right-click on the launcher icons
- Lack of feedback when waiting for something to start was frustrating

Going forward
- Will be testing Unity throughout the Natty cycle
- Hope to be able to correct problems identified throughout the cycle, but this will depend on how complex the changes are
- DX QA will be testing for performance and publishing results
- Results of usability testing will be in the Canonical Design blog - http://design.canonical.com/
- Canonical Design group will post as much of the video of the testing as possible

Work items:
[oreneeshy] Develop and publish public UI guidelines document explaining the overall Unity design: TODO
[johnlea] Evaluate Unity usability issues we already know about and integrate into the 11.04 roadmap: TODO
[unity] Develop Unity user testing protocol for measuring progress in quality of Unity UX: TODO
[charlinepoirier] Publish videos of previous Unity usability tests: TODO
[charlinepoirier] Work with Unity team to publish results of each usability test: TODO