Balsamiq

Install and Update Usability

One of the nice things about working in a small, flat organization is that you have to do support. With no levels of hierarchy and everyone monitoring support, communication is easy and fixes are fast. My experience today in trying to short circuit what appeared to be a small issue that is causing some pain to customers is a good example of how small can mean nimble and responsive.

The past few days of my first week I've been monitoring support and answering a few Get Satisfaction questions where I could. That's a part of everyone's job here. But it also helped with one of my primary goals this week, which was to get get a handle on the priorities and begin working on a few of these small projects. I've been getting up to speed on the apps, filing bugs, and looking at the UX priorities. I've also been thinking about how to attack the low hanging fruit. What I saw while doing some of this work in parallel, is that I might be able to do some triage and kill a few birds with one stone here and there.

The Upgrade Problem

I saw a few posts about issues people have with upgrading, knowing I had the same discomfort with the upgrade process myself. But I got back to work, and filed the issue in the back of my mind. I've also been doing a content inventory of the small Balsamiq.com site, and trying to get a handle on the IA there. I was looking at the really long and verbose pages we have, and in the process was going through scenarios like finding the upgrade link. I knew it would be hard for some people to figure out. It was too much work.

Then I came to the AIR upgrade badge and wondered how much of an issue that was. If you can think back to the first time you installed an AIR app, you might recall how seemingly quick the process was, when compared to downloading a package and executing it on your machine (running setup.exe on Windows or dragging a .app from a disk image on Mac). It's almost scary how seamless AIR installs are after that. Almost.

The issue for some with AIR installations, especially upgrades, is that they introduce different way of upgrading than what we're familiar with. You won't know what it feels like until you do it once. Hell, it can feel unfamiliar for a few times until it you've done it enough that it feels natural. I thought the only way to really know if this was pissing people off was to test it.

Doing Ninja-Usability Testing

There are various gradations of usability testing, but the ones I'm interested in are those that are cost effective while being easy enough for me to do quickly and remotely. Discount usability testing is about low cost methods for testing using only a handful of people that run through scenarios. The outcome is often that the most obvious problems are uncovered with the least investment in time and resources. Jakob Nielsen originated this idea. A radical version of this is the 5 second test, which I became aware of after reading an article describing the process at UIE.

For my needs, I wanted to just figure out this one problem. What do people experience when they try to upgrade Mockups for the Desktop? So I turned to remote usability testing tools. OpenHallway is one I've been testing out--they're a sponsor of Konigi and I'm writing a more in-depth of review of the service soon. What I wanted to do was to give people this simple scenario.

You've been using Balsamiq Mockups, and you want to upgrade to the latest version. Can you find the upgrade installer in the site?

Get to the point where you think you've found it and tell us where you think it is. Tell us what you would do at that point to upgrade.

So I put a call out on Twitter and stopped at 6 respondents. I wanted to test 1 problem with a few people. I don't know much about ninjas, but that seems pretty ninja-like to me.

The Feedback

This the feedback I got. My notes are quick and dirty. There are no task markers because there's only one task. There's no quantitative data to analyze. Just observation and common sense.

Tester 1
Wrote: You click on the "Install now" button

Notes: Started by scrolling through home page. Cliked Support > FAQ. Succeeded after searching the FAQ. Succeded in under 1 minute.

Tester 2
Wrote: After navigating through several pages, I found out that to upgrade the software, you simply download the latest copy off of the site.

Notes: Used the Support link after skimming the home page. Scrolled through the Support page and didn't find it and returned to Home/Products link. Clicked Desktop link in hero. Then appeared to read the page until scrolling down to Upgrade instructions. Found it, didn't sound confident at first. about 1'30" to complete task.

Tester 3
Wrote: Final thoughts:
1/ difficult to find download area
2/ difficult to see if by downloading the latest version it will be installed without deleting anything
3/ looks a bit like apple website without the blingbling effects

Notes: Skimmed the home page. Clicked the Products page, found home/products confusing. Went to desktop mockups product page. Then support. After filing with browsing, saying "not very useful stuff," went to search for "download" and tried the Examples page. Found the Desktop link and guessed that it was an install process.

Tester 4
Wrote: it wasn't there or at least i couldn't find it i was looking for upgrade or something similar

Notes: Failed or gave up task.

Tester 5
Notes: Scanned the home page for an upgrade link. Clicked Desktop product link and found the Upgrade heading below the fold. Then wondered "Would download the demo is the same as just downloading? I certainly don't want to buy it again." Successfully started the download, but wondered if she was doing the right thing.

Tester 6
NOTE: She took the test AFTER I had already made changes by creating a download page and adding the download link to the Header.

Wrote: The first thing I was looking for was a download link. It was on the top right of the first screen in the main navigation. I would have expected that if a new version is recently out that the home page would promote it more obviously though (so I looked in the main area before I looked at the navigation, which is where I expect to find it in a permanent fashion).

On the download page I had to interact with the Flash widget in order to get the download started. That was not expected, I was expecting to see some obvious "Download Version X for Y Platform" button - don't really understand the benefit of that little widget.

All in all it was straightforward to do it, but things just did not look like how I expected. The site could really use a visual refresh so that core content, navigation and important call to actions (like "Download the latest X") are visually differentiated.

Thanks!

Notes: She was successful in finding the Install badge quickly, but again the default AIR install badge is not familiar enough.

Taking the Low Hanging Fruit

Did you notice the funny note on the last one? She took the test after I had already made the changes after seeing what needed fixing in the first 5 tests. You could argue that those fixes were common sense anyway, but the audio from the users thinking out loud provided even more feedback about specific things with the AIR badge and what they expected. I didn't transcribe those notes here. Below is the plan I wrote up.

Immediate Plan
1. Create a download page with very little copy. Be sure to put in right keywords in title for search.
2. Add Download link in header.
3. Add a more prominent Download call to action in support page.
4. Think about interim ways to make call to action clear in home page without a major redesign.

Upcoming Plan
1. Vet the design of the updated download button (AIR badge) we already mocked up.
2. Add a link in Balsamiq or Help menus of desktop that goes to our new download page.

I've already done 1 and 2 from the first list. Download is now in the header and links to the new Download page.

Lessons Learned

It's been a fun first test and it was good to try using a service that's simple enough for my needs. I started by putting a session together at around 7am. I recruited via Twitter, got 6 respondents and waited while they ran their tests. By 10 I was watching their videos and wrote up notes and a plan for changes. I had lunch, and was making the changes by 1pm. That's how I want to do things around here.

---