Doing User Surveys To Narrow Down Your Design Comps

So you're doing a Web site redeisgn, you have an Information Architecture in place, and a few design comps you're trying to choose from. Now what?

If you've gone through the usability testing for the IA, the last thing you want to do is try to test each of your designs. Why not do a simple user survey to narrow down the designs to one, and then do your usability testing on just that one?

Here's how I did it.

Narrow Down Your Designs to the Top 3
Choose no more than three designs to collect survey data on.

Generate the Questions
My questions were pretty simple. Two demographic-type questions, a series of questions about user preferences for some design factors, and a series of questions relating to each of the 3 designs. The questions for each design should be the same. I used Likert scale questions for a more statistically valuable result.

I struggled with the decision to include an open comment box. Did I really want to wade through the distracting jibes, and unrelated rants? In the end, I decided to include them. I'm telling myself now that I'm going to get some frustrating results, but that I'm going to get a jewel of a comment somewhere. I hope I'm right.

Generate a Test Environment
I'm test one design with some Flash, and two that are fairly static designs. Because of time constraints, I built out the Flash-inclusive design, and for the other two, I just used Jpegs and some simple CSS to make them appear like Web pages.

Generate the Survey
I used Survey Monkey to build the survey. It was really simple. The only negative is that I can't link to my test pages using Survey Monkey. Instead, I have to tell them to open the pages in a separate browser window. It's a disappointing hurdle, but my I'm fortunate that my user testees are college IT students, so they should be able to handle it.

I used the free version of Survey Monkey, which is feature-rich enough for my purposes.

Well, that's what I've done. If you'd like to see the final survey, you can link to it here:

Survey

Once I compile the results, and choose one design, I can focus my usability testing and improvement efforts on one design and save myself some time. Hopefully you can too.