Preference Test: Example Survey Questions

About Your Survey The main goal of of this assignments to get users to indicate which of two two variants they prefer by exposing them to those options and enabling them to rate them. You should come up with a system of questions that determines “which variant is better?”. We’ve included in the examples below a SUS survey and a SUPR-Q survey, in addition to some other example questions. As indicated in the notes, SUS and SUPR-Q questions typically are asked at the end of a test, enabling users to think about and rate their entire experience. While we have included examples of SUS and SUPR-Q surveys, your design does not have to conform exactly to either of these templates or scoring formulas. In your own survey, it’s fair to ask any or all of the questions from the SUPR-Q survey. It’s fair to ask any or all of the questions from the SUS survey.

You will also see that there are suggested questions that you can use to focus on rating a particular aspect of the experience; perhaps you just want people to rate labeling, button color, or which graphic they prefer. These questions typically appear just after you have exposed people to an interaction, graphic, or screen. You can also include open-ended questions that enable users to be more descriptive.

The key point of this assignment is that it is critical to elicit from the users which variant they prefer using rating scale questions, at the very least.

Use the following guiding principles for developing your scoring formula: * It should be easy for people to interpret the meaning of each scale point. * The meaning of scale points should be interpreted in the exact same way by all participants * The scale should include enough points to differentiate participants from each other (at least 5) * Responses should be reliable, meaning, individual participants should answer the same way every time they are exposed to a condition and asked that question. (While at least 4 demographic questions are required in your survey, they should not be presented as a scale.)

About the Example Surveys We are including here examples of SUS and SUPR-Q surveys. Each of these survey types has different purposes, and each contains a scoring formula appropriate to their design

• For more info on SUS design, see this resource
• For more info on SUPR-Q design, see this resource

System Usability Scale (SUS) Survey

How a SUS Score is calculated: * SUS questions each use a 5-point likert scale. * For each of the odd numbered questions, subtract 1 from the score. * For each of the even numbered questions, subtract their value from 5. * Take all of these values and add them up. * Multiply this total by 2.5. * You will now have a score that will be out of 100. It is NOT a percentage.

What SUS results mean: The Average SUS Score is 68. This means there is room for improvement. 80.3 or higher is good 51 or below is a cause for concern.

SUPR-Q Survey

Credibility:

If you are measuring an e-commerce experience, it is appropriate to substitute the following two questions for the Credibility questions to ensure you are understanding participant’s perceptions of the purchase/business experience.

I feel comfortable purchasing from this website. I feel confident conducting business with this website. Net Promoter Score (NPS): This is the Net Promoter Score (or NPS) question; it is used to measure customer’s willingness to recommend a company’s products or services to other people.

How a SUPR-Q Score is calculated:

Average the responses for the first 7 questions. Then, add 1/2 of the score for Likelihood to Recommend (NPS)

Example SUPR-Q Survey Score

In the above example, the score (9.14) was reached using the following formula:

Average the scores for questions 1,2,3,4,6,7, and 8. Then add half the score for question 5 (the NPS). Additional Questions

You may also use one of the following additional example questions:

• I feel that the controls are easy to use.
• I feel that the images are appealing.
• I feel that the menus are easy to use.
• I think that the text easy to understand.
• The labels are understandable.
• The number of steps in the process is appropriate.
• The form fields are easy to complete.

NOTE: You can include any of the above questions in your survey, in addition to any questions that you generate that are better tailored for your specific context.

When you are ready to submit your results, you will do so on the next item.