The Art of Measuring Project Success

Ask your stakeholders how they define a successful design project and you’ll often hear:

• “It’s easy to use.”
• “The users like it.”
• “It’s fast."

When the project wraps, these requirements are quickly discounted because they aren’t measureable. The application releases without consideration of the “ease of use”, leaving no way to know if the project is really a success. Vague usability requirements are tossed in a project vision blindly because many project managers and requirements analysts aren’t exposed to user experience methodology. We can’t blame them when, as usability professionals, we haven’t taken the time to show them how to measure the experience.

Take action
Ask to help document the usability requirements for a new project. Work 1-on-1 with the person in charge of the vision and business requirements so you can will have a strong influence, and establish an understanding of your work.

Encourage open communication. Ask the project manager what success looks like and when you hear “it’s easy to use”, follow up with additional questions.

• “How would you define easy?”
• “How long should the primary task take?”
• “How much reduction in help desk calls would you like to see?”

Remind project stakeholders that “easy” is relative. When users are asked if they feel something is easy to use, they are likely to say yes, even when it’s clear they struggled during usability testing. Turn the attention from “easy to use” to measures like the number of times a user has to backtrack, the number of misclicks, how many times a user expresses frustration in testing, and the number of times help documentation is accessed.

Encourage accountability
Refer to measurable usability requirements from planning through release. Use the requirements as a basis for usability testing and iteration, tracking improvements along the way. When the project wraps, be part of the debrief session and present the results of this effort to the project team. Let them see how success can be measured quantitatively.
Imagine a project manager bragging:

• We cut call center costs by 35% by adding inline help to the application.
• Our employees are 20% more efficient when submitting a purchase requisition. This has freed up more than 3,500 work hours this year.
• We found that 27% of our customers abandoned the checkout process last year. After redesign, only 7% have failed to complete the purchase process.

Ensure usability requirements are measureable. It’s better for you, your customer, and your organization. Be proactive and provide leadership as a representative of your product’s customer. Don’t let vague requirements stand in the way of defining a project’s success.