As the word implies, a stakeholder is anyone that has a stake in the interface that is being tested. Stakeholders can include (but are not limited to) the following:

• Upper management (especially for new products)
• The product manager and owner of the interface
• The engineering and development team
• The UX team
• The design team
• The marketing team
• The sales team

In a small company or a company that is new to usability testing, the stakeholders can be virtually anyone within that company: the CEO, a UX contractor, product management, and even the quality assurance team. In larger companies, especially those with more experience in usability testing, the stakeholders are often representatives of the in-house UX and UI teams or the product owner, if the company does not have a dedicated UX team.

Whether or not each of these potential stakeholders needs to be...

The UX researcher running the usability study has to thoroughly understand the interface that will be tested. This is a prerequisite for being able to write an effective discussion guide or script. The following are some questions that we like to ask the study sponsor during a kick-off session:

• How would you describe the purpose of the interface?: Admittedly, it's rare that we don't have any idea what the product does before we start a usability study, but it is always useful (and interesting) to understand how the study sponsor views it. In extreme cases, this view may strongly deviate from the UX researcher's perception of the product. An unnecessary discussion can be avoided by asking the actual users in the study how they perceive the product.
• What are the most important functions of the product?:  This answers the question of...

At the start of a study, its goals must be clear. The goals drive most of the other elements of the framework.

Goals can be anything from all-encompassing (wanting a general feel of how usable a website is) to very specific (wanting to evaluate a new feature and how the users like it, or why the conversion rate went down after a website redesign). At this point in the study, it is more important to ensure that the goals are realistic (that is, that they can actually be achieved with a usability study) than very detailed.

For example, if the goal is to evaluate whether a new marketplace website can go live in its current state, then the goal is not feasible for a usability study: the study can only provide results about the usability of the website, and while we absolutely agree that usability should be a strong deciding factor, a go-live decision for a marketplace...

The funds available to finance the usability study can influence the following:

• The goal: A small budget might require that the goals are reduced to what can actually be achieved in the frame of that budget.
• The scope: The range of tasks to be completed by the participants determines the time required by them, as well as the effort required by the UX researcher, and subsequently, the necessary funding.
• The test environment: Development effort might be required to provide working prototypes, or to ensure that the test environment is populated with representative data; devices might have to be shipped to the participants. The cost for these study preparation tasks has to be covered by the available budget, too.
• The participants: Elaborate profile requirements for the participants may increase the cost of recruiting suitable candidates. The number of participants is also...

The scope is a difficult subject to tackle because stakeholders very often want to use a usability study to cover everything at once, and the UX researcher will have to scale the expectations down to a feasible scope. The scope is constrained by what can reasonably be asked of the participants within the timeframe of the study. A video-based remote usability study will have a different test duration than a survey-based one, for example. The budget also influences the timeframe, and subsequently, the scope. 

The goal will determine whether the study's scope should comprise the entire product, only certain functions of it, or even just individual screens. If the goal is to determine the usability status quo of a product, for example, then the scope will encompass the entire product, whereas comparing three design alternatives...

While the previous sections were targeted towards ensuring that the study meets the expectations of the study sponsor, this section is about ensuring that the study participants can execute the test as smoothly as possible.

Some of the questions that need to be answered are listed below.

A website might be targeted at both large-screen and small-screen devices; a product might only offer a companion app for iPhones, and so on. These requirements must be documented and taken into consideration when recruiting the participants.

When testing websites, an additional question to ask is which browsers and browser versions are relevant; when testing apps, it's...

Getting the right participants is key to obtaining relevant results. The following profile requirements should be covered during the kick-off session too.

Frequently, the study sponsor can provide the demographics of the participants to be recruited, based on who their typical users are. Having no hard requirements is an acceptable response, because while an educative app for toddlers will only be relevant to parents of young children (and their children, of course), an email application, for example, is very broadly used, and it would probably be difficult to narrow it down to specific demographics.

Some common demographic characteristics that might be relevant include the following...