Heuristic Evaluation Plan

1 Introduction

The website kastner-oehler.at, which we will be evaluating is the webstore of the in Graz-based store Kastner-Öhler, which has several shops in Austria. This store offers clothes and accessories for women, men and children. It also provides a Beauty and Home & Living area, as well as toys for children. On the website you will find more expensive brands, than less expensive brands. The whole purpose of the evaluation is to identify as many potential usability problems as possible. We will be evaluating the whole website and deliver input for the next design phase.

2 Evaluation Methodology

Heuristic evaluation is used for analysing user interfaces and finding their usability problems. Those findings can then be used for an iterative design process. Due to the fact that one person alone cannot identify all of the problems a small set of evaluators is used, which independently examine and judge the interface based on usability principles (heuristics). The recommended number of evaluators is between three to five people, because there is no greater benefit when using more evaluators. It is also recommended to use people with different fields of expertise to improve the quality of the evaluations. Such an evaluation usually lasts one to two hours but can be longer if the interface is very complex. During the evaluation the evaluators use a list of heuristics and compare them to the different interface elements. After everyone is done with their evaluation, the results are aggregated and discussed. This can be done by either the evaluators themselves, or an observer. With an observer the evaluation results are available rather soon and in case there are some problems with operating the interface, the observer can also assist the evaluators. In the end the output consists of usability problems, which are determined and explained based on the heuristics by the evaluators. This complete process is based on Nielsen's explanation on how to conduct a heuristic evaluation. [Nie1994]

Heuristic evaluation is method used to measure the usability of a user interface. The evaluators are using already established heuristics, like in our case, the “Andrews General Usability Heuristics”, will be used. Which is a slightly adapted version of Nielsen’s revised set of then usability heuristics. This document contains 10 heuristics, or principles which are basically a practical way to solve problems or make decisions. Professional evaluators will make use of the given heuristics to systematically evaluate the, in our case, usability of an interface, like a website. As an example, the first heuristic in the mentioned document is the feedback. Which describes how and how fast, the website/user interface should react. In this case, there are requirements like for example, the reaction time of the website. For a respond that will take not longer than 10 seconds, display a busy cursor, for more than 10 seconds show the user a progress bar. With the help of the given heuristics, the evaluators will create a list of their found problems and describe them, e.g. as pictures or screen recordings, as well as a textual description and steps to reproduce the found problem. [Inte2023]

3 User Profiles

Since Kastner-oehler.at is mainly an online store for fashion and clothing, it’s target group are people who are interested in buying new clothes and have a certain degree of fashion understanding. The product catalog of this online store includes clothing for children, men, women and additionally extra sections for certain groups such as the “Infected”-section, which mainly targets teenagers. Furthermore, the online store has sections for other products such as beauty products or things in the field of home and living to offer. Therefore, the site is trying to attract a very broad field of potential customers, which in general are very hard to group. One way of grouping these users, is after their own fashion understanding respectively their fashion sense and their reasoning for the site visit. People which are into fashion and beauty are more interested in finding new trendy clothes and products and are more selective in their choosing and therefore need more options of selection and filtering to get to the things which they have in mind. In contrast users, which are not really into fashion, want to get multiple recommendations of standard things so that they can search and compare which of the recommendations suits them best. But for all of them the main task is the searching and buying of products.

4 Extent of the Evaluation

In the context of this practical we will evaluate the complete website Kastner-oehler.at.

5 Evaluators and Evaluation Environments

The evaluators will use the evaluation environments shown in Table 1.

Viola and Daniel will copy their video clips captured from her iPad and his phone to a Windows PC and use LosslessCut 3.53 on Windows 11. Daniel and Mario will use Kdenlive 22.08.1 to cut and edit their videos. Alfred will use Sony Vegas 17 to cut and edit his videos.

Since Viola's iPad has a native resolution larger than FullHD the video clips will be transcoded to 1080x1920 using Handbrake 1.6.1 on a Windows PC. Daniel will also use HandBrake 1.6.1 to transcode his videos, because the resolution of his phone is too high. Alfred and Mario don't have to transcode anything, as they will already record the videos in the right resolution.

References

[Nie1994]

Jakob Nielsen; How to Conduct a Heuristic Evaluation; 01 Nov 1994. https://nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/

[Inte2023]

Interaction Design Foundation; Heuristic Evaluation; retrieved on 27 Mar 2023 https://www.interaction-design.org/literature/topics/heuristic-evaluation

A Materials

The following materials will be used by the evaluation team.