Heuristic Evaluation Report

3.1 Evaluation Methodology

Heuristic evaluation is a way to evaluate a user interface by looking at it and judging the interface according to certain rules. This opinion based method is not meant to replace more thorough evaluations like empirical studies but rather a cost effective way to weed out obvious problems with an interface and improve usability in most situations, developed by Jakob Nielsen and Rolf Molich in 1990 [Nie1990].

For the heuristic inspection an evaluation plan is needed. For the evaluation plan it is firstly important to choose the heuristics. Depending on the heuristics chosen different aspects of the design will be evaluated, leading to different potential problems being spotted.

It is also important to choose the right inspectors. Inspectors should have a high degree of knowledge in the field of human computer interaction. They should also know about the domain of the system. Because it is difficult to find one person with all these skills, multiple inspectors from different backgrounds are recommended.

The inspectors fill out the reports immediately, this makes recording the sessions not very useful. If the inspector talks about their thoughts during the inspection it can make sense to record them. Depending on the prior knowledge of the inspector it may be necessary to give them some information on how to run an evaluation. [Wil2009].

For this evaluation, the "Andrews General Usability Heuristics", shown in Appendix A.1, will be used. These are based on and slightly adapted from Nielsen's revised set of ten usability heuristics [Nie1994a].

The process of an evaluation is as follows: The inspectors visit the website for the first time and get a time frame of one hour to inspect different aspects of the website as accurately as possible, recording them in the process. Afterwards they either choose clips from the video or record seperate clips to show their findings and describe them in the log files. Afterwards the inspectors meet up and share their results. Now the evaluators combine their findings and rank them for importance and severity. The result is a report which can then be discussed with the client.

Finally, after compiling the identified user interface defects into an aggregate, recommendations are deduced. This interpretation of the findings can be very skewed depending on the skill and domain knowledge of the inspectors, thus it is advised to involve real users and/or domain experts. [Sto2005]

3.2 User Profiles

The website tries to attract everyone who wants to engage in sports. From the casual athlete to the determined fitness-freak. Since the website has a really broad collection of sports equipment it really offers something for everyone. We came up with the following groups:

• Dedicated/high performance group: People who follow a certain sport on a professional level, like boxers, weightlifters, etc.
• Casual workout group: People who set themselves a goal, like loosing weight, building up strength, etc. These people probably visit gyms regularly and want some of the sports equipment in their own home.
• Entertainment group: People who search for entertainment more than having a set goal in mind. This group is more interested in trampolines, table tennis, slacklines, etc.
• Wellness, Yoga, Massage group: People who enjoy a more calm sports like yoga or gymnastics or just want to relax a bit.
• Sportswear, additional features group: People who don't look for sport devices but rather sportswear, or small fitness helpers like pulse monitors or cycling watches. We also added users who are interested in sport nutrition to this group.
• Committed givers: People who want to buy a present for children, like small scooters, gokarts or stall bars.

3.3 Extent of the Evaluation

We evaluated the website by trying to use it like any other user would. In this process we encountered some problems, which we will describe in the following sections. In some cases we tried to explicitly find issues by doing things that a normal user might not do. Generally we tried checking all pages and pressing all the buttons to find inconsistencies or things that go against the usability heuristics. We also tried to find issues by doing different things at the same time. Whenever we found an issue we recorded it in a log file. We tried to cover as much of the website as possible in the time of the evaluation. Any button that changed the url like the carrer-portal were not covered.

3.4 Evaluators and Evaluation Environments

The evaluation environments used by each evaluator are shown in Table 1. For this evaluation, mobile devices were operated in portrait mode.

David Bom used ffmpeg to record and cut the video and audacity to do the same with audio. Ffmpeg has been used to upsample the audio to stereo, but everything else was kept as is.

Raphael used the native screen recording software from Samsung to record his findings. After that he put the clip in Adobe Premiere Pro and started cutting and made the voice overs. After that they were encoded and rendered using the Adobe Media Encoder.

David Knill used the native IOS Screen Recorder to record his findings and the native "Voice Memos" App to record his voice. After that he put the clips in Davinci Resolve 18 to combine Audio and Video and to cut the clips. After that he rendered the clips using the recommended settings. Since the native resolution was lower than the maximum the clips were exported in native resolution.

Alexander used OBS Studio to record his screen. This software supports different formats and resolutions, therefore no further transcoding was needed. After rewatching the video he then recorded short clips representing the findings if they were reproduceable.

4.1 Top Three Positive Findings

The top three positive findings according to their average (mean) positivity ratings are described in more detail below. The positivity rating scheme used to rank the positive findings is shown in Table 2.

4.2 List of All Positive Findings

Table 3 shows a list of all the positive findings which emerged from the evaluation, sorted in descreasing order of average (mean) positivity, i.e. the most positive are at the top of the table. The name codes assigned to each evaluator are shown in Table 4.

4.3 Top Five Problems

The top five problems according to their average (mean) severity ratings are described in more detail below. Problem number 1 is the problem (negative finding) with the highest mean severity. The severity rating scheme used to rank the problems is shown in Table 5.

4.4 List of All Problems Found

Table 6 shows a list of all the problems found (negative findings) in the evaluation, sorted in descreasing order of average (mean) severity. The name codes assigned to each evaluator are shown in Table 7.

A.1 Heuristics

The evaluators used the Andrews General Usability Heuristics 2013 found in file: heuristics.pdf.

A.2 Individual Evaluation Logs

The evaluators will use the following (plain text) log files to collect notes during their individual evaluations:

• David Bom: log-db.txt.
• Raphael Hutten: log-rh.txt.
• David Knill: log-dk.txt.
• Alexander Niederreiter: log-an.txt.

A.3 Spreadsheet

The filled-out, sorted spreadsheet of all problems and positives including their ratings is helist.xlsx.