Heuristic Evaluation Report

706.021 Human-Computer Interaction 3VU SS 2006

Group 1-03

Marco Breithuber
Benjamin Hopfer
Eghosa Omoniyi Ikulogun
Ulrich Krispel

Heuristic Evaluation of the Web Site

http://www.billa.at/

Report of 02nd May 2006

1 Executive Summary

This report describes the heuristic evaluation of the website www.billa.at. A heuristic evaluation is a cost efficient usability inspection method performed by a few - in this case four - usability professionals. These experts identify the user groups of the site, search for usability problems and rate their severity. Based on this ratings a ranking of the problems is generated and possible improvements of the most severe problems are suggested. Positive aspects of the site are mentioned too.

The first positive impression of www.billa.at is the amount of content - which is huge and encourages the visitors to stay on the site and come back on a regular basis. Whenever that much information is provided, a solid structure for this information is needed. This is solved quite well on the homepage, by following web standards with a main menu at the top and a more detailed menu on the side. The colors chosen for the site are comfortable to interact with. Also a good idea is to visually separate the different sections with different color themes, which unfortunately is applied not consequent. The last main positive aspect of the site is the thought-out application of metaphors.

The most severe problems we found were:

• Error Message for signing up: A unexpected error message is shown when signing up.
• Shop quantity input problem: Sending forms with the ENTER button makes problem, especially in the shop.
• Inconsistant Colors and Links: Colors and links are not consistent throughout the site.
• Shopping Cart can't be emptied: The procedure for emptying a shopping cart is tedious.
• Amount of "Feinkost": Specifying the amount of delicatessen is unlogical and misleading.

2 Introduction

A heuristic evaluation is a cost efficient method to test the usability of an interface. It is based on set of the most important usability principles which are called heuristics. A small group of usability experts (evaluators) check the interface against the list of heuristics. The usability heuristics used in our evaluation are described in Appendix A. Each evaluator checks the interface individually and writes down the problems he found in a logfile. The evaluators also take a screenshot for each problem found. After each evaluator finished this check, the evaluation manager generates a large list with all problems found into one spreadsheet and sends it to each evaluator. For each problem a severity rating is assigned independently by each evaluator. These ratings are averaged and the problems are sorted by their severity. The report presented to the client includes both this list as well as suggested solutions for the major problems.

2.1 User Profiles

Basically everyone cares where he is purchasing his consumer products. So the target group of the site www.billa.at is the whole population. Nevertheless only specific subset of the whole population has the knowledge or needs to use the site. We will concentrate on these specific groups and list them in order of importance:

• Bargain Hunters: This group is always searching for special offers to safe some money. We can further specify the age of the group between 20 and 40. Younger people probably won't buy products from BILLA themselves and only a few older people are familiar with using the Internet.
• "Vorteils-Club"-Members: Partially a subset of the first group. To access further services of billa.at the users of this group are willing to invest the additional effort of registering. This users will come back frequently.
• Online Shoppers: The lack of time or ability to purchase items in a "real" shop forces this user group to buy their needed products online.
• The Curious: This group is seeking for further information maybe attracted by some kind of advertisement.
• Job Seekers: Searching for an interesting job, they browse the homepages of various potential employers.
• Investors: This group is looking for a lucrative company to invest in.

2.2 Extent of the Evaluation

The whole site www.billa.at was inspected. The evaluators focussed on big problems in higher sections, but also reported minor problems that showed up during the evaluation.

2.3 Structure of the Report

The following section three will describe the evaluation environments used by the evaluators. Afterwards in section four positive impressions of www.billa.at are discussed followed by an detailed analysis of the main problems in section five. In section six all problems that were found are presented in a table. Appendix A shows the heuristics used by the evaluators and Appendix B contains the individual log files of each evaluator.

3 Evaluation Environment

A description of the hardware and software used by each evaluator.

4 Positive Impressions

4.1 Informative Content

The site offers a huge amount of interesting content related to BILLA. Information that is not expected on the site like information on wholesome groceries, horoscopes, the SMS service, the huge and searchable database of recipes surprises interested visitors and keeps them on the site. Additionally the site also offers all the information that is expected, like the current special offers, the "Filialfinder" to search for nearby shops and the "almost expected" online shop. Figure 1 shows all menue items overlapped. This can't be reproduced on the site, it is just to show all the content at once.

Figure 1: Offered content of the site (all menu items overlapped with a graphics editor).

4.2 Fairly well structured

The rich content provided, makes structuring of the information a must. The site achieves that very well in most places. The navigation for accessing the information needed follows web "quasi standards" with the main menue on the top and finer structure on the side as shown in Figure 2.

Figure 2: The navigation structure: Main top menue (marked green) and finer side menue (marked blue).

4.3 Visual Appearence

The colors, fonts and design are - with few exceptions - comfortable and to read and interact with. Also a good idea is to separate the different main sections by giving them different color themes. Unfortunately, this approach is not applied consequently leading to many sections sharing the same colors. Figure 3 shows the colors and fonts of the section "Kulinarik". Compare it with Figure 1 and Figure 2 to see the power that lies in thought-out color separations.

Figure 3: Comfortable design, thought out separation through colors.

4.4 Usage of metaphors

Evaluators also stated that metaphors are used often and wisely throughout the page. For example clickable images, the austria map in the "Filialfinder" and most of the items inside the shop make their meaning immediately clear. Figure 4 shows the Austrian map inside the "Filialfinder" as example for good useage of a metaphor (The different sections of the map are clickable).

Figure 4: Metaphor "Austria map" inside the "Filialfinder".

5 Analysis of the Main Problems

The following section discusses the five main problems the evaluators found.

5.1 Error Message for signing up

When clicking "Noch nicht registriert" at the homepage, the system expects that a valid email address had already been entered in the field "eMail". This is not the behaviour a normal internet user would expect, as this is not the case on other sites. Furthermore the error message given by the system is: "Bitte geben Sie eine gültige Email - Addresse ein". This error message is neither precise nor multi-level and it is a bit offensive too.

Suggested Solution: The system for registering supports multiple steps. In step one, the user has to enter an email address. If the email address was entered before the click on "Noch nicht registriert", the system may still behave like now, taking the user directly to step two. If no email address was entered, the user should be taken to step one instead of getting an error message.

Figure 5: The error message for this problem.

5.2 Shop quantity input problem

Entering a quantity in the online shop and pressing enter leads to a blank page in opera. Pressing back doesn't even lead to the form the number had been entered in, but to the page before.

Suggested Solution: The root of the problem is the less severe rated problem, that sending a form by pressing the button ENTER is not allowed. In opera this restrictive behaviour even causes this severe problem. Our suggested solution would be to allow sending forms by pressing the ENTER key.

Figure 6: A screenshot of the blank page.

5.3 Colors and links not consistent

It is good to layout different sections of the page with different colours. The page tries to achieve that, but is not consistent at all (see Figure 1). It seems like the author has chosen 3 different colours for the first 3 items (e.g. "Angebote", "Kulinarik" and Vorteilsclub") and copied the other items' layout from the first one (all red too).
The links are also not colour and font face consistent over the whole page - sometimes even matching normal headings -, which makes the recognition of a clickable text hard (Figure 7).

Suggested Solution: We would suggest to consequently apply comfortable different colors to each section to seperate them. It would also be important to make headings and links different and consistent over the whole page either by giving them always the same color, or a color related to the section color. Links should always be underlined too.

Figure 7:
A screenshot showing the inconsistency of links.

5.4 Shopping Cart can't be emptied

The online shop misses an option to empty the shopping cart at once. The only way to achieve this is to manually enter "0" into each textbox for the amount and to press "Aktualisierung" afterwards. After having purchased 20 products the user has to enter "0" 25 times and press "Aktualisierung" to empty the shopping cart.

Suggested Solution: Provide a "empty shopping cart" button.

Figure 8:
A screenshot showing that there is no option to empty the shopping cart at once.

5.5 Amount of "Feinkost"

If someone wants to buy cheese ore sausage in the online shop, the amount has to be specified in kilogramms. Most users would expect that the amount means "dag" as in every shop in Austria. Furthermore the description is misleading and confusing: "1000 gramm Kilogramm".

Suggested Solution: The amount should be specified in "dag" instead of kilogramms and the simple and not confusing label "dag" should be right of the "amount" box.

Figure 9:
A screenshot showing the misleading amount information.

6 List of Problems Found

Aggregated list of all problems found, in descending order of average severity.

Appendix A: Heuristics Used by the Evaluators

The evaluators used the set of heuristics found here heuristics.pdf

In the hardcopy, include a printout of the list of heuristics.

Appendix B: Individual Evaluation Logs

The raw, individual evaluation logs of each evaluator. In the hardcopy include printouts of each of the evaluator logs.

XHTML 1.1   CSS2