Heuristic Evaluation Plan

706.021 Mensch-Maschine-Kommunikation 3VU SS 2006

Group 5-06

Joachim Lechner
Marcel Medwed
Markus Tatzgern
Michael Hofmann

Evaluation of the Web Site

http://www.one.at/

HE Plan of 06.04.2006

1 Evaluation Methodology

The Heuristic Evaluation is a technique to analyse the usability of interface designs (software). For this purpose a number of evaluators in the range of unexperienced to experienced investigates the design (typically 3-5 test users per user group result in the best cost-benefit ratio (see How to Conduct a Heuristic Evaluation )). Generally it is the case that some experienced testers will find the same number of issues as a greater amount of unexperienced ones.

For the evaluation of the interfaces so called usability heuristics are used (see Ten Usability Heuristics and this document on heuristics). Each test user tests the webpage against these generic recognised usability principals. The main goal is to find as many of the existing usability problems as possible, but also positive aspects should be noted. Additionally screenshots and notes may be used to describe the results. To achieve a high fault detection coverage it is necessary to combine the individual evaluation results. This originates from the fact that the single test results only cover a subset of the total issues. Furthermore it is required that the evaluations are independent from each other. Influences may lead to a lower overall percentage of detected faults, after the results are combined. The individual results are aggregated together by the evaluation manager (remove duplicates, merge same results, categorise, etc.). Afterwards each evaluator independently assigns a severity rating for each problem. Finally the average value of the weights is calculated to find the issues with the highest rating.

This method doesn't provide information on how to correct the found usability issues. It is a very cheap and relatively fast method for finding design problems, which not only finds major problems but also minor ones. Another advantage of the heuristic evaluation is that it can be applied very early in the development process (e.g. pen-and-paper). This results in an early detection of design problems.

2 User Profile

Mainly the side tries to attract the following three types of users. This is already obvious through the main page of the webside.

• Private customers
• Business customers (firms)
• Prepaid card customers (TAKE ONE)

Additionally these users can be grouped into the following two categories:

• Persons, who are already ONE customers. They want to use the additional features provided by ONE, which can be accessed via the webpage (view invoices, download games/news/pictures/..., online SMS, change settings for the connection (change customer data, cost control, etc.), etc.).
• Persons, who want to become customers of ONE. These persons want to collect information about the contract possibilities. Also roaming-customers are covered by this category.

Furthermore we found the following special user groups, which also get some attention:

• Acoustically impaired people (SMS FOR FREE Gehörlos)
• Disabled people (All in ONE Fair)
• Students (ONE home4students)
• Persons, who want to use the Internet, but don't have a possibility to connect at the current location (I-ONE)
• Press
• Persons, who want to get a job at ONE
• Business competitors

Overall the webpage is designed for all ages. But of course a personal computer with an Internet connection and basic websurfing knowledge is required. Furthermore money is required for using the available offerings. Because of this the webpage attracts persons older than about 14 years (our esteem).

3 Evaluators

The webpage will be evaluated by four students of Telematics at the University of Technology Graz Austria. Management and administrative tasks will be carried out by Marcel Medwed. Details of the environments we will use for the evaluation are presented in the next section.

• Michael Hofmann
• Joachim Lechner
• Marcel Medwed
• Markus Tatzgern

4 Evaluation Environment(s)

Tabular overview of the different evaluation environment(s) which will be used. Demographic data, hardware, browser, type and speed of internet connection, etc. which each evaluator will be using.

Appendix A: Heuristics

A description of the heuristics which will be used can be found under heuristics.pdf

In the hardcopy a printout of this list of heuristics is included.

Appendix B: Evaluation Log Files

The following files contain the individual results of evaluators.
log-leechjoe.txt
log-koermy.txt
log-tatze.txt
log-mhofmann.txt

In the hardcopy a printout of all log files is included.

Appendix C: Skeleton Spreadsheet

Raw skeleton spreadsheet for evaluation manager helist.xls

XHTML 1.1   CSS2