Gerwald Tschinkel
Luis Dean Kals
Rene Langgartner
This report documents a heuristic evaluation of http://www.mediamarkt.at, the website of a local market for electric appliances. The goals of this heuristic evaluation were to evaluate the usability of the website using a set of heuristics and establish a prioritized list of issues for the developers' attention. The findings along with the suggestions will guide future changes in the software's interface.
Reseachers utilized "Andrews General Usability Heuristics". Each team member independently performed tasks multiple times, recorded usability issues and which heuristic violations occurred and then rated each issue in terms of severity. Researchers later consolidated and prioritized usability issues according to severity.
This evaluation identified 26 usability issues in total. We described 5 usability issues in detail and proposed solutions for each of them in the report. The following are the most severe issues outlined in the report:
Sorting Problems:
It is impossible to sort items based on the price, which is a
standard feature in most online stores.
Webcode:
No real description on how to reach or recieve a webcode, yet it is listed
on every link featuring a product (and some other subsites).Couldn't find
a webcode for various products, couldn't find one at all. It says it
navigates to a specific MediaMarkt website however, we've never noticed a
webcode and based on the facts its primarily under products, we guess its
meant to help with product search. However no product lists one in any
findable way.
Inconsistent naming "Wunschliste" and "Einkaufsliste":
After adding a product to your wishlist, it gets directly placed into the
shopping list.
Registering confirmation email:
Registering a user account doesn't send the confirmation email in
reasonable time and already logged in, but without confirming the
registration email, the costumer cannot do anything.
Search does not find specific existing page:
There is a Service called "Küchengeräte Einbaucheck". If you type
this exact string into the search, you will find a lot of "Küchengeräte",
but not the specific page.
More detailed information is given in the Heuristic evaluation.
Reseachers utilized "Andrews General Usability Heuristics", which is based on Nielsen's ten usability heuristics to evaluate the website. The ten heuristics can be seen in Table 1.
Individual work:
Each team member independently performed tasks multiple times, recorded
usability issues and which heuristic violations occurred, they rated
each issue in terms of severity.
Work in group:
Later, the researchers discussed their independent findings. Afterwards
they consolidated and prioritized usability issues according to severity.
No | Usability Heuristics |
---|---|
1 | The system should give appropriate feedback within an appropriate time. (Feedback) |
2 | The system should speak the user's language. (Speak the User's Language) |
3 | The user should be free to explore the system without penalty. (Reversible Actions) |
4 | The same word, phrase, action, or situation should always mean the same thing. (Consistency) |
5 | Prevention is better than cure. (Error Prevention) |
6 | Place knowledge in the world. (Recognition Rather Than Recall) |
7 | Provide accelerators for frequent users, unseen by novice users. (Flexibility and Efficiency of Use) |
8 | "Less is more". (Aesthetic and Minimalist Design) |
9 | A good error message helps users recognize, diagnose, and recover from errors. (Good Error Messages) |
10 | It may be necessary to provide help and documentation. (Help and Documentation) |
Product Surfers:
Online Customers:
Potential Store Customers:
The reseachers analysed the parts of the User Interface which are accessible for a common visitor of the website and which she/he uses during her/his visit. The researchers did not check for failures in the sourcecode or anything of the like. The main exercise was to simulate all types of visitors (see 2.1) and how these types use the website, trying to detect as many failures/problems as possible. The goals of this heuristic evaluation were to evaluate the usability of the website using a set of heuristic and disclose a prioritized list of issues to developer's attention. The findings along with the suggestions will guide future changes in the software's interface.
The hardware and software configurations are listed in Table 2.
Evaluator | Luis Kals | Gerwald Tschinkel | Rene Langgartner |
---|---|---|---|
Age | 22 | 29 | 21 |
Sex | male | male | male |
Web Browser | Mozilla/5.0 en | IE 8.0 en | Opera 10.10 de |
Operating System | Ubuntu 9.10 | Windows 7 Ultimate | Ubuntu 9.10 |
Internet Connection | 8 mbps, AON, cable | 8 mbps, UPC, dsl | 10 mbps, cello, cable |
Monitor Size | 15" | 13,3" | 15,4" |
Monitor Colours | 32-bits | 32-bits | 32-bits |
Monitor Resolution | 1280 x 800 | 1280 x 800 | 1366 x 768 |
Browser Resolution | 1280 x 800 | 1280 x 800 | 1366 x 768 |
Date of Evaluation | 2010-04-11 | 2010-04-11 | 2010-04-13 |
Time of Evaluation | 18:00-21:30 | 18:00-22:00 | 10:00-11:00 |
The appearance of the products is very professional with good highlighting as can be seen from the figure 1.
The researchers also gave positive comments for the good overview on the searching process. Also the visible state of actuality is very positive. These impressions were highlighted with red circles in figure 1.
All the needed information is listed (Store Opening Date; Address and Owner; etc.). A Telephone, Fax and Email address are listed. A feedback form is also included for questions. It also includes a functional google-earth map that shows the location. 2
When looking for a specific range of products, it lists the products in the same category that others were looking for. The font dictating how often the product was searched/how popular. In Figure 3 "Das Interessiert Andere Kunden" is highlighted with a red circle.
The researchers also mentioned the fast loading speed.
The 5 most severe problems from the helist are described in the following chapter.
It is impossible to sort items based on the price, which is a standard feature in most online stores. 4
There is no real description on how to reach or recieve a webcode, yet it is listed on every link featuring a product (and some other subsites). The researchers couldn't find a webcode for various products, couldn't find one at all. It says it navigates to a specific MediaMarkt website however, we've never noticed a webcode and based on the facts its primarily under products, we guess its meant to help with product search. However no product lists one in any findable way. 5
After adding a product to your wishlist, it gets directly placed into the shopping list. 6
The shopping list is highlighted with a red circle.
Registering a user account doesn't send the confirmation email in reasonable time and already logged in, but without confirming the registration email, the costumer cannot do anything. 7
There is a Service called "Küchengeräte Einbaucheck". If you type exactly this string in the search, you will find a lot of "Küchengeräte", but not the specific page. 8
No. | Short Title | Description | Screenshot | Heuristic | How Reproducible | Found | Severity | |||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
LK | GT | RL | LK | GT | RL | Av | ||||||
1 | Search does not find specific page | There is a Service called "Küchengeräte Einbaucheck". If you type exactly this string in the search, you will find a lot of "küchengeräte", but not the specific page. | tschinkelgerwald-prob12.png | Minimalist Design | search | y | y | - | 4 | 3 | 4 | 3.7 |
2 | Sorting Items | Impossible to sort items based on Price [A standard feature in most online stores]. | KalsLuis-prob03.png | Flexibility and Efficiency of Use | Product Search | y | y | y | 4 | 3 | 4 | 3.7 |
3 | Webcode | No real description on how to reach or recieve a webcode, yet it is listed on every link featuring a product (and some other subsites).Couldn't find a webcode for various products, couldn't find one at all. It says it navigates to a specific MediaMarkt website however, I've never noticed a webcode and based on the facts its primarily under products, I guess its meant to help with product search. However no product lists one in any findable way. | KalsLuis-prob04.png | Aesthetic and Minimalist Design | Product Search / Product Descriptions, etc | y | y | - | 4 | 3 | 4 | 3.7 |
4 | Registering a user account doesn't send the confirmation email in reasonable time. | And I am logged in, but i cannot do anything, as i haven't confirmed the registration email. | tschinkelgerwald-prob05.png | Good Error Messages | klick on Registrieren, and fill out the data. | y | y | - | 4 | 3 | 4 | 3.7 |
5 | Account Creation - Still not officially Logged in | Although officially logged in on the website it still denies me basics and says I am not logged in. However wishlists work. | KalsLuis-prob02.png | Good Error Messages | General Item Comments | y | - | - | 3 | 3 | 3 | 3.0 |
6 | Inconsistent naming "Wunschliste" and "Einkaufsliste" | After you added a product to your wishlist, it gets directly placed into your shopping list. | tschinkelgerwald-prob06.png | Consistency and Standards | go on an product details page, and click on "auf meine Wunschliste" | - | y | y | 2 | 3 | 4 | 3.0 |
7 | Search does not have an „Alle Kategorien“ option | If you are not sure to which category a product belongs have to try several til you find the right one | langgartnerre-prob01.png | Flexibility and Efficiency of Use | search | - | - | y | 3 | 3 | 2 | 2.7 |
8 | Click on Register leads to an empty page | Seems to be a clicking-bug. | tschinkelgerwald-prob04.png | Bug | go on any product details, then at the end, click on "Jetzt für "Mein MediaMarkt" registrieren." | - | y | - | 3 | 2 | 3 | 2.7 |
9 | Wunschlisten - Choosing lists | When adding an object to a wishlist, I take for granted I can choose when pressing on 'add' my wishlist when clicking. However I have to choose that wishlist as active beforehand however the wishlist is right at the bottom of the screen and thus silly to keep scrolling down - activating and then choosing the product. | KalsLuis-prob06.png | Consistency | Product Info / when Logged in | y | - | - | 3 | 2 | 2 | 2.3 |
10 | Intransparent categories of products | „Software“ vs. „Software – Spiele und Anwendungen“ where is the difference? | langgartnerre-prob01.png | Minimalist Design | search | - | - | y | 2 | 3 | 2 | 2.3 |
11 | Inconsistent display of product preview | There are two different ways of displaying a product short overview. Compare tschinkelgerwald-prob02.png and tschinkelgerwald-prob03.png | tschinkelgerwald-prob03.png | Consistency and Standards | click on "Honeywell" in the Tag Cloud, than click on the product, than click on "more from honeywell" | - | y | - | 3 | 2 | 2 | 2.3 |
12 | Wuenschlisten – Swapping | When swapping between two wunschlisten, it took the second one and labeled it aktiv. It was badly listed. However I had to press the same button again to aktivate the Einkaufsliste, However I received no feedback on the change (It still labeled Musik as being active), It was also the same button i used to active the other list. If I didn't refresh the page, it didn't show the change. Aktivieren looks like an einkaufsliste using the same red arrow and is a design flaw. | KalsLuis-prob07.png | Reversible Actions/Feedback/Consistency | Wuenschliste | y | - | - | 4 | 2 | 1 | 2.3 |
13 | Wuenschlist - More than two lists | When swapping between more 'wuenschlisten', one vanishes and it states einkaufsliste is aktiv (and not the one activated). It seems to always be the one with the lowest alphabetical name. Refreshing shows the correct active list but the final vanishes again on changing / still same problem as above when changing. | KalsLuis-prob08.png | Reversible Actions/Feedback/Consistency | Wuenschliste | y | - | - | 4 | 1 | 2 | 2.3 |
14 | Wunschliste | Difficult to add a product to "Wunschliste", because of bad placing and design | langgartnerre-prob02.png | Minimalist Design | Product Search / Product Descriptions, etc | y | - | y | 3 | 2 | 1 | 2.0 |
15 | Validity of prices | At each product you see the text "Angebot gültig am 10.04.2010, MediaMarkt Seiersberg" But the problem is, that now (test-date) we have the 11.04.2010 (=Sunday) - so the prices are only valid in the past. My expectation would have been, that the prices are at least valid until the next day the shop has opened | tschinkelgerwald-prob09.png | ? | product details | - | y | - | 3 | 1 | 2 | 2.0 |
16 | MediaMarkt top 10 | Although logged in, it says I can register for Mein Mediamarkt. However I just registered! | KalsLuis-prob05.png | Bug | Top 10 | y | - | - | 2 | 2 | 2 | 2.0 |
17 | Confusing arrangement of search results | The result is not arranged very well. very small, with a lot of additional text around it, that is not as neccessary. | tschinkelgerwald-prob02.png | Minimalist Design | click on "Honeywell" in the Tag Cloud | - | y | - | 2 | 2 | 2 | 2.0 |
18 | Top offers are just announced as screenshots | There are no links to detailed product informations | langgartnerre-prob05.png | Flexibility and Efficiency of Use | Top 10 | y | - | y | 2 | 3 | 1 | 2.0 |
19 | Screen gets stuck | When using the search without entering a message and without selecting a category the screen gets „stuck“. | langgartnerre-prob06.png, langgartnerre-prob07.png | Bug | Press "Fotoservice"; Use searchengine without a term and category; If you press "Service" (or any other category) the site does not change | - | - | y | 2 | 2 | 2 | 2.0 |
20 | Deleting items from the product comparison list | When you want to delete something from the product comparison list, there is a checkbox an a link at each product. if you click on the link, nothing happens. you first need to click on the checkbox and than click on the link. | tschinkelgerwald-prob08.png | Consistency and Standards | product comparison | - | y | - | 2 | 2 | 1 | 1.7 |
21 | Bad positioning of job exchange | People, who want to find the job category may have some problems | langgartnerre-prob04.png | Flexibility and Efficiency of Use | Bottom of mainpage | - | - | y | 1 | 2 | 2 | 1.7 |
22 | Too much information in Contact Form | When using the webform to send a request; too many various options are listed for various places. If checking from the Seiersberg website this is unneeded because you only need a general requiry or a requiry to the store. | KalsLuis-prob01.png | Aesthetic and Minimalist Design | registration/newuser.html | y | - | - | 1 | 3 | 1 | 1.7 |
23 | Wrong layout of text | The text "aus dieser liste entfernen" has a wrong layout (bug) | tschinkelgerwald-prob11.png | Bug | "Wunschliste" | - | y | - | 1 | 1 | 2 | 1.3 |
24 | Contact page - opening times not very readable | The Google Maps Popup shows all Contact information, but because of a technical problem, some text is not in the popup and is therefore not very readable. | tschinkelgerwald-prob01.png | Minimalist Design | Contact page (Start -> click on "Kontakt") | - | y | - | 1 | 1 | 1 | 1.0 |
25 | Obsolete information on contact page | On the contact page, the christmas opening times are still available (in April) | tschinkelgerwald-prob01.png | Minimalist Design | Contact page (Start -> click on "Kontakt") | - | y | - | 1 | 1 | 1 | 1.0 |
26 | Confusing icon, on the list "in which category am i" | On the right side of the products details page, you can find a list, that shows you in which product category you are. the sign next to the current category is an red X – more known as when it comes to deleting something. | tschinkelgerwald-prob07.png | Consistency and Standards | go an a products details page | - | y | - | 1 | 1 | 1 | 1.0 |
Code | Meaning |
---|---|
LK | Luis Kals |
GT | Gerwald Tschinkel |
RL | Rene Langgartner |
y | Found by this evaluator |
Severity | Meaning |
---|---|
4 | Catastrophic problem |
3 | Serious problem |
2 | Minor problem |
1 | Cosmetic problem |
0 | Not a problem |
Av | Average severity |
The evaluators used the set of heuristics found here heuristiken.pdf
. In the hardcopy, you will find a printout of the heuristics.
The raw, individual evaluation logs of each evaluator. The hardcopy includes printouts of each of the evaluator logs.
Name | Log File |
---|---|
Luis Kals | log-kals.txt |
Gerwald Tschinkel | log-gtschinkel.txt |
Rene Langgartner | log-langgartner.txt |
The filled-in spreadsheet
(heliste.xls
)
of all problems including severity ratings.
The hardcopy includes a printout of the spreadsheet.