Heuristic Evaluation Report


506.021 1KU Human-Computer Interaction WS 2001/2002

Group 2

Arto Paasovaara
Susanna Kivelä
Oliver Strasser
Michael Fussenegger

Evaluation of the web site

http://www.webalbum.at

Report of 27th November 2001




1. Executive Summary

Four person group did a heuristic evaluation to webbased photalbum system www.webalbum.at during 21.11.2001 - 26.11.2001. This chapter provides insights to the results of the evaluation.

Many usability problems were discovered but also there were some good features which provide good base for a new, more userfriendly interface. The biggest problems could be solved with a new decent interface design, that takes into account the five biggest problems found in evaluation.

The biggest problems found were:
- There is no warning when deleting whole album. It can be accidentally clicked resulting loss of big workload.
- Only jpeg image format is supported. Error prevention on image upload page concerning this is not good enough.
- It is possible for user to choose item which cannot be delivered (wrong resolution of picture)
- Film archive and Bild album and their subpages offer confusing interface, certain options are found only from certain subpage, without logical reason.
- If error-messagebox is given on loading page, system do not continue action and user can wait many minutes before realising (page still suggests it is loading something)

For more detailed information about biggest problems see chapter 4. Analysis of the Main Problems.
Also there are quite many consistency problems which are not big problems itself, but combined they lower quality of the system.

The following features of the interface were considered to have a good idea behind them: First page is not crowded with irrelevant information, Registration form is short and fast to fill, Basic system for viewing images is quite good, using loading page is a plus (when working) and possibility to give access to album for a friend is nice to have. But as well as many other features, they also have some usability problems (for detailed information see chapter 5. List of problems found).


2. Evaluation Environment

Evaluator Arto Paasovaara Susanna Kivelä Oliver Strasser Michael Fussenegger
Age 24 23 23 24
Sex male female male male
Web Browser Netscape 4.6 Netscape 4.6 IE 5.0 IE 5.0 / Konquerer
OS Windows NT 4.0 Windows NT 4.0 Windows 2000 Windows ME / Linux
Connection University University Cable T1
Monitor Colours 16 bits 16 bits 32 bits 32 bits
Monitor Resolution 1024x768 1024x768 1024x768 1024x768 / 1152x864
Monitor Size 17" 17" 19" 17" / 19"
Date of Evaluation 21-22.11.2001 21-22.11.2001 26.11.2001 23.11.2001
Time of Evaluation 13:15-16.45,
10.30-11.30
13:15-16.45,
10.00-11.00
18.00-19.30 15:00-17:00


3. Positive Impressions


4. Analysis of the Main Problems

4.1

There is no warning when deleting whole album. It is possible to think that delete icon can be used to select and delete some pictures, even when there is description on the same page (figure 1a). Also it is possible to accidentally click it and lose many hours work. There definitely should be some kind of warning ("You you really want to delete whole album, Yes/No").

Fotoalben page
Figure 1a:Fotoalben page

4.2

It is not possible to upload other image types than jpg, which restricts user a lot. This is told in the Help-page (Figure 2a), but not on the image upload page (Figure 2b). Also it is possible to upload unsupported images, they just don't appear to album

Hilfe page
Figure 2a:Hilfe page
Bilder uploaden
Figure 2b:Bilder uploaden

4.3

You can order things which cannot be delivered (because of the wrong resolution, see figure 3a). When adding to shopping cart, list should only hold items which are possible for chosen picture, not all items (figure 3b). Also there is not always error message about unsupported format (can depend on the browser) and the item still is added to shopping cart (figure 3b).

Resolution note on Hilfe page
Figure 3a:Resolution note on Hilfe page
Warenkorb anzeigen
Figure 3b:Warenkorb anzeigen

4.4

Several actions are divided to album (figure 4a) and film archive (figure 4b). Layout suggests that they are almost the same option, so why confuse user by dividing them to 2 different options? Upload archive could be just one of the albums. Also both of them are divided to several subpages some of which have very important options which others do not have. This dividing has no good reason, as all of them should be found in same place so finding certain option would not be so hard.

Layout and options on Fotoalben page
Figure 4a:Layout and options on Fotoalben page
Layout and options on Filmarchiv page
Figure 4b:Layout and options on Filmarchiv page

4.5

If error message has been given in separate message box (on loading page), page still suggests that it is continuing normally current action. This happens on all errors given on loading page.

Error message in messagebox
Figure 5a:Error message in messagebox
Nothing has happened after 2 minutes
Figure 5b:Nothing has happened after 2 minutes


5. List of Problems Found

In descending order of severity.

No. Description Found Severity Location
A S O M A S O M Av
1 System does not ask for user confirmation when deleting album. Also it is possible to think that action could be used for deleting single images yy  44444album
2 It is not possible to upload other image types than jpg y   43343.5image upload
3 You can order things which cannot be delivered (because of the wrong resolution). When ordering item, on the list there should be only those items, which are possible for the picture size.  yyy44333.5add to shopping cart
4 Several actions are divided to album and film archive. Also both of them are divided to several subpages some of which have very important options which others do not have. This dividing has no good reason, as all of them should be found in same place. y   44333.5 album index
5 If error message has been given in separate message box, page still suggests that it is continuing normally current action yy y344 33.5loading
6 In image upload, if you upload something else than jpg, there is no error message, picture just won't be there.    y43333.25image upload
7 On a certain Browser settings (unknown what) browser crashed every time when trying to access image upload page  y  43333.25image upload
8 Link to account information changes can be found only right after login, that page cannot be accessed later y y 33433.25 after login
9 You get the list of possible dealers only after you have filled an ordering form, it should be accessible before. y  y33343.25ordering form
10 After registering user should be automatically logged in yy  42333help page
11 If there was error in registration, you cannot go back to registration page from error page (unless using browser's back button) and if you go back by using links of the left frame, you will lose all given data y   24243registration
12 When loading page, browser informs that giving rights (which system is asking) is highly riskable, can scare users away yyy 33333 image upload
13 Registration can be only found from login page, it should be clearly on the start page yy  3324 3 left frame
14 When registration is successful, user is not clearly shown what to do next. Also message for successful registration should be more clear ("Der Zugang wurde für Sie eingerichtet") yy  32332.75registration
15 It is not clear (at least from the name) what is difference of Fotoalben and Filmarchiv y   32422.75left frame
16 You cannot add images to album from image album, only from archive    y33232.75album
17 When uploading, there is progressbar, but it only blinks the last "box", and this for many seconds and finally seemed to crash the browser although Windows task list claimed it to be running normally  y  43122.5image upload
18 No possibility to log out yy  44022.5left frame
19 Empty textfields for password suggests that user needs to fill them every time they change other information, otherwise password would be set empty y   32322.5 change registration
20 In "album anzeigen" I had three times the same picture, but only one of them would be shown (others showed only X for missing picture)  y  32322.5view album
21 In "fotoalben" anleitung section is confusing especially when you don't have an album yet, you would think the icons are clickable. (and they distract from the more important option --> Neues Album anlegen) yy y22332.5album
22 Password must be at least 5 digits, this is told only in error message, not before  y  21332.25register
23 Sometimes it is possible to "do something" with upcase text, sometimes not yy  22232.25many pages
24 Using 2 comboboxes and 1 link for deleting, copying and buying is not easy interface. y   33122.25album index
25 Edit page sometimes informs that you cannot choose username which is already in use (form in insert state instead of edit state)  y  33212.25change registration
26 If album has no picture, page does not clearly show what page is used for and still, user is asked to save before leaving page. Page can be very confusing. yy  32222.25 edit album
27 "Bestellung absenden" should not be different option on main shopping cart page. User should always check first what he/she is ordering and how much it costs, before going to real order page y   32222.25shopping cart
28 In "film archiv" there is possibility to delete upload archive, but then again it can't be deleted (you get error message) yy  21322archive
29 In warenkorb, on "warenkorb anzeigen" page the links are unappealing (some would say that whole interface is confusing because of links) yy  21232view shopping cart
30 If error happens when giving guest login information, there is no clear exit to next step, you need to choose "Startseite" from left frame y   32122guest login
31 There is no way to delete images from archive or album (in view album) y   11422view album
32 For novice user it's maybe hard to understand that "kostenlos anmelden" equals "jetzt registrieren" (that jetzt registrieren is a way to continue)  y  32122login
33 There is no description or picture of the offered items and some of the item names are not very descriptive yy  12322price list
34 Guest login is possible to be thought as real login, yet it does not let you in with right username and password. So it is not clear, who should use it and how. yyy 22222guest login
35 There is no indication what function "Film Importieren" does yy  22312archive
36 If there is no pictures in album, viewing window throws error message "Fehler beim Holen der Beschreibung" which should not be visible to user. yy  32122view album
37 There is no information on the webpage about where the user can use this service from. Austria is obviously (webalbum.at) but what about Germany for example.   y 22222help
38 In film archive, pictures can be moved only from index page, not from view.    y22222archive
39 There is same error message header "Fehler bei Anmeldung" in registration and when editing account info (when you are already "Anmeldung") y   22121.75 change registration
40 Deleting from shopping cart is not clear (put 0 to appropriate field and click link "Änderungen speichern >") y   22211.75view shopping cart
41 Using diashow can be confusing. If you check checkbox, diashow starts. But you cannot (?) select certain pictures to show and use them and also checking checkbox is confusing way to start show. Stopping show is as confusing as starting (uncheck checkbox) y   23111.75view album
42 Registration form does not indicate how long/short username or password should be. y   21221.75registration
43 Registration is needed for accessing many options and there is no screensshots or examples shown of them for unregistered user (no tour) y   11321.75many pages
44 After anmeldung you come to a page where you are told what you can do now, text has upcase words and you would think they would be links to those things but they are not  y  22121.75login
45 The ordering system is way too insecure concerning misuse. No credit card is needed.   y 11321.75order items
46 Image uploading works differently based on did user grant or deny above mentioned rights y   22021.5 image upload
47 If user gives email address which is not valid (like "eeeee") pages will show "Internal Server Error" message. y   11221.5forgot password
48 The name "filmarchiv" for upload-archive is misleading as one would suppose it is the archive for films.   y 21211.5many pages
49 "bilden einfugen" takes to normal album page. This should be indicated, otherwise user can think that "bilden einfugen" is different option with different actions. Also Description uses word "Archiv" even when link goes to Film Album. yy  22111.5shopping cart
50 In "bilder uploaden", it is possible to upload "nothing", this should not be possible  y  21121.5 image upload
51 With ordering form you can get up to five different errormessages at the same time (each in different message box)    y12121.5ordering form
52 In the text that explains the meanings of different possibilities I would expect links  y y21111.25 start page
53 Preisliste should have headings for the columns of the table  y  11121.25price list
54 If user gives valid email address but address does not belong to anyone who has registered, you can wait over 2 minutes on "weiter zun anmeldung" page without nothing happening y   22101.25 forgot password
55 Sometimes image name is replaced with text "ew_" y   11121.25image upload
56 Buttons in the lower right corner is confusing with few pictures only (in album). They probably change image column, but even with one column you can press them, as if something should happen (buttons should probably be "grayed out" in case like this) y   11121.25view album
57 There is no clear exit from editing user information  y  22011.25change registration
58 Command for closing window is not consistent (Verwerfen/Fenster schliessen)  y  11121.25 many pages
59 On album pages some options are pop-up windows, some normal pages without good reason yy  21111.25many pages
60 When changing user data, it's not shown for which username you are changing information  y  21011change registration
61 It possible to get two framesets inside each others (by refreshing page)  y  11021left frame
62 Text links are not underlined.    y11021many pages
63 In album view, nothing indicates that you are at last picture (you can still press forward button normally)    y11111view album
64 "Ausrichtnung" cannot be changed (or can be changed but this change is not processed) (add: User needs to save before this change takes place) y   12000.75edit album
65 Even if saving changes ("Änderung Speichern") is not clicked, changes to image name and description is stored. (How saving works is weird) yy  11010.75edit album
66 Shopping cart indicates price of 25, even if you have not selected any articles to cart y   21000.75view shopping cart
67 Hilfe has typo (WErden) yy  11010.75help page
68 If no email address is given, error message is: "Email-Adresse nicht im passenden Format" and also no example of right format is given y   11100.75 registration
69 There is several synonyms used for username (Benutzername, Zugangskennung) y   11010.75registration
70 Icon for the price list is confusing (looks like computer chip or something) y   11010.75many pages
71 There is no clearly marked exit from album index page y   11010.75album index
72 Different name for same option: Album betrachten / album anzeigen  y  11010.75album
73 In "album anzeigen"-window there is text with some location information, that seems to be telling the location where the picture was uploaded from, no need for that text, at least when there is no explanation. (Later discovered that it is the name of the picture, anyway it should not have whole path as a name)  y  10110.75view album
74 Preisliste has an empty space that looks like an empty column, user can think there should appear something  y  10010.5price list
75 In Hilfe when you go ahead with a link, you can see only the half of the header of the subject you chose  y  10010.5help page
76 Fonts are not used consistently, different font types and sizes are mixed randomly yy  11000.5many pages
77 Textfields used in forms are too long considering data they take y   11000.5many pages
78 In album schicken there is used several names for same thing (versenden,schicken,verschicken) yy  11000.5email friend
79 In "album schicken" "email empfenger" description is not clear (is email address asked or name of the receiver)  y  10010.5email friend
80 Upper frame is unappealing and images may suggest that they are links y   01000.25 top frame
81 Maximum length for username is different in registration (20) and login (21) y   01000.25login
82 Layout of the info page is bad.    y00010.25info

A = Arto Paasovaara, S = Susanna Kivelä, O = Oliver Strasser, M = Michael Fussenegger,
Av = Average.


6. Appendix: Individual Problem Lists

Individual problem lists of each evaluator.

6.1 Arto Paasovaara

No. Description Location
1 If error message has been given in separate message box, page still suggests that it is continuing normally current action loading
2 If error happens when giving guest login information, there is no clear exit to next step, you need to choose "Startseite" from left frame guest login
3 If user gives email address which is not valid (like "eeeee") pages will show "Internal Server Error" message. forgot password
4 If user gives valid email address but address does not belong to anyone who has registered, you can wait over 2 minutes on "weiter zun anmeldung" page without nothing happening forgot password
5 Icon for the price list is confusing (looks like computer chip or something) left frame
6 Registration is needed for accessing many options and there is no screensshots or examples shown of them for unregistered user (no tour) many pages
7 There is no description or picture of the offered items and some of the item names are not very descriptive price list
8 Registration can be only found from login page, it should be clearly on the start page left frame
9 It is not clear (at least from the name) what is difference of Fotoalben and Filmarchiv left frame
10 It is not possible to upload other image types than jpg image upload
11 Upper frame is unappealing and images may suggest that they are links top frame
12 Fonts are not used consistently, different font types and sizes are mixed randomly many pages
13 Registration form does not indicate how long/short username or password should be. registration
14 You cannot go back to registration page (unless using browser's back button) and if you go back by using links of the left frame, you will lose all given data registration
15 Textfields used in forms are too long considering data they take many pages
16 If no email address is given, error message is: "Email-Adresse nicht im passenden Format" and also no example of right format is given registration
17 There is several synonyms used for username (Benutzername, Zugangskennung) registration
18 When registration is successful, user is not clearly shown what to do next. Also message for successful registration should be more clear ("Der Zugang wurde für Sie eingerichtet") registration
19 Maximum length for username is different in registration (20) and login (21) login
20 Guest login is possible to be thought as real login, yet it does not let you in with right username and password guest login
21 Link to account information changes can be found only right after login, that page cannot be accessed later after login
22 Empty textfields for password suggests that user needs to fill them every time they change other information, otherwise password would be set empty change registration
23 There is same error message header "Fehler bei Anmeldung" in registration and when editing account info (when you are already "Anmeldung") change registration
24 When loading page, browser informs that giving rights (which system is asking) is highly riskable, can scare users away image upload
25 Image uploading works differently based on did user grant or deny above mentioned rights image upload
26 If album has no picture, page does not clearly show what page is used for and still, user is asked to save before leaving page edit album
27 System does not ask for user confirmation when deleting album. Also it is possible to think that action could be used for deleting single images album
28 There is no way to delete images from archive or album (in view album) view album
29 There is no clearly marked exit from album index page album index
30 There is no indication what function "Film Importieren" does archive
31 Shopping cart indicates price of 25, even if you have not selected any articles to cart view shopping cart
32 Indication of what currency is used is before price, and at first I did not see it. price list
33 Sometimes image name is replaced with text "ew_" image upload
34 Using diashow can be confusing. If you check checkbox, diashow starts. But you cannot (?) select certain pictures to show and use them and also checking checkbox is confusing way to start show. Stopping show is as confusing as starting (uncheck checkbox) view album
35 Buttons in the lower right corner is confusing with few pictures only (in album). They probably change image column, but even with one column you can press them, as if something should happen (buttons should probably be "grayed out" in case like this) view album
36 Album index has options which album view should also have (delete,copy image to album etc). Actually there is no logical reason that "indexprint" and "album view" are different actions. album index
37 Using 2 comboboxes and 1 link for deleting, copying and buying is not easy interface. album index
38 "Ausrichtnung" cannot be changed (or can be changed but this change is not processed) edit album
39 Even if saving changes ("Änderung Speichern…") is not clicked, changes to image name and description is stored. edit album
40 Deleting from shopping cart is not clear (put 0 to appropriate field and click link "Änderungen speichern >") view shopping cart
41 "bilden einfugen" takes to normal album page. This should be indicated, otherwise user can think that "bilden einfugen" is different option with different actions. Also Description uses word "Archiv" even when link goes to Film Album. shopping cart
42 "Bestellung absenden" should not be different option on main shopping cart page. User should always check first what he/she is ordering and how much it costs, before going to real order page shopping cart

6.2 Susanna Kivelä

No. Description Location
1 Preisliste has an empty space that looks like an empty column, user can think there should appear something price list
2 Preisliste should have pictures of the products as an example price list
3 Preisliste should have headings for the columns of the table price list
4 In Hilfe when you go ahead with a link, you can see only the half of the header of the subject you chose help page
5 Hilfe has typo (WErden) help page
6 Meaning of Gastzugang is unclear (who uses it etc.) start page
7 Registering should be available from startseite, not only from Anmeldung left frame
8 After registering user should be automatically logged in. registration
9 After anmeldung you come to a page where you are told what you can do now, text has upcase words and you would think they would be links to those things but they are not login
10 When changing user data, it's not shown for which username you are changing information change registration
11 Sometimes systemn stops to loading page without never continuing loading
12 There is no clear exit from editing user information change registration
13 Edit page sometimes informs that you cannot choose username which is already in use (form in insert state instead of edit state) change registration
14 Password must be at least 5 digits, this is told only in error message, not before register
15 After successfully editing user information, it is not clear where to go next (no exit) change registration
16 Errormessages are with different font style than everything else many pages
17 On a certain Browser settings (unknown what) browser crashed every time when trying to access image upload page image upload
18 Upload image page throws dangerous looking Java security window where user needs to choose grant or deny. It frightens user. image upload
19 If there is no pictures in album, viewing window throws error message "Fehler beim Holen der Beschreibung" which should not be visible to user. view album
20 If there is no pictures in album, "album bearbeiten" page is confusing (there is no indicator that album does not have pictures) edit album
21 When uploading, there is progressbar, but it only blinks the last "box", and this for many seconds and finally seemed to crash the browser although Windows task list claimed it to be running normally… image upload
22 No possibility to log out left frame
23 Different name for same option: Album betrachten / album anzeigen album
24 In "album anzeigen"-window there is text with some location information, that seems to be telling the location where the picture was uploaded from, no need for that text, at least when there is no explanation. view album
25 In "album anzeigen" I had three times the same picture, but only one of them would be shown (others showed only X for missing picture) view album
26 Command for closing window is not consistent (Verwerfen/Fenster schliessen) many pages
27 In "fotoalben- indexprint" the possible operations are implemented quite unusually, they are not very logical (2 combobox and link for processing) album index
28 In "album bearbeiten" when you change the name of one picture, the change cannot be seen in the page, and you have to save it twice to get it actually saved edit album
29 See the one above. The second save is badly designed. Could be a button or something else edit album
30 Sometimes it is possible to "do something" with upcase text, sometimes not many pages
31 You can destroy a whole album with an accidental mouse click because you are not asked if you really want to delete it album
32 In album schicken there is used several names for same thing (versenden,schicken,verschicken) email friend
33 In "album schicken" "email empfenger" description is not clear (is email address asked or name of the receiver) email friend
34 In "fotoalben" anleitung section is confusing especially when you don't have an album yet, you would think the icons are clickable album
35 In "film archiv" there is possibility to delete upload archive, but then again it can't be deleted (you get error message) archive
36 What is film importieren for? archive
37 In "bilder uploaden", it is possible to upload "nothing", this should not be possible image upload
38 In warenkorb, "bilder einfügen" goes to fotoalbum, user would think that it is new option shopping cart
39 In warenkorb, on "warenkorb anzeigen" page the links are unappealing view shopping cart
40 In the text that explains the meanings of different possibilities I would expect links start page
41 For novice user it's maybe hard to understand that "kostenlos anmelden" equals "jetzt registrieren" (that jetzt registrieren is a way to continue) login
42 It possible to get two framesets inside each others (by refreshing page) left frame
43 On album pages some options are pop-up windows, some normal pages without good reason many pages

6.3 Oliver Strasser

No. Description Location
1 Description of possible tasks ("Anmelden/registieren" etc.) in main-frame on starting page should be links (user would think so) start page
2 The link "Gast-Zugang" (guest-account) seems to be a test-account but obviously isn't as you get asked for username and password guest login
3 You need to login again whenever you want to change the user-data because you get that option only directly after logging. Once he goes further the link is unaccessible. change registration
4 When accessing image upload, a security window pops up informing the user that a special software for upload is needed. Some information about this software should be given before to ensure the user that this software is secure. image upload
5 If the installation of the upload-software is canceled no error message appears. The user can go on to the upload page and try to upload a picture. This, because of the missing upload software did not work for me.(It worked for the others) The problem is that the user never again gets asked to install this software (on same computer) image upload
6 After uploading a picture the picture's path (c:\mypics...) doesn't disappear from the upload form which leaves the user unsecure whether the upload worked or not. image upload
7 The name "filmarchiv" for upload-archive is misleading as one would suppose it is the archive for films. image upload
8 The descriptions of the possible tasks at the Fotoalben-subpage seems like links and distracts from the more important task --> Neues Album anlegen album
9 There is no information on the webpage about where the user can use this services from. Austria is obviously (webalbum.at) but what about germany for example. help
10 The ordering system is way too insecure concerning misuse. No credit card is needed. order items
11 Information about the pictures resolution should be given. How should a friend of mine know about the resolutions of my uploaded pictures? (Important as the resolution is important factor when ordering something) add to shopping cart

6.4 Michael Fussenegger

No. Description Location
1 Text links are not underlined. many pages
2 Icons in option descriptions look like links. album, archive
3 You cannot add images to album from image album, only from archive album
4 When you get Javascript error (messagebox) in loading page, system does not continue anywhere loading
5 In film archive, pictures can be moved only from index page, not from view. archive
6 In album view, nothing indicates that you are at last picture (you can still press forward button normally) view album
7 In image upload, if you upload something else than jpg, there is no error message, picture just won't be there. image upload
8 Layout of the info page is bad. info
9 If you order a picture, you get all the sizes, nothing suggests you cannot order that size... (it depends on resolution) add to shopping cart
10 With ordering form you can get up to five different errormessages at the same time (each in different message box) ordering form
11 You get the list of possible dealers only after you have filled a ordering form, it should be accessible before. ordering form