Figure 3:Test environment
506.021 1KU Human-Computer Interaction WS 2001/2002
| Arto Paasovaara |
| Susanna Kivelä |
| Oliver Strasser |
| Michael Fussenegger |
Six test users performed Thinking aloud test to webbased photalbum system www.webalbum.at during 7.12.2001 - 8.12.2001. Their computer experience varied from 1 year to 18 years. This chapter provides insights to the results.
All users were able to perform most of the given tasks but were something having big problems. Also they did not find site very appealing because of the problems they encountered.
Most important recommendations for improving site's usability are:
-Simplify using albums and archive. Merge these pages or name them more descriptively. Make possible actions easier to find (for example users did not always see "Neues album anlegen"-link)
-Improve image uploading. Have only one way to upload images (the one without Java). Also it should be possible to choose where pictures are placed after uploading. Upload should probably just be one possible action in album pages and in this case
it should load images to current album.
-Inform user more. Tell the user where he/she finds uploaded pictures. Tell the minimum length of the password (and username) in registration form. Provide examples with error messages. Check if there would be other events that would need more information to help the user.
-Make things clickable. Make headers of descriptions on first page links or get rid of them. Make pictures and texts under Anleitung links, or clarify layout so there is no need for Anleitung.
For more detailed recommendations, see chapter 8. List of Recommendations.
Most of the users thought the idea behind this web site is good.
| Web Browser | Internet Explorer 5.0 |
| Operating system | Windows ME |
| Connection | T1 |
| Monitor Colours | 32 bits |
| Monitor Resolution | 1024x768 |
| Monitor Size | 17" |
| Mouse | Logitech Cordless |
| Videocamera | Canon G1000/G1500 |
| Date of evaluation | Pilot: 7.12.2001, Other: 8.12.2001 |
| Time of Evaluation | Pilot: 19:00-20:00 Other: 14:30-18:30 |
Group did not have access to digital camera, so there is no photo about test environment. Instead there is picture of it:
|
Figure 3:Test environment |
One extra test user were taken because microphone were accidentally off in session TP2. Session TP2 were stored because there was some good points in the session.
| Test User | TP0 (Pilot) | TP1 | TP2 | TP3 | TP4 | TP5 |
| Date of Test | 7.12.2001 | 8.12.2001 | 8.12.2001 | 8.12.2001 | 8.12.2001 | 8.12.2001 |
| Time of Test | 19:00-20:00 | 14:30-15:15 | 15:20-16:10 | 16:20-17:00 | 17:05-18:00 | 18:00-18:35 |
| General Information | ||||||
| Name | Martin | Stefan | Harald | Michael | Karla | Christopher |
| Sex | male | male | male | male | female | male |
| Age | 26 | 23 | 25 | 27 | 23 | 21 |
| Education | Sec. school, Chemistry | Sec. school, Computer eng. | Sec. school, Medicin | Sec. school, Computer eng. | Sec. school, Pharmacy | Sec. school, History, German, Philosophy |
| Experience with Computers | ||||||
| Used PC | 18 years | 11 years | 1 year | 10 years | 2 years | 5 years |
| Hours per week | 20 | 50 | 5 | 4 | 2 | 10 |
| Computer normally used | Windows | Windows/Unix | Windows | Windows/Unix | Windows | Windows |
| Experience with Internet and the Web | ||||||
| Emails received/sent per week | 30/10 | 30/28 | 10/10 | 30/10 | 20/20 | 2/2 |
| Web hours per week | 10 | 10-15 | 3 | 2 | 2 | 1 |
| Web Browser | IE | NN | IE | IE/Opera | IE | IE |
| Webmaster experience | none | 4 years | none | 2 years | none | none |
| Application spesific questions | ||||||
| Uses dig. (video)cameras | few times | few times | few times | often | few times | often |
| Registered to Web Service | yes | yes | no | yes | yes | yes |
| Edited digital images | often | few times | few times | few times | never | never |
| Time hh:mm:ss |
Event | Description |
| TP0 (Pilot), German | ||
| Comment: Problems with the camera: after certain time the camera went to stand by, so there are several "blackouts" on session TP0 | ||
| 00:05:50 | T1B | |
| 00:06:40 | T1E | |
| 00:06:50 | T2B | |
| 00:09:05 | T2E | |
| 00:09:15 | T3B | |
| 00:10:25 | Password too short | Recommendation 3 |
| 00:10:45 | Username already in use | |
| 00:X:X | User forgot his password | |
| 00:13:30 | Problems because logged in after first logging as a guest | Bug, not critical in normal use |
| 00:13:50 | Internal server error, probably because of guest login&real login | |
| 00:16:15 | User tries to click descriptive images on album page | Recommendation 4 |
| 00:16:20 | Does not see "Neues album anlegen"-link | Recommendation 1 |
| 00:16:35 | T3E | |
| 00:16:45 | T4B | |
| 00:17:45 | User tries to drag and drop pictures from Windows Explorer to upload interface | See 6. Analysis and discussion |
| 00:19:35 | Login again | |
| 00:19:50 | Tries to find copying from album anzeigen | Recommendation 1 |
| 00:20:15 | Tries uploading again | |
| 00:25:30 | T4E | |
| 00:25:40 | T5B | |
| 00:26:10 | Consulting Info (not Hilfe...) | Recommendation 7 |
| 00:26:30 | Now in Hilfe | |
| 00:29:10 | T5E | |
| 00:29:20 | T6B | |
| 00:29:40 | User tries every possible picture size to see which comes without quality loss | Task needs little adjusting |
| 00:31:00 | T6E | |
| 00:31:10 | T7B | |
| 00:32:10 | "I have to look everywhere" (for changing password) | Recommendation 5 |
| 00:33:25 | T7E (Task failed) | |
| 00:33:30 | T8B | |
| 00:33:50 | Mysteriously pictures in the album have been disappeared. Reason not found | |
| 00:36:20 | T8E | |
| TP1 (Test User 1), English | ||
| Comment: Only session with English Thinking Aloud & Interviews. Tasks has been changed after TP0 | ||
| 00:48:35 | T1B | |
| 00:49:10 | "I can't click on that" (Texts on first page) | Recommendation 4 |
| 00:49:45 | "They should put Euros here" (Preisliste) | |
| 00:49:45 | T1E | |
| 00:50:05 | T2B | |
| 00:51:45 | T2E | |
| 00:52:00 | T3B | |
| 00:xx:xx | T3E | |
| 00:xx:xx | T4B | |
| 00:55:34 | "This is irritating" (Java security Window on Image upload) | |
| 00:56:40 | "Do you want to debug"-messagebox | |
| 00:57:50 | Tries to find uploaded pictures from photoalbum | Recommendation 1 |
| 00:58:25 | "Why this is different" (Different upload interface) | Recommendation 2 |
| 00:59:10 | "Where are they?" (looking for uploaded pictures) | Recommendation 3 |
| 01:00:55 | "I don't want to read this stuff" (Album bearbeiten) | |
| 01:02:00 | T4E | |
| 01:02:15 | T5B | |
| 01:04:02 | "I can lose a bit of quality" (After trying to find picture without quality loss) | |
| 01:04:30 | "No idea where I could get those pictures" (No delivery info shown yet) | Recommendation 3 |
| 01:05:50 | Order form give many error messages same time | |
| 01:06:30 | "You want to know everything before I get to know where I can get it" (Still no info where items will be delivered) | |
| 01:06:45 | T5E | |
| 01:07:30 | T6B | |
| 01:08:10 | Checking Hilfe and Info (for info about changing account information) | Recommendation 5 |
| 01:08:45 | Tries searching Hilfe with Browser's find, but no information about changing account information | Recommendation 7 |
| 01:09:10 | "Are you sure I can change them" | |
| 01:09:50 | T5E | |
| TP2 (Test User 2), German | ||
| Comment: By accident microphone is off, so no voise in this session | ||
| 01:26:05 | T1B | |
| 01:27:10 | T1E | |
| 01:28:20 | T2B | |
| 01:30:45 | T2E | |
| 01:31:00 | T3B | |
| 01:33:25 | Too short password | Recommendation 3 |
| 01:33:30 | User thinks there is still problem with password, problem is actually in username (name for username is not consistent) | Consistency problem |
| 01:39:00 | T3E | |
| 01:39:15 | T4B | |
| 01:39:xx | Trying to find image upload from Album bearbeitung | Recommendation 2 |
| 01:45:10 | After Java security warning goes away from image upload (uncertain) | Recommendation 2 |
| 01:47:00 | User helped (with upload) | |
| 01:47:15 | Problems with uploading (User thought browse button is not right action) | Recommendation 2 |
| 01:50:00 | T4E (Task failed, time limit) | |
| 01:50:45 | T5B | |
| 01:55:20 | T5E | |
| 01:55:30 | T6B | |
| 01:56:xx | Checks Hilfe,Info and Anmeldung for account info changes, no help found | Recommendation 7 |
| 01:59:15 | T6E (Task failed) | |
| TP3 (Test User 3) 2:05:40, German | ||
| 02:10:35 | T1B | |
| 02:11:45 | Went to price list, for some reason went back before finding price for fotopuzzle | |
| 02:13:00 | T1E | |
| 02:13:15 | T2B | |
| 02:14:35 | T2E | |
| 02:14:45 | T3B | |
| 02:15:40 | Password too short | Recommendation 3 |
| 02:17:00 | T3E | |
| 02:17:10 | T4B | |
| 02:18:00 | Ignored the Java Security window, clicked yes | |
| 02:21:45 | T4E | |
| 02:21:55 | T5B | |
| 02:2x:xx | Ignored quality loss message box, clicked ok | |
| 02:23:55 | User ordered actually 3 pictures | |
| 02:25:00 | T5E | |
| 02:25:15 | T6B | |
| 02:26:xx | User tries everything to find logout button, then place for changing password | Recommendation 5 |
| 02:27:30 | T6E | |
| TP4 (Test User 4) 2:33:30 , German | ||
| 02:38:00 | T1B | |
| 02:38:19 | Thinks that the texts on first page are links | Recommendation 4 |
| 02:38:45 | T1E | |
| 02:39:00 | T2B | |
| 02:40:40 | T2E | |
| 02:40:50 | T3B | |
| 02:43:20 | Password and password check are different | |
| 02:38:00 | Left @ from the email address, error message: "Not in right format" | Recommendation 3 |
| 02:48:50 | Clicking description images on album page | Recommendation 4 |
| 02:50:50 | T3E | |
| 02:50:55 | T4B | |
| 02:38:00 | Copying and pasting images from Windows Explorer | See 6. Analysis and discussion |
| 02:52:xx | User tries everything | |
| 02:54:55 | Makes copy of the picture in Windows Explorer | |
| 02:56:25 | Second upload try | |
| 02:56:40 | "I have uploaded them but I don't know where" | Recommendation 3 |
| 02:58.xx | Hilfe helped (found film archive with it) | |
| 03:00:35 | T4E | |
| 03:00:42 | T5B | |
| 03:00:45 | First action: Hilfe | |
| 03:04:50 | T5E | |
| 03:05:05 | T6B | |
| 03:08:05 | User deleted album, wants to make a new account also (gets irritated?) | |
| 03:09:20 | T6E (Task failed) | |
| TP5 (Test User 5), German | ||
| 03:19:30 | T1B | |
| 03:20:20 | T1E | |
| 03:20:30 | T2B | |
| 03:22:00 | T2E | |
| 03:22:10 | T3B | |
| 03:22:30 | Tries to click texts (thinks as a links) | Recommendation 4 |
| 03:23:10 | Fills also optional fields in registration form but not email | |
| 03:26:10 | T3E | |
| 03:26:20 | T4B | |
| 03:26:35 | Tries to find image upload from album bearbeitung | Recommendation 2 |
| 03:28:xx | Uploads pictures one at the time | |
| 03:29:30 | "I do something different" and tries using right mouse button ("It never helped before") | |
| 03:35:50 | T4E | |
| 03:34:10 | T5B | |
| 03:35:05 | Problem because of popup windows (Window on the background: seems like clicking button does not do anymore anything) | Recommendation 6 |
| 03:37:48 | Went to loading page with browser's back-button = would never continue | |
| 03:38:20 | Help given: Popup windows are already opened | |
| 03:40:27 | T5E | |
| 03:40:32 | T6B | |
| 03:41:10 | T6E (User saw the link after logging in. Also user said that it is the place where he normally would search for it.) | |
R1 Simplify using albums and archive
| Severity: | 3.5 |
| TPs: | TP0, TP1, minor with others also |
| Timestamps: | 00:16:20 and 01:19:50 (TP0), 00:57:50 (TP1). |
It was not clear to users what is the difference between album and archive and users were not able to predict what options would be found from each subpage.
There are not many exact points in sessions when this happens, but this confused and hindered almost every test user.
Suggestion: Merge these pages or name them more descriptively. Make possible actions easier to find (for example users did not always see "Neues album anlegen"-link)
| Severity: | 3.5 |
| TPs: | TP1, TP2, TP5 |
| Timestamps: | 00:55:34 and 00:58:25 (TP1), 01:39:xx and 01:45:10 and 01:47:15 (TP2), 03:26:35 (TP5). |
Some users tried to find image upload from album pages, they did not recognise "Bilder uploaden", one even thought it was for uploading pictures from the web.
Also Java security window coming when accessing upload page seem to be irritating, even confusing. Interviews showed also that there should be some kind of possibility to upload straight to a certain album.
Suggestion: Have only one way to upload images (the one without Java). Also it should be possible to choose where pictures are placed after uploading. Upload should probably just be one possible action in album pages and in this case
it should load images to current album.
| Severity: | 3.5 |
| TPs: | TP0, TP1, TP2, TP3, TP4 |
| Timestamps: | 00:10:25 (TP0), 00:59:10 and 01:04:30 (TP1), 01:33:25 (TP2), 02:15:40 (TP3), 02:38:00 and 02:56:40 (TP4) |
There where some cases where user would have needed some more information what to do next or how something should be done. Half of the test users provided too short password,
one had problems with appropriate format of email address. The most important situation where users need to know what to do next is after uploading. No-one did not know where he/she would
find the uploaded pictures.
Suggestion: Tell the user where he/she finds uploaded pictures. Tell the minimum length of the password (and username) in registration form. Provide examples with error messages. Check if there would be other events that would need more information to help the user.
| Severity: | 3.25 |
| TPs: | TP0, TP1, TP4, TP5 |
| Timestamps: | 00:16:15 (TP0), 00:49:10 (TP1), 02:38:19 and 02:48:50 (TP4), 03:22:30 (TP5) |
There are many pictures and texts, which users thought as links, including: headers of descriptive texts on first page and icon descriptions under "Anleitung" (Album and Archive).
Suggestion: Make headers of the descriptions on the first page links or get rid of them. Make pictures and texts in "Anleitung" links, or clarify layout so there is no need for "Anleitung".
| Severity: | 3.0 |
| TPs: | TP0, TP1, (others also) |
| Timestamps: | 00:32:10 (TP0), 01:08:10 (TP1) |
Changing account information for example changing password is hard to find because it is only shown right after logging in. Also it is not mentioned in Hilfe how access account information.
Registering should probably have its own link, users needed to think some before checking "Anmeldung" page. Some users also asked for a possibility to log out.
Suggestion: Link for Registering, Logout and Changing account information should be in the left frame (maybe as its own group).
R6 Avoid popup-windows or improve them
| Severity: | 2.75 |
| TPs: | TP5 (some other also) |
| Timestamps: | 03:35:05 (TP5) |
If user opens a popup window and forgets to close it, and later tries to access same action again, it can seem that link is not working as popup window already exists in the
background (and won't pop up).
Suggestion: Avoid using popup windows, keep events in the frameset. If it is not possible, make popup windows to pop up from background so user sees it is working.
| Severity: | 2.5 |
| TPs: | TP0, TP1, TP2 |
| Timestamps: | 00:26:10 and 00:32:10 (TP0), 01:08:45 (TP1), 01:56:xx (TP2) |
Difference between Hilfe and Info is unclear, some users tried to find help from Info. Also Help pages should cover every option or possibility shown on pages.
Suggestion: Change name of the "Info" to "Corporate Info" or similar so it would not be confused as a help. Cover all options on pages (changing account info, what is Film Id etc.)
