Heuristic Evaluation Report
INB.05021UF Human-Computer Interaction 3VU SS 2020
Group GT-07
Nicole Krämer
Philip Samer
Tanja Tatschl
Marcel Weiss
Heuristic Evaluation of the Web Site
https://www.oebb.at/
Report of 21th September 2020
1 Executive Summary
Overall the website offers good usability and fulfills it's purpose with neat and mostly easy to use features. Nonetheless it still harbours some problems, which will be explained in more detail in the following report.
The evaluation was conducted by four evaluators, each one individually gathering positive and negative findings. The findings of all team members were individually scored in their severity to determine most critical and positive findings.
The most severely rated problem is a check box in the "terms of use" section that would not save the users setting regarding cookies. The check box itself on the "terms of use" page is also hidden in a scrollable text. For the users the placement of the box may come across questionable. The fact that the site does not save the users choice about cookies can lead to dissatisfaction.
What stood out as a positve finding was the good overall construction of the Homepage, which instantly presents some of the most important features it has to offer directly available for use. We had no problems in finding the functionality to take on the most common tasks.
A following negative finding was the missing information and confusing options around the first class ticket for weekly and monthly tickets. The ticket options state first and second class. Though looking down at the extras section first class is still selectable there. Also the price for first class equals the choosen basis ticket. Clear Indication and information for the user is missing here.
Another good finding we agreed on was the possibility to virtually walk through the different sections of a train.
The advertisement and note to use the ÖBB apps on the website are pretty useful, but the link to the Huawei App Gallery doesn't correctly redirect the user.
The ability to mark the most often used or most relevant train connections in the ticket shop proves to be a valuable asset of the website.
On the other hand a rather unpleasing finding was the fact that to disable the use of cookies the user has to navigate through the site to the "Terms of Use" and uncheck a check box there. This is not a conventional way of disabling and enabling the use of cookies.
The ability to also mark the most often used or most relevant train connections in the ticket shop also proves to be a valuable asset of the website.
Our team did encounter a few problems that we rated severy and in our opinion need to be changed. But while certainly having its flaws, the basic functionality of the website is given.
2 Introduction
The website to be evaluated is "www.oebb.at", the official website of the austrian railway network. The website is used to gather information about the timetable, ticket costs and services of the oebb and to acquire tickets. We want to evaluate this web site, because it is very important for many people depending on public transportation to have a reliable platform.
3 Evaluation Procedure
This section describes the procedure used in the heuristic evaluation.
A heuristic evaluation is a usability test which can be taken on any kind of computer application or device to find usability flaws.
The evaluation was conducted by a team consisting of four members. Each evaluator individually evaluated the website and filmed short clips of the positive and negative findings. After the evaluation, the findings of every team member were put together and individually scored in their severity or positivity. After the scoring they were sorted in their new order.
The tests were conducted individually by every evaluator on a Desktop PC or laptop and on a mobile phone.
3.1 Evaluation Methodology
Heuristic evaluation is defined by Jakob Nielsen and Rolf Molich as an informal method of analyzing an interface design in 1990. [Nie1990]
Heuristic evaluation is a practical method to help developers evaluate a user interface design with a small set of test users. It is a fast, cheap and easy approach to detect usability problems. This analysis is intended to improve the user-friendliness of interfaces, for example a website. To achieve a significant evaluation result, a group of 5-8 people should examine the design for usability problems. The evaluators have to understand the purpose of the system or product and what the target users goals are. To understand this needs, user personalities have to be defined before the evaluation. It is important that the test users evaluate the user interface individually. This ensures a higher percentage of the heuristic problems found. The individual test users capture the positives and negatives they discovered ranked according to their severity. The notes must be precisely specified and identified. [Phil2017]
For this evaluation, the "Andrews General Usability Heuristics", shown in Appendix A.1, will be used. These are based on and slightly adapted from Nielsen's revised set of ten usability heuristics. This illustration provides guidance for the evaluators, pointing out what they should pay attention to and what the system or product should be able to do. For each heuristic problem a scale from 0-5 is used. [Nie1994].
It is good practice to evaluate the product at least twice. A roughly walk through the interface followed by a more detailed analysis. If the system is used by several kinds of devices, it also makes sense to perform the evaluation on a few different ones. This way it can be ensured that the interface is likely to work for the end user as well. Once the results of the individual evaluators have been fully documented, they can exchange views and determine which heuristic problems are serious and which are minor and list them. This way, the expert knows what to focus on during the revision. [Nie1995]
3.2 User Profiles
The typical users of the website are people who use the public transport system. The variety of the user is therefore huge. Users that come to your mind are seniors, teenagers, environmentally conscious, tourists, disabled people, students, workers, school classes and families. These users can be grouped in four categories: people with handicaps (seniors, disabled), groups (tourists, school classes, families), people with technical knowhow (student, worker, teenager), people without technical knowhow (seniors, workers).
The general user will use the website to book a ticket and find all available trains on the timetable.
People with handicaps can use the website to gather information about the handicapped accessible of the train station and trains. This group will also often want to make seat reservations.
Groups will use the website to book a large number of tickets and try to find some information about a group discount or the capacity of the train. If you are in a big group it is important to have a reliable information source to plan in advanced.
People, who travel alone and have no experience with online booking, need a pretty simple and manageable website. Additionally the information about changes of arrival and departure time should be very easy to find.
For people who have experience with the website will want a fast way to book a ticket, in the best case, the system remembers the usual journeys and makes a suggestion. Frequent users will properly use the mobile version more than the other groups to book and find the right train.
3.3 Extent of the Evaluation
We are evaluating the entire english version of the website https://www.oebb.at/, excluding external links (adds, related links, ..).
3.4 Evaluators and Evaluation Environments
The mobile evaluation environments used by each evaluator are shown in Table 1. The PC evaluation environments used by each evaluator are shown in Table 2. For this evaluation, mobile devices were operated in portrait mode.
| Evaluator | Nicole Krämer (NK) | Philip Samer (PS) | Tanja Tatschl (TT) | Marcel Weiss (MW) |
|---|---|---|---|---|
| Age | 23 | 23 | 23 | 21 |
| Sex | female | male | female | male |
| Device | HUAWEI P20 lite | HUAWEI P smart | HUAWEI P30 lite | HUAWEI P10 lite |
| OS and Version | Android 8.0.0 | Android 8.0.0 | Android 9.0 Pie | Android 8.0.0 |
| Screen Size | 5.84" | 5.65" | 6.15" | 5.2" |
| Screen Resolution | 2280 x 1080 | 2160x1080 | 2312x1080 | 1080x1920 |
| Web Browser | Firefox 80.1.2 | Chrome 85.0.4183.81 | Android Browser | Chrome 85.0.4183.81 |
| Ad Blocker | none | none | none | none |
| Internet Connection | A1 | WiFi | A1 | WiFi |
| Download Speed | 39 mbps | 47 mbps | 100 mbps | 23 mbps |
| Browser Resolution | 360x631 | 360x598 | 360x647 | 360x574 |
| Screen Recording Software | AZ Screen Recorder | AZ Screen Recorder | AZ Screen Recorder | Android Screen Recorder |
| Date of Evaluation | 2020-09-11 | 2020-09-07 | 2020-09-10 | 2020-09-09 |
| Time of Evaluation | 18:00-22:00 | 18:00-20:00 | 18:00-19:30 | 19:30-20:00 |
| Evaluator | Nicole Krämer (NK) | Philip Samer (PS) | Tanja Tatschl (TT) | Marcel Weiss (MW) |
|---|---|---|---|---|
| Age | 23 | 23 | 23 | 21 |
| Sex | female | male | female | male |
| Device | ASUS VivoBook (Laptop) | PC | Laptop | PC(laptop) |
| OS and Version | Linux Mint 19.1 tessa | Windows 10 Home | Windows 10 Home | Windows 8.1 Pro |
| Screen Size | 15.6" | 24" | 15.6" | 15.6" |
| Screen Resolution | 1920x1080 | 1920×1080 | 1920x1080 | 1920x1080 |
| Web Browser | Mozilla Firefox 75.0 | Edge 85.0.564.44 | Chrome 84.0.4147.135 | Brave 1.13.82 |
| Ad Blocker | AdBlock Plus, DoNotTrackMe, Ghostery | none | AdBlock Plus | none |
| Internet Connection | A1 | WiFi | WiFi | WiFi |
| Download Speed | 39 mbps | 47 mbps | 28 mbps | 23 mbps |
| Browser Resolution | 1920x938 | 1920×1080 | 1536x674 | 1920x1080 |
| Screen Recording Software | OBS Studio 25.0.8 | OBS Studio 25.0.8 | OBS Studio 25.0.8 | OBS Studio 25.0.8 |
| Date of Evaluation | 2020-09-12 | 2020-09-08 | 2020-09-11 | 2020-09-09 |
| Time of Evaluation | 18:00-22:00 | 08:15-10:30 | 18:00-19:30 | 18:30-19:00 |
On all operating systems the evaluators will use LosslessCut to extract smaller video clips from their screen recording sessions.
4 Results of the Evaluation
This section describes the results of the heuristic evaluation.
4.1 Top Three Positive Findings
The top three positive findings according to their average (mean) positivity ratings are described in more detail below. The positivity rating scheme used to rank the positive findings is shown in Table 3.
| Positivity | Meaning |
|---|---|
| 4 | Extremely Positive |
| 3 | Major Positive |
| 2 | Minor Positive |
| 1 | Cosmetic Positive |
| 0 | Not a Positive |
P01. Good Homepage Overview
| Title: | Good Homepage Overview |
|---|---|
| Description: | The important key-features of the website are instantly noticeable. |
| Video Clip(s): | p01-mw-pc-overview.mp4, |
| Location (How Reproducible?): | Start Page |
| Mean Positivity: | 3.25 |
The two evaluators who discovered the site map both remarked that important key-features of the website are instantly noticeable. So you have important informations on the first sight and don't waste any time to find them. This are for example book ticket, real time infos, route information and search connection as shown in Figure 1.
P02.Virtually walk through the train
| Title: | Virtually walk through the train |
|---|---|
| Description: | You can go virtually thorough the train and look around to see how the train looks like. |
| Video Clip(s): | p02-ps-pc-virtual-train.mp4 |
| Location (How Reproducible?): | Travel planning & services - in the train- our trains |
| Mean Positivity: | 3.00 |
The OEBB site offers a feature that takes you virtually through the train. So you can see how it looks like on the train and get an overview where the canteen is, where you are sitting etc. You can see how it is done in Figure 2.
P03. Journey start page
| Title: | Journey start page |
|---|---|
| Description: | The Journey start page will show you your often needed connections. |
| Video Clip(s): | p03-nk-pc-journey-startpage.mp4, |
| Location (How Reproducible?): | On ticket site (tickets.oebb.at) select your journey - click "put on start page" - choose preferences - click on "next" |
| Mean Positivity: | 3.00 |
Interesting for people who often travel with the OEBB is the feature as shown in Figure 3. The journey start page shows the user his most frequently used connections. This saves time if you have to travel the same route often.
4.2 List of All Positive Findings
Table 4 shows a list of all the positive findings which emerged from the evaluation, sorted in descreasing order of average (mean) positivity, i.e. the most positive are at the top of the table. The name codes assigned to each evaluator are shown in Table 5.
| No. | Title | Description | Video Clip(s) | Location (How Reproducible?) | Found By | Positivity | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NK | PS | TT | MW | NK | PS | TT | MW | Mean | |||||
| 1 | Good Homepage Overview | 'The important key-features of the website are instantly noticeable. For example the ticket shop search function or route planer.' | p01-mw-pc-overview.mp4 | Start Page | y | y | 3 | 3 | 3 | 4 | 3.25 | ||
| 2 | Virtually walk through the train | You can go virtually thorough the train and look around to see how the train looks like. | p02-ps-pc-virtual-train.mp4 | Travel planning & services - in the train- our trains | y | 2 | 4 | 3 | 3 | 3.00 | |||
| 3 | Journey start page | will show you your often needed connections | p03-nk-pc-journey-startpage.mp4 | On ticket site (tickets.oebb.at) select your journey - click "put on start page" - choose preferences - click on "next" | y | 3 | 4 | 3 | 2 | 3.00 | |||
| 4 | Ticket button | 'The ticket button is the whole time at the top of the page,so you can directly go to book you ticket.' | p04-ps-mobile-book-button.mp4 | - | y | 2 | 2 | 2 | 3 | 2.25 | |||
| 5 | Journey preview | informs you of all trains planned to use and on which platforms they will be leaving and arriving | p05-nk-pc-journey-preview.mp4 | On ticket site (tickets.oebb.at) select your journey - click on "journey preview" | y | 2 | 2 | 3 | 2 | 2.25 | |||
| 6 | Print journey preview | Print option for the journey preview. Comfortable and useful feature | p06-nk-pc-print-journey-preview.mp4 | On ticket site (tickets.oebb.at) select your journey - click on "journey preview" - click on "print" at the top of the page | y | 3 | 2 | 2 | 2 | 2.25 | |||
| 7 | Timetable Images | 'Provided images with all the timetables of the different train lines. For example: Can be downloaded for offline use.' | p07-mw-pc-timetables.mp4 | Start Page - Timetable - Timetable images | y | 2 | 3 | 2 | 2 | 2.25 | |||
| 8 | Easy preference selection | Ticket store allows to easily change preferences | p08-nk-mobile-easy-preference-selection.mp4 | homepage of the ticket site - select journey - click on "filters" | y | 2 | 2 | 2 | 2 | 2.00 | |||
| 9 | Loading Feedback | Good loading feedback | p09-tt-pc-laoding-feedback.mp4 | book ticket | y | y | 2 | 2 | 2 | 2 | 2.00 | ||
| 10 | Urgent News | 'On the Starting Page you will find a clickable header called "Urgent News". There you can read about the current situation regarding public transport.' | p10-mw-mobile-urgent-news.mp4 | Starting Page - "Urgent News" | y | 2 | 1 | 2 | 3 | 2.00 | |||
| 11 | Information about trains | Some trains have a nice description with a good visualisation. | p11-ps-pc-information-visualisation.mp4 | Travel planning & services - in the train- our trains | y | 1 | 1 | 2 | 2 | 1.50 | |||
| 12 | Indication to external website | The menu has a box with an arrow if the link opens a new tap to an external website. | p12-ps-pc-indication-in-menu.mp4 | Menu | y | y | 1 | 1 | 2 | 2 | 1.50 | ||
| 13 | Construction Work | Easy to see/find construction work | p13-tt-pc-construction-work.mp4 | real time info | y | 1 | 2 | 1 | 2 | 1.50 | |||
| 14 | Efficient Ticket Shop | 'The ticket shop is easy accessible and also easy to use. It''s design is also somewhat minimalistic and not overflowed with useless information.' | p14-mw-pc-efficient-ticket-shop.mp4 | Start Page - Book Ticket(Button) | y | 1 | 2 | 2 | 1 | 1.50 | |||
| 15 | Swiping through relevant sites | Easy to swipe through topic related links | p15-nk-mobile-swipe-relevant-sites.mp4 | Homepage → tickets and customer cards | y | 2 | 1 | 1 | 1 | 1.25 | |||
| 16 | Warning Message | Warning message is very good | p16-tt-mobile-warning-message.mp4,p16-tt-pc-warning-message.mp4 | book ticket/select ticket/add to basket | y | 1 | 1 | 2 | 1 | 1.25 | |||
| 17 | Help | Help is easy to find | p17-tt-mobile-finding-help.mp4 | menu/oebb customer service | y | 2 | 1 | 2 | 0 | 1.25 | |||
| 18 | Contrast button | With a button on the top of the website the contrast can be changed. | p18-ps-mobile-contrast.mp4 | Button on the top | y | 2 | 1 | 0 | 1 | 1.00 | |||
| 19 | Homepage menu | Even though a lot of links and categories the homepage menu felt intuitive and easy to use. | p19-nk-mobile-homepage-menu.mp4 | Homepage - open top left menu | y | 2 | 1 | 1 | 0 | 1.00 | |||
| 20 | Online Form Consistency | 'Every online form is consitent in choosing their highlighters colors and symbols.' | p20-mw-mobile-online-form-consistency.mp4 | Anywhere | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 21 | Layout | Layout uses the whole window. | p21-ps-mobile-layout.mp4 | - | y | 1 | 1 | 0 | 1 | 0.75 | |||
| Code | Evaluator |
|---|---|
| NK | Nicole Krämer |
| PS | Philip Samer |
| TT | Tanja Tatschl |
| MW | Marcel Weiss |
| y | Found by this evaluator |
4.3 Top Five Problems
The top five problems according to their average (mean) severity ratings are described in more detail below. Problem number 1 is the problem (negative finding) with the highest mean severity. The severity rating scheme used to rank the problems is shown in Table 6.
| Severity | Meaning |
|---|---|
| 4 | Catastophic problem |
| 3 | Serious problem |
| 2 | Minor problem |
| 1 | Cosmetic problem |
| 0 | Not a problem |
N01. Doesn’t save privacy setting
| Title: | Doesn’t save privacy setting |
|---|---|
| Description: | If you uncheck the box in “Terms of use” and then do something else on the site it want safe that and will still use your data. |
| Video Clip(s) | n01-ps-pc-checkbox.mp4 |
| Heuristic: | - |
| Location (How Reproducible?): | Bottom- Terms of use |
| Mean Severity: | 4.0 |
The main problem with the privacy sewttings is shown in Figure 4. It is not user friendly to change the settings after the user specifically uncheck the checkbox.
It would be better if there is an actually save button to save the settings.
N02. First class extra price
| Title: | First class extra price |
|---|---|
| Description: | No clear indication if first class is included, needs to be included and if the price actually is the same as the basis ticket. |
| Video Clip(s): | n02-nk-mobile-first-class.mp4 |
| Heuristic: | - |
| Location (How Reproducible?): | On the ticket site (tickets.oebb.at) choose a journey and select "weekly and monthly tickets" (choose route if more than 1 available) |
| Mean Severity: | 3.75 |
When buying a weekly or monthly ticket, the information and options around the first class provided for confusion. The ticket options state first and second class. Though looking down at the extras section first class is still selectable there. Also the price for first class equals the choosen basis ticket, eg. monthly. Not only is there no clear indication whether or not the extra needs to be booked to use first class, the price also is irritating for exactly doubling costs which adds to the confusion.
A solution to that would be a clear presentation of information. Do I have a ticket that goes first class? Remove the extra if its included, if not, remove the misleading text on the ticket information. Clearly indicate the price for the extra equals the ticket price to assure the user that this is intended, if so.
N03. Hidden checkbox
| Title: | Hidden checkbox |
|---|---|
| Description: | On the page “Terms of use“ is a hidden checkbox if you want to not allow the website to analyse you. |
| Video Clip(s): | n03-ps-mobile-hidden-checkbox.mp4 |
| Heuristic: | - |
| Location (How Reproducible?): | clicking on “Terms if use” |
| Mean Severity: | 3.75 |
There is a function that is very important but hidden. If you as a user don't want to be spied on and your actions and clicks to be tracked, there is a button you can disable. Many users do not even notice that this function is available on the terms of use page as shown in Figure 6.
Make the check box clearly visible or initiate a pop up when entering the website.
N04. Feedback form - misleading question
| Title: | Feedback form - misleading question |
|---|---|
| Description: | The Question "Are you happy with the service you have received from us?" only has options to participate or not participate in a survey. |
| Video Clip(s): | n04-nk-pc-feedback-misleading-question.mp4 |
| Heuristic: | - |
| Location (How Reproducible?): | Homepage - travel planning and services - ÖBB customer service - complaint, praise and ideas - take necessary steps of the form and select "I want an answer" |
| Mean Severity: | 3.25 |
The evaluator noticed that if you are satisfied with the service and you want to make this known in the feedback, you should participate in a survey. This question can be misleading. After all, you are already writing the feedback and you do not want to have to participate in a survey. This problem is described in Figure 7.
Add an additional check box, which the users can tick or untick for participating in the survey, when sending a feedback form.
N05.Printout Timetable
| Title: | Printout Timetable |
|---|---|
| Description: | You can not print the timetable. |
| Video Clip(s): | n05-tt-pc-print-timetable.mp4 |
| Heuristic: | A04 Consistency |
| Location (How Reproducible?): | Book Ticket/select ticket/my basket/link under "total amount" |
| Mean Severity: | 3.25 |
Once you have booked your ticket, you should be able to print it out in the overview at the end. Unfortunately this does not work with all browsers. You can only view the timetable on the page without being able to print or download it. You can see this problem in Figure 8.
Adding extra information about the use and the requirements to use the feature.
4.4 List of All Problems Found
Table 7 shows a list of all the problems found (negative findings) in the evaluation, sorted in descreasing order of average (mean) severity. The name codes assigned to each evaluator are shown in Table 8.
| No. | Title | Description | Video Clip(s) | Heuristic | Browser / OS | Location (How Reproducible?) | Found By | Severity | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NK | PS | TT | MW | NK | PS | TT | MW | Mean | |||||||
| 1 | Doesn’t save privacy setting | If you uncheck the box in “Terms of use” and then do something else on the site it want safe that and will still use your data. | n01-ps-pc-checkbox.mp4 | - | Edge / Windows | Bottom- Terms of use | y | 4 | 4 | 4 | 4 | 4.00 | |||
| 2 | First class extra price | 'No clear indication if first class is included or needs to be included and if the price actually is the same as the basis ticket. ' | n02-nk-mobile-first-class.mp4 | - | Firefox 80.1.2/ Android 8.0.0 | On the ticket site (tickets.oebb.at) choose a journey and select "weekly and monthly tickets" (choose route if more than 1 available) | y | 3 | 4 | 4 | 4 | 3.75 | |||
| 3 | Hidden checkbox | On the page “Terms of use“ is a hidden checkbox if you want to not allow the website to analyse you. | n03-ps-mobile-hidden-checkbox.mp4 | - | Chrome/ Android | clicking on “Terms if use” | y | 4 | 4 | 3 | 4 | 3.75 | |||
| 4 | Feedback form - misleading question | 'The Question "Are you happy with the service you have received from us?" only has otions to participate or not participate in a survey. Misleading question' | n04-nk-pc-feedback-misleading-question.mp4 | - | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa | Homepage - travel planning and services - ÖBB customer service – complaint praise and ideas - take necessary steps of the form and select "I want an answer" | y | 3 | 4 | 4 | 2 | 3.25 | |||
| 5 | Printout Timetable | You can not print the timetable | n05-tt-pc-print-timetable.mp4 | A04 Consistency | Chrome 84.0.4147.135 | 'Book Ticket/select ticket/my basket/link under "total amount"' | y | 3 | 3 | 4 | 3 | 3.25 | |||
| 6 | Dark pattern | Uses cookies and if the user doesn’t want that he must go under “Terms of Use” and uncheck a box. | n06-ps-pc-dark-pattern.mp4 | A10 Help and Documentation | Edge / Windows | Bottom- Terms of use | y | 3 | 4 | 3 | 3 | 3.25 | |||
| 7 | App in Huawei gallery | The link to the app in the Huawei App Gallery is only a normal link to the Huawei App Gallery and not directly to the ÖBB app. | n07-ps-pc-huawei-app.mp4 | A05 Error Prevention | Edge / Windows | Mobile ticketing - link to the Huawei page | y | 3 | 3 | 3 | 4 | 3.25 | |||
| 8 | Open new taps | Some of the links used on the website open a new tap in the browser (Edge). | n08-ps-pc-links-with-new-tap.mp4 | A04 Consistency | Edge / Windows | Tickets & customer card- sparschiene | y | 2 | 4 | 4 | 2 | 3.00 | |||
| 9 | Not working menu | “Lower Austria” has no item in the drop-down menu but you can see a strange line. (In the German version are several items) | n09-ps-pc-menu-error.mp4 | - | Edge / Windows | Regional offers - hover over lower Austria | y | 2 | 4 | 3 | 3 | 3.00 | |||
| 10 | Radticket – adding bicycles error message | Under "Who is Going" the Einfach-Raus-Radticket offers the option to add bicycles. It is unclear if bicycles are already included or need to be added there. Adding leads to an error message | n10-nk-pc-unclear-add-bicycles-radticket.mp4n10-nk-mobile-unclear-add-bicycles-radticket.mp4 | A05 Error Prevention | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa; Firefox 80.1.2/ Android 8.0.0 | 'Ticket store - Einfach-Raus-Radticket - "Who is Going"- add bicycles - click on "next"' | y | 3 | 3 | 4 | 2 | 3.00 | |||
| 11 | Scotty info box | After clicking on the info box about Scotty you get to a website that has no information about Scotty. | n11-ps-pc-about-scotty.mp4 | A10 Help and Documentation | Edge / Windows | On the Scotty page at the top “About SCOTTY” | y | 3 | 3 | 3 | 3 | 3.00 | |||
| 12 | FAQs missing | There is no FAQ site | n12-tt-pc-faqs-missing.mp4 | A10 Help and Documentation | Chrome 84.0.4147.135 | none | y | y | y | 3 | 3 | 3 | 3 | 3.00 | |
| 13 | Change of tickets | If you want to change the number of tickets you have to click on the change button and select who is going and to get back you have to click on next. | n13-ps-pc-number-of-tickets.mp4 | A04 Consistency | Edge / Windows | Homepage - Book ticket | y | y | 3 | 3 | 2 | 3 | 2.75 | ||
| 14 | ÖBB logo | If you click on the ÖBB logo it returns to the home page. If you switch to another side the logo is still the same but it leads to the switched home page. (Without Scotty) | n14-ps-pc-click-on-logo.mp4 | A04 Consistency | Edge / Windows | clicking on the “ÖBB” logo at the top on the different websites | y | 2 | 2 | 3 | 3 | 2.50 | |||
| 15 | Scotty uses only a third of the window. | The whole site uses only 1/3 of the browser window. | n15-ps-pc-scotty-layout.mp4 | A08 Aesthetic and Minimalist Design | Edge / Windows | Scotty | y | 2 | 3 | 3 | 2 | 2.50 | |||
| 16 | Duplicated item names | The menu under “upper Austria” has two times the item “Airport Bus Linz” and the second one is a link to another site. | n16-ps-pc-wrong-item-name.mp4 | A05 Error Prevention | Edge / Windows | Regional offers - upper Austria | y | 2 | 2 | 3 | 3 | 2.50 | |||
| 17 | Radticket - error message | displayed error message is not suggesting a solution to the problem | n17-nk-pc-errormessage-bicycles-radticket.mp4 | A09 Good Error Messages | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa | 'Ticket store - Einfach-Raus-Radticket - "Who is Going" - add bicycles - click on "next"' | y | 2 | 2 | 3 | 3 | 2.50 | |||
| 18 | wrong link - station information | Station information link on homepage shows a page about the ÖBB app for smartphone | n18-nk-pc-station-information-link.mp4n18-nk-mobile-station-information-link.mp4 | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa ; Firefox 80.1.2/ Android 8.0.0 | Homepage - station information | y | 2 | 3 | 3 | 2 | 2.50 | ||||
| 19 | Error Message - Airport | Error message is displayed when trying to look up connection to airport in a certain way. This message shows up from now on on different kinds of input until it is closed and opened again. | n19-nk-pc-airport-error-message.mp4 | A02 Speak the users language / A09 Good error messages | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa | 'Homepage - regional offers - styria - airport shuttle- select station and change time - search connection (https://fahrplan.oebb.at/bin/query.exe/en?to=Graz%20Airport)' | y | 3 | 2 | 2 | 3 | 2.50 | |||
| 20 | Payment methods - link error | Link to payment methods is not working | n20-nk-mobile-payment-methods-error.mp4 | Firefox 80.1.2/ Android 8.0.0 | On the ticket page (tickets.oebb.at) open the menu (top left) - click on "Payment methods" | y | y | 2 | 3 | 2 | 3 | 2.50 | |||
| 21 | Refresh Site | 'Update this site and you will end up on start page from book ticket ' | n21-tt-mobile-refresh-site.mp4 | A03 Reversible Actions | Android Browser | Book Ticket/selecet ticket /go to your basket/click the link under "total amount" and refresh the site | y | y | 2 | 3 | 3 | 2 | 2.50 | ||
| 22 | Facts on the bottom of the site | Almost every site has the same two facts - which is not necessary. | n22-ps-pc-same-paragraphs.mp4 | A08 Aesthetic and Minimalist Design | Edge / Windows | Almost on every site at the bottom | y | y | 2 | 3 | 2 | 2 | 2.25 | ||
| 23 | Green checkmarks and red dots | For a list sometimes a green checkmark is used and sometimes a red dot. In some cases it is not really clear if something is allowed or not. (Example ÖBB Österreichcard/ Door-to-door luggage/ ÖBB lounge) | n23-ps-pc-green-checkmarks-red-dots.mp4 | A02 Speak the Users’ Language | Edge / Windows | - | y | 2 | 3 | 2 | 2 | 2.25 | |||
| 24 | Tickets website is overloaded | On the website where you can book a ticket is too much stuff on the screen. | n24-ps-mobile-overloaded-page.mp4 | A08 Aesthetic and Minimalist Design | Chrome/ Android | clicking on book tickets | y | 1 | 3 | 2 | 3 | 2.25 | |||
| 25 | Unnecessary home button | If you click on the home button the new page has only a button to go back to the ticket page and a menu button. The menu button should be directly on the ticket page. | n25-ps-mobile-unnecessary-home-button.mp4 | A08 Aesthetic and Minimalist Design | Chrome/ Android | clicking on book tickets- home button | y | y | y | 1 | 4 | 2 | 2 | 2.25 | |
| 26 | Menu item doesn’t work | In the drop-down menu “travel planning & services” under “After your trip” is an item “Praise criticism & feedback” which does currently not work. | n26-ps-pc-no-link.mp4 | - | Edge / Windows | Travel planning & services - after your trip - Praise criticism & feedback | y | y | 2 | 2 | 3 | 2 | 2.25 | ||
| 27 | Error – Airport preselected | 'Error message is displayed when trying to look up connection to airport in a certain way. This message shows up from now on on different kinds of input until it is closed and opened again.' | n27-nk-pc-airport-error.mp4 | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa | 'Homepage - regional offers - styria - airport shuttle - select station and change time - search connection (https://fahrplan.oebb.at/bin/query.exe/en?to=Graz%20Airport)' | y | 3 | 2 | 2 | 2 | 2.25 | ||||
| 28 | Error Message (Search Connection) | 'When entering the same location in the route planer there will be an error message. The error message might seem confusing because of the specifc terms and error codes used.' | n28-mw-pc-confusing-error-message.mp4 | A09Good Error Messages | Brave / Windows 8.1 Pro | Start Page - Search Connection - enter the same location in the route | y | 3 | 2 | 2 | 2 | 2.25 | |||
| 29 | English interface with German names | If you want to enter you entrance and end station German names are suggested. (Would be okay if the name of the Station is German but you enter the city name) | n29-ps-pc-suggestion-in-german.mp4 | A04 Consistency | Edge / Windows | Home page | y | 2 | 1 | 3 | 2 | 2.00 | |||
| 30 | Homepage empty | The home icon of the ticket store points to a site that seems to be too empty | n30-nk-mobile-home-ticket-store.mp4 | A08 Aesthetic and minimalist design [Nearly no visual information at all] | Firefox 80.1.2/ Android 8.0.0 | Somewhere in a buying process on the ticket site (tickets.oebb.at) click the home icon. | y | 2 | 3 | 1 | 2 | 2.00 | |||
| 31 | German paragraphs in English version | Sometimes on the web site German paragraphs are not translated. Some sites are entirely in German. | n31-ps-pc-only-german.mp4 | A04 Consistency | Edge / Windows | Accessibility (at the bottom) | y | 2 | 1 | 2 | 2 | 1.75 | |||
| 32 | Drop down menu | After hovering over the menu bar the drop-down items are shown. If you click to fast the general website is instead shown. General if you move over the drop-down menu it is often incorrect. | n32-ps-pc-drop-down-menu.mp4 | A05 Error Prevention | Edge / Windows | Hovering over the drop-down menu at the top of the page | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 33 | Inconsistency with S-Bahn | Each region has the S-Bahn item only “Burgenland” and “Lower Austria” not. | n33-ps-pc-S-Bahn-inconsistency.mp4 | A04 Consistency | Edge / Windows | Regional offers | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 34 | Some items in the menu are no links | On the menu are some items which only have another drop down menu and not directly a page. | n34-ps-mobile-missing-links.mp4 | A04 Consistency | Chrome/ Android | Menu | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 35 | Link to German version | If you click on the bottom on the logo “ÖBB” you are send to the German website. | n35-ps-mobile-logo-german-link.mp4 | A04 Consistency | Chrome/ Android | - | y | 2 | 1 | 2 | 2 | 1.75 | |||
| 36 | no pre-selected ticket-type. | 'If on a specific information site regarding a ticket-type this type is not selected when continuing by "Book ticket now" button' | n36-nk-pc-book-ticket-selection.mp4 | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa | specific information site regarding a ticket-type -"Book ticket now" button | y | 2 | 1 | 2 | 2 | 1.75 | ||||
| 37 | Additional ticket types look like advertisement | 'On the phone advertisements at the bottom of a website are common. Especially the white box smaller than the size to the page with the phrasing "you might also be interested in" reminds on an ad' | n37-nk-mobile-additional-options-advertisement.mp4 | A04 Consistency | Firefox 80.1.2/ Android 8.0.0 | homepage of the ticket site (tickets.oebb.at) - open top left menu | y | 3 | 1 | 1 | 2 | 1.75 | |||
| 38 | Inconsistent text and field sizes | On the start site of scotty there are fields with different | n38-nk-mobile-inconsistent-size.mp4 | A04 Consistency [Use size to mark importance] | Firefox 80.1.2/ Android 8.0.0 | Homepage - click on "search connection" | y | y | 2 | 2 | 2 | 1 | 1.75 | ||
| 39 | Back button | Back button does not lead back to site previously visited | n39-nk-mobile-back-button.mp4 | A03 Reversible action | Firefox 80.1.2/ Android 8.0.0 | Homepage - regional ticket - "book ticket now" - back button | y | y | 2 | 2 | 2 | 1 | 1.75 | ||
| 40 | Reservation | Field becomes too large if you click on it | n40-tt-mobile-reservation-field.mp4 | A07 Flexibility and Efficiency of Use | Android Browser | Book Ticket/select a ticket/journey preview/reservation field | y | 2 | 2 | 2 | 1 | 1.75 | |||
| 41 | Add Person | Add or edit a person is not easy to find | n41-tt-mobile-hire-person.mp4n41-tt-pc-hire-person.mp4 | A06 Recognition Rather Than Recall | Android Browser | Book Ticket/select ticket | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 42 | Discount Feature not easily found | 'To add discount to your purchase in the ticket shop you have to click the "change amount of tickets"-button.' | n42-mw-pc-discount-feature.mp4 | A06 Recognition Rather Than Recall | Brave / Windows 8.1 Pro | Starting Page - Book Ticket(Button) - "change amount of tickets" | y | 1 | 3 | 1 | 2 | 1.75 | |||
| 43 | Booking and searching connection have a complete other layout | If you want to book a ticket or search something you have to go on a different website with another layout. | n43-ps-pc-different-layouts.mp4 | A04 Consistency | Edge / Windows | Home page - Scotty and Tickets and Services | y | 1 | 2 | 2 | 1 | 1.50 | |||
| 44 | Tickets and Services | The site to book tickets uses only half of the window. | n44-ps-pc-tickets-layout.mp4 | A08 Aesthetic and Minimalist Design | Edge / Windows | Homepage - Book ticket | y | 1 | 2 | 2 | 1 | 1.50 | |||
| 45 | Scotty map layout | The Scotty page with the map uses only the half of the screen. | n45-ps-mobile-scotty-layout.mp4 | - | Chrome/ Android | clicking on “Real-time infos on scotty.oebb…” | y | y | y | 1 | 2 | 1 | 2 | 1.50 | |
| 46 | Traffic Information | On the traffic information site there is no indicator of how many problems there are in each category | n46-nk-pc-traffic-information.mp4 | Mozilla Firefox 75.0/ Linux Mint 19.1 tessa | Homepage - route information | y | 1 | 2 | 1 | 2 | 1.50 | ||||
| 47 | Scotty links to german ticket store version | Link points to german version of the ticket store | n47-nk-mobile-link-german.mp4 | Firefox 80.1.2/ Android 8.0.0 | On Scotty look up a connection - click on "ticket and prices" | y | 2 | 1 | 1 | 2 | 1.50 | ||||
| 48 | Barrier-free | Missing functions like read-aloud | n48-tt-mobile-read-aloud-size.mp4 | A10 Help and Documentation | Android Browser | none | y | 2 | 1 | 1 | 2 | 1.50 | |||
| 49 | Fares-links not highlighted or underlined | 'Links on the "Fares" Site aren''t highlighted or underlined.' | n49-mw-pc-fares-links-not-highlighted.mp4 | A04 Consistency | Brave / Windows 8.1 Pro | Starting Page - Scroll down - Fares | y | 1 | 2 | 1 | 2 | 1.50 | |||
| 50 | Search connection slow [also on mobile] | After clicking on “book ticket” it takes sometimes a few seconds to open the new window. | n50-ps-mobile-loading-time.mp4, n50-ps-pc-loading-time.mp4 | - | Edge / Windows | Homepage - Book ticket | y | 2 | 1 | 1 | 1 | 1.25 | |||
| 51 | Link to German version | If you click on “online” on the “All options to get your ticket“ it takes you to the German version of the website. | n51-ps-pc-german-link.mp4 | - | Edge / Windows | Tickets & customer cards - “All options to get your ticket” | y | y | y | 1 | 1 | 1 | 2 | 1.25 | |
| 52 | Arrow style | The arrow which opens/close an area look like the arrows to go to a new page (links). | n52-ps-mobile-arrows.mp4 | A05 Error Prevention | Chrome/ Android | - | y | y | 1 | 1 | 1 | 2 | 1.25 | ||
| 53 | Overlapping area | The top areas are overlapping each other if you want to buy some tickets. | n53-ps-mobile-overlapping-areas.mp4 | A08 Aesthetic and Minimalist Design | Chrome/ Android | clicking on book tickets- select stations | y | 1 | 2 | 2 | 0 | 1.25 | |||
| 54 | Table bad overview | Confusing Table | n54-tt-mobile-table-overview.mp4 | A07 Flexibility and Efficiency of Use | Android Browser | Planned construction work | y | 1 | 1 | 1 | 2 | 1.25 | |||
| 55 | Too much on Homepage | Homepage looks overloaded | n55-tt-pc-homepage-too-much.mp4 | A08 Aesthetic and Minimalist Design | Chrome 84.0.4147.135 | homepage | y | y | y | 1 | 1 | 1 | 2 | 1.25 | |
| 56 | Too Many Images and Textboxes | 'Some parts of the website focus a bit too much about their appearance while loosing efficiency. Users might get confused by the overflowing information.' | n56-mw-pc-confusing-too-many-images.mp4 | A08 Aesthetic and Minimalist Design | Brave / Windows 8.1 Pro | Anywhere | y | 1 | 1 | 1 | 2 | 1.25 | |||
| 57 | Punctuality website only in german | The website is only available in german. | n57-mw-pc-punctuality-only-german.mp4 | A04 Consistency | Brave / Windows 8.1 Pro | Starting Page - Punctunality | y | 1 | 1 | 1 | 2 | 1.25 | |||
| 58 | Confusing Search Feature | 'The Search feature isn''t sorted and has a weird design and to many shown entries at once.' | n58-mw-pc-search.mp4 | A08 Aesthetic and Minimalist Design | Brave / Windows 8.1 Pro | Starting Page - Search | y | 1 | 1 | 2 | 1 | 1.25 | |||
| 59 | Unimportant Advertisement | The Website is filled with unnecessary advertisement of either their own business or related ones. | n59-mw-pc-unimportant-advertisement.mp4 | A08 Aesthetic and Minimalist Design | Brave / Windows 8.1 Pro | Everywhere | y | 1 | 2 | 1 | 1 | 1.25 | |||
| 60 | Digital pictures with text are only in German | If a picture has only a text or statistic it is always in German. (Not including real pictures from Austria) | n60-ps-pc-picture-in-german.mp4 | A04 Consistency | Edge / Windows | click on “information now” - “Latest news” | y | y | y | 1 | 1 | 1 | 1 | 1.00 | |
| 61 | Scotty has German in the title | The title of Scotty has a slogan in German. | n61-ps-pc-german-slogen.mp4 | - | Edge / Windows | Scotty page | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 62 | Drop-Down menu in front of map settings | If you go on the Scotty web site under Train radar two drop-down menu are open and in front of the settings of the map. | n62-ps-pc-menu-overlap-settings.mp4 | A08 Aesthetic and Minimalist Design | Edge / Windows | Scotty - Train radar | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 63 | Tickets and Services sometimes only in German | The payment methods and privacy statement are only in German available. | n63-ps-pc-tickets-links-in-german.mp4 | A04 Consistency | Edge / Windows | Homepage - Book ticket- click at the top right to open menu | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 64 | Tickets and Services search takes some time | After entering the stations the search for the trains take a few seconds. (Is shown with a circle) | n64-ps-pc-load-train-list.mp4 | - | Edge / Windows | Homepage - Book ticket- enters cities and click on the first button below | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 65 | Clock disappear | On the home page of Tickets and Services is a clock in the background which is hard to see and it disappear if you search something. | n65-ps-pc-clock.mp4 | A04 Consistency | Edge / Windows | Homepage - Book ticket | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 66 | Same menu item twice | The Einfach-Raus-Ticket appears under “Student and groups” and Leisure & Vacation”. | n66-ps-pc-same-link-different-menu.mp4 | - | Edge / Windows | Tickets & customer cards - drop-down menu | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 67 | ÖBB nightjet link | Under “our trains” are all trains of the ÖBB and if you click on them you get to an information site about that train. If you click on nightjet you get on the ÖBB nightjet website with no information about the train. | n67-ps-pc-train-information.mp4 | A04 Consistency | Edge / Windows | Travel planning & services - in the train- our trains | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 68 | No drop down menu | The last item in “travel planning & services” is ÖBB customer service which has no extra drop down menu but it would be more constantly with one. | n68-ps-pc-customer-service-menu.mp4 | A04 Consistency | Edge / Windows | Travel planning & services - in the menu | y | 1 | 2 | 1 | 0 | 1.00 | |||
| 69 | ÖBB customer service contact form | If you want to send a contact form you need to select a category but it looks like a list rather than something where you can click on. | n69-ps-pc-contact-form.mp4 | A02 Speak the Users’ Language | Edge / Windows | ÖBB customer service- Complaint praise &idea | y | 0 | 2 | 1 | 1 | 1.00 | |||
| 70 | Only one item in drop-down menu | “Burgenland” has a drop-down menu with only one item. | n70-ps-pc-only-one-item.mp4 | A04 Consistency | Edge / Windows | Regional offers - hover over Burgenland | y | 0 | 2 | 2 | 0 | 1.00 | |||
| 71 | Crowded Site | 'On the homepage of subdomain scotty (https://fahrplan.oebb.at/bin/query.exe/) the presentation of information seems off' | n71-nk-mobile-crowded-page.mp4 | A08 Aesthetic and minimalist design | Firefox 80.1.2/ Android 8.0.0 | Homepage - click on "search connection" | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 72 | Abbrevations | Not all abbreviations are explained | n72-tt-mobile-abbreviations.mp4 | A02 Speak the Users’ Language | Android Browser | ' menu/tickets and customer card/students and groups/region and city tickets' | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 73 | Several People | Long list if you add person | n73-tt-pc-list-person.mp4 | A08 Aesthetic and Minimalist Design | Chrome 84.0.4147.135 | book ticket/change person | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 74 | "Go back" Button not highlighted | The "Go back" button in the ticket shop to go back to the starting page is kind of hidden and not easily found. | n74-mw-pc-go-back-button-hidden.mp4 | A06 Recognition Rather Than Recall | Brave / Windows 8.1 Pro | Starting Page - Book Ticket(Button) - Menu Bar - Go back to oebb.at | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 75 | Poor Menu Design | 'Menu text isn''t fully readable and the drop downs aren''t easily clickable.' | n75-mw-mobile-menu-poor-design.mp4 | A08 Aesthetic and Minimalist Design | Chrome / Android 8.0.0 | Starting Page - Menu | y | 1 | 1 | 1 | 1 | 1.00 | |||
| 76 | Maps are in German | Every map which is used is in German. | n76-ps-pc-german-maps.mp4 | - | Edge / Windows | Scotty - Train radar | y | 1 | 0 | 1 | 1 | 0.75 | |||
| 77 | Duplicated pictures | “S-Bahn Austria” uses for different regional S-Bahn lines the same picture side by side. | n77-ps-pc-duplicated-picture.mp4 | - | Edge / Windows | Regional offers- S-Bahn | y | 1 | 1 | 0 | 1 | 0.75 | |||
| 78 | Printout Timetable | No preview available | n78-tt-mobile-timetable-printout.mp4 | A07 Flexibility and Efficiency of Use | Android Browser | Book Ticket /select ticket /add to basket/ link under "total amount" | y | 0 | 1 | 1 | 1 | 0.75 | |||
| 79 | Star symbol | The star symbol to determine the mandatory fields is not really clear a normal asterisk would be better. | n79-ps-pc-asterisk.mp4 | A02 Speak the Users’ Language | Edge / Windows | ÖBB customer service- Complaint praise &idea | y | 1 | 1 | 0 | 0 | 0.50 | |||
| 80 | Drop down menu with “+” | To open the drop down menu you need to click on the “+”-button and then on an arrow. | n80-ps-mobile-inconsistency-menu.mp4 | A04 Consistency | Chrome/ Android | Menu on the top left. | y | y | 0 | 1 | 0 | 1 | 0.50 | ||
| 81 | Wrong link name | On the site ÖBB Plus is a link which says “tickets.oebb.at/de/shop” (to the German website) but it is actually to “tickets.oebb.at/en/shop” (to the (correct) English website). | n81-ps-pc-wrong-link-name.mp4 | - | Edge / Windows | Tickets & customer cards- leisure & vacation - öbb plus | y | 0 | 0 | 0 | 1 | 0.25 | |||
| 82 | Point in a link | On the site “Requests for fare payment” there is a link “online ticket or using your mobile phone” it looks like two different links with a dot between “using” and “your” because of the underline. | n82-ps-pc-underline.mp4 | A05 Error Prevention | Edge / Windows | Travel planning & services - “Requests for fare payment” | y | 1 | 0 | 0 | 0 | 0.25 | |||
| 83 | Chance to contrast | Nothing changes by contrast | n83-tt-pc-change-contrast.mp4 | A07 Flexibility and Efficiency of Use | Chrome 84.0.4147.135 | homepage | y | 0 | 0 | 1 | 0 | 0.25 | |||
| 84 | No Breadcrumps | No breadcrumps to parent page | n84-tt-pc-no-breadcrumps.mp4 | A03 Reversible Actions | Chrome 84.0.4147.135 | on every child page | y | 0 | 0 | 1 | 0 | 0.25 | |||
| 85 | Contact Information Not Easily Found | 'The contact information isn''t easily found. Users have to have previous knowledge to instantly know that the contact information is usually at the bottom of the website.' | n85-mw-pc-contact-information-not-easily-found.mp4 | A06 Recognition Rather Than Recall | Brave / Windows 8.1 Pro | Starting Page | y | 0 | 0 | 0 | 1 | 0.25 | |||
| Code | Evaluator |
|---|---|
| NK | Nicole Krämer |
| PS | Philip Samer |
| TT | Tanja Tatschl |
| MW | Marcel Weiss |
| y | Found by this evaluator |
References
- [Nie1990]
- Jakob Nielsen and Rolf Molich;
Heuristic Evaluation of User Interfaces;
Proc. Conference on Human Factors in Computing Systems (CHI’90).
ACM. Seattle, Washington, USA, Apr 1990, pages 249–256.
doi:10.1145/97243.97281 - [Nie1994]
- Jakob Nielsen;
Enhancing the Exploratory Power of Usability Heuristics;
Proc. Conference on Human Factors in Computing Systems (CHI’94).
ACM. Boston, Massachusetts, USA, Apr 1994, pages 152–158.
doi:10.1145/191666.191729 - [Nie1995]
- Jakob Nielsen;
How to Conduct a Heuristic Evaluation;
01 Jan 1995.
https://nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/ - [Phil2017]
- Miklos Philips;
Heuristic Analysis for UX – How to Run a Usability Evaluation;
Visited 2020.09.09.
https://www.toptal.com/designers/usability/usability-analysis-how-to-run-a-heuristic-evaluation
A Materials
The following materials were used or created by the evaluation team.
A.1 Heuristics
The evaluators used the Andrews General Usability Heuristics 2013
found in file:
heuristics.pdf.
A.2 Individual Evaluation Logs
The individual log files for each evaluator are provided below:
- Nicole Krämer:
log-nk-mobile.txtandlog-nk-pc.txt. - Philip Samer:
log-ps-mobile.txtandlog-ps-pc.txt. - Tanja Tatschl:
log-tt-mobile.txtandlog-tt-pc.txt. - Marcel Weiss:
log-mw-mobile.txtandlog-mw-pc.txt.
A.3 Spreadsheet
The filled-out, sorted spreadsheet of all problems and positives
including their ratings is helist.xlsx.