Heuristic Evaluation Report
Human-Computer Interaction SS 2023
Group G5-09
Daniel Scharf
Viola Brod
Alfred Sebernig
Mario Knezevic
Heuristic Evaluation of the Web Site
kastner-oehler.at
Report of 28th Apr 2023
1 Executive Summary
With an ever-increasing functionality, one must ask oneself at some point whether those functions are still understandable and useful for the user. That’s why, this report shows all our findings of the website “kastner-oehler.at” from the heuristic evaluation. The procedure of the evaluation began with an individual evaluation from every evaluation team member. Here every team member did an individual heuristic evaluation of the website and tried to discover negative and positive findings. During this process everyone created short video clips, which show their findings. Afterwards the complete team come together to discuss their findings and rank then by severity and positivity. This ranking is now the starting point for this heuristic evaluation report.
One of the most problematic functionalities of the website is the filter option, because in our evaluation different negative findings were found in the context of filtering. The most severe finding is, that the filter options do not load properly every time when trying to add filters. This error has the consequence, that the user is not able to add filters and therefore the complete filter functionality isn’t usable anymore. Other filter related problems are that it is not possible to apply multiple filter options at once with one click on filtering and therefore it is very cumbersome to add multiple filters and that additional filters change already existing filters.
In contrast to the filter related problems, there is also a positive finding for the filter option. During the scrolling process, the possibility to add and change filters stays on the top of the screen and therefore it is not necessary to completely scroll up to the top of the page to make these actions.
Another negative finding is related to the main search bar of the website. It is not possible to find common categories, like for example trousers for men, because you can only search for specific items and brands. This is very annoying for users, which just want to browse through the website and do not yet have something specific in mind.
One more source for multiple problems, are wrong or missing error messages. This can be seen especially during the login- and registration process. One instance of this is, when entering wrong login credentials, the system does not give an appropriate error message and just assumes that you do not want to login, but to register. In addition, there are also wrong or missing error messages in relation to the birthyear and postal-code fields during the registration process.
On the positive site, there are multiple good support possibilities for users on the website, like an extensive FAQ page, a minimalistic support-chat window, or a size table for clothing size determination.
These now mentioned points are just a quick overview for the complete heuristic evaluation. The complete evaluation with all negative and positive findings can be found in the following chapters.
2 Introduction
The website kastner-oehler.at, which we will be evaluating is the webstore of the in Graz-based store Kastner-Öhler, which has several shops in Austria. This store offers clothes and accessories for women, men and children. It also provides a Beauty and Home & Living area, as well as toys for children. On the website you will find more expensive brands, than less expensive brands. The whole purpose of the evaluation is to identify as many potential usability problems as possible. We will be evaluating the whole website and deliver input for the next design phase.
3 Evaluation Procedure
This section describes the procedure used in the heuristic evaluation.
3.1 Evaluation Methodology
Heuristic evaluation is an inspection technique, where usually several people are evaluating a user interface, like a website. For the evaluation a set of heuristic principles, which is also called Heuristics, are used to determine the problems as well as the positives of a website. The evaluation can be done by one single person, but since one person alone can not find all the problems, multiple people are included. Therefore, the interface is viewed from multiple different angles and the result also contains a broader set of findings. The evaluation method consists of basically three big tasks. One of them being the planning. It is essential to set a clear goal which should be achieved by the evaluation. Furthermore, the potential user group as well as the heuristics should be defined. In the next step the evaluators inspect the interface and write down problems or positive findings based on the previous defined heuristics. In the end the evaluation team reviews all findings together. For this all findings get accumulated, rated in terms of severity and presented in form of a report. [Muniz2016]
Heuristic evaluation is method used to measure the usability of a user interface. The evaluators are using already established heuristics, like in our case, the “Andrews General Usability Heuristics”, will be used. Which is a slightly adapted version of Nielsen’s revised set of then usability heuristics. This document contains 10 heuristics, or principles which are basically a practical way to solve problems or make decisions. Professional evaluators will make use of the given heuristics to systematically evaluate the, in our case, usability of an interface, like a website. As an example, the first heuristic in the mentioned document is the feedback. Which describes how and how fast, the website/user interface should react. In this case, there are requirements like for example, the reaction time of the website. For a respond that will take not longer than 10 seconds, display a busy cursor, for more than 10 seconds show the user a progress bar. With the help of the given heuristics, the evaluators will create a list of their found problems and describe them, e.g. as pictures or screen recordings, as well as a textual description and steps to reproduce the found problem. [Inte2023]
3.2 User Profiles
Since Kastner-oehler.at is mainly an online store for fashion and clothing, it’s target group are people who are interested in buying new clothes and have a certain degree of fashion understanding. The product catalog of this online store includes clothing for children, men, women and additionally extra sections for certain groups such as the “Infected”-section, which mainly targets teenagers. Furthermore, the online store has sections for other products such as beauty products or things in the field of home and living to offer. Therefore, the site is trying to attract a very broad field of potential customers, which in general are very hard to group. One way of grouping these users, is after their own fashion understanding respectively their fashion sense and their reasoning for the site visit. People which are into fashion and beauty are more interested in finding new trendy clothes and products and are more selective in their choosing and therefore need more options of selection and filtering to get to the things which they have in mind. In contrast users, which are not really into fashion, want to get multiple recommendations of standard things so that they can search and compare which of the recommendations suits them best. But for all of them the main task is the searching and buying of products.
3.3 Extent of the Evaluation
In the context of this practical we will evaluate the complete website Kastner-oehler.at.
3.4 Evaluators and Evaluation Environments
The evaluators will use the evaluation environments shown in Table 1.
| Evaluator | Daniel Scharf (DS) | Alfred Sebernig (AS) | Mario Knezevic (MK) | Viola Brod (VB) |
|---|---|---|---|---|
| Age | 22 | 24 | 22 | 21 |
| Gender | male | male | male | female |
| Device | Samsung Galaxy A51 | HUAWEI MateBook D 14 | ASUS VivoBook S14 S433UA-AM223 | iPad Air 4th Generation |
| OS and Version | Android 10 | Windows 10 Home 22H2 | Linux Mint 20.2 Cinnamon | iPadOS 16.3.1 |
| Screen Size | 6.5″ | 14″ | 14″ | 10.9″ |
| Screen Resolution | 1080×2400 | 1080x1920 | 1080x1920 | 2360×1640 |
| Web Browser | Samsung Internet Browser 20.0.3.10 | Google Chrome 111.0.5563.111 (64-bit) | Firefox 111.0 64Bit | Safari 16.3 |
| Ad Blocker | AdGuard 2.7.0 | none | AdblockPlus | none |
| Internet Connection | WiFi | WiFi | WiFi | WiFi |
| Download Speed | 7.8 mbps | 80 mbps | 75 mbps | 280 mbps |
| Screen Recording Software | Samsung Bildschirmrecorder | OBS Studio 29.0.2 | OBS Studio 29.0.2 | iOS Screen Recording |
| Recording Resolution | 1080x2400 | 1920x1080 | 1920x1080 | 1668×2224 |
| Planned Date of Evaluation | 2023-04-14 | 2023-04-14 | 2023-04-15 | 2023-04-15 |
Viola and Daniel will copy their video clips captured from her iPad and his phone to a Windows PC and use LosslessCut 3.53 on Windows 11. Daniel and Mario will use Kdenlive 22.08.1 to cut and edit their videos. Alfred will use Sony Vegas 17 to cut and edit his videos.
Since Viola's iPad has a native resolution larger than FullHD the video clips will be transcoded to 1080x1920 using Handbrake 1.6.1 on a Windows PC. Daniel will also use HandBrake 1.6.1 to transcode his videos, because the resolution of his phone is too high. Alfred and Mario don't have to transcode anything, as they will already record the videos in the right resolution.
4 Results of the Evaluation
This section describes the results of the heuristic evaluation.
4.1 Top Three Positive Findings
The top three positive findings according to their average (mean) positivity ratings are described in more detail below. The positivity rating scheme used to rank the positive findings is shown in Table 2.
| Positivity | Meaning |
|---|---|
| 4 | Extremely Positive |
| 3 | Major Positive |
| 2 | Minor Positive |
| 1 | Cosmetic Positive |
| 0 | Not a Positive |
P01. Filter options stays while scrolling
| Title: | Filter options stays while scrolling |
|---|---|
| Description: | The option "Filtern&Sortieren", is used for selecting filters for the product list to achieve better search results. The option is on top of the search results, but during the scrolling process it stays on the screen and does not disappear when you scroll away from the top. So, you can easily adapt the filtering even when you are already scrolled down and search through the products. |
| Video Clip(s): | p01-ds-pos01-filter-option-stays.mp4 |
| Location (How Reproducible?): | Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Alle Artikel" → Scroll down and you see, that the filtering option follows the scrolling |
| Mean Positivity: | 3.25 |
One evaluator realized that the filter options stay while scrolling. So, it is easy to adapt the filters during browsing through different sites. Especially, for older people it is now easier filtering their shopping experience. In Figure 1 you can see the filter options while scrolling.
P02. Extensive FAQ
| Title: | Extensive FAQ |
|---|---|
| Description: | Very clear design, you can also find a phone number for support in there in a big font size. Underneath you got all the questions grouped into different categories |
| Video Clip(s): | p02-as-pos02-faq.mp4 |
| Location (How Reproducible?): | Above Search bar click "Hilfe & FAQ" |
| Mean Positivity: | 3.25 |
Two evaluators noticed the FAQ section with a wide-ranging selection of questions and instructions. This makes it very easy for the customers to answer their own questions without struggling to find the answer. In 2 you can see the FAQ section.
P03. Drop down menu
| Title: | Drop down menu |
|---|---|
| Description: | It is easy to navigate through the different categories because of the drop down menus. |
| Video Clip(s): | p03-vb-pos03-drop-down-menu.mp4 |
| Location (How Reproducible?): | Select different categories on the drop down menu |
| Mean Positivity: | 2.25 |
With this big drop down menu you can select each filter on your own and you can see at a glance e.g. which different types of accessories the category accessories has. In 3 you can see that there are many options to choose a head.
4.2 List of All Positive Findings
Table 3 shows a list of all the positive findings which emerged from the evaluation, sorted in descreasing order of average (mean) positivity, i.e. the most positive are at the top of the table. The name codes assigned to each evaluator are shown in Table 4.
| No. | Title | Description | Video Clip(s) | Location (How Reproducible?) | Found By | Positivity | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DS | AS | MK | VB | DS | AS | MK | VB | Mean | |||||
| 1 | Filter options stays while scrolling | The option "Filtern&Sortieren", is used for selecting filters for the product list to achieve better search results. The option is on top of the search results, but during the scrolling process it stays on the screen and does not disappear when you scroll away from the top. So, you can easily adapt the filtering even when you are already scrolled down and search through the products. | p01-ds-pos01-filter-option-stays.mp4 | Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Alle Artikel" → Scroll down and you see, that the filtering option follows the scrolling | y | 4 | 3 | 3 | 3 | 3.25 | |||
| 2 | Extensive FAQ | Very clear design, you can also find a phone number for support in there in a big font size. Underneath you got all the questions grouped into different categories | p02-as-pos02-faq.mp4 | Above Search bar click "Hilfe & FAQ" | y | y | 3 | 4 | 3 | 3 | 3.25 | ||
| 3 | Drop down menu | It is easy to navigate through the different categories because of the drop down menus. | p03-vb-pos03-drop-down-menu.mp4 | Select different categories on the drop down menu | y | 2 | 2 | 3 | 2 | 2.25 | |||
| 4 | shopping cart feedback | When adding an item to the shopping cart a pop up is displayed confirming that the item was added successfully. | p04-mk-pos02-shopping-cart-feedback.mp4 | add item to shopping cart | y | 2 | 3 | 2 | 2 | 2.25 | |||
| 5 | Minmalistic support-chat window | You can just click the chat symbol on the right of every page of the website. You just need to put in your name or e-mail adress and you can start a chat. | p05-as-pos03-chat.mp4 | On the right site of the screen click the Chat Symbol | y | 2 | 3 | 2 | 2 | 2.25 | |||
| 6 | Check for out of bounds orders | If you order an item, which is only available in a limited number anymore, then the system checks if you order exceeds this number and reduces the number to the max available number. If you try to increase this number again in the shopping cart menu, you will get an error message which clarifies why this is no longer possible. | p06-vb-pos02-error-message.mp4 | Go to the main page and then to the sale section. Search for an item which is in limited stock (as of time of this logging and recording for example: COPENHAGEN Chelsea Boots CPH685) and order more as there are in stock. Afterwards go to the shopping cart menu (shopping bag icon on the right upper corner) and here you will see that only the possible number of items is in your shopping cart. If you increase the number of items here, you will get an error message and nothing will change in your order. | y | y | y | 1 | 3 | 2 | 2 | 2.00 | |
| 7 | Navigation through brands | It is very easy to search for a brand and the ones which are only available in the stores are grey and when you click on it, it shows you where to get the brand. | p07-vb-pos01-navigation-brands.mp4 | Marken | y | 2 | 2 | 2 | 2 | 2.00 | |||
| 8 | Size table for size determination | If a user is not sure, which size of clothing he needs, then there is a size table option to determine your desired size. This option is put next to every size request so that the user is able to enter this option every time when he is unsure about which size he needs. In this size table are explanations how you should measure yourself and what these measurements correlate to in clothing size. | p08-ds-pos02-size-table.mp4 | The size table can be found on the main page when you scroll down to the section "Kundenservice" and there is the option "Größentabellen" or if you click on a clothing item which is available in multiple sizes, then there is a option "Größentabelle" next to the size selector. | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 9 | Clear account page | "If you are logged in and want to check your account informations or orders, you can click on the account symbol in the upper right corner and land on your account page. You got every information that you might need on this page, which fits into one screen size. " | p09-as-pos01-account.mp4 | Login → Account Symbol | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 10 | Extra view for dressed items | In the product catalogue page, there is an extra option where the user can toggle between two different types of preview images. One is only the clothing item and the other one is a person wearing the item. This option stays next to the filtering option on top of the screen while scrolling. | p10-ds-pos03-out-of-bounds-check.mp4 | Main Page → "Menü" (three strips in the upper left corner) → "Neu" → "Alle Artikel" → Next to the filter option are two symbols, one shirt-symbol and one person symbol. These two buttons can be used to toggle between the two different preview image styles. | y | 2 | 1 | 1 | 1 | 1.25 | |||
| Code | Evaluator |
|---|---|
| DS | Daniel Scharf |
| AS | Alfred Sebernig |
| MK | Mario Knezevic |
| VB | Viola Brod |
| y | Found by this evaluator |
4.3 Top Five Problems
The top five problems according to their average (mean) severity ratings are described in more detail below. Problem number 1 is the problem (negative finding) with the highest mean severity. The severity rating scheme used to rank the problems is shown in Table 5.
| Severity | Meaning |
|---|---|
| 4 | Catastophic problem |
| 3 | Serious problem |
| 2 | Minor problem |
| 1 | Cosmetic problem |
| 0 | Not a problem |
N01. Filter Options do not load properly every time
| Title: | Filter Options do not load properly every time |
|---|---|
| Description: | The website offers an option to filter items in the product catalogue. This "Filtern&Sortieren"- option contains multiple subcategories like sort order, brand, size, etc. from which the user can choose his preferred combination. But sometimes, when opening the filter options, the filters don't load correctly and therefore if you then press on a subcategory, the selected category does not open correctly. If you open the "Filtern&Sortieren"-option and a small symbol appears for a short time which states that the filter options are loaded, then everything works. But this symbol does not appear every time and then the filter selection does not work. |
| Video Clip(s) | n01-ds-neg03-filter-dont-load.mp4 |
| Heuristic: | |
| Only When: | |
| Location (How Reproducible?): | Main Page → "Menü" (three strips in the upper left corner) → "Neu" → "Damen"→ "Alle Artikel" → "Filtern&Sortieren"-option: If the loading symbol with "Filter werden geladen" does not appear select "Sortieren" and you will notice that the sort order selection will not open correctly. If the loading symbol appears, select any sort option, press "Ergenisse anzeigen" and then open the "Filtern&Sortieren"-option and try again." |
| Mean Severity: | 3.50 |
In Figure 4 you can see, that the filter function sometimes doesn't work. If the loading symbol "Filter werden geladen" doesn't appear, then the sort order selection will not work properly.
Probably, there is an error in the handling of the filter function. We can just recommend to fix this error by looking through your implementation of the filter function. Maybe there is an error with the loading symbol or something.
N02. Can't find common categories in the search function
| Title: | Cant find common categories in the search function |
|---|---|
| Description: | If you are trying to search for a common phrase in the search bar like "Herren Hosen" you wont get any results that lead you to this category. You can only search for specific items. |
| Video Clip(s): | n02-as-neg09-search.mp4 |
| Heuristic: | A07 Flexibility and Efficiency of Use |
| Only When: | |
| Location (How Reproducible?): | Switch between the given categories |
| Mean Severity: | 3.50 |
In Figure 5 you can see, that if you are searching e.g. for "Herren Hosen" the search function just shows you the men's category and one specific brand which sells men's trousers, but it won't show you the category trousers for men.
This issue could be solved by optimizing your search function and maybe including your categories, which are already filter options, to the search function.
N03. Filter changes another filter
| Title: | Filter changes another filter |
|---|---|
| Description: | If you want to filter just for women clothes, there will be a huge amount of findings, but if you also want to filter for men clothes additionally to the women clothes, then the findings of the women clothes change to the number of findings for the men clothes. |
| Video Clip(s): | n03-vb-neg03-filter-changes-filter.mp4 |
| Heuristic: | A05 Error Prevention |
| Only When: | |
| Location (How Reproducible?): | Damen → Filter Damen → Filter Herren |
| Mean Severity: | 3.25 |
One evaluator realized, if you select the category "Frauen" the website shows you thousands of results. But if you also want to be shown the category "Herren" suddenly the amount of results for "Frauen" is the same amount as the one from "Herren". This is shown in Figure 6.
This problem can be resolved by letting the customers choose different categories to show them even if filtering by women and men is conflicting.
N04. Button doesn't work
| Title: | Button doesn't work |
|---|---|
| Description: | The button "Fragen zum Online Shop?" sometimes doesn't work, so when you try to click on it nothing happens. The site is not loading, no window pops up or whatsoever. |
| Video Clip(s): | n04-vb-neg10-button-not-working.mp4 |
| Heuristic: | A01 Feedback |
| Only When: | On iPad |
| Location (How Reproducible?): | Click on button "Fragen zum Online Shop?" |
| Mean Severity: | 3.25 |
In Figure 7 you can see that our evaluator on the iPad, had problems with the button "Fragen zum Online Shop?" because nothing happened when clicking on it. The grey circle around the button shows that our evaluator really tried clicking on it.
This issue can just be resolved by finding out why the button sometimes doesn't work. Especially on the iPad, maybe it has something to do with the mobile version.
N05. Failed login - no feedback
| Title: | Failed login - no feedback |
|---|---|
| Description: | When trying to log into your account with a wrong email adress (eg. misspelled) the site will redirect you to the registration page. So they assume that you want to create a new account with this email adress, but you get no feedback if you typed in wrong login credentials or if the email even exists. |
| Video Clip(s): | n05-as-neg07-login.mp4 |
| Heuristic: | A09 Good Error Messages |
| Only When: | On iPad |
| Location (How Reproducible?): | Login with wrong login credentials |
| Mean Severity: | 3.25 |
If you try to log in to your account with a wrong email address, the website assumes, that there doesn't exist an account with this email address (which is right) and you should registrate with this email address. There is no feedback that maybe you typed in a wrong email address. This process is shown in Figure 8.
You should implement an error message if someone wants to log into their account, but there exists no account with this email. An error message like "Wrong email or password, or registrate".
4.4 List of All Problems Found
Table 6 shows a list of all the problems found (negative findings) in the evaluation, sorted in descreasing order of average (mean) severity. The name codes assigned to each evaluator are shown in Table 7.
| No. | Title | Description | Video Clip(s) | Heuristic | Only When | Location (How Reproducible?) | Found By | Severity | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DS | AS | MK | VB | DS | AS | MK | VB | Mean | |||||||
| 1 | Filter Options do not load properly every time | The website offers an option to filter items in the product catalogue. This "Filtern&Sortieren"- option contains multiple subcategories like sort order, brand, size, etc. from which the user can choose his preferred combination. But sometimes, when opening the filter options, the filters don't load correctly and therefore if you then press on a subcategory, the selected category does not open correctly. If you open the "Filtern&Sortieren"-option and a small symbol appears for a short time which states that the filter options are loaded, then everything works. But this symbol does not appear every time and then the filter selection does not work. | n01-ds-neg03-filter-dont-load.mp4 | Main Page → "Menü" (three strips in the upper left corner) → "Neu" → "Damen"→ "Alle Artikel" → "Filtern&Sortieren"-option: If the loading symbol with "Filter werden geladen" does not appear select "Sortieren" and you will notice that the sort order selection will not open correctly. If the loading symbol appears, select any sort option, press "Ergenisse anzeigen" and then open the "Filtern&Sortieren"-option and try again." | y | 4 | 4 | 3 | 3 | 3.50 | |||||
| 2 | Cant find common categories in the search function | If you are trying to search for a common phrase in the search bar like "Herren Hosen" you wont get any results that lead you to this category. You can only search for specific items. | n02-as-neg09-search.mp4 | A07 Flexibility and Efficiency of use | Switch between the given Categories | y | 4 | 4 | 3 | 3 | 3.50 | ||||
| 3 | Filter changes another filter | If you want to filter just for women clothes, there will be a huge amount of findings, but if you also want to filter for men clothes additionally to the women clothes, then the findings of the women clothes change to the number of findings for the men clothes. | n03-vb-neg03-filter-changes-filter.mp4 | A05 Error Prevention | Damen → Filter Damen → Filter Herren | y | 3 | 4 | 3 | 3 | 3.25 | ||||
| 4 | Button doesn't work | The button "Fragen zum Online Shop?" sometimes doesn't work, so when you try to click on it nothing happens. The site is not loading, no window pops up or whatsoever. | n04-vb-neg10-button-not-working.mp4 | A01 Feedback | Click on button "Fragen zum Online Shop?" | y | 3 | 3 | 3 | 4 | 3.25 | ||||
| 5 | Failed login - no feedback | When trying to log into your account with a wrong email adress (eg. misspelled) the site will redirect you to the registration page. So they assume that you want to create a new account with this email adress, but you get no feedback if you typed in wrong login credentials or if the email even exists. | n05-as-neg07-login.mp4 | A09 Good Error Messages | Login with wrong login credentials | y | 2 | 3 | 4 | 3 | 3.00 | ||||
| 6 | Filter - Cant apply multiple options at once | If you want to filter for a specific brand AND a specific size, you can specify those options but if you click on the filter button it will only apply the filters of the filter button you actually clicked. Every filter category has its own filter button | n06-vb-neg01-confirm-filter.mp4 | A07 Flexibility and Efficiency of use | Go into some category and mark at least two filter options (eg. size and brand) → click on filter | y | y | y | 3 | 3 | 3 | 3 | 3.00 | ||
| 7 | bad visibility on map | When searching the nearest location to the one provided by the user, and then clicking on the locations, the popups overlap eachother. | n07-mk-neg06-map.mp4 | scroll down to the end → Standorte / Häuser → enter zip code 8010 → open info of locations | y | 3 | 3 | 3 | 3 | 3.00 | |||||
| 8 | Categories blocking the screen | If you are on above the categories with your cursor and you want to scroll down to the products a window with more specific categories will pop up and block the screen which can only be removed by moving your cursor outside of the window, even when scrolling it wont disappear. | n08-as-neg01-categorywindow.mp4 | A07 Flexibility and Efficiency of use | Place cursor on the top middle of the website (for example searchbar), then move the cursor down to the product section | y | 2 | 3 | 3 | 3 | 2.75 | ||||
| 9 | Birth year | In the account settings you can enter your birthday, but there is no age controll, so you can type 2050 as your birth year and there is no error message | n09-vb-neg04-birthyear.mp4 | A09 Good Error Messages | Account → Meine Daten Bearbeiten → Geburtsdatum | y | 2 | 3 | 2 | 4 | 2.75 | ||||
| 10 | Ad-Blocker blocks offers and personal recommendations | With Ad-Blocker enabled some of the offers and personal recommendations on the website are blocked and appear to be loading forever. These offers are not an extra advertising of a third party, but offers of kastner-oehler themself and because of this, they should not be blocked. The loading signs indicate that something will come but, nothing is going to happen. | n10-ds-neg01-block-offers-with-adblocker.mp4 n10-ds-neg01-block-offers-without-adblocker.mp4 | A01 Feedback | Ad-blocker is disabled | Go to the main page of kastner-oehler.at an scroll down a bit. Some of the offers appear to be loading endlessly because the ad-blocker is enabled. One such case for example is above the field for the newsletter subscription on the main page. | y | 3 | 2 | 3 | 2 | 2.50 | |||
| 11 | Huge Advertisement banner on almost every page | On the most categories is a huge banner with advertisement which takes up the whole page. Also very repetitive. | n11-as-neg05-banner.mp4 | A08 Aesthetic and Minimalist Design | Click on every categorie listed above | y | 2 | 3 | 2 | 3 | 2.50 | ||||
| 12 | From shopping cart back to the shop | If you click on the shopping cart the whole menu bar at the top of the page disappears, for some people it can be hard finding their way back to the original website, which is at the bottom of the page | n12-vb-neg08-shopping-cart.mp4 | A03 Reversible Actions | Warenkorb → scroll down → Zurück zum Shop | y | 2 | 2 | 3 | 3 | 2.50 | ||||
| 13 | No feedback on clicking | It is very hard to see on which word or category you just clicked because there is no feedback on the main page. | n13-vb-neg09-no-feedback.mp4 | A01 Feedback | Startseite → click through the categories | y | 2 | 3 | 2 | 3 | 2.50 | ||||
| 14 | misleading country flags | At the bottom of the website there are different country flags, visualising the countries available for international shipping. | n14-mk-neg07-country-flags.mp4 | A02 Speak the Users’ Language | scroll down to the bottom → click one of the country flags | y | 3 | 2 | 2 | 3 | 2.50 | ||||
| 15 | Jumping through product lists | If you, for example, already found an item which you like but then accidently closed the browser or just though, you will purchase it later and can't remember the name, but you remember the site in the product catalogue you have to scroll down every time to come to the next site button so that you can jump to the desired site in the catalogue. And additionally, it is only possible to jump to the next page and not directly to the desired page. | n15-ds-neg04-jumping-through-pages.mp4 | A07 Flexibility and Efficiency of Use | Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Bekleidung" → "Hemden” → "Businesshemden" → You know that the shirt you want is on page 4 of the product catalogue, so you have to scroll down completely three times to jump to the desired page. | y | 2 | 3 | 2 | 2 | 2.25 | ||||
| 16 | registration - no feedback if account was created sucessfully | When creating an account with valid informations and you click on register, you will just get redirected to the starting page without any indications if the account was created sucesfully or any further instructions | n16-as-neg06-register.mp4 | A01 Feedback | Register an account with valid information | y | 2 | 2 | 3 | 2 | 2.25 | ||||
| 17 | Different meaning of same action | It is possible to open a category (right side, where filters are located) by clicking on the name. When trying to close it again by clicking on the name the user gets redirected instead. | n17-mk-neg03-category.mp4 | A04 Consistency | Click twice on one category name | y | 2 | 3 | 2 | 2 | 2.25 | ||||
| 18 | No checks if postal code is valid | In the registration process you are asked to but in your postal code and your country, but the system does not check, if the postal code is valid in the selected country. Therefore, it is possible to enter a postal code which does not exist in the selected country. | n18-ds-neg06-no-check-postal-code.mp4 | A05 Error Prevention | Main page → Login/Registration (Symbol of a person in the upper right corner) → "Jetzt Registrieren" → For country select "Austria" and as a postal code enter a 4-digit number which starts with 0. The system will accept it although there does not exist a postal code in Austria which starts with a 0. | y | 2 | 1 | 2 | 3 | 2.00 | ||||
| 19 | Color changing symbols with different actions | When scrolling through the product list, items which are available in more than one colour have a selection of all available colours underneath. These colour-indicators have different actions in different contexts. In the looks category if you click on one of the colours, the preview picture of the clothing item changes colour without opening a new page, but if you click on one colour in a normal product catalog page, the single product page opens with the selected colour. | n19-ds-neg09-different-colour-changer.mp4 | A04 Consistency | First go to: Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Bekleidung" → "Hemden" → "Businesshemden" → Scroll down to one shirt, which is available in multiple colours → Click on one of the different colours, then the single product page opens with the selected colour. On the other hand, go to: Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Looks" → "New Neutral" → Scroll down to an item, which is available in multiple colours → Click the colour, then the preview image changes colour without opening the single product page. | y | 2 | 2 | 2 | 2 | 2.00 | ||||
| 20 | no return from pluscard to service | If the user clicks on the "PLUSCARD" picture in the service tab the user gets redirected to the PLUSCARD section but can't go back to the service section. | n20-mk-neg01-services-pluscard.mp4 | A03 Reversible Actions | Mehr K&Ö → Services → Pluscard image | y | 2 | 1 | 2 | 3 | 2.00 | ||||
| 21 | Unnecessary big size selector | In the filter options is the option "Grösse" to select the desired size, but instead of one combo box for selection or one text field with auto completion to enter the desired size, there is a large table of buttons with all different sizes. This is especially annoying when you enter a category with not only one type of clothing, because then different size systems are put together in this list of sizes and make it very hard to read, particularly on small devices like mobile phones. | n21-as-neg04-sizetable.mp4 | A08 Aesthetic and Minimalist Design | Main Page → "Menü" (three strips in the upper left corner) → "Neu" → "Alle Artikel" → "Filtern&Sortieren"-option → "Grösse" | y | y | 1 | 2 | 2 | 2 | 1.75 | |||
| 22 | No error message | When you want to add an item 10 times into the shopping cart, there is no error message, the site loads a long time and the "Warenkorb" button turns grey, but you don't get a message that the maximum amount of one item is 9. | n22-vb-neg02-no-error-message.mp4 | A09 Good Error Messages | add an item 9 times to the shopping cart → try it a 10th times and there is no error message | y | 2 | 1 | 2 | 2 | 1.75 | ||||
| 23 | Gift card paypal logo | At the gift card section you can select between different values of gift cards, but the paypal logo just appears if you select 500€ | n23-vb-neg05-giftcard.mp4 | A04 Consistency | Geschenkkarte → Geschenkkarte bestellen → Geschenkkarte → 500€ | y | 1 | 3 | 2 | 1 | 1.75 | ||||
| 24 | Underlined words are no links | On the FAQ page are some words underlined, but you cannot click on them - other underlined words are links | n24-vb-neg07-underlined-words-no-link.mp4 | A04 Consistency | Hilfe & FAQ → dringenden Fällen/konkreten Produktberatungen are no links | y | 2 | 1 | 1 | 3 | 1.75 | ||||
| 25 | image - action not clear | When clicking on the image under the "Unsere Vorteile für Sie" text, the user gets redirected to the FAQ. | n25-mk-neg09-faq-image.mp4 | A02 Speak the Users’ Language | scroll down to the "Unsere Vorteile für Sie" text→ click on the image below | y | 2 | 1 | 2 | 2 | 1.75 | ||||
| 26 | Wrong error message in registration process | In the registration process you are asked to enter your postal code (PLZ) and your city. If you only enter your postal code and leave the city empty and press enter/register, then you get an error message next to the city field which states:"Die von Ihnen eingegebene PLZ ist ungültig. Bitte überprüfen Sie die Eingabe und korrigieren Sie diese." The system does say something is wrong, but instead of saying the city is missing it says that the postal code is wrong. | n26-ds-neg05-wrong-error-city.mp4 | A09 Good Error Messages | Main page → Login/Registration (Symbol of a person in the upper right corner) → "Jetzt Registrieren" → Enter a valid postal code, leave the city field empty and press enter | y | 1 | 1 | 2 | 2 | 1.50 | ||||
| 27 | Minimalistic symbols for account. wishlist and shopping card | On the top right of the website you can find some symbols, but they are designed very minimalistic and not everyone will recognise them instantly. Even when hovering over them there wont be any indication where this symbols will lead you. You have to try them out | n27-as-neg02-symbols.mp4 | A02 Speak the Users’ Language | Visible on every page, for example the landing page | y | 1 | 1 | 2 | 2 | 1.50 | ||||
| 28 | Switching categories - no feedback | "When switching categories eg. Herren to Damen you will get barely any indication that you actually switched the category. To find out you have to scroll down to the products or check the URL " | n28-as-neg08-switching.mp4 | A01 Feedback | Switch between the given Categories | y | 2 | 1 | 2 | 1 | 1.50 | ||||
| 29 | No feedback on filter use | When trying to apply a filter at the K&Ö Momente section there is no real feedback that the filter has been applied. | n29-mk-neg04-filter-feedback.mp4 | A01 Feedback | y | 2 | 1 | 1 | 2 | 1.50 | |||||
| 30 | Two back-to-top buttons with different symbols | When scrolling down on the website two buttons appear on the bottom site of the screen. One on the bottom right site which stays even when not touching the screen with an arrow to the top and second smaller one in the middle of the bottom border which only appears when scrolling and has an arrow to the top but this time which a dash above it. During the scrolling process there are two buttons on screen which both do the same thing (jump to the top of the site) but have different symbols. They also don't appear at the same time when you reload the site. | n30-ds-neg02-two-back-to-top-buttons.mp4 | A04 Consistency | Only on mobile devices | Go to the main site of kastner-oehler.at an scroll down. After a few scrolls first the button on the right side appears and after a few more scrolls the second one in the middle appears and both have the same usage but different symbols. | y | 1 | 1 | 1 | 2 | 1.25 | |||
| 31 | Buttons not loading correctly | In the product catalogue view, there are further selection possibility for subcategories at the top of the item list. In some subcategories, if there are no further subcategories available, then there is (if currently available) the selection between new and sale items visible. If you select the sale category, only sale items get shown. If you again click on this sale button the side reloads, but during the reloading process the "new"-button gets shown again. This can lead to confusion, because the user is does not know at the beginning what now happens. | n31-ds-neg10-buttons-loading-wrong.mp4 | A01 Feedback | Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Tracht" → "Gilets" → Click on "Sale" (white button) → Click on "Sale" (now black button) again → during the reloading of the page the "Neu" button gets visible again. | y | 1 | 2 | 1 | 1 | 1.25 | ||||
| 32 | Flags just on Austrian website | At the bottom of the website are different flags of the countries to which they ship, if you click on one of the flags you are redirect to their website, there are no flags at the bottom | n32-vb-neg06-flags-on-bottom.mp4 | A04 Consistency | Startseite → scroll down → click on flag → scroll down | y | 1 | 1 | 2 | 1 | 1.25 | ||||
| 33 | email address not linked | While on the other parts of the website the email addresses are all clickable, the ones in the Presse section are not. | n33-mk-neg05-email.mp4 | A04 Consistency | y | 2 | 1 | 1 | 1 | 1.25 | |||||
| 34 | multiple same links | When trying to buy gift cards, "Guthabenabfrage" is linked three times on the same page. The other options are also linked twice. One link should be enough for all options. | n34-mk-neg10-gift-cards.mp4 | A08 Aesthetic and Minimalist Design | go to Geschenkkarte on the top | y | 2 | 1 | 1 | 1 | 1.25 | ||||
| 35 | Breadcrumbs disappear when refreshing site | Breadcrumbs (the list of pages which someone clicked through to come to the current page) are not loaded in correctly when jumping through different looks in the "Styles and Looks"- section. The breadcrumbs disappear and appear when reloading the site and jumping between looks randomly. | n35-ds-neg08-breadcrumbs-disappear.mp4 | Main page → "Menü" (three strips in the upper left corner) → "Herren" → "Looks" → "New Neutrals" → Jump between looks ("Vorheriger Look" and "Nächster Look") and watch for changes in the breadcrumbs. Often the last breadcrumb for "New and Neutral" doesn't load at all, then you have to restart the process from the main page. | y | 1 | 1 | 1 | 1 | 1.00 | |||||
| 36 | Recall previous search requests from the search bar | The website does not save previous search request for the search bar. If a person searches for something specific, there could be a chance that they are interested in this type of clothing and might want to search for it again in the future. | n36-ds-neg11-recall-search-request.mp4 | A07 Flexibility and Efficiency of Use | Go to the main page and type a search request into the search bar at the top, for example "Adidas Socken" After the search completed, go back to the main page, and enter the search bar again. There are no previous search requests as recommendations for searching. | y | 1 | 1 | 1 | 1 | 1.00 | ||||
| 37 | Mixed up german and english terms | On the top of the page at the categories (Herren, Damen, Kinder...) you can find german and english terms. Since the website is only available in german language it can be hard for someone to find the right items that doesnt speak english | n37-as-neg03-mixedterms.mp4 | A04 Consistency | Visible on every page, for example the landing page | y | 1 | 1 | 1 | 1 | 1.00 | ||||
| 38 | misleading indication at reviews | When hovering over the displayed reviews there is some kind of indication that the reviews are clickable, but they are not. | n38-mk-neg08-reviews.mp4 | A02 Speak the Users’ Language | scroll down to the reviews → hover over one review | y | 1 | 1 | 1 | 1 | 1.00 | ||||
| Code | Evaluator |
|---|---|
| DS | Daniel Scharf |
| AS | Alfred Sebernig |
| MK | Mario Knezevic |
| VB | Viola Brod |
| y | Found by this evaluator |
References
- [Muniz2016]
- Fabio Muniz;
An Introduction To Heuristic Evaluation;
30.05.2016
https://usabilitygeek.com/heuristic-evaluation-introduction/ - [Inte2023]
-
Interaction Design Foundation;
Heuristic Evaluation;
retrieved on 27 Mar 2023
https://www.interaction-design.org/literature/topics/heuristic-evaluation
A Materials
The following materials were used or created by the evaluation team.
A.1 Heuristics
The evaluators used the Andrews General Usability Heuristics 2013
found in file:
heuristics.pdf.
A.2 Individual Evaluation Logs
The individual log files for each evaluator are provided below:
- Daniel Scharf:
log-ds.txt. - Alfred Sebernig:
log-as.txt. - Mario Knezevic:
log-mk.txt. - Viola Brod:
log-vb.txt.
A.3 Spreadsheet
The filled-out, sorted spreadsheet of all problems and positives
including their ratings is helist.xlsx.