Heuristic Evaluation & Reports

All original names & content is renamed as example.com for this case study.

I have provided heuristic evaluation and reports for ecommerce department design team in Rakuten using a number of evaluation criteria based on set of user experience rules. It reveal insights and helped design teams to enhance product usability from early in development.

To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and

does not necessarily reflect the views of Rakuten ecommerce division.

Design Exercise for Guest User Checkout

Index

The Scope
- Business Strategy
- User Profile
- User Environments
- Task Analysis & User Goals
Criteria
- Evaluation Criteria
- Severity Criteria
Framework
- The Fogg behavior model: A framework for thinking about ecommerce checkouts
Evaluation Metric System
Evaluations
Report

The Scope

Business Strategy

User Profile

Design Implication

User Environments

Task Analysis & User Goal

Tasks

Provide shipping and billing address
Choose shipping option
Payment
Order confirmation

Goals

Make successful checkout easily

Design: Mobile & Desktop

Criteria

Evaluation Criteria

Jakob Nielsen’s 10 Usability Heuristics for User Interface Design are probably the most commonly used set of usability heuristics. There are others such as the list of six Design Principles for Usability by Don Norman, and the 20 Usability Heuristics by Susan Weinschenk and Dean Barker listed below. There is even a set that contains no less than 247 Web Usability Guidelines by Dr. David Travis.

More than 10 evaluation criteria will possibly render no significant change in the outcome, while a minimum of 5 evaluation criteria will ensure the analysis has some depth.

Using the following Jakob Nielsen’s 10 Usability Heuristics.

Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
Match between system and the real world
The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
User control and freedom

Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing.
Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
Recognition rather than recall

Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use

Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

Severity Criteria

A set of maximum 5 severity criteria
More than 5 severity criteria can overcomplicate the outcomes and not present as useful, while less than 3 severity criteria will not allow for clear differentiation between the severity states.

Framework

A framework for thinking about ecommerce checkouts.

The bottom line is this: Behavior = Motivation x Ability x Trigger.

We want to aim for the top right of the graph—high motivation, easy to do, a trigger in place.

If we have high motivation and low ability (difficult to do), what we’ll get is frustration. If it’s low motivation but easy to do (e.g. take out the trash), we get annoyance.

Evaluation Metric system

Each of the 10 evaluation criteria has been assigned 3 usability rules that support and drive the specific outcome for the heuristic (this number is not set, it can vary). The compliance to each of the usability rules are measured against 4 severity criteria as illustrated in the table below.

Evaluations

Heuristic #1: Visibility of system status

1.1. Is the user aware of their current position within the user journey?

Both mobile and desktop interface is not having the breadcrumb trail to ensures that the user is always aware of where they are within their journey on the example.com.

Recommendation: Would be good to display the breadcrumb on top of the body section. For mobile to avoid lengthy breadcrumb can have one step previous navigation with clear navigation name.

1.1. Is the user aware of their current position within the user journey?

Both mobile and desktop interface form all input fields are having label is inside of the input field. When user enter the information/value, label is hidden and user can’t understand the input filed name.

Recommendation: Would be good to display label on top of the every input fields to always user understand.

1.2. Is the user notified about changes clearly in their user journey?

During checkout added/selected item if not available, user is not notified before the “Place order and pay”.

Recommendation: Introduce review order category/step in the guest user checkout and display if the item is no longer available and remove the price for that specific product/item.

1.3. Is the user aware of factors that can have a significant impact on their experience?

For products/items “Ship by” has given delivery speed only as “1, 2 and 2-5 business days”. Users will find a challenge to work out when to expect delivery.

Recommendation: Display exact date and time limit for each products which is user added in their cart. Example: Order within X hours/mins(based on user interaction live time), get it by day(Ex. Mon, Dec 13. 9:00 AM to 6:00PM).

Heuristic #2: Match between system and the real world

2.1. Are UI elements and interaction prompts easily recognisable?

Shipping address and ship by edit action is looking part the content and text is not case sensitive as other words and link button.

Recommendation: Can use icon or text link button on top right corner of the each sections.

2.2. Does the online experience replicate the familiarity of offline actions and behaviour?

Credit/Debit card number, date and CVV is not as offline format in the card to users to recognise easily and provide card details.

Recommendation: Card details can display in the following way:

First line:

Credit/Debit card number

Second line(total 3 input fields):

Expiry month and year (two separate input fields)
CVV number

2.3. Does the website make use of acronyms, technical terms or jargon that need an explanation? If acronyms are used, are they clearly explained?

For payment mentioned just “card number” only. It will lead confuse/slow-down to understand which card number user has to enter.

Recommendation: Communicating correct card name would be better to user understand and quickly provide. Example: “Credit card number, Debit card number”.

Heuristic #3: User control and freedom

3.1. Is the user able to use the core sections of the website without signing up?

Yes, this main task is to user for guest checkout.

3.2. Does the user have control over their personal information?

User can edit their addresses, ship by option and payment card details. However recommended edit changes on above #2 match between the system and real world evaluation.

3.3. Is the user able to checkout freedom of their choice to choose delivery option?

Delivery/Pickup: Recently increasing importance of store pickup, sites need to ensure that this aspect of their checkout process is optimized. Example.com does not have store pickup option.

Recommendation:

Delivery: Can provide two options - One is to receive to their address and second one is to pickup the nearest store pickup.

Heuristic #4: Consistency and standards

4.1. Is there a consistent design standard for all call to actions (CTA) on the website?

Yes, both mobile and desktop CTA’s are having consistent design. However as recommended above (Match between system and the real world) new architecture design all CTA button name should change to specific steps.

4.2. Is there a consistent design standard for form controls?

There is an inconsistency on shipping address form. In mobile interface “Apt / Other” input field is open by default to enter to users but in desktop interface is hidden in default and user have to click “Add apartment, building, floor” link button to enter the details.

Recommendation: Should have both devices same input fields for shipping address. For mobile all input fields can have in single row for better space to enter details to users. For desktop address input fields can have full size in singe row.

4.3. Is there a consistent design standard for headings?

“Save your information for next time” is not consistent for name and font size. Other page heading is single word and big size font face.

Recommendation: Title can reduce around 3-4 words and apply the same heading font size as other pages.

Heuristic #5: Error prevention

5.1. Are there helpful constraints that prevent the user from making mistakes?

There are no UI constraints to prevent the user from entering incorrect values in forms.

Mandatory / Optional: Not mentioned what are fields are mandatory and optional to users.

Error: Within the shipping address section, the user has the option to add numeric characters in the City field.

Recommendation: Can provide user validation in all the fields related to specific inputs. So that user can correct the inputs immediately when they make mistake.

5.2. Is the user guide with suggestions to prevent incorrect actions?

There is no user guide as help text for all input fields to understand the user what to fill to avoid error and quick to complete their task.

Recommendation: All complex input fields should have guidance help text. Example for CVV: We can communicate what is CVV and display example image where it’s available on credit/debit card to understand users easily.

5.3. Is the user presented with forgiving formatting for information?

Often the user can feel overwhelmed by the number of fields when filling in forms and can easily make a mistake. By using forgiving formatting to relieve the user’s cognitive load, will not only ensure that more accurate information is provided but also make the experience of filling in forms more endurable. Example.com is not having pre-populated information for this guest user checkout.

Recommendation: When the user enter their address, by default using Google we can pre-populate shipping section details such as phone code number, address, city input fields.

Heuristic #6: Recognition rather than recall

6.1. Is the user presented with navigational items that reduce cognitive load and aids recall?

The checkout process (the architecture) is not easy to understand for shipping address, payments and confirmation. It’s grouped by category but the category is not designed to reduce cognitive load to users to understand the number of steps and progress status towards completion while providing information.

Recommendation: We can separate the information as following 3 steps:

Delivery
1. Will get the users shipping and billing address.
Payment
1. Will have an option to choose the payment method.
Review & place order
1. Display the number of items with clear delivery date & time and total amount with all breakup details.
Confirmation
1. Once success the payment will confirm the order with details where and how they can track and can get every stage communication for overall journey.

Overall display the live progress would be good based on user filling the every steps to know the status clearly what’s done and what needs to be filled.

6.2. Is the user presented with customised content based on previous actions?

While user place the order, they can’t review the number of item. They need to go back to review and there is no navigation to go back.

Recommendation: Once user selected payment option and provided the correct information display the number of items with clear delivery date and time to review then can take action “Place order and pay”. And can provide previous page navigation thru breadcrumb/back-button.

6.3. Is the user presented with Luhn validation?

Example.com doesn’t auto-format spaces in the “Card number” field, making it that bit harder for users to check their inputs.

Recommendation: Automatically formats spaces in the card number field(1234 - 5678 - 9012 - 3456), making it easier for users to check what they are typing. And would be good to display the what card is user entered. Example: “VISA”.

Heuristic #7: Flexibility and efficiency of use

The Nielsen Norman Group refers to “accelerators” which are often unseen by the novice user, but are essential for the experienced user to advance them in their user journey.

7.1. Is the user presented with shortcuts to end goals?

Yes, the overall journey to checkout the guest user checkout improve the checkout experience to customers and business.

7.2. Is the user able to tailor frequent actions?

Yes, After the successful checkout confirmation page is asking save password for next time.

7.3.Is the user presented with ambient information for quick actions?

Confirmation page have handful information. But still can add more primary information to user understand such as number of items with delivery date & time. If user is not created password or created password what’s the next action from example.com and how user can understand about delivering the product/items.

Recommendation: Would be good to display the number of items ordered with delivery date & time, and next actions from example.com for delivery.

Heuristic #8: Aesthetic and minimalist design

8.1. Is the user interface design simple and easy to understand?

8.2. Is the user clear on what all the icons mean and why they’re included in the design?

8.3. Are all forms easy to understand and effortless to fill in?

No to all the above three questions. Have already mentioned above evaluation. Please refer #1 to #7 criteria. Overall example.com guest user check need to improve for aesthetic and minimalist design.

Heuristic #9: Help users recognise, diagnose & recover from errors

9.1. Is the user presented with error messages (as opposed to no message) when adding incorrect information in a form control?

All form fields have validation messages when incorrect information has been entered.

9.2. Is the user presented with human-readable error messages that offer useful information on how to rectify the problem?

All input fields on the example.com lacks useful descriptions on how to rectify the problem. The standard “Uh oh! There was an error” message will not help the user recover from the error.

Recommendation: Please provide error help text clearly for every input fields. Not the generic error message to all input fields. Example error message for City: “Please enter correct city name. Numeric numbers can’t be city name”. Suggesting the city name thru Google based on user location would be good.

9.3. Is the user presented with polite error messages that don’t blame the user for the error?

The validation messages are not descriptive nor polite.

Recommendation: The tone of error messages is extremely important as well. Error messages need to clearly convey information to the user and if an error is serious, the tone should reflect that. Using words like “please” could lessen or increase the severity of a message depending on how they are used.

Heuristic #10: Help and documentation

10.1. Is the user presented with clear steps/guidelines to use the product service?

No, as mentioned above the overall steps(#1, #6) is not easy for guest user checkout.

Recommendations: Please refer #1 and #6

10.2. Does the user have access to documentation with relevant topics to help reach their goal?

Help centre is not linked with example.com guest user checkout experience.

Recommendations: Would be good to provide the help link to each specific group for the overall guest user checkout.

10.3. Is the user presented with other channels of communication to enquire assistance to reach their goal?

There is a Contact page link is available on the bottom of the screen. But contact page interface is not available to review. And order confirmation page also not having contact information to users.

Recommendations: In order confirmation screen, good to display customer contact details such as chat, email and phone.

Report

From the level of compliance for the usability rules, a score is allocated for each of the 10 heuristics for example.com