In this article, I explore, categorize, and standardize heuristic evaluation methodologies and data visualization to help inform which method to choose.

Like many UX designers, my go-to heuristic evaluation approach is Jakob Nielsen’s 10 Usability Heuristics for User Interface Design, first published in 1994 and updated in 2020. However, placing all issues found during an assessment into Nielsen’s ten options often felt forced; I would discover discussion-worthy user experience problems that did not fit the mold.

A brief moment of searching for alternative heuristics led to the question, how do I choose the appropriate method? Determining the heuristic similarities and, perhaps more importantly, the differences, were vital to understanding which to choose and when.

What is a heuristic evaluation?

A heuristic evaluation is a process where evaluators assess the usability of an interface against established usability principles.

Usability expert Nielsen Norman Group states:

“Heuristic evaluation is a usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the “heuristics”).”

Dictionary.com defines heuristic as:

<aside> 💡 heuristic *adjective

serving to indicate or point out; stimulating interest as a means of furthering investigation.
encouraging a person to learn, discover, understand, or solve problems on his or her own, as by experimenting, evaluating possible answers or solutions, or by trial and error: a heuristic teaching method.
of, relating to, or based on experimentation, evaluation, or trial-and-error methods.
Computers, Mathematics. pertaining to a trial-and-error method of problem solving used when an algorithmic approach is impractical.*

</aside>

Notice the last definition: “…when an algorithmic approach is impractical”. This statement is a good summary of the subjective, qualitative nature of heuristic evaluation methods. And while this subjective nature is primarily true for the frameworks discussed here, the System Usability Scale (SUS) attempts to quantify heuristic evaluations. Or, for more measurable results, you may want to consider the PURE method for evaluating ease of use.

Approach

My approach to understanding the criteria was to first narrow down the heuristic frameworks to those of a similar number of heuristics. I chose ten frameworks with 6–12 heuristics. Frameworks with more heuristics tend toward checklists and are less heuristic.

Note: I’m using the term framework to collectively refer to each author’s set of heuristics, guidelines, principles, criteria, or rules.

Standardize the data

I categorized all heuristics against core, recognized usability standards from the established sources of Nielsen Norman Group, International Organization for Standardization (ISO), Interaction Design Foundation, and usability.gov.

I’m terming these usability standards ‘quality components’ based on Nielsen Norman Group’s definition that usability is a quality attribute that assesses how easy user interfaces are to use and that usability is defined by 5 quality components: