This week, Nielsen Norman Group published the 5th edition of their UX Metrics and ROI report. We are honoured to be featured in it with a case study related to our business unit Acumbamail, alongside brands such as Atlassian, Shopify and Luxottica.

The Acumbamail case

The case study covers a strategic experiment carried out jointly by the central Group Design department and the local Acumbamail Design team, with the final aim to improve usability and ultimately business metrics on the Acumbamail website.

The goals were set as follows:

  • To improve conversion rates both to paid subscriptions and to free accounts
  • To enhance the frictionless approach

  • To solve all the usability problems in the step between the pricing page (or the primary CTA in the website) and the landing on the platform.

The process involved a first step (testing to understand), a second step (redesign), and a third and final step (test to validate).

Problems discovered thanks to usability tests

  • The password creation instructions were not in contextual proximity of the microtask area (usability problem)

  • The shape of the pop-up prevented people from finalising the task when the screen resolution was slightly lower than normal (business problem: it prevented people from converting to the product)


The solution

Once the problems had surfaced, we moved on to redesign the affected website areas and to test whether the changes brought the desired results.

The redesign involved two actions:

  • Forcing the final pop-up to stay above the fold

  • Reshaping the pop-up modal in order to accomodate side by side input fields and suggestions/feedback for password creation

The design metrics we tracked

During the different phases of the project we tracked some specific metrics with specific tools. Quantitative usability testing and Analytics were the main tools we used. The metrics we tracked and monitored were:
  • Subscription time on task: -10%
  • Subscription error rate: -48%
  • Subscription Conversion rate: +29%
  • Subscription NPS: +5000% (from 1 to 51)

Why is UX ROI so important

Design is a way for improving business KPIs. Design has a quick ROI and a long term ROI. The first is perceived as soon as you get an easier to use or more effective interface or service. The second is perceived when business KPIs are altered by design actions somewhere else.

The tracking of those ROIs has an effect on many levels:

  • It demonstrates the value of our design efforts
  • It shows how design changes impact the bottom line
  • It can increase the trust on design and designers
  • It increases the design maturity of our organisation
So no matter the size of the revenue, a good design improves what matters to our users and business.

About the report

The report is the 5th edition of a collection of 44 real-life case studies highlighting how teams have used UX metrics to assess the impact of their design changes.

The 297-page report also includes advice for applying these concepts to own work, as well as detailed quantitative analysis of the 99 total case studies collected for all editions of this report over 14 years.

At the end of the theoretical part, the report lists 44 case studies from different industries (Enterprise & B2B, Nonprofit & Education, Healthcare & Insurance, Entertainment & Tourism, Transportation & Real Estate, Ecommerce, Finance, Intranet/Internal, Miscellaneous)

Each study includes:

  • Metrics collected

  • Methodology used

  • Description of the product or service

  • Description of the design problem(s) that the team attempted to address

  • Description of the solution(s) implemented

  • Results of the design change

About Nielsen Norman Group

NNG is the leading voice in the field of User Experience since 1998.

Their founders, Jakob Nielsen and Don Norman, are recognized around the world for their leadership in defining the field of UX.

Together, they founded Nielsen Norman Group, an elite firm dedicated to improving the everyday experience of using technology.

Article by Chiara Scesa
Design Lead, MailUp Group

ft 1000 2021
ft 1000
deloitte 3
deloitte 2
saas 1000