How I Redesigned Settings to Increased Engagement by 135%

Define

Next, I facilitated a series of discovery workshops with stakeholders and a cross-functioanl team to identify the problem we would like to solve and some of our measures of success. One exercise I used to help determine the problem statement was the 5 Why's exercise.

Problem Statement

How might we provide a way for users to control the delivery method for notifications, the timing of notifications and they type of notifications they receive?

Ideate

I led the same cross-functional team in two ideation sessions where I leveraged journey maps to help us do brainwriting exercises, storyboarding and sketches to figure out ways to empower the user to have more control over notifications. At the conclusion of the sessions, the team decided on expanding the current notifications center UI to accommodate more features, and I created a quick sketch of notification settings and preferences.

Information Architecture Questions

Coming out of the workshops, the developers had an idea of how they wanted to approach the UI and structure the API. However, I still needed to run these ideas that may impact the experience by our users. An outstanding question I had was how do users access the settings and preferences.

Using a Moderated Cardsorting Exercise to Help with Navigation

From a UX perspective, I had questions about the information architecture. I used a moderated card sorting exercise to determine the best way to handle the navigation to the settings.

Unmoderated A/B Testing to Help with Interaction Design

I also had questions about controls. Should I use a checkbox verses a switch to enable the communication settings? I then used an unmoderated A/B test to determine whether to use checkboxes or switches. The control group used checkboxes while the variable used the switch. I surveyed approximately 379 users over a 7 day period, which met my statistical significance criteria at 90% confidence level. Both options performed well, but I found that checkboxes were slightly better for the users.

Prototype

I ended up expanding the current accounts application to allow for several different settings and preferences for the platform. There was a little redesign that I had to do because the current settings application had only one screen and had no navigation.

The original prototype was done in Figma leveraging a set design system and tokens. I have recreated this prototype in mockplus for this usecase.

View Prototype

Test

I returned to the initial set of users who left negative feedback about the notifications center and asked if they would participate in a usability study. I scheduled a interviews with 20 users with the help of calendly. I also created a test plan and test script.

I worked with our stakeholders and cross-functional team members to make sure that at least one of them were available for each session.

I then facilitated moderated user sessions where there were three tasks related to settings and the second half was open to feedback.

Outcome

- After releasing the updates to the settings application, user engagement increased by 135% in the first 3 months.

- User sentiment improved to 91% of users surveyed selecting satisfied or very satisfied with their overall experience.

- The promotors for the NPS score for the overall platform increased by 10%.

Lessons Learned

Users want to be included in the process. When I contacted users about their feedback, they offered even more ideas than what we initially considered and wanted to take part in future usability tests.

Introducing new controls to users can be difficult, time consuming and may not be worth the effort. Most of the applications on this existing system utilize checkboxes. There are only a handful of instances where a switch is used and most users never encountered them. Sticking with a checkbox was just easier for us and the user.

Giving users more control over their experience can lead to more engagement. When users found out that they could set communication preferences, more power users began to encourage others on their team to use the functionality, which led to more engagement numbers.

Even the best designs need time to grow on users. After releasing the updates to the account area, we realized that more work should have been done to communicate these updates to users prior to launch. Many users complained about disorientation because things had moved or changed.