leader in ux research, service design, & visual design
Rapid Evaluative Research
2 Mini Case Studies
Competitor Product Marketing Sites, Persistent File Protection (PFP)
Competitor Product Marketing Sites
🔎 Research Categories: Evaluative
📝 Project Type: Cognitive Walkthrough; Competitive Analysis
🕵️♀️ Role/Contribution: Research Lead, UX Strategist
🗓️ Timeline: 2.5 weeks
🛠️ Relevant Tools: UserTesting, FigJam
🤝 Cross-Functional Team: UX Researcher, UX Manager, UX Designer, Marketing Designer
👥 Stakeholder Teams: UX, Product, Product Marketing, Marketing
🔒 Users: 18 Users (UserTesting)
*Note: The visuals on this page came from my final UX Research report and presentation and are meant to showcase my research reporting ability.
🥅 High-Level Research Objectives:
Ensure that users can easily find and access information about the file-sharing product on the website
Assess the clarity and effectiveness of the website content in conveying information about the file-sharing product
Discover how users react to the file-sharing product’s website in comparison to competitor product websites
Background
Problem Statement
We need to understand how best to update the product marketing website for our file-sharing product so that it allows users to easily navigate and learn about the product to drive usage and sales.
Research Goals
Investigate and analyze user preferences regarding our file-sharing product’s website compared to our competitors’ product websites.
Understand whether our file-sharing product’s website accurately communicates the product’s offerings to users.
Learn how users prefer to navigate and discover information when perusing product marketing websites.
Key Insights
Information Overload
Approximately 50% of users felt our product’s website was too lengthy and inundated them with too much information. While navigating all three sites, users noted the scrollability and the amount of information on each site.
Recommendation: Shorten the length of the site to keep it more scrollable and navigable.
Formatting
The section of our website that was “chunked out” with a bulleted list was a big hit because people could read and digest the information quicker than the paragraphs of information. Users preferred more negative space because it allowed their eyes moments to rest.
Recommendation: Chunking more information and introducing more negative space will make the site easier to read
Overly Technical Language
Users found the use of very technical cybersecurity jargon off-putting. When a website used very technical language, this hindered their understanding of what the product does and also caused them to lose interest when skimming the information.
Recommendation: Our site performed well in this area with 83% of users feeling that our file-sharing product was explained well in an easy-to-understand way.
For further information on findings and insights, please contact me.
Group 1 (top row), Group 2 (middle row), Group 3 (bottom row) used for Cognitive Walkthroughs
(Left to Right) User Preferences between Websites in Group 1, Group 2, and Group 3
Research Impact
Business
The easier the website is to navigate and understand, the more users we can sign up, which contributes to the efficiency of our marketing sales funnel.
Strategy
This project informed a second usability study for our file-sharing product’s sign-up flow. The product’s marketing website acted as the starting point in the sign-up flow, leading you to a page where you could sign up (or login). Insights from the second usability study were similar, with users feeling that the sign-up flow had moments where it didn’t provide enough information and other moments that were too wordy. Based on insights from both studies, the updates to the file-sharing WebApp included:
100% of users tested wanted the page after sign-up to be removed. This page was meant to explain the product but users complained that it repeated the product’s marketing site content, which they have just seen.
Users felt the sign-up page was too bare when coming from the lengthy and wordy product marketing website. It was important to strike a happy medium so that users didn’t feel suddenly displaced after going from an informative marketing site to a sign-up page. We decided to provide more context on the sign-up page while being mindful of not going too information heavy.
User
This research made the file-sharing product’s marketing website easier to skim for users so that they could understand the product quickly and, just as important, begin using the product easily.
Methodologies
Affinity Map of 18 Cognitive Walkthroughs
Some of the key methodologies for this project are listed below, with explanations as to why they were chosen for this project:
Cognitive Walkthrough (Competitive Analysis): I chose to approach this evaluative research by combining cognitive walkthroughs with competitive analysis. In three different groups, we had participants view three product marketing websites: our file-sharing product’s website and two competitor product websites. I requested they employ a “talk-aloud” protocol about each site. Without clicking on anything, I had them view the site and discuss what they were doing as well as what caught their eye. After they had viewed all three sites in their grouping, I requested that they talk about the site that they favored the most out of the three and why. The combination of cognitive walkthroughs and competitive analysis provided a holistic evaluation of user interactions, allowing for a nuanced understanding of users' cognitive processes, preferences, and comparative insights.
Research Report: Because this project worked more closely with the Marketing team than our Product team, I had to ensure that my research report targeted the appropriate audience. This meant translating my insights to fit the mental model of our Marketing team, including content writers, branding specialists, etc.
Reflections & Learnings
An executive-level stakeholder interrupted the research presentation multiple times throughout. I made sure to patiently pause to answer questions, but also kept them short to keep focus and move the presentation along. This allowed for everyone, including that stakeholder to receive all of the valuable and actionable insights.
The research-backed cybersecurity personas I had developed before this usability study came in handy! They provided a nuanced understanding of how different users might engage with our marketing product site, especially since the level of security understanding ran from security expert to security novice.
Persistent File Protection (PFP)
🔎 Research Categories: Evaluative
📝 Project Type: Usability Testing; UX Writing
🕵️♀️ Role/Contribution: Research Lead, UX Strategist, UX Writer
🗓️ Timeline: 1.5 weeks
🛠️ Relevant Tools: UserTesting, FigJam, Loom
🤝 Cross-Functional Team: UX Researcher, UX Manager, UX Designer, Product Manager, Engineering Manager
👥 Stakeholder Teams: UX, Product, Engineering, Executive Team
🔒 Users: 9 Users (UserTesting)
*Note: The visuals on this page came from my final UX Research report and presentation and are meant to showcase my research reporting ability.
🥅 High-Level Research Objectives:
Determine whether users understand the functionality of Persistent File Protection (PFP)
Compare the understandability of Persistent File Protection (PFP) vs. Prevent Download
Understand whether the current tool tip helps users understand the feature
Background
What is Persistent File Protection (PFP)?
The "Persistent File Protection (PFP)" feature adjusts the configuration of an encrypted file so that when a recipient initiates a download, what they download is not the actual file itself but rather a bookmark. Upon opening the bookmark, they are directed to a file-viewing WebApp that validates the viewer before allowing them to view the encrypted file/message, ensuring an additional layer of security and privacy.
Problem Statement
As we work towards fully shifting our products over to the new design system, we need to address the usability of some of our core product settings, such as “Persistent File Protection (PFP)”.
Research Goals
Define how well users understand what Persistent File Protection (PFP) does based on the name of the feature
Learn whether re-naming the feature to Prevent Download increases the ability of users to understand the functionality of the feature
Discover the usability of the tool tip as it is currently written
Key Insights
Persistent File Protection (PFP) vs. Prevent Download
❌ 80% Fail Rate when users were asked to describe what “Persistent File Protection” means on the page.
✅ 75% Success Rate when users were asked to describe what “Prevent Download” means on the page.
Recommendation: Change Persistent File Protection (PFP) to Prevent Download across all products. Also, actually prevent a download. When this feature was first introduced a decade ago, there was no history in the file-viewing WebApp, so it made more sense that it would download a bookmark to redirect you there to view the file. Now that there is also a file-sharing WebApp with a history section, there is no need for users to be able to download a bookmark. They can see all the history of files shared and received in one place, so preventing the download simplifies both the user experience and the product architecture.
Tool Tip
Only 10% of users correctly understood what the tool tip meant.
Fun fact: at the start of this project, during stakeholder interviews and information-gathering, we uncovered that most of our employees didn’t fully understand Persistent File Protection and the tool tip either!
Recommendation: Partner with our content and product marketing teams to rewrite the tool tip, especially now that the feature will actually be preventing download.
For further information on findings and insights, please contact me.
Usability Testing Prototypes: Persistent File Protection (PFP) vs. Prevent Download
Research Impact
Business
“Persistent File Protection” has been core to the product since its inception and appears in every product but, because it was not understood by our users, it was rarely used. The change in this feature allows us to prepare for the future in which our file-sharing and file-viewing WebApps are combined into one product.
Strategy
Many stakeholders within the organization were even unsure of what “Persistent File Protection” meant on a technical level. Renaming it to “Prevent Download” also allowed us to simplify the way this feature actually works on the backend, streamlining the user flow for our users but also streamlining how our product functions.
User
Reduce users’ confusion about settings within our products that are meant to help them protect their data. This also allows users more control over how they protect their files and data.
Methodologies
Dimension Analysis: Plotting users on continuum style dimensions to see themes/patterns
Some of the key methodologies for this project are listed below, with explanations as to why they were chosen for this project:
Usability Testing (Content): To understand whether users understood our phrasing (and reduce bias), we conducted two usability tests through UserTesting:
Users see the feature called Persistent File Protection, explain what they think it means, view the tool tip, explain what they think Persistent File Protection means after the added context, and then what they would think the feature meant if it was called Prevent Download.
Users see the feature called Prevent Download, explain what they think it means, view the tool tip, explain what they think Prevent Download means after the added context, and then what they would think the feature meant if it was called Persistent File Protection.
Dimension Analysis: Dimension analysis is a form of affinity mapping or cluster mapping where you use a line (also called a dimension) to plot users either on a continuum (ex: experience level // less experienced —> more experienced) or categorically (ex: feelings // sad, mad, happy, disappointed). We wanted to know how well the users understood the different wording and plotting them on a continuum helped us see their different levels of understanding in relation to one another.
Reflections & Learnings
Despite the clear evidence that users could not understand what Persistent File Protection meant, our CEO was reticent to change the wording despite hearing our final recommendation to change it to Prevent Download. Because it had always been called Persistent File Protection (since the inception of the company 10 years prior), he had a personal attachment to the phrase. We enlisted the help of our executive team such as the SVP of Engineering/Design and Engineering Managers to help us relay to him that it was the right decision.
Normally, our UX Designers could perform usability tests on their own. However, I guided our UXD closely for this because of how ubiquitous this feature was in our products.
This went beyond UX Design and UX Research and required us to put on our UX Writing/Content Design hats. We went through multiple iterations of “Prevent Download” before settling on that for the usability test.