Designing a unified Testing platform To increase user trust.
ROLES
Principal Designer
PERIOD
Jun 2019 - Sep 2020
Collaborated with
Product, Research, Engineering, Growth, Support, Sales
In 2019, Sauce Labs approached me to replicate the success I achieved at BrowserStack by auditing and redesigning the existing Sauce product experience.
I WAS HIRED BY CEO Charles RAMSEY and CMO TERRI AVNAIM, to audit the existing product and design a plan for redesigning the product experience.
Performed a Deep product and team audit that surfaced issues around product flaws and inter-team dynamics.
Based on my work with BrowserStack, CEO Charles Ramsey and CMO Terri Avnaim reached out to me to help Sauce Labs uncover and fix key product problems. Over 3 weeks, I met with teams in San Francisco and Berlin to audit the problems with the product, workflows and team communication which I presented to the management team.
creating a unified Navigation will increase user trust and drive product discovery and adoption.
Based on learnings from the audit, we created a data dictionary and nomenclature that simplified communication with customers and across the organisation. The redesigned navigation included the new hierarchy of products and was implemented.
simplifying the live testing experience and reducing the pricing would increase usage by freelancers and design studios.
Sauce Labs was losing customers to BrowserStack because of an inferior Manual Testing experience. I updated the Manual Testing experience to align with BrowserStack's Live Testing experience with additional support for app testing and view logs from within the same UI.
Reducing onboarding friction will allow more self-serve users to sign up.
The existing Sign in/Sign up pages added friction points to the authentication experience reducing the number of self-serve registrations. We reduced the number of mandatory fields and A/B tested multi-step onboarding forms to reduce the onboarding and sign in friction. Within 3 months of shipping the new authentication flows, self-serve registrations increased by 10%. However this increase coincided with COVID making it unclear as to the reason for the increase.
in-depth version
Performing an audit
🚩 The problem
Between June and August 2019, I met with different teams - Design, Engineering, Product, Sales and Support - located across San Francisco, Berlin and Warsaw, to get their thoughts and insights about the product, process and user experience. The interviews also gave me a chance to ask questions around data and analytics, company health, team structure, and collaboration dynamics within the company.
✅ Outcomes
After meeting the teams, I compiled the feedback and presented my findings to the management team. Below is some of the feedback and primary concerns that came up during a few of the interviews.
💆🏽♂️ Challenges
Teams were cautious when talking with me as having come from the competitor, BrowserStack, I was approached as an outsider who didn't "understand" Sauce Labs.
Sauce had acquired testing tools like TestObject and Screener.io, but the products had not been fully integrated yet and there as no formal plan for unifying the products.
Key Design decisions
creating a unified Navigation will increase user trust and drive product discovery and adoption.
🚩 The problem
Links to products and features are scattered across the header and sidebar without any clear hierarchy.
Navigation does not highlight and link to acquired brands like TestObject and Screener.io, reducing discoverability.
No standard naming convention or nomenclature.
We first had to understand the current sitemap and identify the primary issues with the existing navigational experience.
Proposed options for organising the information hierarchy and navigation.
✅ The solution
We created a nomenclature and data dictionary for all teams across Sauce Labs to use to harmonise communication within the organisation and when speaking with customers.
We explored different layout options for the navigation including a horizontal style header.
We divided the sidebar into 2 sections - user session actions at the top and user account/system actions at the bottom.
Reposition minutes and tunnel usage to the header and provide detailed information and actions for Sauce Connect.
simplifying the live testing experience would increase usage by freelancers and design studios.
🚩 The problem
Sauce had positioned itself as the leaders in Selenium and automated testing. However, the explosive growth of BrowserStack ate away at their customers and hence they needed a manual testing product that could compete with BrowserStack's Live.
All users trying Sauce for the first time would always try the Manual Testing experience first to quickly gauge the performance, reliability, stability and usability of the platform. Since the Manual Testing experience was clunky, slow and sometimes confusing, this would put off potential customers.
All Sauce Plans came with manual testing, though the price point was geared towards enterprises and not startups and smaller studios, who were a large segment of manual testers.
Enabling localhost testing from the console was unintuitive and hard to use. It was also difficult which localhost tunnels were active and which weren't.
✅ The solution
Using what I learned during my time at BrowserStack, we converted the long list style UI into a list of collapsible jobs that could be opened to view results or run a new test. We also replaced the light theme UI with a dark UI to focus attention on the browser+OS selection tool.
Users could easily add multiple files to their workspace or session by dropping files directly onto the screen. We designed the different states of the uploads, including error states if uploads failed.
Different states for uploading files
Display job details directly in the UI so users don't have to go to other properties. We also loaded the UI with pre-built apps so that users could get started immediately without having to upload an app from scratch.
We added a customizeable toolbar that allowed users to get started with their most commonly used browser+OS or platform+device combination in a single click.
Different states of the new local testing extension
role based dashboards provide Actionable data for different team members.
🚩 The problem
The current dashboard focussed only on SDE related details like builds, automation and live testing. However, from the audit we learned that project and product managers wanted to use the dashboard to optimise their development pipeline, identify problematic devices and perform a cost analysis for their device cloud to determine the ROI.
Empty states did not provide enough information for users to get started by themselves without reading lengthy docs.
Less technical Live Testing users found the UI heavily focussed on automation and hence confusing.
The existing UX of the Sauce Labs login pages
✅ The solution
Added an option to allow users to assign roles to invited team members - engineer or manager. Based on the role, the dashboard would display relevant information and tabs.
For the engineering dashboard, we added glanceable data about running/passed/failed jobs at the top of the page, followed by a list of collapsable jobs to see the results and logs.
A Project Manager is more interested in metrics that affect the user experience like failing OS versions, failed device runs, cost efficiency of the private cloud stack etc.
🔄 Other Design directions
We tried a Launchpad experience where users select individual products that they want to use. This would instantly bring all Sauce Labs products into a single platform while also allowing teams to control the UI and UX of their products.
🚩
This approach was abandoned before the visual design stage since the management team did not buy into the Launchpad experience and instead wanted a user-focussed Dashboard experience that provides summarised and actionable data.
Reducing onboarding friction will increase sign ups by self-serve users.
🚩 The problem
Users were unaware which data center they had to log in to and how this affected their account.
Sauce Labs, TestObject and Screener.io each had their own sign in pages that looked completely different. This not only confused users, but increased abandonment.
Error messaging and notifications were cryptic and did not help users solve the problems.
Marketing messages caused friction instead of guiding sign up process.
The existing UX of the Sauce Labs login pages
✅ The solution
Two-step sign up process that reduces cognitive load of sign up
Display progress, errors and notifications clearly to reduce confusion and guide the user in the right direction.
By default, assign user account to data center based on their location - US or EU.
✅ Other Design options
Two-step sign up process that reduces cognitive load of sign up
Display progress, errors and notifications clearly to reduce confusion and guide the user in the right direction.
Outcomes
We implemented the new navigation and the nomenclature that was shipped across the company. This simplified documentation helping sales and support staff communicate with customers with confidence.
Internal testing showed that the new dark UI made the experience "feel softer" on the eyes. Teams also liked the new Live Testing and Job Details page that make the page scannable for key details.
The new designs helped users understand how to download, setup and use Sauce Connect. By providing links to the documentation, users could solve their own problems, reducing Sauce Connect related support tickets by 10%.
Challenges
A new Design Director was hired to juggle large projects alongside the smaller design requirements needed for sprints. This took away resources from the unified platform and extended the timeline proposed in the audit.
While the designs elevated the user and developer experience, the team lacked the engineering support to build a design system and components that aligned with the new design direction.
After Charles Ramsey left the company, the new CEO Andrew Miles was charged with "shifting left". This reduced the importance of implementing the unified platform across teams.
As a result of Covid, half the design team was let go and the remaining designers were assigned to product teams. The unified platform was put on hold. Sauce Labs eventually redesigned their brand and product experience from scratch.
Learnings
Organizational buy-in across geographies takes a long time and may not come to fruition.
Large, cross-organisational projects require buy-in from department heads that are often too busy with their own team and responsibilities. Breaking the ice with different teams and getting teams involved and excited about the benefits of a unified platform was the first step to getting buy-in from team heads.
claim Design's place at the table.
As a design consultant coming from the competitor into an engineering-driven environment, I pushed to have the unified platform given equal importance as other engineering efforts as design issues were directly affecting the product and business outcomes.
Foster healthy inter-team communication, especially if teams are remote.
Many of the problems I uncovered at Sauce Labs were due to mis-communication between teams, especially due to the different timezones and task prioritisation. Always ensure that teams have fully understood and are onboard with the objectives. This should be a written confirmation that all teams sign off on before doing any work.
Acknowledgements
I am grateful to Charles Ramsey, Terri Avnaim, and Brad Edelberg for giving me the opportunity to work with the Sauce Labs team. I enjoyed working with, and learning from a very talented team of designers - Michael Wiekrykas, Seth Kornfeld, Joan Jasak, Josh King, Steve Leslie, Wanda Seto, Anna Nawrocka and Marek Pohl - and look forward to having the chance to work with them again.
I would also like to thank Marcia Foley for helping me get around and get familiar with who's who, while staying laser focussed on the job at hand.
VISIT SAUCE LABS
Frankensteined in Mumbai.
Fueled by YouTube, 90s cartoons & Heavy Metal.
