browserstack

A native approach to responsive testing.

BrowserStack launched Responsive to give customers a quick responsive tool that showed users what their websites look like on real
devices. I lead the design and front end
development teams and was
responsible for designing the UX,
designing and coding the scaling algorithm
and ensuring project milestones were met.

In 2013, BrowserStack launched Screenshots to rapidly test website layouts across more than 500 desktop and mobile devices. Screenshots provides a faster and cheaper alternative to Live, but does not offer interactive testing features.

BrowserStack Screenshots was the inspiration for the design of Responsive.

We studied user data for a period of 10-12 weeks and observed that users ran more tests on iOS and Samsung devices, than desktop browsers. The high usage of mobile presented an opportunity to build a product specific to mobile testing, using the Screenshots API. The results generated through the API were accurate, though the experience of viewing the results felt unnatural compared to testing on a real device in your hands.

Numerous, freely available, testing solutions already existed, though most only provide viewport testing, without accounting for difference in OSes, screen size, device size, aspect ratio and on and on. While these responsive tools work for quick and dirty tests, testing on a real device or emulator is essential for accurate results.

We wanted to display results in a format which accurately represented how users would view websites in the real world. The team felt that Screenshots, along with the other responsive tools, failed to answer critical questions like:

Which devices are the most relevant to test on?

The mobile market is highly fragmented and it's almost impossible to determine the right devices to test on. How relevant are older devices and how far back do you go? User studies and OS/browser usage data told us that this depends on the size of the company's users. Start ups tend to test on the smallest set of devices, while enterprises are obligated to test on larger sets. BrowserStack tracks user sessions to learn more about which OSes, browsers and devices are frequently used. We found that Samsung and iOS devices were the most frequently tested Oses; this made sense since they own a majority of the mobile market share.

The aim of Responsive was to be quick - enter a URL and get your results. By limiting the devices to a curated set, we defined the relevant devices to test on, based on the size of the company. The higher usage of iOS devices meant that we needed different iOS versions in the mix. We added Samsung and Amazon devices due to their popularity. For the sake of completeness, we also decided to include large desktop resolutions. Since the list would be updated automatically based on device popularity, we chose not to define the exact list of devices, but instead, defined a category of devices that should be included.

  • popularity
    3 Phones
    2 tablets
  • variety
    4 Phones
    2 tablets
  • completeness
    2 Desktop
    Resolutions

Does seeing the device hardware improve the testing experience?

Since we were going for a realistic experience, the results needed to look real. A key factor in making the experience authentic is being able to view results within a realistic looking interface.

The form factor of a phone affects how users interact with it. Screen size, device size, placement of hardware buttons and UI elements such as the status and navigation bar impact how websites are displayed and how much content is available above the fold.

  • Status bar

    Provides system specific information and is a persistent UI element. Top of the UI stack.

  • Device hardware

    Location of hardware buttons on a device determines how quickly users can access these buttons while browsing.

  • Browser

    Websites are rendered within the browser, which is lower in the UI stack than the status and navigation bars.

Does the device look similar to a real device?

Using the scaling formula, we were able to scale devices to actual size depending on the monitor size selected. Larger monitors scale down the devices, while smaller screens scale up the device size, when viewed on the appropriate device.

We ported the entire Wikipedia display size matrix into a JSON hash map that enabled us to easily look up device widths and heights in millimeters, organized by aspect ratio. For exmaple, for an aspect ratio of 5:4, a sample entry in the hash map for a 3.5" screen is,

{
  "aspectRatio": "5:4",
  "displays": [{
    "diag-in": 3.5,
    "diag-cm": 9,
    "width-cm": 7,
    "height-cm": 6,
    "area-cm": 39
  }],
  "aspectRatioValue": 1.25
}

This JSON file contains a list of all the devices supported on Responsive and their corresponding measurements (in mm). For example, for the iPhone 6, the entry in the JSON object is,

"iphone6" : {
    device:"iphone 6",
    port:67.1, land:138.1,
    leftOffset:4.544483333333333, topOffset:19.90725,
    width:57.912, height:100.34058333333333,
    land_left:16.891, land_bottom:4.4238333333333335,
    land_width:104.16116666666666, land_height:58.71633333333333,
    urlcss: {
      top: 0, left: 3.4572222222222226, width: 50,
      height: 4.494388888888889, font: 2.592916666666667, textalign: 'center'
    },
    urlcss_land: {
      top: 0.60325, left: 23, width: 58
    }
}

The attributes saved in this JSON format are mm values. These values need to be converted to pixels in order for the scaling algorithm to render accurate results. With the device width known, the dimensions of the viewport in which the screenshot is displayed are set. The width, height, top, left, bottom and offset positions are also determined from the hash table and converted into pixels. In landscape mode, the width of the viewport is equivalent to the height of viewport in portrait mode.

The 15.4" MacBook Pro is the default monitor size (PPI 110). Since users may not be familiar with their monitor size, we pre-populated a dropdown list of devices to choose from. The devices were chosen based on Google Analytics data for the 10 most popular screen sizes used to access BrowserStack.

function convertMMtoPixels (size_in_mm, offset) {

  var wd_in = size_in_mm / (2.54*10)
  wd_screen_px = wd_in * screenPPI;
  return wd_screen_px;

  var resolutionX = screen.width;
  screenDiagnol_calc_inches = screenDiagnol_calc / screenPPI;
  var aspectRatio = screenAspectRatio;
  if (aspectRatio == 1.33) {
    var screenDiagonal = screenDiagnol_calc_inches * 25.4;
    var angle = 36.86989764584402;
  } else if (aspectRatio == 1.6) {
    var screenDiagonal = screenDiagnol_calc_inches * 25.4;
    var angle = 32.0053832080835;
  } else if (aspectRatio == "ipad") {
    var screenDiagonal = 246.38;
    var angle = 53.8;
  } else {
    var screenDiagonal = screenDiagnol_calc_inches * 25.4;
    var angle = 29.357753542791272;
  }
  return ((1 + (offset / 100)) * (Math.round(width * (resolutionX / ((Math.cos((angle / 180) * Math.PI)) * screenDiagonal)) * 10) / 10));
};

Final Designs

Low fidelity wireframes were sketched on paper but most of the design was done in the browser, adapting elements of the Screenshots interface. We kept the workflow similar and introduced new visual elements such as the device slider and the information panel.

Final design implemented on site

Take the tour.

Spreading the word

I designed and coded the newsletter that was sent to 30,000 users, informing them of the launch of Responsive.

BrowserStack Newsletter

User Reactions

What I learned

  • 1. Don’t design in the browser.

    After deploying Responsive, I moved back to designing in Photoshop/Sketch first and then building in the browser. I got very dependant on seeing instant results and would often keep playing with code till the UI looked usable. However, sketching with pencil and paper and using tools to design, gives you the freedom to think, ideate and execute a number of ideas before settling on one. Designing in the browser gave me tunnel vision and narrowed my scope of design.

  • 2. Build products for users, not yourself.

    Responsive was built to complement the Screenshots service. However, while Free Tier usage was high, the increase in subscriptions was marginal and in the end, we hadn’t really addressed any crucial market requirements. We made responsive testing more accurate and simpler, but did not improve the testing process enough to warrant a paid subscription. Early validation would have helped us understand the use cases and if Responsive even needed to

  • be built.

    I was happy with the early responses from customers including Ethan Marcotte himself. However, since the goal of increasing Screenshots subscriptions was not met, I don’t consider the project to be very successful.

  • 3. Use modern technologies and provide graceful fallbacks for older browsers.

    Much of the front end technologies required to emulate mobile interfaces and devices required use of HTML5, CSS3 and modern JavaScript technologies such as matchMedia and JS transforms. We decided to use technologies that modern browsers support and provide fallbacks for users on older browsers. Majority of the users used modern browsers and so this was a no brainer for us.

Acknowledgements

I worked with a small but effective team, that was able to design, build and deploy iteratively. Many thanks to Ahmed Abbas for all the JS work and helping me build the scaling algorithm. I'd also like to thank the Rails team - Rahul Nawani and Utsav Kesharwani - for building a great API that helped us reuse much of the existing code from Screenshots.