Entries in testing (1)

Friday
Jan032014

Device Lab progress

A month or-so-ago we set about building a device lab to help with our cross-device testing. Once we'd figured out which device and operating system combinations we wanted to start with we looked at the cheapest ways of obtaining them. Some had to be bought new, but others could be picked up relatively cheaply 2nd hand.

Early December was like Christmas-come-early with a flurry of devices arriving at the office. Pretty much learning as we went, we started figuring out a set up process to help make things easier as new devices came in. We're still adding to it and most things are commonsense, but if you're thinking of setting up your own device lab then there may be some useful points:
  1. build an asset register. To help keep track of what devices we have, where they came from and associated peripherals it's useful to maintain a register.
  2. turn-off auto updates. Where possible this applies to both the operating system and browsers/applications. We purposefully picked up some devices that were running older versions of an OS and it was important to make sure we maintained these.
  3. provide WiFi access. We're planning on using a variety of tools to help make testing across multiple devices easier. These all rely on being on the same network. 
  4. install browsers and choose a default. On a number of the devices we've installed a mixture of browsers. Installing these upfront and choosing a default makes life easier – where possible remember to turn off auto update on browsers if you want to maintain a specific version. 
  5. clear the browser history and cache. If you've picked up a 2nd hand device then it'll be useful to clear the browser cache (this is also good practice for post-testing). Most of the devices we picked up were already wiped, but one went straight into gmail logged in as the previous owner, eep!
  6. update display settings. Whilst testing the chances are you won't be directly interacting with all the devices all the time. Updating the display settings so that the device doesn't automatically turn-off the display will save with the hassles of repeatedly unlocking or adjusting screen brightness.
With the devices set-up it was time to look at how best to test across multiple devices at once. We looked at a couple of options including Adobe's Edge Inspect. But for the time being we're moving forward with using Ghostlab – we found this worked across all the devices, had plenty of flexibility and handy debugging using Weinre. (Thanks to folks who commented on the previous post suggesting looking at these!).

Our set-up at the moment isn't perfect, but we're about to go through a major office refurbishment after which we're hoping to have a dedicated space for a permanent device bench. Once this happens the next step will be to get ourselves set-up such that we can open the device lab to local developers.

We currently have 14 devices with a variety of operating systems. The list at the moment looks like this:
  • ZTE Open – Firefox OS – Gecko 18
  • Samsung Galaxy SII – Android 4.1.2 (Jelly Bean) – Android 4.1.2 (WebKit 534)
  • Nokia Lumia 800 – Windows Phone 7.5 (v797.01) – IE9
  • Hudl (Tesco) – Hudl 1.3 (Android 4.2.2) – Chrome (28.0.1500.94)
  • Samsung Galaxy Tab 2 – Android 4.1.1 (Jelly Bean) – Android 4.1.2
  • Apple iPhone 5
  • Apple iPad 4 – iOS 6.1 – Mobile Safari
  • LG Nexus 5 – Android 4.4 (KitKat) – Chrome (v30.0.1.5999.105)
  • Kindle Fire – Android v2.3 (customised for Amazon Kindle) – Amazon Silk
  • BlackBerry Curve 9320 – BlackBerry OS 7.1 – Blackberry Browser
  • Nexus 10
  • Nokia Lumia 820 – Windows Phone 8 (8.0.10328.78) – Internet Explorer 10
  • Blackberry Z10 – Blackberry 10 (10.2.0.424) – Evolution
  • Surface – Windows RT – Internet Explorer 10