« Device Lab progress | Main | projectPlutus – a creative hack weekend »
Friday
Nov222013

Building a device lab

At the Money Advice Service we're currently working on a series of web improvement initiatives. This includes reviewing our existing taxonomy, site search strategy and also building a new responsive front end. For the past few months the tools we’ve developed in-house have been built in a responsive manner, however once live they’ve lived within an unresponsive frame. The enhancements from the web improvements initiative will change this, and along with our goal of being ‘mobile first’ we’ve been thinking about the tools we need to help design, develop and test across small screen devices.

One of the things we’re doing is building a physical device lab that’ll complement the use of online emulators. This is at a fairly early stage, but over the course of the next few weeks we'll be moving forward and I’ll update with progress. For now I thought it might be useful to write some notes around why we’re building a device lab, how we decided on the devices to start with, and what our aspirations are for the future.

Why 
There are a number of reasons why we’ve opted to invest in a physical device lab over relying purely on emulators. Emulators definitely have a place and will be a part of our workflow. They provide good coverage and can be quick and easy to use, especially on static, content based sites. However having access to the physical device provides a number of advantages:

1 – Firstly, it’ll be no great surprise that the number of visits to the site on small screen devices has increased rapidly. Looking at analytics data* there’s been an average increase of 398% year-on-year. The increased usage meant that optimising the site for small screen devices became a growing priority, and with that the need for more effective testing.

 2 – an emulator can replicate how a site will look given a device’s particular capabilities (screen resolution, browser support etc) but some of the subtilise around how the site will behaviour on a device can be lost, for example the responsiveness of a touch screen or having to navigate using a trackball or keys.

3 - the third advantage is that having physical devices available on site really re-enforces the mobile-first mindset right across the team, from product owners, to designers, to dev and QA. It means that everyone can look at the latest product developments across a variety of devices rather than just the thing that they have in their pocket or on their desk.

Getting the device lab set-up
Having decided that we wanted to a physical device lab the next question was what devices to get. Thankfully the interwebs came to the rescue and we were able to look at plenty of useful resources from the likes of Jeremy Keith, PPK and Stephanie Rieger. They’ve all written a bunch of stuff on the topic, and are much more knowledgable than me, but I figured I’d summarise a few of the things we found useful.

1 – in the first instance we looked at existing traffic to the site to see what the most common devices, OS, and browsers were within our existing audience. The data showed us that when it came to small screen devices the vast majority of traffic came via Apple iPad and Apple iPhone devices (36% and 33% respectively) with a further 7 devices making up 80% of our total mobile traffic. Looking the operating systems, 80% of our mobile traffic came from 12 devices, of which 9 were different variants of Apple's iOS. The browsers hitting the site also told a very similar story.

2 – looking at out own data analytics provided a great snapshot of our immediate audience, however we also wanted to consider the wider device/browser landscape. At the Money Advice Service our target audience is the whole of the UK and our goal is to make everything we produce available to as many people, in as many ways, as possible. Previous experience has shown that in making a site optimised for mobile then traffic from mobile devices is likely to increase and broaden. To help get a wider view we looked at data from GDS and data.gov.uk.

3 – with a view of device traffic to both our site and our target audience we were confronted with a fairly daunting list of potential devices. Through looking at existing device labs we were able to take some inspiration as to the best combination of devices for us to start out with. Combing this information from the data we’d captured from the first two points gave us a good starting point.

4 – finally, we looked at the devices we already had in the office. Having a set budget we wanted to be frugal with our spending. The most common devices we wanted to form part of a dedicate lab, but some of the less popular devices we didn’t want to duplicate what we already had in the office knowing they could probably be called on for testing purposes as and when needed.

Where we are now, and where we want to be
Right now we’re at the point of starting to collect devices. Newer ones have been ordered, and for some of the older models we’re scouring eBay and 2nd-hand electronic shops. By Christmas we should have the basis of a good device lab.

We’re also starting to figure out a way of making testing as easy as possible. We'll probably using Adobe Edge Inspect and Forward to help with this, but very keen to get any tips or suggestions! There’s also the logistics of how to manage devices, store them, and maintain the right version of operating system. As we figure some of this stuff out I plan to write a follow-up post on some lessons learnt.

Our aspiration for the future is to be able to open up our device lab so that others can also benefit from it too. The potential of being able to share access to devices and with that learn from each other about problems we’re facing and ways to overcome them is hugely exciting. We’re in the process of starting an office refurbishment that’s due to be completed in the spring of 2014, so hopefully that’ll give us the opportunity to set something up for then.   


* data taken from April 2010 through November 2013

References (83)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cloud authority
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cloud authority
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cloudauthority.org
    mattlucht.com - blog - Building a device lab
  • Response
    Response: Going in l0.kz
    mattlucht.com - blog - Building a device lab
  • Response
    Response: her latest blog
    mattlucht.com - blog - Building a device lab
  • Response
  • Response
    Response: www.amazon.com
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: how you can help
    mattlucht.com - blog - Building a device lab
  • Response
    Response: bola
    mattlucht.com - blog - Building a device lab
  • Response
    Response: buy email list
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: bursa taruhan
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: navigate here
    mattlucht.com - blog - Building a device lab
  • Response
    Response: judi poker online
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cerita sex
    mattlucht.com - blog - Building a device lab
  • Response
    Response: visit website
    mattlucht.com - blog - Building a device lab
  • Response
    Response: harga mesin bubut
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: vegas338
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: pescollection
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cerita sex
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: judi bola
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cara deposit
    mattlucht.com - blog - Building a device lab
  • Response
    Response: go to website
    mattlucht.com - blog - Building a device lab
  • Response
    Response: agen tangkasnet
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cara daftar sbobet
    mattlucht.com - blog - Building a device lab
  • Response
    Response: prediksi-jitu
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: judi online
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: forex market
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: link web site
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: GoPro Suction Cup
    mattlucht.com - blog - Building a device lab
  • Response
    Response: igo hot
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: tipos de celulitis
    mattlucht.com - blog - Building a device lab
  • Response
    Response: que es celulitis
    mattlucht.com - blog - Building a device lab
  • Response
    Response: 64gb Micro SD
    mattlucht.com - blog - Building a device lab
  • Response
    Response: watch here
    mattlucht.com - blog - Building a device lab
  • Response
    Response: gopro pole mount
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: GoPro Accessories
    mattlucht.com - blog - Building a device lab
  • Response
    Response: website
    mattlucht.com - blog - Building a device lab
  • Response
    Response: GoPro Attachments
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: Blog Semua Suka
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: Blog Semua Suka
    mattlucht.com - blog - Building a device lab
  • Response
    Response: visit these guys
    mattlucht.com - blog - Building a device lab
  • Response
    Response: go here
    mattlucht.com - blog - Building a device lab
  • Response
    Response: GoPro Selfie Stick
    mattlucht.com - blog - Building a device lab
  • Response
    Response: cerita dewasa
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: essay helper
    mattlucht.com - blog - Building a device lab
  • Response
    is a Buddhist Qibao agate. weigh weight: gold for the pr TRX Force Kit Outlet oportion of Sky Blue Kanken Mini 19. TRX Force Kit Outlet 1, 4, don't use the mop silver Sky Blue Kanken Mini milk and TRX Professional Door Anchor wash water, 2, the size of ...
  • Response
    stciker-sampai-hati
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    mattlucht.com - blog - Building a device lab
  • Response
    Response: Go Pro Remote
    mattlucht.com - blog - Building a device lab

Reader Comments (4)

Hey I know that PhoneGap used to use their own tool Weinre http://people.apache.org/~pmuellr/weinre/docs/latest/ to test across over 50+ devices at the same time.

November 25, 2013 | Unregistered CommenterBenJam

Thanks @BenJam, I'll take a look. It sounds as though it could be quite a useful tool.

November 25, 2013 | Unregistered CommenterMatt Lucht

You might be interested in checking out Ghostlab, which our team found out about recently. We haven't done much in-depth work with it yet, but our initial run-through with it seems to be promising. You also might be able to do multi-device testing by getting creative with Grunt: http://blog.mattbailey.co/post/50337824984/grunt-synchronised-testing-between-browsers-devices

November 25, 2013 | Unregistered CommenterPatrick

Awesome, thanks @Patrick – I'll give Ghostlab and Grunt a go and see how we get on.

November 25, 2013 | Unregistered CommenterMatt Lucht

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>