Getting to beta service assessment

It’s important to focus on the leanest version of a service during alpha

Last week we blogged about completing an alpha and getting ready to move into the beta phase. This post looks at moving into the beta phase and getting to beta service assessment.

Start small

In the early stages of a beta, it’s important to focus on the leanest version of a service that can meet the needs you identified at alpha. That means you can start getting feedback from users quickly to inform further iterations and extensions of that initial, minimal service.

For Teaching Vacancies, we revisited our storymap and worked as a team to prioritise the parts of the service which were crucial for users to complete a core task. That included things like listing a new job or completing a search for relevant listings. It also helped us to think about which features we could look at later on to improve the experience for users or support other related tasks.

For a minimum viable product that we could take into a private beta, we identified two high-level priorities:  

We also identified further work to understand more about how hiring staff put together job listings, and if and how application packs and detailed prospectuses should feature in the service.

A clear vision

With this scope and our learnings from alpha, we were able to set a clear vision for the beta, and articulate the goals and objectives that would help us realise that vision.

‘Vision: a national search and listing service for teacher vacancies’
Vision: a national search and listing service for teacher vacancies

Setting goals

We set goals outlining our thinking on what that service would provide:

Based on our alpha findings and storymapping, and on our vision and goals for beta, we deprioritised some features that we’d explored and prototyped in alpha. Whilst these extra features made it easier for jobseekers to find roles that met their specific criteria, users could still complete searches and find appropriate roles without these enhancements.

We’ve since implemented some of these in the beta service as part of our continuous iteration, including search by distance from a given location, a filter for jobs which are suitable for newly qualified teachers, sign-in access for school hiring staff that are responsible for multiple schools, and the ability to share a posted job listing on social media.  

Regular iteration isn’t just for alpha  

We continued to learn and adapt the service throughout beta, and you can see some examples of the changes we’ve made and the findings they’re based on in our open code repository. We tagged ‘design decisions’ to help us track our iterations over time.

Making a service for everyone

Once we had an end to end working service for the core journeys we’d prioritised, we placed extra emphasis on our usability testing: making sure that we included users who placed themselves lower on the digital inclusion scale, and those with access needs or using assistive technologies.

This is an important part of beta development, ensuring the service we build will work for all users. It also helped us think about what support users would need, and how we’d provide that in a way that was accessible to everyone. This isn’t something we see as an added extra, accessibility is now a legal requirement for public sector websites. It’s also a really good way of ensuring good design for everyone, regardless of any specific needs.

Moving into private beta and beyond

With a functioning, accessible minimum viable product, it’s time to take the service into private beta.

We worked with colleagues at the Department for Education to work out what characteristics we needed to capture to make sure our private beta users would give a representative picture of our whole user base. We looked at data including school type and age range, geographical location and socioeconomic indicators, and built a private beta user base of 60 schools.

Follow-up research with these private beta participants, and analytics from their use of the service, helped us identify the fixes or improvements we needed to make before releasing the service to more users, and informed our thinking around scaling and support.

The service passed its beta service assessment in October 2018 and is now used by thousands of schools across the country, listing hundreds of jobs. You can read more about how we rolled the service out in this blog post.