You can now search for courses you're interested in on FutureLearn. In this post, our Product Manager, Keira Poland, explains the new feature, how we developed it and what’s next.
You can now search for courses you’re interested in on FutureLearn. In this post, our Product Manager, Keira Poland, explains the new feature, how we developed it and what’s next.
Ever since the number of courses we offer reached double digits, people have suggested ways to make it easier to find the course they want. As a result, we’ve introduced some of these ideas – including course categories and sorting by upcoming, in progress and to be announced – to make it easier to find a course.
But as the number of available courses has grown, the number of requests for a search function has increased too – requests from our learners, partners and even the FutureLearn team. In almost any digital experience, searching for something you’re looking for has now become a basic expectation for a lot of us and we believe that FutureLearn is no different.
Our hypothesis about search
Our hypothesis, shared by most of you, was that introducing ‘Search to find a course’ would make finding a course on a topic you care about simpler, quicker and more satisfying, and would ultimately help more people join and start learning with FutureLearn.
As of last week, we’ve made this search functionality visible to everyone who visits FutureLearn (click on the magnifying glass at the top of most pages or visit www.futurelearn.com/search to try it out.) But the process of designing, developing and testing search, which we’ve been through over the last three months, has often made us question our hypothesis.
How did we start with search?
We knew that we wanted to introduce search, but the questions we couldn’t immediately answer were “Where?” and “How?” Where would search most benefit our learners in finding a course? What design and interaction would provide the best experience?
We decided to design a number of ideas and then try them out using an A/B testing approach, so that we could learn which one would be our best starting position.
What is A/B testing?
A/B testing – also known as split testing – allows you to test two ideas side by side at the same time. Typically, one of the two ideas is your original design, while the second is the result of a hypothesis that you think would improve a specific outcome. In this case we were testing the existing FutureLearn site with and without search, to understand the impact on course enrolments.
The test works by splitting the audience that visits a page into two groups. As a new visitor visits the page, they are randomly assigned to one of the two groups (let’s call them Group A and Group B) until there is a 50/50 split between them. As the page loads, Group A sees Design A and Group B sees Design B. At the same time, we are able to record which group each visitor is in, and whether or not they decide to enrol on a course, which helps us figure out which design is performing best.
What did we learn about search?
Our first test introduced a search bar on our main courses page. We knew that most visitors to this page would be looking to find a course, so we guessed that search would have most benefit here too. Seems like a fair assumption, yes?
What we actually found was that surfacing search in this way resulted in a small but noticeable drop in the rate of course enrolments. “No way!” we cried, “How could this possibly be?”
We scratched our heads, removed the search bar, and went back to the data to uncover some home truths. What we found was really interesting: people who were offered search and used search (Group B1) were equally as likely to enrol as people who were not shown search at all (Group A). It was the people who were offered search and did not use search (Group B2) who were less likely to enrol and whose activity was contributing to the drop in conversion that we were seeing.
For some reason, search was getting in the way of finding a course for those learners who didn’t want to use it.
Testing another approach to search
We still believed that search would benefit learners, especially those with a clear idea of what they were looking for, so we refocused our efforts on how to introduce the feature without distracting people who preferred to browse our courses. We settled on testing search in our main navigation that sits atop most pages of the FutureLearn experience.
Happily for our learners, our partners and ourselves, the results of this test were more encouraging. There was no significant difference in the rate at which people enrolled, whether they were shown a version of the site with or without search. While the results weren’t exactly the ringing endorsement we were hoping for, we are now more confident that we’ve found a way to surface search that we can build on and improve.
What next for search?
This is where the fun begins! Now that search is live and people can use it everyday, we can start to understand more about how and why they use it, so that we can make it better in the future. We’re already busy analysing data from search itself and talking to learners about their experience of using it. This will help us identify any pain points in using search and prioritise what changes we need to make next.
We’d love to hear what you think about search on FutureLearn and any ideas you have to improve it. Drop us a note in the comments below to share your thoughts.