Netflix migration to the cloud

Have you ever pondered the sheer magnitude of Netflix as a company and contemplated the intricate scaling demands their software must meet to serve tens of millions of users concurrently?

As you may be aware, Netflix relies on cloud computing to deliver its vast library of movies and series. However, what you may not know is that Netflix’s journey to the cloud is relatively recent, as they previously operated using on-premises infrastructure. In this section, we will unravel the story behind Netflix’s transition to the cloud.

Migrating Netflix

The story that follows is condensed from Netflix’s article detailing their migration from on-premises systems, a transformation spanning the years from 2008 to 2016. You can explore the full article here
for a more in-depth understanding.

Netflix, the colossal video streaming service we know today, entertains well over 200 million subscribers across more than 190 countries. However, when it was founded in 2007, Netflix wasn’t always like that. Initially, Netflix operated as an online DVD rental service, literally mailing physical DVDs to customers’ homes.

During this early period, Netflix managed its own data centres (a huge space with many computers acting as servers to power up a system or website), and in 2008, they suffered a major database corruption which as a result had the entire service shut down for three whole days. Such outages are a significant business loss for companies of this scale (both financially but also in terms of customer experience). Faced with this critical issue, Netflix’s leadership had to choose from the following two choices:

1. Invest in expanding their workforce and fortifying their data centres to enhance resilience and robustness
2. Embrace the cloud and migrate their entire service to this scalable platform.

Netflix was experiencing rapid growth, and the first option would have posed substantial difficulties in meeting the surging demand. It became evident that the cloud presented the most viable solution looking forward.

The migration journey commenced in 2008, unfolding gradually as Netflix strategised the optimal approach to transition their operations to the cloud. The process culminated in 2016 when Netflix closed its entire data centre infrastructure. At this point, Netflix was entirely in the cloud.

According to Netflix, their growth from 2007 to 2015 was a staggering 1,000-fold. They underscored that “supporting such rapid growth would have been extremely difficult out of our own data centres; we simply could not have racked the servers fast enough.” The cloud empowered Netflix to swiftly expand its services to accommodate the burgeoning demand of millions of users, often in a matter of minutes.

According to Netflix, they rely on the cloud for (among other things):

• computing and storage needs
• business logic
• databases (storing relational data)
• big data processing and analytics
• recommendations
• transcoding (converting media to different resolutions/encodings, for example)
• sending millions of emails to subscribers.

Netflix acknowledges that the journey to fully embrace the cloud involved “a lot of hard work, and we had to make a number of difficult choices along the way.” This statement highlights the fact that migrating to the cloud necessitates a shift towards a cloud-native approach, entailing the rebuilding of technology infrastructure and a fundamental transformation in operational strategies.

It’s crucial to reiterate a point discussed in a previous step: adopting the cloud does not inherently guarantee resilience, robustness, or cost-efficiency. Achieving these benefits often requires the comprehensive reevaluation and, in some cases, the complete reconstruction of existing systems to leverage cloud technologies. Simply moving an existing system to the cloud does not guarantee resilience, robustness, or cost-efficiency.

Netflix Open Connect: sometimes cloud offering isn’t enough

While cloud offerings provide robust solutions, they may not always meet every requirement, particularly for companies of Netflix’s scale. Some organisations opt to implement their systems despite the availability of equivalent cloud services. Netflix, for example, introduced its Content Delivery Network (CDN) called Open Connect, specifically optimised to deliver the best viewing experience.

A CDN is essentially a distributed network of servers designed to distribute content (e.g. webpages, images, videos) by caching it in proximity to end-users. This caching strategy facilitates rapid data transfer and ensures high service availability. For you, it means that the time it takes for a movie to load on your TV by the time you click on one is minimal.

Imagine the internet as a vast highway, with movies, videos, images, and other online content travelling on this “motorway”. This digital “motorway” is shared by numerous companies, organisations and users, not just Netflix. Netflix typically streams movies during “rush hours”, when internet usage peaks. To expedite this process, Open Connect stores movies on servers located in neighbourhoods (a concept known as caching), ensuring that the content is readily available locally through your Internet Service Provider (ISP; the company you pay to have internet at home) rather than traversing the congested public “motorway”, i.e. the internet.

In essence, Netflix has created its private express lanes to deliver movies to your doorstep more efficiently, bypassing the usual internet traffic congestion. During off-peak hours, Netflix leverages this infrastructure to deliver new movies and catalogue updates to your neighbourhood, ensuring swift access when you’re ready to watch.

In summary, Netflix’s unique requirements compelled them to develop their CDN, emphasising the importance of tailoring solutions to specific needs, even when cloud services are available. It’s important to note, however, that this approach is relatively uncommon and often beyond the reach of both small and even large companies, primarily due to the intricacies of maintaining such infrastructure and the associated costs involved.

The next step will give you the opportunity to test your understanding of the activity so far.