prime video case study serverless

InfoQ Software Architects' Newsletter

A monthly overview of things you need to know as an architect or aspiring architect.

View an example

We protect your privacy.

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

• English edition
• Chinese edition
• Japanese edition
• French edition

Back to login

Login with:

Don't have an infoq account, helpful links.

• About InfoQ
• InfoQ Editors

Write for InfoQ

• About C4Media

Choose your language

Special Memorial Day Sale with significant discounts of up to 60% off . Register now.

Get practical advice from senior developers to navigate your current dev challenges. Use code LIMITEDOFFERIDSMUNICH24 for an exclusive offer.

Level up your software skills by uncovering the emerging trends you should focus on. Register now.

Your monthly guide to all the topics, technologies and techniques that every professional needs to know about. Subscribe for free.

InfoQ Homepage News Prime Video Switched from Serverless to EC2 and ECS to Save Costs

Prime Video Switched from Serverless to EC2 and ECS to Save Costs

This item in japanese

May 03, 2023 2 min read

Rafal Gancarz

Prime Video, Amazon's video streaming service, has explained how it re-architected the audio/video quality inspection solution to reduce operational costs and address scalability problems . It moved the workload to EC2 and ECS compute services, and achieved a 90% reduction in operational costs as a result.

The Video Quality Analysis (VQA) team at Prime Video created the original tool for inspecting the quality of audio/video streams that was able to detect various user experience quality issues and trigger appropriate repair actions.

The initial architecture of the solution was based on microservices responsible for executing steps of the overall analysis process, implemented on top of the serverless infrastructure stack. The microservices included splitting audio/video streams into video frames or decrypted audio buffers as well as detecting various stream defects by analyzing frames and audio buffers using machine-learning algorithms . AWS step functions were used as a primary process orchestration mechanism, coordinating the execution of several lambda functions . All audio/video data, including intermediate work items, were stored in AWS S3 buckets and an AWS SNS topic was used to deliver analysis results.

Source: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

The team designed the distributed architecture to allow for horizontal scalability and leveraged serverless computing and storage to achieve faster implementation timelines. After operating the solution for a while, they started running into problems as the architecture has proven to only support around 5% of the expected load.

Marcin Kolny , senior software engineer at Prime Video, shares his team's assessment of the original architecture:

While onboarding more streams to the service, we noticed that running the infrastructure at a high scale was very expensive. We also noticed scaling bottlenecks that prevented us from monitoring thousands of streams. So, we took a step back and revisited the architecture of the existing service, focusing on the cost and scaling bottlenecks.

The problem of high operational cost was caused by a high volume of read/writes to the S3 bucket storing intermediate work items (video frames and audio buffers) and a large number of step function state transitions.

The other challenge was due to reaching the account limit of the overall number of state transitions, because the orchestration of the process involved several state transitions for each second of the analyzed audio/video stream.

In the end, the team decided to consolidate all of the business logic in a single application process. Kolny summarizes the revised design:

We realized that distributed approach wasn't bringing a lot of benefits in our specific use case, so we packed all of the components into a single process. This eliminated the need for the S3 bucket as the intermediate storage for video frames because our data transfer now happens in the memory. We also implemented orchestration that controls components within a single instance.

The resulting architecture had the entire stream analysis process running in ECS on EC2, with groups of detector instances distributed across different ECS tasks to avoid hitting vertical scaling limits when adding new detector types.

After rolling out the revised architecture, the Prime Video team was able to massively reduce costs (by 90%) but also ensure future cost savings by leveraging EC2 cost savings plans. Changes to the architecture have also addressed unforeseen scalability limitations that prevented the solution from handling all streams viewed by customers.

About the Author

Rate this article, this content is in the cloud computing topic, related topics:.

• Architecture & Design
• Cloud Architecture
• Architecture
• Cloud Computing

Related Editorial

Related sponsored content, popular across infoq, uber migrates 1 trillion records from dynamodb to ledgerstore to save $6 million annually, deepthi sigireddi on distributed database architecture in the cloud native era, ahrefs joins others in suggesting that on-premises hosting can be more cost effective than cloud, architecture modernization with nick tune, from mainframes to microservices - the journey of building and running software, java news roundup: openjdk updates, piranha cloud, spring data 2024.0.0, glassfish, micrometer, related content, the infoq newsletter.

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%

The move from a distributed microservices architecture to a monolith application helped achieve higher scale, resilience, and reduce costs..

At Prime Video, we offer thousands of live streams to our customers. To ensure that customers seamlessly receive content, Prime Video set up a tool to monitor every stream viewed by customers. This tool allows us to automatically identify perceptual quality issues (for example, block corruption or audio/video sync problems) and trigger a process to fix them.

Our Video Quality Analysis (VQA) team at Prime Video already owned a tool for audio/video quality inspection, but we never intended nor designed it to run at high scale (our target was to monitor thousands of concurrent streams and grow that number over time). While onboarding more streams to the service, we noticed that running the infrastructure at a high scale was very expensive. We also noticed scaling bottlenecks that prevented us from monitoring thousands of streams. So, we took a step back and revisited the architecture of the existing service, focusing on the cost and scaling bottlenecks.

The initial version of our service consisted of distributed components that were orchestrated by AWS Step Functions . The two most expensive operations in terms of cost were the orchestration workflow and when data passed between distributed components. To address this, we moved all components into a single process to keep the data transfer within the process memory, which also simplified the orchestration logic. Because we compiled all the operations into a single process, we could rely on scalable Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Elastic Container Service (Amazon ECS) instances for the deployment.

Distributed systems overhead

Our service consists of three major components. The media converter converts input audio/video streams to frames or decrypted audio buffers that are sent to detectors. Defect detectors execute algorithms that analyze frames and audio buffers in real-time looking for defects (such as video freeze, block corruption, or audio/video synchronization problems) and send real-time notifications whenever a defect is found. For more information about this topic, see our How Prime Video uses machine learning to ensure video quality article. The third component provides orchestration that controls the flow in the service.

We designed our initial solution as a distributed system using serverless components (for example, AWS Step Functions or AWS Lambda ), which was a good choice for building the service quickly. In theory, this would allow us to scale each service component independently. However, the way we used some components caused us to hit a hard scaling limit at around 5% of the expected load. Also, the overall cost of all the building blocks was too high to accept the solution at a large scale.

The following diagram shows the serverless architecture of our service.

The initial architecture of our defect detection system.

The main scaling bottleneck in the architecture was the orchestration management that was implemented using AWS Step Functions. Our service performed multiple state transitions for every second of the stream, so we quickly reached account limits. Besides that, AWS Step Functions charges users per state transition.

The second cost problem we discovered was about the way we were passing video frames (images) around different components. To reduce computationally expensive video conversion jobs, we built a microservice that splits videos into frames and temporarily uploads images to an Amazon Simple Storage Service (Amazon S3) bucket. Defect detectors (where each of them also runs as a separate microservice) then download images and processed it concurrently using AWS Lambda. However, the high number of Tier-1 calls to the S3 bucket was expensive.

From distributed microservices to a monolith application

To address the bottlenecks, we initially considered fixing problems separately to reduce cost and increase scaling capabilities. We experimented and took a bold decision: we decided to rearchitect our infrastructure.

We realized that distributed approach wasn’t bringing a lot of benefits in our specific use case, so we packed all of the components into a single process. This eliminated the need for the S3 bucket as the intermediate storage for video frames because our data transfer now happened in the memory. We also implemented orchestration that controls components within a single instance.

The following diagram shows the architecture of the system after migrating to the monolith.

The updated architecture for monitoring a system with all components running inside a single Amazon ECS task.

Conceptually, the high-level architecture remained the same. We still have exactly the same components as we had in the initial design (media conversion, detectors, or orchestration). This allowed us to reuse a lot of code and quickly migrate to a new architecture.

In the initial design, we could scale several detectors horizontally, as each of them ran as a separate microservice (so adding a new detector required creating a new microservice and plug it in to the orchestration). However, in our new approach the number of detectors only scale vertically because they all run within the same instance. Our team regularly adds more detectors to the service and we already exceeded the capacity of a single instance. To overcome this problem, we cloned the service multiple times, parametrizing each copy with a different subset of detectors. We also implemented a lightweight orchestration layer to distribute customer requests.

The following diagram shows our solution for deploying detectors when the capacity of a single instance is exceeded.

Our approach for deploying more detectors to the service.

Results and takeaways

Microservices and serverless components are tools that do work at high scale, but whether to use them over monolith has to be made on a case-by-case basis.

Moving our service to a monolith reduced our infrastructure cost by over 90%. It also increased our scaling capabilities. Today, we’re able to handle thousands of streams and we still have capacity to scale the service even further. Moving the solution to Amazon EC2 and Amazon ECS also allowed us to use the Amazon EC2 compute saving plans that will help drive costs down even further.

Some decisions we’ve taken are not obvious but they resulted in significant improvements. For example, we replicated a computationally expensive media conversion process and placed it closer to the detectors. Whereas running media conversion once and caching its outcome might be considered to be a cheaper option, we found this not be a cost-effective approach.

The changes we’ve made allow Prime Video to monitor all streams viewed by our customers and not just the ones with the highest number of viewers. This approach results in even higher quality and an even better customer experience.

• Artificial Intelligence
• Generative AI
• Cloud Computing
• CPUs and Processors
• Data Center
• Edge Computing
• Enterprise Storage
• Virtualization
• Internet of Things
• Network Management Software
• Network Security
• Enterprise Buyer’s Guides
• United States
• Newsletters
• Foundry Careers
• Terms of Service
• Privacy Policy
• Cookie Policy
• Copyright Notice
• Member Preferences
• About AdChoices
• E-commerce Links
• Your California Privacy Rights

Our Network

• Computerworld

6 lessons from the Amazon Prime Video serverless vs. monolith flap

Prime video developers fine-tuned their microservices architecture to address underlying issues that enlisting network expertise might have caught earlier..

A software-development team caused quite a stir recently with a blog post describing how it abandoned a serverless architecture project in favor of a monolith—and slashed cloud infrastructure costs by 90% in the process.

But this wasn’t just any team; the post was written by Marcin Kolny, a senior software-development engineer at Amazon Prime Video.

Since Amazon is one of the leading advocates for serverless computing , not to mention the market leader in cloud services, the post was viewed as either a commendable act of openness or the very definition of throwing your company under the bus. Either way, it triggered a passionate back and forth on social media platforms that focused on larger questions:

• Has the whole serverless/microservices/service-oriented architecture (SOA) movement been overhyped?
• Has the traditional monolithic approach to software development been underestimated?
• Is it time for a market correction similar to what we’re seeing with cloud in general, where some companies are moving apps from the cloud back to the data center and rethinking their cloud-first strategies?

Now that the dust has settled a bit, a closer examination of the Prime Video team’s experience reveals some key lessons that enterprises can apply going forward. But also importantly, the issues they faced highlight the need for early input from networking pros when the application-planning process is just getting underway.

What went wrong?

The first question that needs to be asked is: Was this an edge case, an outlier, or does it have broader implications in general? The Amazon team was dealing with real-time video streams, so not exactly your average enterprise app, but the takeaways are universal to any development process involving data-intensive, low-latency applications.

Prime Video was building a tool to analyze video streams for quality issues, such as video freezes or lack of synchronization between audio and video. In a complex, multi-step process, a media converter broke the streams into video frames and audio buffers that were then sent to defect detectors. Each defect detector, software that uses algorithms to identify defects and send real-time notifications, was running as its own microservice .

Two problems became apparent as the team began to scale the application: there were too many expensive calls to Amazon S3 storage, and the process was difficult to orchestrate.

The Amazon Prime Video team explained, “We designed our initial solution as a distributed system using serverless components (for example, AWS Step Functions or AWS Lambda), which was a good choice for building the service quickly. In theory, this would allow us to scale each service component independently.”

“However, the way we used some components caused us to hit a hard scaling limit at around 5% of the expected load. Also, the overall cost of all the building blocks was too high to accept the solution at a large scale. To address this, we moved all components into a single process to keep the data transfer within the process memory, which also simplified the orchestration logic.”

The high-level architecture remained the same, and the original code was able to be reused and was quickly migrated to the new architecture, which consolidated the workflow into a single Amazon Elastic Container Service (ECS) task.  

“Moving our service to a monolith reduced our infrastructure cost by over 90%. It also increased our scaling capabilities. Today, we’re able to handle thousands of streams and we still have capacity to scale the service even further,” the team wrote.

Reactions run the gamut

The post triggered lengthy discussions on social media. David Heinemeier Hansson, co-owner and CTO at SaaS vendor 37signals, was quick to jump into the fray. Hansson caused something of a stir himself recently when he decided to pull his company’s applications and data out of the Amazon public cloud.

Hansson fired off a blog post that took this basic position: “I won’t deny there may well be cases where a microservices-first architecture makes sense, but I think they’re few and far in between. The vast majority of systems are much better served by starting and staying with a majestic monolith. ”

Hansson argues that the microservices/SOA approach works for large enterprises and hyperscalers, but not necessarily for smaller organizations. “If you’re Amazon or Google or any other software organization with thousands of developers, it’s a wonderful way to parallelize opportunities for improvement. Each service can be its own team with its own timeline, staff, and objectives. It can evolve independently, at least somewhat, of whatever else the rest of the constellation is doing. When you reach a certain scale, there simply is no other reasonable way to make coordination of effort happen. Otherwise, everyone will step on each other’s feet, and you’ll have to deal with endless merge conflicts.”

But the problem with breaking an application into multiple pieces is that it increases complexity. “Every time you extract a collaboration between objects to a collaboration between systems, you’re accepting a world of hurt with a myriad of liabilities and failure states,” Hansson says.

He adds that in today’s tech culture, the traditional monolithic application has become “a point of derision.” But he wants the culture to “embrace the monolith with pride.”

His definition of a monolith is “an integrated system that collapses as many unnecessary conceptual models as possible, eliminates as much needless abstraction as you can swing a hammer at. It’s a big fat ‘no’ to distributing your system lest it truly prevents you from doing what really needs to be done.”

Adrian Cockcroft, an industry veteran whose resume includes stints at Sun Microsystems, eBay, Netflix, Battery Ventures and AWS, weighed in with a different take.

He argues that the Prime Video team essentially used inaccurate terminology; they didn’t really go back to a monolith; they were simply refactoring their initial implementation, which Cockcroft describes as a best practice. 

Cockcroft says, “The Prime Video team followed a path I call Serverless First, where the first try at building something is put together with Step Functions and Lambda calls. When you are exploring how to construct something, building a prototype in a few days or weeks is a good approach. Then they tried to scale it to cope with high traffic and discovered that some of the state transitions in their step functions were too frequent, and they had some overly chatty calls between AWS Lambda functions and S3. They were able to re-use most of their working code by combining it into a single long-running microservice that is horizontally scaled using ECS, and which is invoked via a Lambda function. The problem is that they called this refactoring a microservice-to-monolith transition, when it’s clearly a microservice-refactoring step and is exactly what I recommend people do.” 

Cockroft does agree that microservices have been somewhat oversold, and there has been some backlash as organizations realize that “the complexity of Kubernetes has a cost, which you don’t need unless you are running at scale with a large team.”

He adds, “I don’t advocate ‘serverless only’, and I recommended that if you need sustained high traffic, low latency, and higher efficiency, then you should re-implement your rapid prototype as a continuously running autoscaled container, as part of a larger serverless-event driven architecture, which is what they did.”

6 takeaways IT pros should remember

There important lessons that enterprise IT leaders can learn from the Amazon Prime Video example.

1. It’s not about the technology

“Don’t start with technology; start with goals,” recommends Pavel Despot, senior product marketing manager at Akamai. “Start with what you want to accomplish and build for the requirements presented.”

Vijay Nayar, founder and CEO at Funnel-Labs.io, agrees. “If you approach a problem and state that microservices or a monolith system is or isn’t the answer before you’ve even heard the problem, you’re shooting first and then asking questions. It’s reckless and leads to bad decision making.”

2. Analyze the trade-offs

Microservices bring flexibility, enabling independent development, deployment, and scalability of individual services. They also introduce complexity, including the need for service discovery, inter-service communication, and managing distributed systems.

Going the serverless route has the advantage of fast deployment because the underlying infrastructure upon which you’re building the application is spun up by the service provider on demand.

3. The original design has to be right

The underlying architecture that the application will run on has to be correct in the first place, or else any attempt to move from prototype to production will run into scaling problems.

David Gatti, an AWS specialist and CTO, says, “If you design the architecture incorrectly, it won’t work, will be expensive, and complicated. The idea of passing data from one Lambda to another to do some work is not a great idea; do all the processing in one Lambda.” He says the Amazon Prime Video team “made a bad design based on the workload needed, and now they are doing the right thing. This does not mean that all serverless is bad.”

4. Simplify languages and dependencies

Hansson says that “one of the terrible side effects of microservices madness is the tendency to embrace a million different programming languages, frameworks, and ecosystems.” He recommends no more than two languages; one tuned for productivity that can be used the vast majority of the time, and a second high-performance language used for addressing hot spots.

Nayar adds, “If you split your service into 100 different tiny services, and you can’t figure out where problems emerge from, and the spiderweb of dependencies among them makes deployment a nightmare, then that’s because you split your services without thinking about keeping their purpose clear and their logic orthogonal.

5. Target specific use cases

Cockcroft says enterprise workloads that are intermittent and small scale are good candidates for the serverless approach using Amazon Step Functions and Lambda.

“When microservices are done right, they often target a narrow, isolated, and usually performance-critical segment of the system,” adds Hansson.

And Despot points out that while the serverless approach provides flexibility, the requirement that microservices talk to each other and to backend databases can impact latency.

6. Consider cost

Because the providers of serverless computing charge for the amount of time code is running, it might not be cost effective to run an application with long-running processes in a serverless environment.

And then there’s the lesson that the Amazon Prime Video team learned: storage costs can bite you if you’re not careful. AWS storage pricing is based on tiers, with Tier 1 fast-access more expensive than slower tiers. On top of that, customers are charged for every data request and data transfer, so overly chatty applications can rack up charges pretty quickly.

Related content

T-mobile deal could mean advantages for uscellular customers, 2024 global network outage report and internet health check, how to deploy wpa3 for enhanced wireless security, ampere updates roadmap, heads to 256 cores, newsletter promo module test.

Neal Weinberg is a freelance technology writer and editor. He can be reached at [email protected] .

More from this author

Download our hybrid cloud data protection enterprise buyer’s guide, download our sase and sse enterprise buyer’s guide, hpe-juniper’s ai story resonates, but customer concerns linger, what is iot the internet of things explained, most popular authors.

Show me more

Elon musk’s xai to build supercomputer to power next-gen grok.

Regulators sound out users on cloud services competition concerns

Backgrounding and foregrounding processes in the Linux terminal

Has the hype around ‘Internet of Things’ paid off? | Ep. 145

Episode 1: Understanding Cisco’s Converged SDN Transport

Episode 2: Pluggable Optics and the Internet for the Future

Has the hype around ‘Internet of Things’ paid off?

Are unused IPv4 addresses a secret gold mine?

Preparing for a 6G wireless world: Exciting changes coming to the wireless industry

DEV Community

Posted on May 4, 2023

Microservices May Not Always Be the Answer: Lessons from Amazon Prime Video

Amazon Prime Video recently shared a case study about their decision to replace their serverless, microservices architecture with a monolith. The result? A 90% reduction in operating costs and a simpler system. This is a big win and it highlights a larger point for our industry: microservices may not always be the answer.

The Prime Video team initially designed their system as a distributed system using serverless components, with the idea that they could scale each component independently. However, they hit a hard scaling limit at around 5% of the expected load.

This experience shows that in practice, microservices can actually complicate your system unnecessarily. While service-oriented architectures like microservices make perfect sense at the scale of Amazon, it can wreak havoc when it's pushed into the internals of a single-application architecture.

It's important to note that microservices are not a one-size-fits-all solution. Instead, we should evaluate our options and choose what works best for our specific use case. In many cases, a monolith may be the way to go.

This doesn't mean we should abandon microservices altogether, but rather approach them with a critical eye. We should consider the complexity of our system, the scalability needs, and the maintenance costs when making architecture decisions.

Let's not get caught up in buzzwords or the latest trends. Instead, let's focus on finding the best solution for our unique challenges. The lessons from Amazon Prime Video show that sometimes, a simpler approach can be more effective.

Reference : Reference : https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

Top comments (0)

Templates let you quickly answer FAQs or store snippets for re-use.

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink .

Hide child comments as well

For further actions, you may consider blocking this person and/or reporting abuse

Deploying Nuxt 3 App in AWS EC2 with Nginx and PM2

AJAY SHRESTHA - May 15

Creating a Storage Account for a Public Website on Azure

Blessing Oseyenum - May 22

OpenAI GPT-4o, Free and Powerful AI Model Tool in 2024

Saif Hussain - May 14

SMTPGet Review- Analyzing Client Reviews, Ratings, and User Experience

Otis Milburnn - May 14

We're a place where coders share, stay up-to-date and grow their careers.

The Amazon Prime Video Monolith Shift: Dissecting Microservices, Serverless, and the Real-World Cost

Explore amazon prime video's monolith shift, analyzing the real-world cost of microservices and serverless computing in a detailed case study..

Introduction

The Amazon Prime Video team’s recent case study has revealed an interesting shift from a serverless microservices architecture to a monolithic approach. This change resulted in a significant 90% reduction in operating expenses. This development has sparked discussions about the distinctions between serverless and microservices, and how to evaluate their respective merits and drawbacks. By examining Amazon Prime Video’s real-world experiences, we can gain valuable insights into serverless and microservices architectures, as well as their implications for system complexity and cost.

Distinguishing Between Serverless and Microservices

It is essential to understand the differences between serverless and microservices architectures. While microservices involve breaking down applications into smaller, independent services, serverless computing refers to an execution model where the cloud provider dynamically manages the allocation of resources. These two concepts can be used together, as seen in Amazon Prime Video’s initial architecture, but they can also be employed independently.

In the case study, Amazon Prime Video moved away from serverless components, not necessarily microservices. The team found that the serverless components in their architecture, such as AWS Step Functions and Lambda, were causing scaling bottlenecks and increasing costs. By removing these serverless components and simplifying their architecture, Amazon Prime Video was able to achieve significant cost savings.

Finding the Balance

The Amazon Prime Video case study demonstrates the importance of finding the right balance between serverless and microservices architectures for specific use cases. While serverless computing may offer benefits such as scalability and reduced operational overhead, it may not always be the optimal solution for every application or system. Similarly, microservices can provide increased flexibility, but they may also introduce unnecessary complexity in some situations.

Developers must carefully assess their project requirements and constraints before deciding which architectural patterns to adopt. In the Amazon Prime Video case study, the team found that removing serverless components from their architecture and transitioning to a monolithic approach led to better cost and performance optimization.

The Amazon Prime Video case study offers valuable insights into the real-world challenges and costs associated with serverless and microservices architectures. It highlights the importance of understanding the distinctions between these concepts and their suitability for various use cases. By maintaining a balanced approach and continuously evaluating the trade-offs between different architectures, developers can make well-informed decisions that optimize both cost and performance.

Thanks for reading Hacktivate! Subscribe for free to receive new posts and support my work.

🔗 Connect with me on LinkedIn! I hope you found this article helpful! If you’re interested in learning more and staying up-to-date with my latest insights and articles, don’t hesitate to connect with me on LinkedIn . Let’s grow our networks, engage in meaningful discussions, and share our experiences in the world of software development and beyond. Looking forward to connecting with you! 😊 Follow me on LinkedIn ➡️

Ready for more?

Check out our Cloud Native Services and book a call with one of our experts today! 

• Case Studies
• WTF is Cloud Native Collections
• Talk to our ChatBot

Cloud native , Microservices , Architecture

Reflections on Amazon Prime Video’s Monolith Move

Recently an Amazon Prime Video (APV) article about their move from serverless tools to ECS and EC2 did the rounds on all the tech socials. A lot of noise was made about it, initially because it was interpreted as a harbinger of the death of serverless technologies, followed by a second wave that lashed back against that narrative. This second wave argued that what had happened was not a failure of serverless, but rather a standard architectural evolution of an initial serverless microservices implementation to a ‘microservice refactoring’.

This brouhaha got me thinking about why, as an architect, I’ve never truly got onto the serverless boat, and what light this mini-drama throws on that stance. I ended up realising how Amazon and AWS had been at the centre of two computing revolutions that changed the computing paradigm we labour within.

Before I get to that, let’s recap the story so far.

The APV team had a service which monitored every stream viewed on the platform, and triggered a process to correct poorly-operating streams. This service was built using AWS’s serverless Step Functions and Lambda services, and was never intended to run at high scale.

As the service scaled, two problems were hit which together forced a re-architecture. Account limits were hit on the number of AWS Step Function transitions, and the cost of running the service was prohibitive.

In the article’s own words: ‘The move from a distributed microservices architecture to a monolith application helped achieve higher scale, resilience, and reduce costs. [...] We realised that [a] distributed approach wasn’t bringing a lot of benefits in our specific use case, so we packed all of the components into a single process.’

The Reactions

There were more than a few commentators who relished the chance to herald this as the return of the monolith and/or the demise of the microservice. The New Stack led with an emotive ‘ Amazon Dumps Microservices ’ headline, while David Heinemeier Hansson, as usual, went for the jugular with ‘ Even Amazon Can’t Make Sense of Serverless or Microservices ’.

After this initial wave of ‘I told you so’ responses, a rearguard action was fought by defenders of serverless approaches to argue that reports of the death of the serverless and microservices was premature, and that others were misinterpreting the significance of the original article.

Adrian Cockroft, former AWS VP and well-known proponent of microservices fired back with ‘ So Many Bad Takes - What Is There To Learn From The Prime Video Microservices To Monolith Story ’, which argued that the original article did not describe a move from microservice to monolith, rather it was ‘clearly a microservice refactoring step’, and that the team’s evolution from serverless to microservice was a standard architectural pathway called ‘Serverless First’. In other words: nothing to see here, ‘the result isn’t a monolith’.

The Semantics

At this point, the debate has become a matter of semantics: What is a microservice? Looking at various definitions available , the essential unarguable point is that a microservice is ‘owned by a small team’. You can’t have a microservice that requires extensive coordination between teams to build or deploy.

But that can’t be the whole story, as you probably wouldn’t describe a small team that releases a single binary with an embedded database, a web server and a Ruby-on-Rails application as a microservice. A microservice implies that services are ‘fine-grained […] communicating through lightweight protocols’.

If a microservice is 'fine-grained', then there must be some element of component decomposition in a set of microservices that make up a set of applications. So what is a component? In the Amazon Prime Video case, you could argue both ways. You could say that the tool itself is a component, as it is a bounded piece of software managed by a small team, or you could say that the detectors and converters within the tool are separate components mushed into a now-monolithic application. You could even say that my imagined Ruby-on-Rails monolithic binary above is a microservice if you want to just define a component as something owned by a small team.

And what is an application? A service? A process? And on and on it goes. We can continue deconstructing terms all the way down the stack, and as we do so, we see that whether or not a piece of software is architecturally monolithic or a microservice is more or less a matter of perspective. My idea of a microservice can be the same as your idea of a monolith.

But does all this argumentation over words matter? Maybe not. Let’s ignore the question of what exactly a microservice or a monolith is for now (aside from ‘small team size’) and focus on another aspect of the story.

Easier to Scale?

The second paragraph of AWS’s definition of microservices made me raise my eyebrows:

‘Microservices architectures make applications easier to scale and faster to develop , enabling innovation and accelerating time-to-market for new features.’ Source: https://aws.amazon.com/microservices/

Regardless of what microservices were, these were their promised benefits: faster to develop, and easier to scale. What makes the AVP story so triggering to those of us who had been told we were dinosaurs is that the original serverless implementation of their tool was ludicrously un -scalable:

We designed our initial solution as a distributed system using serverless components (for example, AWS Step Functions or AWS Lambda), which was a good choice for building the service quickly. In theory, this would allow us to scale each service component independently. However, the way we used some components caused us to hit a hard scaling limit at around 5% of the expected load . Source: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

and not just technically un-scalable, but financially too:

Also, the overall cost of all the building blocks was too high to accept the solution at a large scale. Source: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

To me, this doesn’t sound like their approach has made it ‘easier to scale’.

Faster to Develop?

But what about the other benefit, that of being ‘faster to develop’? Adrian Cockroft’s post talks about this, and lays out this comparison table:

This is where I must protest, starting with the second line, which states that ‘traditional’, non-serverless/non-microservices development takes ‘months of work’ compared to the ‘hours of work’ microservices applications take to build.

Anyone who has actually built a serverless system in a real world context will know that it is not always, or even usually, ‘hours of work’. To take one small example of problems that can come up: 

...to which you might add: difficulty of debugging, integration with other services, difficulty of testing scaling scenarios, state management, getting IAM rules right… the list goes on.

You might object to this, and argue that if your business has approved all the cloud provider’s services, and has a standard pattern for deploying them, and your staff is already well versed in the technologies and how to implement them, then yes, you can implement something in a few hours.

But this is where I’m baffled. In an analogous context, I have set up ‘traditional’ three-tier systems in a minimal and scalable way in a similar time-frame. Much of my career has been spent doing just that, and I still do that in my spare time because it’s easier for me for prototyping to do just that on a server than wiring together different cloud services.

The supposed development time difference between the two methods is not based on the technology itself, but the context in which you’re deploying it . The argument made by the table is tendentious. It’s based on comparing the worst case for ‘traditional’ application development (months of work) with the best case for ‘rapid development’ (hours of work). Similar arguments can be made for all the table’s comparisons.

The Water We Swim In

Context is everything in these debates. As all the experts point out, there is no architectural magic bullet that fits all use cases. Context is as complex as human existence itself, but here I want to focus on two areas specifically:

The governance context is the set of constraints on your freedom to build and deploy software. In a low-regulation startup these constraints are close to zero. The knowledge context is the degree to which you and your colleagues know how a set of technologies work. It’s assumptions around these contexts that make up the fault lines of most of the serverless debate. Take this tweet from AWS, which approvingly quotes the CEO of Serverless:

"The great thing about serverless is that you don't have to think about migrating a big app or building out this huge application, you just have to think about one task, one unit of work." - @austencollins , Founder & CEO @goserverless w/ @danilop . https://t.co/TVfP1CFCNS pic.twitter.com/yCuY8ChvmM — Amazon Web Services (@awscloud) January 30, 2020

The great thing about serverless is that you don't have to think about migrating a big app or building out this huge application, you just have to think about one task, one unit of work. @austencollins, Founder & CEO @goserverless

I can’t speak for other developers, but that’s almost always true for me most of the time when I write functions (or procedures) in ‘traditional’ codebases. When I’m doing that, I’m not thinking about IAM rules, how to connect to databases, the big app, the huge application. I’m just thinking about this one task, this unit of work. And conversely, if I’m working on a serverless application, I might have to think about all the problems I might run into that I listed above, starting with database connectivity.

You might object that a badly-written three-tier system makes it difficult to write such functions in isolation because of badly-structured monolithic codebases. Maybe so. But microservices architectures can be bad too, and let you ‘think about the one task’ you are doing when you should be thinking about the overall architecture. Maybe your one serverless task is going to cost a ludicrous amount of money (as with APV), or is duplicated elsewhere, or is going to bottleneck another task elsewhere.

Again: The supposed difference between the two methods is not based on the technology itself, but the context in which you’re working . If I’m fully bought into AWS as my platform from a governance and knowledge perspective, then serverless does allow me to focus on just the task I’m doing, because everything else is taken care of by the business around me. Note that this context is independent of 'the problem you are trying to solve'. That problem may or may not lend itself to a serverless architecture, but is not the context in which you're working . Here I’d like to bring up a David Foster Wallace parable about fish:

There are these two young fish swimming along and they happen to meet an older fish swimming the other way, who nods at them and says “Morning, boys. How’s the water?” And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes “What the hell is water?”

When you’re developing, you want your context to be like water to a fish: invisible, not in your way, sustaining you. But if I’m not a fish swimming in AWS’s metaphorical water, then I’m likely to splash around a lot if I dive into it.

Most advocates of serverless take it as a base assumption of the discussion that you are fully, maturely, and exclusively bought into cloud technologies, and the hyperscalers’ ecosystems. But for many more people working in software (including our customers), that’s not true, and they are wrestling with what, for them, is still a relatively unfamiliar environment.

A Confession

I want to make a confession. Based on what you’ve read so far, you might surmise I’m someone who doesn’t like the idea of serverless technology. But I’ve spent 23 years so far doing serverless work. Yes, I’m one of those people who claims to have 23 years experience in a 15-year old technology.

In fact, there’s many of us out there. This is because in those days we didn’t call these technologies ‘serverless’ or ‘Lambda’, we called them ‘stored procedures’.

https://twitter.com/ianmiell/status/921767022056345602

Serverless seems like stored procedures to me. The surrounding platform is just a cloud mainframe rather than a database. — Ian Miell (@ianmiell) October 21, 2017

I worked for a company for 15 of those years where the ‘ big iron ’ database was the water we swam in. We used it for message queues (at such a scale that IBM had to do some pretty nifty work to optimise for our specific use case and give us our own binaries off the main trunk), for our event-driven architectures (using triggers), and as our serverless platform (using stored procedures).

The joy of having a database as the platform was exactly the same then as the joys of having a serverless platform on a hyperscaler now. We didn’t have to provision compute resources for it (DBA’s problem), maintain the operating system (DBA’s problem), or worry or performance (DBA’s problem, mostly). We didn’t have to think about building a huge application, we just had to think about one task, one unit of work. And it took minutes to deploy.

People have drawn similar analogies between serverless and xinetd .

xinetd, but the daemons are containers. Not a bad idea, but certainly not the solution to everything, which is how some "serverless experts" seem to be approaching this. — Timothy Van Heest (@turtlemonvh) September 5, 2019

Serverless itself is nothing new. It’s just a name for what you’re doing when you can write code and let someone else manage the runtime environment (the ‘water’) for you. What’s new is the platform you treat as your water. For me 23 years ago, it was the database. Now it’s the cloud platform.

Mainframes, Clouds, Databases, and Lock-In

The other objection to serverless that’s often heard is that it increases your lock-in to the hyperscaler, something that many architects, CIOs, and regulators say they are concerned about. But as a colleague once quipped to me: “Lock-in? We are all locked into x86”, the point being that we’re all swimming in some kind of water, so it’s not about avoiding lock-in, but rather choosing your lock-in wisely.

It was symbolic when Amazon (not AWS) got rid of their last Oracle database in 2019, replacing them with AWS database services. In retrospect, this might be considered the point where businesses started to accept that their core platform had moved from a database to a cloud service provider. A similar inflection point where the mainframe platform was supplanted by commodity servers and PCs might be considered to be July 5, 1994, when Amazon itself was founded. Ironically, then, Amazon heralded both the death of the mainframe, and the birth of its replacement with AWS.

The Circle of Life

With this context in mind, it seems that the reason I never hopped onto the serverless train is because, to me, it’s not the software paradigm I was ushered into as a youngengineer. To me, quickly spinning up a three-tier application is as natural as throwing together an application using S3, DynamoDB, and API Gateway is for those cloud natives that cut their teeth knowing nothing else.

What strikes this old codger most about the Amazon Prime Video article is the sheer irony of serverless’s defenders saying that its lack of scalability is the reason you need to move to a more monolithic architecture. It was serverless’s very scalability and the avoidance of the need to re-architect later that was one of its key original selling points!

But when three-tier architectures started becoming popular I’m sure mainframers of the past said the same thing: “What’s the point of building software on commodity hardware, when it’ll end up on the mainframe?” Maybe they even leapt on articles describing how big businesses were moving their software back to the mainframe, having failed to make commodity servers work for them, and joyously proclaimed that rumours of the death of the mainframe was greatly exaggerated.

And in a way, maybe they were right. Amazon killed the physical mainframe, then killed the database mainframe, then created the cloud mainframe. Long live the monolith!

Engineering Ethics: It’s Not Just The Money We’re Talking Ab...

How eBPF enables Cloud Native Innovation and Performance

• Latest News
• WTF is Cloud Native
• Website terms of use
• Privacy policy
• Data Privacy for Job Seekers
• Cookie Notice
• Events Code of Conduct

Talk to sales

Stay in touch.

© 2024 Container Solutions

Prime Video Switched from Serverless to EC2 and ECS to Save Costs

• May 18, 2023

Prime Video, Amazon's video streaming service, has explained how it re-architected the audio/video quality inspection solution to reduce operational costs and address scalability problems . It moved the workload to EC2 and ECS compute services, and achieved a 90% reduction in operational costs as a result.

The Video Quality Analysis (VQA) team at Prime Video created the original tool for inspecting the quality of audio/video streams that was able to detect various user experience quality issues and trigger appropriate repair actions.

The initial architecture of the solution was based on microservices responsible for executing steps of the overall analysis process, implemented on top of the serverless infrastructure stack. The microservices included splitting audio/video streams into video frames or decrypted audio buffers as well as detecting various stream defects by analyzing frames and audio buffers using machine-learning algorithms . AWS step functions were used as a primary process orchestration mechanism, coordinating the execution of several lambda functions . All audio/video data, including intermediate work items, were stored in AWS S3 buckets and an AWS SNS topic was used to deliver analysis results.

Source: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

The team designed the distributed architecture to allow for horizontal scalability and leveraged serverless computing and storage to achieve faster implementation timelines. After operating the solution for a while, they started running into problems as the architecture has proven to only support around 5% of the expected load.

Marcin Kolny, senior software engineer at Prime Video, shares his team's assessment of the original architecture:

While onboarding more streams to the service, we noticed that running the infrastructure at a high scale was very expensive. We also noticed scaling bottlenecks that prevented us from monitoring thousands of streams. So, we took a step back and revisited the architecture of the existing service, focusing on the cost and scaling bottlenecks.

The problem of high operational cost was caused by a high volume of read/writes to the S3 bucket storing intermediate work items (video frames and audio buffers) and a large number of step function state transitions.

The other challenge was due to reaching the account limit of the overall number of state transitions, because the orchestration of the process involved several state transitions for each second of the analyzed audio/video stream.

In the end, the team decided to consolidate all of the business logic in a single application process. Kolny summarizes the revised design:

We realized that distributed approach wasn't bringing a lot of benefits in our specific use case, so we packed all of the components into a single process. This eliminated the need for the S3 bucket as the intermediate storage for video frames because our data transfer now happens in the memory. We also implemented orchestration that controls components within a single instance.

The resulting architecture had the entire stream analysis process running in ECS on EC2, with groups of detector instances distributed across different ECS tasks to avoid hitting vertical scaling limits when adding new detector types.

After rolling out the revised architecture, the Prime Video team was able to massively reduce costs (by 90%) but also ensure future cost savings by leveraging EC2 cost savings plans. Changes to the architecture have also addressed unforeseen scalability limitations that prevented the solution from handling all streams viewed by customers.

• Cloud Computing
• Architecture & Design
• Cloud Architecture

Upcoming Posts

Magento 2 on Kubernetes – how do I do that?

AWS Auto Scaling With Varnish – How Your Magento Shop Can Handle Increased Traffic Easily!

Performance Boost With Database Auto-Scaling For Magento With Amazon Aurora

Search the site

Amazon Prime Video team throws AWS Serverless under a bus

Monoliths are sexy again.

• Share on Twitter
• Share on Facebook
• Share on Pinterest
• Share on LinkedIn
• Share on WhatsApp
• Share via Email

Amazon Prime Video has dumped its AWS distributed serverless architecture and moved to what it describes as a “monolith” for its video quality analysis team in a move that it said has cut its cloud infrastructure costs 90%.

The shift saw the team swap an eclectic array of distributed microservices handling video/audio stream analysis processes for an architecture with all components running inside a single Amazon ECS task instead.

(Whether this constitutes a "monolith" as it is described in a Prime Video engineering blog that has triggered huge attention its or instead is now one large microservice is an open question; it has saved it a lot of money following the approach Adrian Cockcroft describes as "optimiz[ing] serverless applications by also building services using containers to solve for lower startup latency, long running compute jobs, and predictable high traffic.")

Senior software development engineer Marcin Kolny said on Prime’s technology blog that toolings built to assess every video stream and check for quality issues had initially been spun up as a “distributed system using serverless components" but that this architecture "caused us to hit a hard scaling limit at around 5% of the expected load" and the "cost of all the building blocks was too high to accept the solution at a large scale.”

This post is for subscribers only

Already have an account? Sign In

“I need to always have levers” – Wells Fargo’s platform chief

Google cloud's matt renner: “half of all ai pocs are just a bad idea", microsoft unleashes new deputy cisos, will tie leadership compensation to security, an attacker could run you up a huge aws bill just by sending rejected requests to an s3 bucket and there’s little you can do about it, the uk election manifestos and the importance of a tech mission, home office may put child abuse database on the cloud, a citi popup showed a trader 711 alerts: they clicked it away –  and generated $196 billion in trading orders by mistake, containerise everything what nutanix’s evolution says about the changing face of it – and the rise of the platform engineer.

Serverless Cost Insights - Part 2: Serverless Cost Savings

Introduction

Welcome back to the second installment of our deep dive into serverless computing and its financial implications . In Part 1 , “ Serverless Cost Insights: Total Cost of Ownership Factors Explained ”, we explored the various factors that contribute to the Total Cost of Ownership (TCO) in serverless architectures . We untangled the complexities and nuances that make serverless computing both an attractive and challenging prospect for businesses striving for technological advancement and cost-efficiency .

In Part 2, we will put theory into practice. Whether you’re a cloud guru , a coding wizard or a business leader seeking to unlock serverless cost savings , this part is for you. We will explore expert tips and actionable strategies to make your serverless journey as economical as it is awesome.

From investing in your team's serverless expertise to re-architecting applications like a pro, we'll equip you with the knowledge and tools to make informed decisions , optimize resources and drive down costs .

In his AWS re:Invent 2023 Keynote , Amazon Chief Technology Officer and Vice President Dr. Werner Vogels focused on the topic of cost optimizatio n for cloud architectures . He shared laws from The Frugal Architect to highlight principles and practices to consider in order to build cost-aware, sustainable, and modern architectures .

So, buckle up and get ready to maximize your serverless savings ! Let's make sure your serverless initiatives are as financially savvy as they are technologically advanced .

Tips to Maximize Serverless Cost Savings

Invest in your team and grow together.

Ready to unlock the power of serverless? It's not just about flipping a switch - it's about equipping your team with the skills they need to succeed.

🚗 Think of it like learning to drive a car. You wouldn't expect a new driver to just hop in and go, right? They need training to navigate the roads and laws.

In the same way, your team needs a roadmap to navigate the constantly evolving world of serverless technology .

Investing in employee education might seem like a cost upfront , but it's a worthwhile investment that leads to:

•   Enhanced Efficiency

• Your team learns best practices and optimization techniques, helping them develop their serverless expertise and leading to sustained efficiency gains.
• Skill development leads to faster workflow and resolution of technical challenges.

•   Optimized Resource Utilization

• Eliminate resource waste by empowering your team to maximize the potential of every serverless technology.
• Informed resource allocation leads to cost savings and efficient application scaling.

•   Fostered Innovation and Adaptability

• Deeper understanding of how serverless opens doors for creative problem-solving and adaptability to evolving technological landscapes.
• Your team becomes equipped to tackle new challenges with confidence and ingenuity.

•   Increased Developer Productivity

• Skilled developers and architects design efficient serverless architectures, optimizing workloads and resource allocation, ultimately reducing costs.
• Streamlined workflows lead to higher developer output and faster project completion times.

•   Informed Decision-Making

• Comprehensive knowledge of architectural nuances and cost implications of serverless design patterns, such as microservices, enables informed decisions.
• This translates to optimized performance, reduced costs, and future-proofed applications.

•   Reduced Technical Debt

• Avoiding serverless anti-patterns, such as using Lambda functions for orchestration or nested function calls , prevents unnecessary complexity and ensures efficient operation.
• Proactive training minimizes technical debt and ensures long-term sustainability of your serverless infrastructure.

Here are a few ideas for how you can invest in your team:

• Training programs: Send your team to serverless workshops, conferences, or online courses.
• Mentorship: Pair experienced developers with new hires to share knowledge and best practices.
• Internal hackathons: Encourage your team to experiment and build new serverless applications.
• Create a learning environment: Provide access to resources, tools, and documentation to support your team's learning.

Remember, your team is your greatest asset! By investing in their growth, you're paving the way for a successful and cost-effective serverless journey .

🐠   As the old saying goes: “ If you give someone a fish, you feed them for a day. If you teach someone to fish, you feed them for a lifetime .”

Investing in your team's knowledge and skills through serverless training is not just about skill-building, it sets the stage for sustained success and innovation.

Tailored Serverless Success: Training and Solutions

Investing in employee training is crucial for successful serverless adoption. But generic training won't cut it. You need tailored solutions that speak directly to your unique business needs .

✂️ Tailor-made clothing fits perfectly because it's customized to an individual’s preferences, measurements, and the occasion.

Similarly, customized serverless solutions are crafted to align precisely with your specific needs.

Tailored Training Workshops for Effective Serverless Adoption

Imagine a training program designed just for your team, like a perfectly tailored suit. It fits your skill level, experience, and specific business goals. This is what customized serverless training offers:

• Focused learning: No more generic IT training. Experts provide specialized knowledge and practical experience directly relevant to your business.
• Direct application: Learn by doing! Workshops enable your team to apply serverless concepts to your specific business scenarios right away.
• Engaging experience: Interactive workshops keep your team actively engaged and excited about learning new technologies.

Tailor-Made Serverless Solutions: Aligning with Your Business Needs

Remember Dr. Vogels ' words at his re:Invent 2023 keynote : " Systems that Last Align Cost to Business ". Just like a tailor-made garment, your serverless solutions should be crafted to fit your business perfectly.

• Focus on your unique use cases: Don't waste resources on features you don't need. Prioritize solutions that align with your specific business drivers and offer the most value.
• Combine business knowledge with technical expertise: This powerful synergy ensures your serverless journey is both technically sound and strategically aligned with your business goals.

Embracing Tailored Training for a Serverless Future

🎤 Serverless Swiftly with Taylor! 😂

Tailored Serverless Training is about equipping you with the tools and knowledge to adapt to the ever-changing technological landscape.

Serverless isn't just about technology, it's about transforming your business . Customized workshops and training programs help you:

• Embrace innovation: Equip your team with the skills they need to adapt to the ever-changing technological landscape.
• Update existing systems: Learn how to integrate serverless technology seamlessly into your existing infrastructure.
• Build from scratch: Create new serverless applications that drive business growth.

Re-architect and Refactor Applications for Serverless Migration

Before embarking on a serverless journey, it's important to distinguish between managed cloud services and serverless computing.

Managed cloud services like PaaS (Platform-as-a-Service) and IaaS (Infrastructure-as-a-Service) offer a platform for deploying and running applications without the complexities of directly managing infrastructure. Developers, however, still retain responsibility for application configuration and maintenance . AWS Elastic Beanstalk and Amazon RDS are examples of PaaS services, offering simplified operations but still requiring client involvement in certain management aspects. Amazon EC2 (Elastic Compute Cloud) is an example of an IaaS offering that provides abstraction on the operating system level.

Serverless computing goes a step further by elevating the abstraction level. With serverless, the cloud provider fully manages the underlying server infrastructure . This helps reduce operational responsibilities such as scaling and maintenance , allowing developers to focus on their code .

Migrating from Monolithic to Serverless

The transition from traditional monolithic architectures to serverless often involves multiple phases, each offering a different level of efficiency and optimization . AWS Community Builder Marin Radjenovic outlines a valuable migration strategy in this Medium article.

In this Medium article , AWS Community Builder Marin Radjenovic talks about AWS cloud migration strategies . The transition from monolithic architecture to serverless involved several phases, each offering different levels of efficiency and optimization.

and shares a migration strategy which involves several phases

Phase 1: Traditional Monolithic Architecture

In this initial phase, a single server hosts the entire application consisting of an API service layer and a relational database. This approach can lead to resource allocation challenges and scalability limitations , impacting both costs and performance.

Phase 2: Transition to Managed Cloud Services (Re-platforming)

Re-platforming involves moving the application to a cloud environment with minimal code changes . While this improves scalability and operational efficiency , it doesn't fully leverage the cloud's agility and can still incur significant costs , especially due to ongoing management needs.

Phase 3: All-in on Serverless (Re-architect and Refactor)

Refactoring the application to a serverless architecture involves breaking it down into independent functions which allows fo r granular scalability. This transition not only simplifies scaling and operations but also leads to a more cost-effective model, paying only for the resources used.

Each phase of this journey offers opportunities for optimization and cost savings, with serverless providing the most significant benefits in terms of flexibility , scalability and cost-effectiveness , with each function managed independently.

Benefits and Considerations

Migrating to serverless offers significant benefits:

• Reduced operational burden: Developers can focus on core business logic instead of infrastructure management.
• Enhanced scalability: Serverless functions automatically scale based on demand, significantly reducing the need for manual intervention.
• Cost-efficiency: Clients only pay for the resources used, leading to substantial cost savings compared to traditional models.

However, it's important to consider the complexity of the migration process. Re-architecting and refactoring applications can be challenging and require expertise in serverless technologies . Additionally, serverless applications can be more difficult to monitor and debug than traditional applications due to their distributed nature.

Optimization and Cost Savings

Migrating to serverless is more than just a technology shift; it's a strategic move towards a more efficient , scalable , and cost-effective architecture . The transition offers an opportunity to modernize the technology stack , streamline the architecture , identify areas for improvement, and reducing waste . While complex, this journey can yield significant rewards in terms of operational agility and cost savings.

Additional Insights

• Security: Security is paramount in serverless environments. Developers need to be aware of the shared responsibility model and implement appropriate security measures.
• Cost optimization: Strategies like cold starts and throttling can help achieve optimal cost-efficiency.
• Monitoring and debugging: Implementing appropriate monitoring and debugging tools is crucial for maintaining operational efficiency.

By carefully planning your migration strategy and addressing potential challenges, you can leverage the power of serverless computing to achieve significant benefits for your organization.

💪 Slowly but surely. One step at a time. It’s a marathon, not a sprint.

Understand Architectural Tradeoffs: Balance Cost and Performance in Serverless Environments

While serverless architectures offer significant benefits of scalability and agility , there are crucial tradeoffs to consider when deciding on the best option for your application. This tip explores key tradeoffs and provides insights to guide your decision-making process.

The transition to cloud-based, serverless architectures offers significant benefits, but like Dr. Werner Vogels ' insightful law in The Frugal Architect , it's not without important tradeoffs that can impact cost savings . This tip explores key tradeoffs to consider when deciding between a traditional vs serverless architecture.

When Traditional Infrastructure Might Be Better

• Always-on requirements: For applications requiring constant uptime, such as certain Big Data and machine learning workloads, traditional infrastructure may offer better cost-efficiency due to predictable resource needs.
• Extensive scaling needs: While serverless excels at handling fluctuating workloads, it can become costly under high, consistent loads. As highlighted by Allen Helton , serverless may not be the most cost-effective option for applications with constant, high-volume traffic.
• Amazon Prime Video case study: This case study illustrates the limitations of serverless in certain high-demand scenarios. While Prime Video initially benefited from serverless scalability, it eventually switched to a traditional setup due to increased usage and associated costs. However, it's important to remember that this is an exception , and most workloads may never encounter such scale .

Advantages of Serverless Solutions

• Greenfield Projects : Serverless is ideal for new projects, allowing for iterative refinement based on performance metrics and cost considerations.
• Modular architecture: Adopting a serverless design can lead to a more modular architecture, facilitating targeted optimizations and cost management, similar to the approach eventually adopted by Prime Video.
• Non-Production Environments : Serverless excels in environments with sporadic usage, aligning costs with actual usage instead of projected needs.
• Unpredictable/Fluctuating Workloads : Automatic resource scaling based on real-time usage promotes cost efficiency and reduces resource waste.

Cost Comparison Illustration

Yan Cui’s LinkedIn Post provides a visual representation that illustrates the cost-effectiveness of serverless for handling unpredictable traffic . Serverless minimizes waste through pay-per-use pricing separating it from the potential over-allocation of resources in traditional setups.

Hybrid Architecture Considerations

A hybrid architecture approach combining serverless with traditional components can be the optimal solution for certain situations. This allows leveraging the scalability and cost benefits of serverless for specific parts of a system while using traditional infrastructure where it's more suitable for performance or cost reasons.

Key Takeaways

• Tailor your architecture: Building a serverless application requires careful consideration of your specific needs to optimize both cost and performance.
• Invest in training: Equipping your team with serverless expertise can significantly impact cost savings and development efficiency.
• Understand the tradeoffs: Knowing the strengths and limitations of both serverless and traditional architectures allows for informed decision-making.
• Start small and iterate: Begin your serverless journey with a small project and gradually scale it after learning from your initial experience.

By carefully considering these factors, you can leverage the power of serverless computing while achieving your desired cost and performance objectives.

🛠️ Building a serverless application is like building a house - you need a solid foundation, the right materials, and a skilled architect to ensure it stands the test of time.

Embrace Best Practices for Efficient and Cost-Effective Serverless Computing

With serverless computing , adhering to best practices is crucial for maximizing efficiency and cost-effectiveness. A wealth of guidelines and standards exist, crafted by experts and early adopters , to guide developers and architects toward successful implementations.

The serverless community and cloud providers like AWS have generously shared numerous valuable rules , recommendations , and proven methods . Following these guidelines saves time and effort, helping prevent potential pitfalls like anti-patterns which can lead to problems down the line.

Harnessing the collective experiences and insights of serverless experts helps minimize waste and boost productivity . The best practices shared are gathered from the serverless community's real-world experiences, ensuring their practical applicability.

Finding and Implementing Best Practices

• Explore official documentation: Cloud providers like AWS, Azure, and Google Cloud Platform offer comprehensive documentation on best practices for serverless development on their respective platforms.
• Join the serverless community: Engage with serverless blogs, forums, Discord, Slack and online communities like Serverless Stack Exchange. These platforms offer valuable discussions, insights, and best practices shared by experienced developers and experts.
• Utilize available tools and resources: Tools like AWS Powertools for Lambda and platforms like CloudGTO can streamline the implementation of best practices and provide helpful guidance.

By actively seeking out and implementing best practices, you can unlock the full potential of serverless computing and build efficient, scalable, and cost-effective applications.

This revised version emphasizes the broader concept of best practices without providing potentially limiting specific examples. It encourages readers to take an active role in their learning and empowers them to discover the best practices most relevant to their specific needs and project context.

📢 Further Tips:

👉 Do not reinvent the wheel: Utilize existing libraries and frameworks to avoid unnecessary development effort.

👉 Learn from others: Share your experiences and contribute to the serverless community.

👉 Avoid working in silos: Collaborate with other developers and leverage their expertise.

Evolving and Optimizing Serverless Applications over Time

As serverless applications become increasingly complex, their evolution and optimization become crucial for maintaining optimal performance and cost-effectiveness. This section explores key strategies to ensure your serverless applications remain agile and efficient over time.

Observability: The First Step to Optimization

Observability forms the cornerstone of effective serverless optimization. By collecting comprehensive data and metrics through dedicated tools, you gain valuable insights into your application's performance, usage patterns, and potential bottlenecks. This data empowers you to make informed decisions about where and how to implement improvements, ensuring your serverless applications run smoothly and cost-effectively.

Dr. Werner Vogels, in his 2023 re:Invent keynote, eloquently stated: " Unobserved Systems Lead to Unknown Costs. " This emphasizes the importance of monitoring and measuring application metrics. Just like a hidden utility meter encourages wasteful habits, making application metrics readily available promotes informed decision-making and cost optimization. While implementing robust observability practices requires an initial investment, the long-term benefits far outweigh the initial cost.

💪 Be proactive. Collect data and statistics to assess the performance and efficiency of workloads

Be Proactive: Embrace Continuous Improvement

Continuous improvement should be incorporated into your development process. To avoid technical debt and legacy applications, it's vital to evolve your applications alongside the rapidly changing technological landscape. Stay informed about the latest serverless advancements and explore new design patterns to uncover optimization opportunities for your application. For instance, replacing Lambda functions with direct service integrations between Amazon API Gateway and DynamoDB can simplify your architecture, reduce code maintenance, and improve cost-efficiency, security, and bug resilience.

Neglecting continuous improvement leads to serverless applications becoming outdated and falling into legacy status. Legacy applications are not only expensive to maintain due to the lack of active management but also pose potential security risks. In severe cases, organizations may struggle to find specialists who possess the expertise to manage applications built on outdated technologies, leading to further cost increases. This starkly highlights the importance of ongoing improvement and modernization for maintaining a healthy and cost-effective serverless environment.

Key takeaways

• Utilize observability tools to gain deep insights into your application's performance and identify optimization opportunities.
• Continuously update your architecture and code base to stay ahead of technological advancements and avoid technical debt.
• Regularly review and implement improvements based on data-driven insights.
• Make continuous improvement an integral part of your team's development lifecycle.

By embracing these strategies, you can ensure your serverless applications remain agile, efficient, and cost-effective throughout their lifespan. Remember, the journey of serverless optimization is a continuous one, requiring ongoing effort and dedication.

💡 Strive to keep architecture and code up to date, apply improvements/enhancements based on observability data and evolve architecture as newer and better technology becomes available

Embrace Continuous Learning: Unlock Your Serverless Potential

In today's rapidly evolving technological landscape, continuous learning is no longer optional but essential for success, especially in the realm of serverless applications. As previously discussed, investing in employee skill development benefits both individuals and the overall business. A key attribute for thriving in the serverless domain is the eagerness and willingness to acquire knowledge and continually improve oneself.

Challenge the Status Quo and Embrace Experimentation

Dr. Werner Vogels' Law VII from The Frugal Architect aptly states: "Unchallenged Success Leads to Assumptions." During his re:Invent 2023 keynote, he emphasized the importance of putting aside our egos and embracing an open mind as technology evolves and advances. Otherwise, we risk hindering our own progress and falling victim to the "We have always done it this way" mentality, as Grace Hopper famously warned against.

Don't be afraid to experiment and explore new approaches, even if they deviate from traditional thinking. The serverless community is a collective learning space; by sharing our learnings and experiences, we contribute to its maturity and help each other grow. Remember, we won't know if something works unless we try it and analyze the results to determine its effectiveness.

Leverage Proven Patterns and Embrace New Solutions

Many existing problems have already been tackled by the community. Leverage proven patterns and best practices developed through collaborative efforts. This approach will save you time and effort while ensuring predictable outcomes.

However, new problems might require more creative solutions. Don't shy away from exploring uncharted territory. What's the worst that could happen? You might not find a solution immediately, but you'll be one step closer than if you never tried. This iterative process of prototyping, testing, and iterating on ideas is crucial for pushing the boundaries and unlocking new possibilities in the serverless space.

Actionable Steps: Foster a Culture of Continuous Learning

• Training and Development: Implement training programs, workshops, and hackathons to equip your team with the latest serverless knowledge and skills.
• Community Engagement: Encourage participation in online forums and communities focused on serverless technologies to stay updated and share learnings with peers.
• Knowledge Sharing: Create a knowledge repository where team members can document valuable insights, best practices, and lessons learned.
• Mentorship Program: Establish a mentorship program where experienced team members can guide and support junior colleagues, fostering a culture of knowledge transfer.
• Professional Development: Allocate resources for attending conferences and workshops to ensure your team remains at the forefront of industry advancements.

By embracing continuous learning and implementing these actionable steps, you can unlock the full potential of your serverless applications and empower your team to thrive in the ever-changing landscape of cloud computing.

🧠 Keep an open mind and be pleasantly surprised with the unlimited possibilities and potential in our technological space today!

Evaluating Cloud Providers

Choosing the right cloud provider for your serverless applications is crucial for success. It's not just about cost, it's about finding a platform that aligns with your specific requirements and unlocks the full potential of your serverless architecture.

Matching Technical Requirements

Start by thoroughly evaluating your application's technical demands. Consider factors like computing power, storage capacity, scalability requirements, reliability needs, and any specialized services you might require. Carefully assess how different cloud providers measure up to these specific needs.

Assessing Quality of Support

Don't underestimate the importance of reliable support. Excellent customer service, responsive technical assistance, comprehensive documentation, and readily available troubleshooting resources are crucial for minimizing downtime and resolving issues quickly.

Community Strength and Ecosystem

Look for a provider with a thriving user community and robust ecosystem. A strong community fosters ongoing innovation and offers a wealth of shared resources, learning opportunities, and collaboration potential.

Navigating Vendor Lock In

Here are some resources to consider for avoiding vendor lock-in:

• Cloud-to-Cloud Migration by DryvIQ: This resource discusses best practices for migrating between cloud providers.
• Multi-Cloud Strategies by Information Week: This article explores the benefits of deploying workloads across multiple cloud providers to leverage the best available offerings.
• Cloud Repatriation by CIO: This article explores the concept of reverse-cloud migration (moving back to on-premises) and discusses scenarios where it might be beneficial.

Remember: these strategies involve significant changes to your application's architecture and design, requiring careful planning and deep technical expertise. Managing multiple cloud environments, ongoing support, and community considerations are also crucial factors to evaluate.

While avoiding vendor lock-in is a key driver for these strategies, it comes with its own set of challenges and costs that require careful management. Ultimately, the best approach depends on your specific requirements and priorities.

⚖️ Make informed decisions; don't blindly follow the crowd. Choose the cloud provider that best aligns with your application's needs and provides the optimal balance between flexibility and cost-efficiency.

Benefits of Serverless Adoption: Unleashing Agility, Efficiency and Innovation

Increased development speed.

• Faster Time to Delivery: Serverless simplifies deployment and testing, enabling rapid development cycles. This allows teams to iterate quickly, respond to market changes, and deliver applications faster.
• Fail Fast and Iterate Sooner: The cost-effective nature of serverless encourages experimentation and rapid iteration. Teams can test ideas quickly, identify issues early, and arrive at optimal solutions faster.
• Reduced Overhead: Eliminate the need to manage and maintain servers, freeing up valuable developer time for core business logic development.

Reduced Operational Overhead and Improved Work-Life Balance

• Shift Operational Responsibility: Cloud providers handle infrastructure management, freeing development teams from tedious tasks and allowing them to focus on higher-value activities.
• Automated Operations: Serverless functions automatically scale up and down based on demand, eliminating the need for manual intervention and enabling efficient resource utilization.
• Reduced Costs: Eliminates the need to purchase, manage, and maintain hardware, leading to significant cost savings.
• Improved Work-Life Balance: By automating tasks and reducing operational overhead, serverless improves work-life balance for developers and IT staff.

👉 Offloading monotonous tasks to cloud providers means better work-life balance!! 😊

Reduced Upfront Cost

• Pay-Per-Use Model: Serverless functions are billed only for the time they are used, eliminating the need for large upfront investments in hardware. This makes serverless especially attractive for startups and organizations with unpredictable workloads.
• Flexible Scaling: Easily scale resources up or down as your needs change, avoiding overprovisioning and optimizing costs.

Optimized Resource Allocation

• Pay-per-use: The pay-per-use model eliminates the risk of over-provisioning and under-provisioning resources, ensuring optimal resource utilization and avoiding unnecessary costs.
• Automatic Scalability: Serverless functions automatically scale up and down based on demand, ensuring resources are always available when needed and reducing costs during periods of low activity.

Enables Experimentation and Proof of Concepts

• Lower Barrier to Entry: Serverless simplifies access to complex technologies like AI and Machine Learning, making them more accessible to developers of all skill levels.
• Cost-Effective Experimentation: The pay-as-you-go model allows for experimentation and proof-of-concept projects without significant upfront costs. This encourages innovation and exploration of new technologies.
• Rapid Iteration: Serverless enables rapid iteration and testing of new ideas, allowing developers to refine solutions quickly and efficiently.

Additional Benefits

• Improved Security: Serverless applications benefit from the inherent security features of the cloud, including automatic patching, vulnerability scanning, and access control.
• Increased Agility: Serverless allows organizations to respond quickly to changing market demands and adapt their applications as needed.
• Greater Focus on Core Business Logic: By offloading infrastructure management, developers can focus on their core business logic and deliver greater value to their customers.

Overall, serverless computing offers a compelling solution for organizations of all sizes, providing significant benefits in terms of development speed, operational efficiency, cost savings, and innovation. By embracing serverless, businesses can unlock new possibilities and achieve their technology goals with greater agility and effectiveness.

Well, folks, that's all she wrote!

Whew, that was a marathon of serverless cost-saving tips ! We covered everything from cloud migration to cost optimization and even threw in some memes and jokes.

But seriously, if you're looking to unleash the power and agility of serverless computing , you're now armed with the knowledge you need to make it happen. Remember, serverless isn't just a buzzword, it's a game-changer . So, go forth and conquer the cloud, one serverless function at a time!

And hey, if you get stuck, don't hesitate to reach out. We're all on this serverless journey together, and there's no shame in asking for help.

P.S. We're not saying this article was short, but if you've read this far, you're clearly a dedicated serverless enthusiast. We salute you! Now go forth and serverlessly conquer the world!

Need to Contact Serverless Guru?

Thank you for spending your time with us! We hope our insights and tips have added value to your serverless journey and equipped you with useful knowledge.

Your perspectives and feedback are not only welcome, they're essential for us and the wider serverless community. Whether you have a question, a comment, or an insight to share, we're eager to hear from you.

Connect with us using the contact form at the bottom of this page or drop us an email at [email protected] . We strive to respond promptly and look forward to engaging in meaningful conversations with you!

The dream team

At Serverless Guru, we're a collective of proactive solution finders. We prioritize genuineness, forward-thinking vision, and above all, we commit to diligently serving our members each and every day.

Looking for skilled architects & developers?

More from Serverless Guru

Choose your character: same service, different tools ( part 4 ), event driven architecture: overview and intro, lift and shift to lambda with docker and web adapters, join the community.

Serverless Cost Insights - Part 2: Expert Tips to Maximize Serverless Cost Savings

Looking for Senior AWS Serverless Architects & Engineers?

Kimberly Lu

Board game enthusiast who enables fun escapades for my cat Charlie. Foodie who loves to cook and bake. Enjoys 3D printing. A jokester.

Choose Your Character: Same Service, Different Tools ( Part 3 )

Request Validation with API Gateway Models

A Deep Dive Into AWS PrivateLink Support For DynamoDB