Virtual machines, containers, and serverless computing

Now that we have a better understanding of the monolithic and microservice architectures, let's look at the Amazon Web Service (AWS) building blocks for creating serverless microservices.

But first we'll cover virtual machines, containers, and serverless computing, which are the basic building blocks behind any application or service hosted in the public cloud.

Virtual machines are the original offering in the public cloud and web hosting sites, containers are lightweight standalone images, and serverless computing is when the cloud provider fully manages the resources. You will understand the benefits and drawbacks of each approach and we will end on a detailed comparison of all three.

Virtual machines

In traditional data centers, you would have to buy or lease physical machines and have spare capacity to deal with additional web or user traffic. In the new world, virtual machines were one of the first public cloud offerings. You can think of it as similar to physical boxes, where you can install an operating system, remotely connect via SSH or RDP, and install applications and services. I would say that virtual machines have been one of the key building blocks for start-up companies to be successful. It gave them the ability to go to market with only small capital investments and to scale out with an increase in their web traffic and user volumes. This was something that previously only large organizations could afford, given the big upfront costs of physical hardware.

The advantages of virtual machines are the pay per usage, choice of instance type, and dynamic allocation of storage, giving your organization full flexibility to rent hardware within minutes rather than wait for physical hardware to be purchased. Virtual machines also provides security, which is managed by the cloud provider. In addition, they provide multi-region auto-scaling and load balancing, again managed by the cloud provider and available almost at the click of a button. There are many virtual machines available, for example, Amazon EC2, Azure VMs, and Google Compute Engine.

However, they do have some drawbacks. The main drawback is that it takes a few minutes to scale. So, any machine that needs to be spun up takes a few minutes, making it impossible most to scale quickly upon request. There is also an effort in terms of configuration where the likes of Chef or Puppet are required for configuration management. For example, the operating system needs to be kept up to date.

Another drawback is that you still need to write the logic to poll or subscribe to other managed services, such as streaming analytics services. In addition, you still pay for idle machine time. For example, when your services are not running, the virtual machines are still up and you're still paying for that time even if they're not being actively used.

Containers

The old way with virtual machines was to deploy applications on a host operating system with configuration-management tools such as Chef or Puppet. This has the advantage of managing the application artifacts' libraries and life cycles with each other and trying to operate specific operating systems, whether Linux or Windows. Containers came out of this limitation with the idea of shipping your code and dependencies into a portable container where you have full operating-system-level virtualization. You essentially have better use of the available resources on the machine.

These containers can be spun up very fast and they are essentially immutable, that is, the OS, library versions, and configurations cannot be changed. The basic idea is that you ship the code and dependencies in this portable container and the environments can be recreated locally or on a server by a configuration. Another important aspect is the orchestration engine. This is the key to managing containers. So, you'd have Docker images that will be managed, deployed, and scaled by Kubernetes or Amazon EC2 container service (ECS).

The drawbacks are that these containers generally scale within seconds, which is still too slow to actually invoke a new container per request. So, you'd need them to be pre-warmed and already available, which has a cost. In addition, the cluster and image configuration does involve some DevOps effort.

Recently AWS introduced AWS Fargate and Elastic Kubernetes Service (EKS), which have helped to relieve some of this configuration-management and support effort, but you would still need a DevOps team to support them.

The other drawback is that there's an integration effort with the managed services. For example, if you're dealing with a streaming analytics service, you still need to write the polling and subscription code to pull the data into your application or service.

Finally, like with virtual machines, you still pay for any containers that are running even if the Kubernetes assists with this. They can run on the EC2 instance, so you'll still need to pay for that actual machine through a running time even if it's not being used.

Serverless computing

You can think of service computing as focusing on business logic rather than on all the infrastructure-configuration management and integration around the service. In serverless computing, there are still servers, it's just that you don't manage the servers themselves, the operating system, or the hardware, and all the scalability is managed by the cloud provider. You don't have access to the raw machine, that is, you can't SSH onto the box.

The benefits are that you can really focus on the business logic code rather than any of the infrastructure or inbound integration code, which is the the business value you are adding as an organization for your customers and clients.

In addition, the security is managed by the cloud provider again, auto-scaling and the high availability options also managed by the cloud provider. You can spin up more instances dynamically based on the number of requests, for example. The cost is per execution time not per idle time.

There are different public cloud serverless offerings. Google, Azure, AWS, and Alibaba cloud have the concept of Functions as a Service (FaaS). This is where you deploy your business logic code within a function and everything around it, such as the security and the scalability, is managed by the cloud provider.

The drawback is that these are stateless, which means they have a very short lifetime. After the few minutes are over, any state maintained within that function is lost, so it has to be persisted outside. It's not suitable for a long-running processes. It does have a limited instance type and a duration too. For example, AWS Lambdas have a duration of 15 minutes before they are terminated. There's also constraints on the actual size of the external libraries or any custom libraries that you package together, since these lambdas need to be spun up very quickly.

Comparing virtual machines, containers, and serverless

Let's compare Infrastructure as a Service (IaaS), Containers as a Service (CaaS), and Functions as a Service (FaaS). Think of IaaS as the virtual machine, CaaS as pool of Docker containers and FaaS an example will be Lambda functions. This is a comparison between IaaS, CaaS, and FaaS:

The green elements are managed by the user, and the blue elements are managed by the cloud service provider. So, on the left, you can see that IaaS, as used with virtual machines, have a lot of the responsibility on the user. In CaaS, the operating-system level is managed by the provider, but you can see that the container and the runtime are actually managed by the user. And, finally on the right, FaaS, you can see the core business logic code and application configuration is managed by the user.

So, how do you choose between AWS Lambda containers and EC2 instances in the AWS world? Check out the following chart:

If we compare virtual machines against the containers and Lambda functions on the top row, you can see that there is some configuration effort required in terms of the maintenance, building it for high availability, and management. For the Lambda functions, this is actually done on a pre-request basis. That is, it's request-driven. AWS will spin up more lambdas if more traffic hits your site to make it highly available (HA), for example.

In terms of flexibility, you have full access in virtual machines and containers, but with AWS Lambda, you have default hardware, default operating system, and no graphics processing units (GPU) available. The upside is that there is no upgrade or maintenance required on your side for Lambdas.

In terms of scalability, you need to plan ahead for virtual machines and containers. You need to provision the containers or instances and decide how you are going to scale. In AWS Lambda functions, scaling is implicit based on the number of requests or data volumes, as you natively get more or fewer lambdas executing in parallel.

The launch of virtual machines is usually in minutes and they can stay on perhaps for weeks. Containers can spin up within seconds and can stay on for minutes or hours before they can be disposed of. Lambda functions, however, can spin up in around 100 milliseconds and generally live for seconds or maybe a few minutes.

In terms of state, virtual machines and containers can maintain state even if it's generally not best practice for scaling. Lambda functions are always stateless, when they terminate their execution, anything in memory is disposed of, unless it's persisted outside in a DynamoDB table or S3 bucket, for example.

Custom integration with AWS services is required for virtual machines and Docker containers. In Lambda functions, however, event sources can push data to a Lambda function using built-in integration with the other AWS services, such as Kinesis, S3, and API Gateway. All you have to do is subscribe the Lambda event source to a Kinesis Stream and the data will get pushed to your Lambda with its business logic code, which allows you to decide how you process and analyze that data. However, for EC2 virtual machines and ECS containers, you need to build that custom inbound integration logic using the AWS SDK, or by some other means.

Finally, in terms of pricing, EC2 instances are priced per second. They also have a spot instance that uses market rates, which is lot cheaper than on-demand instances. The same goes for containers, except that you can have many containers on one EC2 instance. This makes better use of resources and is a lot cheaper, as you flexibility to spread different containers among the EC2 instances. For AWS Lambda functions, the pricing is per 100 milliseconds, invocation number, and the amount of random-access memory (RAM) required.