There are some technical prerequisites before moving ahead. We are going to demonstrate AWS through a web console and the AWS CLI. The following prerequisites should be considered:

• All the demonstration has been tested on a Linux machine with Ubuntu 16.04. We have shared the links to each library used in this book. You can get the detailed information about the installation and configuration with a specific platform.
• We are using open source libraries and software. Hence, for each library, we are going to share its official documentation links. You can refer to those links for detailed information about a specific library.

Web hosting has changed drastically since it started. Physical server machines were shared among many web applications, and it was a really big challenge when it came to scale. It proved to be very expensive for any individual or company to afford an entire server machine to host their web application.

But, thanks to virtualization, the need for a physical server for any web application has been eliminated. Virtualization provides the ability to create many virtual servers as opposed to a single physical server.

Now, the new era of serverless is making the developer's life easier, as we can focus our hard work on development instead of investing time and money on deployment.

Amazon introduced Amazon Elastic Compute Cloud (Amazon EC2) as a cloud computing solution. Amazon EC2 makes it possible to create an array of virtual servers or instances the Amazon Cloud without investing in hardware. You can scale it as per your requirements in terms of networking, computing, and storage.

The serverless approach is nothing but the process of eliminating the manual workload of setting up the hosting environment. Cloud service providers provide serverless services, and so you never actually own any server. Instead, the cloud service provider executes your code in a high-availability infrastructure.

Many cloud service providers introduced different services for the serverless infrastructure. Amazon introduced AWS Lambda as a compute service, where you just provide your code and AWS Lambda executes the code in a highly scalable infrastructure by itself. You don't need to worry about managing the services manually. You need to pay for the compute time of your code execution, and there are no charges when your code is not running.

AWS Lambda executes the code as needed in response to events such as data storage events on an S3 bucket, Amazon DynamoDB events, and HTTP request events via the API Gateway. AWS Lambda is able to execute the code based on scheduled time events via AWS CloudWatch Events. AWS Lambda supports Python, Node.js, C#, and Java.

You need to write a function, which will be executed by AWS Lambda on your behalf.

AWS Lambda is implemented on a container-based model that supports a runtime environment and executes the code as per the Lambda function configuration. When the Lambda function is invoked, it launches the container (an execution environment) based on the AWS Lambda configuration and enables the basic runtime environment, which is required to execute the code.

Let's start with some practical work:

1. To create a Lambda function, you must have an AWS account. If you don't have an AWS account, then you need to sign up on AWS (https://aws.amazon.com/) by providing some basic contact and payment information, as it's essential information required by Amazon.
2. Go to the Lambda home page (https://console.aws.amazon.com/lambda/home). Click on the Create a function button. This will redirect you to the Create function page, which is described in the next step. Take a look at the following screenshot:

3. AWS provides three different options to create a Lambda function, such as Author from scratch, Blueprints, and Serverless Application Repository. We will be using the Blueprint option, which has some built-in Lambda functions. We can choose these blueprints based on our requirements from the search bar, where you can filter by tag and attributes or search by keywords:

4. Let's choose a hello-world-python blueprint. Once we choose the blueprint, we need to set up the basic information about the Lambda function. This information includes the Lambda function's Name and Role, as shown in the following screenshot:

5. Here, Name will be a unique identification for your Lambda function and Role defines the permissions of your Lambda function.

There are three options available for creating a role:

• Choose an existing role
• Create new role from template(s)
• Create a custom role
  Let's look at them in more detail:
  • Choose an existing role: This allows you to select the previously created role.
  • Create new role from template(s): Here, you need to define a role name. AWS Lambda provides ready-made built-in role policy templates that have pre-configured permissions. These are based on other AWS services-related permissions required by the AWS Lambda function. On any role selection, Lambda will automatically add the logging permission to CloudWatch (AWS logging service), as this is the basic permission required by Lambda.
  • Create a custom role: AWS provides an additional privilege to create a customized role to access AWS Lambda. Here, you can define the role based on your requirement.

6. Let's create the HelloWorld Lambda function with some role. Here, I chose the S3 object read-only permission policy template.

7. The following screenshot describes the newly created HelloWorld Lambda function:

The Lambda function includes three sections:

• Configuration
• Triggers
• Monitoring

Let's look at detailed information about the configuration and monitoring. We will have a separate section for triggers. 

Lambda execution depends on the configuration setting. Configuring the Lambda function requires the following details:

• Function code
• Environment variables
• Tags
• Execution role
• Basic settings
• Network
• Debugging and error handling

Here, you are required to write the code. The Lambda function has a predefined pattern for writing the code. While writing the code, you need to understand the context. Lambda provides three kinds of feasibility, which decides your runtime execution for the code:

• Code entry type: This section provides three options to decide the entry type for your code, such as editing code inline, uploading a ZIP file, and uploading a file from Amazon S3.
• Runtime: This section provides options to decide the runtime programming language context for your code, such as Python, C#, NodeJS, and Java.
• Handler: A handler defines the path to your method/function, such as <filename>.<method_name>. For example, if you want to execute a function named as a handler, which is defined in main.py, then it would be main.handler.

Let's get back to our newly created hello world function named lambda_handler.

Here, the handler value is defined as lambda_function.lambda_handler, where lambda_function.py is the filename and lambda_handler is the method name:

def lambda_handler(event, context): 
    print("value1 = " + event['key1']) 
    print("value2 = " + event['key2']) 

Lambda_handler accepts two positional arguments, event and context:

• event: This argument contains event-related information. For example, if we configure the Lambda function with Amazon S3 bucket events, then we would get S3 bucket information in event arguments, such as bucket name, region, and so on.
• context: This argument contains the context-related information that may be required during runtime for code execution.

You can set the environment variables in key-value pairs, which can be utilized in your code.

You can use tags for grouping and filtering your Lambda functions. You may have multiple Lambda functions with different regions, so tags help make Lambda functions more manageable.

As we previously discussed the role and permission while creating the Lambda function, Lambda provides the capability to edit the existing role that you chose at the time of the Lambda function creation.

Under basic settings, you can configure the memory and execution timeout. Lambda supports memory from 128 MB to 1,536 MB. Timeout execution is in seconds; the default timeout execution Lambda supports is 300 seconds. This setting helps you to control the code execution performance and cost for your Lambda function.

In the network section, you can configure the network access to your Lambda function.

AWS provides a VPC (Virtual Private Cloud) service to create a virtual network, which allows access to AWS services. You can also configure the networking as per your requirements.

We will discuss the Lambda function with VPC in the upcoming chapters. As of now, we will choose No VPC in the network section.

AWS Lambda automatically retries the failed asynchronous invocation. But you can also configure the DLQ (Dead Letter Queue), such as the SQS queue or SNS topic. To configure the DLQ, the Lambda function must have permission to access DLQ resources.

Now that we understand the configuration, let's go ahead with the execution of the Lambda function.

Let's look at the Monitoring section, which describes the activity related to our Lambda function. It can be used to analyze the performance of our Lambda function execution.

AWS CloudWatch is a monitoring service for AWS resources and manages all activity logs. It creates metric data to generate statistical data. CloudWatch enables real-time monitoring of AWS resources. It also monitors hardware information related to AWS EC2 or RDS database instances and other resources.

Lambda monitoring sections display the last 24 hours' analytics data related to the Lambda function's activity and performance. The following screenshot shows the monitored analytics information about our hello world Lambda function:

Let's move on to the next section, where we are going to look at the Lambda function execution.

AWS Lambda supports several methods of execution. Let's start with the basic execution from its own web console interface. AWS Lambda provides the capability to test the function manually, where you can define the test event context. If you want to test against some other Amazon services, then there are built-in event templates available.

The following screenshot demonstrates the test event creation:

As shown in the preceding screenshot, a single Lambda function can have a maximum of 10 test events and the test events are persisted, so you can reuse them whenever you want to test your Lambda function.

I created the test event with the event name as HelloWorld and now I am going to execute the HelloWorld function, when converting the Lambda function as a Python microservice, as shown in the following code:

from __future__ import print_function 
import json 
 
print('Loading function') 
 
def lambda_handler(event, context): 
    print("Received event: " + json.dumps(event, indent=2)) 
    print("value1 = " + event['key1']) 
    print("value2 = " + event['key2']) 
    print("value3 = " + event['key3']) 
    return "Hello World" 

Here, we are printing the event data and then returning back to the Hello World string:

Lambda manages some information on every request execution, such as a request ID and billing information. The Lambda price model is based on the time consumption on request processing, whereas the request ID is the unique identification of every request.

In the Log output, you can see all the print statements output. Now, let's raise an error and see how Lambda responds and returns the logs.

We are going to replace the current code with the following snippet:

from __future__ import print_function 
import json 
 
print('Loading function') 
 
 
def lambda_handler(event, context): 
    print("Received event: " + json.dumps(event, indent=2)) 
    raise Exception('Exception raised manually.') 

The following screenshot is the log snippet of the execution result:

Here, Lambda responded with the complete stack trace information and logged it as well. You can check the CloudWatch logs, as CloudWatch is preconfigured with the AWS Lambda execution.

We learned about the Lambda function execution from the Lambda console, and now it's time to execute the Lambda function from a schedule trigger. In our project, we often need to have a cron job schedule to execute some functionality at a particular time period.

Lambda triggers will help us to set up the triggers based on events. Let's move ahead to introduce the trigger to our hello world function.

The Lambda function can be configured in response to events. AWS provides a list of triggers that support lots of events. These triggers belong to their associated AWS services.

You can add a trigger to your Lambda function from the triggers section.

I am going to slightly modify the hello world Lambda function. Here, we are printing the request ID, which is received in the context object as an aws_request_id attribute. It also prints the timestamp:

Now, we are going to add a trigger to our Lambda function that will execute our Lambda function every minute.

The following screenshot shows the Add trigger flow, where you can easily configure any trigger from the left-hand panel with your Lambda function:

We are going to configure the CloudWatch Events trigger. CloudWatch Events deliver near real-time system events that describe the changes in AWS resources.

You can set up simple event rules with operational events in AWS resources as they occur, and you can also schedule automated events that self-trigger based on cron or the rate expression.

Here, we are going to schedule a rate expression to execute our hello world Lambda function every minute. We need to select CloudWatch Events from the triggers dropdown.

To create the CloudWatch event rule, we are going to create a new rule. We need to set up the rule with some required information, such as the rule name, which is a unique identifier. So, we are going to name the rule as hello-world-every-minute and the rule type as either the event pattern or schedule expression. In our case, it would be a schedule expression as the rate (1 minute), as shown in the preceding screenshot.

Once we set the trigger and enable it, the scheduled event would get triggered as per the schedule expression. Let's see our hello world Lambda logs after five minutes.

To view the logs related to any services, you need to do the following:

1.  Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
2. In the navigation pane, choose Logs
3. Select the log group related to the HelloWorld Lambda function

The following screenshot describes the CloudWatch log access:

By selecting the HelloWorld Lambda function log groups, you see the logging activity related to our HelloWorld Lambda function. The following screenshot shows the logs of the HelloWorld function:

Here, you can see that our hello world Lambda function is executed exactly every minute since the time we have enabled the trigger.  

Now, let's move ahead to create a serverless RESTful API.

Let's understand the microservice scenario where we are going to deploy a serverless hello world function that will respond to HTTP events through the API Gateway.

The Amazon API Gateway service enables you to create, manage, and publish a RESTful API to interact with AWS resources at any scale. The API Gateway provides an interface where you expose the backend through the REST application programming interface.

To enable the AWS serverless infrastructure, the API Gateway plays an important role, as it can be configured to execute the Lambda functions. 

Now, we are going to configure an API Gateway service to executes the Lambda function

Here is the hello world function:

When we integrate the AWS Lambda function with the API Gateway, the Lambda function must return a dictionary object with the required keys as statusCode, headers, and body.  The value of the body attribute must be in a JSON string. Hence, we converted the Python dictionary into a JSON string.

It's time to integrate the API Gateway with the Lambda function. As we have seen in our previous discussion about triggers, we are going to add a trigger with the API Gateway:

We are going to create an API Gateway service with the name as LambdaMicroservice. The API Gateway enables you to create and maintain a deployment stage as per your requirement.

If you want to secure your API then you have two options—using AWS IAM and opening it with the access key, or keeping it as open, making it publicly available.

AWS IAM (Identity Access Management) is an AWS cloud service that is helpful in creating a secure access credential in order to access AWS cloud services.

Opening with the access key feature allows you to generate the key from the API Gateway console. In our case, we are going to keep the security open only, as we need to access our API publicly:

Once you add and save the changes, the REST API is ready within a few seconds. The invoke URL is our REST API endpoint.

Let's hit the invoke URL using the curl command-line tool and see what happens:

$ curl https://cfi6872cxa.execute-api.us-east-2.amazonaws.com/prod/HelloWorld
{"message": "Hello World returned in JSON"}

That's it. We are done with creating a serverless RESTful API using AWS Lambda and the API Gateway. Now, we are going to see how we can interact with the AWS services using the AWS CLI.

The AWS CLI is an open source tool developed on top of AWS SDK for Python using the Boto library, which provides commands to interact with AWS services. With the very minimum configuration, you can manage any AWS services from the CLI. It provides direct access to AWS services and you can develop shell scripts to manage your resources.

For example, if you want to upload the file to the S3 bucket, then you can do so by just a single command from the CLI:

$ aws s3 cp index.html s3://bucket-name/ 

aws s3 cp is a shell-like command that performs the multi-part file upload operation in order to complete the operation.

It also supports customization for some of the AWS services. You can see the list of AWS services supported by aws-cli by using the --help command.

awscli is available to as a Python distributor package. You can easily install it with the pip command, as described in the following code:

$ pip install awscli --upgrade  

Here are the prerequisites:

• Python 2 with version 2.6.5+ or Python 3 with version 3.3+
• Unix, Linux, macOS, or Windows

awscli directly accesses AWS services but we need to configure and authenticate it in order to access AWS services.

Run the aws configure command to configure the AWS CLI with your Amazon account:

You can get the AWS access key ID and AWS secret access key from the My Security Credentials option, as shown in the following screenshot:

Let's configure our hello world Lambda function and triggers using the awscli utility command.

The AWS CLI supports all available AWS services. You can see a detailed description of the aws command using aws help and it will also list all the available services.

We are interested in Lambda, as we are going to create a Lambda function with a simple hello world context:

$ aws lambda help  

This will list a complete description of the Lambda service and all the available commands that are required to manage the AWS Lambda service.

Here, we are going to create a new Lambda function with the aws lambda create-function command. To run this command, we need to pass the required and optional arguments.

Make sure you have a role with permission for the lambda:CreateFunction action.

Previously, in the AWS Lambda console, we chose the code entry point as inline editing. Now, we will be using a ZIP file as a deployment package.

Before creating the Lambda function, we should create a Lambda function deployment package.

This deployment package will be a ZIP file consisting of your code and any dependencies.

If your project has some dependencies, then you must install the dependencies in the root directive of the project. For example:

   $ pip install requests -t <project-dir> OR  
   $ pip install -r requirements.txt  -t <project-dir> 

Here, the -t option indicates the target directory.

Create a simple lambda_handler function in a file named as handler.py, as shown in the following screenshot:

Now, let's make a deployment package as a ZIP file consisting of the preceding code:

Now, we are ready to create the Lambda function. The following screenshot describes the command execution:

You can see that, in the AWS Lambda console, the Lambda function immediately got created:

Let's discuss the required and optional parameters that we used with the aws lambda create-function command:

• --function-name (required): The name is self-explanatory. We need to pass the Lambda function name that we are intending to create.
• --role (required): This is a required parameter where we need to use the AWS role ARN as a value. Make sure this role has permissions to create the Lambda function.
• --runtime (required): We need to mention the runtime environment for the Lambda function execution. As we mentioned earlier, AWS Lambda supports Python, Node.js, C#, and Java. So these are the possible values:
  • python2.7
  • python3.6
  • nodejs
  • nodejs4.3
  • nodejs6.10
  • nodejs4.3-edge
  • dotnetcore1.0
  • java8
• --handler (required): Here, we mention the function path that will be an execution entry point by the AWS Lambda. In our case, we used handler.lambda_function, where the handler is the file that contains the lambda_function.
• --description: This option lets you add some text description about your Lambda function.
• --zip-file: This option is used to upload the deployment package file of your code from your local environment/machine. Here, you need to add fileb:// as a prefix to your ZIP file path.
• --code: This option helps you upload the deployment package file from the AWS S3 bucket.

You should pass a string value with a pattern, such as the one shown here:

 "S3Bucket=<bucket-name>,S3Key=<file-name>,S3ObjectVersion=<file-version-id>". 

There are many other optional parameters available that you can see with the help command, such as aws lambda create-function help. You can use them as per your requirement.

Now we will see the Lambda function invocation using the command $ aws lambda invoke.

The Lambda CLI provides a command to directly invoke the Lambda function:

$ aws lambda invoke --function-name <value> <outfile> 

Let's look at the parameters:

• --function-name (required): This parameter asks for a Lambda function name
• outfile (required): Here, you need to mention a filename where the returned output or response by the Lambda function will be stored

Here are other optional parameters available that you can list by the help command.

Let's invoke our recently created HelloWorldCLI function:

When we invoked the Lambda function, it immediately responded with a status code and the Lambda function returned the output data stored in the newly created lambda_output.txt file by the lambda invoke command.

This is a subcommand of the aws lambda command and is used to create event mapping for your Lambda function.  $ aws lambda create-event-source-mapping supports only Amazon Kinesis stream and Amazon DynamoDB stream events mapping. We will discuss event mapping with the Amazon API Gateway and CloudWatch event using Zappa in the upcoming chapters.

In this chapter, we learned about the manual process of creating a simple AWS Lambda and configuring some triggers against it. Also, we looked at the AWS Lambda configuration using the AWS CLI. It's really amazing to implement a serverless application. These AWS services play an essential part in creating a serverless infrastructure, where you can develop your application and deploy it as serverless.

1. What would be the benefits of being serverless?
2. What is the role of Amazon S3 in the serverless infrastructure?