Learning Lambda — Part 1
Setting the Stage
This is Part 1 of Learning Lambda, a tutorial series about engineering using AWS Lambda. To see the other articles in this series please visit the series home page. To be alerted about future installments subscribe to our newsletter, follow Symphonia on twitter @symphoniacloud, and our blog at The Symphonium.
What is Lambda?
‘Serverless’ is a new cloud computing approach to architecting and building applications. Serverless vendors like Amazon Web Services, Microsoft, and Google perform infrastructural heavy lifting, empowering your teams to focus on their customers and business goals.
The aspect of Serverless that has received the most excitement over the last 18 months is Serverless Compute, otherwise known as Functions-as-a-Service (FaaS). FaaS is a new way of deploying server side logic, oriented around deploying individual functions, or operations. The traditional way to deploy server side software is to start with a host instance — for example a virtual machine instance or a container. We then deploy our application within that host. This application is typically an operating system process, and usually contains code for several different operations.
FaaS changes this significantly. We strip away both the host instance and the application process from our model, and instead focus on just the operations, or functions, that express our application's logic. We upload those functions individually to a vendor-supplied FaaS platform, and that platform performs all the work to actually run our code on our behalf.
FaaS has many interesting benefits. It often requires less work and risk than traditional server-side software development since you don't have to worry about managing as many deployment or infrastructure concerns. This is good for ongoing development, and is especially useful when you want to prototype something new. It's also a wonderful platform for scaling — the FaaS provider will scale your application from zero to many parallel instances, and back again, without any work on your part. Finally it's very efficient from a infrastructure cost perspective — the FaaS provider will only charge you when your code is executing (you aren't charged when your app is idle) — and does so to a high level of precision (e.g. at a 100 millisecond granularity.)
FaaS is inherently an event-driven approach. Beyond providing a platform to host and execute Serverless Compute code a FaaS Vendor also integrates with various synchronous and asynchronous event sources. An example of a synchronous source is an HTTP ‘API Gateway’ (which we'll see more of in the next section.) An example of a asynchronous source is a hosted message bus.
Lambda, from Amazon Web Services (AWS), is the most widely adopted FaaS implementation currently available. It was launched in the fall of 2014 and over the last 2 ½ years has grown in maturity and usage. Companies use Lambda to process in excess of 1 billion events per day, to give you a sense of scale. At the time of writing Lambda is integrated with more than 15 different types of event source, enabling it to be used for a wide variety of different applications.
This series of articles will give you a deep dive into programming on the Lambda platform. We start from first principles and we explain what is going on in the Lambda environment as we proceed. The aim of this series is not to give you cookie cutter examples but instead to give you a thorough understanding of the platform and programming model so that you'll have confidence building your own Lambda applications.
In this series we'll be using Java (and its JVM) as our development language, but Lambda natively supports several other languages too. We won't be using any complicated aspects of the Java language, so most developers will be able to follow along. Occasionally we'll dip into Java and/or JVM specifics, but they'll be the exception rather than the norm. We hope in the future to give versions of this series for different languages and runtimes.
If you're seeking more knowledge about Serverless in general, rather than a deep-dive into Lambda, I recommend our article from last year on Serverless Architectures. If you already have a decent understanding of Lambda in general you should follow our blog anyway since we are also writing about more advanced / real-world usages of Lambda.
What does a Lambda application look like?
Invoking (calling) Lambda functions follows two basic patterns. The first is synchronous invocation (named RequestResponse by AWS), where the caller is blocked, and the function returns a result. The second is asynchronous (Event in AWS parlance) where the function is called solely for the purpose of a side effect (e.g. updating a database.) If a lambda function is written in a way that returns a result, but is invoked asynchronously then the result is discarded by the platform. We'll get further into the difference between these later in the series, but for now let's look at an example of each of these usages.
First up, we can use Lambda to provide a Web API. While Lambdas aren't HTTP servers themselves we can use another component, API Gateway, to provide the HTTP and routing logic that we typically have within a web service.
The API Gateway has a configured Lambda function for each route that it provides. When it receives an HTTP request for a specific route (e.g.
GET /products/123) it makes a synchronous invocation to the Lambda platform specifying the correct Lambda function to call (e.g.
ProductsLambda) and passing the HTTP request arguments (e.g.
id=123.) The Lambda platform instantiates or locates a running instance of the Lambda function and then calls it with the argument passed by the API Gateway.
Once the Lambda function is running we're in our code and can do whatever we want with the request. Often times in such a scenario we'll want to access a database or other backend service — AWS DynamoDB is a popular database to use with Lambda, partly due to its similar scaling capabilities. Once we've finished processing the request we return a response. The Lambda platform passes this back to the API Gateway, which then maps our Lambda function's response to an appropriate HTTP response, returning this back to the original caller.
A very common use case for Lambda is file processing. Let's imagine a mobile application that can upload photos to a remote server, which we then want to make available to other parts of our product suite, but at different image sizes.
With Lambda we can tie the event of an object being created in an S3 bucket to a specific Lambda function (in this case one named
ImageReceiver). When the file is created S3 invokes the Lambda platform, specifying the function to be called, and passing a path to the file. As with the previous example the Lambda platform instantiates or locates a running instance of the lambda function and then calls it with the request details passed this time by S3. The difference now though is that this is an asynchronous invocation — no value is returned to S3 and nor does S3 wait for a return value.
This time our Lambda function exists solely for the purpose of a side-effect — it loads the file specified by the request parameter and then creates new, resized, versions of the file in a different S3 bucket. With the side-effects complete the Lambda function's work is done. Since it created files in an S3 bucket we may choose to create further Lambda functions that process these, creating a processing pipeline.
Does the name ‘Lambda’ imply Java / C# Lambda expressions?
No — AWS Lambda has nothing to do with Java Lambda Expressions, nor the the C# equivalent. You are free to use Java Lambda expressions within your AWS Lambda if you'd like (since AWS Lambda supports Java 8), or not, but the two types of technology are unrelated.
We're nearly at the end of this introduction and all that remains is to set up a few pre-requisites so that we're ready to get coding next time.
First of all you're going to need an AWS account with appropriate privileges. You can create an AWS account using the Create account button on the AWS home page here. For what we're going to do the ‘free tier’ that Amazon give new accounts will provide more than enough capacity. We suggest you do actually create a new account for this series, outside of any production AWS account you may have at work, since certain aspects of AWS Lambda (e.g. concurrent invocation limits) are account-wide.
Even if you don't create a new account we suggest using one with full AWS privileges. If you don't you'll hit some distracting access issues. If you absolutely can't get an account with full AWS privileges, you'll at least need full Lambda privileges — those will be enough for the first few parts of the series.
AWS Lambda supports Java 8, and we'll be assuming you have Java 8 available in your development environment. If you don't already have Java 8 available you may find the download page here to be useful.
We'll also be using Maven so make sure you have that installed and up-to-date. We use Apple Macs for our development and install Maven with homebrew, but you should feel free to use whatever installation variant you want.
To validate your environment open a terminal and run
mvn -v . You should see some output that starts with something like the following:
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015–11–10T11:41:47–05:00) Maven home: /usr/local/Cellar/maven/3.3.9/libexec Java version: 1.8.0_112, vendor: Oracle Corporation
Any 3.x version of Maven and 1.8.x version of Java should be ok, but we'll be using the versions above.
Finally you should be comfortable creating Java projects, using Maven, in your development environment of choice. We'll use the free version of IntelliJ IDEA, but you should feel free to use whatever you want. We won't be using any IDE-specific tools or plugins in this series.
This was the first part of our ‘Learning Lambda’ series. We gave an overview of Lambda and its benefits, gave a couple of examples of how Lambda is used, talked about the name ‘Lambda’, and finally described some pre-requisites if you want to follow along with the rest of the series.
You can read Part 2 here. To see the other articles in this series please visit the series home page. To be alerted about future installments subscribe to our newsletter, follow Symphonia on twitter @symphoniacloud, and our blog at The Symphonium.