Learning Lambda — Part 5

Command Line Tools

Symphonia
Mike Roberts
Mar 21, 2017

Command Line Tools

This is Part 5 of Learning Lambda, a tutorial series about engineering using AWS Lambda. To see the other articles in this series please visit the series home page. To be alerted about future installments subscribe to our newsletter, follow Symphonia on twitter @symphoniacloud, and our blog at The Symphonium.

image

Welcome to Part 5 of Learning Lambda! If you didn’t read Part 4 you’ll probably want to do that before continuing here.

Last time we dug into the Java Lambda Runtime programming model, specifically with regard to input, output and context. This time we’re going to step away from code and into tooling.

So far I’ve talked only about using the AWS Web Console for all deployment, configuration, and testing. This is fine when you’re just getting started, but gets tedious pretty quickly. In fact remember all those examples that I gave last time in Part 4? I didn’t use the Web Console. Instead I used the AWS CLI — Command Line Interface. In this installment of the series I’m going to show you how you can also use the CLI.

Automating AWS, and setting up the CLI

Those of you that have used the AWS CLI before, for other AWS Services, should feel free to skip this first section.

Amazon and AWS are built on APIs. In fact in this classic story of Amazon’s API mandate we see that “All teams will henceforth expose their data and functionality through service interfaces” and “All service interfaces, without exception, must be designed from the ground up to be externalizable.” What this means is that almost anything that we can do through the AWS Web Console UI we can also do using the AWS API and CLI.

The AWS API is a large collection of HTTP endpoints that we can call to perform actions within AWS. While calling the API directly is perfectly supported, it’s also a little laborious due to things like authentication / request signing, correct serialization, etc. For this reason AWS give us two tools to make things easier — SDKs and the CLI.

You can view all the different SDKs here, for example here is the documentation for the Java SDK. But we’re not going to use the SDKs right now, instead we’re going to use the AWS CLI — Command Line Interface. The CLI is a tool you can use from a terminal — it wraps the AWS API, and so almost anything available through the API is also accessible via the CLI.

You can use the CLI on Mac OS, Windows and Linux, however all the examples and suggestions we give from here on in are Mac oriented. If you’re using a different operating system for your development machine then you should combine the instructions here with whatever is specified in the CLI documentation.

To install the CLI you can follow the instructions here. If you’re using a Mac and Homebrew, installing the CLI is as simple as running brew install awscli. To validate your install of the CLI run aws --version from a terminal prompt, and it should return something similar to: aws-cli/1.11.59 Python/2.7.10 Darwin/16.4.0 botocore/1.5.22 . The precise output will depend on your operating system, among other factors.

Once the CLI is installed you’ll need to configure it — run aws configure and give the information for your setup. More help is available here. Once you’ve configured the CLI you can run aws configure again, and you’ll see your settings, something like:

$ aws configure 

AWS Access Key ID [********************]:  
AWS Secret Access Key [********************]:  
Default region name [us-west-2]:  
Default output format [json]:

If you have something that looks like this you’re good to go. Aside — you may also want to setup CLI command completion.

The Lambda CLI

We can use the CLI to create, configure, deploy, view, inspect, invoke and delete Lambda functions. The full list of commands we can use is visible here. Note that since the CLI is ‘just’ a wrapper to the AWS API this list is precisely the same (bar some formatting differences) as the Lambda API documentation here.

At time of writing there are 25 different operations you can invoke through the Lambda CLI. We’re not going to go into all of them here, but are going to discuss some of them. We’ll be using more of these commands later in this series.

Before we get into the CLI Lambda commands there’s just one thing we need to do — capture the identifier of the IAM role that our Lambda functions will run as. If you’ve already followed along through Part 2 you should have a role named lambda_basic_execution (and if you don’t, you might want to go and follow through Part 2 before we go on). Let’s capture that role identifier in a shell variable using the IAM get-role command of the AWS CLI:

$ LAMBDA_ROLE_ARN=`aws iam get-role --role-name lambda_basic_execution --query Role.Arn --output text`

Quick aside — I learned about this kind of variable capture using --query from this great talk from AWS re:Invent 2016 by Kyle Knapp.

Now if you execute echo $LAMBDA_ROLE_ARN you should see something like arn:aws:iam::YOUR_ACCOUNT_ID:role/lambda_basic_execution . This is an ARN that identifies the role that we want our Lambda to assume. An ARN is an ‘Amazon Resource Name’ — a way to unambiguously identify any type of object or resource globally within AWS.

In the examples below we’ll assume that LAMBDA_ROLE_ARN is set in this way.

create-function

Let’s go back to the first Lambda function we wrote in Part 2:

First we’re going to rebuild our code as we did in Part 2 using Maven. Then we’re going to deploy this code as part of configuring a new Lambda function, using the AWS CLI. Run the following commands from your project’s root directory:

$ mvn package
…
[INFO] BUILD SUCCESS
…
$ aws lambda create-function --function-name CliLambda --runtime java8 --role $LAMBDA_ROLE_ARN --handler io.symphonia.SimplestLambda::handler --zip-file fileb://target/learning-lambda-1.0-SNAPSHOT.jar --memory-size 512

If this is successful you should see something like this as output:

{
    "CodeSha256": "CODE_SHA...",
    "FunctionName": "CliLambda",
    "CodeSize": 863546,
    "MemorySize": 512,
    "FunctionArn": "arn:aws:lambda:YOUR_DEFAULT_REGION:YOUR_ACCOUNT_ID:function:CliLambda",
    "Version": "$LATEST",
    "Role": "arn:aws:iam::YOUR_ACCOUNT_ID:role/lambda_basic_execution",
    "Timeout": 3,
    "LastModified": "2017-03-19T21:09:21.600+0000",
    "Handler": "io.symphonia.SimplestLambda::handler",
    "Runtime": "java8",
    "Description": ""
}

What we did here with the create-function command was exactly the equivalent of what we did using several pages of UI wizard in the AWS Web Console in Part 2 — we’ve created a Lambda function, configured it with various settings, and uploaded a first version of the code. I think you’ll agree that using the CLI is a lot less time consuming!

If we break down the command we see the following structure

  1. aws — execute the AWS CLI command
  2. lambda — specify which AWS service we are using
  3. create-function — specify which of the Lambda service commands we want to run
  4. --function-name CliLambda — specify a unique (within our AWS account & region) name for the new function
  5. --runtime java8 — this is a Java Lambda
  6. --role $LAMBDA_ROLE_ARN — specify the role the Lambda should execute as, using the role identifier we captured earlier
  7. --handler io.symphonia.SimplestLambda::handler — the package, class name, and method name of our handler function
  8. --zip-file fileb://target/learning-lambda-1.0-SNAPSHOT.jar — the JAR file that contains our compiled Lambda function
  9. --memory-size 512 — configuration for how much RAM to give our Lambda (in MB)

create-function has other parameters which we didn’t use here, for instance --timeout to specify a different ‘Timeout’ setting (we’ll go over timeouts later in the series). To see all the options for a command, you can run aws lambda COMMAND_NAME help, e.g. in this case aws lambda create-function help, or of course you can refer to the documentation.

update-function-code

Before we test our new function I actually want to make a code change. Update the handler code as follows:

With that done, execute mvn package again, to rebuild the code, then run the following:

$ aws lambda update-function-code --function-name CliLambda --zip-file fileb://target/learning-lambda-1.0-SNAPSHOT.jar

You should see output very similar to that you saw with create-function .

update-function-code uploads a new version of our deployed code, effectively it’s a subset of create-function. We invoked both of these commands using the--zip-file parameter, which takes a local zip file (a JAR file is just a zip file with some optional Java specific metadata included.) Alternatively you can tell these commands to use a file from AWS S3 (that you’ve previously uploaded), using the S3 related optional parameters. This can be more appropriate for automated deployment pipelines.

invoke

Now let’s test our Lambda function, using the invoke CLI command :

$ aws lambda invoke --invocation-type RequestResponse --function-name CliLambda --payload \"world\" outputfile.txt

If this is successful you should see the following output:

{
   "StatusCode": 200
}

There are 4 arguments here:

  1. --invocation-type RequestResponse — We are invoking the Lambda function synchronously and waiting for a response.
  2. --function-name CliLambda — The Lambda function we want to call
  3. --payload \"world\" — the input to pass to the handler, in this case the JSON string \"world\"
  4. A file to output the function response to

Given that fourth argument, we can cat the output file:

$ cat outputfile.txt
"Hello, world"

In fact I typically combine these two commands into one shell request (see below for example).

The third argument, --payload, can take a value inline as we did here. For more complicated inputs you can specify a file to use. As an example create a file name test.json and write "from file" in it. Now we can execute the following version of invoke :

$ aws lambda invoke --invocation-type RequestResponse --function-name CliLambda --payload file://test.json outputfile.txt; cat outputfile.txt
{
    "StatusCode": 200
}
"Hello, from file"

update-function-configuration

The final Lambda CLI command I’m going to describe here is update-function-configuration. As you would expect this allows us to change the configuration parameters that we specified when we called create-function. Let’s update our Lambda to use a different handler function, specifically the StringIntegerBooleanLambda.handlerInt function from Part 4:

$ aws lambda update-function-configuration --function-name CliLambda --handler io.symphonia.StringIntegerBooleanLambda::handlerInt

Note that you only need to specify the function you’re updating, and then any configuration parameters that need to change. All other parameters will keep their original values.

Now let’s invoke our function to make sure the update really happened:

$ aws lambda invoke --invocation-type RequestResponse --function-name CliLambda --payload "5" outputfile.txt; cat outputfile.txt
{
   "StatusCode": 200
}
105

Yes it did!

These 4 commands, create-function, update-function-code, update-function-configuration, and invoke are all you need to do most of what we’ve done in this series without using the web console. One exception is looking at logs. Of course that’s possible through the API / CLI too, but for now stick to the console for that.

Other tools

In this installment you’ve seen an introduction to the AWS CLI, specifically to manipulate Lambda functions. However this is a fairly low-level abstraction for Lambda applications and as you can imagine there are other tools which build on the CLI and API to provide a higher level interface. I’m going to mention some here — for any more significant detail you should explore the links provided.

CloudFormation and SAM

CloudFormation is AWS’ declarative deployment tooling. You can use it to define, configure and deploy entire application ecosystems, including EC2 instances, Dynamo databases, and, yes, Lambda functions.

CloudFormation can get a little verbose so AWS also offer SAM — the Serverless Application Model. SAM allows you to define Serverless applications, i.e. those consisting of several Lambda functions, event sources, etc. SAM is an abstraction on top of CloudFormation and provides some useful shortcuts for Serverless scenarios.

CloudFormation and SAM don’t offer complete abstractions of the CLI / API — there are some things you can do in the lower-level tools that you can’t do in CloudFormation / SAM — but for applications of any significant complexity you’ll likely want to look into these higher level tools.

Terraform

Terraform is a popular infrastructure management tool, made available from the good folks at HashiCorp. Terraform offers Lambda integration and so you can use it to deploy and configure your Lambda applications. Note however that Terraform isn’t officially sanctioned by AWS and so you should test to make sure that it does what you’re expecting — these third party tools are in early days (at least with regard to Lambda) and so can occasionally have bugs and/or unexpected behavior.

Maven

Since we’re already using Maven, there are also plugins that will automate deployment, etc. One that we at Symphonia have used successfully is lambda-maven-plugin.

Eclipse

Finally there is an official Eclipse plugin for Lambda deployment, however I don’t recommend using this. Getting to grips with the command line tooling, as we’ve done in this part, is not complicated. Further, beyond helping you out with experimentation, it sets you down a good path for automating deployment pipeline tooling. The Eclipse plugin doesn’t help in this regard at all.

Next time

That brings us to the end of Part 5 of Learning Lambda. I introduced the AWS CLI and API, and we used a few commands to automate our workflow. I also briefly introduced some higher level tools.

Next time we’re going to look in depth at Lambda Invocation, and you can read Part 6 here. To see the other articles in this series please visit the series home page. To be alerted about future installments subscribe to our newsletter, follow Symphonia on twitter @symphoniacloud, and our blog at The Symphonium.

Need help with Lambda, or other Serverless technologies? We’re the experts! Contact us at Symphonia for expert advice, architectural review, training and on-team development.

« Symphonia's Spring Tour! Learning Lambda — Part 4 »