Lambda Powertools - great defaults for batteries that aren't quite (but should be) included

AWS Lambda is my preferred runtime environment for cloud-based code. Lambda has gained many, many, features over the eight years since it was launched. Most of these are focussed on the Lambda runtime - more CPUs, more memory, different CPU architectures, many more language runtimes, custom runtimes, integration with approximately a bajillion other services, etc.

One thing that the Lambda service hasn’t progressed with as quickly are things that we consider more at development time - thoughtful logging, common design solutions driven by Lambda’s constraints, and general patterns to help keep code clean as our Lambda-based applications grow.

Many of us outside of AWS have given our ideas of how best to approach these (I even wrote a book on it), and AWS provide a lot of guidance, but until recently there wasn’t much from AWS that was more tangible, in the form of service features or libraries.

Now though AWS has “Lambda Powertools” - a suite of utilities and libraries that are used at development time to improve the maintainability and operability of Lambda applications. The Python and Java versions of Lambda Powertools have been out for two years, but the first JavaScript / TypeScript version was released this past summer (2022). I do significantly more JS / TS coding these days than anything else, so this release was when I took much more of an interest.

My opinion can be summarized as “if you don’t already have mature solutions for these kinds of things, then definitely take a look at Lambda Powertools.” This article describes a little more why I think this, but I also mention a few concerns.

What is Lambda Powertools?

Lambda Powertools is a bundle of small, open sourced, libraries that can be used as part of solving certain requirements that are common to many, or all, Lambda-based applications. Since these requirements are code-time related, Powertools is a language specific library bundle, and at time of writing there are different bundles for each of Python, Java, and JavaScript / TypeScript.

For example the Lambda Runtime has a logging capability that all messages written to “standard out” are captured for subsequent analysis within CloudWatch Logs. This is a great runtime feature, however when you’re thinking about building applications that run in a production environment you need to consider what messages you log in different environments, and in what format. In other words you need log levels, and structured logging. The Lambda service out-of-the-box doesn’t do anything to help you in this regard.

Enter the Logging module of Lambda Powertools. First of all it provides a simple, clean API for both log levels and structured logging. If you use Powertools Logging across your whole application you’ll have a consistent way of logging just what you want, in a format that can be more easily analyzed during an incident.

In addition to just being an API, the Powertools Logging module also provides additional features - configuration, logging system level events (cold starts), enriching your log messages with contextual detail, and more. In other words it’s a good logging library and it adds extra value because it’s aware of its Lambda context.

Why Powertools is better than many other options

Because of this combination of elements - clean API + Lambda awareness - Powertools Logging already wins over other libraries that are runtime-generic. As such I recommend Powertools Logging in its own right if you don’t already have a solid logging solution in place.

Things get even better when you combine multiple Powertools modules together, because while they are separate libraries that you can choose to use independently, they also share common themes and configuration.

Here are two examples.

First, the Logging, Tracing, and Metrics modules all enrich events with a configured “service name” of the application. But a nice part of how this is implemented is that they can all (optionally) use a common environment variable, POWERTOOLS_SERVICE_NAME, to configure this behavior. This reduces the amount of overall system configuration.

Second - all the JavaScript / TypeScript modules provide the capability for code-setup to be performed in “Middy” middleware. Middy is an increasingly popular library for structuring setup code in multi-Lambda-Function applications, and provides a “cognitive on-ramp” from frameworks like Express that also use a middleware approach.

Because all the Powertools modules support Middy then it means that the code for initializing different Powertools modules looks the same, and therefore is easier to understand and maintain.

Why not Powertools?

Powertools is a well implemented solution to common concerns, however it’s not unique. Many organizations that are further down their Lambda-adoption lifecycle may already have perfectly good solutions in place for these kinds of needs. For example if you already use a good logging library, you’re using Datadog for metrics, and you’re using Honeycomb for tracing, then Powertools doesn’t have anything to offer you - at least not at the moment in the JavaScript libraries.

However the Python and Java versions of Powertools offer a lot more than just monitoring solutions, and start pushing much more into the land of “concrete serverless patterns” - things like how to solve the “at least once execution” behavior of Lambda. When the JS/TS version of Powertools includes ideas like this I might give it a more universal recommendation.

Batteries aren’t quite included?

In the title I said “batteries aren’t quite included”. What I mean by this is that Powertools comes from AWS but it doesn’t come from the Lambda team. In fact it doesn’t even currently come from a service team (the organizations that build AWS services), instead it’s been developed and managed by solutions architects in the AWS “field team”. While this may sound a bit like AWS “inside baseball” it introduces a few concerns:

• Powertools is an external addition to, not a unified part of, the Lambda service. This means there are no runtime elements of Powertools, and you won’t see any Powertools specific behavior in the AWS Web Console. Compare this with, say, Datadog or Honeycomb where your development-time implementation is much more closely tied to what you see at runtime.
• Powertools is not part of the Lambda roadmap or team, and so it’s not universal for all Lambda applications. For example right now there is only support from Powertools for Python, Java, and JavaScript / TypeScript apps, while Lambda itself supports many more languages.
• The longer-term outlook for Powertools is, to me, a little less certain than a “normal” AWS service. AWS have done a reasonable job with supporting their non-runtime aspects (think CLI + language SDKs). But I still have a slight concern about this.
• Occasionally the Powertools team might make choices that are inconsistent with the rest of AWS. At present there are a couple of examples of this. First, the documentation lives outside of the regular AWS knowledge base. Second, the Powertools team are asking us to use Lambda Layers for dependency management so that they can track usage (which I don’t recommend - stick with just the NPM library unless you have other reasons for using Layers).

Why does Powertools exist?

Developing production applications is more than just building domain logic, and deploying to a production runtime. There are many other activities involved - architecture, code packaging, testing, configuration, deployment, monitoring, integration with other systems, and more besides. While the Lambda service provides many wonderful runtime capabilities, and AWS provide a lot of learning material, AWS still hasn’t provided us with a complete “serverless platform” that provides a unified experience over most of the development and operations lifecycle of a serverless application.

Powertools doesn’t address all of these areas either, but it does at least make a start on some of them.

In summary I feel similarly to Powertools as I do to CDK. In isolation I think it’s a very useful addition to the AWS serverless landscape, and I congratulate the individual teams taking the initiative to produce it. I just wish that there was a coherent vision coming from AWS around these areas that didn’t require satellite teams to solve legitimate, pressing, and at this point years-old concerns.