I lately started playing with AWS Lambda for few reasons. I become interested in serverless architecture, ways to save money while running apps and I wanted finally to learn Python. I’m a network engineer, not a software developer. I like cloud computing and see it as important part of market now. So that was an opportunity for me to learn something new. Now I want to share my knowledge with you and show you how to import your Python application to Lambda.

In my tutorial I want to show you that Lambda and programming is something interesting that you can use for everyday work whatever you do. Of course Lambda tutorials are already available on Internet but they show you how to make new application from scratch. I want to show you how to import your own small Python application to Lambda, required changes to the code, python environment, testing approach and finally how to expand it using other AWS services. This post is just first chapter!

What is FaaS?

Lambda is service that allows us to run code without maintaining whole required infrastructure. This approach is commonly known under names as serverless computing, Function as a Service (FaaS) or microservices. Last name is quite wide to cover general approach of splitting the code into smaller independent parts.

To describe FaaS simply – you as a developer care about code, the infrastructure to run it is transparent and maintained by cloud operator. You don’t have to pay to maintain whole infrastructure instead you pay just for code execution. By its nature serverless applications are executed by triggers – if something happen then function is executed. There are plenty of documents about FaaS and serverless architecture on Internet and primarily on cloud service providers.

AWS Lambda was the first service of this type and I agree it was a game changer in the industry. It’s intensively developed by AWS engineers and new features are expected any time. Other players have their own similar products – Azure Functions in Azure and Google Cloud Functions in Google Cloud. Keep in mind that features and available programming languages differs between the cloud!

My Python developer environment

If you get back in time a little, or just scroll down posts on my blog, you will find that I already presented few apps here that I wrote in Perl, Node.js and Swift. As IDE tool for coding I used free Atom suite and Apple XCode for Swift. I started using it for writing in Python but I was missing context help or function auto competition etc. Plugins like autocomplete-python and Kite were not really good. Don’t get me wrong – Atom is fantastic modular tool for developers but somehow it is not that good if you just started to learn particular programming language.

I still like Atom but for Python my friend recommended me switching to PyCharm which is available as free and paid product. PyCharm is much better if you focus on Python. It can do pretty much everything as Atom with plugins but a little better, smoother and have very good debugger included as well as run tool with console. And you can easily manage libraries and Python versions used to run your app.

Remeber – always use tools that suites your needs and personal preferences. I’m just showing you what I like

My Python script – Certificate signing tool

Do you know Let’s Encrypt service? It’s a service and Certificate Authority that you can use to obtain free SSL certificate for your website. It’s CA certificate is trusted by all browsers by default. Signing process is simplified and you usually use script to get your request signed. It’s working fine as long as you own your domain and it’s public.

I run virtual labs on VIRL quite often, I help setup test environments and PoC labs using mix of physical hardware and virtualized. I often need to have SSL certificates there. In post How to act as your own local CA and sign certificate request from ASA I showed you how using openssl you can create your own CA certificate and act as signing authority. Adding this CA certificate to lab computers or engineers laptops will make all device certificates signed by this CA trusted and this is what I often need. Presented method is nice but you have to copy requests files to VM where you sign it, you must keep the VM running and secure and if you perform signing often it is not really flexible.

My Python app in version I present here is simplified version for demo purposes. I may release full version on GitHub one day.

from OpenSSL import crypto

def createCertificate(req, issuerCert, issuerKey, serial, notBefore, notAfter, digest="sha256"):
    cert = crypto.X509()
    cert.set_serial_number(serial)
    cert.gmtime_adj_notBefore(notBefore)
    cert.gmtime_adj_notAfter(notAfter)
    cert.set_issuer(issuerCert.get_subject())
    cert.set_subject(req.get_subject())
    cert.set_pubkey(req.get_pubkey())

    cert.add_extensions((
        crypto.X509Extension(b'basicConstraints', False,
                             b'CA:FALSE'),
        crypto.X509Extension(b'keyUsage', True,
                             b'digitalSignature, keyEncipherment'),
        crypto.X509Extension(
            b'authorityKeyIdentifier', False, b'keyid:always', issuer=issuerCert),
        crypto.X509Extension(
            b'extendedKeyUsage', False, b'serverAuth, clientAuth')
    ))

    cert.sign(issuerKey, digest)
    return cert

ca_cert_file = open('/Volumes/home/CA/ca/signing-ca.crt', 'r')
ca_cert = crypto.load_certificate(crypto.FILETYPE_PEM, ca_cert_file.read())
ca_cert_file.close()
ca_key_file = open('/Volumes/home/CA/ca/root-ca/private/root-ca.key', 'r')
ca_key = crypto.load_privatekey(crypto.FILETYPE_PEM, ca_key_file.read(), b'test')
ca_key_file.close()
crt_req_file = open('asav-3.virl.lan.crt.csr')
crt_req = crypto.load_certificate_request(
    crypto.FILETYPE_PEM, crt_req_file.read())
crt_req_file.close()
new_crt_serial = 990099

new_crt = createCertificate(
    crt_req, ca_cert, ca_key, new_crt_serial, 0, 230400)

msgCertificateParameters = crypto.dump_certificate(crypto.FILETYPE_TEXT, new_crt)
print(msgCertificateParameters.decode('utf-8'))
msgCertificateParameters = crypto.dump_certificate(crypto.FILETYPE_PEM, new_crt)
print(msgCertificateParameters.decode('utf-8'))

open('asav-3.virl.lan.crt', 'wb').write(crypto.dump_certificate(crypto.FILETYPE_PEM, new_crt))

This Python application is available on my GitHub.

First thing you should notice are static definitions of file names. We will change that when we run it on Lambda and add frontend and triggers in next chapters of this tutorial. This app base on several examples like this that I found on Internet.

Application consist of main code and createCertificate() function called from main section. It’s opening CSR request, CA certificate and the key file and along with few other parameters provide them as arguments to createCertificate() function.

The createCertificate() itself create X509 certificate variable, set provided parameters and add few more like X509 Extensions, sign the certificate and then return it.

When main block of code gets certificate I’m writing it’s parameters to console output and signed certificate file on disk.

I think even if you are beginner in Python you shouldn’t have problem understanding this application. It can be executed in Python version 2.7 and 3.6 as well. I used pyOpenSSL library here – it’s available on most Linux distributions but not on AWS Lambda (it will be important shortly!). Feel free to modify paths and play with the script by yourself.

Create Lambda function

To create function in Lambda we need to open Lambda service dashboard and click Create a Lambda function button. In first step of the creator we need to choose template of code we may wish to use. At this moment we will use Blank Function template. In next step we define triggers, so what events will execute the function. Let’s leave it with no triggers now.

Then we configure Lambda function parameters like function name and language we will use.

Lambda Function Creator

To upload code we need to create ZIP archive on our local computer, download it from S3 bucket or just paste into the provided editor. Last option is good only for very short applications but we will use it now. Remeber that we selected empty template for our function? But there is sill definition of function called lambda_handler() in text area. This name is connected with other parameters you need to set.

Lambda function handler

It’s very important to understand the Handler field. It contains two names separated by dot (.). First one is lambda_function. This is the name of the python file with the code we upload to Lambda for execution in the ZIP archive. It is not relevant if you paste your code to text area. Second part is lambda_handler and it’s the function name that is our main one and have to be executed. Now you know why in text field you have lambda_handler() defined.

Each Lambda function must have proper policies defined so it can run and access other services. We will focus on this later. For now create new role and assign Simple Microservice permissions policy.

Remove everything in text area and copy-paste source code without any changes there. Because Lambda charge us per execution it’s worth to adjust Timeout in Advanced settings so in case of errors AWS won’t charge us for waiting. I use value of 3 seconds. We can now review provided configuration and finish the creator.

Invoking function on Lambda

To execute the code click Test button. You will be prompted for JSON structure where you provide trigger parameters but ignore it for now by accepting default values. We won’t use them now in our code.

The execution will fail and following error message appear on the screen

{
 "errorMessage": "Unable to import module 'lambda_function'"
}

In log output we find detailed information

Unable to import module 'lambda_function': No module named OpenSSL

That was expected because pyOpenSSL is not included in Lambda by default. So what we have to do is create locally sandboxed Python environment with pyOpenSSL and import it to Lambda. Please check my post How to create Python sandbox archive for AWS Lambda with step-by-step instruction how to do that.

Our archive must contain Python sandbox with required libraries and lambda_function.py file with the code in root of the archive. And we need to put main part of our program in function we defined as Handler parameters. It cannot be outside de function because Lambda need to have start point for the program. So pretty much what we do is:

Lambda requires all code to be within functions and we need to specify in handler which function Lambda have to execute

Important: python files must be have read permission set for owner, group and other. You can use command chmod uog+r lambda_function.py to set it before adding to archive. This is not listed clearly in the documentation. Error message is also misleading – logs will tel you that module cannot be important but won’t list permissions as a reason.

You may experience error like that

Unable to import module 'lambda_function': No module named cryptography.hazmat.bindings.openssl.binding

That means archive with Python sandbox was not created properly and some dependencies are missing. That’s most likely if error is related to non-standard library installed via pip into sandbox.

If Python script is properly parsed and executed in this example you will experience other error

{
  "stackTrace": [
    [
      "/var/task/lambda_function.py",
      44,
      "lambda_handler",
      "ca_cert_file = open('/Volumes/home/CA/ca/signing-ca.crt', 'r')"
    ]
  ],
  "errorType": "IOError",
  "errorMessage": "[Errno 2] No such file or directory: '/Volumes/home/CA/ca/signing-ca.crt'"
}

This JSON structure provide information about what kind of problem was experienced. It contains detailed information about file, code line and executed code that failed. In my example it just cannot open the file defined in static path which we expect and fix in next chapter

AWS Lambda CLI

In most cases you will upload ZIP archive with your code to Lambda. Using web browser and AWS Lambda dashboard for every change is not flexible way to work. Instead it’s better to use AWS CLI and make process automated in scripts.

Remember that if you want to use AWS CLI you need to have access key generated for each user using IAM and permission assigned. remember you may need to add information about region or profile if you don’t have those set in configuration file

To invoke function (with no trigger parameters) use

aws lambda invoke --function-name myFirstLambdaFunction outfile

First parameter is function name, the ‘outfile’ is name of file where we will write log output from function. What you see on console is only status code of executing the invoke method.

It’s also handy using CLI to update new ZIP archive to function

aws lambda update-function-code --function-name myFirstLambdaFunction --zip-file fileb://lambda_function.zip

We need to specify function name that we need to update and archive to upload. Note the structure of path to file, it starts with fileb:// (that’s not a typo, there is ‘b’ which stands for binary) and then path to archive. In my example archive is in same folder from which we execute the command.

The modified code that you can test on Lambda by yourself is available on my GitHub.

Summary

In this blog post we created our first Lambda function and (almost) executed first app. We used Python application which can be successfully run from command line but required small changes so it can be run on Lambda. What we had to do is:

1. Create sandbox Python environment with pyOpenSSL library because it’s not available by default on Lambda
2. Because we left default handler definition (lambda_function.lambda_handler) we put the code into file lambda_function.py and added it to root of the archive. Also the main part of code had to be put inside lambda_handler() function.
3. We uploaded archive to Lambda and invoked our function.

In next chapter I will show you how to use S3 service in Lambda so we can store there all files.