Hello kid. You may be here because you clicked on a promising search result about a method to run Python 2.7 lambda after the deprecation of the official AWS runtime. You can celebrate: this post is exactly what you are looking for.
First of all, I would like to be clear:
IT IS A BAD PRACTICE TO CONTINUE DEPLOYING DEPRECATED RUNTIMES AS PYTHON 2.7 IN AWS LAMBDA FUNCTIONS
I strongly recommend migrating to a Python 3.x runtime. However, you may be trapped in a planning, a budget, or an evil dependency supporting only Python 2.7, so… let’s do it…
How to do it ?
AWS Lambda executors are sort of virtual machines, put into life by a magic AWS thing called firecracker. Inside the “micro VM”, there is an Amazon Linux OS, with all the required libraries to run your application. When deploying AWS Python 2.7 lambda, you actually instruct AWS to deploy the “Python 2.7 OS image” the service team prepared for you.
These “official” runtimes have limited customization. You can add some of your libraries, change the function handler, and so on, but you cannot change the underlying OS binaries, and it is fine because it is not required for many workloads.
However, since december 2020, you can now control (almost) entirely the OS you deploy in the micro-VM ! You can just provide a Docker image, the entrypoint to run and that’s it !
Moreover, for test purposes, AWS provides the Docker images it uses in its official runtimes. In particular, the Python 2.7 lambda runtime image can be pulled from a public AWS ECR repository.
Do you see it happening ? YES ! We can take the “official” Python 2.7 Docker image, put our original source code inside, add a bit of magic and we will have a lambda running in a runtime very very similar to the official legacy 2.7 one !
A Python 2.7 custom lambda runtime using Docker
A first step
We must create a DockerFile derivating from the official Amazon Python 2.7 Docker image, and add our source code in it:
And… It will not work because you cannot specify the “lambda handler” when deploying Docker-based lambda functions.
Get around the missing « lambda handler » parameter
When you configure an official Python runtime, you just need to provide the “lambda handler” representing the file and the function to run. Then, the bootstrap included into the Python 2.7 lambda OS will read it from a reserved _HANDLER environment variable magically pushed in the executor, and run the specific file and function. The mechanism is similar when you use a custom runtime: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html#runtimes-custom-build
Unfortunately, the _HANDLER environment variable is a reserved word. You cannot monkey patch the Lambda variables on your deployed Lambda-Docker function with it to tell the Python bootstrap to use it.
However, this is “just” an environment variable somewhere in the environment, and if we can control the lambda entrypoint, we can inject it. Good news: you CAN control the entrypoint !
How do the lambda entrypoints work ?
When running an AWS Lambda, the executor will look for the “bootstrap” file. Official runtimes usually store it in /var/runtime/bootstrap, in custom runtimes you must provide it with the function source code, and with Docker-based lambda function, you can tell the executor where it is by specifying an “ENTRYPOINT” configuration parameter (see screenshot above). Inside the original AWS Python 2.7 Docker image, there already is this bootstrap file in /var/runtime/bootstrap. This bootstrap will then invoke Python 2.7 and run the /var/runtime/awslambda/bootstrap.py file, executing your user-defined function.
You can have a look at the original /var/runtime/bootstrap file when executing a shell in the official image:
ADD bootstrap /var/runtime/bootstrap
RUN chmod 755 /var/runtime/bootstrap
ADD function.py /var/task/function.py
Let’s replace the image with an updated one:
docker build -t my-custom-python-runtime .<br>docker tag my-custom-python-runtime:latest 123456789012.dkr.ecr.eu-west-1.amazonaws.com/my-custom-python-runtime:latest<br>docker push 123456789012.dkr.ecr.eu-west-1.amazonaws.com/my-custom-python-runtime:latest
And… it works !
START RequestId: 24774993-bd18-4d72-af16-9ac63d91182b Version: $LATEST
END RequestId: 24774993-bd18-4d72-af16-9ac63d91182b
REPORT RequestId: 24774993-bd18-4d72-af16-9ac63d91182b Duration: 1.06 ms Billed Duration: 523 ms Memory Size: 128 MB Max Memory Used: 45 MB Init Duration: 521.89 ms
An attention point here: you can see that the cold boot delay hit the performance very hard (+500ms). Docker-based functions runtimes may be slower to start than ready-to-use runtimes.
This article has demonstrated how you can continue creating new Python 2.7 lambda after the official runtime deprecation. These steps will provide an environment execution very similar to the one currently deployed in the official runtime, because you directly use the Amazon Python 2.7 Docker image.
However, I strongly encourage your team to migrate to an officially supported runtime, such as Python 3.x for your lambdas. The Python 2.7 runtime will not receive security updates anymore, and continuing to use it is a risk you must seriously take into account.
Comme toujours, nous nous efforçons de suivre le rythme des innovations et de v...
A PROPOS DE L'AUTEUR
C'est assez jeune que Nicolas s'est mis à faire travailler les machines à sa place. S'il a commencé à coder en C/C++, il est rapidement tombé dans le Java. Linuxien convaincu, il a pendant longtemps écrit son code avec Vim. Certains disent que c'est un troll, d'autres qu'il pose juste beaucoup de questions...
Le blog reBirth
Nous luttons contre les raccourcis intellectuels, proposons des alternatives, challengeons les pratiques, partageons nos expériences et provoquons une réaction. En ce sens nous ENTREPRENONS et révélons les singularités.