Published 2021-04-29. Last modified 2021-05-03.
Time to read: 3 minutes.

This page is part of the posts collection, categorized under AWS Lambda, Django, Docker, Linux, Python.

This article is a work in progress. Some of it may be incorrect, and some thoughts might lead nowhere. I am publicly posting it in this state so I can discuss it with others. This article will be improved as information becomes available.

Goal

As previously discussed, Buildah is a drop-in replacement for using docker build and a Dockerfile. Buildah’s build-using-dockerfile, or bud argument makes it behave just like docker build does.

The goal of this article is to use Buildah / podman to create an Open Container Initiative (OCI) container image with a Django app, including the Python 3.8 runtime installed. The Django app will start when the container is created. The code for the Django app will be stored on the local machine where its source code can be edited, and it will be mapped into the container from the host system. Changes made to the code from the host system will be immediately visible inside the container.

TODO

Background: AWS publishes Deploying Python with an AWS base image, but that does not discuss running or testing. Create a Lambda function with the console is a more complete article, but is focused on using the web browser console, using Docker, and Node.js. So many differences from the desired goal make the articles difficult to translate to AWS CLI, Buildah / podman and Python.

Talk about the AWS Lambda Runtime Interface Emulator, compare and contrast with the AWS Lambda Python Runtime Interface Client.

Compare these AWS Lambda Runtimes with other, equivalant runtimes.

OCI images are swapped in when AWS Lambda is invoked. Do larger images cost more to use? If so, discuss.

Deploy Python Lambda function with Container Image

Consider this Dockerfile, which launches a Python 3.8 command-line application in a manner compatible with AWS Lambda:

Dockerfile

FROM public.ecr.aws/lambda/python:3.8

COPY app.py ./
CMD ["app.handler"]

Following is a small Python app called app.py, which will be launched by the Dockerfile. The Python app can be run as an AWS Lambda program because it implements the handler entry point.

app.py

import sys

def handler(event, context):
    return f"Hello from AWS Lambda using Python {sys.version}!"

Build Image

Buildah builds the image, just the same way that Docker would:

Shell

$ buildah bud -t hello .
STEP 1: FROM public.ecr.aws/lambda/python:3.8
Getting image source signatures
Copying blob 03ac043af787 skipped: already exists
Copying blob 420e64b38334 done
Copying blob ff259f25b075 done
Copying blob 3ff716981d54 done
Copying blob 6b6e623a48a8 done
Copying blob 9aa8f1e66d54 done
Copying config 67dc3a2a54 done
Writing manifest to image destination
Storing signatures
STEP 2: COPY app.py   ./
STEP 3: CMD ["app.handler"]
STEP 4: COMMIT hello
Getting image source signatures
Copying blob 683073d39306 skipped: already exists
Copying blob 658871a69e1f skipped: already exists
Copying blob 6fa16f35d11e skipped: already exists
Copying blob d6fa53d6caa6 skipped: already exists
Copying blob 61c062506436 skipped: already exists
Copying blob 1c1d66a5fd95 skipped: already exists
Copying blob 33af9dc6463a done
Copying config 98862dfd20 done
Writing manifest to image destination
Storing signatures
--> 98862dfd208
98862dfd2087152ee821553d6cb1c033e735af06e5f11c814bcc9300fb65584e 

Test Lambda function Locally

Before calling the Lambda API from a local container, first run the container. Containers default to running in the foreground, but the -d option causes a container to be run as a background process. This container is given the name hello, the external HTTP endpoint at 9000 is mapped to internal port 8080, and the latest version of the hello lambda function is run in the container.

Shell

$ podman run \
  -d \
  --name hello \
  -p 9000:8080 \
  hello:latest
d4d296e4c91d01c98d312e3f79599dca53990d95218e94bbdfbbac6a43cde9e8 

Call the local version of the Lambda API:

Shell

$ curl \
  -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" \
  -d '{}'
"Hello from AWS Lambda using Python 3.8.9 (default, Apr 20 2021, 13:58:54) \n[GCC 7.3.1 20180712 (Red Hat 7.3.1-12)]!" 

Stop the container called hello.

Shell

$ podman stop hello
96cc1b1ed92368a1165d6a6ad0b1e5544d4ac751b64e94df33bf2322e6d7b30c 

Create AWS ECR Repository

AWS provides a registry for OCI-compatible image repositories called the AWS Elastic Container Registry (ECR).

Shell

$ aws ecr create-repository help
CREATE-REPOSITORY() CREATE-REPOSITORY()

NAME
create-repository -

DESCRIPTION
Creates a repository. For more information, see Amazon ECR Repositories
in the Amazon Elastic Container Registry User Guide .

See also: AWS API Documentation

See 'aws help' for descriptions of global parameters.

SYNOPSIS
create-repository
--repository-name <value>
[--tags <value>]
[--image-tag-mutability <value>]
[--image-scanning-configuration <value>]
[--cli-input-json <value>]
[--generate-cli-skeleton <value>]

OPTIONS
--repository-name (string)
The name to use for the repository. The repository name may be spec-
ified on its own (such as nginx-web-app ) or it can be prepended
with a namespace to group the repository into a category (such as
project-a/nginx-web-app ).

--tags (list)
The metadata that you apply to the repository to help you categorize
and organize them. Each tag consists of a key and an optional value,
both of which you define. Tag keys can have a maximum character
length of 128 characters, and tag values can have a maximum length
of 256 characters.

(structure)
The metadata that you apply to a resource to help you categorize
and organize them. Each tag consists of a key and an optional
value, both of which you define. Tag keys can have a maximum
character length of 128 characters, and tag values can have a
maximum length of 256 characters.

Key -> (string)
One part of a key-value pair that make up a tag. A key is a
general label that acts like a category for more specific tag
values.

Value -> (string)
The optional part of a key-value pair that make up a tag. A
value acts as a descriptor within a tag category (key).

Shorthand Syntax:

Key=string,Value=string ...

JSON Syntax:

[
{
"Key": "string",
"Value": "string"
}
...
]

--image-tag-mutability (string)
The tag mutability setting for the repository. If this parameter is
omitted, the default setting of MUTABLE will be used which will al-
low image tags to be overwritten. If IMMUTABLE is specified, all im-
age tags within the repository will be immutable which will prevent
them from being overwritten.

Possible values:

o MUTABLE

o IMMUTABLE

--image-scanning-configuration (structure)
The image scanning configuration for the repository. This setting
determines whether images are scanned for known vulnerabilities af-
ter being pushed to the repository.

scanOnPush -> (boolean)
The setting that determines whether images are scanned after be-
ing pushed to a repository. If set to true , images will be
scanned after being pushed. If this parameter is not specified,
it will default to false and images will not be scanned unless a
scan is manually started with the StartImageScan API.

Shorthand Syntax:

scanOnPush=boolean

JSON Syntax:

{
"scanOnPush": true|false
}

--cli-input-json (string) Performs service operation based on the JSON
string provided. The JSON string follows the format provided by --gen-
erate-cli-skeleton. If other arguments are provided on the command
line, the CLI values will override the JSON-provided values. It is not
possible to pass arbitrary binary values using a JSON-provided value as
the string will be taken literally.

--generate-cli-skeleton (string) Prints a JSON skeleton to standard
output without sending an API request. If provided with no value or the
value input, prints a sample input JSON that can be used as an argument
for --cli-input-json. If provided with the value output, it validates
the command inputs and returns a sample output JSON for that command.

See 'aws help' for descriptions of global parameters.

EXAMPLES
Example 1: To create a repository

The following create-repository example creates a repository inside the
specified namespace in the default registry for an account.

aws ecr create-repository \
--repository-name project-a/nginx-web-app

Output:

{
"repository": {
"registryId": "123456789012",
"repositoryName": "sample-repo",
"repositoryArn": "arn:aws:ecr:us-west-2:123456789012:repository/project-a/nginx-web-app"
}
}

For more information, see Creating a Repository in the Amazon ECR User
Guide.

Example 2: To create a repository configured with image tag immutabil-
ity

The following create-repository example creates a repository configured
for tag immutability in the default registry for an account.

aws ecr create-repository \
--repository-name sample-repo \
--image-tag-mutability IMMUTABLE

Output:

{
"repository": {
"registryId": "123456789012",
"repositoryName": "sample-repo",
"repositoryArn": "arn:aws:ecr:us-west-2:123456789012:repository/sample-repo",
"imageTagMutability": "IMMUTABLE"
}
}

For more information, see Image Tag Mutability in the Amazon ECR User
Guide.

Example 3: To create a repository configured with a scanning configura-
tion

The following create-repository example creates a repository configured
to perform a vulnerability scan on image push in the default registry
for an account.

aws ecr create-repository \
--repository-name sample-repo \
--image-scanning-configuration scanOnPush=true

Output:

{
"repository": {
"registryId": "123456789012",
"repositoryName": "sample-repo",
"repositoryArn": "arn:aws:ecr:us-west-2:123456789012:repository/sample-repo",
"imageScanningConfiguration": {
"scanOnPush": true
}
}
}

For more information, see Image Scanning in the Amazon ECR User Guide.

OUTPUT
repository -> (structure)
The repository that was created.

repositoryArn -> (string)
The Amazon Resource Name (ARN) that identifies the repository.
The ARN contains the arn:aws:ecr namespace, followed by the re-
gion of the repository, AWS account ID of the repository owner,
repository namespace, and repository name. For example,
arn:aws:ecr:region:012345678910:repository/test .

registryId -> (string)
The AWS account ID associated with the registry that contains
the repository.

repositoryName -> (string)
The name of the repository.

repositoryUri -> (string)
The URI for the repository. You can use this URI for Docker push
or pull operations.

createdAt -> (timestamp)
The date and time, in JavaScript date format, when the reposi-
tory was created.

imageTagMutability -> (string)
The tag mutability setting for the repository.

imageScanningConfiguration -> (structure)
The image scanning configuration for a repository.

scanOnPush -> (boolean)
The setting that determines whether images are scanned after
being pushed to a repository. If set to true , images will be
scanned after being pushed. If this parameter is not speci-
fied, it will default to false and images will not be scanned
unless a scan is manually started with the StartImageScan
API.



CREATE-REPOSITORY()

The following creates an AWS ECR image repository in called hello within the test namespace. Images are scanned for known vulnerabilities after they are pushed to the repository.

Shell

$ aws ecr create-repository \
  --repository-name test/hello \
  --image-scanning-configuration scanOnPush=true
{
  "repository": {
      "repositoryArn": "arn:aws:ecr:us-east-1:031372724784:repository/test/hello",
      "registryId": "031372724784",
      "repositoryName": "test/hello",
      "repositoryUri": "031372724784.dkr.ecr.us-east-1.amazonaws.com/test/hello",
      "createdAt": 1620232146.0,
      "imageTagMutability": "MUTABLE",
      "imageScanningConfiguration": {
          "scanOnPush": true
      }
  }
} 

Tag Image

podman tag – Assigns a new image name to an existing image. A full name refers to the entire image name, including the optional tag after the :. If there is no tag provided, then podman will default to latest for both the image and the target-name.   – From man podman-tag.

Shell

$ IMAGE_NAME=hello

$ IMAGE_VERSION=0.1

$ podman tag $IMAGE_NAME:$IMAGE_VERSION \
  $REGISTRY/$IMAGE_NAME:$IMAGE_VERSION

$ podman images
REPOSITORY                                                  TAG              IMAGE ID      CREATED         SIZE
localhost/hello                                             0.1              98862dfd2087  39 minutes ago  622 MB
752246127823.dkr.ecr.us-east-1.amazonaws.com/hello          latest           98862dfd2087  39 minutes ago  622 MB
public.ecr.aws/lambda/python                                3.8              67dc3a2a54fb  25 hours ago    622 MB
752246127823.dkr.ecr.us-east-1.amazonaws.com/ancientwarmth  latest           5d18ea34fc30  28 hours ago    2.03 GB
localhost/ancientwarmth                                     latest           5d18ea34fc30  28 hours ago    2.03 GB
<none>                                                      <none>           40ef32b39cf4  5 days ago      622 MB
docker.io/library/amazonlinux                               latest           53ef897d731f  5 days ago      170 MB
docker.io/amazon/aws-lambda-python                          3.8              e12ea62c5582  9 days ago      622 MB
docker.io/library/alpine                                    latest           6dbb9cc54074  2 weeks ago     5.88 MB
docker.io/lambci/lambda                                     build-python3.8  714c659c9f6f  3 months ago    2.03 GB 

Push Image to ECR

Podman will use the IAM credentials for the dev profile in ~/.aws/credentials to log into that AWS account:

~/.aws/credentials

[default]
aws_access_key_id = ********************
aws_secret_access_key = ****************************************
region = us-east-1

[dev]
aws_access_key_id = ********************
aws_secret_access_key = ****************************************
region = us-east-1

Shell

$ export AWS_PROFILE=dev

$ AWS_ACCOUNT="$( aws sts get-caller-identity \
  --query Account \
  --output text
)"

$ AWS_REGION="$( aws configure get region )"

$ REGISTRY="$AWS_ACCOUNT.dkr.ecr.$AWS_REGION.amazonaws.com"

$ aws ecr get-login-password \
    --region "$AWS_REGION" | \
  podman login \
    --password-stdin \
    --username AWS \
    "$REGISTRY"
Login Succeeded! 

Now that podman is logged into AWS, use podman push the image to AWS ECR:

Shell

$ podman push test/$IMAGE_NAME \
  $REGISTRY/$IMAGE_NAME:$IMAGE_VERSION
Getting image source signatures
Copying blob 692590faf2d1 [--------------------------------------] 8.0b / 8.2MiB
Copying blob 397718cff58d [--------------------------------------] 8.0b / 206.2MiB
Copying blob 9ca787b1c91c [--------------------------------------] 8.0b / 93.1MiB
Copying blob ef26f5221b79 [--------------------------------------] 8.0b / 196.7MiB
Copying blob 0a3f69c27a89 [--------------------------------------] 8.0b / 316.4MiB
Copying blob 5b3cbb76df75 [--------------------------------------] 8.0b / 1.1GiB
Copying blob e9cad39831b0 [--------------------------------------] 8.0b / 3.5KiB
Error: Error copying image to the remote destination:
Error writing blob: Error initiating layer upload to /v2/ancientwarmth/blobs/uploads/ in
752246127823.dkr.ecr.us-east-1.amazonaws.com:
name unknown: The repository with name 'hello' does not exist in the registry with id '752246127823' 

The results of an image scan for the new repository can be retrieved as follows:

Shell

$ aws ecr describe-image-scan-findings \
  --repository-name test/hello \
  --image-id imageTag=tag_name

Deploy Python Lambda function with Container Image

Podman can invoke the app using an OCI container with Amazon Linux 2 and Python 3.8:

Shell

$ podman container run -ti \
  public.ecr.aws/lambda/python:3.8 \
  blog/docker/podman/app.py
Trying to pull public.ecr.aws/lambda/python:3.8...
Getting image source signatures
Copying blob 1de4740de1c2 done
Copying blob 03ac043af787 done
Copying blob 2e2bb77ae2dc done
Copying blob 842c9dce67e8 done
Copying blob df513d38f4d9 done
Copying blob 031c6369fb2b done
Copying config e12ea62c55 done
Writing manifest to image destination
Storing signatures
time="2021-05-02T23:38:30.971" level=info msg="exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)" 

Share to Twitter, Hacker News, LinkedIn.