(aws-lambda-python-alpha): bundling fails with .venv dir in "entry" dir #22585
Closed
Description
Describe the bug
If a .venv directory is present in the function source directory (i.e. the value of entry to PythonFunction), the asset bundling fails after apparently trying to find the Python binary inside it. This presumably occurs because in my virtualenv, bin/python is a symlink to the original Python used as part of the virtualenv creation.
I often create a virtualenv in my function's directory to allow for code completion etc while working on the code.
I don't believe this used to happen; this project was successfully deployed on 2.35.0a0 and fails with 2.{46,47}.0a0. I have not bisected the library versions to verify.
Expected Behavior
cdk synth would succeed.
Current Behavior
% cdk synth
#1 [internal] load build definition from Dockerfile
#1 sha256:2cd2e2070c44df558ddddd66c54d2d34cc57055097780ab0811ad693df39bb6e
#1 transferring dockerfile: 1.28kB done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:452d082e50a39ce5b9ce6ad02ec379aac436b131d15ce9c1bd04817a1aab4273
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for public.ecr.aws/sam/build-python3.9:latest
#3 sha256:c037d438954a63dd514169e8b70bd8c1f25f7620154f690e39b0cde3ca7a28d9
#3 DONE 0.5s
#4 [1/2] FROM public.ecr.aws/sam/build-python3.9@sha256:0c64875b97960e69e8f17a1c780e2efd2db3a80ca50bc1abc5967f6ee9b73e4d
#4 sha256:59d114c6ecb4d0f4ce048aa96caa848d13b68f73dee357a97d26f844713fe9aa
#4 DONE 0.0s
#5 [2/2] RUN python -m venv /usr/app/venv && mkdir /tmp/pip-cache && chmod -R 777 /tmp/pip-cache && pip install --upgrade pip && mkdir /tmp/poetry-cache && chmod -R 777 /tmp/poetry-cache && pip install pipenv==2022.4.8 poetry && rm -rf /tmp/pip-cache/* /tmp/poetry-cache/*
#5 sha256:f19856fe4384001478c5a68440ec33198ae19c73da3a1809ae18bd0258b16b66
#5 CACHED
#6 exporting to image
#6 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#6 exporting layers done
#6 writing image sha256:1fa9791ea44d050b3f7fd06a80efc36eb27ecad35727b902c659fce52cefd431 done
#6 naming to docker.io/library/cdk-8a5e4a750532e5be099ef9223a66e2fb24a66b4837382d6e05feb49d7c9dfdf1 done
#6 DONE 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Bundling asset CdkAppconfigChangeCalendarExtnStack/check_fn/Code/Stage...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
cp: cannot stat ‘/asset-input/.venv/bin/python’: No such file or directory
cp: cannot stat ‘/asset-input/.venv/bin/python3’: No such file or directory
cp: cannot stat ‘/asset-input/.venv/bin/python3.9’: No such file or directory
jsii.errors.JavaScriptError:
Error: Failed to bundle asset CdkAppconfigChangeCalendarExtnStack/check_fn/Code/Stage, bundle output is located at /Users/sewardjm/src/cdk-appconfig-change-calendar-extn/cdk.out/asset.eb48a51f2a1be51e1217c2c5c80976983ad77c46566c31399d2a69357bbd1499-error: Error: docker exited with status 1
at AssetStaging.bundle (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/core/lib/asset-staging.js:2:614)
at AssetStaging.stageByBundling (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:4506)
at stageThisAsset (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:1867)
at Cache.obtain (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/core/lib/private/cache.js:1:242)
at new AssetStaging (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2262)
at new Asset (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:736)
at AssetCode.bind (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4628)
at new Function (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:2803)
at new PythonFunction (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/jsii-kernel-Q5NaOz/node_modules/@aws-cdk/aws-lambda-python-alpha/lib/function.js:40:9)
at Kernel._create (/private/var/folders/s7/_09xb8j906j6kxr9cd_dl9840000gr/T/tmps0mbv5hk/lib/program.js:7485:29)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/app.py", line 10, in <module>
CdkAppconfigChangeCalendarExtnStack(
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/.venv/lib/python3.10/site-packages/jsii/_runtime.py", line 109, in __call__
inst = super().__call__(*args, **kwargs)
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/cdk_appconfig_change_calendar_extn/cdk_appconfig_change_calendar_extn_stack.py", line 28, in __init__
function = aws_python.PythonFunction(
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/.venv/lib/python3.10/site-packages/jsii/_runtime.py", line 109, in __call__
inst = super().__call__(*args, **kwargs)
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/.venv/lib/python3.10/site-packages/aws_cdk/aws_lambda_python_alpha/__init__.py", line 599, in __init__
jsii.create(self.__class__, self, [scope, id, props])
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/.venv/lib/python3.10/site-packages/jsii/_kernel/__init__.py", line 321, in create
response = self.provider.create(
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/.venv/lib/python3.10/site-packages/jsii/_kernel/providers/process.py", line 350, in create
return self._process.send(request, CreateResponse)
File "/Users/sewardjm/src/cdk-appconfig-change-calendar-extn/.venv/lib/python3.10/site-packages/jsii/_kernel/providers/process.py", line 332, in send
raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: Failed to bundle asset CdkAppconfigChangeCalendarExtnStack/check_fn/Code/Stage, bundle output is located at /Users/sewardjm/src/cdk-appconfig-change-calendar-extn/cdk.out/asset.eb48a51f2a1be51e1217c2c5c80976983ad77c46566c31399d2a69357bbd1499-error: Error: docker exited with status 1
Reproduction Steps
CDK code:
from aws_cdk import Stack
from aws_cdk import aws_lambda
from aws_cdk import aws_lambda_python_alpha as aws_python
from constructs import Construct
class CdkAppconfigChangeCalendarExtnStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
function = aws_python.PythonFunction(
self,
"check_fn",
index="index.py",
runtime=aws_lambda.Runtime.PYTHON_3_9,
handler="lambda_handler",
entry="lambda",
)In the lambda/ directory,
python -v venv .venvThen synth.
Possible Solution
I was hoping there would be an option to exclude files/directories from the build but couldn't see one in the docs.
Or, CDK should not attempt to use the existing virtualenv during the build, or emit a warning if one is found, which would have aided my troubleshooting.
Additional Information/Context
No response
CDK CLI Version
2.46.0 (build 5a0595e)
Framework Version
No response
Node.js Version
v18.10.0
OS
macOS 12.6
Language
Python
Language Version
Python 3.10.4
Other information
No response