Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logGroupIdentifier not working as expected in boto3 logs client call to get_log_events() #8371

Closed
T1ckL35 opened this issue Dec 6, 2024 · 1 comment · Fixed by #8442
Closed

Comments

@T1ckL35
Copy link

T1ckL35 commented Dec 6, 2024

While testing against code that uses the boto3 logs client get_log_events() logGroupIdentifier parameter we have noticed that it always returns the following, even when a log group, log stream and events have been correctly created via Moto:
E botocore.errorfactory.ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the GetLogEvents operation: The specified log group does not exist.

Other slight variations include the output of the describe_log_groups and describe_log_streams calls. Both below calls return an arn of the pattern arn:aws:logs:us-east-2:123456789012:log-group:test-log-group whereas looking in an actual AWS log group shows the arn pattern to differ slightly, instead being arn:aws:logs:us-east-2:123456789012:log-group:/test-log-group:* (additional forward slash after :log-group and the addition of :* at the end. When creating these in moto this is not seen. Examples:
WARNING root:tests.py:88 {'logGroups': [{'logGroupName': 'test-log-group', 'creationTime': 1733482480718, 'metricFilterCount': 0, 'arn': 'arn:aws:logs:us-east-2:123456789012:log-group:test-log-group', 'storedBytes': 0}], 'ResponseMetadata': {'RequestId': '7ptVcnqFn72o2i6w98KS7HShz7dGZHETMW6GX6m1PviCPtKA4M33', 'HTTPStatusCode': 200, 'HTTPHeaders': {'server': 'amazon.com', 'date': 'Fri, 06 Dec 2024 10:54:40 GMT', 'x-amzn-requestid': '7ptVcnqFn72o2i6w98KS7HShz7dGZHETMW6GX6m1PviCPtKA4M33'}, 'RetryAttempts': 0}}
WARNING root:tests.py:91 {'logStreams': [{'logStreamName': 'test-log-stream', 'creationTime': 1733482480719, 'arn': 'arn:aws:logs:us-east-2:123456789012:log-group:test-log-group:log-stream:test-log-stream', 'storedBytes': 0}], 'ResponseMetadata': {'RequestId': 'ByX1iiSQ0tNtxZ7iO3Nu62wDbKpvd6KMcjxqyRJMBWehsLcrQf7n', 'HTTPStatusCode': 200, 'HTTPHeaders': {'server': 'amazon.com', 'date': 'Fri, 06 Dec 2024 10:54:40 GMT', 'x-amzn-requestid': 'ByX1iiSQ0tNtxZ7iO3Nu62wDbKpvd6KMcjxqyRJMBWehsLcrQf7n'}, 'RetryAttempts': 0}}

Example code used when trying to use the logGroupIdentifier. Changing it to logGroupName as both a string and the arn does work as expected. Unfortunately the lambda code we are testing against is rather complex meaning we may not be able to easily change it:
cwlogs = boto3.client('logs') response = cwlogs.get_log_events( logGroupIdentifier=log_group_arn.rstrip(":*"), logStreamName=log_stream_name, )
Using the latest released moto version at time of writing
Cheers
Simon

@bblommers
Copy link
Collaborator

Hi @T1ckL35! The describe_log_groups call should return two ARN's:

  • arn - to be used in IAM policies. That has the :* ending
  • logStreamArn - to be used in every other CloudWatch call

See the Response Structure in the docs here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs/client/describe_log_groups.html

Moto does not yet return the logStreamArn though, so I've raised a PR that:

  • Returns the variable
  • Alters the logic of describe_log_streams/get_log_events to accept only the proper logGroupArn, and raise a validation error when using the IAM arn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants