Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQS receive_message VisibilityTimeout attribute not returned #4272

Closed
LuchiLucs opened this issue Sep 16, 2024 · 4 comments
Closed

SQS receive_message VisibilityTimeout attribute not returned #4272

LuchiLucs opened this issue Sep 16, 2024 · 4 comments
Assignees
Labels
bug This issue is a confirmed bug. closed-for-staleness p3 This is a minor priority issue response-requested Waiting on additional information or feedback. sqs

Comments

@LuchiLucs
Copy link

LuchiLucs commented Sep 16, 2024

Describe the bug

Calling the receive_message() method of the SQS client with MessageSystemAttributeNames key and value ["All", "VisibilityTimeout"] or ["All"] seems not to return a response whose message attribute contains the intented key under its attributes.

Moreover the Python SDK documentation here has that key in the deprecated key AttributeNames and not in MessageSystemAttributeNames. The REST API is here.

Assuming each message has a custom visibility timeout already set. How should I retrieve this information?

Expected Behavior

The returned response should contain the specified attribute.

Current Behavior

The returned response seems not to contain the specified attribute.

Reproduction Steps

def receive_messages(
    self,
    queue_url: str,
    max_number: int = 1,
    wait_time: int = 0,
    visibility_timeout: int = 2 * 60 * 60,
    attribute_names: list[str] = ["All", "VisibilityTimeout"],
):
    response = self.sqs_client.receive_message(
        QueueUrl=queue_url,
        MaxNumberOfMessages=max_number,
        WaitTimeSeconds=wait_time,
        VisibilityTimeout=visibility_timeout,
        MessageSystemAttributeNames=attribute_names,
    )
    return response

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.35.2

Environment details (OS name and version, etc.)

Linux (AWS EC2)

@LuchiLucs LuchiLucs added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Sep 16, 2024
@LuchiLucs LuchiLucs changed the title SQS receive_message SQS receive_message VisibilityTimeout attribute not returned Sep 16, 2024
@adev-code adev-code self-assigned this Sep 16, 2024
@adev-code adev-code added investigating This issue is being investigated and/or work is in progress to resolve the issue. sqs p3 This is a minor priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Sep 16, 2024
@adev-code
Copy link

Hi @LuchiLucs, thanks for reaching out. Testing from my side, having "All" as a value and pulling the message from the SQS for the first time by a consumer, the output will show the values for "MessageSystemAttributeNames". When pulling the same message right after you have pulled it, it may not show the list of attributes. This is because the message that you have pulled previously have been taken out already from the SQS Queue and already been delivered to a consumer. This is also shown on the visibility timeout documentation ( https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html ). That said, please try to create a new message then try to pull the message from SQS for the first time with "All" as a value and see the results. If VisibilityTimeout is not specified, it has a 30 seconds default value. So the longer the VisibilityTimeout, the longer the message will not be retrieved again.

If you would like to get the same message again right away, you can set a low "VisibilityTimeout" (VisibilityTimeout=0) - this way you would be able to get the output for the parameters you have included. Also, please take note that "VisibilityTimeout" would not be a value for "MessageSystemAttributeNames".

I have raised an internal ticket to our SQS Documentation team for clarifying the retention of outputs and for the VisibilityTimeout to be added to the API docs.

If you have any questions, please do let me know. For other review, please provide full debug logs by adding boto3.set_stream_logger('') to your code and redacting any sensitive information.
Thank you

@adev-code adev-code added response-requested Waiting on additional information or feedback. and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Sep 18, 2024
@LuchiLucs
Copy link
Author

LuchiLucs commented Sep 22, 2024

@adev-code Thank you for your review. I'm confused whether I should be able to see the set custom value of the key VisibilityTimeout with the mentioned version of the Python SDK, or the current REST API.
As far as I understood, when I pull a message from the queue for the first time (I'm using FIFO queues) the message becomes in-flight and the queue is blocked while processing the message untill it finishes, by removing the message handle from the queue, or if timeout expires.
In my particular use-case, as in the provided example, I'm using the receive_message method to at the same time specified the VisibilityTimeout desired value, by setting this per message and not using the default queue value, and also getting the message medadata, i.e. the MessageSystemAttributeNames key which, as far as I understood, should contain the attributes, and one of these is the VisibilityTimeout . In other words, what I expect to date, it to set and get, at the same time the VisibilityTimeout with that API call. Am I missing something?

@github-actions github-actions bot removed the response-requested Waiting on additional information or feedback. label Sep 23, 2024
@adev-code
Copy link

Hi @LuchiLucs, thanks for the update. As per the documentation for receive_message (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs/client/receive_message.html#SQS.Client.receive_message), the 'VisibilityTimeout' is not part of the response. It is only used as an input to configure the VisibilityTimeout. I have submitted a feature request to the SQS Team about if this can be added to the response. Please refer to an SDK changelog (https://raw.githubusercontent.com/boto/boto3/develop/CHANGELOG.rst) for updates about this going forward. Please let me know if you have any other questions. Thanks!

@adev-code adev-code added the response-requested Waiting on additional information or feedback. label Sep 23, 2024
Copy link

github-actions bot commented Oct 4, 2024

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

@github-actions github-actions bot added closing-soon This issue will automatically close in 4 days unless further comments are made. closed-for-staleness and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Oct 4, 2024
@github-actions github-actions bot closed this as completed Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a confirmed bug. closed-for-staleness p3 This is a minor priority issue response-requested Waiting on additional information or feedback. sqs
Projects
None yet
Development

No branches or pull requests

2 participants