[BUG] Spring Cloud Stream Service Bus delivers a message sometimes twice #43428
Description
Describe the bug
Last Month we had 6000 cases (out of 5 Mio), where a Message gets delivered twice with Spring Cloud Stream Service Bus.
- The message stays for the Timeout in the Queue (2 Min, thats the minor problem)
- Then we process the message and complete it
- Then the message gets delivered again. (after another 2,8 mins)
- Then Spring Cloud Stream Azure is doing a renewMessageLock which fails, becuase the message is already completed
But thats an internal exception, thus we process the message again. That the major problem.
At the End complete fails, because the message is already processed. - the error happens at arbitrary time. Not only on high Load on service Bus (Premium) and Container App.
Exception or Stack Trace
com.azure.messaging.servicebus.ServiceBusException: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue. For more information please see ...
To Reproduce
unclear
Expected behavior
A Message get delivered only one time.
Screenshots
** Logs **
Logs from Service Bus Side (From Azure Support Ticket)
Message sent to queue | 0:12:01:01.529093
First delivery (msg lock) | 0:12:01:01.529272
Abandoned | 0:12:03:01.520244
Second delivery | 0:12:03:01.530253
Completed Successfully | 0:12:03:02.480113
what we can see in the application Insights Logs (100% of all logs are saved in Insight at that time)
first processing on 2024-11-28T12:03:01.527Z (UTC)
complete on 2024-11-28T12:03:02.366Z (UTC)
second processing on 2024-11-28T12:05:48.082Z (UTC) why?
**Setup **
- OS: Linux Docker Image
- Library/Libraries:
spring-messaging-azure-4.15.0,
spring-messaging-azure-servicebus-4.15.0
spring-cloud-azure-stream-binder-servicebus-4.15.0
spring-cloud-stream-4.0.3
spring-boot-3.1.5 - Java version: 17
- App Server/Environment: Azure Container Apps
- Frameworks: Spring Boot
Information Checklist
- [ x ] Bug Description Added
- [ x ] Repro Steps Added
- [ x ] Setup information Added
Metadata
Assignees
Labels
Type
Projects
Status
In Progress