Skip to content

[🐛 BUG]: Redialer does not resume pipeline after delivery channel was closed by RabbitMQ #2091

Open
@algirdasci

Description

@algirdasci

No duplicates 🥲.

  • I have searched for a similar issue in our bug tracker and didn't find any solutions.

What happened?

When RabbitMQ closes delivery channel without connection interruption, Redialer does not restore channel and stops processing messages.

Discussion: here

Version (rr --version)

2024.2.1

How to reproduce the issue?

  1. Start RoadRunner
  2. Stop or pause RabbitMQ container
  3. Wait until timeout is reached
  4. Start or resume RabbitMQ container

RR just says pipeline will be recreated, but it is not recreated and messages are stopped to process.

Relevant log output

2024-10-10T14:03:14+0000        DEBUG   rpc             plugin was started      {"address": "tcp://127.0.0.1:6001", "list of the plugins with RPC methods:": ["lock", "kv", "metrics", "resetter", "informer", "app", "jobs"]}
2024-10-10T14:03:14+0000        DEBUG   amqp            pipeline was started    {"driver": "amqp", "pipeline": "rpc", "start": "2024-10-10T14:03:14+0000", "elapsed": 4}
2024-10-10T14:03:15+0000        DEBUG   server          worker is allocated     {"pid": 84205, "max_execs": 0, "internal_event_name": "EventWorkerConstruct"}
2024-10-10T14:03:44+0000        DEBUG   amqp            delivery channel was closed, leaving the AMQP listener
2024-10-10T14:03:44+0000        DEBUG   amqp            exited from redialer
2024-10-10T14:03:44+0000        DEBUG   amqp            number of listeners     {"listeners": 0}
2024-10-10T14:03:44+0000        ERROR   amqp            amqp connection closed  {"error": "Exception (504) Reason: \"channel/connection is not open\""}
2024-10-10T14:03:44+0000        ERROR   amqp            pipeline connection was closed, redialing       {"error": "Exception (501) Reason: \"read tcp 172.23.0.2:57384->192.168.65.254:5672: i/o timeout\"", "pipeline": "rpc", "driver": "amqp", "start": "2024-10-10T14:03:44+0000"}
2024-10-10T14:04:45+0000        ERROR   amqp            backoff operation failed, pipeline will be recreated    {"error": "amqp_driver_redial: Exception (501) Reason: \"read tcp 172.23.0.2:44474->192.168.65.254:5672: i/o timeout\""}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

B-bugBug: bug, exception

Type

Projects

  • Status

    🔖 Ready

Relationships

None yet

Development

No branches or pull requests

Issue actions

    [🐛 BUG]: Redialer does not resume pipeline after delivery channel was closed by RabbitMQ · Issue #2091 · roadrunner-server/roadrunner