Monitor queue size with durable sessions enabled and community version #14407
-
|
I am using community version 5.8 with durable sessions enabled. How do I know if messages are accumulating for a specific client? or how many messages are left for it to finish processing everything? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
|
Hello,
Durable sessions don't have the message queue, this is why this option is not available. They stream messages from a database.
Durable sessions don't accumulate messages, unlike memory sessions. When a durable session subscribes to a topic-filter, this topic-filter is marked as "durable". All new MQTT messages sent to such topics are saved into a builtin database which stores them on disk. In reality it's more complicated than that, but the bottom line is that with this design the broker will never run out of RAM due to a lagging client, as clients don't accumulate messages in RAM.
This is a valid concern. We are planning to add lag monitoring tools in the future. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @ieQu1 So durable sessions are handled as streams (like in kafka). All messages for a topic goes to the disk, they are persisted inside the builtin db and are kept there until they expire (retention period). Then all subscriber has an iterator (pointer) of the last processed message. Current situation is then than there is nothing on the API or even by accessing the filesystem of the emqx server to get information on how many messages are in each stream/topic or which are the current values for each subscriber iterator What happens if I subscribe to a topic-filter whit a new client? Will it start its iterator pointing to the oldest message in the stream or the newest one? Last question. Is there any way to connect and inspect the built-in database? |
Beta Was this translation helpful? Give feedback.
-
Right.
Yes, right now there's no such API, but we're planning to add it. Monitoring database size is an important feature. Our current focus is on optimizing throughput of durable sessions, though. Currently total disk utilization of the builtin DB can be monitored using
The iterators will point at the latest message in the stream. The new client won't receive any messages published before it subscribed to the topic, as per MQTT standard.
The builtin database provides an Erlang interface that caters to the durable session. |
Beta Was this translation helpful? Give feedback.
Right.
Yes, right now there's no such API, but we're planning to add it. Monitoring database size is an important feature. Our current focus is on optimizing throughput of dur…