EMQX Docker Cluster Installation with Load Balancer or DNS Configuration #14435

pitambariu · 2024-12-18T05:58:02Z

pitambariu
Dec 18, 2024

Hi Team,

Could someone please help me set up an EMQX 3-node cluster across three different nodes using AWS Load Balancer and Auto Scaling?

I have attempted the configuration, but the nodes are not connecting to each other.

Thanks in advance!

Answered by id

Dec 18, 2024

I do not have time to test this right now. Try to put double quotes around the whole line like in this example in the documentation https://docs.emqx.com/en/emqx/latest/deploy/install-docker-ce.html

environment:
- "EMQX_NODE_NAME=emqx@emqx1-poc.xxx.com"
- "EMQX_CLUSTER__DISCOVERY_STRATEGY=static"
- "EMQX_CLUSTER__STATIC__SEEDS=[emqx@emqx-poc.xxx.com,emqx@emqx1-poc.xxx.com]"

View full answer

id · 2024-12-18T07:41:12Z

id
Dec 18, 2024
Maintainer

Hi @pitambariu.

You can take a look at this example with Terraform https://github.com/emqx/emqx-usage-example/tree/main/iov-s3-aws. It's a little more complex than what are you asking about, but does the job.

The trick there is to use static hostnames used in static_seeds, and have instances themselves update Route53 records when they come up. The difference with your requirements is that there we are using 1 ASG per AZ, and each ASG is static, min = max = desired = 1, so we can assign one hostname per ASG.

In your scenario you may use similar approach, but use DNS discovery and update entries in a single SRV record instead https://docs.emqx.com/en/emqx/latest/deploy/cluster/create-cluster.html#configure-autocluster-by-dns-records. Then may be run a Lambda periodically to clean up defunct records.

0 replies

pitambariu · 2024-12-18T09:42:32Z

pitambariu
Dec 18, 2024
Author

version: '3.8'

services:
emqx:
image: emqx:latest
container_name: emqx-node2
environment:
- "EMQX_NODE_NAME=emqx@emqx1-poc.xxx.com"
- node.cookie=secretcookie
- cluster.discovery_strategy=static
- "EMQX_CLUSTER_DISCOVERY_STRATEGY=static"
- "EMQX_CLUSTER_STATIC_SEEDS=emqx@emqx-poc.xxx.com,emqx@emqx1-poc.xxx.com"
- node.dist_listen_min=4370
- node.dist_listen_max=4400
- log.level=debug
- EMQX_LISTENER__TCP__EXTERNAL__ACCEPTORS=64
- EMQX_LISTENER__TCP__EXTERNAL__MAX_CLIENTS=1000
- WITH_EPMD=1
network_mode: "host"
ports:
- "1883:1883"
- "8883:8883"
- "18083:18083"
- "4369:4369"
- "4370-4400:4370-4400"
healthcheck:
test: ["CMD", "emqx_ctl", "status"]
interval: 30s
timeout: 10s
retries: 3
restart: always

Is this compose file correct

After running the above compose file, the node is unable to connect to the cluster

Please suggest any modifications to the Compose file or any other recommendations, as I am running it on an Ubuntu server

1 reply

id Dec 18, 2024
Maintainer

What do emqx1-poc.xxx.com and emqx1-poc.xxx.com resolve to from EC2 instances?

pitambariu · 2024-12-18T10:02:33Z

pitambariu
Dec 18, 2024
Author

Hi these dns resolved to respective instance IP Best Regards, Pitambar Sen *|* DevOps Engineer M : +91 6360371510 *|* E : ***@***.*** ***@***.***> *-------------------------------------------------------**-**-* *[image: Infinite Uptime] <https://www.infinite-uptime.com/>* *www.infinite-uptime.com <http://www.infinite-uptime.com/> **|** Socialize with us [image: Follow our LinkedIn Page] <https://www.linkedin.com/company/infinite-uptime/> [image: Engage with us on Facebook] <https://www.facebook.com/infiniteuptime/> [image: Connect on YouTube] <https://www.youtube.com/channel/UCceOqZfxozvbEm3SDGPS6bg>* *-------------------------------------------------------**-**-* [image: Accelerate Industrial Digital Transformation] <https://www.infinite-uptime.com/digital-reliability>

…

On Wed, Dec 18, 2024 at 3:29 PM Ivan Dyachkov ***@***.***> wrote: What do emqx1-poc.xxx.com and emqx1-poc.xxx.com resolve to from EC2 instances? — Reply to this email directly, view it on GitHub <#14435 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A5C37FXATE2KWASPOZODD4D2GFBPHAVCNFSM6AAAAABTZ5XA2OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRQGM2DQMY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

0 replies

id · 2024-12-18T10:16:57Z

id
Dec 18, 2024
Maintainer

environment:
- EMQX_NODE_NAME="emqx@emqx1-poc.xxx.com"
- EMQX_CLUSTER__DISCOVERY_STRATEGY=static
- EMQX_CLUSTER__STATIC__SEEDS="emqx@emqx-poc.xxx.com,emqx@emqx1-poc.xxx.com"
- EMQX_NODE__DIST_LISTEN_MIN=4370
- EMQX_NODE__DIST_LISTEN_MAX=4400
- EMQX_LOG__CONSOLE_HANDLER__LEVEL=debug
- EMQX_LISTENER__TCP__EXTERNAL__ACCEPTORS=64
- EMQX_LISTENER__TCP__EXTERNAL__MAX_CLIENTS=1000

3 replies

pitambariu Dec 18, 2024
Author

getting below error
root@ip-10-1-3-40:/home/ubuntu# docker logs -f emqx-node2
ERROR: Invalid node name, should be of format '^[0-9A-Za-z_-]+$'.
ERROR: Invalid node name, should be of format '^[0-9A-Za-z_-]+$'.

id Dec 18, 2024
Maintainer

I do not have time to test this right now. Try to put double quotes around the whole line like in this example in the documentation https://docs.emqx.com/en/emqx/latest/deploy/install-docker-ce.html

environment:
- "EMQX_NODE_NAME=emqx@emqx1-poc.xxx.com"
- "EMQX_CLUSTER__DISCOVERY_STRATEGY=static"
- "EMQX_CLUSTER__STATIC__SEEDS=[emqx@emqx-poc.xxx.com,emqx@emqx1-poc.xxx.com]"

Answer selected by id

pitambariu Dec 19, 2024
Author

Thanks it working

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EMQX Docker Cluster Installation with Load Balancer or DNS Configuration #14435

{{title}}

Replies: 4 comments 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

EMQX Docker Cluster Installation with Load Balancer or DNS Configuration #14435

pitambariu Dec 18, 2024

Replies: 4 comments · 4 replies

id Dec 18, 2024 Maintainer

pitambariu Dec 18, 2024 Author

id Dec 18, 2024 Maintainer

pitambariu Dec 18, 2024 Author

id Dec 18, 2024 Maintainer

pitambariu Dec 18, 2024 Author

id Dec 18, 2024 Maintainer

pitambariu Dec 19, 2024 Author

pitambariu
Dec 18, 2024

Replies: 4 comments 4 replies

id
Dec 18, 2024
Maintainer

pitambariu
Dec 18, 2024
Author

id Dec 18, 2024
Maintainer

pitambariu
Dec 18, 2024
Author

id
Dec 18, 2024
Maintainer

pitambariu Dec 18, 2024
Author

id Dec 18, 2024
Maintainer

pitambariu Dec 19, 2024
Author