Three node elastic cluster using individual docker-compose files and swarm on single virtual machine

[slteksystemsllc]

Hello, I have been watching all of your videos trying to sort out the best way to start a three node elastic cluster using docker swarm running on the same virtual host and I am struggling to find anything about this topic other then you mentioning using docker swarm to deploy on a single host is a possible scenario.

I have three docker compose files that look something like this with the other two nodes not exposing ports. Any help would be appreciated, I would normally do this all in one compose but am testing before deploying a swarm across multiple VMs and have seen this work elsewhere just can't sort out why master election if failing in swarm mode.

docker-compose-elasticsearch1.txt
docker-compose-elasticsearch2.txt
docker-compose-elasticsearch3.txt

Any ideas would be super helpful. I have seen this done in a previous SIEM project but can not figure out why the cluster wont form correctly. Thanks and best regards!!

[slteksystemsllc]

okay so spent most of the day trying to sort this out and have found what I think is the error however having trouble solving it. Long story short when master election happens over swarm the three nodes can see eachother but node 1 is discovering node 2 and 3 over different subnet not the same subnet as node 2 and 3, using docker network inspect commands I can see the two nodes are using eth1 and the node 1 host is trying to do discovery on eth0.

Apparently I am not alone from reading this elastic/elasticsearch#49916 and this deviantony/docker-elk#455.

So now I am really confused lol. I realize this may not be the most helpful question but I am really hoping someone has tried to boot up three elastic nodes using swarm in the same VM before and had successful master election and network connectivity etc.

[BretFisher]

Sorry, I'm not a elasticsearch admin, so I won't be of much help.

[slteksystemsllc]

Thanks @BretFisher i actually resolved it and will post a working explanation soon. It was an issue of network discovery for the containers in swarm mode. More to follow but appreciate the response. Cheers