ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Schoreels <jonathan.schore...@gmail.com>
Subject Re: Suggestion about Docker Swarm Services and Scaling Ignite automatically
Date Tue, 08 May 2018 09:05:59 GMT
Hi Denis,

It seems like one workaround would be to configure the URL to one of the
manager node, and then the Docker API allows to loop over all the nodes net
interfaces :
https://github.com/bitsofinfo/docker-discovery-swarm-service#status. The
problem is it needs to know which node is a manager and which is a worker.

However, that's still a limitation of docker. After some searching on
"Docker Swarm Peer Discovery", I've found another implementation of the
similar problem, but for rabbitmq, which also relies on a third party
service discovery : Consul. 
Here : https://github.com/rabbitmq/rabbitmq-peer-discovery-consul
One integration with ignite can be found here :
https://github.com/andrea-zanetti/ignite-consul/blob/master/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/consul/TcpDiscoveryConsulIpFinder.java

One other interesting possibility would be to use a dns lookup based on "dig
<servicename> short" response. This implementation has been made for example
here for zookeeper :
https://github.com/itsaur/zookeeper-docker-swarm/blob/master/docker-swarm-entrypoint.sh. 
Indeed, Docker Swarm updates a DNS with all the containers loadbalanced
behind. We could cycle through those information and get all the nodes IP.

I've personnally tested it with scaling, and the new IP addresses are
dynamically added :

bash-4.4# dig tasks.zookeeper1 +short
10.0.0.12

docker service scale stack_zookeeper1=2
stack_zookeeper1 scaled to 2
overall progress: 2 out of 2 tasks 
1/2: running   [==================================================>] 
2/2: running   [==================================================>] 
verify: Service converged 

bash-4.4# dig tasks.zookeeper1 +short
10.0.0.12
10.0.0.17

Any thoughts ? Isn't it a bit too low level ?

Jonathan



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Mime
View raw message