Leaving Swarm
My little story of why I switched to Docker Swarm. And few weeks later left it for "standard Docker".
First I had one server, which ran Docker. I used nginx-proxy image to automatically run images. That was quite easy, functioning, so I was happy.
Then, when I bought second server, I wanted to have them connected. I needed communication to be encrypted, because they are not just virtual servers on the same physical machine. I wanted just one DNS record, and servers to decide which will server which request.
Nginx
Nginx-proxy was not swarm-ready, so I couldn't use this one easily. Firstly I thought of connecting them via 2 nginx instances. But I couldn't find an easy way to do this. I was reading about creating custom CA, or using LE certificates to secure communication between the node. Then I also needed to configure nginx.conf and other stuff.
So I looked for something easier.
Traefik
Traefik looked like a good alternative. It is pretty simple, has nice features, is Swarm-ready.
I found problems with redirect to HTTPS. Also problem with proxy being unable to run if LE server is down was reported.
So once again I had to search another solution.
Docker flow proxy
Docker Flow Proxy (DFP) is a really nice image. It uses HAProxy, is highly configurable, supports plugin system (e.g. you can plug in LE support image).
The problems I found might be related to Swarm or anything else, but I discovered them after using DFP, so I suspect this image.
- I often get dangling images.
- Sometimes after server restart / docker restart some images won't run correctly. They show as running. If you
docker stack psthem, you see no problems, but visiting the URL - you'll get disappointed. - Very often I cannot
docker execto running container. - The same applies to removing images or containers. The command will hang and that's it.
- I was often unable to even successfully run
sudo service docker restart. First had tops aux|grep dockerand kill concrete processes, to be able to restart docker service.
Swarm advantages
I will not list all advantages of Swarm - they are multiple. For full list please refer to Docker docs.
Swarm / cluster acts as one machine
When it comes to IP, it does. You can have one server with IP 1.2.3.4 and another one with IP 9.8.7.6.
It's enough to run some proxy image on 1.2.3.4. If your DNS record points to 9.8.7.6, it will still work.
Nice, that was one of my goals.
Scalability
You can super simply "duplicate" containers. For example if you run 1 PHP container and it gets overloaded, you just run one docker command to add new instances. Swarm will automatically load-balance the traffic.
Swarm problems
Swarm networks
The problem with Swarm itself is, its networks (at least the configuration I needed) can't pass you the real IP address.
I suspect this of being responsible for getting into spamlist. And you can imagine whole lot of situations when you real IP is necessary - be it analytics, request limiting, session fixation, ...
Swarm quorum
Another problem can be Swarm manager quorum. I experienced the problem once. Probably forgot it since I read the documentation to Swarm.
You need more than half of managers to be available to run manager commands. My situation (with 2 servers, both being managers) is the worst one :D
If I had only one manager, I couldn't execute manager commands from the other node.
Custom images
If you have your own image and want to deploy it to Swarm, you have to use your own registry container, or build image on each node, or if you restrict container to only some nodes, you have to have this image available on them.
Data persistency
If you want to persist your data, it isn't all that easy. You can use docker config. However, if I'm not mistaken, this approach only supports 500kB of data. Even if you could use this for source codes, it's still not solution for database data or file uploads. You'd need to dump data with cron - e.g. every 15 minutes or so.
Binding to host is also possible, but you need to specify placement constraint (by which you lose one of the advantages of Swarm - redeploying containers on any working node).
You could even use SSHFS to persist data (the Docker Swarm plugin didn't work though).
Debbuging
With Swarm it's pretty harder to debug where the problem is. If you run pure docker-compose, you'll normally spot the problem instantly or quickly. With Swarm it can take quite few minutes.
Conclusion
So finally I'm moving back to non-swarm solution. I didn't need all the fancy features Swarm provides.
It was interesting to learn about all this though, and maybe I'll use this knowledge in future.