Up until now, we were sending reconfigure and remove requests to our proxy. That greatly simplified the configuration. Instead of changing HAProxy config ourselves, we let the service reconfigure itself. We used Consul to persist the state of the proxy. Can we improve the existing design by leveraging Docker Remote API? I think we can.
Instead of sending reconfigure and remove requests, we can have a service that would monitor the cluster state through the API. Such a tool could detect new and removed services and send the same request to the proxy like the one we would send manually.
We can go even further. Since the API allows us to retrieve any information related to the cluster, we don't need to store it in Consul anymore. Whenever a new instance of the service is created, it can retrieve all the information it needs from the API.
All in all, we can use the API to fully automate changes to the proxy configuration as well as its state...