I've got a cleanup idea in my mind, which I'd like to pass by you all to gather different opinions and experiences.
Ansible configuration files (hosts and group_vars basically) are considered our truth. To configure our components we have a mixture of ways to pass those values to them, though. The most common way is:
- Write the values into whisk.properties (arbitrary step, which is not necessary for the rest)
- Push the values into Consul's KV store
- Component comes up, gets only Consul parameters via its environment and reads all the values it needs from Consul
Some of our values are already passed by Environment vs. Consul though, for no apparent reason like WHISK_VERSION_NAME in the Controller.
In my opinion, our configuration handling is a bit out of hand at this point and I haven't found a reason why we're using Consul for configuration storage at all. It also defeats ansible's purpose of keeping track of configuration changes.
An example: Say I want to change the database of a running OpenWhisk deployment because the main one crashed. How do I do this today:
- Change the values in Consul - easy, but not in line with our single-point-of-truth anymore.
- Know each component that relies on those values - hidden knowledge.
- Restart those components to make sure they reload the configuration.
What I propose is: Configuration through Environment solely. The example would then be:
- Change the values in the Ansible configuration.
- Redeploy through the usual commands - Ansible only updates/restarts the components with changed configuration automatically (might need some extra tweaking as I think our scripts currently force a restart every time and on every component, but you get the idea)
I believe that's much simpler to handle, more in line with how Ansible etc. are designed to work and it streamlines configuration management even more. As a bonus, we lose the deploy time dependency on Consul, making the deployment simpler in general.