Allow configuration by config file

[paulgb]

We've outgrown command-line only configuration.

When it comes to configuration files there's a tricky balance; on one hand we don't want to stray too far from expected practices; on the other hand, common practices are fractured and leave a lot to be desired.

Some things to consider:

• It's common to have a fairly static configuration but want to set some values at runtime. How should this be accomplished?
  • Caddy approach: env var templating in configuration file
  • Nginx approach doesn't have built-in env var substitution, but the docker image implements it based on envsubst
  • docker registry allows overriding individual yaml leaves by turning the full path to that leaf into a variable name. This is good for overriding string values, but is awkward for list/array/set values.
• It's important to be able to automatically generate configuration from popular languages, so it should be a relatively common format.
• It's important to be able to add comments to configuration, so if we were to go with something like JSON, we'd want to use a flavor that supported comments. And trailing commas.

Some options:

• TOML: Rust projects tend to favor it.
• YAML: Popular with Go/CNCF projects, but No Body Wants To Write YAML. Superficially a superset of JSON, making it very easy to generate universally, but not without footguns.
• JSON: Good for generating but not for human-generated configuration.
• Dhall: Has some nice neat ideas but overcomplicated for our use case and still pretty obscure.
• CUE: Exposes less complexity than Dhall although it has a similarly thoughtful design. It could be a nice user experience with the right tooling, but I don't think the tooling is mature enough yet. Mostly targets the Go ecosystem currently.
• Nickel: Cool but too programming-languagey

[paulgb]

There are Rust libraries that allow us to read from multiple formats and also to allow environment overrides.

https://github.com/SergioBenitez/Figment
https://crates.io/crates/config

A minimal feature set for us probably includes:

• Ability to use toml (for human-created) and json (for machine-created)
• Reasonable errors on schema validation
• Ability to override with env or cli options
• Serde, ideally

[paulgb]

I've evaluated Figment and config-rs and written up my thoughts in this issue comment.

I find some of figment's design decisions more aligned with my expectations, but it hasn't been touched in over a year and has some out-of-date dependencies. I'm inclined to fork it and update the deps ourselves and then use it.

[paulgb]

Closed by #149