[config] YAML with template

[at15]

Transferred from: dyweb/Ayi#65
Related issues: embed script language dyweb/Ayi#51, xephon-b config xephonhq/xephon-b#6, runner command config dyweb/Ayi#63

TODO

• render template using vars from previous documents
• render template using vars from current document
• render template using environment variables, test broke, mixed use of env and envs
• access data using the dot syntax like viper https://github.com/spf13/viper#getting-values-from-viper
  • Get https://github.com/spf13/viper/blob/master/viper.go#L591
  • split the access path into slice and then search the data
  • find https://github.com/spf13/viper/blob/master/viper.go#L851
  • case insensitive by looping through the map and try to find it, though I think the most efficient way would be to try the exact match first, if there is a consistent style of using lower case, then the loop is not needed
• allow disable support for var and env
• (b8df376) support for parse yaml into golang struct instead of interface{}, like what I did in xephon-k https://github.com/xephonhq/xephon-k/blob/master/pkg/config/bench.go
  • viper got a Unmarshal https://godoc.org/github.com/spf13/viper#Unmarshal
• maybe c.data["vars"] should be removed, thought about it in [config] Use built in template package instead of pongo2 #15
• allow revert a failed parse, a.k.a keep a deep copy (snapshot?) of previously succeeded parse result
• use struct tag for validation [config] Support validate config struct using tag #19

Get methods

• Get ddd0545
• GetOrFail ddd0545
• GetOrDefault ddd0545
• GetString
• GetInt
• GetBool
• GetStringSlice
• GetSlice ?
• GetMap?
• special time string, now, now+1d, might be something from Reika?
• environment override, like influxdb did, a.b.c can be override by A_B_C https://github.com/influxdata/influxdb/blob/master/cmd/influxd/run/config.go#L192 applyEnvOverrides

Set method

• Set
• SetDefault
  • fall back to default data when parsed data does not have the wanted key
  • SetDefaultVar?

Load file

• load from multiple locations and merge in order (just shallow merge, may support deep merge in the future?)
• keep the path of loaded config files for debugging
• allow strict and loose mode, if one file failed to parse, either we revert what's in it and just use what we got or we panic
• traverse all the way back to /? like npm ?? (I saw some program traverse parent folders to find the nearest match, just could find one for now, maybe git?)

Legacy

TODO

• specification
  • merge the example from runner
• summarize the related libraries, the comments in these related issues all have some useful/inspiration libraries
• write the test cast based on spec
• implementation (need to divide into several stages)

Won't fix

we no longer use pongo2, so they are all fixed ....

• extra blank line shows up Eat my newline flosch/pongo2#94, there is a PR Fix for issue #94 "Eat my newline" flosch/pongo2#165 for that
• pongo2 render false to False, but Yaml spec support a lot of values http://yaml.org/type/bool.html

[at15]

Found viper complains about go-yaml as well

// mergeMaps merges two maps. The itgt parameter is for handling go-yaml's
// insistence on parsing nested structures as map[interface{}]interface{}
// instead of using a string as the key for nest structures beyond one level
// deep. Both map types are supported as there is a go-yaml fork that uses
// map[string]interface{} instead.

I think it would be a good idea to write a new yaml parser for golang now

• support multiple documents
• stream? encode, decode
• keep the order of objects as the appears in file (a.k.a have an ordered map)
• decode to struct, like JSON unmarshal

[at15]

now I am considering using https://golang.org/pkg/text/template instead of pong2 because we don't need that much feature, though the golang syntax is a little bit strange ....

[at15]

so current version is using go's template, and has function to get environment variables, it is rendered twice to make use of variables declared in current file, it supports multiple yaml in same document (by split before unmarshal), also we no longer have so many getter and setter, we chose to unmarshal it into a go struct, which provides usage and type safety.

We may change to a DSL, like https://github.com/dhall-lang/dhall-lang, but that would be another story, sorry didn't have a release for config (patpat config) .... there is no semver in all projects where I am the only active developer and user ....