feat: Allow flattening of OTLP key-value lists

[MikeGoldsmith]

Which problem is this PR solving?

When translating OTLP kv lists we currently just encode as a JSON string which isn't particularly useful. This PR extends the extraction behaviour to traverse kv lists to generate multiple event fields. The max depth is set to 5 levels.

For example, this kv attribute:

key: "data"
value: {
  "key1": "val1",
  "key2": true,
}

Previously would be stored like this:

data: "{\"key1\": \"val1\","key2":true}"

After the change, the same kv list would be stored like this:

data.key1: "val1"
data.key2: true

This change exposes the AddAttributesToMap func for consumers to use directly. This will be useful until metrics translation can be moved into this library (eg from shepherd).

• Closes Unpack and flatten array and key-value structures when feasible #235

Short description of the changes

• Update logic to extract OTLP data types to traverse kv lists instead of always encoding as JSON string, with a max depth of 5
• Expose AddAttributesToMap as a public function so consumers can use directly
• Update log translation to set the body field with a JSON string of the attribute to preserve backwards compatibility
• Removes unused & broken truncation event fields

[MikeGoldsmith]

@kentquirk @cartermp I've added an example options struct in 52c5529 to be passed into translate calls (trace and logs) that can control some of the behaviours you've pointed out as concerning (max depth, array & map flattening and body prefix).

I think this has has two big benefits:

1. we can opt into the new behaviour by utilising launch darkly flags in Shepherd or expose via refinery config, defaulting to current behaviour then flip the switch when we want to
2. paves the way to move metric translation into husky, as it requires customer level controls for some things

Note: this isn't complete, eg hasn't been added to public API calls yet, but shows the intention of the options

[kentquirk]

I like the options pattern, but instead of passing around the options as an extra parameter on all these function calls, what do you think of the idea of making these functions be methods on a type that carries the options? It's not significantly more efficient from a performance standpoint but it neatly separates the function signatures and also helps organize the code a bit.

[MikeGoldsmith]

I like the options pattern, but instead of passing around the options as an extra parameter on all these function calls, what do you think of the idea of making these functions be methods on a type that carries the options? It's not significantly more efficient from a performance standpoint but it neatly separates the function signatures and also helps organize the code a bit.

Interesting - so create a version of the translator with these options set. I'll try it out.

[TylerHelmuth]

In case it is useful, here is the flatten function we added recently for OTTL: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/ottlfuncs/func_flatten.go

[kentquirk]

Back to basics! This seems correct to me.

[TylerHelmuth]

I like this simplified version

[cartermp]

Awesome work.