[8.17](backport #6229) Cache conditions when applying variables to config

[mergify]

What does this PR do?

Our configurations can have conditional sections. The conditions are expressed in an EQL-like syntax, and we parse the expressions lazily on evaluation. However, we don't cache the parsed expression, and parse it every time it's evaluated. This change instead only parses the condition expression once, and it's then cached in the AST node.

Why is it important?

When there are a lot of variables from dynamic providers - notably in Kubernetes, on a Node with a lot of Pods - we end up spending significant resources on re-parsing the condition expressions for every var entry.

See benchstat showing the effect of this change on the benchmark from #6180 (not that this includes #6184, as otherwise we just make a copy before applying and never actually use the cache):

goos: linux
goarch: amd64
pkg: github.com/elastic/elastic-agent/internal/pkg/agent/application/coordinator
cpu: 13th Gen Intel(R) Core(TM) i7-13700H
                                      │ bench_noclone.txt │            bench_eql.txt            │
                                      │      sec/op       │   sec/op     vs base                │
Coordinator_generateComponentModel-20         32.36m ± 4%   27.45m ± 2%  -15.17% (p=0.000 n=10)

                                      │ bench_noclone.txt │            bench_eql.txt             │
                                      │       B/op        │     B/op      vs base                │
Coordinator_generateComponentModel-20        25.38Mi ± 0%   20.51Mi ± 0%  -19.17% (p=0.000 n=10)

                                      │ bench_noclone.txt │            bench_eql.txt            │
                                      │     allocs/op     │  allocs/op   vs base                │
Coordinator_generateComponentModel-20         580.4k ± 0%   490.1k ± 0%  -15.57% (p=0.000 n=10)

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool

How to test this PR locally

Unit tests are sufficient, the benchmark in #6180 helps as well.

Related issues

• Relates Elastic agent uses too much memory per Pod in k8s #5835
• Relates Memory Consumption Issue with Elastic Agent on Kubernetes with high number of resources #5991

---

This is an automatic backport of pull request #6229 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of cab5754 has failed:

On branch mergify/bp/8.17/pr-6229
Your branch is up to date with 'origin/8.17'.

You are currently cherry-picking commit cab5754e4b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1733411331-cache-conditions-in-config.yaml
	modified:   internal/pkg/agent/transpiler/ast.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/agent/transpiler/ast_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[pkoutsovasilis]

@swiatekm gentle reminder for the conflicts 🙂

[swiatekm]

@swiatekm gentle reminder for the conflicts 🙂

This needs to wait until after the 8.17.1 release anyway. I've got a long line of PRs I'm going to be rebasing and merging once that's done.

[elastic-sonarqube]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube

[mergify]

This pull request has not been merged yet. Could you please review and merge it @swiatekm? 🙏