Not an issue, but a question: how to easily use same set of LAMS rules across multiple repos in Github Actions?

[wsjk]

If there is a better place for this inquiry, i'm happy to move it over there.

Our Looker instance setup has multiple separate projects that are all imported to a central project.

All the Looker projects run LAMS via Github Actions. This is hard to maintain because each project defines the LAMS rules separately in their own manifest.lkml file. So if we want to update list of LAMS rules or add a custom one, we ask each Looker project repo owner to individually make updates in their respective repos.

I was thinking maybe we could create a composite GHA for all Looker project repos to reference. And let's say the repo for this composite action contains the LAMS rule list/definitions. The question is how can I have this composite GHA run LAMS using the rules in action repo instead of the manifest.lkml file in the Looker project repo?

[fabio-looker]

Starting in v3.3 I added the ability to source manifest arguments from a few different locations, with different levels of priority. The lowest level being "manifest defaults", which can be overriden by the manifest file in a project, and which in turn can be overridden by an explicitly identified manifest file as a command line argument.

So, you should be able to copy in rules that you want by default, or to force, by simply copying in those files during the GHA's initial steps, then passing them to the appropriate command line arguments:

[wsjk]

Do you have examples of how the json object or json file would look?

[fabio-looker]

Here is an example of the JSON format: https://github.com/looker-open-source/look-at-me-sideways/blob/master/__tests__/dummy-projects/29-manifest-loading/inline-json/index.test.js#L3

Alternately, if you want to get the JSON representation for your existing lkml file, you can npm install -g lookml-parser and run lookml-parser --input=manifest.lkml --conditional-comment=LAMS --whitespace=2 --file-output=none, and look at the value of the manifest key

[wsjk]

Hmm seems like if I use manifest or manifest-default arguments in a Github Action and reference a yml in GHA repo, it doesn't seem to recognize the exemptions json file in the repo that's actually running the GHA. Do I need to pass the path of the file with exemptions as cmd line ar arg as well?

[fabio-looker]

Additional file types (JSON, YAML) (or even alternate filenames with a lkml type for that matter) are not picked up by default within a LookML project, as they have no inherent meaning there like manifest.lkml does. Alternate file types/names are only interpreted as manifest values if they are explicitly referenced by either of the two manifest command line arguments.

[fabio-looker]

Whoops, I misunderstood your question. so it's not a JSON manifest file that is not being loaded, but the exemptions... let me think about why that could be.

[wsjk]

yea i send the path of yml file with rules to check in manifest cmd line argument and it seems to work as expected. The only problem is that it's no longer picking up the list of files to be exempt from rule checks (ie lams-exemptions.ndjson)

[wsjk]

Hmm actually right now i'm just testing my GHA from teh GHA repo. I'm not actually running the GHA in the Looker project repo in final implementation. So maybe that's the problem

[fabio-looker]

Will you attach or email me your run output?

[wsjk]

Run lams --reporting=save-no --verbose --manifest ${GITHUB_ACTION_PATH}/lams-manifest.yml
Parsing project...
> Parsing done!
Getting manifest and rule info...
> Manifest and exemptions done
Checking rules... 
> K3
> K4
> E1
> E2
> E6
> E7
> T1
> T2
> W1
> K7
> F4
> ES1
> no_sql_in_dimensions
> no_sql_in_measures
> no_vw_in_filename
> no_vw_in_view_name
> Rules done!
   	Rule   	Location                                       	Description
🛈 	.......	project........................................	Override manifest settings read from YAML path)
💬 	.......	...............................................	Override manifest properties: rule
💬 	.......	...............................................	Override rules: K3, K4, E1, E2, E6, E7... +10
🛈 	.......	...............................................	Final manifest properties: rule
🛈 	.......	...............................................	Final rules: K3, K4, E1, E2, E6, E7... +10
💬 	.......	file:lams-exemptions.ndjson....................	No central LAMS exemptions found. ENOENT: no such file or directory, open '/home/runner/work/looker-actions/looker-actions/lams-exemptions.ndjson'
🛈 	E1.....	...............................................	Rule E1 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	E2.....	...............................................	Rule E2 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	E6.....	...............................................	Rule E6 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	E7.....	...............................................	Rule E7 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	ES1....	...............................................	Rule ES1 summary: 2 matches, 0 matches exempt, and 2 errors
🛈 	F4.....	...............................................	Rule F4 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	K3.....	...............................................	Rule K3 summary: 4 matches, 0 matches exempt, and 0 errors
🛈 	K4.....	...............................................	Rule K4 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	K7.....	...............................................	Rule K7 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	LAMS3..	...............................................	Rule K3 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule K4 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule E1 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule E2 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule E6 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule E7 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule T1 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule T2 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule W1 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule K7 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	LAMS3..	...............................................	Rule F4 is a built-in rule. Some rule declaration properties (e.g. match, rule_expr) have been ignored.
🛈 	no_sql_	...............................................	Rule no_sql_in_dimensions summary: 123 matches, 0 matches exempt, and 0 errors
🛈 	no_sql_	...............................................	Rule no_sql_in_measures summary: 34 matches, 0 matches exempt, and 0 errors
🛈 	no_vw_i	...............................................	Rule no_vw_in_filename summary: 4 matches, 0 matches exempt, and 0 errors
🛈 	no_vw_i	...............................................	Rule no_vw_in_view_name summary: 4 matches, 0 matches exempt, and 0 errors
🛈 	T1.....	...............................................	Rule T1 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	T2.....	...............................................	Rule T2 summary: 0 matches, 0 matches exempt, and 0 errors
🛈 	W1.....	...............................................	Rule W1 summary: 1056 matches, 0 matches exempt, and 0 errors
❌  	ES1....	ived_table.view/v:some_view	Derived table must use explore_source
❌  	ES1....	_exemption.view/v:some_view	Derived table must use explore_source
Error: Process completed with exit code 1.

[fabio-looker]

Hmm actually right now i'm just testing my GHA from teh GHA repo. I'm not actually running the GHA in the Looker project repo in final implementation. So maybe that's the problem

Yes, it will look for lams-exemptions.ndjson in the cwd directory by default. You can provide a different cwd as an argument or pass an explicit path to the exemptions with lams_rule_exemptions_path, but probably the easiest thing is to run LAMS in your project directory