From 882a0c27379186d0de37f93ef05f78c3276e0d62 Mon Sep 17 00:00:00 2001
From: Brewster Malevich <bmalevich@rhg.com>
Date: Tue, 29 Mar 2022 13:53:06 -0700
Subject: [PATCH] Add READMEs to infra and workflow dirs

---
 infrastructure/README.md       |  7 +++++++
 workflows/README.md            |  7 +++++++
 workflows/parameters/README.md |  5 +++++
 workflows/templates/README.md  | 24 ++++++++++++++++++++++++
 4 files changed, 43 insertions(+)
 create mode 100644 infrastructure/README.md
 create mode 100644 workflows/README.md
 create mode 100644 workflows/parameters/README.md
 create mode 100644 workflows/templates/README.md

diff --git a/infrastructure/README.md b/infrastructure/README.md
new file mode 100644
index 00000000..43dd2387
--- /dev/null
+++ b/infrastructure/README.md
@@ -0,0 +1,7 @@
+# infrastructure
+
+Cloud infrastructure, configurations.
+
+- `terraform/` contains Terraform environments to provision cloud resources, including Kubernetes clusters.
+
+- `kubernetes-gcp/` contains manifests and resources deployed onto the Google Kubernetes Engine (GKE) clusters after they've been provisioned with Terraform. After initial setup, these are automatically synced by ArgoCD within the clusters. 
diff --git a/workflows/README.md b/workflows/README.md
new file mode 100644
index 00000000..19139f43
--- /dev/null
+++ b/workflows/README.md
@@ -0,0 +1,7 @@
+# workflows
+
+Resources to orchestrate the larger biascorrection/downscaling workflow with Argo Workflows.
+
+- `workflows/parameters/` contains parameters for individual bias-correction/downscaling jobs.
+- `workflows/templates/` contains `WorkflowTemplate` manifests. These are resuseable workflows that are automatically pulled onto the cluster.
+- `*.yaml` Single use `Workflow` manifests for one-off jobs and prototypes.
diff --git a/workflows/parameters/README.md b/workflows/parameters/README.md
new file mode 100644
index 00000000..24cb6f42
--- /dev/null
+++ b/workflows/parameters/README.md
@@ -0,0 +1,5 @@
+# parameters
+
+Parameters files describing input-data specifics for various CMIP6 GCMs and variables.
+
+These files are fed to workflows described in `../templates`.
diff --git a/workflows/templates/README.md b/workflows/templates/README.md
new file mode 100644
index 00000000..c7f0f1fb
--- /dev/null
+++ b/workflows/templates/README.md
@@ -0,0 +1,24 @@
+# templates
+
+Various `WorkflowTemplate` specifying reuseable jobs or job steps for orchestrating biascorrection/downscaling work. Resources listed in `kustomization.yaml` are automatically pulled onto the cluster.
+
+The most important "top-level" work is in the `e2e-*-jobs.yaml` files which are fed parameter files from `../parameters/`. These download, clean, and biascorrect/downscale CMIP6 data. Manually launched from the command line this might look like:
+
+```shell
+argo submit --from workflowtemplate/e2e-tasmax-jobs.yaml \
+  -f "https://raw.githubusercontent.com/ClimateImpactLab/downscaleCMIP6/master/workflows/parameters/ACCESS-CM2-tasmax.yaml"
+```
+
+We might also set labels for the work like so:
+```shell
+argo submit --from workflowtemplate/e2e-tasmax-jobs.yaml \
+  -f "https://raw.githubusercontent.com/ClimateImpactLab/downscaleCMIP6/master/workflows/parameters/ACCESS-CM2-tasmax.yaml" \
+  --labels "source_id=Access-CM2,variable_id=tasmax,env=prod"
+```
+
+The majority of the other `WorkflowTemplate` files specify single-tasks used as steps in larger workflows. Each file is generally 200 - 500 lines long with the hope of being simpler and easier to digest.
+
+A few other files to note:
+
+- `clean-era5.yaml` Needs to be run on its own before the larger workflow is run. This performs additional preprocessing to ERA5, in addition to the processing described in `notebooks/`. This is needed so the ERA5 can be used as a reference dataset.
+- `create-pr-references.yaml`  Also needs to be run before the larger workflow is run for precipitation. It reads in the cleaned ERA5 data to create precipitation reference dataset. This allows us to carefully handle wet-day adjustments and run the larger precipitation workflow without repeating reference-creation steps. 
\ No newline at end of file