Added further utilities readme files to Wiki (#840)

* Update to latest * Update to latest * Update to latest * Update to latest * Update to latest * Update to latest
Azure · Dec 23, 2021 · cb72e20 · cb72e20
1 parent 550e0fa
commit cb72e20
Show file tree

Hide file tree

Showing 15 changed files with 280 additions and 65 deletions.
diff --git a/README.md b/README.md
@@ -115,7 +115,7 @@ This repository includes a collection of advanced and curated Modules consisting
 
 | Name | Status | Docs |
 | - | - | - |
-| [ConvertTo-ARMTemplate](https://github.com/Azure/ResourceModules/blob/main/utilities/tools/ConvertTo-ARMTemplate.ps1) | [![.Platform: Test - ConvertTo-ARMTemplate.ps1](https://github.com/Azure/ResourceModules/actions/workflows/platform.convertToArmTemplate.tests.yml/badge.svg?branch=main)](https://github.com/Azure/ResourceModules/actions/workflows/platform.convertToArmTemplate.tests.yml) | [link](https://github.com/Azure/ResourceModules/wiki/UtilitiesConversionScript) |
+| [ConvertTo-ARMTemplate](https://github.com/Azure/ResourceModules/blob/main/utilities/tools/ConvertTo-ARMTemplate.ps1) | [![.Platform: Test - ConvertTo-ARMTemplate.ps1](https://github.com/Azure/ResourceModules/actions/workflows/platform.convertToArmTemplate.tests.yml/badge.svg?branch=main)](https://github.com/Azure/ResourceModules/actions/workflows/platform.convertToArmTemplate.tests.yml) | [link](https://github.com/Azure/ResourceModules/wiki/UtilitiesConvertToARMTemplate) |
 
 ## Contributing
 

diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md
@@ -21,7 +21,10 @@ If you're unfamiliar with Infrastructure as Code, or wonder how you can use the
   - [Design](./TestingDesign)
   - [Usage](./TestingUsage)
 - [Utilities](./Utilities)
-  - [Bicep to ARM conversion script](./UtilitiesConversionScript)
+  - [Bicep to ARM conversion script](./UtilitiesConvertToARMTemplate)
+  - [Register Azure DevOps pipelines script](./UtilitiesRegisterAzureDevOpsPipeline)
+  - [Set Module ReadMe script](./UtilitiesSetModuleReadMe)
+  - [Get formatted RBAC roles script](./UtilitiesGetFormattedRBACRoleList)
 - [Pipelines](./Pipelines)
   - [Design](./PipelinesDesign)
     - [Parameter File Tokens](./ParameterFileTokens)

diff --git a/docs/wiki/ModulesDesign.md b/docs/wiki/ModulesDesign.md
@@ -76,7 +76,7 @@ Microsoft.Sql
   └─ databases [child-module/resource]
 ```
 
-In this folder we recommend to place the child-resource-template alongside a ReadMe (that can be generated via the `.github\workflows\scripts\Set-ModuleReadMe.ps1` script) and optionally further nest additional folders for it's child-resources.
+In this folder we recommend to place the child-resource-template alongside a ReadMe (that can be generated via the [Set-ModuleReadMe](./UtilitiesSetModuleReadMe) script) and optionally further nest additional folders for it's child-resources.
 
 The parent template should reference all it's direct child-templates to allow for an end-to-end deployment experience while allowing any user to also reference 'just' the child-resource itself. In the case of the SQL-server example the server template would reference the database module and encapsulate it it in a loop to allow for the deployment of n-amount of databases. For example
 
@@ -180,7 +180,7 @@ module <mainResource>_rbac '.bicep/nested_rbac.bicep' = [for (roleAssignment, in
 Here you specify the platform roles available for the main resource.
 
 The `builtInRoleNames` variable contains the list of applicable roles for the specific resource to which the nested_rbac.bicep module applies.
->**Note**: You can find a helper script `Get-FormattedRBACRoles.ps1` in the `utilities\tools` folder of the repository. You can use this script to extract a formatted list of RBAC roles used in the CARML modules based on the RBAC lists in Azure.
+>**Note**: You use the helper script [Get-FormattedRBACRoles.ps1](./UtilitiesGetFormattedRBACRoleList) to extract a formatted list of RBAC roles used in the CARML modules based on the RBAC lists in Azure.
 
 The element requires you to provide both the `principalIds` & `roleDefinitionOrIdName` to assign to the principal IDs. Also, the `resourceId` is target resource's resource ID that allows us to reference it as an `existing` resource. Note, the implementation of the `split` in the resource reference becomes longer the deeper you go in the child-resource hierarchy.
 
@@ -194,7 +194,6 @@ var builtInRoleNames = {
   'Contributor': subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'b24988ac-6180-42a0-ab88-20f7382dd24c')
   'Reader': subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'acdd72a7-3385-48ef-bd42-f606fba81ae7')
   // <optionalAdditionalRoles>
-  // You can find a helper script `Get-FormattedRBACRoles.ps1` in the `tools` folder of the repository to fetch the roles.
 
 }
 
@@ -524,7 +523,7 @@ Its primary components are in order:
 - A **Template references** section listing relevant resources [ARM template reference](https://docs.microsoft.com/en-us/azure/templates).
 
 Note the following recommendations
-- Use our module generation script `Set-ModuleReadMe` that will do most of the work for you. Currently you can find it at 'utilities\tools\Set-ModuleReadMe.ps1'. Just load the file and invoke the function like this `Set-ModuleReadMe -TemplateFilePath '<pathToModule>/deploy.bicep'`
+- Use our module ReadMe generation script [Set-ModuleReadMe](./UtilitiesSetModuleReadMe) that will do most of the work for you.
 - It is not recommended to describe how to use child resources in the parent readme file (for example 'How to define a [container] entry for the [storage account]'). Instead it is recommended to reference the child resource's ReadMe instead (for example 'container/readme.md').
 
 # Parameter files

diff --git a/docs/wiki/Utilities.md b/docs/wiki/Utilities.md
@@ -6,6 +6,9 @@ This section and its sub-sections describe tools and utilities provided in this
 
 ### _Navigation_
 
-- [Bicep to ARM conversion script](./UtilitiesConversionScript)
+- [Bicep to ARM conversion script](./UtilitiesConvertToARMTemplate)
+- [Register Azure DevOps pipelines script](./UtilitiesRegisterAzureDevOpsPipeline)
+- [Set Module ReadMe script](./UtilitiesSetModuleReadMe)
+- [Get formatted RBAC roles script](./UtilitiesGetFormattedRBACRoleList)
 
 ---
diff --git a/docs/wiki/UtilitiesConversionScript.md b/docs/wiki/UtilitiesConversionScript.md
diff --git a/docs/wiki/UtilitiesConvertToARMTemplate.md b/docs/wiki/UtilitiesConvertToARMTemplate.md
@@ -0,0 +1,34 @@
+# Bicep to ARM conversion script
+
+At the time of writing bicep is still in a beta phase. For this reason, some people may want to wait for bicep's _General Availability_ and prefer to use ARM Templates for the time being.
+For these scenarios, the CARML library provides a script that uses the Bicep Toolkit translator/compiler to support the conversion of CARML Bicep modules to ARM templates.
+This page documents the conversion utility and how to use it.
+
+---
+
+### _Navigation_
+
+- [Location](#location)
+- [How it works](#what-it-does)
+- [How to use it](#how-to-use-it)
+  - [Examples](#examples)
+
+---
+# Location
+
+You can find the script under `/utilities/tools/ConvertTo-ARMTemplate.ps1`
+
+# How it works
+
+The script finds all 'deploy.bicep' files and tries to convert them to json-based ARM templates
+by using the following steps.
+1. Remove existing deploy.json files
+1. Convert bicep files to json
+1. Remove bicep metadata from json
+1. Remove bicep files and folders
+1. Update pipeline files - Replace .bicep with .json in pipeline files
+
+# How to use it
+
+For details on how to use the function please refer to the script's local documentation.
+> **Note:** The script must be loaded before the function can be invoked
diff --git a/docs/wiki/UtilitiesGetFormattedRBACRoleList.md b/docs/wiki/UtilitiesGetFormattedRBACRoleList.md
@@ -0,0 +1,94 @@
+# Get formatted RBAC roles Script
+
+Use this script to format a given raw 'Roles' table from Azure to the format required by either bicep or ARM in any RBAC deployment.
+
+---
+
+### _Navigation_
+
+- [Location](#location)
+- [How it works](#what-it-does)
+- [How to use it](#how-to-use-it)
+  - [Examples](#examples)
+
+---
+# Location
+
+You can find the script under `/utilities/tools/Get-FormattedRBACRoles.ps1`
+
+# How it works
+
+1. From the provided raw and plain roles list, create a list of only the contained role names
+1. Fetch all available roles from Azure
+1. Go through all provided role names, match them with those from Azure to get the matching RoleDefinitionId and format a string like `'<roleName>': subscriptionResourceId('Microsoft.Authorization/roleDefinitions','<roleDefinitionId>')` for each match
+1. Print the result to the terminal
+
+# How to use it
+
+The script does not accept any custom parameter per se, but expects you to replace the placeholder in the `rawRoles` variable inside the script
+
+```PowerShell
+$rawRoles = @'
+    <paste the table here>
+'@
+```
+
+To get the list of roles in the expected format:
+1. Navigate to Azure
+1. Deploy one instance of the service you want to fetch the roles for
+1. Navigate to the `Access Control (IAM)` blade in the resource
+1. Open the `Roles` tab
+1. Set the `Type` in the dropdown to `BuiltInRole`
+
+   <img src="media/rbacRoles.png" alt="Complete deployment flow filtered" height="300">
+
+1. Select and copy the entire table as is to the PowerShell variable.
+
+   The result should look similar to
+
+   ```PowerShell
+   $rawRoles = @'
+   Owner
+   Grants full access to manage all resources, including the ability to assign roles in Azure RBAC.
+   builtInRole
+   General
+   View
+   Contributor
+   Grants full access to manage all resources, but does not allow you to assign roles in Azure RBAC, manage assignments in Azure Blueprints, or share image galleries.
+   BuiltInRole
+   General
+   View
+   Reader
+   View all resources, but does not allow you to make any changes.
+   BuiltInRole
+   General
+   View
+   '@
+   ```
+1. Execute the script. The output for the above example would be
+
+    ```yml
+    VERBOSE: Bicep
+    VERBOSE: -----
+     'Owner':       subscriptionResourceId('Microsoft.Authorization/roleDefinitions','8e3af657-a8ff-443c-a75c-2fe8c4bcb635')
+     'Contributor': subscriptionResourceId('Microsoft.Authorization/roleDefinitions','b24988ac-6180-42a0-ab88-20f7382dd24c')
+     'Reader':      subscriptionResourceId('Microsoft.Authorization/roleDefinitions','acdd72a7-3385-48ef-bd42-f606fba81ae7')
+    VERBOSE:
+    VERBOSE: ARM
+    VERBOSE: ---
+     "Owner":       "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions','8e3af657-a8ff-443c-a75c-2fe8c4bcb635')]",
+     "Contributor": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions','b24988ac-6180-42a0-ab88-20f7382dd24c')]",
+     "Reader":      "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions','acdd72a7-3385-48ef-bd42-f606fba81ae7')]",
+    ```
+1. Copy the output into the RBAC file into the `buildInRoleNames` variable. Again, for the same example using bicep this would be:
+
+   ```bicep
+   var builtInRoleNames = {
+      'Owner': subscriptionResourceId('Microsoft.Authorization/roleDefinitions', '8e3af657-a8ff-443c-a75c-2fe8c4bcb635')
+      'Contributor': subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'b24988ac-6180-42a0-ab88-20f7382dd24c')
+      'Reader': subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'acdd72a7-3385-48ef-bd42-f606fba81ae7')
+   }
+   ```
+
+For further details on how to use the function please refer to the script's local documentation.
+> **Note:** The script must be loaded before the function can be invoked
diff --git a/docs/wiki/UtilitiesRegisterAzureDevOpsPipeline.md b/docs/wiki/UtilitiesRegisterAzureDevOpsPipeline.md
@@ -0,0 +1,43 @@
+# Register Azure DevOps Pipelines
+
+Use this script to automatically register all specified Azure DevOps pipelines in a target Azure DevOps project. This is especially useful to register the initial module pipelines as there is one for each module in the repository.
+
+---
+
+### _Navigation_
+
+- [Location](#location)
+- [How it works](#what-it-does)
+- [How to use it](#how-to-use-it)
+  - [Examples](#examples)
+
+---
+# Location
+
+You can find the script under `/utilities/tools/Register-AzureDevOpsPipeline.ps1`
+
+# How it works
+
+1. Get all pipelines in a given target folder (for example `.azuredevops/modulePipelines`)
+1. Fetch all currently registered pipelines in the target Azure DevOps project
+1. Compare the local defined and remote-registered pipelines to detect which need to be created and which skipped
+1. Create all pipelines that are missing
+1. Optionally register the pipelines also for build validation (i.e. they registered to be required for Pull Requests)
+
+# How to use it
+
+> **Note:** You'll need the 'azure-devops' extension to run this function: `az extension add --upgrade -n azure-devops`
+
+The steps you'd want to follow are
+1. (if pipelines are in GitHub) Create a service connection to the target GitHub repository using e.g. oAuth
+1. Create a PAT token for the Azure DevOps environment in which you want to register the pipelines in
+1. Run this script with the corresponding input parameters
+1. Create any required element required to execute the pipelines. For example:
+   - Library group(s) used in the pipeline(s)
+   - Service connection(s) used in the pipeline(s)
+   - Agent pool(s) used in the pipeline(s) if not using the default available agents
+
+For further details on how to use the function please refer to the script's local documentation.
+> **Note:** The script must be loaded before the function can be invoked
+
+
diff --git a/docs/wiki/UtilitiesSetModuleReadMe.md b/docs/wiki/UtilitiesSetModuleReadMe.md
@@ -0,0 +1,52 @@
+# Module ReadMe Generation Script
+
+Use this script to generate most parts of a templates ReadMe file. It will take care of all aspects but the description & module-specific parameter usage examples. However, the latter are added for default cases such as `tags` or `roleAssignments` if the corresponding parameter exists in the provided template.
+
+For further information about the parameter usage blocks, please refer to the [section](#special-case-parameter-usage-section) below.
+
+---
+
+### _Navigation_
+
+- [Location](#location)
+- [How it works](#what-it-does)
+  - [Special case: 'Parameter Usage' section](#special-case-parameter-usage-section)
+- [How to use it](#how-to-use-it)
+  - [Examples](#examples)
+
+---
+# Location
+
+You can find the script under `/utilities/tools/Set-ModuleReadMe.ps1`
+
+# How it works
+
+1. Using the provided template path, the script first makes sure to convert it to ARM if necessary (i.e. if a path to a bicep file was provided)
+1. If the intended readMe file does not yet exist in the expected path, it is generated with a skeleton (with e.g. a generated header name, etc.)
+1. It then goes through all sections defined as `SectionsToRefresh` (by default all) and refreshes the section content (for example for the `Parameters`) based on the values in the ARM template. It detects sections by their header and regenerates always the full section.
+1. Once all are refreshed, the current ReadMe file is overwritten. **Note:** The script can be invoked with a `WhatIf` in combination with `Verbose` to just receive an console-output of the updated content.
+
+## Special case: 'Parameter Usage' section
+
+The 'Parameter Usage' examples are located just beneath the 'Parameters' table. They are intended to show how to use complex objects/arrays that can be provided as parameters (excluding child-resources as they have their own readMe).
+
+For the most part, this section is to be populated manually. However, for a specific set of common parameters, we automatically add their example to the readMe if the parameter exists in the template. At the time of this writing these are:
+- Private Endpoints
+- Role Assignments
+- Tags
+- User Assigned Identities
+
+To be able to change this list with minimum effort, the script reads the content from markdown files in the folder: `utilities/tools/moduleReadMeSource` and matches their title against the parameters of the template file. If a match is found, it's content is added to the readme alongside the generated header. This means, if you want to add another case, you just need to add a new file to the `moduleReadMeSource` folder and follow the naming pattern `resourceUsage-<parameteRName>.md`.
+
+For example, the content of file `resourceUsage-roleAssignments.md` in folder `moduleReadMeSource` is added to a template's readMe if it contains a parameter `roleAssignments`. The combined result is:
+
+```markdown
+### Parameter Usage: `roleAssignments`
+
+<[resourceUsage-roleAssignments.md] file content>
+```
+
+# How to use it
+
+For details on how to use the function please refer to the script's local documentation.
+> **Note:** The script must be loaded before the function can be invoked
diff --git a/docs/wiki/media/rbacRoles.png b/docs/wiki/media/rbacRoles.png
diff --git a/utilities/tools/Get-FormattedRBACRoles.ps1 → ...ities/tools/Get-FormattedRBACRoleList.ps1 b/utilities/tools/Get-FormattedRBACRoles.ps1 → ...ities/tools/Get-FormattedRBACRoleList.ps1
diff --git a/utilities/tools/Register-AzureDevOpsPipeline.ps1 b/utilities/tools/Register-AzureDevOpsPipeline.ps1
@@ -30,13 +30,16 @@ Defaults to 'Azure/ResourceModules'
 Optional. The type of source repository. Either 'GitHub' or 'tfsgit' (for Azure DevOps).
 Defaults to 'GitHub'.
 
+.PARAMETER GitHubPAT
+Optional. A personal access token for the GitHub repository with the source code.
+
 .PARAMETER GitHubServiceConnectionName
 Optional. The pre-created service connection to the GitHub source repository if the pipeline files are in GitHub.
 It is recommended to create the service connection using oAuth.
 
 .PARAMETER AzureDevOpsPAT
-Required. The access token whith appropirate permissions to create Azure Pipelines.
-Usually the System.AccessToken from an Azure Pipeline instance run has sufficent permissions as well.
+Required. The access token with appropriate permissions to create Azure Pipelines.
+Usually the System.AccessToken from an Azure Pipeline instance run has sufficient permissions as well.
 Reference: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/access-tokens?view=azure-devops&tabs=yaml#how-do-i-determine-the-job-authorization-scope-of-my-yaml-pipeline
 Needs at least the permissions:
 - Release: Read, write, execute & manage
@@ -68,6 +71,29 @@ Register-AzureDevOpsPipeline @inputObject
 
 Registers all pipelines in the default path in the DevOps project [Contoso/CICD] by leveraging the given AzureDevOpsPAT and creating a service connection to GitHub using the provided GitHubPAT
 
+.EXAMPLE
+$inputObject = @{
+    OrganizationName      = 'Contoso'
+    ProjectName           = 'CICD'
+    SourceRepository      = 'Azure/ResourceModules'
+    AzureDevOpsPAT        = '<Placeholder>'
+}
+Register-AzureDevOpsPipeline @inputObject
+
+Registers all pipelines in the default path in the DevOps project [Contoso/CICD] by leveraging the given AzureDevOpsPAT and using a pre-created service connection to GitHub
+
+.EXAMPLE
+$inputObject = @{
+    OrganizationName      = 'Contoso'
+    ProjectName           = 'CICD'
+    SourceRepositoryType  = 'tfsgit'
+    SourceRepository      = 'Azure/ResourceModules'
+    AzureDevOpsPAT        = '<Placeholder>'
+}
+Register-AzureDevOpsPipeline @inputObject
+
+Register all pipelines in a DevOps repository with default values in a the target project
+
 .NOTES
 You'll need the 'azure-devops' extension to run this function: `az extension add --upgrade -n azure-devops`