Easy ACI - Comprehensive example for ACI & NDO

YAML Schema for auto-completion, Help, and Error Validation:

If you would like to utilize Autocompletion, Help Context, and Error Validation, (HIGHLY RECOMMENDED), make sure the files all utilize the .eza.yaml file extension.

And Add the Following to YAML: Schemas in Visual Studio Code: Settings > Search for YAML: Schema: Click edit in settings.json. In the yaml.schemas section:

"https://raw.githubusercontent.com/terraform-cisco-modules/easy-imm/main/yaml_schema/easy-imm.json": "*.ezi.yaml"

Soon the Schema for these YAML Files have been registered with SchemaStore via utilizing this .eza.yaml file extension. But until that is complete, need to still add to settings.

Use Cases

There are two examples in this Repository. The first example is shown in the primary folders:

• access-policies/
• admin/
• fabric-policies/
• switch/
• system_settings/
• tenants/
• virtual-networking/

IMPORTANT NOTE

Notice the eza.yaml extension on the files. This is how the data.utils_yaml_merge.model is configured to recognize the files that should be imported with the module.

The other examples are shown in the ./RICH/ Folder Structure. This is what is being used for our lab in Richfield, Ohio. In these examples there are four sub-folders.

• RICH/Asgard - The First ACI Fabric
• RICH/Wakanda - The Second ACI Fabric
• RICH/Odin - Nexus Dashboard Orchestrator managing the stretched Tenants Between the two Fabrics
• RICH/shared_settings - YAML Data that is the same between Asgard and Wakanda

The Structure of the YAML Files is very flexible. You can have all the YAML Data in a single file or you can have it in multiple individual folders like is shown in this module. The important part is that the data.utils_yaml_merge.model is configured to read the folders that you put the Data into. In the single Data Center example the data is read from all the folders in the root path described above (access-policies, admin, fabric-policies, switch, system-settings, tenants, virtual-networking). In comparison, the Asgard and Wakanda Fabrics, read files in their respective home directory plus they shared data found in the ./RICH/shared_settings folder.

Additionally because the ./RICH/Odin/ Nexus Dashboard Fabric Only supports pushing configuration with the tenants module, currently, only the built_in_tenants and tenants modules are defined. The additional function for Odin is that it is pulling in the switch_profiles, from both Asgard and Wakanda, to build EPG -> Static Path Bindings in NDO. And since NDO is being used to build the Tenant policies, there is an additional module used in Asgard and Wakanda, which is the AAEP to EPG module for mapping EPG's to the Attachable Access Entity Profiles, as this is not yet supported by Nexus Dashboard Orchestrator.

Modify global_settings.eza.yaml to match environment

global_settings.eza.yamls contains variables related to authentication to the controller and an optional global annotations for tagging objects.

Notes for the global_settings.eza.yamls

• Controller: Defines all the settings for Authentication to an APIC or Nexus Dashboard Orchestrator Controller.
• management_epgs: necessary if you are using inband EPG's, and or the ooband EPG is not default. Both true for our use case.
• annotations: helpful, but not required. This is used on Tenant Objects that support the annotations attribute. You can customize this according to anything desired to add, but by default the use case is the version of the easy_aci module is being added.

Note 1: Modules can be added or removed dependent on the use case. The primary example shown is consuming/showing a full environment deployment. But in the ./RICH/Odin/ example, it is just using the tenant modules.

Note 2: The reason for the seperation of built_in_tenants vs tenants is to make sure objects are always created in common/mgmt first. So they can be consumed by user tenants or Admin/Fabric etc (management EPGs for example). If nothing is being configured in common/mgmt/infra the built_in_tenant is not necessary.

Cloud Posse tfenv

Command line utility to transform environment variables for use with Terraform. (e.g. HOSTNAME → TF_VAR_hostname)

Recently I adopted the tfenv runner to standardize environment variables with multiple orchestration tools. tfenv makes it so you don't need to add TF_VAR_ to the variables when you add them to the environment. But it doesn't work for windows would be the caveat.

In the export examples below, for the Linux Example, the 'TF_VAR_' is excluded because Cloud Posse tfenv is used to insert it during the run.

Make sure you have already installed go

go

go install github.com/cloudposse/tfenv@latest

Add go/bin to PATH

GOPATH="$HOME/go"
PATH="$GOPATH/bin:$PATH"

Aliases for .bashrc

Additionally to Save time on typing commands I use the following aliases by editing the .bashrc for my environment.

alias tfa='tfenv terraform apply main.plan'
alias tfap='tfenv terraform apply -parallelism=1 main.plan'
alias tfd='terraform destroy'
alias tff='terraform fmt'
alias tfi='terraform init'
alias tfim='terraform import'
alias tfp='tfenv terraform plan -out=main.plan'
alias tfu='terraform init -upgrade'
alias tfv='terraform validate'

Environment Variables

Note that all the variables in variables.tf are marked as sensitive. Meaning these are variables that shouldn't be exposed due to the sensitive nature of them.

The first three variables are for authentication to either APIC or NDO controllers. The rest are for configuration parameters in the Fabric. These sensitive variables are all combined together in locals.tf into five objects:

• access_sensitive
• admin_sensitive
• fabric_sensitive
• system_sensitive
• tenant_sensitive

Using this object structure like this allows for the amount of a single variable to be flexible enough for any environment. In previous iterations of this repository many of the variables were added with five iterations to cover most use cases, but if more were needed it wasn't possible to add more. By combining the variables into the object model, you can add or remove variables as needed.

Compare the files ./locals.tf, RICH/Asgard/locals.tf, and RICH/Odin/locals.tf. Notice in the Asgard example most of the variables are taken down to a single iteration, as additional instances are not being consumed. And with the Odin example, where only the tenant module is utilized, the other iterations of sensitive variables have been removed.

The sensitive variables are still exported as individual variables as I am unsure of a good method of loading the variables into the environment in a better method, but would always appreciate feedback if there is an easy way to build an object model in an export command.

Terraform Cloud/Enterprise - Workspace Variables

At a Minimum for APIC

• Add variable apic_password with the value of <your-apic-password> and sensitive set to true

At a Minimum for Nexus Dashboard Orchestrator

• Add variable ndo_password with the value of <your-ndo-password> and sensitive set to true

Add Other Variables as discussed below based on use cases

IMPORTANT: ALL EXAMPLES BELOW ASSUME USING tfenv in LINUX

Minimum Sensitive Variables for ACI

Linux

Password Authentication

export apic_password='<your-apic-password>'

Certificate Authentication

export certificate_name='<your-certificate_name>'
export private_key='<your-private_key>'

Windows

$env:TF_VAR_apic_password='<your-apic-password>'

Minimum Sensitive Variables for Nexus Dashboard Orchestrator

Linux

export ndo_password='<your-ndo-password>'

Windows

$env:TF_VAR_ndo_password='<your-ndo-password>'

Sensitive Variables for the Access Module:

• MCP Instance Key: Key to Protect MCP traffic.
• VMM Password: vCenter Password for VMM Integration.

Linux

export mcp_instance_key='<mcp_instance_key>'

export vmm_password='<vmm_password>'

Windows

$env:TF_VAR_mcp_instance_key='<mcp_instance_key>'

$env:TF_VAR_vmm_password='<vmm_password>'

Sensitive Variables for the Admin Module:

Configuration Backup Sensitive Variables

• remote_password: For Password based Authentication.
• private_key and private_key_passphrase: for SSH Key Based Authentication.

Linux

export remote_password='<remote_password>'

export private_key='<ssh_key_contents>'
export private_key_passphrase='<ssh_key_passphrase>'

Windows

$env:TF_VAR_remote_password='<remote_password>'

$env:TF_VAR_private_key='<ssh_key_contents>'
$env:TF_VAR_private_key_passphrase='<ssh_key_passphrase>'

RADIUS Sensitive Variables

• radius_key: Key for Protecting RADIUS Server Communication.
• radius_monitoring_password: If Server Monitoring is Enabled, the Password to use with the test user account.

Linux

export radius_key='<radius_key>'

export radius_monitoring_password='<radius_monitoring_password>'

Windows

$env:TF_VAR_radius_key='<radius_key>'

$env:TF_VAR_radius_monitoring_password='<radius_monitoring_password>'

Smart Callhome Sensitive Variables

• smtp_password: Only Required if the SMTP Server Requires Authentication.

Linux

export smtp_password='<smtp_password>'

Windows

$env:TF_VAR_smtp_password='<smtp_password>'

TACACS+ Sensitive Variables

• tacacs_key: Key for Protecting TACACS Server Communication.
• tacacs_monitoring_password: If Server Monitoring is Enabled, the Password to use with the test user account.

Linux

export tacacs_key='<tacacs_key>'
export tacacs_monitoring_password='<tacacs_monitoring_password>'

Windows

$env:TF_VAR_tacacs_key='<tacacs_key>'
$env:TF_VAR_tacacs_monitoring_password='<tacacs_monitoring_password>'

Only Showing Linux Examples for the Rest for brevity

Sensitive Variables for the Fabric Module:

Note: Multiple Instances.

Note that ntp_key, snmp_authorization, snmp_community, snmp_privacy_key have multiple instances. This is only in the event you need multiple values for each variable. If only one value is needed only define one value in the export.

• NTP Authentication Keys

export ntp_key_1='<ntp_key_1>'

• SNMP Communities

export snmp_community_1='<snmp_community_1>'

• SNMP Authorization Keys and Privacy Keys for SNMP Users

export snmp_authorization_key_1='<snmp_authorization_key_1>'

export snmp_privacy_key_1='<snmp_privacy_key_1>'

Sensitive Variables for the System Settings Module:

Global AES Passphrase Encryption Settings

• aes_passphrase: Used to Encrypt Configuration Backups to protect sensitive attributes.

export aes_passphrase='<aes_passphrase>'

Sensitive Variables for the Tenants Module:

NDO Secrets for AWS or Azure Site Authetnication

export aws_secret_key='<aws_secret_key>'

export azure_client_secret='<azure_client_secret>'

Routing Protocol Authentication

• bgp_password: Only requried for BGP Neighbor Authentication
• ospf_key: Only required for Neighbor Authentication

export bgp_password_1='<bgp_password_1>'

export ospf_key_1='<ospf_key_1>'

• vrf_snmp_community: Only required if the VRF Should use different Communities than the Global SNMP Values. These Communities, that need to be added to the SNMP client_groups, will only be usable by the VRF when configured.

export vrf_snmp_community_1='<vrf_snmp_community_1>'

Execute the Terraform Plan

Terraform Cloud

When running in Terraform Cloud with VCS Integration the first Plan will need to be run from the UI but subsiqent runs should trigger automatically

Terraform CLI

• Execute the Plan - Linux

# First time execution requires initialization.  Not needed on subsequent runs.
terraform init
terraform plan -out="main.plan"
terraform apply "main.plan"

• Execute the Plan - Windows

# First time execution requires initialization.  Not needed on subsequent runs.
terraform.exe init
terraform.exe plan -out="main.plan"
terraform.exe apply "main.plan"

Requirements

Name	Version
terraform	>= 1.3.0
aci	2.13.0
mso	1.0.0
utils	0.2.5

Providers

Name	Version
utils	0.2.5

Modules

Name	Source	Version
access	terraform-cisco-modules/access/aci	3.1.10
admin	terraform-cisco-modules/admin/aci	3.1.10
built_in_tenants	terraform-cisco-modules/tenants/aci	3.1.10
fabric	terraform-cisco-modules/fabric/aci	3.1.10
switch	terraform-cisco-modules/switch/aci	3.1.10
system_settings	terraform-cisco-modules/system-settings/aci	3.1.10
tenants	terraform-cisco-modules/tenants/aci	3.1.10

NOTE:

When the Data is merged from the YAML files, it will run through the modules using for_each loop(s). Sensitive Variables cannot be added to a for_each loop, instead use the variables below to add sensitive values for policies.

Inputs

Name	Description	Type	Default	Required
apic_password	Password for User based Authentication.	string	"dummydummy"	no
apic_private_key	Cisco ACI Private Key for SSL Based Authentication.	string	"blah.txt"	no
ndo_password	Password for Nexus Dashboard Orchestrator Authentication.	string	"dummydummy"	no
mcp_instance_key	The key or password to uniquely identify the MCP packets within this fabric.	string	""	no
radius_key	RADIUS Key.	string	""	no
radius_monitoring_password	RADIUS Monitoring Password.	string	""	no
tacacs_key	TACACS Key.	string	""	no
tacacs_monitoring_password	TACACS Monitoring Password.	string	""	no
apic_ca_certificate_chain_1	Certificate Authority Certificate Chain. i.e. Intermediate and Root CA Certificate.	string	"blah.txt"	no
apic_ca_certificate_chain_2	Certificate Authority Certificate Chain. i.e. Intermediate and Root CA Certificate.	string	"blah.txt"	no
apic_certificate_1	APIC Certificate 1.	string	"blah.txt"	no
apic_certificate_2	APIC Certificate 2.	string	"blah.txt"	no
apic_private_key_1	APIC Certificate Private Key 1.	string	"blah.txt"	no
apic_private_key_2	APIC Certificate Private Key 2.	string	"blah.txt"	no
smtp_password	Password to use if Secure SMTP is enabled for the Smart CallHome Destination Group Mail Server.	string	""	no
remote_password	Remote Host Password.	string	""	no
remote_private_key	SSH Private Key File Location.	string	"blah.txt"	no
remote_private_key_passphrase	SSH Private Key Based Authentication Passphrase.	string	""	no
ntp_key_1	Key Assigned to NTP id 1.	string	""	no
ntp_key_2	Key Assigned to NTP id 2.	string	""	no
ntp_key_3	Key Assigned to NTP id 3.	string	""	no
ntp_key_4	Key Assigned to NTP id 4.	string	""	no
ntp_key_5	Key Assigned to NTP id 5.	string	""	no
snmp_authorization_key_1	SNMP Authorization Key 1.	string	""	no
snmp_authorization_key_2	SNMP Authorization Key 2.	string	""	no
snmp_authorization_key_3	SNMP Authorization Key 3.	string	""	no
snmp_authorization_key_4	SNMP Authorization Key 4.	string	""	no
snmp_authorization_key_5	SNMP Authorization Key 5.	string	""	no
snmp_community_1	SNMP Community 1.	string	""	no
snmp_community_2	SNMP Community 2.	string	""	no
snmp_community_3	SNMP Community 3.	string	""	no
snmp_community_4	SNMP Community 4.	string	""	no
snmp_community_5	SNMP Community 5.	string	""	no
snmp_privacy_key_1	SNMP Privacy Key 1.	string	""	no
snmp_privacy_key_2	SNMP Privacy Key 2.	string	""	no
snmp_privacy_key_3	SNMP Privacy Key 3.	string	""	no
snmp_privacy_key_4	SNMP Privacy Key 4.	string	""	no
snmp_privacy_key_5	SNMP Privacy Key 5.	string	""	no
aes_passphrase	Global AES Passphrase.	string	""	no
aws_secret_key	AWS Secret Key Id. It must be provided if the AWS account is not trusted. This parameter will only have effect with vendor = aws.	string	""	no
azure_client_secret	Azure Client Secret. It must be provided when azure_access_type to credentials. This parameter will only have effect with vendor = azure.	string	"1"	no
bgp_password_1	BGP Password 1.	string	""	no
bgp_password_2	BGP Password 2.	string	""	no
bgp_password_3	BGP Password 3.	string	""	no
bgp_password_4	BGP Password 4.	string	""	no
bgp_password_5	BGP Password 5.	string	""	no
ospf_key_1	OSPF Key 1.	string	""	no
ospf_key_2	OSPF Key 2.	string	""	no
ospf_key_3	OSPF Key 3.	string	""	no
ospf_key_4	OSPF Key 4.	string	""	no
ospf_key_5	OSPF Key 5.	string	""	no
vrf_snmp_community_1	SNMP Community 1.	string	""	no
vrf_snmp_community_2	SNMP Community 2.	string	""	no
vrf_snmp_community_3	SNMP Community 3.	string	""	no
vrf_snmp_community_4	SNMP Community 4.	string	""	no
vrf_snmp_community_5	SNMP Community 5.	string	""	no
vmm_password	Password for VMM Credentials Policy.	string	""	no

Outputs

Name	Description
access	Access module outputs.
admin	Admin module outputs.
built_in_tenants	Built-In Tenants module outputs (common|infra|mgmt).
fabric	Fabric module outputs.
switch	Switch module outputs.
system_settings	System Settings module outputs.
tenants	Tenants module outputs.

Sub Modules

If you want to see documentation on Variables for Submodules use the links below:

Terraform Registry

aaep-to-epgs

access-policies

admin

fabric-policies

switch

system-settings

tenants