Update to Chef Server 13.2.0 and allow in-place upgrading of ElasticSearch

README.md

@@ -37,8 +37,8 @@ You can launch this stack with the push of a button:
 
 | Edition | Version | Est Monthly cost | Launch button |
 | ------- | ------- | ---------------- | ------------- |
-| High Performance | 5.1.1 | [$1500*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-E9251374-D80C-45B7-BED7-C9C9778B1D6C) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.1.1%2Fmain.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
-| Lowest Cost | 5.1.1 | [$320*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=files/calc-165638db8e331664846c0d6654d743377bd3eac6&v=ver20190604sQ) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.1.1%2Fmarketplace.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
+| High Performance | 5.2.0 | [$1500*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-E9251374-D80C-45B7-BED7-C9C9778B1D6C) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.2.0%2Fmain.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
+| Lowest Cost | 5.2.0 | [$320*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=files/calc-165638db8e331664846c0d6654d743377bd3eac6&v=ver20190604sQ) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.2.0%2Fmarketplace.yaml&amp;stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
 
 _*NOTE: Estimated costs are just that, your mileage will vary. Use reserved instances to greatly reduce costs for long-running clusters._
 
@@ -48,7 +48,7 @@ However, the most repeatable and least error-prone way to launch this stack is t
 ```bash
 MYBUCKET=aws-native-chef-server
 MYID=mycompany
-VERSION=5.1.1
+VERSION=5.2.0
 EDITION=main  # main or marketplace
 # Configure the automate_stack_parameters.json and then launch the cloudformation stack:
 # If you're using your own bucket, uncomment the next line:
@@ -69,7 +69,7 @@ If you've made changes to the template content or parameters and you wish to upd
 ```bash
 MYBUCKET=aws-native-chef-server
 MYID=mycompany
-VERSION=5.1.1
+VERSION=5.2.0
 EDITION=main  # main or marketplace
 # If you're using your own bucket, uncomment the next line:
 # aws s3 sync . s3://$MYBUCKET/$VERSION/ --exclude "*" --include "*.yaml" --include "files/*" && \
@@ -94,9 +94,9 @@ otherwise just SSH directly to the public IPs of the chef servers
 
 ## Upgrading the Chef Server
 
-If a new Chef Server or Manage package comes out, the process for upgrading is simple and requires no downtime:
+If a new version of the template and associated AMIs comes out, the process for upgrading is simple and requires no downtime:
 
-1. Using Cloudformation's `update-stack` functionality, update the `ChefServerPackage` and `ChefManagePackage` parameters to the new URLs.
+1. Using Cloudformation's `update-stack` functionality, update to a new version of the template.
   - Confirm in the ChangeSet that this will only `ServerLaunchConfig` resource and no others!
 2. Wait for the update-stack to run, it may take a few minutes for the new metadata to be available
 3. Terminate the bootstrap frontend instance (aka `mystack-chef-bootstrap-frontend`). AutoScale will launch a new one within a few seconds that will pick up the new package versions and upgrade.
@@ -106,6 +106,19 @@ If a new Chef Server or Manage package comes out, the process for upgrading is s
   - alternatively, temporarily increase the desired capacity to launch new instances and then decrease it back to the original level to terminate the old instances
 6. You're up to date!
 
+## Upgrading ElasticSearch
+
+As of template version 5.2.0 the included version of Chef Server is at least 13.2.0, which includes support for ElasticSearch 6. The allowed versions of ElasticSearch are now 2, 5 and 6.
+
+Also as of template version 5.2.0, in-place upgrade of ElasticSearch versions is enabled. Please refer to the AWS documentation for more information:
+
+- https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-version-migration.html
+- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-updatepolicy.html#cfn-attributes-updatepolicy-upgradeelasticsearchdomain 
+
+The process for upgrading is simple but requires some downtime for AWS to process the ElasticSearch domain upgrade. However, if you wish to upgrade to ElasticSearch 6, you *must* first upgrade Chef Server to 13.2.0. Update to template versions 5.2.0 or later, and follow the process above for updating the stack and upgrading Chef Server ensuring that the `ChefElasticSearchVersion` parameter is the same version as currently deployed.
+
+Once the stack update is complete, create a new change set with the same template and configure the `ChefElasticSearchVersion` parameter to the required version. Confirm in the ChangeSet that this will only `ElasticSearchDomain` resource and no others!
+
 # FAQ
 
 ### Is this better than using chef-backend on AWS?

automate.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Automate 2 (auto-recovering standalone) v5.1.1
+Description: AWS Native Automate 2 (auto-recovering standalone) v5.2.0
 
 Parameters:
   # Required Parameters

chef_elasticsearch.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: Chef Server Elasticsearch 5.1.1
+Description: Chef Server Elasticsearch 5.2.0
 
 Parameters:
   # Required Parameters
@@ -44,12 +44,19 @@ Parameters:
   ElasticSearchVersion:
     Description: Version of ElasticSearch to use
     Type: String
-    Default: '5.6'
+    Default: '6.8'
     AllowedValues:
       - '2.3'
       - '5.3'
       - '5.5'
       - '5.6'
+      - '6.0'
+      - '6.2'
+      - '6.3'
+      - '6.4'
+      - '6.5'
+      - '6.7'
+      - '6.8'
   ElasticSearchShardCount:
     Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
     Default: 3
@@ -87,6 +94,11 @@ Resources:
 
   ElasticsearchDomain:
     Type: AWS::Elasticsearch::Domain
+    # Enable in-place upgrade of ES domain
+    # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-updatepolicy.html#cfn-attributes-updatepolicy-upgradeelasticsearchdomain 
+    # https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-version-migration.html
+    UpdatePolicy:
+      EnableVersionUpgrade: true
     Properties:
       ElasticsearchVersion: !Ref ElasticSearchVersion
       ElasticsearchClusterConfig:

chef_rds.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: Chef Server PostgreSQL v5.1.1
+Description: Chef Server PostgreSQL v5.2.0
 
 Parameters:
   # Required Parameters

chef_server_ha.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Chef Server (auto-scaling HA) v5.1.1
+Description: AWS Native Chef Server (auto-scaling HA) v5.2.0
 
 Parameters:
   # Required Parameters
@@ -107,12 +107,19 @@ Parameters:
   ElasticSearchVersion:
     Description: Version of ElasticSearch to use
     Type: String
-    Default: '5.6'
+    Default: '6.8'
     AllowedValues:
     - '2.3'
     - '5.3'
     - '5.5'
     - '5.6'
+    - '6.0'
+    - '6.2'
+    - '6.3'
+    - '6.4'
+    - '6.5'
+    - '6.7'
+    - '6.8'
   ElasticSearchShardCount:
     Description: Number of ElasticSearch hosts to provision at launch (3 recommended)
     Default: 3
@@ -194,7 +201,7 @@ Parameters:
     Type: String
   TemplateVersion:
     Type: String
-    Default: "5.1.1"
+    Default: "5.2.0"
 
 Metadata:
   cfn-lint:

chef_server_ops_dashboard.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native OpsDashboard for Chef Server v5.1.1
+Description: AWS Native OpsDashboard for Chef Server v5.2.0
 
 Parameters:
   # Required Parameters

generate_os_mappings.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-CENTOS_RELEASE='aws-native-chef-server-5.1.*'
+CENTOS_RELEASE='aws-native-chef-server-5.2.*'
 IMAGE_OWNERID="446539779517"
 
 printf "Mappings:\n  AMI:\n"

main.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Chef Stack v5.1.1
+Description: AWS Native Chef Stack v5.2.0
 
 Parameters:
   # Required Parameters
@@ -135,13 +135,14 @@ Parameters:
       'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
       'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
   ChefElasticSearchVersion:
-    Description: Version of ElasticSearch to use. (5.6 recommended for new installs)
+    Description: Version of ElasticSearch to use. (6.8 recommended for new installs)
     Type: String
     Default: '5.6'
     AllowedValues:
     - '5.3'
     - '5.5'
     - '5.6'
+    - '6.8'
   ChefElasticSearchShardCount:
     Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
     Default: 3
@@ -210,7 +211,7 @@ Parameters:
   TemplateVersion:
     Description: The version of this template to use (do not change this unless directed by support)
     Type: String
-    Default: "5.1.1"
+    Default: "5.2.0"
   Route53HostedZone:
     Type: String
     Default: ''
@@ -301,37 +302,37 @@ Conditions:
 Mappings:
   AMI:
     eu-north-1:
-      centos: ami-0b90a61689a1959d9
+      centos: ami-0f35acb0146ca6f78
     ap-south-1:
-      centos: ami-0030dea64c277379b
+      centos: ami-0cda6bedbb52c34b4
     eu-west-3:
-      centos: ami-07e0048518fbff782
+      centos: ami-0877a081573423d44
     eu-west-2:
-      centos: ami-02b14e78bbcfed4c3
+      centos: ami-0f0de2763dfd55e97
     eu-west-1:
-      centos: ami-00cced3425b8e26c9
+      centos: ami-0b95d5a8c9438a534
     ap-northeast-2:
-      centos: ami-06e068bee310906fc
+      centos: ami-0be5ab6a008fccd60
     ap-northeast-1:
-      centos: ami-004f6c060d1c18426
+      centos: ami-0f09cfeeb1b63f613
     sa-east-1:
-      centos: ami-03764a3ef231647f8
+      centos: ami-0f6135f44f2abfd7c
     ca-central-1:
-      centos: ami-0ccf046a8117357e1
+      centos: ami-0bf1cb90ad35e16d0
     ap-southeast-1:
-      centos: ami-0115d2191f108af7c
+      centos: ami-09d1d2cf1e57e267e
     ap-southeast-2:
-      centos: ami-00069b48e01775cb4
+      centos: ami-09e21da53848db264
     eu-central-1:
-      centos: ami-0fe8f1c26a7804f6b
+      centos: ami-0710d5c1b90df9df9
     us-east-1:
-      centos: ami-010b25d7343c5f95f
+      centos: ami-0fdef64afaadabde5
     us-east-2:
-      centos: ami-090f430a3250bf016
+      centos: ami-010b15bc3f5317ce1
     us-west-1:
-      centos: ami-05b0f151b72ff950c
+      centos: ami-0ad28bb7307255f0d
     us-west-2:
-      centos: ami-072e83d774bfd95b6
+      centos: ami-0193af60bbabed8ef
 
 Resources:
   ChefRole:

marketplace.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Chef Stack for Marketplace v5.1.1
+Description: AWS Native Chef Stack for Marketplace v5.2.0
 
 Parameters:
   # Required Parameters
@@ -145,11 +145,12 @@ Parameters:
       'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
       'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
   ChefElasticSearchVersion:
-    Description: Version of ElasticSearch to use. (5.6 recommended for new installs)
+    Description: Version of ElasticSearch to use. (6.8 recommended for new installs)
     Type: String
     Default: '5.6'
     AllowedValues:
     - '5.6'
+    - '6.8'
   ChefElasticSearchShardCount:
     Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
     Default: 2
@@ -198,7 +199,7 @@ Parameters:
   TemplateVersion:
     Description: The version of this template to use (do not change this unless directed by support)
     Type: String
-    Default: "5.1.1"
+    Default: "5.2.0"
   LogsRetentionInDays:
     Description: Specifies the number of days you want to retain cloudwatch log events.
     Type: Number
@@ -279,37 +280,37 @@ Metadata:
 Mappings:
   AMI:
     eu-north-1:
-      centos: ami-0b9af355d644500ce
+      centos: ami-0f35acb0146ca6f78
     ap-south-1:
-      centos: ami-0d55d5856a64a8918
+      centos: ami-0cda6bedbb52c34b4
     eu-west-3:
-      centos: ami-0dedf3f53f2557591
+      centos: ami-0877a081573423d44
     eu-west-2:
-      centos: ami-04c86e4a44288c442
+      centos: ami-0f0de2763dfd55e97
     eu-west-1:
-      centos: ami-0b34ea9402583cc17
+      centos: ami-0b95d5a8c9438a534
     ap-northeast-2:
-      centos: ami-021ab66ea3552f872
+      centos: ami-0be5ab6a008fccd60
     ap-northeast-1:
-      centos: ami-083a80ec77b6c48f5
+      centos: ami-0f09cfeeb1b63f613
     sa-east-1:
-      centos: ami-034441c2a437922b1
+      centos: ami-0f6135f44f2abfd7c
     ca-central-1:
-      centos: ami-0fae7558a8c3d4683
+      centos: ami-0bf1cb90ad35e16d0
     ap-southeast-1:
-      centos: ami-073d575257d1d42ea
+      centos: ami-09d1d2cf1e57e267e
     ap-southeast-2:
-      centos: ami-0595dd171f0627906
+      centos: ami-09e21da53848db264
     eu-central-1:
-      centos: ami-0ad4e9dfe9704c7a5
+      centos: ami-0710d5c1b90df9df9
     us-east-1:
-      centos: ami-093b02d3970c7616d
+      centos: ami-0fdef64afaadabde5
     us-east-2:
-      centos: ami-0aa5e3a0d1fda1498
+      centos: ami-010b15bc3f5317ce1
     us-west-1:
-      centos: ami-08e6c19ae4c1a2267
+      centos: ami-0ad28bb7307255f0d
     us-west-2:
-      centos: ami-01ddf45199de44e71
+      centos: ami-0193af60bbabed8ef
 
 Resources:
   ChefRole:

supermarket.yaml

@@ -1,5 +1,5 @@
 AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Supermarket (auto-recovering standalone) v5.1.1
+Description: AWS Native Supermarket (auto-recovering standalone) v5.2.0
 
 Parameters:
   # Required Parameters