From 963ae6b502185ff89affa7b0c671cbeaebdad37c Mon Sep 17 00:00:00 2001 From: Matthew Hanson Date: Wed, 23 Oct 2019 01:46:06 -0400 Subject: [PATCH 1/5] update versions everywhere for a final 1.2.0 release --- .circleci/config.yml | 8 ++++---- CHANGELOG.md | 3 ++- README.md | 16 ++++++++-------- VERSION | 2 +- 4 files changed, 15 insertions(+), 14 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index f6ede8c..1a87943 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -53,18 +53,18 @@ jobs: for region in us-east-1 us-west-2 eu-central-1 do LVERSION="$(aws lambda publish-layer-version --region ${region} \ - --layer-name geolambda-dev --license-info 'MIT' \ + --layer-name geolambda --license-info 'MIT' \ --description 'Native geospatial libaries for all runtimes' \ --zip-file fileb://lambda-deploy.zip | jq '.Version')" aws lambda add-layer-version-permission --region ${region} \ - --layer-name geolambda-dev --action lambda:GetLayerVersion \ + --layer-name geolambda --action lambda:GetLayerVersion \ --statement-id public --version-number ${LVERSION} --principal '*' LVERSION="$(aws lambda publish-layer-version --region ${region} \ - --layer-name geolambda-python-dev --license-info 'MIT' \ + --layer-name geolambda-python --license-info 'MIT' \ --description 'Geospatial Python libraries' \ --zip-file fileb://lambda-deploy.zip | jq '.Version')" aws lambda add-layer-version-permission --region ${region} \ - --layer-name geolambda-python-dev --action lambda:GetLayerVersion \ + --layer-name geolambda-python --action lambda:GetLayerVersion \ --statement-id public --version-number ${LVERSION} --principal '*' done # deploy Python Lambda layers diff --git a/CHANGELOG.md b/CHANGELOG.md index dfc1731..03bc141 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,10 +6,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ## [Unreleased] -## [v1.2.0] - 2019-10-21 +## [v1.2.0] - 2019-10-24 ### Added - A Python GeoLambda Layer is now published, along with the base Lambda Layer. The base layer *must* be included in any Lambda that uses the Python GeoLambda layer. It includes the Python libraries GDAL, rasterio, pyproj, and shapely. +- Base geolambda includes OpenSSL 1.0.2, as this is required for compiling Python 3.7+. It is not included in the Lambda layer, just the base layer for ease of creating different versions of Python child images. ### Changed - The python diectory, and the new Lambda layer, now uses Python 3.7.4 diff --git a/README.md b/README.md index 1f9a892..35923d9 100644 --- a/README.md +++ b/README.md @@ -40,21 +40,21 @@ The GeoLambda image does not have an entrypoint defined, so a command must be pr If you just wish to use the publicly available Lambda layer you will need the ARN for the layer in same region as your Lambda function. Currently, GeoLambda layers are available in `us-east-1`, `us-west-2`, and `eu-central-1`. If you want to use it in another region please file an issue or you can also create your own layer using this repository (see instructions below on 'Create a new version'). -#### v1.2.0rc2 +#### v1.2.0 | Region | ARN | | ------ | --- | -| us-east-1 | arn:aws:lambda:us-east-1:552188055668:layer:geolambda-dev:6 | -| us-west-2 | arn:aws:lambda:us-west-2:552188055668:layer:geolambda-dev:5 | -| eu-central-1 | arn:aws:lambda:eu-central-1:552188055668:layer:geolambda-dev:5 | +| us-east-1 | arn:aws:lambda:us-east-1:552188055668:layer:geolambda:2 | +| us-west-2 | arn:aws:lambda:us-west-2:552188055668:layer:geolambda:2 | +| eu-central-1 | arn:aws:lambda:eu-central-1:552188055668:layer:geolambda:2 | -#### v1.2.0rc2-python +#### v1.2.0-python | Region | ARN | | ------ | --- | -| us-east-1 | arn:aws:lambda:us-east-1:552188055668:layer:geolambda-python-dev:1 | -| us-west-2 | arn:aws:lambda:us-west-2:552188055668:layer:geolambda-python-dev:1 | -| eu-central-1 | arn:aws:lambda:eu-central-1:552188055668:layer:geolambda-python-dev:1 | +| us-east-1 | arn:aws:lambda:us-east-1:552188055668:layer:geolambda-python:1 | +| us-west-2 | arn:aws:lambda:us-west-2:552188055668:layer:geolambda-python:1 | +| eu-central-1 | arn:aws:lambda:eu-central-1:552188055668:layer:geolambda-python:1 | #### v1.1.0 diff --git a/VERSION b/VERSION index fb6c5c5..26aaba0 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.2.0rc2 +1.2.0 From d6ebc41ec354771a25c57b1c2cad2cefa2992ddc Mon Sep 17 00:00:00 2001 From: Matthew Hanson Date: Wed, 23 Oct 2019 01:52:38 -0400 Subject: [PATCH 2/5] updated README with additional info on Python layer --- README.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 35923d9..ba6a1b3 100644 --- a/README.md +++ b/README.md @@ -10,11 +10,11 @@ This repository also contains examples and pre-built images for specific runtime While GeoLambda was initially intended for AWS Lambda they are also useful as base geospatial Docker images. The version of GDAL included for different versions of GeoLambda is shown here: -| geolambda | GDAL | -| --------- | ---- | -| 1.0.0 | 2.3.1 | -| 1.1.0 | 2.4.1 | -| 1.2.0 | 2.4.2 | +| geolambda | GDAL | Notes | +| --------- | ---- | | +| 1.0.0 | 2.3.1 | | +| 1.1.0 | 2.4.1 | | +| 1.2.0 | 2.4.2 | Separate Python (3.7.4) image and Lambda Layer added | To get the version numbers of all installed packages see the [CHANGELOG](CHANGELOG.md) or the Dockerfile for the specific version. @@ -50,6 +50,8 @@ If you just wish to use the publicly available Lambda layer you will need the AR #### v1.2.0-python +See the [GeoLambda Python README](python/README.md). The Python Lambda Layer includes the libraries `numpy`, `rasterio`, `GDAL`, `pyproj`, and `shapely`. + | Region | ARN | | ------ | --- | | us-east-1 | arn:aws:lambda:us-east-1:552188055668:layer:geolambda-python:1 | From 5dacd3846c96263912f9e7845ef164e005fc7ead Mon Sep 17 00:00:00 2001 From: Matthew Hanson Date: Wed, 23 Oct 2019 02:01:39 -0400 Subject: [PATCH 3/5] misc README updates --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index ba6a1b3..b6bf4ef 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,13 @@ The GeoLambda project provides public Docker images and AWS Lambda Layers containing common geospatial native libraries. GeoLambda contains the libraries PROJ.5, GEOS, GeoTIFF, HDF4/5, SZIP, NetCDF, OpenJPEG, WEBP, ZSTD, and GDAL. For some applications you may wish to minimize the size of the libraries by excluding unused libraries, or you may wish to add other libraries. In this case this repository can be used as a template to create your own Docker image or Lambda Layer following the instructions in this README. -This repository also contains examples and pre-built images for specific runtimes. +This repository also contains additional images and layers for specific runtimes. Using them as a Layer assumes the use of the basee GeoLambda layer. - [Python](python/README.md) ## Usage -While GeoLambda was initially intended for AWS Lambda they are also useful as base geospatial Docker images. The version of GDAL included for different versions of GeoLambda is shown here: +While GeoLambda was initially intended for AWS Lambda it is also useful as a base geospatial Docker image. For detailed info on what is included in each image, see the Dockerfile for that version or the [CHANGELOG](CHANGELOG.md). A version summary is provided here: | geolambda | GDAL | Notes | | --------- | ---- | | @@ -16,8 +16,6 @@ While GeoLambda was initially intended for AWS Lambda they are also useful as ba | 1.1.0 | 2.4.1 | | | 1.2.0 | 2.4.2 | Separate Python (3.7.4) image and Lambda Layer added | -To get the version numbers of all installed packages see the [CHANGELOG](CHANGELOG.md) or the Dockerfile for the specific version. - ### Docker images The Docker images used to create the Lambda layer are also published to Docker Hub, and thus are also suitable for general use as a base image for geospatial applications. @@ -29,16 +27,18 @@ The developmentseed/geolambda image in Docker Hub is tagged by version. Or just include it in your own Dockerfile as the base image. ``` -FROM developmentseed/geolambda:${VERSION} +FROM developmentseed/geolambda: ``` The GeoLambda image does not have an entrypoint defined, so a command must be provided when you run it. This example will mount the current directory to /work and run the container interactively. $ docker run --rm -v $PWD:/home/geolambda -it developmentseed/geolambda:latest /bin/bash +All of the GDAL CLI tools are installed so could be run on images in the current directory. + ### Lambda Layer -If you just wish to use the publicly available Lambda layer you will need the ARN for the layer in same region as your Lambda function. Currently, GeoLambda layers are available in `us-east-1`, `us-west-2`, and `eu-central-1`. If you want to use it in another region please file an issue or you can also create your own layer using this repository (see instructions below on 'Create a new version'). +If you just wish to use the publicly available Lambda layer you will need the ARN for the layer in the same region as your Lambda function. Currently, GeoLambda layers are available in `us-east-1`, `us-west-2`, and `eu-central-1`. If you want to use it in another region please file an issue or you can also create your own layer using this repository (see instructions below on 'Create a new version'). #### v1.2.0 @@ -69,15 +69,15 @@ See the [GeoLambda Python README](python/README.md). The Python Lambda Layer inc ## Development -Contributions to the geolambda project are encouraged. The goal is to provide a turnkey method for developing and deploying geospatial Python based projects to Amazon Web Services. The 'master' branch in this repository contains the current state as deployed to the `developmentseed/geolambda:latest` image on Dockerhub, along with a tag of the version. The 'develop' branch is the development version and is not deployed to Dockerhub. +Contributions to the geolambda project are encouraged. The goal is to provide a turnkey method for developing and deploying geospatial applications to AWS. The 'master' branch in this repository contains the current state as deployed to the Docker Hub images `developmentseed/geolambda:latest` and `devlopmentseed/geolambda-python:latest`, along with a tag of the version. The 'develop' branch is the development version and is not deployed to Docker Hub. -When making a merge to the `master` branch be sure to increment the `VERSION` in the CircleCI config.yml file. Circle will push the new version as a tag to GitHub and build and push the image to Docker Hub. If a GitHub tag already exists with that version the process will fail. +When making a merge to the `master` branch be sure to increment the `VERSION` file. Circle will push the new version as a tag to GitHub and build and push the image to Docker Hub. If a GitHub tag already exists with that version the process will fail. ### Create a new version Use the Dockerfile here as a template for your own version of GeoLambda. Simply edit it to remove or add additional libraries, then build and tag with your own name. The steps below are used to create a new official version of GeoLambda, replace `developmentseed/geolambda` with your own name. -To create a new version of GeoLambda follow these steps. +To create a new version of GeoLambda follow these steps. Note that this is the manual process of what is currently done in CircleCI, so it is not necessary to perform them but they are useful as an example for deploying your own versions. 1. update the version in the `VERSION` file From 83e3c8f81b65d65042ba3c1221043ea667b23789 Mon Sep 17 00:00:00 2001 From: Matthew Hanson Date: Wed, 23 Oct 2019 13:33:14 -0400 Subject: [PATCH 4/5] fix table in README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b6bf4ef..202d1b7 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ This repository also contains additional images and layers for specific runtimes While GeoLambda was initially intended for AWS Lambda it is also useful as a base geospatial Docker image. For detailed info on what is included in each image, see the Dockerfile for that version or the [CHANGELOG](CHANGELOG.md). A version summary is provided here: | geolambda | GDAL | Notes | -| --------- | ---- | | +| --------- | ---- | ----- | | 1.0.0 | 2.3.1 | | | 1.1.0 | 2.4.1 | | | 1.2.0 | 2.4.2 | Separate Python (3.7.4) image and Lambda Layer added | From 7c6ff2595f2eaaffec06c62a19df5487017484db Mon Sep 17 00:00:00 2001 From: Matthew Hanson Date: Wed, 23 Oct 2019 15:33:22 -0400 Subject: [PATCH 5/5] ci: fix deployment of python layer --- .circleci/config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 1a87943..00a0f11 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -62,7 +62,7 @@ jobs: LVERSION="$(aws lambda publish-layer-version --region ${region} \ --layer-name geolambda-python --license-info 'MIT' \ --description 'Geospatial Python libraries' \ - --zip-file fileb://lambda-deploy.zip | jq '.Version')" + --zip-file fileb://python/lambda-deploy.zip | jq '.Version')" aws lambda add-layer-version-permission --region ${region} \ --layer-name geolambda-python --action lambda:GetLayerVersion \ --statement-id public --version-number ${LVERSION} --principal '*'