version 13-06-2024

aws-samples · Jun 13, 2024 · 99d48ca · 99d48ca
1 parent a29cee0
commit 99d48ca
Show file tree

Hide file tree

Showing 22 changed files with 1,929 additions and 229 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,19 @@
 
 All notable changes to this project will be documented in this file.
 
+## version v0.0.8
+
+### Added
+
+- New compute family available Fargate with Graviton 2: You can run your applications using the Fargate (Serverless) launch type with the ARM64 architecture.
+- Upgrade : FFmpeg 7.0.1 for `intel`, `arm`, `amd`, `nvidia`, `fargate`, `fargate-arm`
+- Improve compatibility with several AWS Regions: ap-southeast-2``ap-south-1``sa-east-1``eu-west-3``us-east-1``us-west-2``eu-central-1`
+
+### Changed
+
+- Fix stack destroy issues
+- Fix multi region deployment
+
 ## version v0.0.7
 
 ### Changed

diff --git a/README.md b/README.md
@@ -117,18 +117,19 @@ So, parameters of the solution are
 - `ìnput_url`: AWS S3 url synced to the local storage and tranformed to local path by the solution.
 - `output_file_options`: FFmpeg output file options described in the official documentation.
 - `output_url`: AWS S3 url synced from the local storage to AWS S3 storage.
-- `compute`: Instances family used to compute the media asset : `intel`, `arm`, `amd`, `nvidia`, `fargate`, `xilinx`
+- `compute`: Instances family used to compute the media asset : `intel`, `arm`, `amd`, `nvidia`, `fargate`, `fargate-arm`, `xilinx`
 - `name`: metadata of this job for observability.
 
 The solution has different FFmpeg versions per AWS EC2 instance families.
 
 | **Compute** | **FFmpeg version per default** | **FFmpeg version(s) available** |
 |-------------|--------------------------------|---------------------------------|
-| intel       | 6.0 (snapshot)                 | 6.0, 5.1                        |
-| arm         | 6.0 (snapshot)                 | 6.0, 5.1                        |
-| amd         | 6.0 (snapshot)                 | 6.0, 5.1                        |
-| nvidia      | 6.0 (snapshot)                 | 6.0, 5.1                        |
-| fargate     | 6.0 (snapshot)                 | 6.0, 5.1                        |
+| intel       | 7.0.1                          | 6.0, 5.1                        |
+| arm         | 7.0.1                          | 6.0, 5.1                        |
+| amd         | 7.0.1                          | 6.0, 5.1                        |
+| nvidia      | 7.0 (snapshot)                 | 6.0, 5.1                        |
+| fargate     | 7.0.1                          | 6.0, 5.1                        |
+| fargate-arm | 7.0.1                          | 6.0, 5.1                        |
 | xilinx      | 4.4                            | 4.4                             |
 
 In this example we use the AWS SDK "Boto3" (Python) and I want to cut a specific part of a video. First of all, I uploaded a video in the Amazon S3 bucket created by the solution, and complete the parameters below :
@@ -194,8 +195,8 @@ Per default, AWS Batch chooses by itself an EC2 instance type available. If I wa
 instance_type = 'c5.large'
 result = batch.submit_job(
     jobName=job_name,
-    jobQueue="batch-FFmpeg-job-queue-" + compute,
-    jobDefinition="batch-FFmpeg-job-definition-" + compute,
+    jobQueue="batch-ffmpeg-job-queue-" + compute,
+    jobDefinition="batch-ffmpeg-job-definition-" + compute,
     parameters=command,
     nodeOverrides={
             "nodePropertyOverrides": [
@@ -250,7 +251,7 @@ In this example, we use the AWS CLI. A Step Functions execution receives a JSON
 Parameters of this `input.json are:
 
 - `$.name`: metadata of this job for observability.
-- `$.compute`: Instances family used to compute the media asset : `intel`, `arm`, `amd`, `nvidia`, `xilinx`.
+- `$.compute`: Instances family used to compute the media asset : `intel`, `arm`, `amd`, `nvidia`, `xilinx`, `fargate`, `fargate-arm`.
 - `$.input.s3_bucket` and `$.input.s3_prefix`: S3 url of the list of Amazon S3 Objects to be processed by FFMPEG.
 - `$.input.file_options`: FFmpeg input file options described in the official documentation.
 - `$.output.s3_bucket` and `$.output.s3_prefix`: S3 url where all processed media assets will be stored on Amazon S3.
@@ -301,7 +302,7 @@ The CDK stack is described in the directory `/cdk`.
 
 ## Performance and quality metrics
 
-AWS Customers also wants to use this solution to benchmark the video encoding performance and quality of Amazon EC2 instance families. I analyze performance and video quality metrics thanks to AWS X-Ray service. i define 3 segments : Amazon S3 download, FFmpeg Execution and Amazon S3 upload.
+AWS Customers also wants to use this solution to benchmark the video encoding performance and quality of Amazon EC2 instance families. I analyze performance and video quality metrics thanks to AWS X-Ray service. I define 3 segments : Amazon S3 download, FFmpeg Execution and Amazon S3 upload.
 
 If I switch the AWS SSM (Systems Manager) Parameter `/batch-ffmpeg/ffqm` to `TRUE`, quality metrics PSNR, SSIM, VMAF are calculated and exported as an AWS X-RAY metadata and as a JSON file in the Amazon S3 bucket with the key prefix `/metrics/ffqm`. Those metrics are available through AWS Athena views `batch_FFmpeg_ffqm_psnr`, `batch_FFmpeg_ffqm_ssim`, `batch_FFmpeg_ffqm_vmaf`.
 

diff --git a/application/Taskfile.yml b/application/Taskfile.yml
@@ -13,42 +13,6 @@ tasks:
       - |
         {{.DOCKER_EXE}} version
 
-  kaniko-nvidia:
-    desc: "Build Container image for nvidia"
-    cmds:
-      - task: kaniko-build
-        vars:
-          VARIANT: nvidia2004
-          VERSION: 6.0
-          ARCH: amd64
-
-  kaniko-xilinx:
-    desc: "Build Container image for xilinx"
-    cmds:
-      - task: kaniko-build
-        vars:
-          VARIANT: xilinx2004
-          VERSION: 4.4
-          ARCH: amd64
-
-  kaniko-ubuntu:
-    desc: "Build Container image for ubuntu"
-    cmds:
-      - task: kaniko-build
-        vars:
-          VARIANT: ubuntu2004
-          VERSION: 6.0
-          ARCH: amd64
-
-  kaniko-arm64:
-    desc: "Build Container image for ubuntu"
-    cmds:
-      - task: kaniko-build
-        vars:
-          VARIANT: ubuntu2004
-          VERSION: 6.0
-          ARCH: arm64
-
   docker-xilinx:
     desc: "Build Container image for xilinx"
     deps: [docker-login]
@@ -76,7 +40,7 @@ tasks:
       - task: docker-build
         vars:
           VARIANT: ubuntu2004-amd64
-          VERSION: 6.0
+          VERSION: 7.0
           ARCH: linux/amd64
 
   docker-arm64:
@@ -86,7 +50,7 @@ tasks:
       - task: docker-build
         vars:
           VARIANT: ubuntu2004-arm64
-          VERSION: 6.0
+          VERSION: 7.0
           ARCH: linux/arm64
 
   docker-login:

diff --git a/application/docker-images/6.0/nvidia2004-amd64/Dockerfile b/application/docker-images/6.0/nvidia2004-amd64/Dockerfile
@@ -62,7 +62,7 @@ RUN add-apt-repository ppa:deadsnakes/ppa -y \
 # Build
 FROM devel-base as build
 
-ENV     FFMPEG_VERSION=snapshot \
+ENV     FFMPEG_VERSION=6.1.1 \
         NVIDIA_HEADERS_VERSION=12.1.14.0 \
         AOM_VERSION=3.7.0 \
         FDKAAC_VERSION=2.0.2 \
@@ -127,7 +127,7 @@ RUN \
         DIR=/tmp/nv-codec-headers && \
         git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git ${DIR} && \
         cd ${DIR} && \
-        #git checkout n${NVIDIA_HEADERS_VERSION} && \
+        git checkout n${NVIDIA_HEADERS_VERSION} && \
         make PREFIX="${PREFIX}" && \
         make install PREFIX="${PREFIX}" && \
         rm -rf ${DIR}

diff --git a/application/docker-images/6.0/ubuntu2004-amd64/Dockerfile b/application/docker-images/6.0/ubuntu2004-amd64/Dockerfile
@@ -32,7 +32,7 @@ RUN add-apt-repository ppa:deadsnakes/ppa -y \
 
 FROM base as build
 
-ENV     FFMPEG_VERSION=snapshot \
+ENV     FFMPEG_VERSION=6.1.1 \
         AOM_VERSION=3.7.0 \
         FDKAAC_VERSION=2.0.2 \
         FONTCONFIG_VERSION=2.14.2 \

diff --git a/application/docker-images/6.0/ubuntu2004-arm64/Dockerfile b/application/docker-images/6.0/ubuntu2004-arm64/Dockerfile
@@ -32,7 +32,7 @@ RUN add-apt-repository ppa:deadsnakes/ppa -y \
 
 FROM base as build
 
-ENV     FFMPEG_VERSION=snapshot \
+ENV     FFMPEG_VERSION=6.1.1 \
         AOM_VERSION=3.7.0 \
         FDKAAC_VERSION=2.0.2 \
         FONTCONFIG_VERSION=2.14.2 \