Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfix release v1.3.1 #741

Merged
merged 48 commits into from
Nov 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
230d718
Bump hotfix number
euanmillar Oct 2, 2023
a54d224
Bump deploy action release number
euanmillar Oct 2, 2023
d5fcf91
fix backing up elasticsearch, move all directory creation to the same…
rikukissa Oct 3, 2023
7ed9c33
Merge pull request #743 from opencrvs/hotfix-es-backup
euanmillar Oct 3, 2023
977dbd2
fix backing up elasticsearch, move all directory creation to the same…
rikukissa Oct 3, 2023
5183cf2
Support spouse occupation and educational attainment
euanmillar Oct 3, 2023
5a9d415
deploy metrics to only data1 node
tahmidrahman-dsi Oct 11, 2023
e4384e9
remove option from informant type dropdown
Oct 12, 2023
49e2b80
print message update
Oct 13, 2023
d20cd5e
issue collector form message update
Oct 16, 2023
de193f4
corrector message update
Oct 16, 2023
63ba20f
ocrvs-5795 email message add for search (#685)
Nil20 Oct 17, 2023
d058734
add signature field constant for roles
Oct 3, 2023
689c4bb
organize ui configurable constants
Oct 3, 2023
03b85bb
test svg add for user signatures
Oct 6, 2023
c0cd762
comment add for transformer
Oct 6, 2023
770f249
deprecate style tag from signature handlebar svg
Oct 17, 2023
cec27b0
make sure minio is always deployed to the data1 node
rikukissa Oct 6, 2023
d462b30
Ensure jq is installed on all production servers (#759)
rikukissa Oct 17, 2023
c811309
Include type parameter in name query
Zangetsu101 Oct 18, 2023
da8653f
id verifier message add for age
Oct 18, 2023
7aeec27
corrector id verifier message update
Oct 19, 2023
a1bfc30
Make it possible to confgure handlebars
euanmillar Oct 19, 2023
358f8b4
revert name length used for testing
euanmillar Oct 19, 2023
cba30fd
Merge pull request #762 from opencrvs/ocrvs-6052
euanmillar Oct 20, 2023
4e8c8ab
Merge pull request #761 from opencrvs/ocrvs-6063
euanmillar Oct 20, 2023
4161232
Merge pull request #754 from opencrvs/ocrvs-5794
euanmillar Oct 20, 2023
b4eb19f
Merge pull request #757 from opencrvs/ocrvs-5994
euanmillar Oct 20, 2023
3392065
Merge pull request #756 from opencrvs/ocrvs-6032
euanmillar Oct 20, 2023
b98a240
Fix conflict
euanmillar Oct 20, 2023
c625589
Merge pull request #745 from opencrvs/niue-form-fies
euanmillar Oct 20, 2023
25fa93f
Merge pull request #763 from opencrvs/configurable-handlebars
euanmillar Oct 20, 2023
26958e4
Add registration location to email
euanmillar Oct 23, 2023
984e7d8
Merge pull request #764 from opencrvs/add-rg-loc-to-email
euanmillar Oct 24, 2023
08de054
Add a separate tracking id handler
tahmidrahman-dsi Oct 25, 2023
9788d7a
Add a separate tracking id handler (#768)
tahmidrahman-dsi Oct 26, 2023
901b06b
fix message for collector form (#769)
Nil20 Oct 26, 2023
f9f8ec1
Update tracking id handler
tahmidrahman-dsi Oct 26, 2023
e3bb0f2
Merge branch 'release-v1.3.1' into ocrvs-6093
tahmidrahman-dsi Oct 26, 2023
047611f
Type improvements
tahmidrahman-dsi Oct 26, 2023
5cba087
Merge pull request #770 from opencrvs/ocrvs-6093
tahmidrahman-dsi Oct 26, 2023
9e4fd25
add a placeholder file for custom handlebar helpers (#772)
rikukissa Oct 26, 2023
a200480
Configurable whitelist (#773)
euanmillar Oct 30, 2023
aadd271
Copy amend (#778)
euanmillar Nov 5, 2023
e049853
Endpoint for certificate fonts & configuration (#782)
Zangetsu101 Nov 8, 2023
5d0312d
Some minor fixes as an interim solution (#784)
euanmillar Nov 9, 2023
57a6fbd
Add route and deploy approach to customise metabase (#777)
euanmillar Nov 9, 2023
9bbb478
Fix master conflict
euanmillar Nov 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/deploy-prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ on:
core-image-tag:
description: Core DockerHub image tag
required: true
default: 'v1.3.0'
default: 'v1.3.1'
countryconfig-image-tag:
description: Your Country Config DockerHub image tag
required: true
Expand Down Expand Up @@ -78,7 +78,7 @@ jobs:
sleep 10
done
while true; do
if docker manifest inspect opencrvs/ocrvs-farajaland:${{ github.event.inputs.countryconfig-image-tag }}; then
if docker manifest inspect ${{ secrets.DOCKERHUB_ACCOUNT }}/${{ secrets.DOCKERHUB_REPO }}:${{ github.event.inputs.countryconfig-image-tag }}; then
break
fi
sleep 10
Expand Down Expand Up @@ -110,6 +110,7 @@ jobs:
INFOBIP_API_KEY: ${{ secrets.INFOBIP_API_KEY }}
SENDER_EMAIL_ADDRESS: ${{ secrets.SENDER_EMAIL_ADDRESS }}
SUPER_USER_PASSWORD: ${{ secrets.SUPER_USER_PASSWORD }}
CONTENT_SECURITY_POLICY_WILDCARD: ${{ vars.CONTENT_SECURITY_POLICY_WILDCARD }}
run: |
cd ./${{ github.event.repository.name }}
yarn deploy --clear_data=no --environment=${{ github.event.inputs.deploy-script-environment }} --host=${{ env.DOMAIN }} --version=${{ github.event.inputs.core-image-tag }} --country_config_version=${{ github.event.inputs.countryconfig-image-tag }} --country_config_path=../${{ github.event.repository.name }} --replicas=${{ env.REPLICAS }}
6 changes: 3 additions & 3 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
core-image-tag:
description: Core DockerHub image tag
required: true
default: 'v1.3.0'
default: 'v1.3.1'
countryconfig-image-tag:
description: Your Country Config DockerHub image tag
required: true
Expand Down Expand Up @@ -81,12 +81,11 @@ jobs:
sleep 10
done
while true; do
if docker manifest inspect opencrvs/ocrvs-farajaland:${{ github.event.inputs.countryconfig-image-tag }}; then
if docker manifest inspect ${{ secrets.DOCKERHUB_ACCOUNT }}/${{ secrets.DOCKERHUB_REPO }}:${{ github.event.inputs.countryconfig-image-tag }}; then
break
fi
sleep 10
done


- name: Deploy to ${{ github.event.inputs.environment }}
id: deploy
Expand Down Expand Up @@ -114,6 +113,7 @@ jobs:
INFOBIP_API_KEY: ${{ secrets.INFOBIP_API_KEY }}
SENDER_EMAIL_ADDRESS: ${{ secrets.SENDER_EMAIL_ADDRESS }}
SUPER_USER_PASSWORD: ${{ secrets.SUPER_USER_PASSWORD }}
CONTENT_SECURITY_POLICY_WILDCARD: ${{ vars.CONTENT_SECURITY_POLICY_WILDCARD }}
run: |
cd ./${{ github.event.repository.name }}
yarn deploy --clear_data=${{ github.event.inputs.reset }} --environment=${{ github.event.inputs.environment }} --host=${{ env.DOMAIN }} --version=${{ github.event.inputs.core-image-tag }} --country_config_version=${{ github.event.inputs.countryconfig-image-tag }} --country_config_path=../${{ github.event.repository.name }} --replicas=${{ env.REPLICAS }}
Expand Down
12 changes: 11 additions & 1 deletion infrastructure/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
set -e

BASEDIR=$(dirname $0)
PARENT_DIR=$(dirname $(dirname $0))

# Reading Names parameters
for i in "$@"; do
Expand Down Expand Up @@ -173,6 +174,11 @@ if [ -z "$DOCKERHUB_REPO" ] ; then
print_usage_and_exit
fi

if [ -z "$CONTENT_SECURITY_POLICY_WILDCARD" ] ; then
echo 'Error: Missing environment variable CONTENT_SECURITY_POLICY_WILDCARD.'
print_usage_and_exit
fi

if [ -z "$TOKENSEEDER_MOSIP_AUTH__PARTNER_MISP_LK" ] ; then
echo 'Info: Missing optional MOSIP environment variable TOKENSEEDER_MOSIP_AUTH__PARTNER_MISP_LK.'
TOKENSEEDER_MOSIP_AUTH__PARTNER_MISP_LK=''
Expand Down Expand Up @@ -309,6 +315,9 @@ cp $BASEDIR/emergency-restore-metadata.sh /tmp/opencrvs/infrastructure/emergency
# Copy authorized keys
cp $BASEDIR/authorized_keys /tmp/opencrvs/infrastructure/authorized_keys

# Copy metabase database
cp $PARENT_DIR/src/api/dashboards/file/metabase.init.db.sql /tmp/opencrvs/infrastructure/metabase.init.db.sql

rotate_authorized_keys() {
# file exists and has a size of more than 0 bytes
if [ -s "/tmp/opencrvs/infrastructure/authorized_keys" ]; then
Expand Down Expand Up @@ -446,7 +455,8 @@ docker_stack_deploy() {
NATIONAL_ID_OIDP_ESSENTIAL_CLAIMS=$NATIONAL_ID_OIDP_ESSENTIAL_CLAIMS
NATIONAL_ID_OIDP_VOLUNTARY_CLAIMS=$NATIONAL_ID_OIDP_VOLUNTARY_CLAIMS
NATIONAL_ID_OIDP_CLIENT_PRIVATE_KEY=$NATIONAL_ID_OIDP_CLIENT_PRIVATE_KEY
NATIONAL_ID_OIDP_JWT_AUD_CLAIM=$NATIONAL_ID_OIDP_JWT_AUD_CLAIM"
NATIONAL_ID_OIDP_JWT_AUD_CLAIM=$NATIONAL_ID_OIDP_JWT_AUD_CLAIM
CONTENT_SECURITY_POLICY_WILDCARD=$CONTENT_SECURITY_POLICY_WILDCARD"

echo "Pulling all docker images. This might take a while"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ services:

config:
environment:
- QA_ENV=true
- SENTRY_DSN=${SENTRY_DSN}

metrics:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ services:

config:
environment:
- NODE_ENV=production
- SENTRY_DSN=${SENTRY_DSN}

metrics:
Expand Down
25 changes: 21 additions & 4 deletions infrastructure/docker-compose.deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,10 @@ services:
- '/data/minio:/data'
command: server --console-address ":9001" /data
deploy:
replicas: 1
placement:
constraints:
- node.labels.data1 == true
labels:
- 'traefik.enable=true'
- 'traefik.docker.network=opencrvs_overlay_net'
Expand Down Expand Up @@ -321,7 +325,7 @@ services:
options:
gelf-address: 'udp://127.0.0.1:12201'
tag: 'minio'

setup-elasticsearch-users:
image: ubuntu:bionic
entrypoint: ['bash', '/usr/app/setup.sh']
Expand Down Expand Up @@ -466,7 +470,7 @@ services:
client:
environment:
- COUNTRY_CONFIG_URL=https://countryconfig.{{hostname}}
- HOST={{hostname}}
- CONTENT_SECURITY_POLICY_WILDCARD=${CONTENT_SECURITY_POLICY_WILDCARD}
- MINIO_URL=https://minio.{{hostname}}
deploy:
labels:
Expand Down Expand Up @@ -543,8 +547,10 @@ services:
replicas: 1
environment:
- APN_SERVICE_URL=http://apm-server:8200
- COUNTRY_LOGO_URL=https://countryconfig.{{hostname}}/content/country-logo
- COUNTRY_CONFIG_URL=https://countryconfig.{{hostname}}
- LOGIN_URL=https://login.{{hostname}}
- CLIENT_APP_URL=https://register.{{hostname}}
- DOMAIN={{hostname}}
networks:
- overlay_net
logging:
Expand All @@ -555,7 +561,7 @@ services:
login:
environment:
- COUNTRY_CONFIG_URL=https://countryconfig.{{hostname}}
- HOST={{hostname}}
- CONTENT_SECURITY_POLICY_WILDCARD=${CONTENT_SECURITY_POLICY_WILDCARD}
deploy:
labels:
- 'traefik.enable=true'
Expand Down Expand Up @@ -588,6 +594,9 @@ services:
- APN_SERVICE_URL=http://apm-server:8200
- CERT_PRIVATE_KEY_PATH=/run/secrets/jwt-private-key.{{ts}}
- CERT_PUBLIC_KEY_PATH=/run/secrets/jwt-public-key.{{ts}}
- LOGIN_URL=https://login.{{hostname}}
- COUNTRY_CONFIG_URL=https://countryconfig.{{hostname}}
- CLIENT_APP_URL=https://register.{{hostname}}
- DOMAIN={{hostname}}
deploy:
labels:
Expand Down Expand Up @@ -654,6 +663,8 @@ services:
environment:
- APN_SERVICE_URL=http://apm-server:8200
- CERT_PUBLIC_KEY_PATH=/run/secrets/jwt-public-key.{{ts}}
- LOGIN_URL=https://login.{{hostname}}
- CLIENT_APP_URL=https://register.{{hostname}}
- DOMAIN={{hostname}}
deploy:
labels:
Expand Down Expand Up @@ -725,6 +736,9 @@ services:
- HEARTH_MONGO_URL=mongodb://hearth:${HEARTH_MONGODB_PASSWORD}@mongo1/hearth-dev?replicaSet=rs0
- DASHBOARD_MONGO_URL=mongodb://performance:${PERFORMANCE_MONGODB_PASSWORD}@mongo1/performance?replicaSet=rs0
deploy:
placement:
constraints:
- node.labels.data1 == true
labels:
- 'traefik.enable=false'
replicas: 1
Expand Down Expand Up @@ -779,6 +793,8 @@ services:
- APN_SERVICE_URL=http://apm-server:8200
- CERT_PUBLIC_KEY_PATH=/run/secrets/jwt-public-key.{{ts}}
- MONGO_URL=mongodb://config:${CONFIG_MONGODB_PASSWORD}@mongo1/application-config?replicaSet=rs0
- LOGIN_URL=https://login.{{hostname}}
- CLIENT_APP_URL=https://register.{{hostname}}
- DOMAIN={{hostname}}
deploy:
labels:
Expand Down Expand Up @@ -959,6 +975,7 @@ services:
dashboards:
volumes:
- /data/metabase:/data/metabase
- /data/metabase/metabase.init.db.sql:/metabase.init.db.sql
networks:
- overlay_net
environment:
Expand Down
37 changes: 22 additions & 15 deletions infrastructure/emergency-backup-metadata.sh
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ excluded_collections() {
# Today's date is used for filenames if LABEL is not provided
#-----------------------------------
BACKUP_DATE=$(date +%Y-%m-%d)
REMOTE_DIR="$REMOTE_DIR/${LABEL:-$BACKUP_DATE}"

# Backup Hearth, OpenHIM, User, Application-config and any other service related Mongo databases into a mongo sub folder
# ---------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -240,7 +241,7 @@ echo ""

create_elasticsearch_backup() {
OUTPUT=""
OUTPUT=$(docker run --rm --network=$NETWORK appropriate/curl curl -s -X PUT -H "Content-Type: application/json;charset=UTF-8" "http://$(elasticsearch_host)/_snapshot/ocrvs/snapshot_${VERSION:-$BACKUP_DATE}?wait_for_completion=true&pretty" -d '{ "indices": "ocrvs" }' 2>/dev/null)
OUTPUT=$(docker run --rm --network=$NETWORK appropriate/curl curl -s -X PUT -H "Content-Type: application/json;charset=UTF-8" "http://$(elasticsearch_host)/_snapshot/ocrvs/snapshot_${LABEL:-$BACKUP_DATE}?wait_for_completion=true&pretty" -d '{ "indices": "ocrvs" }' 2>/dev/null)
if echo $OUTPUT | jq -e '.snapshot.state == "SUCCESS"' > /dev/null; then
echo "Snapshot state is SUCCESS"
else
Expand Down Expand Up @@ -286,14 +287,19 @@ else
fi

echo "Creating a backup for Minio"
cd $ROOT_PATH/minio && tar -zcvf $ROOT_PATH/backups/minio/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz . && cd /

LOCAL_MINIO_BACKUP=$ROOT_PATH/backups/minio/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz
cd $ROOT_PATH/minio && tar -zcvf $LOCAL_MINIO_BACKUP . && cd /

echo "Creating a backup for Metabase"

cd $ROOT_PATH/metabase && tar -zcvf $ROOT_PATH/backups/metabase/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz . && cd /
LOCAL_METABASE_BACKUP=$ROOT_PATH/backups/metabase/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz
cd $ROOT_PATH/metabase && tar -zcvf $LOCAL_METABASE_BACKUP . && cd /

echo "Creating a backup for VSExport"
cd $ROOT_PATH/vsexport && tar -zcvf $ROOT_PATH/backups/vsexport/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz . && cd /

LOCAL_VSEXPORT_BACKUP=$ROOT_PATH/backups/vsexport/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz
cd $ROOT_PATH/vsexport && tar -zcvf $LOCAL_VSEXPORT_BACKUP . && cd /

if [[ "$IS_LOCAL" = true ]]; then
echo $WORKING_DIR
Expand All @@ -304,17 +310,18 @@ fi
# Copy the backups to an offsite server in production
#----------------------------------------------------
if [[ "$OWN_IP" = "$PRODUCTION_IP" || "$OWN_IP" = "$(dig $PRODUCTION_IP +short)" ]]; then
script -q -c "rsync -a -r --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/elasticsearch/ $SSH_USER@$SSH_HOST:$REMOTE_DIR/elasticsearch" && echo "Copied elasticsearch backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/minio/${LABEL:-$BACKUP_DATE} $SSH_USER@$SSH_HOST:$REMOTE_DIR/minio" && echo "Copied minio backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/metabase/${LABEL:-$BACKUP_DATE} $SSH_USER@$SSH_HOST:$REMOTE_DIR/metabase" && echo "Copied Metabase backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/influxdb/${LABEL:-$BACKUP_DATE} $SSH_USER@$SSH_HOST:$REMOTE_DIR/influxdb" && echo "Copied influx backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/hearth-dev-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied hearth backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/user-mgnt-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied user backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/openhim-dev-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied openhim backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/application-config-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied application-config backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/metrics-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied metrics backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/webhooks-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied webhooks backup files to remote server."
script -q -c "rsync -a -r --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/performance-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied performance backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/elasticsearch/ && rsync' --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/elasticsearch/ $SSH_USER@$SSH_HOST:$REMOTE_DIR/elasticsearch" && echo "Copied elasticsearch backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/minio/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/minio/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/minio" && echo "Copied minio backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/metabase/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/metabase/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/metabase" && echo "Copied Metabase backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/vsexport/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' /data/backups/vsexport/ocrvs-${LABEL:-$BACKUP_DATE}.tar.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/vsexport/" && echo "Copied VSExport backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/influxdb/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/influxdb/${LABEL:-$BACKUP_DATE} $SSH_USER@$SSH_HOST:$REMOTE_DIR/influxdb" && echo "Copied influx backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/hearth-dev-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied hearth backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/user-mgnt-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied user backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/openhim-dev-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied openhim backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/application-config-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied application-config backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/metrics-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied metrics backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/webhooks-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied webhooks backup files to remote server."
script -q -c "rsync -a -r --rsync-path='mkdir -p $REMOTE_DIR/mongo/ && rsync' --ignore-existing --progress --rsh='ssh -p$SSH_PORT' $ROOT_PATH/backups/mongo/performance-${LABEL:-$BACKUP_DATE}.gz $SSH_USER@$SSH_HOST:$REMOTE_DIR/mongo" && echo "Copied performance backup files to remote server."
fi

# Cleanup any old backups from influx or mongo. Keep previous 7 days of data and all elastic data
Expand Down
6 changes: 3 additions & 3 deletions infrastructure/emergency-restore-metadata.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ done

print_usage_and_exit() {
echo 'Usage: ./emergency-restore-metadata.sh --label=XXX --replicas=XXX'
echo "This script CLEARS ALL DATA and RESTORES'S A SPECIFIC DAY'S or VERSION'S DATA. This process is irreversable, so USE WITH CAUTION."
echo "Script must receive a label parameter to restore data from that specific day in format +%Y-%m-%d i.e. 2019-01-01 or that version"
echo "This script CLEARS ALL DATA and RESTORES'S A SPECIFIC DAY'S or label's data. This process is irreversable, so USE WITH CAUTION."
echo "Script must receive a label parameter to restore data from that specific day in format +%Y-%m-%d i.e. 2019-01-01 or that label"
echo "The Hearth, OpenHIM User and Application-config db backup zips you would like to restore from: hearth-dev-{label}.gz, openhim-dev-{label}.gz, user-mgnt-{label}.gz and application-config-{label}.gz must exist in /data/backups/mongo/ folder"
echo "The Elasticsearch backup folder /data/backups/elasticsearch must exist with all previous snapshots and indices. All files are required"
echo "The InfluxDB backup files must exist in the /data/backups/influxdb/{label} folder"
Expand All @@ -51,7 +51,7 @@ print_usage_and_exit() {
}

if [ -z "$LABEL" ]; then
echo "Error: Argument for the --label is required. You must select which day's or which version's data you would like to roll back to."
echo "Error: Argument for the --label is required. You must select which day's or which label's data you would like to roll back to."
print_usage_and_exit
fi

Expand Down
5 changes: 5 additions & 0 deletions infrastructure/server-setup/playbook-1.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,11 @@
name: python3-pip
state: present

- name: 'Install jq'
apt:
name: jq
state: present

- name: 'Install pexpect python module for ansible expect commands'
pip:
name: pexpect
Expand Down
7 changes: 6 additions & 1 deletion infrastructure/server-setup/playbook-3.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,11 @@
name: python3-pip
state: present

- name: 'Install jq'
apt:
name: jq
state: present

- name: 'Install pexpect python module for ansible expect commands'
pip:
name: pexpect
Expand Down Expand Up @@ -366,7 +371,7 @@
apt:
name: fail2ban
state: present

- name: 'Copy fail2ban jail.local'
copy:
src: ../jail.local
Expand Down
9 changes: 7 additions & 2 deletions infrastructure/server-setup/playbook-5.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@
name: python3-pip
state: present

- name: 'Install jq'
apt:
name: jq
state: present

- name: 'Install pexpect python module for ansible expect commands'
pip:
name: pexpect
Expand Down Expand Up @@ -350,12 +355,12 @@
ufw:
rule: allow
name: OpenSSH

- name: 'Install Fail2Ban'
apt:
name: fail2ban
state: present

- name: 'Copy fail2ban jail.local'
copy:
src: ../jail.local
Expand Down
3 changes: 3 additions & 0 deletions infrastructure/setup-deploy-config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ sed -i "s/{{hostname}}/$1/g" /opt/opencrvs/docker-compose.deploy.yml
KIBANA_ENCRYPTION_KEY=`uuidgen`
sed -i "s/{{KIBANA_ENCRYPTION_KEY}}/$KIBANA_ENCRYPTION_KEY/g" /opt/opencrvs/infrastructure/monitoring/kibana/kibana.yml

# Move metabase file
mv /opt/opencrvs/infrastructure/metabase.init.db.sql /data/metabase/metabase.init.db.sql

# Replace environment variables from all alert definition files
for file in /opt/opencrvs/infrastructure/monitoring/elastalert/rules/*.yaml; do
sed -i -e "s%{{HOST}}%$1%" $file
Expand Down
Loading
Loading