datafest-airflow-workshop

Code used for Datafest Africa workshop on Airflow

Requirements

To set up this project, you'll need to have the following installed on your Machine:

1. Python (>= 3.7 | can be installed here)
2. Docker (can be installed here)
3. Docker Compose (>= v1.29.1 | can be installed here)
4. Git

Setup

1. Clone the repository
2. Create a virtual environment. See here
3. Install requirements

pip install -r requirements.txt

4. Create .env file and update as follows:

ENV=dev
AIRFLOW__CORE__FERNET_KEY=
AIRFLOW_UID=

To get fernet key, see here To get AIRFLOW_UID, for MacOS/Linux users, run the following in your terminal and use the value

AIRFLOW_UID=$(id -u)
echo $AIRFLOW_UID

4. Run setup script:

chmod a+x ./bin/run-airflow.sh
./bin/run-airflow.sh