This repository contains code and documentation for building a robust data engineering pipeline to analyze Uber data. The pipeline is designed using Google Cloud Platform (GCP) services including Make-AI, BigQuery and Looker for data storage, processing, and visualization.
Uber generates vast amounts of data from various sources including ride requests, driver activities, and user interactions. Analyzing this data can provide valuable insights for optimizing operations, improving user experience, and making data-driven decisions.
The data engineering pipeline outlined in this repository aims to:
- Ingest raw Uber data from a csv file
- Extract Process and Transform data using Mage-AI
- Store Transformed data to google Cloud Storage
- Create meaningful visualizations and dashboards using Looker
- Enable stakeholders to explore and gain insights from the data
-> Google Cloud Platform (GCP)
- Compute Engine: For running Mage-AI
- BigQuery: For data warehousing and analytics
- Cloud Storage: For storing raw data files
-> Looker: For data visualization and exploration
-> Make-AI: To build ETL data engineering pipeline
Access the looker dashboard using this uber_data_analytics