Pinned Loading

1. spark-kafka-temp-stream spark-kafka-temp-stream Public

   This project implements a real-time data pipeline that streams CPU temperature data from multiple devices using Kafka and Spark Streaming. The processed data is stored in PostgreSQL and visualized …

   Python 1

2. worldbank-finance-elt-pipeline worldbank-finance-elt-pipeline Public

   Automated ELT pipeline extracting World Bank Finance data, transforming with Pandas, and loading into Azure SQL Database using Apache Airflow.

   Python

3. openweather-data-pipeline openweather-data-pipeline Public

   This project uses Apache Airflow to extract weather data from the OpenWeather API, process it with PySpark, and load it into Azure SQL Database. Intermediate data is stored in Azure Blob Storage. T…

   Python

4. USGS_ELT-dbt-snowflake USGS_ELT-dbt-snowflake Public

   USGS Earthquake ELTL Pipeline: A cloud-based solution that extracts, loads, transforms, and loads real-time earthquake data from the USGS Earthquake API into Snowflake, utilizing dbt for efficient …

   Python