Skip to content

Latest commit

 

History

History
12 lines (9 loc) · 917 Bytes

File metadata and controls

12 lines (9 loc) · 917 Bytes

Building a Real-Time Event Streaming Pipeline with Kafka, BigQuery & PowerBI

  • This is a data engineering project and we have used India - Trade Data dataset (Trade statistics of India for export and import of commodities from 2010–2021) from Kaggle. Let's get started!
  • Tools used : Apache Kakfa(distributed event store and stream-processing platform), Google BigQuery (Data Warehouse), PowerBI(Data Viz)

Complete workflow and execution of this project has been explained in this article.

Key to understand the data :

  • Value - values for export and import of commodities in million US $.
  • Country - Country Exported To
  • Year - Year in which commodities where Exported which is in between 2010 to 2021.
  • Commodities - type of goods which was exported between 2010 to 2021