Skip to content

Latest commit

 

History

History
64 lines (32 loc) · 4.36 KB

README.md

File metadata and controls

64 lines (32 loc) · 4.36 KB
Raw

Project-Two-Bitcoin-Covid-Timeseries-Comparison

This project was started in July 2022 with my mentor Adrian Brudaru's suggestion; everything was done from scratch.

The basic idea was to get daily Bitcoin Prices via an API and ingest them into a database. The second step would be to use the Google Public Covid dataset and get the Daily number of Covid cases in some countries and find out if a correlation between the two exists or not.

Problem Statement:

Bitcoin is a volatile cryptocurrency, and its variations cannot be predicted. However, the Corona Virus COVID-19 significantly impacted the world economy. So we were curious to know how COVID-19 effected Bitcoin Prices. To answer this question, we decided to do an end-to-end project which could ingest daily COVID-19 cases from the top 9 economies of the world and also ingest the everyday Bitcoin prices from Coin Ranking API.

Project Setup

  1. The project was set up on a VM instance on google cloud. So all the frameworks to be used were set up as docker containers. The project uses the following technologies.

So all the important files are stored in the following location in this repo:

  1. In this project, python scripts were used for ingestion from Coin Ranking API. response.request method was used to call the API, and JSON input from the API was converted to pandas data frame and loaded into Bigquery via the pdf.to_gbq method. The used in this script is here:
  1. The following code runs in the Airflow Dag using the python callable function. The Next step is to transform the data used in the analysis. The data transformation was done using the Data Build Tool, also known as dbt. This framework was installed in the Docker container via Sudo mode, and the transformations were done using the following models in dbt.

The entire filesystem of dbt can be seen here dbt_folders

  1. The last step was to email the developer daily that the airflow dags had been run. This was achieved using the email operator in Airflow.

  2. Next the dashboards were made using datastudio, These are the dash boards which were made using data studio:

  • Dashboard One, Comparing Daily bitcoin prices with Covid 19 Cases in USA & India Dashboard One- Daily Variations

  • Dashboard Two, Co-relation coffecient between daily Bitcoin Prices and Covid cases using orginal timeseries and shifted time series data by two weeks. Dashboard Two-Co-relation Coffecient

  • Dashboard Three, Linear regression was done on sample data using Bigquery Model Dashboard Three

So, this completes the projects. :) These Dashboards can show the daily variations of Covid Cases in top Nine economies of the world, i.e.

USA, CHINA, JAPAN, GERMANY, UK, FRANCE, INDIA, CANADA and SOUTH KOREA.

Thank You !