Added MATIC ABI

Installed file-store-csv dependency
Removed old dependency
Removed old ABI
Added code for table(s), db
Modified processor to save Transfers

Author: RaekwonIII

.dockerignore

@@ -2,4 +2,4 @@ DB_NAME=squid
 DB_PORT=23798
 GQL_PORT=4350
 # JSON-RPC node endpoint, both wss and https endpoints are accepted
-RPC_ENDPOINT=
+RPC_ENDPOINT="https://rpc.ankr.com/eth"

.env

@@ -6,3 +6,8 @@
 
 # IDE files
 /.idea
+/data/*
+!*.py
+!*.ipynb
+!*.lock
+!*.toml

.gitignore

@@ -0,0 +1,42 @@
+# This configuration file was automatically generated by Gitpod.
+# Please adjust to your needs (see https://www.gitpod.io/docs/introduction/learn-gitpod/gitpod-yaml)
+# and commit this file to your remote git repository to share the goodness with others.
+
+# Learn more from ready-to-use templates: https://www.gitpod.io/docs/introduction/getting-started/quickstart
+github:
+  prebuilds:
+    # enable for the master/default branch (defaults to true)
+    master: true
+    # enable for all branches in this repo (defaults to false)
+    branches: false
+    # enable for pull requests coming from this repo (defaults to true)
+    pullRequests: true
+    # add a check to pull requests (defaults to true)
+    addCheck: true
+    # add a "Review in Gitpod" button as a comment to pull requests (defaults to false)
+    addComment: false
+
+tasks:
+  - init: |
+      npm i
+      npm i -g @subsquid/cli
+      gp sync-done setup
+      exit
+  - name: Python setup
+    command: |
+      cd data
+      curl -sSL https://install.python-poetry.org | python3 -
+      poetry install
+      gp sync-done py-setup
+      exit
+  - name: Squid procesor
+    command: |
+      gp open src/processor.ts
+      gp sync-await setup
+      sqd process
+  - name: Python data analysis
+    command: |
+      gp open data/analysis.py
+      gp sync-await py-setup
+      cd data
+      poetry run python analysis.py

.gitpod.yml

@@ -1,33 +1,77 @@
-# Gravatar squid
+# MATIC data analytics - A squid example for CSV storage
 
-This squid has been migrated from the [Gravatar subgraph](https://github.com/graphprotocol/example-subgraph). For a step-by-step migration guide, see the [migration docs page](https://docs.subsquid.io/migrate/migrate-subgraph/).
+<p align="center">
+	<img src="assets/white-woodmark.svg">
+</p>
 
-## Quickstart
+<div align="center">
 
-```bash
-# 1. Update the Squid SDK and install dependencies
-npm run update
-npm ci
+[Twitter](https://twitter.com/subsquid) | [Discord](https://discord.gg/subsquid) | [LinkedIn](https://linkedin.com/subsquid) | [Telegram](https://t.me/HydraDevs) | [GitHub](https://github.com/subsquid) | [YouTube](https://www.youtube.com/@subsquid)
 
-# 2. Compile typescript files
-make build
+</div>
 
-# 3. Start target Postgres database and detach
-make up
+## Quick-start
 
-# 4. Start the processor
-make process
+<div align="center">
 
-# 5. The command above will block the terminal
-#    being busy with fetching the chain data, 
-#    transforming and storing it in the target database.
-#
-#    To start the graphql server open the separate terminal
-#    and run. The GraphQL playground will be available at localhost:4350/graphql
-make serve
-```
+[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io#https://github.com/RaekwonIII/local-csv-indexing.git)
+
+</div>
+
+## Introduction
+
+In this article I describe how to use Subsquid's indexing framework for data analytics prototyping.
+I have built an indexer that processes MATIC transactions on Ethereum mainnet, and dumps them on a local CSV files.
+I have then developed a simple Python script to demonstrate how to import these CSVs into a Pandas DataFrame, and perform aggregation operations on this data.
+
+This project is a squid that indexes blockchain information generated by the Transfers of [MATIC](https://etherscan.io/token/0x7d1afa7b718fb893db30a3abc0cfc608aacfebb0) tokens on Ethereum mainnet.
+The indexer writes it to multiple files, divided in chunks of configurable size, in the CSV format.
+
+It also contains a simple Python script (in the `data` folder), which reads the CSV files, imports the data in a [Pandas](http://pandas.pydata.org/) DataFrame, aggregates the data (albeit rather trivially) and plots a bar chart.
+
+MATIC is the native token of the [Polygon project](https://polygon.technology/). It is defined by an ERC-20 standard smart contract, and the tokens are transferred via the contract's `transfer` function, which emits a `Transfer` event.
+Such event is exactly what the squid ETL is indexing, and eventually writing to CSV files using [Subsqduid's `file-store`](https://www.npmjs.com/package/@subsquid/file-store) and [`file-store-csv` libraries](https://www.npmjs.com/package/@subsquid/file-store-csv).
+
+The `analysis.py` Python script is using Pandas to read the data stored in all CSVs, and create a [DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html). Then, using the `groupby` function, it calculates the daily total value of transfers, and it creates a bar chart, using [`matplotlib` Python library](https://matplotlib.org/).
+
+<p align="center">
+	<img src="assets/plot.png">
+</p>
+
+The project is relatively simple, because its purpose is purely demonstrative. Its intent is to showcase the capabilities of Subsquid SDK.
 
-## Dev flow
+### What is a Squid?
+
+> A squid is a project that extracts and transforms on-chain data in order to present it as a GraphQL API. Squids are developed using the Subsquid SDK, which provides extensive tooling to define data schemas, data transfomation rules, and the shape of the resulting API.
+
+We recommend that you read Subsquid docs to understand how it works: https://docs.subsquid.io/
+
+## Prerequisites
+
+- Node 16.x
+- Docker
+- NPM
+
+## Quick-start local indexing
+
+1. Clone the repository
+2. Install dependencies (in a console window): `npm i`
+3. Build the project `sqd build`
+4. Launch the database container `sqd up`
+5. Launch the processor `sqd process`
+6. Launch the GraphQL server (in a separate console window) `sqd serve`
+7. Access the GraphiQL Playground, by running `sqd open http://localhost:4350/graphql` <!-- markdown-link-check-disable-line -->
+
+## Key components
+
+* The `schema.graphql` file is used to define the database and API schemas. A command line tool will automatically generate code from it, which you can find in `src/model/generated`
+* The `db/migrations` folder contains automatically files with SQL statements to modify the database (create, alter, delete tables), similarly to any ORM database interface.
+* The `src/abi` folder contains facade TypeScript code, automatically generated by a command line tool from one, or multiple smart contract ABI(s). This code is used to programmatically interface with the smart contract(s) and decode events and function calls.
+* The main logic of this project is defined in `src/processor.ts`. The `EvmBatchProcessor` class is configured and used to perform request to [Subsquid's Archive for Ethereum blockchain](https://app.subsquid.io/archives), to obtain necessary data. Then some custom logic is implemented to process this data in batches, and save it on the database with the custom defined structure.
+
+[Subsquid documentation](https://docs.subsquid.io/) has dedicated sections and pages describing each of these concepts, it is advised to consult them, before starting to develop your own squid.
+
+## Development flow
 
 ### 1. Define database schema
 
@@ -61,18 +105,17 @@ npx squid-typeorm-migration generate
 
 See [docs on schema updates](https://docs.subsquid.io/develop-a-squid/schema-file/schema-updates/) for more details.
 
-
 ### 4. Import ABI contract and generate interfaces to decode events
 
 It is necessary to import the respective ABI definition to decode EVM logs. 
 
-To generate a type-safe facade class to decode EVM logs, use `squid-evm-typegen(1)`. For example, for Gravatar we fetch the public ABI by the address `0x2E645469f354BB4F5c8a05B3b30A929361cf77eC` and tell the `evm-typegen` to use `Gravity` as the base name:
+To generate a type-safe facade class to decode EVM logs, place the ABI in the `assets` folder and use `squid-evm-typegen(1)`, e.g.:
 
 ```bash
-npx squid-evm-typegen src/abi 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC#Gravity --clean
+npx squid-evm-typegen src/abi assets/ERC721.json#erc721
 ```
-It generates the files `abi.support.ts`, `Gravity.abi.ts` and `Gravity.ts` in the destination folder `src/abi`.
 
+For more details about `squid-evm-typegen` read the [docs page](https://docs.subsquid.io/develop-a-squid/typegen/squid-evm-typegen/)
 
 ## Project conventions
 
@@ -83,14 +126,15 @@ The layout of `lib` must reflect `src`.
 * All TypeORM classes must be exported by `src/model/index.ts` (`lib/model` module).
 * Database schema must be defined in `schema.graphql`.
 * Database migrations must reside in `db/migrations` and must be plain js files.
-* `sqd(1)` and `squid-*(1)` executables consult `.env` file environment variables, in particular, to establish a database connection.
+* `sqd(1)` and `squid-*(1)` executables consult `.env` file for a number of environment variables.
 
 ## GraphQL server extensions
 
 It is possible to extend `squid-graphql-server(1)` with custom
 [type-graphql](https://typegraphql.com) resolvers and to add request validation. See [the docs](https://docs.subsquid.io/develop-a-squid/graphql-api/custom-resolvers/) for more details.
 
+## Learn More
 
-## Disclaimer
+You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
 
-This is alpha-quality software. The Squid SDK may introduce breaking changes in future versions.
+To learn React, check out the [React documentation](https://reactjs.org/).

Makefile

@@ -0,0 +1,41 @@
+{
+    "$schema": "https://cdn.subsquid.io/schemas/commands.json",
+    "commands": {
+      "clean": {
+        "description": "delete all build artifacts",
+        "cmd": ["npx", "--yes", "rimraf", "lib"]
+      },
+      "build": {
+        "description": "Build the squid project",
+        "deps": ["clean"],
+        "cmd": ["tsc"]
+      },
+      "typegen": {
+        "description": "Generate data access classes for an ABI file(s) in the ./abi folder",
+        "cmd": ["squid-evm-typegen", "./src/abi", {"glob": "./abi/*.json"}]
+      },
+      "process": {
+        "description": "Load .env and start the squid processor",
+        "deps": ["build"],
+        "cmd": ["node", "--require=dotenv/config", "lib/processor.js"]
+      },
+      "process:prod": {
+        "description": "Start the squid processor",
+        "cmd": ["node", "lib/processor.js"],
+        "hidden": true
+      },
+      "check-updates": {
+        "cmd": ["npx", "--yes", "npm-check-updates", "--filter=/subsquid/", "--upgrade"],
+        "hidden": true
+      },
+      "bump": {
+        "description": "Bump @subsquid packages to the latest versions",
+        "deps": ["check-updates"],
+        "cmd": ["npm", "i", "-f"]
+      },
+      "open": {
+        "description": "Open a local browser window",
+        "cmd": ["npx", "--yes", "opener"]
+      }
+    }
+  }