Code

[WIP] DO NOT MERGE#4

Code

Open

alexander-morris wants to merge 35 commits intokoii-network:masterkoii-network/task-template:masterfrom GET-Store-CAT:masterGET-Store-CAT/data-gatherer:masterCopy head branch name to clipboard

+11,763-4,587

Commits

Commits on Apr 13, 2023

• adding adapter

  

  alexander-morris

  committed
  4e70d91

  Copy full SHA for 4e70d91

• Ff

  

  alexander-morris

  committed
  6d8aa0c

  Copy full SHA for 6d8aa0c

Commits on Apr 15, 2023

• fixing gatherer

  

  alexander-morris

  committed
  b6b8b95

  Copy full SHA for b6b8b95

• fixing minor interface issues

  

  alexander-morris

  committed
  bb675c4

  Copy full SHA for bb675c4

• fixing list scaffold

  

  alexander-morris

  committed
  0123355

  Copy full SHA for 0123355

Commits on Apr 16, 2023

• adding test-starting

  

  alexander-morris

  committed
  1267dd4

  Copy full SHA for 1267dd4

• fixed twitter handlers

  

  alexander-morris

  committed
  a3a7b98

  Copy full SHA for a3a7b98

• debugging search iterators

  

  alexander-morris

  committed
  5b0f87b

  Copy full SHA for 5b0f87b

• adding data model and fixing last bugs, now just need to tune for the twitter api

  

  alexander-morris

  committed
  66c0a71

  Copy full SHA for 66c0a71

Commits on Apr 18, 2023

• adding arweave

  

  alexander-morris

  committed
  b4c1497

  Copy full SHA for b4c1497

• debugging arweave logic now

  

  alexander-morris

  committed
  ec2d662

  Copy full SHA for ec2d662

• fixed looping

  

  alexander-morris

  committed
  26ddd5f

  Copy full SHA for 26ddd5f

Commits on Apr 19, 2023

• fixing main loop

  

  alexander-morris

  committed
  d4439dd

  Copy full SHA for d4439dd

• fixing main loop

  

  alexander-morris

  committed
  d0e6339

  Copy full SHA for d0e6339

Commits on Apr 26, 2023

• arweave code review

  SomaKoii

  committed
  f9e74bb

  Copy full SHA for f9e74bb

Commits on Apr 27, 2023

• use puppeteer log in to twitter

  SomaKoii

  committed
  0f46e77

  Copy full SHA for 0f46e77

• add twitter verify option

  SomaKoii

  committed
  3f7d4c8

  Copy full SHA for 3f7d4c8

• add twitter stuff in env

  SomaKoii

  committed
  d6bc4ad

  Copy full SHA for d6bc4ad

• update scrape code

  SomaKoii

  committed
  b5f63c9

  Copy full SHA for b5f63c9

• getting comments, likes, shares and views

  SomaKoii

  committed
  6b3ea66

  Copy full SHA for 6b3ea66

• update scraping script

  SomaKoii

  committed
  0173f0f

  Copy full SHA for 0173f0f

Commits on May 2, 2023

• store results of twitter scrape to local leveldb

  aelavell

  committed
  b2fa4eb

  Copy full SHA for b2fa4eb

• update leveldb class

  SomaKoii

  committed
  d353405

  Copy full SHA for d353405

• Merge pull request #2 from aelavell/feat/twitter-leveldb

  Show description for 644a3cc
  somali0128

  authored
  644a3cc

  Copy full SHA for 644a3cc

• fix and update leveldb function

  SomaKoii

  committed
  23a056c

  Copy full SHA for 23a056c

• format leveldb

  SomaKoii

  committed
  367188e

  Copy full SHA for 367188e

• Merge pull request #3 from GET-Store-CAT/@feature/leveldb

  Show description for 8ddf6a1
  somali0128

  authored
  8ddf6a1

  Copy full SHA for 8ddf6a1

• update twitter scraping code work with db-model

  SomaKoii

  committed
  252c0c7

  Copy full SHA for 252c0c7

• update some calls to db.getPending to become db.getPendingList

  aelavell

  committed
  5819ee9

  Copy full SHA for 5819ee9

• Merge pull request #1 from GET-Store-CAT/@feature/twitter-scrape

  Show description for 05e9b60
  somali0128

  authored
  05e9b60

  Copy full SHA for 05e9b60

• Merge pull request #4 from aelavell/feature/arweave

  Show description for b10b30b
  somali0128

  authored
  b10b30b

  Copy full SHA for b10b30b

• add TODO twitter scrape scrollPage()

  SomaKoii

  committed
  20ef0b4

  Copy full SHA for 20ef0b4

Commits on May 3, 2023

• autoscroll with randomized "natural" scrolling, plus test

  aelavell

  committed
  3d00069

  Copy full SHA for 3d00069

• Merge pull request #5 from aelavell/feature/autoscroll

  Show description for ea44428
  somali0128

  authored
  ea44428

  Copy full SHA for ea44428

Commits on May 16, 2023

• fixing readme, wip

  alexander-morris

  committed
  aa106fe

  Copy full SHA for aa106fe