links-as-of-1-17.md

General

Seeding

How the Internet Archive Crawler Works

• https://docs.google.com/document/d/1PeWefW2toThs-Pbw0CMv2us7wxQI0gRrP1LGuwMp_UQ/edit

Seeding the Internet Archive’s Webcrawler

• https://docs.google.com/document/d/1qpuNCmBmu4KcsS_hE2srewcCiP4f9P5cCyDfHmsSAVU/edit

Archiving using the Google Chrome Extension & Sub-primers

• https://docs.google.com/document/d/1L_JYldCwCHxVEW_9nD6llWPtbh_VvPwp2UYE6cGE2g0/edit

Seeding and Sorting Example

• https://docs.google.com/document/d/1ue63H3inYcvWWRXkHRIhE_qPnfeICjqXraTPAO1RVtk/edit

Data Refuge Seed Progress DataRescue Philly DOE

• https://docs.google.com/spreadsheets/d/1pQ_2GkgGwdY3emPhJwL4fSnT8miYpYGPkhABppNSwG0/edit#gid=707149033

Data Refuge Seed Progress DataRescue Philly NOAA

• https://docs.google.com/spreadsheets/d/1pQ_2GkgGwdY3emPhJwL4fSnT8miYpYGPkhABppNSwG0/edit#gid=0

Seeded URL's DataRescue Philly

• https://docs.google.com/spreadsheets/d/1cEGTGgjnNzcd39-2M-vWJxdobhSVaGGuKgyb59SUsFs/edit

EDGI URL Tracker with Agency Coding (Responses) 13:40:06 DataRescue Philly

• https://docs.google.com/spreadsheets/d/1sJiZKx44RSo8xBHmLR1VmJRtHtBx_a2y8NRPI9qcXaQ/edit#gid=766232732

Seeding and Sorting DataRescue Philly Debrief Notes

https://docs.google.com/document/d/10o-MQ2gDK5KfeXkSc2mglZwmWH0CahKrgAGzldptfbA/edit

Baggers

Initial Bagger Workflower and Documentation DataRescue Philly

• https://docs.google.com/document/d/1toEjVTyIhX4d3mCVDTnohPx74umU-R_tGmo43h6kU3E/edit?ts=587a3928

EDGI Uncrawlable Content

• https://docs.google.com/spreadsheets/d/1nSav_VXulUWi6Zf_FVEzbNgkhFJlMZtiK6JMsY8Sdhw/edit#gid=1967480336

Uncrawable Content Spreadsheet Template DataRescue Philly

• https://docs.google.com/spreadsheets/d/1nevyzpc-vwoK6krngEASPJS5_ngrOwD2qYUPxLVesrg/edit

Technical Metadata Bag DataRescue Philly

• https://docs.google.com/document/d/1Xj5iqJxy9y_J_VhGbd5YvEjPC29tQDyssCOD3lKvIaE/edit
• DataRescuePhilly Metadata Schema https://docs.google.com/spreadsheets/d/12JAleU6eF4wgu0hIlQ5efoOUCsZjT-UkRxwAofVZp6c/edit#gid=1879921913

Scrape Methodology by Adam Labay

• https://docs.google.com/document/d/1Za2NRNMS2_smJOQnQucTDVm1S8XsPdMcA5dUrUlX0xo/edit?usp=sharing

Scraping Code by Rob Emanuele

• https://gist.github.com/lossyrob/db5ebc427e43bf211f7874a0b7fdb3c6

EDGI URL Priority List

• How- to: https://docs.google.com/spreadsheets/d/11nCTBATDGABjRhgsFEtukAClsxCodkMgWW1GSLwLBfI/edit#gid=1041328002

  EDGI Toolkit

  EDGI Github

Agency Office Code Lookup Table

• https://docs.google.com/document/d/1Vd9Zb2VWHwYUEG8N-Qc4yqBaP8FUb7426rKRgWWz-1I/edit

NOAA Seeding and Sorting

• https://docs.google.com/spreadsheets/d/1FyENnUIxgauRTWfSKwLnFYfZRu9IGUDqyyu8Fq0vNYI/edit#gid=0