v1.12: adds ICC-Crawler

What's Changed

• Add last line of defence to FAQ by @nisbet-hubbard in #24
• Add ICC-Crawler by @mirium999 in #25

New Contributors

• @mirium999 made their first contribution in #25

Full Changelog: v1.11...v1.12

v1.11: adds PetalBot and facebookexternalhit

What's Changed

• Add PetalBot (and facebookexternalhit?) by @nisbet-hubbard in #22

Full Changelog: v1.10...v1.11

v1.10: adds Scrapy

Thank you to Brian at Skewray Research for calling this one out.

Full Changelog: v1.9...v1.10

v1.9: adds Timpibot

Full Changelog: v1.8...v1.9

v1.8: adds VelenPublicWebCrawler

Thank you to Brian at Skewray Research for calling this one out.

Full Changelog: v1.7...v1.8

v1.7: adds Meta-ExternalAgent

Many thanks to @nisbet-hubbard for raising this as well.

Full Changelog: v1.6...v1.7

v1.6: adds OAI-SearchBot to robots.txt

chore: add OAI-SearchBot

v1.5: drops AdsBot-Google; adds GoogleOther-Image/Video

Full Changelog: v1.4...v1.5

v1.4: add ImagesiftBot, remove peer39

What's Changed

• Adds ImagesiftBot (thanks @corbindavenport!)
• Removes peer39 bots as they are not used to train/related to AI
• Add reference by @glyn in #18

Full Changelog: v1.3...v1.4

chore: updates to bots table, readme

What's Changed

• More bot information and improved README by @corbindavenport in #17

New Contributors

• @corbindavenport made their first contribution in #17

Full Changelog: v1.2...v1.3