Spidering Sources: Capture errors we think are Not Our Fault

[odscjames]

https://tameside-openactive.legendonlineservices.co.uk/OpenActive throws a 500 error.

http://data.tabletennis365.com/ has no DNS entry.

This raises the issue of how we log errors like this.

[odscjames]

New table: `spider_data_catalog_errors'

• URL - URL error occurred on
• Error - the error
• At - time error occurred
• From (or better name) - Basically, a message saying why we were looking at this URL. What data catalog did it come from?

We can hold a lot of data in table; I'd suggest just starting with logging all errors. We can add something later to clean out old errors automatically

[robredpath]

Capturing errors during spidering