Download Raw data - handle HTTP errors

[odscjames]

From #18 (comment)

A feed page returns a status of anything other than a 200, 429, or 503 - feed is in an inconsistent state, and is displaying symptoms of an error that should be fixed. Suggest harvesting is restarted from the beginning of the feed the following day, when the errors may have been resolved. If this happens ideally an error would be shown on the status page saying e.g. "Harvesting halted at {pageUrl}, which returned status code {statusCode}".

[odscjames]

See #40 . That will log the error. Next time we scrape we will try again.

But is there any reason you say "from the beginning of the feed "? Normally we always try and resume a scrape from where we left off.

[nickevansuk]

If the feed is returning something other than 200, 429, or 503 there are a number of things that could have happened to it (e.g. software reinstallation) - best to start from the beginning to ensure the feed is represented accurately, as previously harvested records are likely irrelevant

[robredpath]

Handle HTTP errors encountered during harvesting