Crawler outputs thing ids to screen but does not put anything into the summary.csv

[rtho782]

As per title, the summary CSV is created with the column headers in:

thing_id, file_id, file, license, link

But no other output is generated.

The terminal displays the thing IDs, there are no errors displayed.

[rtho782]

Changing the following section:

def get_thing(thing_id): base_url = "https://www.thingiverse.com/{}:{}" file_ids = [] url = base_url.format("thing", thing_id) contents = get_url(url).text license = parse_license(contents) return license, parse_file_ids(contents)

As follows:

def get_thing(thing_id): base_url = "https://www.thingiverse.com/{}:{}/files" file_ids = [] url = base_url.format("thing", thing_id) contents = get_url(url).text license = parse_license(contents) return license, parse_file_ids(contents)

seems to resolve this?

[rtho782]

The page it is trying to parse to find download links by default doesn't have any matching strings. This fixes it but means it is looking for the individual files rather than the zipped files.