Error scraping mastodon-profile

[Almex-codes]

Describe the bug

An AttributeError is given when scraping for mastodon profile data.

How to reproduce

snscrape mastodon-profile @[email protected]

Expected behaviour

Receiving profile data.

Screenshots and recordings

No response

Operating system

Ubuntu 20.04.

Python version: output of python3 --version

python 3.10.12

snscrape version: output of snscrape --version

snscrape 0.7.0.20230622

Scraper

mastodon-profile

How are you using snscrape?

CLI (snscrape ... as a command, e.g. in a terminal)

Backtrace

AttributeError: 'NoneType' object has no attribute 'find_all'

Log output

2023-09-24 19:51:37.739 INFO snscrape.base Retrieving https://indieweb.social/@daphne/with_replies
2023-09-24 19:51:38.048 INFO snscrape.base Retrieved https://indieweb.social/@daphne/with_replies: 200
2023-09-24 19:51:38.105 CRITICAL snscrape._cli Dumped stack and locals to /tmp/snscrape_locals_wvz4dp16
Traceback (most recent call last):
File "/home/alt/.local/bin/snscrape", line 8, in
sys.exit(main())
File "/home/alt/.local/lib/python3.10/site-packages/snscrape/cli.py", line 323, in main
for i, item in enumerate(scraper.get_items(), start = 1):
File "/home/alt/.local/lib/python3.10/site-packages/snscrape/modules/mastodon.py", line 280, in get_items
yield from self.entries_to_items(soup.find('div', class = 'activity-stream').find_all('div', class = 'entry'), r.url)
AttributeError: 'NoneType' object has no attribute 'find_all'

Dump of locals

No response

Additional context

API still works like a charm, so maybe the data structure was changed. But no idea how snscrape actually works regarding Mastodon. Anyhow, keeping the scrapibility of mastodon intact could be become essential.

[JustAnotherArchivist]

#587