Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge PR 123 from develop into master #124

Merged
merged 5 commits into from
Dec 17, 2024
Merged

Merge PR 123 from develop into master #124

merged 5 commits into from
Dec 17, 2024

Conversation

Criamos
Copy link
Contributor

@Criamos Criamos commented Dec 17, 2024

This PR merges #123 into master, which includes:

  • an update for the headless browser (to browserless v2.24), to guarantee compatibility with playwright v1.49
  • dependency updates (trafilatura v2)
  • crawler update: planet_n_spider v0.0.3

- change: ignore "og:image"-thumbnails and take website-screenshots for each item instead (as discussed with Jan on 2024-12-11)
- feat: mapping from Planet-N's "class_list" WP-JSON property to our "new_lrt"- and "discipline"-Vocabs
- change: all items are considered teaching modules ("Unterrichtsbaustein") by default
- feat: set default license to CUSTOM with Planet-N's description text
  - the license description can be found at https://www.planet-n.de/info/
- LomBase used a default "root"-logger, which made it quite hard to understand where logging messages came from
- by using loguru, individual logging messages are much easier to traceback to individual lines of code
Update headless browser and planet_n_spider v0.0.3
@Criamos Criamos self-assigned this Dec 17, 2024
@Criamos Criamos added the dependencies Pull requests that update a dependency file label Dec 17, 2024
@Criamos Criamos merged commit 7318bd9 into master Dec 17, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant