Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jjjake/internetarchive
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v2.1.0
Choose a base ref
...
head repository: jjjake/internetarchive
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Loading
Showing with 6,193 additions and 4,101 deletions.
  1. +13 −0 .github/dependabot.yml
  2. +18 −0 .github/workflows/lint_python.yml
  3. +20 −0 .github/workflows/pre-commit.yml
  4. +16 −0 .github/workflows/test_install.yml
  5. +18 −0 .github/workflows/tox.yml
  6. +24 −20 .gitignore
  7. +59 −0 .pre-commit-config.yaml
  8. +0 −15 .travis.yml
  9. +1 −0 AUTHORS.rst
  10. +3 −3 CONTRIBUTING.rst
  11. +349 −60 HISTORY.rst
  12. +7 −7 Makefile
  13. +18 −10 README.rst
  14. +4 −3 docs/requirements.txt
  15. +3 −3 docs/source/api.rst
  16. +48 −33 docs/source/cli.rst
  17. +8 −9 docs/source/conf.py
  18. +15 −2 docs/source/index.rst
  19. +47 −78 docs/source/installation.rst
  20. +4 −4 docs/source/jq.rst
  21. +3 −3 docs/source/metadata.rst
  22. +2 −2 docs/source/parallel.rst
  23. +55 −35 docs/source/quickstart.rst
  24. +13 −22 docs/source/troubleshooting.rst
  25. +33 −35 internetarchive/__init__.py
  26. +1 −0 internetarchive/__version__.py
  27. +269 −0 internetarchive/account.py
  28. +323 −338 internetarchive/api.py
  29. +9 −7 internetarchive/auth.py
  30. +85 −102 internetarchive/catalog.py
  31. +32 −17 internetarchive/cli/__init__.py
  32. +0 −78 internetarchive/cli/argparser.py
  33. +210 −0 internetarchive/cli/cli_utils.py
  34. +123 −155 internetarchive/cli/ia.py
  35. +110 −0 internetarchive/cli/ia_account.py
  36. +124 −62 internetarchive/cli/ia_configure.py
  37. +124 −86 internetarchive/cli/ia_copy.py
  38. +137 −112 internetarchive/cli/ia_delete.py
  39. +176 −164 internetarchive/cli/ia_download.py
  40. +115 −53 internetarchive/cli/ia_list.py
  41. +231 −203 internetarchive/cli/ia_metadata.py
  42. +68 −54 internetarchive/cli/ia_move.py
  43. +81 −33 internetarchive/cli/ia_reviews.py
  44. +172 −86 internetarchive/cli/ia_search.py
  45. +140 −132 internetarchive/cli/ia_tasks.py
  46. +290 −224 internetarchive/cli/ia_upload.py
  47. 0 internetarchive/cli/py.typed
  48. +31 −34 internetarchive/config.py
  49. +21 −4 internetarchive/exceptions.py
  50. +247 −153 internetarchive/files.py
  51. +335 −324 internetarchive/iarequest.py
  52. +518 −454 internetarchive/item.py
  53. 0 internetarchive/py.typed
  54. +72 −48 internetarchive/search.py
  55. +217 −222 internetarchive/session.py
  56. +167 −79 internetarchive/utils.py
  57. +3 −5 pex-requirements.txt
  58. +87 −0 pyproject.toml
  59. +88 −14 setup.cfg
  60. +1 −54 setup.py
  61. +3 −4 tests/cli/{test_argparser.py → test_cli_utils.py}
  62. +2 −8 tests/cli/test_ia.py
  63. +69 −24 tests/cli/test_ia_download.py
  64. +18 −26 tests/cli/test_ia_list.py
  65. +7 −9 tests/cli/test_ia_metadata.py
  66. +18 −24 tests/cli/test_ia_search.py
  67. +287 −40 tests/cli/test_ia_upload.py
  68. +19 −30 tests/conftest.py
  69. +11 −11 tests/data/s3_status_check.json
  70. +1 −1 tests/data/scrape_response.json
  71. +3 −4 tests/requirements.txt
  72. +75 −62 tests/test_api.py
  73. +30 −39 tests/test_config.py
  74. +1 −1 tests/test_exceptions.py
  75. +208 −129 tests/test_item.py
  76. +30 −12 tests/test_session.py
  77. +21 −16 tests/test_utils.py
  78. +2 −20 tox.ini
13 changes: 13 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Keep GitHub Actions up to date with GitHub's Dependabot...
# https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot
# https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem
version: 2
updates:
- package-ecosystem: github-actions
directory: /
groups:
github-actions:
patterns:
- "*" # Group all Actions updates into a single larger pull request
schedule:
interval: weekly
18 changes: 18 additions & 0 deletions .github/workflows/lint_python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: lint_python
on: [pull_request, push]
jobs:
lint_python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
cache: pip
python-version: 3.x
- run: pip install --upgrade pip setuptools wheel
- run: pip install .[all]
- run: black --check --skip-string-normalization . || true
- run: ruff check --output-format=github # See pyproject.toml for configuration
- run: pip install -r pex-requirements.txt -r tests/requirements.txt
- run: mypy . # See setup.cfg for configuration
- run: safety check || true # Temporary fix for https://pyup.io/v/51457/f17
20 changes: 20 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# https://pre-commit.com
# This GitHub Action assumes that the repo contains a valid .pre-commit-config.yaml file.
name: pre-commit
on:
pull_request:
push:
branches: [master]
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.x
cache: pip
- run: pip install pre-commit
- run: pre-commit --version
- run: pre-commit install
- run: pre-commit run --all-files
16 changes: 16 additions & 0 deletions .github/workflows/test_install.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: test_install
on:
pull_request:
push:
branches: [master]
jobs:
install_internetarchive:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
setuptools-version: ["45.2.0", "58.1.0", "62.4.0"]
steps:
- uses: actions/checkout@v4
- run: pip install setuptools=="${{ matrix.setuptools-version }}"
- run: pip install .
18 changes: 18 additions & 0 deletions .github/workflows/tox.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: tox
on: [push, pull_request]
jobs:
tox:
runs-on: ubuntu-latest
strategy:
fail-fast: false
max-parallel: 1 # Avoid timeout errors
matrix:
python: ['3.9', '3.10', '3.11', '3.12', '3.13', 'pypy-3.10']
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
cache: pip
- run: pip install tox
- run: tox -e py
44 changes: 24 additions & 20 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,28 +1,32 @@
.cache
.coverage
*.pyc
build
dist
*.egg-info
stairs
itemlist.txt
.DS_Store
.envrc
.idea
.pytest_cache/
.python-version
.tox
TAGS
.vagrant
.venv*
.vscode
*.csv
htmlcov
*.egg-info
*.log
*.pex
pex/
wheelhouse
*.pyc
*gz
.venv*
.cache
.vagrant
.idea
v2/
v3.6/
v3.7/
.pytest_cache/
.python-version
build
dist
htmlcov
itemlist.txt
pex/
stairs
TAGS
trash/
.vscode
v3.10/
v3.7/
v3.8/
v3.9/
wheelhouse
ia.dist/
ia.bin
59 changes: 59 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# To enable these pre-commit hook run:
# `python3 -m pip install pre-commit` or `brew install pre-commit`
# Then in the project root directory run `pre-commit install`
# Learn more about this config here: https://pre-commit.com/

# default_language_version:
# python: python3.10

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: check-builtin-literals
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-yaml
- id: detect-private-key
- id: end-of-file-fixer
- id: mixed-line-ending
- id: requirements-txt-fixer
- id: trailing-whitespace

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.8.5

hooks:
- id: ruff
# args: [ --fix, --exit-non-zero-on-fix ]

- repo: https://github.com/psf/black
rev: 24.10.0
hooks:
- id: black
language_version: python3
args:
- --diff
- --skip-string-normalization

- repo: https://github.com/codespell-project/codespell
rev: v2.3.0
hooks:
- id: codespell # See setup.cfg for args

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.14.1
hooks:
- id: mypy
additional_dependencies:
- tqdm-stubs
- types-jsonpatch
- types-requests
- types-setuptools
- types-ujson
- types-urllib3

- repo: https://github.com/asottile/setup-cfg-fmt
rev: v2.7.0
hooks:
- id: setup-cfg-fmt
15 changes: 0 additions & 15 deletions .travis.yml

This file was deleted.

1 change: 1 addition & 0 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
@@ -13,6 +13,7 @@ Contributors

- Bryce Drennan <internetarchive@brycedrennan.com>
- Max Zettlmeißl <max@zettlmeissl.de>
- Ian Tait <thetaiter@gmail.com>

Patches and Suggestions
-----------------------
6 changes: 3 additions & 3 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@ All pull requests and patches are welcome, but please consider the following:
- Include tests.
- Include documentation for new features.
- If your patch is supposed to fix a bug, please describe in as much detail as possible the circumstances in which the bug happens.
- Please follow `PEP8 <http://legacy.python.org/dev/peps/pep-0008/>`_, with the exception of what is ignored in `setup.cfg <https://github.com/jjjake/internetarchive/blob/master/setup.cfg>`_. PEP8 compliancy is checked when tests run. Tests will fail if your patch is not PEP8 compliant.
- Please follow `PEP8 <http://legacy.python.org/dev/peps/pep-0008/>`_, with the exception of what is ignored in `setup.cfg <https://github.com/jjjake/internetarchive/blob/master/setup.cfg>`_. PEP8 compliance is checked when tests run. Tests will fail if your patch is not PEP8 compliant.
- Add yourself to AUTHORS.rst.
- Avoid introducing new dependencies.
- Open an issue if a relevant one is not already open, so others have visibility into what you're working on and efforts aren't duplicated.
@@ -37,7 +37,7 @@ Running Tests
The minimal requirements for running tests are ``pytest``, ``pytest-pep8`` and ``responses``:

.. code:: bash
$ pip install pytest pytest-pep8 responses
Clone the `internetarchive lib <https://github.com/jjjake/internetarchive>`_:
@@ -68,4 +68,4 @@ To test against all supported Python versions, first make sure you have all of t
$ pip install tox
$ tox
Even easier is simply creating a pull request. `Travis <https://travis-ci.com/>`_ is used for continuous integration, and is set up to run the full testsuite whenever a pull request is submitted or updated.
Even easier is simply creating a pull request. `GitHub Actions <https://docs.github.com/en/actions>`_ are used for continuous integration, and are set up to run the full testsuite whenever a pull request is submitted or updated.
Loading