Installation: cannot import name 'Mapping' from 'collections'

[keithkhl]

Expected behaviour

No error in installation

Actual behaviour

Below error msg

Installing frappe_s3_attachment...
frappe.integrations.doctype.s3_backup_settings.s3_backup_settings.take_backups_daily is not a valid method: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/init.py)
frappe.integrations.doctype.s3_backup_settings.s3_backup_settings.take_backups_weekly is not a valid method: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/init.py)
frappe.integrations.doctype.s3_backup_settings.s3_backup_settings.take_backups_monthly is not a valid method: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/init.py)
Updating Dashboard for frappe_s3_attachment

Step to reproduce

bench get-app https://github.com/zerodhatech/Frappe-attachments-s3.git
bench --site sitename install-app frappe_s3_attachment

While installation, I saw above error

Version

master

Comment

I guess it is only backup related issue that was Frappe's native support for S3 backup, but just want to raise your attention, in case I missed something.

[keithkhl]

From the Frappe website, Integrations -> S3 Backup Settings, I see below error msg from a pop-up

App Versions

{
	"drive": "0.0.11",
	"erpnext": "15.38.4",
	"frappe": "15.44.2",
	"frappe_s3_attachment": "0.0.1",
	"hrms": "15.32.2",
	"lms": "2.8.0",
	"payments": "0.0.1",
	"print_designer": "1.0.0-dev",
	"raven": "2.0.0",
	"wiki": "2.0.0"
}

Route

Workspaces/Integrations

Traceback

Traceback (most recent call last):
  File "apps/frappe/frappe/modules/utils.py", line 254, in load_doctype_module
    doctype_python_modules[key] = frappe.get_module(module_name)
  File "apps/frappe/frappe/__init__.py", line 1483, in get_module
    return importlib.import_module(modulename)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "apps/frappe/frappe/integrations/doctype/s3_backup_settings/s3_backup_settings.py", line 6, in <module>
    import boto3
  File "env/lib/python3.10/site-packages/boto3/__init__.py", line 16, in <module>
    from boto3.session import Session
  File "env/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "env/lib/python3.10/site-packages/botocore/session.py", line 25, in <module>
    import botocore.configloader
  File "env/lib/python3.10/site-packages/botocore/configloader.py", line 19, in <module>
    from botocore.compat import six
  File "env/lib/python3.10/site-packages/botocore/compat.py", line 23, in <module>
    from botocore.exceptions import MD5UnavailableError
  File "env/lib/python3.10/site-packages/botocore/exceptions.py", line 15, in <module>
    from botocore.vendored.requests.exceptions import ConnectionError
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/__init__.py", line 58, in <module>
    from . import utils
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/utils.py", line 26, in <module>
    from .compat import parse_http_list as _parse_list_header
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/compat.py", line 7, in <module>
    from .packages import chardet
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/packages/__init__.py", line 3, in <module>
    from . import urllib3
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/__init__.py", line 10, in <module>
    from .connectionpool import (
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 38, in <module>
    from .response import HTTPResponse
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/response.py", line 9, in <module>
    from ._collections import HTTPHeaderDict
  File "env/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py", line 1, in <module>
    from collections import Mapping, MutableMapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "apps/frappe/frappe/app.py", line 114, in application
    response = frappe.api.handle(request)
  File "apps/frappe/frappe/api/__init__.py", line 49, in handle
    data = endpoint(**arguments)
  File "apps/frappe/frappe/api/v1.py", line 36, in handle_rpc_call
    return frappe.handler.handle()
  File "apps/frappe/frappe/handler.py", line 49, in handle
    data = execute_cmd(cmd)
  File "apps/frappe/frappe/handler.py", line 85, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "apps/frappe/frappe/__init__.py", line 1775, in call
    return fn(*args, **newargs)
  File "apps/frappe/frappe/utils/typing_validations.py", line 31, in wrapper
    return func(*args, **kwargs)
  File "apps/frappe/frappe/desk/form/load.py", line 72, in getdoctype
    docs = get_meta_bundle(doctype)
  File "apps/frappe/frappe/desk/form/load.py", line 83, in get_meta_bundle
    bundle = [frappe.desk.form.meta.get_meta(doctype)]
  File "apps/frappe/frappe/desk/form/meta.py", line 42, in get_meta
    meta = FormMeta(doctype, cached=False)
  File "apps/frappe/frappe/desk/form/meta.py", line 53, in __init__
    self.load_assets()
  File "apps/frappe/frappe/desk/form/meta.py", line 67, in load_assets
    self.load_templates()
  File "apps/frappe/frappe/desk/form/meta.py", line 248, in load_templates
    module = load_doctype_module(self.name)
  File "apps/frappe/frappe/modules/utils.py", line 258, in load_doctype_module
    raise ImportError(msg) from e
ImportError: Module import failed for S3 Backup Settings, the DocType you're trying to open might be deleted.
Error: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

Request Data

{
	"type": "GET",
	"args": {
		"doctype": "S3 Backup Settings",
		"with_parent": 1,
		"cached_timestamp": null
	},
	"headers": {},
	"error_handlers": {},
	"url": "/api/method/frappe.desk.form.load.getdoctype",
	"request_id": null
}

Response Data

{
	"exception": "Error: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)",
	"exc_type": "ImportError"
}

[yarin-zhang]

When installing Frappe S3 Attachment in a Python 3.10+ environment, you may encounter the following error: ImportError: cannot import name 'Mapping' from 'collections'. This issue arises because, starting with Python 3.3+, the Mapping and MutableMapping classes have been moved from collections to collections.abc.

Solution

1. Update Dependencies

First, upgrade boto3 and botocore to the latest versions:

bench pip uninstall boto3 botocore -y
bench pip install boto3 botocore --upgrade

2. Create a Patch File

Set up the necessary file structure within the app directory:

cd apps/frappe_s3_attachment
mkdir -p frappe_s3_attachment/patches
touch frappe_s3_attachment/patches/__init__.py

Then, create the patch file:

from __future__ import unicode_literals
import frappe

def execute():
    """
    This patch updates the collections import to use collections.abc
    """
    try:
        from collections.abc import Mapping, MutableMapping
    except ImportError:
        from collections import Mapping, MutableMapping

3. Add the Patch Record

Update or create patches.txt:

frappe_s3_attachment.patches.fix_collections_import

4. Update requirements.txt

Edit requirements.txt to remove version restrictions on outdated dependencies:

boto3
botocore
python-magic==0.4.18

5. Run Migrations and Restart

bench migrate
bench restart

At least for me, this solution works.

[yarin-zhang]

I think this plugin is a bit old and seems to be poorly maintained and may have many issues.

I saw that the plug-in listed in Frappe cloud is this one.

https://frappecloud.com/marketplace/apps/frappe_s3_attachment

https://github.com/shridarpatil/Frappe-attachments-s3

It is a fork of this repository and has made some adaptations. It shows that it supports frappe 15.

And this one.

https://frappecloud.com/marketplace/apps/dfp_external_storage

https://github.com/developmentforpeople/dfp_external_storage

I suggest you try those first instead of trying to improve this project.