Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Metadata Support to Pluginspace #467

Merged
merged 50 commits into from
Jul 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
d66107d
NEW: move metadata support to plugin space
jcjgraf Feb 25, 2022
9de230d
Add convenience functions to determine metadata support
jcjgraf Jan 7, 2023
6f3d885
Fix missing renames from ExifHandler to MetadataHandler
jcjgraf Jan 7, 2023
9fb1155
Remove get_raw_md, cleanup and docstrings
jcjgraf Jan 7, 2023
26cf2b6
Improve metadata handler by operations.
jcjgraf Jan 8, 2023
50cb581
Rename `imutils.exif` to `imutils.metadata`
jcjgraf Jan 8, 2023
f0740e5
Refactor registry to make client subclass MetadataPlugin
jcjgraf Jan 8, 2023
824fe8e
Fix various mypy and linter issues
jcjgraf Jan 9, 2023
f2af1de
Fix pyexiv2 plugin to conform to refactoring
jcjgraf Jan 9, 2023
23c8fd1
Refactor MetadataPlugin to be an abstract class
jcjgraf Jan 9, 2023
2bd2ded
Improve: lazy import backend packages
jcjgraf Jan 9, 2023
e2e9c4a
Add display of metadata plugin status to version
jcjgraf Jan 9, 2023
446f4bd
Fix conftest for metadata redesign and test version
jcjgraf Jan 10, 2023
b610237
Make piexif / pyexiv2 lazy imports optional
karlch Jan 15, 2023
2d98a51
Temporarily shorten metadata (integration) test
karlch Jan 15, 2023
fd6f951
Remove metadata-related version unit tests
karlch Jan 15, 2023
6bc1742
Fix end2end test markers: exif -> metadata
karlch Jan 15, 2023
65fcb20
Add plugins_loaded signal to api.signals
karlch Jan 13, 2023
56c88df
Remove metadata information from --version
karlch Jan 15, 2023
8a93af6
Tests: rewrite metadata markers
karlch Jan 15, 2023
53e12e7
Update metadatawidget end2end test
karlch Jan 15, 2023
99d29ad
Fix bug in piexif's copy_metadata implementation
karlch Jan 15, 2023
01fa41d
Add metadata wrapper plugin
karlch Jan 15, 2023
1555d20
Extend metadata integration test and markers
karlch Jan 15, 2023
a0fcb6b
Fix lint / mypy
karlch Jan 15, 2023
4448981
Fix typo: iff -> if
karlch Jan 15, 2023
0aa4f8b
Change name/version property of MetadataPlugin to static method
jcjgraf Jan 20, 2023
b3206b8
Cleanup of supression of exception which is never raised
jcjgraf Jan 20, 2023
03fc90f
Import full `abc` for consistency reason
jcjgraf Jan 21, 2023
5507de9
Cleanup overcomplicated expression
jcjgraf Jan 21, 2023
1132667
Make all MetadataHandle methods raise exceptions
jcjgraf Jan 21, 2023
409c804
Change metadat_pyexiv to require full keys
jcjgraf Jan 21, 2023
89dd893
Assert earlier if piexif is available
jcjgraf Jan 22, 2023
aa6a80a
Make widget displaying images in the requested order
jcjgraf Jan 22, 2023
37d9809
Update docs to reflect metadata to plugin space transition
jcjgraf Jan 24, 2023
7b3e2ee
Fix piexif data not displayed in widget
jcjgraf Jan 24, 2023
1820813
Cleaup obsolete comments
jcjgraf Jun 24, 2023
a22982f
Remove broken link
jcjgraf Jun 24, 2023
fb66610
Ensure backward compatibility but allow overwriting
jcjgraf Jun 24, 2023
9266928
Tests: Auto-register metadata backend markers
karlch Jul 5, 2023
db95c4b
Fix typos
karlch Jul 5, 2023
1cbba4a
Refactor (metadata) plugin loading
karlch Jul 5, 2023
c3c1654
Improve subsection titles
jcjgraf Jul 5, 2023
2040326
Add note about default loading behaviour
jcjgraf Jul 5, 2023
b9fb5dd
Fix wrong url pointing to invalid exif page
jcjgraf Jul 5, 2023
1062817
Add url to metadata docs for missing metadata deps warning
jcjgraf Jul 5, 2023
2d16f89
Add link to gexiv2 user metadata plugin
jcjgraf Jul 5, 2023
ebfd9d6
Add note about enabling of multiple plugins
jcjgraf Jul 5, 2023
816dc32
Update changelog to reflect this PR
jcjgraf Jul 5, 2023
bff129b
Docs: minor wording changes
karlch Jul 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,11 @@ Changed:
supported by our testing framework, and 5.11 is out since July 2018. Code will likely
still work with these versions, but as it is no longer tested, there is no guarantee.
* The ``shuffle`` setting was moved into the ``sort`` group.
* Complete refactoring of metadata support. The handler functionality is moved out
to the plugin space, allowing for full flexibility in choosing a suitable backend. By
default, ``metadata_pyexiv2`` or ``metadata_piexif`` is loaded, if the respective
backend is installed. The default behaviour can be overridden by explicitly loading a
metadata plugin.

Fixed:
^^^^^^
Expand Down
66 changes: 0 additions & 66 deletions docs/documentation/exif.rst

This file was deleted.

2 changes: 1 addition & 1 deletion docs/documentation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The following documents are available:
getting_started
commands
configuration/index
exif
metadata
contributing
migrating

Expand Down
7 changes: 3 additions & 4 deletions docs/documentation/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,10 +129,9 @@ Dependencies
- QtSvg (optional for svg support)
* `PyQt5 <http://www.riverbankcomputing.com/software/pyqt/intro>`_ 5.11.3 or newer
* `setuptools <https://pypi.python.org/pypi/setuptools/>`_ (for installation)
* `pyexiv2 <https://python3-exiv2.readthedocs.io>`_ (optional for exif support)
* `piexif <https://pypi.org/project/piexif/>`_ (optional alternative for exif support)

.. include:: pyexiv2.rst
* `pyexiv2 <https://python3-exiv2.readthedocs.io>`_ (optional for metadata support)
* `piexif <https://pypi.org/project/piexif/>`_ (optional alternative for metadata
support)

Package Names For Distributions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
144 changes: 144 additions & 0 deletions docs/documentation/metadata.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
Metadata
========

Vimiv provides optional metadata support. If enabled, then:

#. The ``:metadata`` command and the corresponding ``i``-keybinding is available. It
displays the metadata of the current image.
#. The ``{exif-date-time}`` statusbar module is available. It displays the Exif
creation time of the current image.
#. Metadata is automatically copied from source to destination when writing images to
disk.

Vimiv provides full flexibility to users in terms of what metadata backend to use.
Each backend comes with their advantages and disadvantages, and each user has different
requirements, as well as different package support. Therefore, vimiv provides metadata
backends as independent plugins, that can be loaded as one desires. In addition, users
have the ability to extend vimiv's metadata capabilities using
:ref:`custom plugins<Plugins>`, as described in
:ref:`a later section<create_own_plugins>`.

Vimiv comes with two default plugins:

* ``metadata_piexif`` is based on `piexif`_
* ``metadata_pyexiv2`` is based on `pyexiv2`_

In addition, there are the following user metadata plugins available:

.. _user_metadata_plugins:

.. table:: Overview of user plugins
:widths: 20 80

===================================================================== ===========
Name Description
===================================================================== ===========
`metadata_gexiv2 <https://github.com/jcjgraf/vimiv_metadata-gexiv2>`_ Based on `gexiv2 <https://gitlab.gnome.org/GNOME/gexiv2>`_
===================================================================== ===========


Enable Support
--------------

Metadata plugins are loaded as any other vimiv plugin. To enable one of the default
plugins, simply list its name in the ``PLUGINS`` section of the configuration file. In
addition, you need to ensure that the required backend is installed on your system.

To enable a user metadata plugin, first you need to download it, and put it into the
plugin folder. Afterwards loading is equivalent to the default plugins.

For more information on how to load plugins, please refer to the
:ref:`plugin section<Plugins>`.

If no metadata plugin is specified, vimiv will load one of the two default plugins, if
the respective backend is installed. To disable this default behaviour, specify
``metadata = none`` in the ``PLUGINS`` section.

.. warning::

Default loading may be dropped in a future version. If you rely on metadata support,
we recommend to explicitly specify what backend you want.

.. note::
Multiple metadata plugins can be registered at the same time. If they use distinct
keys, the value of both is combined in the output of ``:metadata``.


Comparison of Different Plugins
-------------------------------

In short, ``metadata_pyexiv2`` is much more powerful than ``metadata_piexif``, though
the dependencies are also more involved to install.

.. table:: Comparison of the two libraries
:widths: 20 15 20 45

======================= =================== ==================== =====================================================================
PROPERTY ``metadata_piexif`` ``metadata_pyexiv2`` Note
======================= =================== ==================== =====================================================================
Backend `piexif`_ `pyexiv2`_
Exif Support True True pyexiv2 can potentially extract more data for the same image
ICMP Support False True
XMP Suppport False True
Output Formatting False True e.g. ``FNumber: 63/10`` vs ``FNumber: F6.3``
Supported File Types JPEG, TIFF Many common types
Ease of installation Simple More complicated pyexiv2 requires some dependencies including the C++ library `exiv2`_
======================= =================== ==================== =====================================================================

We recommend to use ``metadata_pyexiv2`` if the installation of `pyexiv2`_ is not too
involved on your system and ``metadata_piexif`` as a fallback solution or in case you
don't need the full power of `pyexiv2`_ and prefer something more lightweight.
Also consider the list of available
:ref:`user metadata plugins<user_metadata_plugins>`.


Customize Keysets
-----------------

You can configure the information displayed by the ``:metadata`` command by adding your
own key sets to the ``METADATA`` section in your configfile like this::

keys2 = someKey,anotherOne,lastOne
keys4 = newKey,oneMore

where the values must be a comma-separated list of valid metadata keys.

In case you are using `pyexiv2`_, you can find a complete overview of valid keys on the
`exiv2 webpage <https://www.exiv2.org/metadata.html>`_. You can choose any of the Exif,
IPTC, or XMP keys.

`Piexif`_ uses a simplified form of the key. It does not use the ``Group.Subgroup``
prefix, which is present in each of `pyexiv2`_'s keys. However, ``metadata_piexif``
automatically does this truncation, if the provided keys are in the long form.

The ``:metadata-list-keys`` command provides a list of all valid metadata keys, that
the currently loaded metadata plugins can read. This serves as an easy way to see what
keys are available in the image.


.. _create_own_plugins:

Create Own Plugins
------------------

One can extend vimiv's metadata capabilities by creating own metadata plugins. This is
useful if you want to use a different metadata backend.

The rough steps are the following:

#. Create a plugin, that implements the abstract class
``vimiv.imutils.metadata.MetadataPlugin``

#. Implement all required methods

#. Optionally, also implement the non-required methods

#. In the plugin's init function, register the plugin using
``vimiv.imutils.metadata.register``

Please see the default metadata plugins for an example implementation.


.. _exiv2: https://www.exiv2.org/index.html
.. _pyexiv2: https://python3-exiv2.readthedocs.io
.. _piexif: https://pypi.org/project/piexif/
3 changes: 0 additions & 3 deletions docs/documentation/pyexiv2.rst

This file was deleted.

4 changes: 2 additions & 2 deletions pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ faulthandler_timeout = 30
markers =
current: Mark tests during development
imageformats: Require retrieving images from the web to test additional formats
exif: Require exif support
metadata: Require metadata support
jcjgraf marked this conversation as resolved.
Show resolved Hide resolved
piexif: Require piexif
pyexiv2: Require pyexiv2
noexif: Requires exif support NOT to be available
nometadata: Requires metadata support NOT to be available
ci: Run test only on ci
ci_skip: Skip test on ci
56 changes: 22 additions & 34 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,38 +12,36 @@

import pytest

from vimiv.imutils import exif
from vimiv.plugins import metadata_pyexiv2, metadata_piexif


CI = "CI" in os.environ

HAS_PIEXIF = metadata_piexif.piexif is not None
HAS_PYEXIV2 = metadata_pyexiv2.pyexiv2 is not None
HAS_METADATA = HAS_PIEXIF or HAS_PYEXIV2

# fmt: off
PLATFORM_MARKERS = (
("ci", CI, "Only run on ci"),
("ci_skip", not CI, "Skipped on ci"),
)

EXIF_MARKERS = (
("exif", exif.has_exif_support, "Only run with exif support"),
("noexif", not exif.has_exif_support, "Only run without exif support"),
("pyexiv2", exif.pyexiv2 is not None, "Only run with pyexiv2"),
("piexif", exif.piexif is not None, "Only run with piexif"),
METADATA_MARKERS = (
("metadata", HAS_METADATA, "Only run with metadata support"),
("nometadata", not HAS_METADATA, "Only run without metadata support"),
("piexif", HAS_PIEXIF, "Only run with piexif"),
("pyexiv2", HAS_PYEXIV2, "Only run with pyexiv2"),
)
# fmt: on


def apply_platform_markers(item):
"""Apply markers that skip tests depending on the current platform."""
apply_markers_helper(item, PLATFORM_MARKERS)


def apply_exif_markers(item):
"""Apply markers that skip tests depending on specific exif support."""
if os.path.basename(item.fspath) in ("test_exif.py",):
for marker_name in "exif", "pyexiv2", "piexif":
marker = getattr(pytest.mark, marker_name)
def apply_fixture_markers(item, *names):
"""Helper function to mark all tests using specific fixtures with that mark."""
for name in names:
marker = getattr(pytest.mark, name)
if name in item.fixturenames:
item.add_marker(marker)
apply_markers_helper(item, EXIF_MARKERS)


def apply_markers_helper(item, markers):
Expand All @@ -60,8 +58,9 @@ def apply_markers_helper(item, markers):
def pytest_collection_modifyitems(items):
"""Handle custom markers via pytest hook."""
for item in items:
apply_platform_markers(item)
apply_exif_markers(item)
apply_fixture_markers(item, "piexif", "pyexiv2")
apply_markers_helper(item, PLATFORM_MARKERS)
apply_markers_helper(item, METADATA_MARKERS)


@pytest.fixture
Expand Down Expand Up @@ -165,28 +164,17 @@ def tmpdir():
raise ValueError("Use the 'tmp_path' fixture instead of 'tmpdir'")


@pytest.fixture()
def piexif(monkeypatch):
"""Pytest fixture to ensure only piexif is available."""
monkeypatch.setattr(exif, "pyexiv2", None)


@pytest.fixture()
def noexif(monkeypatch, piexif):
"""Pytest fixture to ensure no exif library is available."""
monkeypatch.setattr(exif, "piexif", None)


@pytest.fixture()
def add_exif_information():
"""Fixture to retrieve a helper function that adds exif content to an image."""

def add_exif_information_impl(path: str, content):
assert exif.piexif is not None, "piexif required to add exif information"
exif_dict = exif.piexif.load(path)
import piexif

exif_dict = piexif.load(path)
for ifd, ifd_dict in content.items():
for key, value in ifd_dict.items():
exif_dict[ifd][key] = value
exif.piexif.insert(exif.piexif.dump(exif_dict), path)
piexif.insert(piexif.dump(exif_dict), path)

return add_exif_information_impl
1 change: 1 addition & 0 deletions tests/end2end/features/image/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

@pytest.fixture()
def exif_content():
assert piexif is not None, "piexif required create exif information."
return {
"0th": {
piexif.ImageIFD.Make: b"vimiv-testsuite",
Expand Down
2 changes: 1 addition & 1 deletion tests/end2end/features/image/metadata.feature
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
@exif
@metadata
Feature: Metadata widget displaying image exif information

Scenario: Show metadata widget
Expand Down
Loading