Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract more metadata using exifTool #2645

Merged
merged 5 commits into from
Feb 5, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions meshroom/nodes/aliceVision/ExtractMetadata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
__version__ = "0.1"

from meshroom.core import desc
from meshroom.core.utils import VERBOSE_LEVEL
from pathlib import Path

import pyalicevision as av

import distutils.dir_util as du
import shutil
import glob
import os
import subprocess

Check warning on line 13 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L9-L13

Added lines #L9 - L13 were not covered by tests


class ExtractMetadata(desc.Node):
size = desc.DynamicNodeSize("input")

Check warning on line 17 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L16-L17

Added lines #L16 - L17 were not covered by tests

category = 'Utils'
documentation = '''

Check warning on line 20 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L19-L20

Added lines #L19 - L20 were not covered by tests
Using exifTool, this node extracts metadata of all images referenced in a sfmData and store them in appropriate files.
'''

inputs = [

Check warning on line 24 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L24

Added line #L24 was not covered by tests
desc.File(
name="input",
label="Input",
description="SfMData file input.",
value="",
),
desc.BoolParam(
name="keepFilename",
label="Keep Filename",
description="Keep the filename of the inputs for the outputs.",
value=False,
),
desc.ChoiceParam(
name="extension",
label="Output File Extension",
description="Metadata file extension.",
value="txt",
values=["txt", "xml", "xmp"],
exclusive=True,
),
desc.StringParam(
name="arguments",
label="Arguments",
description="ExifTool command arguments",
value="",
),
desc.BoolParam(
name="insertInSfm",
label="Update sfmData",
description="Insert the extracted metadata in the sfmData file.",
value=False,
),
desc.ChoiceParam(
name="verboseLevel",
label="Verbose Level",
description="Verbosity level (fatal, error, warning, info, debug, trace).",
values=VERBOSE_LEVEL,
value="info",
),
]

outputs = [

Check warning on line 66 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L66

Added line #L66 was not covered by tests
desc.File(
name="output",
label="Result Folder",
description="Output path for the resulting metadata files.",
value=desc.Node.internalFolder,
),
]

def processChunk(self, chunk):
try:
chunk.logManager.start(chunk.node.verboseLevel.value)

Check warning on line 77 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L75-L77

Added lines #L75 - L77 were not covered by tests

if chunk.node.input.value == "" or chunk.node.input.value[-4:].lower() != '.sfm':
error = 'This node need to have a sfmData connected as input.'
chunk.logger.error(error)
raise RuntimeError(error)

Check warning on line 82 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L79-L82

Added lines #L79 - L82 were not covered by tests

if not os.path.exists(chunk.node.output.value):
os.mkdir(chunk.node.output.value)

Check warning on line 85 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L84-L85

Added lines #L84 - L85 were not covered by tests

dataAV = av.sfmData.SfMData()
if av.sfmDataIO.load(dataAV, chunk.node.input.value, av.sfmDataIO.ALL):
views = dataAV.getViews()
for id, v in views.items():
inputFile = v.getImage().getImagePath()
chunk.logger.info(f"Processing {inputFile}")

Check warning on line 92 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L87-L92

Added lines #L87 - L92 were not covered by tests

if chunk.node.keepFilename.value:
outputMetadataFilename = os.path.join(chunk.node.output.value, Path(inputFile).stem + "." + chunk.node.extension.value)

Check warning on line 95 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L94-L95

Added lines #L94 - L95 were not covered by tests
else:
outputMetadataFilename = os.path.join(chunk.node.output.value, str(id) + "." + chunk.node.extension.value)

Check warning on line 97 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L97

Added line #L97 was not covered by tests

if chunk.node.extension.value == 'txt':
cmd = 'exiftool ' + chunk.node.arguments.value.strip() + ' ' + inputFile + ' > ' + outputMetadataFilename
elif chunk.node.extension.value == 'xml':
cmd = 'exiftool -X ' + chunk.node.arguments.value.strip() + ' ' + inputFile + ' > ' + outputMetadataFilename

Check warning on line 102 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L99-L102

Added lines #L99 - L102 were not covered by tests
else: #xmp
cmd = 'exiftool -tagsfromfile ' + inputFile + ' ' + chunk.node.arguments.value.strip() + ' ' + outputMetadataFilename

Check failure on line 104 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

codefactor.io / CodeFactor

meshroom/nodes/aliceVision/ExtractMetadata.py#L104

subprocess call with shell=True identified, security issue. (B602)

Check warning on line 104 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L104

Added line #L104 was not covered by tests

chunk.logger.debug(cmd)
error = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stderr.read().decode()

Check warning on line 107 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L106-L107

Added lines #L106 - L107 were not covered by tests

chunk.logger.debug(error)

Check warning on line 109 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L109

Added line #L109 was not covered by tests

if error != "":
chunk.logger.error(error)
raise RuntimeError(error)
if not os.path.exists(outputMetadataFilename):

Check failure on line 114 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

codefactor.io / CodeFactor

meshroom/nodes/aliceVision/ExtractMetadata.py#L114

subprocess call with shell=True identified, security issue. (B602)
info = 'No metadata extracted for file ' + inputFile
chunk.logger.info(info)
elif chunk.node.insertInSfm.value:
cmd = 'exiftool ' + chunk.node.arguments.value.strip() + ' ' + inputFile

Check notice on line 118 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

codefactor.io / CodeFactor

meshroom/nodes/aliceVision/ExtractMetadata.py#L118

Ambiguous variable name 'l'. (E741)
metadata = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.read().decode()
chunk.logger.debug(metadata)
lmeta = metadata.split('\n')
for i in range(1, len(lmeta)-1):
l = lmeta[i].split(':', 1)
v.getImageInfo().addMetadata('ExifTool:'+l[0].strip(), l[1].strip())

Check warning on line 124 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L111-L124

Added lines #L111 - L124 were not covered by tests

if chunk.node.insertInSfm.value:
outputSfm = os.path.join(chunk.node.output.value, Path(chunk.node.input.value).stem + ".sfm")
av.sfmDataIO.save(dataAV, outputSfm, av.sfmDataIO.ALL)

Check warning on line 128 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L126-L128

Added lines #L126 - L128 were not covered by tests

chunk.logger.info('Metadata extraction end')

Check warning on line 130 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L130

Added line #L130 was not covered by tests

finally:
chunk.logManager.end()

Check warning on line 133 in meshroom/nodes/aliceVision/ExtractMetadata.py

View check run for this annotation

Codecov / codecov/patch

meshroom/nodes/aliceVision/ExtractMetadata.py#L133

Added line #L133 was not covered by tests