Harmonize image file naming between full-text imports to Wikisource and file import to Commons

[Daniel-Mietchen]

Both
https://github.com/konrad/JATS-to-Mediawiki
and
https://github.com/erlehmann/open-access-media-importer
generate file names for images and supplementary files, but the rules used therein are not fully compatible.

Part of #7

[notconfusing]

@DCoetzee do you have any advice for this? Basically, there is one Bot called "Open Access Media Importer" that generates media on Commons, with one naming scheme. We are creating a new bot "Citation Bot" that may upload pwhat would duplicates of what OAMI does. However, it would be nice if the naming conventions could be moved to suit Citation bot rather than OAMI, since Citation bot is uploading journal articles to Wikisource which are expecting a specific file naming convention, whereas whatever is using OAMI-uploaded files are not name sensitive. Moving files however requires admin. What to do?

[Daniel-Mietchen]

The issue of duplicates is not so much between OAMI (which uploads only audio and video files) and the Citation bot (which will mainly do images), but between Citation bot's uploads and images that have already been uploaded manually by Commons users.

To give an idea of the scale that we are talking about, there are currently ca. 25k files from Open Access scholarly sources (cf. http://tools.wmflabs.org/glamtools/glamorous.php?doit=1&category=Open+access+%28publishing%29&use_globalusage=1&ns0=1&depth=5 ), of which 16k are audio/ video (basically all from OAMI): http://tools.wmflabs.org/glamtools/glamorous.php?doit=1&category=Uploaded+with+Open+Access+Media+Importer&use_globalusage=1&ns0=1 . This means there are ca. 9k image files that have been uploaded manually.

With Citation bot, we are planning - in the long run - to upload thousands of full-text articles onto Wikisource and their respective images to Commons, of which a fair share will be amongst the 9k (or more by then) that are already there, so we will have to detect duplicates in one way or another - either before upload to avoid duplicates, or after upload to mark duplicates for deletion. The problem is that there is no good way to capture duplicates with all the possible modifications that may have been made on the way (examples in https://meta.wikimedia.org/wiki/Research:Committee/Areas_of_interest/Open-access_policy/Request_for_Information_on_Public_Access_to_Peer-Reviewed_Scholarly_Publications_Resulting_From_Federally_Funded_Research#Additional_comments ).

This issue #8 is about harmonizing the way that OAMI and Citation bot name the image and media files they upload.

[Daniel-Mietchen]

Reopening, assuming that @notconfusing had just hit the wrong button when closing.

[notconfusing]

@Daniel-Mietchen, I copied the way that OAMI does naming at the moment, and re-uploaded our 11 article test-set test. I looks like it works. So we have a prototype for this now. Can always be changes.

[Daniel-Mietchen]

Some examples of duplicate images:
https://commons.wikimedia.org/w/index.php?title=File:Tracking-Marsupial-Evolution-Using-Archaic-Genomic-Retroposon-Insertions-pbio.1000436.g002.jpg&oldid=126539861
is a duplicate of
https://commons.wikimedia.org/w/index.php?title=File:Phylogenetic_tree_of_marsupials_derived_from_retroposon_data_-_journal.pbio.1000436.g002.png&oldid=124478813 ,
and
https://commons.wikimedia.org/w/index.php?title=File:Taxonomic-revision-of-the-olingos-%28Bassaricyon%29-with-description-of-a-new-species-the-Olinguito-ZooKeys-324-001-g003.jpg&oldid=126538904
is a duplicate of
https://commons.wikimedia.org/w/index.php?title=File:Illustrations_of_the_species_of_Bassaricyon_-_ZooKeys-324-001-g003.jpg&oldid=124698039 .

[notconfusing]

The filenames are being harmonized now. Sometimes there is an issue with same filenames but image size differences - but thats a different bug wpoa/JATS-to-Mediawiki#20 (comment)