Minting of DOIs (and archival?) for blog posts

[mfisher87]

Description

I'm currently a user of Zenodo to automatically generate DOIs for each release of my software projects on GitHub, and I love that my source is automatically archived and citable. All I have to do is flip a switch and Zenodo starts watching my repo for Releases. When one happens, it reads state from my Git repository and, if I have one, a CITATION.cff file, to find the metadata it needs.

I've been thinking that this experience would be amazing to have for each post I make on my Quarto blog. Of course, since my blog is already a repo on GitHub, I can easily get a DOI for the whole thing, but that's not what I'm looking for.

I don't know what this workflow might look like, there are definitely complexities I haven't considered. Perhaps:

• a new Quarto command to add a DOI to a specific post e.g. quarto add-doi posts/my-post/index.md which would reach out to a DOI authority/archival service like Zenodo, authenticate with the author's credentials, send a copy of the live post (by scraping from site-url? maybe modifying to embed image data?) for archival, and request a DOI. Once it gets a DOI, it updates the source file with the DOI in the front-matter.
• a third-party service which generates DOIs from an RSS feed? The needed metadata would need to be available in the feed. But then the user would need to manually go add the DOI to their post's front-matter to complete the setup.
• ???

This feels like a big ask, but also possibly a huge win for adoption by technical publishing communities. As far as I can tell, there is a Wordpress plugin that can do this, some commercial live services that can do this, but nothing else.

Note

I'm already using the citation: true feature to generate citations at the bottom of each post, and I can manually generate DOIs for each post and then add the DOI to the front-matter. This gets the end result I'm looking for, but the process is too manual.

[mfisher87]

The hypothetical new add-doi command could go a step further and, like Zenodo, inspect the Git history to identify if a file has changed since its last DOI was generated, and if so, generate a new sub-DOI the way Zenodo does for each GitHub Release. The front-matter of the doc would have the "all-versions" parent DOI so it wouldn't need to be updated.

This sounds like it would be super fun to explore in depth, but with my current arrangement I'd need to write a grant proposal for a chance of funded time to work on it. I just got done writing another proposal and jeez I didn't know how exhausting it could be!

[jimjam-slam]

This feels like something that a couple of Quarto project scripts could handle! Maybe:

1. a periodic script (quarto run add-doi [post name]) to add some YAML metadata to a post, or a user just adds it themself. Then,
2. a project pre-render script that intercepts posts with the required metadata, compares the history, handles the DOI regen and then injects metadata and/or content for the new DOI.

[mfisher87]

TIL project scripts are a thing, thanks :)

I'm really interested in making this process as accessible as possible. I don't really know much about quarto extensions, can project scripts be installed as extensions?

[jimjam-slam]

They absolutely can! Here's one of my Quarto extensions, sverto. The specifics of it aren't really important, except to say that it contributes a new project type built on HTML that includes project pre-render scripts (three of them, in fact—check out _extension.yml). It also contributes a document format based on HTML that includes a Lua filter. So you can include all of this in one extension! 😊

[mfisher87]

Thanks, James! I'm excited to look in to this more, this seems like a promising path. Time is the main struggle for me right now.

Zenodo's API looks like it does everything I'd need, but I wonder if their migration to InvenioRDM will be disruptive for API users. I haven't looked deeply yet, but exciting their dogfooding their open-source platform :D

The Quarto extension developer guide looks like it'll be really helpful too.

[jimjam-slam]

I feel it—too many exciting things to work on! 😅

[MattF-NSIDC]

I've learned a few new things after getting introduced to some folks with Crossref and The Rogue Scholar.

• The Rogue Scholar will watch a blog's RSS/Atom feed and generate DOIs from that (it currently is operating on two Quarto blogs: https://rogue-scholar.org/blogs/tarleb and https://rogue-scholar.org/blogs/andrewheiss). In Quarto's specific case, my understanding is that RSS lacks the ability to specify author metadata so, for example, there's no standard way to transmit an author's ORCiD for association with the DOI. If Quarto offered an Atom feed with rich author metadata, then generating DOIs from a Quarto blog's feed could be more robust. I think this may warrant a feature request for supporting Atom feeds with rich author metadata.

• There's a Crossref community call on October 31 with a demo relating to this topic. I want to learn more about this before starting development on a Quarto extension.

[MattF-NSIDC]

@jimjam-slam I'm not sure if you'd be notified about this since I made it a new response instead of replying to your thread, but I thought you'd be interested ☝️

[mfenner]

I am the Rogue Scholar developer, and the service is indeed providing option 2, a third-party service which generates DOIs from an RSS feed. This is not specific to Quarto and works with all blogs that have an RSS (or Atom/JSON Feed) feed. Rogue Scholar is currently doing that for four blogs using Quarto. The service registers DOIs with Crossref and is free for up to 50 posts per year. As I have just migrated one of my two blogs to Quarto, and would be happy to work on an improved integration, along the lines of publishing with Netlify, GitHub Pages, etc.

The problem with Zenodo/InvenioRDM is that the DOIs you register there will point to Zenodo and not your blog. But it is a good platform to upload blog content that already has a DOI in one of the many Quarto output formats (e.g. PDF or ePub). I am involved in InvenioRDM development and happy to help with writing a Quarto integration, as that is on my roadmap for the next three months.

[chrisvoncsefalvay]

I hope @mfenner is not going to hate me for teaching people how to potentially hug his site to death, but here's how I solved this problem: a project script. @mfisher87, this may meet your needs.

This is a pre-render script that looks at the first ten posts in the Rogue Scholar feed, obtains the DOI via beautifulsoup4 parsing, and attaches them to the YAML preamble. If you are rendering using a GitHub Actions workflow, you will want to include a post-commit, and you'll also want to make sure that that's conditional in execution on there being a change to the working tree lest it throws an unsightly error.

[mfenner]

@chrisvoncsefalvay of course I very much appreciate your work to make DOI registration easier, especially the part of getting the DOI from Rogue Scholar and adding it to your Quarto blog.

One suggestion: I recently launched a dedicated API (a Python Quart app) and parsing the response will be easier than scraping the web page with Beautiful Soup, so https://api.rogue-scholar.org/blogs/chrisvoncsefalvay instead of https://rogue-scholar.org/blogs/chrisvoncsefalvay. Or a with a more specific query, the Open API documentation (work in progress) is at https://api.rogue-scholar.org/docs.

[chrisvoncsefalvay]

Ah damn. Just when I wrote a nice long blog post about it. Time to rewrite lol.

[mfenner]

Sorry @chrisvoncsefalvay :) But good timing as I am launching my own Quarto blog tomorrow. And your post will be very helpful for integrating the DOIs into the blog.

[mcanouil]

For reference.

• doi for html document #11936

[mfenner]

thank @mcanouil. Rogue Scholar continues to provide DOI registration for science blogs, including a number of Quarto blogs. This is only for science blogs, not for over services built with Quarto, e.g. books, websites, etc. Since the beginning of the year the workflow is simplified, the doi in the Quarto post is used for DOI registration (under some conditions, to make sure the DOI is unique).