Looking for consultants familiar with executablebooks

[jpmckinney]

Hi folks,

I'm not sure where to put this, so pardon the interruption.

I'm the editor of the Open Contracting Data Standard (OCDS) – a free, non-proprietary open data standard for public contracting, implemented by over 30 governments around the world. The OCDS is maintained by an NGO, the Open Contracting Partnership.

Our documentation is written in Markdown and generated using Sphinx – but using recommonmark and Commonmark.py (for which we've had to fix many gettext-related bugs) rather than MyST and markdown-it-py. It also uses a simple theme based on the ReadTheDocs theme, which has many UX issues (notably mixing page links and within-page links in the sidebar).

We would like to switch to using The Executable Book Project's Sphinx theme and extensions, but our in-house capacity is currently stretched on other projects.

So, I was wondering whether the project's maintainers, contributors or community members were interested in consulting work. If so, I'd be happy to chat! You can also reach me at [email protected]

[chrisjsewell]

Thanks for reaching out @jpmckinney, I would certainly be interested in helping out, as it looks like a worthy cause 😄
Although, as with you and probably every other open-source project lol, our capacity is also stretched, so I may not be able to offer substantial amounts of time.

Anyhow, @choldgraf is the holder of the purse strings, so I'm sure he will offer he's thoughts soon

[jpmckinney]

Thank you, @chrisjsewell! We are happy to explore what a reasonable scope of work would look like – and in case I wasn't clear we would be paying market rates for the work. That said, I know many in the project's team might be busy with academic and scientific work, and so might not be available for another contract.

[choldgraf]

hey @jpmckinney thanks for reaching out! It would be great to find a way for the EBP team to help you out.

First of all - the OCDS sounds really cool! I hadn't heard of it before. We <3 open standards in the Jupyter world :-) do you know of any projects that are aggregating datasets that have been published with the OCDS? That would be a pretty cool project - I've seen similar things do this in other fields (e.g., we are starting to work with a project called the Catalyst Cooperative which aggregates and standardizes federally-published climate/energy datasets.

Anyway back to your question, I can think of a few potential paths forward:

• Somebody on the EBP team individually contracts with you if they have the spare time (e.g., @AakashGfude @mmcky @chrisjsewell or @najuzilu seem like the most likely suspects)
• We can work out a contract for the non-profit that I help run, 2i2c, to collaborate with you on this (2i2c develops and runs infrastructure in the Jupyter / open-source world for research/edu/non-profits/etc)
• We can try to scope this inside of the EBP grant and carve out a slice of somebody's time to help you do the conversion being funded on the grant (though as @chrisjsewell mentions - right now a lot of us are close to capacity on bandwidth for the grant itself). In return you could write us a nice letter talking about how useful the EBP tools are whenever we try to ask Sloan (or somebody else) for more money :-)

Could you give an idea of the scope of work that you have in mind? E.g., is this a pure "conversion" thing, or do you also want new content written? Or new UIs developed beyond just changing the theme (e.g., re-working some content using sphinx-panels and such?)

[jpmckinney]

Oops, this is my first time using GitHub's discussions feature, and it looks like I replied out-of-thread below.

[jpmckinney]

We have a tool (built on Scrapy) to collect all published OCDS data (https://kingfisher-collect.readthedocs.io/), and we are presently working on (1) a data service to give people direct access to this data without having to run a spider and (2) a "flattening tool" to convert the JSON to CSV/Excel, while giving the user some control over the "shape" of the tables. (There is already Flatten Tool, but it isn't as configurable as desired, and it's inefficient for large datasets.) There are other, commercial projects that aggregate OCDS data. Related to the Catalyst Cooperative (thanks for the link!), you might be interested in https://icebreakerone.org.

We'd be happy to work out a contract with 2i2c (and it looks like we might find future opportunities to work together, as we have been creating reusable Jupyter notebooks over the last year), or with an individual. I assume the former might be simpler, but I don't have a strong preference. (In either case, I'd be happy to separately write a letter describing how useful EBP's tools are :))

In terms of scope, there's room to scale up. We'd start with:

1. Change the theme to sphinx-book-theme, to get a first set of improvements, like the navigation
2. Switch to MyST (I think required to change theme), which will at least involve changing eval_rst blocks to the MyST equivalent, and we might change how headings are linked

After merging those changes, we can progressively:

1. Customize the theme (e.g. integrate OCP's colors, add a top navigation bar, use elevation to distinguish the right sidebar)
2. Adopt some of the theme's features, like toggles, sidebars within content, margins and glossaries
3. Adopt some of the related extensions, like panels/tabs, comments, copybutton, togglebutton
4. Contribute to a style guide, so that content authors know when to use which of the above features/extensions

The last few would involve closer collaboration with our content authors, to decide on a consistent approach. Another way to organize the work is to start with the style guide, which our content authors can then apply independently.

By doing things progressively, we can stop when appropriate / based on availability / once we reach our ceiling for this project.

Does that make sense? Let me know if you have any questions, and I'd be happy to have a call to discuss as well.

[choldgraf]

hey @jpmckinney - yep that makes sense to me 👍 I'd be happy to jump on a quick call to chat about any of the above if you like. (though lemme know if others have already reached out to you privately about doing the Jupyter Book docs work - I'd be happy for others in the community to take a crack at it if they're interested). What's a good way to get in touch? Feel free to send an email at choldgraf AT 2i2c.org if you like.

[mmcky]

@jpmckinney there is also sphinx-tomyst if your projects are already in rst format to assist with the migration to myst::markdown files. I am using this extension to assist with migration work at quantecon and would be happy to add support through this translator should you need additional directives etc.

[jpmckinney]

Hi @mmcky, our project is already in Markdown, but using recommonmark instead of MyST. Looks very handy for RST projects, though!

[mmcky]

ah -- sorry I missed this:

Our documentation is written in Markdown and generated using Sphinx

If it is using Sphinx as the build engine -- we could give sphinx-tomyst a try I suppose. It consists mainly of a writer/translator that builds myst from the Sphinx AST. I haven't tried this out and I am not sure if recommonmark has special nodes etc. I'd be happy to lend a hand to do some testing @choldgraf if that is an avenue to explore.

[jpmckinney]

Interesting. Our Markdown includes eval_rst blocks like:

```eval_rst
.. toctree::

   a-page
```

which will need to be changed to:

```{toctree}
a-page
```

recommonmark doesn't support linking to headers on other pages, so we had to write relative URLs (as would appear in the generated HTML), instead of relative paths or Sphinx references like in MyST. As such, we'll have to manually edit those in the Markdown.