Using GitHub discussions and issues for public preprint feedback

[castedo]

This idea discussion is about what works well, or not, when publicly providing critical feedback on a preprint via GitHub issues and discussions. I am particularly grateful for feedback and any lessons learned. Public negative feedback has all kinds of social challenges. But publicly sharing communication and thoughts on the web also has major advantages for the advancement of our collective knowledge. The tskit-dev community, and countless open-source projects, have demonstrated this kind of advantage. How to strike the right balance I wonder.

To help balance all the negative stuff I post, I'm going to also post a GitHub discussion for all the parts of the preprint that I think are spot on and/or I thought were really great/true points to make.

[jeromekelleher]

Thanks @castedo, this is a great idea!

[castedo]

Any insights/thoughts @jeromekelleher on open/closable threaded discussions for public preprint feedback?

Software engineering has a rich tradition of using threaded discussions, in the form of issues/bugs/tickets/etc..., with states like open and closed. I ended up doing all my points of feedback (#439 #440 #441 #442) as open/closable GitHub issues.

One downside to open/closable state is it can create social drama over whether an issue should or should not be closed. I think we can both think of lots of upsides to open/closable state. But for public preprint feedback, my inclination now is that it is not worth the social risk. I'm thinking threaded discussions with IDs but without open/close state (like this GitHub discussion) are better. The authors can use a variety of way to categorize the IDs, perhaps sometimes in private. And then feedback-givers can keep track of IDs in their own list if they really care that much.

[jeromekelleher]

I like the model of tracking these comments via discussion threads @castedo. In general, I think issues are best for things that are intended to be closed - either through resolution via changes or as "wontfix". Threaded discussions are better because you don't have the linear narrative of issues, which can get quite confusing when people go off one tangents (which seems inevitable for preprint feedback).

So, my take would be things like "this is a typo at line xxx" is an issue, and anything more substantive/subjective should be a discussion.

[jeromekelleher]

So, for example, when we get reviews back from the journal, I'll create a set of issues to track the things that we need to change, and I know we're ready to resubmit when all these have been closed. Having lots of vague discussion-oriented issues lying around makes this workflow much harder.

[castedo]

When you say "the linear narrative of issues" is that because GitHub issues do not have this "Write a reply" input field and "Reply" button for each top-level comment that appear in a GitHub discussion?

[castedo]

For example, I've just tattooed numbers #439, #440 #441 #442 on my right hand so that I never ever forget to track how my perfect suggestions are graciously incorporated into the next version exactly the way I think they should be. 🤣 But seriously, feel free to close them as use them as you see fit.

[jeromekelleher]

Perhaps we should convert the subjective ones to discussions?

[castedo]

Whatever works for you. To me all four of those issues (the ones now tattooed on my right hand) are subjective. Based on our discussion I think I would submit those types of feedback points as GitHub discussions rather than issues. I'm thinking records like GitHub issues should be created by authors unless it's like the "typo at line xxx" thing you mention.

[castedo]

OK, now I feel a bit silly. 🤦 These GitHub discussions do have open/close state too. There are "Close as " ("resolved", "outdated", "duplicate") actions.