What about an ancillary Scholia-integrated project for feedback to pre-preprint papers and preprint papers?

[prototyperspective]

The Wikipedia Library (a site for active Wikipedia editors to gain access to locked-away sources) or a tool similar to it (started from the same code) could become a way to publish preprints to a large community who can provide feedback and (pre-)peer-review instead or before publishing preprints completely publicly (e.g. often causing news reports).

This could work similar to subreddits on reddit in terms of how you discover preprints (you can subscribe to topics and people can interact – e.g. curate – in various ways) and the community could initially consist of existing active Wikipedia editors. It could also be integrated into the proposed Scholia Watchlist (#1733).

It could have many advantages such as better interconnecting peers and preprints and being something between preprint repository and conventional journal. Something in-between may currently be missing but useful in many cases.

Unlike preprint sites like biorxiv, we could enable proper review features such as comments for highlighted text...for example using the etherpad or anything similar (the software already exists; you could use cryptpad to see an example of this). People more knowledgable in how large conventional peer-review systems work could probably give some useful insights into how this could be designed and which things to consider.

---

My initial conclusion about this was that:

1. it'll probably just drain valuable researchers' time who are either drawn into doing peer-reviewing or looking into / replying to lots of time-wasting commentary without much substance (due to either there being lots of reviewers and/or lack of sufficient domain-specific expertise of those giving feedback).

However, it could also save time, especially relatively more valuable time by doing some initial peer-reviewing work before more senior peers do review as well as helping filter the flood of papers or better direct research (such as due to getting some early feedback during an early draft stage). Moreover, this could for example be used by students by which these could self-educate by giving feedback and interacting with papers (basically a type of project-based learning).

2. It won't work due to a lack of peers.

Like The Wikipedia Library (and these two could be combined for example), it could start off with a user-base of active Wikipedia editors (which includes a form of proof of constructive open contributions) who are initially given access. The more it is used, the more people would start using it (all of which could benefit Scholia) so over time many subjects have some peers which get notified or actively explore new prepreprints. There could also be groups that find this concept useful and jump onboard to for example do rapid early reviewing before preprints are published (e.g. 2 days afterwards) instead of them getting published in the open right away (which could lead to e.g. misinterpretations in the media and so on).

I don't think that preventing reports about such pre-preprints before they are published openly or discarded would be a major problem...the same also applies to "claims of discovery" for which e.g. cryptography could be used.

What do you think?

---

_{I'll revise this post over time. Initially, I intended to post this as an issue but it's more of an ancillary project that could then be integrated into Scholia once at a reasonable beta state. Maybe this Wiki-Journal proposal could be relevant.}

_{I think it does have the potential to change a lot about the scientific/academic publishing process towards open and efficient science, in particular for the better albeit it may be difficult.}

_{I don't think I'm drastically over- or underestimating the amount of work required to get a working version of this implemented (not from scratch but from existing tools) and a working version that's actually useful and widely used.}

[fnielsen]

Just a note: The Wikipedia Library does - AFAIU - not contain any data, - it is just a gateway to paywalled publishers.