One of our goals for package onboarding at rOpenSci is to provide a service to our authors by making their packages more visible once they pass through peer review. As such, we are planning an experiment in which we automatically submit accepted packages to the Journal of Open Source Software if authors request it, with JOSS’s editors being able to accept on the basis of the RO’s reviews.
JOSS has some slightly different requirements than RO. First, they require a short (<500 words)
paper.md document with a high-level review of the software. Software must also be deposited into a repository such as figshare or Zenodo that provides a DOI prior to submission. JOSS’s software quality requirements are largely general principles, as they are language-agnostic. In general they are less stringent than RO’s R package requirements.Thus, with a few small additions an RO package can easily be published in JOSS.
However, so that our reviews are usable by JOSS, we would need to make some parts a little more structured and explicit. I’ve started a pull request that updates our onboarding repository to do so. Author submissions, and both editor and reviewer comments will include checklists in addition to freeform comments. These checklists cover everything in the JOSS reviewing checklist (e.g., https://github.com/openjournals/joss-reviews/issues/28), but with our more stringent and R-package-specific requirements. They provide a little more structure to our reviews, and attempt to move some gatekeeping checks up to the editors prior to assigning reviewers. This approach requires that our reviewers review the short
paper.md as well, should authors choose to submit to JOSS.
You can see what all of the checklists would look like together in a review in this test issue.
I hope for some community feedback before we start this process. I am very excited about this experiment - it will reduce the dual-submission effort for authors and lead to more visibility and credit for our packages. Yet I have two primary concerns:
- We already ask a lot of our reviewers - are the additional checks and
paper.mdreviewing too onerous?
- Will the checklist format of the reviewer’s template lead to less thorough reviews?