The prize was created to recognise initiatives to reduce waste in health research.
In this article we will share how BMJ Open included Penelope.ai into its author workflow as part of a larger pilot where we aim to capture the majority of their submitting authors.
We reviewed BMJ Open’s author guidelines and customised Penelope’s checks and feedback to reflect BMJ Open’s policies and author resources. The Editor could adjust our customisation where necessary using our journal dashboard.
To integrate the tool, the Editor altered their “Submit article” button, to divert it to a Penelope.ai page giving authors the option either to check their manuscript using Penelope first, or to submit via Manuscript Central without a check.
Authors that choose to use Penelope are invited to upload their manuscript. Within two minutes, they receive an email with a link where they can view the feedback.
The feedback indicates what they need to change to comply with journal guidelines. The tool verifies article structure, abstract structure, title page completion, declarations, tables, figures and referencing. It also flags up whether a reporting checklist is applicable to their work. You can see an example here.
Monitoring and Evaluation
BMJ Open’s Editor can use the Penelope.ai dashboard to view how many authors use the tool, and any feedback they leave. Within the first 2 weeks, 300 authors used the tool and rated it an average of 9/10.
At the end of a month we will compare manuscripts before / after Penelope usage, and compare manuscripts that were checked with Penelope.ai with those that weren’t. We will investigate Penelope’s effect on submission processing times and desk rejections.
For research articles, we will measure whether the tool increases the usage of reporting guidelines when applicable.
We have been working hard with the EQUATOR network to expand and improve our guideline finder for medical researchers. The new version includes more guidelines, and a clearer set of questions to guide authors to the most appropriate reporting guideline. The new guideline finder has been integrated into Penelope.ai. Authors of medical research are presented with the same set of questions when they upload their manuscript, and Penelope.ai then tailors the advice it gives accordingly.
Authors using Penelope to check their work can now see their feedback online. Take a look at a demo here. This is a great enhancement to Penelope as it offers a smoother user experience. The feedback is easier to read, can be enriched with bullet points, emphasis and hyperlinks.
Journal editors, peer reviewers, and other journal staff can now use the Penelope dashboard to check manuscripts themselves. They can control which checks run, and customise the wording of the feedback it gives to both editors and authors. The dashboard allows them to monitor usage and authors’ experience of the tool.
Today we are delighted to announce that BMJ Open will be inviting their authors to try Penelope. We have created a version of Penelope customised to their journal guidelines, and we will be excited to learn what authors think of the tool over the next few months.
We are thrilled to be working with the BMJ, whose staff have been so generous during our development, giving us feedback and advice from day 1. The BMJ publishing group has a long-standing commitment to research integrity; it is an ever-present voice in discussions around improving research, a champion of reporting guidelines and a partner of the Methods In Research on Research PhD programme.
We are particularly excited to be working with BMJ Open, an innovative journal that is fully committed to transparency and reproducibility. In the words of Adrian Aldcroft, Editor of BMJ Open, “We want to embrace technology and tools that can help us achieve a high editorial standard whilst also simplifying the publication process for our authors. We are excited to pilot the Penelope tool with the hope that it improves the author experience, simplifies the submission process, and helps us achieve our editorial goals.”
You can read more on BMJ Open's blog.
Our latest update now allows journals to specify the exact wording they want Penelope to use when it gives feedback to an author, including the links it provides for extra info. This means Penelope can mirror the instructions of a journal’s author guidelines.
Journals can also specify the exact wording for headings and where items should go. For example, should it be “Conflicts of Interest” or “Competing Interests”? Should informed consent be addressed in the methods section or in a separate ethics section?
This level of customisation means that Penelope can truly become an extension of a journal's workforce.
This month our founder, James, was delighted to be invited to present at the UKSG conference about the scholarly ecosystem. It was a great conference, with some really interesting talks showcasing innovative projects being lead by a wide array of publishers, libraries and institutions. You can watch James’ presentation here, where he discusses latest results from our first 500 users, and read a write-up of the event on the scholarly kitchen.