Case Study - BMJ Open

In this article we will share how BMJ Open included Penelope.ai into its author workflow as part of a larger pilot where we aim to capture the majority of their submitting authors. 

Setup

We reviewed BMJ Open’s author guidelines and customised Penelope’s checks and feedback to reflect BMJ Open’s policies and author resources. The Editor could adjust  our customisation where necessary using our journal dashboard. 

Integration 

To integrate the tool, the Editor altered their “Submit article”  button, to divert it to a Penelope.ai page giving authors the option either to check their manuscript using Penelope first, or to submit via Manuscript Central without a check. 

Author flow

Authors that choose to use Penelope are invited to upload their manuscript. Within two minutes, they receive an email with a link where they can view the feedback. 

The feedback indicates what they need to change to comply with  journal guidelines. The tool verifies article structure, abstract structure, title page completion, declarations, tables, figures and referencing. It also flags up  whether a reporting checklist is applicable to their work. You can see an example here

Monitoring and Evaluation

BMJ Open’s Editor can use the Penelope.ai dashboard to view how many authors use the tool, and any feedback they leave. Within the first 2 weeks, 300 authors used the tool and rated it an average of 9/10. 

At the end of a month we will compare manuscripts before / after Penelope usage, and compare manuscripts that were checked with Penelope.ai with those that weren’t. We will investigate Penelope’s effect on submission processing times and desk rejections. 
For research articles, we will measure whether the tool increases the usage of reporting guidelines when applicable. 
 

New guideline finder for medical researchers

We have been working hard with the EQUATOR network to expand and  improve our guideline finder for medical researchers. The  new version includes more guidelines, and a clearer set of questions to guide authors to the most appropriate reporting guideline. The new guideline finder has been integrated into Penelope.ai. Authors of medical research are presented with the same set of questions when they upload their manuscript, and Penelope.ai then tailors the advice it gives accordingly. 
 

BMJ Open begins Penelope trial

Today we are delighted to announce that BMJ Open will be inviting their authors to try Penelope. We have created a version of Penelope customised to their journal guidelines, and we will be excited to learn what authors think of the tool over the next few months. 

We are thrilled to be working with the BMJ, whose staff have been so generous during our development, giving us feedback and advice from day 1. The BMJ publishing group has a long-standing commitment to research integrity; it is an ever-present voice in discussions around improving research, a champion of reporting guidelines and a partner of the Methods In Research on Research PhD programme.

We are particularly excited to be working with BMJ Open, an innovative journal that is fully committed to transparency and reproducibility. In the words of Adrian Aldcroft, Editor of BMJ Open, “We want to embrace technology and tools that can help us achieve a high editorial standard whilst also simplifying the publication process for our authors. We are excited to pilot the Penelope tool with the hope that it improves the author experience, simplifies the submission process, and helps us achieve our editorial goals.”

You can read more on BMJ Open's blog.  

Update: Journal personalisation

Our latest update now allows journals to specify the exact wording they want Penelope to use when it gives feedback to an author, including the links it provides for extra info. This means Penelope can mirror the instructions of a journal’s author guidelines. 

Journals can also specify the exact wording for headings and where items should go. For example, should it be “Conflicts of Interest” or “Competing Interests”? Should informed consent be addressed in the methods section or in a separate ethics section?

This level of customisation means that Penelope can truly become an extension of a journal's workforce. 

Penelope at UKSG conference

This month our founder, James, was delighted to be invited to present at the UKSG conference about the scholarly ecosystem. It was a great conference, with some really interesting talks showcasing innovative projects being lead by a wide array of publishers, libraries and institutions. You can watch James’ presentation here, where he discusses latest results from our first 500 users, and read a write-up of the event on the scholarly kitchen