Research Integrity at CHIBE: A Q&A with Jingsan Zhu
CHIBE spoke with Director of Data Analytics Jingsan Zhu, MS, MBA, about the importance of research transparency and replicability, what CHIBE is doing to promote research integrity, and concrete steps our research teams can take.
What concerns you the most about ensuring research integrity and enhancing public trust in science? Specifically, what do you see as the biggest challenges?
The main challenge faced by many researchers is that the data management process is often not robust enough to support research transparency and replicability. This often results from data management becoming an afterthought while research is ongoing. Certain remediation may take place when the current system, or rather the lack of a system, leads to some kind of data chaos.
Data management should never be a reflective action. It should be carefully planned before the study begins, with data roles, responsibilities, documentation processes (including coordination efforts), and the application of data management tools clearly outlined in a Data Management Plan that coincides with the research protocol (or better yet, as part of the research proposal). This plan should be actively discussed among research team members, and there should be an oversight structure to ensure research is conducted according to the plan. It is not a high bar to clear, but it does need careful planning and engagement with the research lead, data manager, and all research staff.
The biggest challenge is that if research teams believe data management is something that happens organically as the project progresses, they may not fully appreciate the proactive nature of data management planning. As a result, they will not adopt best practices until it is too late.
As the lead of the Data Management Taskforce, what efforts have been made to improve data management practices and promote research integrity?
It’s a privilege to lead the Data Management Taskforce, which comprises some of my amazing coworkers who are passionate about improving research practices through the lens of data management. Over the past 5 years, we have conducted due diligence, including literature reviews and outreach to various research stakeholders through meetings, presentations, interviews, and surveys, as well as performing project management process audits. We presented our findings to our PIs, project management staff, and analysts group and solicited feedback.
More recently, we hosted a panel during the CHIBE annual retreat to discuss the important role of data management in research integrity. This included engaging audiences with mini-case studies to further hone some of the fundamental ideas on research transparency.
A suite of templates and tools has been developed over time to reflect best practices, covering wide-ranging areas of data management activities. These include Data Analysis Plan development, Study Closeout procedures, Data De-identification processes, Data Sharing Guidelines and repository choices, Data Management Plan checklists, DUA tracking templates, and much more. These tools and resources are readily accessible through PENN+BOX, which also hosts training resources for onboarding new staff or faculty members.
If you would like access to these resources, please email me at jingsan@upenn.edu.
What do you see as progress in our research practices as a result? Is there more work to be done?
I am excited to see an encouraging trend: more study teams are actively discussing the role of data management during research management discussions and data meetings. The outreach by the Data Management Taskforce to our research groups, engagement with center and division leadership, and other efforts have certainly raised awareness. Nothing makes me happier than when people reach out to us requesting certain data management tools we’ve built or access to our general resource folder on Box.
When we talked to project management staff, they are more actively thinking about the data QC process in collaboration with analysts. When we interviewed our analysts, they indicated a strong desire for more rigorous code-check mechanisms. To me, these are all very encouraging signs.
However, we have more work to do. One critical element to ensure our collective success is to encourage our academic leadership to commit resources and support to further strengthen data management functions across all study groups. One suggestion is to create a dedicated data manager role who can work closely with project leads across study groups, promote standardized processes, deploy data management tools, help with training needs, and support data management oversight.
Secondly, unless our PIs are deeply committed to evidence-based best data practices and play the role of not only a strong advocate but also lead the charge, the staff-initiated efforts can only take the success of the project so far.
What are some concrete steps that our research groups could take today?
For our PIs and research leads: I wish you could start your research meetings by asking about data management processes and identifying any gaps that the team can address immediately.
For our project management staff: I’d like you to identify a documentation process to log and track operational and research decisions with strong version control that is accessible to both analysts and non-analyst staff and faculty. Teams normally have some form of decision-logging processes, but they are often managed by different staff roles in a highly fragmented way, including storage choice. Avoid using emails as the main tool for documentation purposes!
For our analysts: I’d like to ask you to establish routines for code checks and other quality checks of your programs. Engage project leaders to allocate time and resources (such as inviting collaborative reviewers) for this important aspect of research.
As you can see, we can all start to take small steps toward the goal of conducting honest and higher quality research, and the best time to act is now.
Do you have feedback, questions, or ideas for other research integrity-related content you are interested in? Reach out to jingsan@upenn.edu.
(Photo courtesy of Margo Reed Studio)