Using Digital Tools in Clinical, Health and Social Care Research: A Mixed-Methods Study of UK Stakeholders

Contents

Research Team and Collaborators:

University of Warwick 

• Dr Sophie Clohessy, Research Fellow, WMG
• Dr Mark T. Elliott, Co-Primary Investigator, WMG
• Dr Carla Toro, Co-Primary Investigator, WMS
• Professor Theodoros N. Arvanitis, Co-Investigator, WMG
• Umer Rashid, Research Intern, WMG
• Estie Hughes, Research Intern, WMG

NIHR CRN West Midlands

• Carly Craddock, Chief Operating Officer NIHR Clinical Research Network
• Mark Evans, Research Data and Digital Transformation Manager

Acknowledgements:

We thank all participants for taking part in the study. We would also like to thank members of the steering group for their contributions on developing online survey and qualitative questions. Thank you to CRN West Midlands for their support in organising the online workshop in order to disseminate our findings. 

Abbreviations:

CRN- Clinical Research Network

CPRD- Clinical Practice Research Datalink

MS Teams- Microsoft Teams

NIHR- National Institute for Health and Care Research

PROMS- Patient Reported Outcome Measures 

UHCW- University Hospitals Coventry and Warwickshire

Funding:

We would like to acknowledge that this work was supported by the National Institute for Health and Care Research, Clinical Research Network West Midlands Improvement and Innovation Strategic Funding. 

Executive Summary

The COVID-19 pandemic rapidly accelerated the use of digital tools in health, social care and clinical research studies and reports have suggested that digital tools will be used increasingly in the future. Indeed, a report by the UK government [see ‘The Future of Clinical Research Delivery: 2022 to 2025 implementation plan’ (Department of Health and Social Care, June 2022)] sets out a future vision of research delivery which includes research enabled data and digital tools. 

A team of academic researchers based at WMG, University of Warwick have conducted research in collaboration with NIHR CRN West Midlands. The aim of this report is to present the research findings. Within the report, we explore which digital tools are used in the CRN West Midlands region. We also try to understand people’s experiences using digital tools, what worked well and not so well. We investigate the following types of questions; Which digital tools are people using? Which digital tools are effective? Are there any barriers to digital tool use? 

This report covers two phases of data analysis. Firstly, we compare the findings of three separate online surveys with the following participant groups within the CRN West Midlands region:

1) Researchers and related staff

2) Research and Development staff

3) Previous participants/carers of clinical research (Low participant rate achieved) 

Secondly, using data from qualitative interviews with eight stakeholders in the NIHR CRN West Midlands region we built upon the results of the online survey and explored people’s experiences of digital tools in further depth. Participants who took part in the survey were given the opportunity to take part in qualitative interviews, where they were asked semi-structured questions about their experiences using digital tools and asked to expand on some of their survey answers (e.g., most effective tool case studies). After applying thematic analysis to the data, we found ten themes (summarised below).  

• Definition of Digital Tools in Clinical Research: Participants generally agreed that a digital tool used within clinical research was a device that can connect to the internet, or a form of online technology. The main reasoning behind using a digital tool was to make research more efficient. Digital tools have been adopted across all stages of clinical research including management, recruitment, and data collection.
• Impacts of the COVID-19 Pandemic on Digital Tool Use: It is evident from our analysis that the COVID-19 pandemic has rapidly increased the use of digital tools within clinical research. Through this change in working practices, it appears attitudes among staff and research participants is one of acceptance towards digital tools. 
• Perceived Benefits of Digital Tools:  Advantages were considered from two perspectives: 1) Research participants, 2) Staff using digital tools. For research participants, benefits included: convenience, accounting for individual differences, more opportunities for people to take part in research. Staff benefits included: efficient use of time and easy connectivity between colleagues (particularly if based in different locations). 
• Perceived Drawbacks of Digital Tools: Perceived drawbacks included technical issues, difficulty reading interactions over digital means (compared to face-to-face), learning how to use new digital tools/systems. For some digital tools, the levels of engagement are unclear from different participant groups (e.g., which tools work best for a particular group or task).
• Selection of a Digital Tool: Participants cited little choice in the digital tool they used. One of the main reasons was because most digital tools were widely used in an organisation (for example MS Teams, SharePoint, WhatsApp). However, some participants described the process of designing and using a bespoke digital tool. Most participants reported discovering tools via word of mouth. Recommendations from participants included a central place (e.g., website) to share information about digital tools. A standardised process outlining how to choose a tool. For example, a digital checklist before choosing a digital tool i.e., people input requirements of a digital tool and a programme would offer suggestions of appropriate tools. Further examples included case studies of effective tools (by stage and participant group) and a local/national network to share expertise.
• Barriers: Several barriers to digital tool use were referenced, including attitudes of staff/research participants, conflict of carrying out patient centred job and time to use digital tools. Furthermore, resources are a key driver for digital tool use (budget, physical resources, digital expertise), clear resources varied across organisations in the West Midlands region. Several ways to overcome barriers were mentioned. For example, due to the rapid adoption of digital tools, some staff used a hybrid approach using both paper and digital tools to carry out the same tasks (e.g., recruitment of participants undertaken using paper and digital methods).  For some staff, a hybrid approach increased their workload mainly via data management. Furthermore, individual differences exist in digital tool use and therefore one size does not fit all. For example, some research participants may not feel comfortable using their video camera when using MS Teams (or similar software) and therefore a way to overcome this is to take a flexible approach with participants. Participants explained participants/staff attitudes can be a barrier, offering training in multiple modalities can help.
• Training: Most participants had received some training on digital tools, training was provided using different methods including demos and videos. However, sometimes training was not always specific for digital tool use within research. 
• Cost Efficiency of Digital Tools: Little appears to be known about the costs or cost effectiveness of digital tools compared with paper approaches. Some participants expressed an interest in carrying out cost effectiveness research relating to digital tools they were using. There was a recognition this research would be important when making a case for using digital tools in future. 
• Future Digital Tool Use: Most participants believed they would be using digital tools increasingly in the future. Important themes highlighted by participants included: concern of pressure placed on nurses to keep up to date with digital tools whilst caring for patient, need for collaboration with digital experience and clinical staff when developing tools. Participants also highlighted several gaps for research to inform future tool use including which tools are most appropriate for research stage and participant group, with some participants even conducting their own research to evaluate the effectiveness of digital tools.  
• Most Effective Tools (Case Studies): A number of tools were proposed as examples of effective digital tools for research across a range of research stages, including recruitment and management of clinical research. 

 Conclusions 

Key findings taken from our scoping literature review, online surveys, and qualitative interviews suggest the following:

• The use of digital tools is a fundamental part of clinical research, even more so since the COVID-19 pandemic accelerated use of these tools; adoption was observed across all stages of clinical research including set up, management, recruitment, consent, data collection, retention, and intervention delivery.
• Tools are mostly viewed favourably by participants, with increased efficiency cited as a leading benefit. Tools can account for individual differences (e.g., materials offered in different languages). Despite this, some drawbacks exist.
• Generally, limited choice in digital tools used (mostly widely used in organisations, not bespoke), some participants did report designing/utilising bespoke tool in collaboration with clinical research staff and a digital team. 
• Digital exclusion may be a potential barrier for some participants to take part in research. Organisations should ensure research projects are inclusive for people who may not have access to digital tools or the internet. 
• One method to promote inclusivity for research participants is via a hybrid approach (using both paper/digital methods). However, some qualitative participants reported this presented its own challenges (e.g., increased workload relating to data management across both approaches). 
• One size does not fit all - research staff should be willing to accommodate personal preferences when using digital tools (e.g., some participants might not feel comfortable with their video camera on).
• Appears to be a disparity of resources across region including 1) Budget for digital tools 2) Physical resources (e.g., poor internet connection or limited digital equipment) 3) Digital expertise (some organisations had access to specialist digital teams embedded in R&D).
• Concern of pressures placed on patient facing staff (e.g., nurses) using digital tools (as clinical research demands digital expertise, concern this will only increase in the future).
• Training is key to target attitudes of staff/research participants, build confidence and demystify digital tools. 
• Several resources were suggested by interview participants for future digital tools use (for example, standardising a process for selecting a digital tool, case studies of effective tools based on research stage and stakeholders, local/national network to share digital expertise).

 Future research suggestions

As an outcome of the research project, a number of future research suggestions were made:

1. Investigate experiences of participant/carers with relation to using digital tools. Findings in this report point to the fact that participant/carers are a key user of digital tools, therefore it is vital to understand use from their perspective and if they require support for any aspect of the experience. 
2. Build local case studies of effective digital tools (specific to research stage/stakeholders)
3. Develop guidance on process of choosing a tool 
4. Develop local/national network bringing together expertise in digital tools
5. Evaluate effectiveness of tools (specific to research stage/stakeholders) 
6. Evaluate cost effectiveness of digital tools (particularly in relation to non-digital approaches).

 Digital tool use in clinical research: Guidance 

Based on the findings in this report, research teams may wish to consider the following recommendations for future digital tool selection:

• Is there an organisational process mapped out for all departments to follow when choosing a digital tool? 
• Is guidance available on different types of digital tools and different stages of clinical research? 
• Is this stored in a central, accessible place? Do staff know where to access this? 
• Is there a digital tool champion or digital expert within the organisation that staff can contact? 
• Do staff know who to contact in the first instance with a query about digital tool use? 
• Is adequate training for digital tools available? 
• If so, is this training fit for purpose? (i.e., does it cover research specific training as well as generic use). 
• Is training available in different modalities? (video, face-to-face) 
• Is training available for research participants? Is this available in different languages? 
• Can a training needs analysis be undertaken for any staff that encounter digital tools used in clinical research? 
• What is the baseline knowledge and skills required to use digital tools in your department? 
• If there are gaps in knowledge, how will you approach this? 
• Have you analysed costs of digital tools compared with paper-based approaches? 

 Digital tool use in clinical research: Key considerations

The following guidance is directed at research staff working with digital tools. Below are key recommendations when using digital tools taken from our report. 

Concern for digital exclusion

• Digital exclusion may be a potential barrier for some research participants to take part in research. 
• Research projects need to be inclusive for people who:
• may not have access to digital devices (e.g., tablets),
• may not have an internet connection, 
• require study materials in another language, 
• have limited digital literacy (may require additional assistance on how to use digital components of a study). 
• To ensure inclusivity when using digital tools: 
• Researchers should provide study materials in different languages
• Ensure there is adequate training on how to use digital tools (and offering training in different modalities i.e., video, face-to-face)
• Where possible, provide a physical location which research participants can visit to use digital devices (e.g., computer) in order to take part in the research study. 

One size does not fit all

• Research staff should consider individual differences among research participants when using digital tools.
• Research staff should be willing to accommodate personal preferences when using digital tools (e.g., some participants might not feel comfortable with their video camera on, some participants may prefer to use a chat function rather than a microphone). Before beginning data collection, it might be helpful to explain to research participants that data collection can be flexible to accommodate their preferences (where appropriate). 
• Some participants may prefer a hybrid approach to data collection (some digital, some paper based). If appropriate, researchers should offer options to research participants. 

Demystifying tools and training

This study highlighted that limited knowledge about digital tools is a barrier to digital tool use. Before embarking on using digital tools: 

• Do research staff/research participants understand how to use the tools? 
• Are adequate instructions provided on how to engage with the digital aspects of the study? 
• Do participants know who to contact if they encounter technical issues with the tools? 

Does the digital tool fit the job?

If you are involved in selecting a digital tool, have you researched different tools that could be used? 

• Are you using the most appropriate tool for the job? 
• By using a digital tool, are you adding efficiency to a research stage? Do you require additional resources to manage the digital tool? 

Scoping literature review summary

A scoping literature review was carried out to identify the types of digital tools/platforms used across clinical research within articles published 2019-2022. The following key findings were identified:

• The COVID-19 pandemic greatly accelerated the use of digital tools, with many papers detailing changes to their methodology as a result of the pandemic. 
• Digital tools were used across a range of research stages (e.g., consent, recruitment, data collection) by a range of stakeholders (e.g., research participants, research staff, clinical delivery staff). 
• Many papers reported using a hybrid approach, using both digital and paper-based methods.
• Digital tools were reported being used most for recruitment.
• Few tools were referenced for set up of clinical research. 
• Some articles measured effectiveness of digital tools. However, there was a variety of methods to do this, including online surveys, qualitative questions, and comments on effectiveness of the tool in the discussion. The heterogeneity of the methods makes it difficult to compare the effectiveness of a tool. 
• Some articles made direct comparisons of digital tools to paper-based tools.

Conclusions based on the scoping review findings:

• This scoping review offers an understanding of the breadth of tools reported since the start of the COVID-19 pandemic.
• Finding from this scoping review provides a general sense of digital tool examples and some themes that may arise from using digital tools in clinical research. However, the review does not provide an in-depth insight into which digital tools are effective and why, alongside other key issues related to their use within clinical research. 

The review offers a general insight, next we used quantitative surveys and qualitative interviews understand which tools are used within the West Midlands area.

Methods

Study design

This research was approved by the University of Warwick’s Biomedical and Scientific Research Ethics Sub-Committee. The project used a variety of methods to learn about stakeholders’ experiences with digital tools in clinical research (see Figure 1 for overview). 

Figure 1: Overview of studies included in this report 

Online surveys

Participants

Three distinct surveys were aimed at the following participant groups, with eligibility criteria shown in Table 1. 

Table 1: Eligibility criteria by survey type 

Recruitment 

Participants were recruited primarily using a database of clinical, health and social care research studies held by the National Institute of Health Research Clinical Research Network (NIHR CRN) West Midlands. For the Researchers and Related Staff survey, clinical, health or social care projects that had been completed or still ongoing within the past three years were identified by staff within the NIHR CRN West Midlands team. Although, we were primarily interested in including digital tools that had been used since COVID-19 in the past two years, recruited studies were included from the past three years to ensure a suitable number of studies were included in the survey. Most research was paused to deliver urgent public health research during the pandemic, and then possibly restarted during the two-year window.  

The email invitation and survey link were distributed to the lead researcher or main contact for the selected research studies. A similar process was followed for the R&D survey, R&D managers were contacted at all West Midlands NHS partner organisations. To further widen the opportunity to participate, a short study description and survey links were advertised on Twitter via the NIHR CRN West Midlands account. Participant and carers were recruited via CRN West Midlands Patient and Public Involvement and Engagement Team by sending to Public Research Champions, where they were also asked to disseminate to their contacts. After a period of 2 weeks, a follow-up reminder email was sent to all contacts via the same mechanism as the initial survey invitation. All three surveys were live for one month in total. After the first data collection period (27th June- 26th July 2022), participant numbers were lower than expected; therefore, a second phase of recruitment took place (14th October-15th November 2022). An email prompt was sent to all contacts via the same mechanism as the initial survey invitation.

Procedure

Participants accessed the surveys via the Qualtrics platform (https://www.qualtrics.com/uk/), For eligibility criteria for each survey see Table 1. Before taking part, all participants read an information sheet and ticked a consent form to agree to their participation. Before answering any questions, participants were offered a definition of a digital tool and examples of digital tools (Table 2). The definition of a digital tool varied between the three surveys to account for different digital tools being used in a variety of contexts e.g., employees working directly on clinical research versus employees working in an R&D department. After survey completion, participants were presented with a debriefing sheet which repeated the objectives of the study and information about how the survey data would be stored. 

Research staff survey

Participants were re-directed to the end of the survey if they did not meet the eligibility criteria (see Table 1) determined via initial screening questions. A further screening question determined if participants had used digital tools in their research studies. Participants who answered yes, were directed to the main questionnaire. If participants had delegated digital tool use to colleagues they were re-directed to the end of the survey, whilst participants who answered ‘no’, were presented with a separate set of questions that included identifying any barriers they had encountered that had prevented them from using digital tools and their intentions to use digital tools in future research studies. 

Participants who had not previously used digital tools were also provided the option to participate in a qualitative interview study to explore their thoughts towards digital tools in further detail.

Table 2: Definition of digital tool by survey type

Participants who answered ‘yes’ to using digital tools in their clinical, health or social care research were then directed to questions which focused on participants experience of what they perceived to be most effective and least digital tools used in their role (blocks presented in a randomised order, questions were the same in each block) (see Table 3 for question themes). These participants were also provided the option to participate in a qualitative interview study to explore their thoughts towards digital tools in further detail.

R&D staff survey 

Similar to above, participants who had not used digital tools were directed to a separate set of sub questions and invited to take part in a qualitative interview. 

Eligible participants answered three blocks of questions about their perceived most effective digital tool used in 1) set up, 2) recruitment, and 3) management of clinical research studies (blocks were presented in a randomised order, questions were the same in each block). 

If participants did not have a digital tool example for set up, recruitment or management they skipped this block of questions and they were re-directed to the next part of the survey (see Table 3 for question themes). Participants were also provided the option to participate in a qualitative interview study.

Participant and carer survey  

Participants were asked if they had taken part in clinical, health or social care studies within the past two years (themselves or on behalf of someone else), if they answered no or not sure they were directed to the end of the survey. Eligible participants answered a number of questions about their experience of using digital tools including advantages, disadvantages and barriers

(see Table 3 for question themes by survey type). 

Data analysis

A mixed approach to analysis was undertaken for all three surveys. Multiple choice and scale answers were analysed using descriptive statistics; free text answers were analysed thematically. Questionnaire responses were collected anonymously and aggregated for analysis. Results from all three surveys were analysed, discussed, and verified with the project team. 

Table 3: Question themes by survey type 

One-to-one interviews 

At the end of the online survey with R&D staff and Researchers and related staff, participants were given the option to sign up to take part in an interview to discuss their experiences using digital tools in further depth. Before conducting the interviews, participants were emailed a participant information leaflet and given the opportunity to ask questions. Informed consent was taken electronically, and semi structured interviews were conducted using MS Teams lasting up to one hour. The interview guide was developed based on the findings of the online survey. With the participant’s consent, some of the answers they provided in the online survey were discussed during the interview e.g., most effective digital tool. This provided the opportunity for participants to give greater detail/context to their answers in the survey.

Participants

Qualitative interviews were undertaken with two participant groups: 

1. Researchers and related staff working on clinical research projects over the past two years (2020-2022), either in a research capacity or operational support; aged 18 years or older; employing organisation based in West Midlands region of the UK. 

Research and Development staff working in within a hospital’s Research & Development (or similar) department, with a role to support or coordinate clinical research taking place within, or in partnership with the Hospital over the past two years (2020-2022); 18 years or older; hospital must be a research active NHS Trust in the West Midlands region.

Recruitment 

During the online survey, participants were invited to take part in the qualitative interviews and expressed their interest by providing their name, job role and email address. Participants were informed upon expressing their interest that their survey answers would be discussed in greater detail during the qualitative interviews (this was also agreed to on the consent form).

Procedure 

All interested participants were sent an electronic information and consent sheet via email and a mutually convenient time and date was arranged via Doodle poll (https://doodle.com/free-poll). Participants returned consent via email; this was also confirmed verbally at the beginning of each interview. Semi-structured interviews were undertaken by the Research Fellow (SC) via MS Teams and took between 45-60 minutes (conducted between 20th October- 9th November 2022). The interview topic guides were informed by the online survey and participants were informed that their survey answers would be discussed in greater detail during the qualitative interviews (referenced on the consent form). See Table 4 for an overview of question themes by survey type. All interviews were recorded and transcribed using MS Teams. Data collection was stopped at the point of data saturation, this was defined as the point no new themes were observed.

Data analysis 

Transcripts were anonymised and cross-checked for accuracy by the first author (SC) and Research Assistant (UR) against the recordings. After each interview, interviewer SC summarised key themes to inform the coding framework. Interview transcripts were then uploaded into the software programme NVivo v12. Thematic analysis was used following an inductive approach, meaning that codes were derived from the data. To reduce bias, UR independently coded 25% of all interviews using the same method as the first author and developed a separate coding framework. To enhance validity, SC and UR discussed the two coding frameworks to produce one final coding framework. Themes were identified, reviewed, and defined iteratively. Themes and subthemes were then presented and demonstrated by representative quotes.

Table 4: Qualitative interview question theme by participant type

Results – Phase 1 online surveys

Researchers and related staff results 

A total of 52 people completed the survey, of which eight participants answered that they delegated tool use to colleagues, and therefore were re-directed to the end of the survey. Two participants answered ‘no’ to having not used digital tools; one participant provided reasoning for not using digital tools which was ‘stick to what worked in the past’ (selected from drop-down menu of answers). If participants only had experience of one digital tool, they were asked questions about this specific example. One participant answered questions about one digital example, however, they did not provide the name of the digital tool and was therefore removed from the analysis.  

41 participants partially or fully completed the survey (see Table 5 for participant characteristics). As an introductory question, participants were asked to list all of the digital tools they have used over the past two years, 11/41 people completed the survey up until this point. 30 participants provided one digital tool example which they deemed to be 1) least effective tool and/or 2) most effective tool and 19/30 participants answered a number of questions about these tools. 

Table 5: Research staff survey participant characteristics

Table 6 provides an overview of the most effective and least effective tools by research stage. Table 7 provides an overview of the most effective and least effective digital tools question responses.  In the context of most effective tools, on a scale of 0 - 10 (0 no more effective than other tools to 10 much more effective than other tools) participants gave a mean rating of 8.1±1.3. This result suggests that participants were mostly positive about the tools they deemed to be efficient. In contrast, the least effective tools, were rated on a scale of 0 - 10 (0 = no less effective than other tools, 10 = much less effective than other tools) with a mean rating of 4.3±2.0. 

Table 6: Overview of most effective and least effective digital tools examples by research stage. Entries show the actual responses from participants including context where relevant. 

Table 7: Most effective and least effective digital tools results 

Next, participants were asked to provide free text answers to advantages and disadvantages with regards to their most effective digital tool. Participants identified seven broad categories of advantages, of these the most common advantages were the accessibility to either participating or carrying out research that the various digital tools facilitated, ease of use, convenience for participants and staff, and time savings of using digital tools compared to paper-based methods. The most common disadvantages found were the potential for digital exclusion potential technological issues and software limitations (see Table 8 below for a summary of themes). 

Table 8: Key themes of (A) advantages and (B) disadvantages of most effective digital tools from the research staff perspective, with example extracts from the survey

Next, participants were asked to provide free text answers to advantages and disadvantages with regards to their least effective digital tool. 

Table 9 - Key themes of (A) advantages and (B) disadvantages of least effective digital tools from the research staff perspective, with example extracts from the survey.

Next, participants explained their reasons for selecting their most effective tool (Table 10) and their reasons for selecting their least effective tool (Table 11). 

Table 10 - Key themes summarising reasons most effective tool was selected.

Table 11 - Key themes summarising reasons least effective tool was selected.

Summary of most and least effective digital tools results

• Most tools referenced were from ongoing clinical trials, with the largest number digital tools were referenced from data collection (most effective digital tools) and participant recruitment (least effective)
• Most tools were widely used in an organisation (opposed to a bespoke tool) 
• A large proportion of participants were ‘not sure’ about the cost of the tool.
• Approx. half participants received training for their digital tool examples.
• Most tools deemed somewhat or highly essential to achieve goals.
• Most tools deemed somewhat effective integration with other systems.
• Large proportion of tools were deemed somewhat/easy to use in terms of tech literacy. 
• Participants selected a range of 1-5 goals for using digital tools, the majority selected 3-5 goals.
• The most popular research goal for using a digital tool was to “Reduce time/Increase efficiency”.
• Participants selected a range of 1-5 users for using digital tools, the majority selected 1 user.  
• The most popular users of digital tools were research participants (most effective digital tools) and research delivery (least effective digital tools). 
• In context of most effective tools, on a scale of 0 - 10 (0 no more effective to 10 much more effective) most participants rated their tool 7 and above.
• In context of least effective tools, on a scale of 0 - 10 (0 = no less effective than other tools, 10 = much less effective than other tools) just over a third (35%) of participant rated their tool 5/10. 

Research and Development staff survey

A total of 25 people completed the survey (Table 12), of which three participants answered ‘no’ to having not used digital tools; three participants provided reasoning for not using digital tools (selected from drop-down menu of answers). 22 participants partially or fully completed the survey. Participants were informed they would be asked about digital tools used in R&D. 

Table 12 - R&D Survey Participant Characteristics

A block of questions was repeated for three different areas of clinical research studies: set up, recruitment and management. Within each block participants were asked to first provide examples of digital tools used at that stage (for recruitment one participant only answered up until this stage). Next, participants (n=21) were asked to provide a digital tool they deemed to be most effective within that particular stage and subsequently answered subsequent questions about this example (Table 13). 

Table 13 - R&D staff summary of digital tool use

Table 18: Most effective digital tools examples by research stage used by employees working in R&D

Next, participants were asked to provide free text answers to advantages and disadvantages with regards to their most effective digital tool by R&D research stage. 

 Advantages and disadvantages of R&D setup tools 

• Advantages 

Connectivity was found to be a major strength, with sharing of information, accessibility on any PC with a NHS log in and the option for all those required within the study being able to access relevant information, e.g., 

• “Accessible on any PC with a NHS log in can be shared with any use within the trust”.

• Disadvantages 

The disadvantages presented were related to the potential loss of information, and the need for training, and initial set up, e.g. 

• “Would be easy to lose information”. 

• “It is a completely different way of working and managing folders so did require training”.  

 Advantages and disadvantages of R&D management tools 

• Advantages 

R&D management found that their digital tools aided with connectivity. There was “quicker sign off for projects with instant notifications to all parties”. It was also stated that there was benefit from the production of visual data in the form of “reports/charts/graphs from audit”. The quality of data has been improved as there are recordings instead of a second facilitator, therefore “all communication verbal/non-verbal/contexts can be accessed again in analysis”.  

• Disadvantages 

The main disadvantages provided were initial training period required, time taken to use the relevant digital tools and need for good communication and integration within existing systems.  

• “Some staff require support, and ICT training to use, change of practice can take time”.

• “The amount of editing the transcripts needed to anonymise and make them accurate”. 

Advantages and disadvantages of R&D recruitment tools

• Advantages 

The use of digital tools such as social media were found to have wide reach, with different populations that could be targeted, and was easy to do.  

• “Free to use & widely available” 

• “Wide reaching when using the Trust social media account” 

• “Easy and quick to do” 

• Disadvantages 

Ineligibility of people, risk of selection bias and time taken for posts to be uploaded on trust social media pages were disadvantages provided by the participants.  

• “People will sign up who have an interest in something they want to share about the study topic. Risk your data being not representative”.

• “A number of patients were not eligible for study”.

“As this a Trust managed account, it can sometimes take a while for our posts to be uploaded”.

Summary of Research and Development survey results 

• Most examples were provided for management of clinical studies (n=11)

• In terms of learning about tools, most cited reasons: set up (recommended by others and for recruitment/management (developed in house). 
• Across stages, approximately half of participants reived training on the examples provided. 
• The training period ranged from less than a day to over 5 days with most training taking place under 1 day. 
• The majority of tools were widely used in an organisation (opposed to being a bespoke tool)
• With regards to cost information, most participants had either no information relating to costs or there was no fee associated with the digital tool (open source). 
• In terms of cost effectiveness ratings, for set up/recruitment tools scores were evenly spread. For management, participants rated scores on the upper end of cost effectiveness

Participant and carer survey

Four participants completed the participant/carer survey (Table 14), all of which were individuals who had previously participated or were currently participating in a clinical, health or social care research study (none of which were carers).  

Table 14 - Participants/carers demographics  

Participants were asked to select digital tools they had encountered from a drop-down list, participants could select multiple answers, the number of digital tools used by participants ranged from one to five for each participant. The most frequently selected digital tools were completing an online questionnaire and a mobile app for consent or provided consent electronically (i.e., website) (see Table 15 below).  

Table 15 - Types of digital tools encountered by participants of clinical, health and social care research studies (multi-responses).  

Participants were asked on a 7-point scale, ‘What would you (or someone you care for) prefer when participating in a research study’, with (1) representing “All Paper-based and/or in-person methods” through to (7) “All digital and/or remote platform methods”, with responses ranging from 2-7. 

Participants were asked ‘Do you think the overall experience for?participants?would be worse or better if more research studies included digital methods?’ (1= much worse through to 7= much better), answers ranged from 2-7, with the majority of participants believing the experience would be better if more studies used digital tools. The same question was repeated for carers, with most participants believing that the experience would be better for carers if more studies used digital tools. 2/4 participants (50%) provided three of each category: advantages, disadvantages (Table 16), and barriers (Table 17) when considering a prospective participant/carer joining a clinical research study that utilises digital tools.  

Table 16 - Advantages and disadvantages towards joining a clinical research study using digital tools identified by previous/current participants of clinical, health and social care studies 

Table 17 - Barriers towards joining a clinical research study using digital tools identified by previous/current participants of clinical, health and social care studies 

Survey results summary

The online survey provided an initial overview of which tools stakeholders based in the CRN West Midlands have used alongside a number of insights about these tools. The online survey results were aggregated and presented descriptively.

The online survey results offer a snapshot into experiences of tool use. The main aim of the online survey was to act as a recruitment tool for qualitative interviews. Qualitative interviews were conducted to expand on the online survey results and understand about participants experiences with digital tools in greater detail e.g., why their tools were deemed effective/ineffective, were there any barriers to use, advantages/disadvantages of tools. 

Results - Phase 2: Qualitative interviews

A total of eight participants took part; five were participants who had completed the survey for R&D employees, (two of which stated on the survey that they had not previously used digital tools), three participants had completed the researchers and related staff survey. Participants were employed by a variety of organisations in the CRN West Midlands area. 

Ten main themes were identified across the transcripts, see Table 18 for summary of themes. Each theme will be described below and substantiated with representative direct quotes from the transcripts. 

Table 18 -Summary of themes identified.

 Theme 1 - Definition of Digital Tools in Clinical Research 

Across all participants there was a consensus that a digital tool used within clinical research was a device that can connect to the internet, or a form of online technology. The main reasoning behind using a digital tool was to make research more efficient. 

“I think a digital tool is anything that you can use online basically. So, it can be a handset, it can be a tablet or a website…it's anything that can connect to the Internet that can ease the process of undertaking research.” 

“A digital tool is something what you know, obviously online or whatever, or a device that is, you know, working that produces something or helps you do something helps you. Either by quickening up a process, making things easier, recording data, recording all sorts of different things.” 

Although digital tools were mostly viewed in a positive light across participants, one participant questioned whether digital tools did in fact improve research all of the time.

“I suppose it's anything that uses technology to either hopefully improve things, whether it does all the time is a different question.” 

Participants frequently referred to widely used tools such as those implemented by their employing organisation, for example MS Teams. Participants also referred to bespoke digital tools that were specifically designed to fulfil a local need. One participant defined a digital tool in the context of designing a digital tool. They explained the aim of designing a bespoke tool was to increase automation of research processes to enable nurses to spend more time on clinical activities.

“So, a digital tool would be for me is mainly as built around efficiency. So…anything that we can do in terms of automation or anything we can do to assist nurses in spending less time with non-clinical activities naturally giving them more of an opportunity to get back out on the ward whether that be like recruitment or direct patient care through the clinical trial.” 

References were made to digital tools used across different stages of clinical research, for example participant recruitment, management of research studies and intervention delivery. It appears that some digital tools are primarily research participant facing, whereas some may only be used by the research team. There was recognition by some participants that tools used differ depending on someone’s job role. 

“I believe one has to distinguish between the tools to carry out the research and the tools are interventions like digital health interventions. I guess in terms of tools to carry out the research, one can consider things like teams or online systems for video conference as digital tools that can be used to engage participants. Or one could use social media for example as a way of advertising the project.” 

“I suppose it depends what element part of R&D you're working in…So for me digital tools might mean something different to perhaps a research nurse who works on delivering research.” 

“Something that you would use to either aid recruitment of a participant into a study or that sort of the management and oversight of every search study.” 

Contrary to the participant responses above, one participant offered a broad definition of a digital tool in clinical research. This participant considered different types of tools which are used for different stages within research projects ranging from management to data collection. 

“It helps make it easier to collect data…easier to track where research is [for example] digital solutions for portfolio management and I know the CRN [Clinical Research Network] uses Edge and CPMS [Central Portfolio Management System]… And then thinking about the patient facing role of digital research…giving devices to patients or using patients own devices to actually report outcome measures directly to research teams…using technology, whether it's Internet based technology or sort of phone, tablet, laptop-based technology to hopefully improve research even like an e site file system or a trial master file or trial master file system or just using teams and SharePoint, this is a digital solution.” 

Theme 2 - Impacts of the COVID-19 pandemic on digital tool use

Although many participants stated they used some digital tools prior to the COVID-19 pandemic, there was a good level of agreement that the pandemic had rapidly accelerated the use of digital tools in research. 

“It sort of from my perspective sped up the development of them because you have to act quickly and get them to do things you weren't doing before. But we were definitely having a push towards that direction of travel.” 

Many participants reported that using digital tools were essential for carrying out research projects during the pandemic due to the guidance imposed at the time which prevented and/or limited face-to-face contact with others. Participants described how changes to research had to be implemented quickly, moving from paper-based methods to digital methods (e.g., paper consent forms to online consent or data collection via online questionnaire platforms or MS Teams).

“Obviously the whole point was that we needed to reduce foot fall into the NHS. GP’s weren’t seeing patients face to face so obviously we rolled out the e-consent which meant that patients wouldn’t have to come in and sign a consent form. And that's probably how it's affected it, it's just pushed us to be even more remote.” 

“We wanted to continue the research where we could do so…we had to move it online. And people just embraced that. We had our IT team. We had enough resources to get everything onto online platforms in order to collect the data on an online questionnaire as opposed to a paper questionnaire.” 

“We made an adaptation to the intervention, but also we made adaptations to the research process, so we enabled consent to be taken online, we enabled… people to be interviewed or to be assessed in [a] quantitative assessment online via teams or by a phone, so that we obviously allowed people to participate even there was a physical distance directive in place.” 

There was a consensus that digital tool use is now accepted by many staff and research participants. One participant explained how the COVID-19 pandemic enabled them to demonstrate the positive benefits of using digital tools and as a result change the viewpoint of nurses towards digital tools who may have previously been more resistant to using digital tools. 

“We proved that the digital way is working, so that's massive. It definitely gave us the push needed cause we had no choice. People had to start using digital tools and actually once we got through it, you go, oh, this is actually really good…so it gave us the opportunity to kind of change best work in practice and actually through doing it, that's almost the consultation period during covid…covid is not finished but obviously now hopefully it’s started to resume back to some form of new normality. You go actually, we're not gonna go back to that way before because this way works really well…And it gave us a lot more buy in, particularly with some nurses who may be resistant to change.” 

Theme 3 - Perceived benefits of digital tools

Perceived benefits for participants of clinical research

A benefit identified by many participants was the convenience of taking part in an online study for research participants. Participants perceived this method of data collection made it easier for research participants to participate compared with physically visiting a location or completing a paper form and returning it via post. However, it was implied there may be individual differences when using digital tools. For example, participants who are comfortable with IT may find this method beneficial but for others this may not be the case.

“For the patients that are happy and IT savvy and things it saves a trip to the post box or whatever to fill in the paper questionnaire if people are happy to do it, especially because it's like transferable, you can answer it on your whether it be a mobile phone or a tablet or a PC.” 

Other advantages of digital tools included catering for individual differences of research participants. One participant explained they conducted focus groups via MS Teams. They found that some research participants were receptive to using the chat function on MS teams (where a question response is typed rather than spoken aloud). The participant reflected that some people who may have felt uncomfortable speaking in a group may prefer to contribute in this manner. 

“You can ask people not only to talk but also to use the chat, which I think helps some people who are bit kind of less able to interact in a group. Sometimes you hear from people that you would not normally necessarily have… the kind that might feel kind of worried about talking in public, but they can use the chat so [they] can still contribute to things in groups.” 

There was recognition that offering research participants the option to take part in an online study, may enable more people to take part in research projects where they couldn’t previously. One participant explained that research studies previously using exclusively non digital methods may have meant a certain demographic were more likely to take part i.e., visiting a physical building might require money, time, and ability to travel. It is possible that collecting data via digital means may result in a more diverse group of research participants. 

“The advantages are that we potentially can reach patient populations that wouldn't normally take part in research, those that potentially have a lower mobility, aren't able to travel as much could actually take part in the trial. And it opens it up to people that actually in the past, I think research has a problem where actually you tend to get those people that have a lot potentially more time on their hands, they're potentially more affluent, they potentially have to work less, they’re maybe even closer to retirement or are retired and therefore you sort of limit your patient population whereas and obviously not able to say if it has been able to do this erm but hopefully that the tool can open things out to people that couldn’t previously take part in research”.

Perceived benefits for staff                                

Nearly all participants referenced that using digital tools is a more efficient use of time. For example, using digital tools to create study materials and help manage research studies saved time for clinical staff. This was achieved either by clinical staff spending less time on admin tasks and more time with patients or using digital tools to carry out tasks that previously would have been carried out face to face and as a consequence, reducing the number of visits with patients. 

“You know it has reduced the number of times that district nurses have had to go and see patients in the community.” 

Digital tools were evaluated in relation to paper methods. A participant reported a benefit that online research materials can bring to both research staff and participants. They explained research staff can create a drop-down box for participants to select the language they require for study materials rather than printing research documents in several different languages. 

“It’s easier on an online tool to have a number of different languages …if you’re printing off something that’s a hell of a lot of paper…and you’re sort of sometimes guessing what language people are wanting things in. If there’s a drop-down box that could all of a sudden change it to French or you know Arabic, it’s a much easier thing to be able to do.” 

Another benefit identified was the ease of connectivity between researchers and collaborators. One participant explained WhatsApp had been used to communicate between people working on a research project as well as a means to support colleagues. 

“WhatsApp has been used to keep contact with researchers between meetings, which I felt was quite effective…so it helps to have a WhatsApp group and say what are you doing? Can I help? It helps between meetings to keep things going. But I think it would be more within research teams [rather] than a way of collecting data from participants… It's also to…kind of push the progress through in between meetings to make sure that people kind of are reminded about targets and so on... I mean obviously like there’s a limit to what people can do but I guess it’s good to have this kind of continuous kind of messaging through between meetings so people…remain motivated and engaged.” 

It appears that some digital tools have multiple purposes within research. For example, as described in a previous section by one participant, MS Teams was used for facilitating workshops and focus groups with participants. Another participant described the benefits of using MS Teams whilst working with colleagues from different organisations based in different locations, specifically in relation to supporting the management of research studies. 

It’s meant that actual collaboration between the two of us is a lot easier…and to be fair, it’s enabled us to actually support our sites a lot better because of the rollout of teams.” 

Theme 4 - Perceived drawbacks of digital tools

In an earlier section, participants discussed the ease of meeting with colleagues online, especially when based in different locations and organisations. However, there was a recognition that meeting colleagues online does not always enable people to strengthen or develop relationships in the same way as meeting face to face allows. Similarly, conducting research online may create issues with understanding expressions from participants as explained below.

“In my experience online is good to get a lot of work done rapidly but sometimes you can't read face to face interactions. You can create maybe bonds that are bit beyond the business like you can have a coffee together so there’s a little bit of a difference there.” 

It was highlighted that technical issues may be encountered when using digital tools. Examples cited include a poor internet connection and that researchers may require additional technical support if something were to go wrong. However, it was noted that participants may also encounter issues and require support when conducting face to face research. 

“Some participants might struggle with connection and so on. So, there's a number of technical issues that [could occur] but again that’s very similar to a logistical issue that you would have in a face-to-face workshop.” 

A further challenge identified was staff and research participants learning how to use a new tool or system. However, participants explained that training was often offered to participants. 

“The further negatives are probably people having to use a new system if they don't use it that often then potentially forgetting how to use it. But obviously we've tried to … reduce that by offering videos, offering…training sessions.” 

A further drawback raised was engagement with studies that use digital tools. A participant below explained there is limited knowledge around which type of participants engage with research utilising digital tools. The benefits of research using digital tools was mentioned in a previous section of this report. Some participants believed digital tools provide research opportunities to a greater pool of participants. On other hand, some participants believed that research using digital tools might exclude other research participants from taking part. One participant explained below they were conducting research to investigate engagement with digital tools for recruitment and retention. 

“I think drawbacks are still engagement and I think this is what we're trying to investigate at the moment… what is engagement like across different participant groups and age ranges and demographics and ethnicities and stuff like that… I'm not sure it is inclusive.” 

Theme 5 - Selection of a digital tool

No choice in digital tool selection

A number of participants noted they had little choice in the digital tools used within their research projects. Some participants explained they used tools provided by their employing organisation, with the main reason due to cost. Despite having no choice in the digital tools that were used, participants did see some benefits of using a digital tool that people were already familiar with.

“It was basically chosen because it was free, and that's what the university had.” 

“I know that there are things that you can buy…and sort of add in and utilize for screening and things. However, I think because of the complexity of the hospital and things, it's easier to have something that is widely already in place rather than actually look for something external and then have to fund that.” 

“The tool is provided by the university… since it is the one that we normally use every day then obviously one is a bit more comfortable with it compared to other tools.”

One participant explained that although they didn’t choose the specific tool (REDCap), they did provide input with adapting and testing the software to meet the needs of their specific project. 

“So [employing organisation removed] host the REDCap and then different researchers can change it to do different things and so I was involved in the design…and the testing of that and then obviously the rollout but the actual selection of the REDCap software I wasn't involved in.” 

Current methods of acquiring knowledge about digital tools

Participants described a variety of methods in which they discovered information about new digital tools. Talking to other people was a popular method, some described asking colleagues for advice directly about tools. One participant working in R&D explained that they expected their colleagues would go to the R&D team initially before asking friends or colleagues for advice. Others explained specific scenarios where they have learnt about tools. One participant described how they had attended meetings with colleagues from across the West Midlands region where case studies of effective digital tools had been presented. 

“Things tend to be sort of by word of mouth I think a lot relies upon the fact that you happen to have a conversation you know when somebody says ohh, hold on a minute you could do it this way or you could use this and it's probably the current way things happen.” 

“I think we've got our Principle Investigator's pretty well trained now to come to us with their queries first of all, rather than, you know, going out and asking their friend who works in Newcastle or whatever it is. So I think in the first instance, I would expect people to come to the R&D team and if we don't know the answer to that, then we can sign post most appropriately.” 

“You go to the people you know…and so the network itself is really good cause we've got sort of 25 trusts … we go to the managers forums and all sorts of things. And if I can then phone [and ask] what are you using for this? I've got this issue... I went to the industry group…and one of those was around the fact that this particular trust is using Power BI behind what they're doing. It was a different perspective, and it is like a case study. They showed you what they can do and what's happening and everything else. And it's sort of sharing that good practice.” 

Recommendations for selecting tools in the future 

Participants were asked what would help researchers when deciding which digital tool to use and where this advice might be best placed. Participants proposed there needs to be a process to follow when choosing a digital tool as there currently isn’t a known standard procedure to follow. As mentioned in the previous section, one of the main ways people currently find out about digital tools is via word of mouth. However, there was a recognition that this method cannot be standardised. Some participants felt that often research staff are not often aware of digital tool options available at the start of a project and this sometimes may result in suboptimal choices. There was a sense that the whole process required demystifying, alongside a clear explanation/description for each digital tool and how they can be used in a project. 

“Just demystifying the use of them would be helpful. You hear the term digital tool, that's quite fancy sounding, but by breaking it down into what that actually is. So it's an electronic CRF [case report form]…you just access it via this website and then people can be like, ok, I can do that or if it's a patient reported outcome measure, maybe it could be done online. You could do it via your phone and just explain things a bit better to people, I think it all just comes down to communicating and making sure what you're trying to tell people is really clear... And I guess it’s just knowing which tools are right for the job and for each occasion? It might just be you only have one tool that you can use.” 

“From previous experience, I think people get so fixed on I've always used this and this must be what I will use. However actually you know there’s times when you need to step back and think what is it I want it to do…what do I need from this and is that the best thing for it? So there does need to be a process…I've seen recently where you've got people who are using digital tools in their research which are not really fit for the purpose, but because they're not aware of other options when they start the research… So, I think there needs to be a lot of awareness [of] how the digital world can help and support research that bit is sort of missing.” 

One participant suggested a checklist that people can select their needs for a project and then a program could recommend appropriate digital tools. 

“Almost a digital checklist and then it points to what program, even if it doesn't say what the program is, it tells you the type of program you need…depending on where you are with it and what you're doing. Are you the sponsor? Well, if you sponsor a study you'll need a very different [tool] to if you’re delivering the study… is it multi-site? …So I think if that was a process in terms of give you an idea and point in the direction of what you need, it would be useful.” 

One recommendation which participants strongly agreed on was to provide prospective digital tool users with case studies of effective tools that have been used in previous projects. As explained above, it seems knowledge is important when selecting the most appropriate tool. One participant suggested sharing expertise about digital tools across the West Midlands region. 

“Case studies [are] really good…I think a website probably would help.” 

“I think it would be good to have case studies of people who have used some tools and made their own adaptations [for a] service user or participant group.” 

“If you don't know what's available you, you don't know what you don't know. There could be a digital solution to the problem you've got but if that's not your background, if you haven't encountered it, you wouldn't know to search for it.... I think a checklist...suggestions or case studies essentially and almost you know sharing best practice across regions…It could be national, you've got NHS digital they possibly have a repository of research tools that we've got access to free of charge.” 

In addition to checklists, case studies and a repository of available tools, suggestions were made for more specific information to be widely available in a public domain. For example, participants suggested making benefits and drawbacks of various tools more publicly available as well as guidance on what the best type of tools for specific stages of research and stakeholders. 

“It would be helpful to have guidance for research of what are the pros and cons of different tools. I believe [the] university provides training for using different tools. So, I guess kind of from the research, what is probably missing is what are the best tools for particular types of research or for particular types of stakeholders.” 

There was a sense that providing research staff with information about digital tools could be distributed more widely than the organisation they work (e.g., University) or on a regional/national level. 

“It might make sense to make it more widely available than our university.” 

Several participants mentioned the idea of a wider network/community to share digital tool ideas and learn from others expertise. 

“We've got things like the Research Design Service…the idea is that if you have a research question, you can go to the research design service and they will help, put you in touch with statisticians, health, economics or whatever you know. Maybe we need something that sort of signposts people to digital element of it…and sort of a showcase you know these are actually some of the things that are out there, these are the expertise we have in the region would be really helpful.” 

“A community of practice might be helpful because then you can bring people together and people have different experiences…and then you can kind of exchange information. So, I guess developing a network of practitioners working [on] research using tools… that might help to improve the way we use them.” 

Theme 6 - Barriers

Barriers to implementing and using digital tools

A number of barriers to using digital tools were identified including the attitudes of research participants towards digital tools. One participant explained that despite collecting data via digital means during COVID-19, some research participants were still more comfortable with using paper methods. A participant explained some research participants were concerned about clicking on a survey link, fearing whether it is a legitimate survey. This concern appeared to be heightened among an older age group of research participants as explained below. 

“It feels like people were happy to make the move because you couldn't [use] paper. But following COVID some people have gone back to paper…especially older people […] they said that they'd be worried about clicking on a link and whether it was, you know, bonafide or not…whether it is safe to click on the link.” 

Attitudes of staff towards digital tool use was also commented on. Indeed, some participants referred to the fact that some staff have negative feelings towards using digital tools. Indeed, some participants believed these attitudes could be due to a lack of experience with digital tools. 

“I would say sometimes, perhaps it's the staff who would be using the tools. It's their lack of perhaps confidence in wanting to use the tool or a lack of knowledge perhaps. And that can be quite a barrier, especially if they're going into it with the kind of attitude that says ohh no…I can't possibly learn how to do this and that's a really frustrating attitude to bump into especially as you know you can provide as much training as you like, but there's still gonna be like, no still not interested in learning.” 

“Digital tools are there for everyone. Some teams grab with them and run with them really quickly. Others are really resistant still and I think it's what you know. And it's not clear cut on age at all, but there definitely is a demographic that can feel more comfortable with it, and others that don’t and I think that comes down to experience.”

Attitudes of staff were also considered from all levels of seniority. One participant noted that the approach managers take towards digital tools is key to whether they are embraced by other staff. 

“If you get the most senior member of staff on board and they see the benefit the rest of the team tend to follow but if you get resistance at the top, like with any change management it filters down and then there's resistance throughout. So user readiness is probably the biggest one.” 

Healthcare staff were mentioned regularly in relation to using digital tools. For example, participants commented on the lack of opportunities for certain staff, particularly nurses to use digital tools. There appears to be conflict between carrying out a patient centred role (e.g., nurse, clinician) and using digital tools in order to carry out research activities. It appears that nurses are expected to know how to use digital tools despite not necessarily having sufficient knowledge or time to do so. Another participant commented on clinicians perhaps not knowing about or planning for digital tool use adequately, they referred to clinicians whose priority is to be with patients. 

“They are trained as nurses first and foremost…I certainly don't imagine they have a lot of time to sit down and draw massively complex pivot tables or anything like that. Their first priority is always gonna be their patients and you know, hear stories about them working super long shifts, they don't take a break for the entire time. When are they gonna have time to sit down and play with Excel.” 

“I'm not being disrespectful to nurses because I was a nurse myself. But you know, nurses and others aren't always as techie as us that are using it all the time, so it's got to be really straightforward. It's got to be fairly easy to use because if it's not easy to use, people don't use it. Also, they think I haven't got time now and then they forget to go back to it and you know, it's only as useful as the data that's actually put into it.” 

“I think a lot of it is lack of awareness of what is out there, but also not having that foresight to think this might be what I need… particularly clinicians will just sort of focus on I just need to recruit a patient and get them into a trial without actually thinking about all the other things.” 

Several participants commented on resources as a barrier to using digital tools. Resources identified included technical barriers (e.g., internet connection). However, there was a recognition that issues may also occur when facilitating a face-to-face workshop. Lack of physical resources such as not enough hardware or hardware that doesn’t work was also highlighted as a barrier for use. 

“Taking an iPad to clinic to do Proms (patient reported outcome measures) or anything like that wouldn't be very good because the broadband provision isn't that great. So if you're too far away from wherever the main hubs of the broadband is you can't get the signal, or if you go too deep into the hospital, you can't really get a signal because the walls are too thick. 

Sometimes it's just simply something as easy as that that means it's not feasible to use the tools… other barriers can just be the lack of actual hardware. So, if there's not enough handsets…or if the handset keeps resetting itself… if it doesn't hold its charge and you've given someone one with the duff battery for them to take away. So sometimes they can be a real nuisance.” 

“Some participants might struggle with connection and so on. There's a number of technical issues…but again that's very similar to a logistical issue that you would have in face-to-face workshops.” 

Participants appear to agree that the size of an organisation’s or department’s budget often dictated which tools were available for use. 

“If they've got no budget for IT, then they'll need to whip up an Excel spreadsheet.” 

There was a recognition that expertise was required when using and developing digital tools which presents additional costs. This was also connected to the size/resources of organisations which appeared to vary across participants. For example, one participant noted they went to region wide meeting and discovered a bigger trust had digital tools whilst their department did not have access to the same resources. Participants also mentioned costs for licences as well as staff and maintenance costs. There was a sense that although investment in digital tools is expensive, it would save on staff time in the long term. 

“That's where we find out that these bigger trusts have got all these zany things that are going on behind and you think, oh, why can't we have something like that? Because it makes life so much easier…some of it is about expertise in putting those tools together in the first place. You know, we've got ideas as to what would make life easier for us, which ultimately makes life easier for staff and for patient and, you know, end users. But it's the expertise behind that… I know what I want and I've seen people's, you know, presentations about what things can do…we don’t have the expertise behind it so we would be expecting other people to do it. So, the barrier really is sometimes the experience of putting your idea into something that will actually work. This cost is always going to be a barrier in the NHS…if we don't invest in something, how are we ever going to move forward… and so if we invest in there, then actually we might be able to start, you know, save on staff time. So I think the biggest barrier is expertise.” 

“There will be an ongoing maintenance and as things evolve and we want to add more to it or use it in different ways so that element of it, the resource element of it are the main sort of barriers that we face at the moment.” 

Overcoming barriers to digital tools

Some participants discussed ways in which they had overcome barriers to digital tool use. One participant explained they had conducted a research study to test out different methods of participant retention. As discussed previously, some participants had expressed fear of receiving an electronic survey link. Within the study, some participants were sent a text message (SMS) to inform them they would receive an email containing a survey link to reassure them that it was a legitimate request whereas some participants did not receive any notification at all. The participant found no difference in participation rates between the methods.

“I did a questionnaire study and used 18 practices…I sent a prenotification text to nine practices and nine practices I just sent the link to the study and there was absolutely no difference in participation rate.” 

One participant recognised some research participants may not want to switch on their video when communicating online. To accommodate this, the participant explained a flexible approach was undertaken by which research participants could take part using audio only.

“We keep an open mind as to whether people want to show their face or not. Because some people are not comfortable with showing their house, you know it’s quite invasive in a way. So, I think [if] they don’t want to show where they live and if some people want to keep the cameras on, we say we prefer them to keep the camera on. But if they want to keep off and just participate by audio [we’re] also quite open to that.” 

It was further noted that a barrier of digital tool use for some research participants is not having access to the internet, digital equipment (e.g., computer) or not wishing to use digital tools. To accommodate this and prevent digital exclusion some researchers have adopted a hybrid approach in which they use both digital and paper-based methods. There was agreement that although COVID-19 had increased use of digital tools, some research participants stated a preference for paper-based tools, participants perceived this preference was higher amongst older research participants. 

“There are some people who might find it more convenient, so you will never be able to engage them in kind of face to face because of convenience and time and so on. But also, there might be people who struggle to access IT or digital tools because of literacy. So, I guess any research that considers using only digital tools to collect data there might also be some level of thinking about people [who may be excluded] and to have other strategies to try and include them.” 

“We've got an online feedback survey that asks patients what their experiences were of taking part in research, did they feel informed? Were they informed of the outcomes and all of those different things...But most of our patients seem to be an older age group weren't able to use the technology. They don't all have, you know, smartphones, they don't all have access to iPads…so the numbers dramatically went down in the survey that year. And as soon as we were able to start printing out, sending things out to people, it's gone back up again. So, we do need to have dual.” 

“The negative is that there's a potential that maybe people who aren't technologically aware would struggle to take part in the study. We took the view that the COVID-19 pandemic made people potentially more technologically aware, people are more likely to be using zoom, more likely to sort of use QR codes, lots of different things that the COVID-19 has done potentially that would help. And also we always say that if someone hasn't got the sort of technology to take part, then there are ways that relatives can be the people that we send things out to, that actually their GP can support them so instead of a GP sending something out, you can almost take that remote nature away and patients can come in and complete it on a practice computer and then actually they can come in every six months, do their follow up questionnaires so there is that, it isn’t an all or nothing because this is a remote study it has to be remote no you can actually take that remote nature away and just have a patient complete it on a practice computer.” 

However, not all participants were positive about a hybrid approach. One participant explained how due to uncertainty about participant response rates for paper vs digital methods, a hybrid method towards recruitment and retention was adopted. However, they explained that using paper makes data management more complex for research staff. 

“The advantages are it's quicker and it's not as costly and things it's like we costed up for a study to send a massive paper mail out and the paper mail out alone was costing quarter of a million. And I said, well, if we did it by text message… it costs like next to nothing. So obviously that sways them. But then they're like, yeah, but what sort of response rate are we gonna get? So, which is why therefore all of our studies now are being designed on multiple platforms. So, we're doing this text message first and you know online first be it whatever route. And if that doesn't work, then mop up everybody else with paper and then you get data in all sorts of different places and it just makes the whole study more complex…

…a lot of our studies that have been traditionally done on paper, or might have even been started on paper, then have moved to online platforms and then you've got participants who are on paper and on digital platforms, so the most chaos it's causes for the data management teams because all data was on one platform. But with the hybrid of platforms now there's lots of data everywhere, so actually it's increased the amount of work for data cleaning and validation and things like that, and where all the data is and the study management systems probably haven't been set up in order to cope with the changes. And there wasn't enough time to design them in order to cope with it. So at the moment with data that was collected over sort of the COVID period and with lots of different methods of collecting the same data. There's a lot more work now it feels in data cleaning for the department.” 

Theme 7 - Training 

The majority of participants stated that their department provided training on how to use digital tools. Training was provided using different methods including demos, videos and face to face training sessions. Participants explained that they had provided training options and support to a number of different stakeholders, not simply those who work in research. 

“We've tried to make it as easy as possible with multiple different sort of training, so we offer training sessions, we offer training videos that they're able to watch themselves, we offer slides, we basically offer whatever type of training may want we’ll offer them and potentially trying to bring people with us that way, just extra supporting them as much as possible.” 

“When we first set up a site, we provide training on how to enter data on the REDCap system. We also show them what patients have to fill out as well. So then if needed, GP’s could support their patients. We also offer support to patients as well if they need it filling out the forms…the university also have a sort of REDCap community as well where people bring in share and problems that sort of and they discuss and see if other people know how to get around them. So that's sort of training sort of I've had we provide sites and then obviously any new staff that starts here I train them on the system as well.” 

“So, because it's kind of built and put together in house, we've got a PDF guide that we put together. But what I tend to do is that I will go to the team, whether it's on MS Teams or actually in person, and I'll just do a 10-15 minute demo.” 

On the other hand, it was noted that training can be generic and not specific to features of digital tools that might be useful when carrying out clinical research. One participant explained they learnt how to use MS teams through experience, they observed that learning on the job led to them discovering new features such as the ‘raise hand’ function on MS teams. 

“We did receive the general training for teams but to do workshops on teams, I think everybody had to figure that out at that time there was no training available then… training on specific tools might not cover what you do when you use it for research.” 

Theme 8 - Cost efficiency of digital tools 

No participants could provide exact details on digital tool costs. This was mainly because many of the tools were paid for by the employing organization (e.g., a university) and therefore participants did not have access to this information. One participant did disclose the significant differences in cost between contacting people using a paper mail out compared with text messaging. There was agreement that digital tools were more cost effective than using paper-based tool across various stages of research. One participant compared carrying out workshops/interviews with research participants via MS teams or in person. They highlighted indirect costs such as savings in travelling to either facilitate interviews with research participants or with colleagues. 

“I'd say people can see the advantages and the advantages are it's quicker and it's not as costly and things…we costed up for a study to send a massive paper mail out and the paper mail out alone was costing quarter of a million. And I said, well, if we did it by text message or whatever, it costs next to nothing.”

“I think the saving is certainly that you reduce the amount of travel. And potentially…since now teams has been approved by ethics for recording interviews that also saves more in terms of having to buy recorders. And so there's quite a lot of savings there.” 

The costs of designing a bespoke tool were discussed. It was acknowledged that the staff time to create the tool is costly. However, there were a number of benefits to this initial cost namely that correct management would ensure that patients completed a study on time.

“There's no cost obviously apart from the time of the staff to build it. That's back tenfold in the time it saves in managing patients and keep them in line with the protocol. If a patient goes outside of [protocol timeline], the knock-on effect with the study is ten-fold and then you've also got the reputational damage to your site. And so it's cost neutral at worst…it's the amount of savings you have and increase in quality of research outweighs the cost, in my opinion.” 

One participant expressed an interest in conducting research into the costs of a trial they had been working on which used digital tools. They expressed their desire to compare the costs of the digital tools with a trial compared with using standard methodology. The participant believed that concrete evidence to demonstrate the cost-effectiveness of digital tools would help them to conduct similar research in the future. 

“No idea how much it costs or management or anything like that. But again, I think that's absolutely something that we as a team want to look at because if we can show that actually this way and digitising research like this can reduce costs because it makes it much easier to do research... it's absolutely something that we'd love to do analysis on in terms of the cost of this research compared to a conventional trial.” 

Theme 9 - Future digital tool use

Views on future digital tool use 

All participants stated they would use digital tools in the future and anticipated the majority of research studies would utilise them. One participant mentioned two projects in their department that planned to use e-consent and that they were anticipating more projects would incorporate digital tools. Another participant explained a willingness to implement more digital tools into their department. 

“We don't get them very often, but I wouldn't be at all surprised if we start seeing a lot more of that kind of thing.”

“So I've said to them, you know, is this the best method to be collecting data? Is there anything more efficient? Is there something fancier? Have we got any investment that we can build up for delivery of clinical research studies? and you know that's what I'm interested in moving forward how can we be more efficient with delivering our research using digital tools.” 

Concern was expressed for pressure placed on nurses to know how to use digital tools in research projects. There was a belief that these expectations on nurses would only increase in the future.

“The technological expectations of nurses has grown massively…you can bet that sponsors and commercial companies are going to try and push to try and use digital tools as well because it's cost saving and efficiency…there is a massive expectation on nurses to be able to just know stuff and some of what the trials asks for. It's not always straightforward.” 

On the other hand, with regards to the general population there was a belief that most people (i.e., future research participants) will use digital tools in the future. One participant believed that increased use of digital tools and resulting familiarity over time might help to reduce exclusion of participants who do not currently use digital tools. 

“All these things will change as people, the population gets more comfortable with accessing digital tools. Maybe in 20 years and everybody will really use them. So digital exclusion might reduce over time I feel.” 

Research to inform future digital tool use

Generally, there was a belief there isn’t sufficient research in the literature to inform digital tool choice. One participant explained that people approach them for advice on using digital tools. Specifically, this example related to which recruitment and retention tools to use and whether to use digital vs paper-based methods. The participant explained that more research needs to be published to help people make these types of decisions. 

“All the researchers here come and say to me…what's your evidence for using it? What’s are your response rates for those via paper and those that answered via e-mail or text message? They just come and ask us what the evidence is and you know what were response rates like in that study? What were they like in that study when you use that method and that method? I just think that needs to be published a bit more so people can make informed decisions around what they're going to use for their particular sample.” 

Some participants mentioned they were conducting research internally to inform choices around future digital tools. One participant explained they were carrying out research in their department to understand the skills of people working in their team/department as well as which training is required.

“So what I've been tasked with is trying to…get a baseline of where we are with digital skills and what kind of systems people are using. And …to see if there's anything we can do to standardize the workflows and you know get some digital SOPs [standard operating procedures] in place. And make sure everyone's singing from hymn sheet. And I'm gonna find out where they're at with their training needs and how confident they are using digital applications. I think in the NHS you have to have a certain level of competence with digital stuff and virtually everyone knows how to turn on computer. But I find that some people nurses more than others, and that's just a generalization lack a bit of confidence… And I think as part of the work I'm doing to find out the baseline of people's knowledge. And I'm also going to be working to put together various training options for people. So just collecting what information is already available and not creating a whole new training package, but certainly just putting together a useful list of training packages that could be utilized. So, whether it's trust, internal trainings such as Excel training…or if it's external training, that could be used….And I plan to test its usefulness by just doing the training myself and seeing how good it is… do I understand this? Yes. Will a nurse understand this?.” 

Gaps for digital tools

A number of suggestions were proposed by participants for areas within clinical research where digital tools could be utilised. One of the main areas where gaps were spotted by participants was management of clinical research studies. 

Two participants commented on how more research studies should use electronic methods to manage their files instead of paper versions. One participant explained electronic file management is particularly beneficial when sharing files across different organisations, but there is a belief that there is still some hesitation among people working in research to use such methods. 

“Every study has a site file and study level has a trial master file and actually using those as sort of e-site files it's definitely increasing and obviously we're doing it, but I think people are still wedded to…paper and still worry about their electronic signatures…you're able to track electronic signatures quite easily, whether it's using tools such as DocuSign and whether it's using tools such as Adobe Pro that allows you to sign documents and track that signature, or really simply just someone emailing to say I have signed this document and it just means that actually everything's there it means that we're able to work across corporations. Obviously working with CPRD, we both need access to that trial master file. We're able to do that as different organizations. We're able to access it, you're able to restrict access to certain sections. For example, there's a section that CPRD should only be able to see…so you're able to limit it that way.”

“R&D, it's supposed to be sort of the fore front of things. I think sometimes we are a little bit slower uptake of things, so to think of trial management …there’s things like electronic and trial master files where documents are kept electronically rather than the fact we print everything off and keep it all in an actual physical file. So, I think there's sort of that digital tools and you can get the digital tools which will help manage that. There's things such as delegation log…so [a] delegation log is a bit of paper…if you're working on a study, you will write down who you are, what elements of you’re doing and you will have authorization from the person who's managing that study the PI that they're happy for you to do those roles. That's still largely paper based, but actually there are digital tools where you can capture that information digitally and we don't widely use those.”

“Archiving is going to be a huge, big issue for us because we have so [many] protocols… so we can have 17 protocols in the life of a study…we have to keep them all for eight years as a minimum for some of them 25 to 30 years. For others, we're gonna run out of space. But if there is a way of getting all that information and having it somewhere [e.g.] a cloud…having it somewhere that it can be stored something that is time proof that if somebody needed to look at it in 20 years’ time… they would be able to get that information. And I think it's quite worrying a lot of research departments [are] supposed to be heading towards being, you know, zero paper by 2040, I don't know how we're gonna do it in research if we don't get something to be able to archive. I mean at the moment we've got a shared drive… we're trying to save things at the moment in there…we're also sending it off to paper as well which doesn't really make sense.”

Future collaboration required between clinical and digital expertise

Two participants recognised that they had access to a digital team which other organisations in the region may not have. Both participants explained that they perceive a need for digital experts to help with digital resources as research becomes more complex. One participant also expressed the benefits of collaboration with nurses in developing digital tools as seen below. 

“Clinical trials seem to be becoming more complex and the data seems to be growing. So, I think there's a real need for digital project managers like myself to come in and assess whether we can bring in any tools to enable that process of…automating what we can. And, as I said, ensure that nurses have a lot more clinical time…I think the need is growing. I think people are more inclined to look at digital support with studies, I think it's changing, and I think historically it was just nurse workforce…nurses would be expected to pick up these sorts of administrative tasks. And I think now we're looking at getting people in who can do it and it's a lot more efficient for them as well…

…I'm really keen for, like [an] open door policy, like the nurses have got something that they think could be automated or if they have a project where it could really help them in their role; whether that's managing patients, managing studies, managing staff, we’re really keen to hear and try these projects out and what tends to happen is we try it with one particular area, we then demo it to the lead nurse or the lead teams, and then if it's good we'll then roll it out to every team.” 

Theme 10 - Most effective tools: case studies

Widely used tools

Participants provided a number of widely used tools in organisations (i.e., not bespoke) as examples of most effective tools in clinical research. These tools were used across different stages of research including study set up and facilitating workshops/focus groups. The section below will discuss a selection of the tools mentioned by participants. 

• Setup and management case study: Power BI

One participant identified Microsoft Power BI as their most effective tool for set up and management of studies. 

“Power BI is software that you can use to present your data in an easy-to-read format and it's very interactive. We’ve used it in our clinical trials, but we've now moved it across to the whole of our R&D space… It's almost like a dashboard…so I can present in a meeting and go this is what the research portfolio looks like. These are the studies that we've got in setup that are open, in follow up that are closed, this is how our recruitment is. We can project our recruitment. We can look at showing how our commercial portfolio is doing to our non-commercial portfolio, whether we've got observational studies, interventional studies, specialty recruitment. At a specialty level or a research delivery team level, it helps them manage their research portfolio because they can see what's studies have got closing shortly, what's studies are in setup, they can see studies which aren't recruiting well and look at what why is that… what are the issues so it's from a governance point of view, it's a really powerful tool to help us to monitor and track things, but also enable our delivery teams to monitor and track their research activity and see what they're doing”.

The participant explained the methods used for presenting data about a portfolio of studies before Power BI was implemented. 

“The way it was sort of done before was a very old-fashioned way, Excel spreadsheets and it was very flat data, you couldn't interact you could just pull data from Edge into an Excel spreadsheet and then in Excel you would create pie chart, but it wouldn't be interactive”.

The participant also described that compared to manually using excel spreadsheet or creating pie charts, using Power BI to do the same tasks was time saving and that Power BI gives live information. 

“Power BI has a dashboard where you can see live information, how we're doing the recruitment, what’s our research portfolio. And so the fact that this does all of that is very helpful and great. It can be refreshed by clicking a button daily, so it's near to live as you can get currently”

“The time it saves…that is a big advantage and the fact you can constantly refresh it at the click of a button”

• Multipurpose case study: Microsoft Teams

A participant working in a research position highlighted Microsoft Teams as their most effective tool.  

The tool appears to be both multiuser (end user included: health staff, principal investigators, research delivery workforce and participants) as well as multipurpose e.g., meeting between colleagues, facilitation of workshops and focus groups. 

“Teams… we have used it quite substantially. Probably the best example is a study in which we did a consultation across the country, we did face to face interviews, focus groups and a series of workshops with different professionals and service users and carers and that helped because we were able to do it at a pace that potentially having to organise face to face meetings wouldn't have allowed…people were able to agree to meetings within one or two weeks because it was an online meeting”. 

MS Teams has not just been used for data collection but also other means, for example online consultation with people with mental health problems. 

“We use for different forms of data collection….we use it to organise qualitative interviewing e.g., focus groups and workshops...we have actually carried out a consultation with people with self-reported mental illness, in which we have developed an online guidance for involving service users and carers people with lived experience in developing digital tools” 

The benefits of MS teams were considered, from a research perspective using MS Teams over face-to-face methods was perceived to be an effective use of time. Furthermore, the participants believed that some people may prefer to communicate via online means as described below. However, it appears that communicating online compared to face to face makes it more challenging to build rapport. 

“There are people, for example, with mental health problems that do struggle with social interaction…one person who had ASD difficulties, all self-reported of course we didn't diagnose them, but they mentioned that they preferred to [take part in] research online more than face to face…so some people actually might prefer it. And some others may not, I guess from my point of view, the experience with online consultation is that we get quite a lot of work done and discuss quite a lot of things. But the difference with offline is that there is less of a social networking side there. So, there's less space for it and it’s hard to organise that in online consultations. But to do the focus group for purely research, they're quite an effective way of using time and don't require travel and so on.” 

Bespoke tools

The next section will provide examples of bespoke digital tools which were designed to fulfil a specific need.  This is in contrast to the majority of digital tool examples provided during in the interviews which were widely used tools. Please note specific names of bespoke tools have been omitted to ensure participants identities remain anonymous. 

• Management case study

Two participants referred to a bespoke management tool as their most effective digital tool. The tool was designed and developed by R&D staff supported by a digital team embedded in an R&D department in collaboration with clinical research staff (primarily nurses). Built in MS Excel, the tool is used to manage patients through a study protocol. Below, one participant describes the tool. 

“It meets multiple needs, so it's we build them individually off the back of each protocol…so when you put the patients recruited date it then predicts all their follow up visits for you in keeping with the protocol. It then has a traffic light system which would tell you when they're due to come up so the teams can see it and have oversight quite quickly of all the participants where they are, what happens at each individual stage. You then hover your mouse over visit 2 and a little info box will come up and tell you a brief synopsis of what needs to be done at that visit. So going quickly, either give you what needs to be done, you can click on it and it takes you through to the worksheets that we create off the back of it.” 

One participant explained the primary reason for developing the tool was to standardise study management. 

“It came around a sort of a want to offer standardised high-quality research and oversight... I’ve worked in lots of different research areas, and they’ve always got slightly different way of working. Some of the nurses and the clinical teams have sort of had to go at building their own Excel spreadsheets. Some of the sponsors and the studies provided them for us and then some used paper diaries. So, but when you look at it, none of them were quite really fulfilling everyone's needs, there were sort of doing a bit here and a bit there and I wanted to standardize it across the whole of R&D because it wasn't quite equitable for our patients and our staff. Some were getting access to. So that's why we standardized and went we’re gonna build these and offer them out across the board.” 

Prior to development of this bespoke management tool, it appears that studies were managed via paper means and/or nurses had attempted to build their own databases to manage studies.

“Nurses that trying to build databases, you know, it's not our skill set, but some can do sort of just a small amount with them and then others were very much using paper which is not the way to do it.” 

Both participants deemed the bespoke management tool to be effective for a number of different reasons including helping staff keep patients to the protocol time frames. One participant explained that the tool can be tweaked to suit needs of the users (i.e., nurses) and new features can be added. They also explained that it has many benefits due to being developed in excel, including tool template can be sent to teams located at different sites and used without the R&D staff visiting that particular site as well as being user friendly. 

“In terms of what it gains and what it offers for clinical deliveries…if you use [tool name] your patient should stay within [the] protocol visit window…and you can plan a lot easier and again should someone go off…someone can come in and pick it up quite easily and you can ensure the patient is safe and kept up to date.” 

“One of the benefits of it is we've built it in Excel. It doesn't look like excel when you use it. Everyone's got excel within the trust, and there's no additional cost apart from the hours to build it, we weren't having to purchase another system. So that was why we chose that setup to build it. And then we're working with other sites and helping them have access. We can send them a template, we don't need to go there…they upload the template, then they can use it themselves. So that's why we built it that way just for that reason.” 

“I think we’re on version seven or eight now, so they're really user friendly and they're really locked down so you can't break them. I've had some nurses do some really weird things and they’ve broken [it] but generally as times gone by, we've ironed out all the things. Before if a nurse dragged a formula away and deletes it, it would affect everything else…but in the department we've [become] more savvy at locking things down. Generally …you should be able to click and use it.” 

A further benefit highlighted by participants is the fact that the tool was developed in collaboration with clinical research staff and has incorporated their ideas into the tool such as automatically generating reports opposed to looking through paper documents. 

“It's made by the people delivering research…I think together we’ve developed the service over the years and the tracker itself and that's what makes it better than an out-of-the-box bit of kit from it or third-party developer…one of the things we built in which was one of the ideas from a nurse, she had nearly a 1000 patients on a study but she just wanted to see who she needed to see this month, and rather than scrolling down the 1000 patients with a pen seeing “Who do I need to see?” 

She clicks the month button and it will automatically generate your report in PDF and just say these are your patients for the month…I think that's really where the track of kind of stands out. It's just that bit of being able to organize and manage yourself and your team a lot more effectively and actually you don't want to spend time trawling through bits of paper, paper calendars and diaries.” 

• Recruitment case study 1

One participant described digital methods used in a clinical trial to recruit participants who meet the inclusion criteria of a study. Using data from the Clinical Practice Research Datalink (CPRD), researchers are able to know how many patients within each GP surgery meet the inclusion criteria for a study. Advantages of this include reduction in time and money spent on recruitment, as well as increases the likelihood of recruiting relevant participants. It is also beneficial for the patient, as the GP surgery may have relevant information on file that research studies can use (e.g., blood pressure). This is in turn also reduces the workload for the research team.

“Practices sign up to CPRD [Clinical Practice Research Datalink] and they've got around 20% of practices across England. So CPRD are able to extract their coded medical data. So any information that when you go to the GP and they say you've got diabetes, they put what they call a medical code in your patient records. So, you've got diabetes… CPRD able to extract that medical code...it's just that coded that this patient has diabetes, their blood pressure was this, they're on these drugs. They're able to extract that from practice records. So CPRD and [employing university] have spotted that actually there's a potential for using this in an interventional trial. So, it means there is an ability to extract that medical info, we can first of all pre-screen remotely the entire of CPRD's population, so over 13 million records we're able to pre-screen against our trial inclusion and exclusion criteria. So, we're then able to actually instead of the old-fashioned research way, which is you ask practices that do they want to take part in research? They say yes. Sounds interesting. You set them up, you spend time, money, setting them up, and actually they've got one patient…they haven’t got the patients you need. Whereas with this ability of CPRD’s, we're able to know that practice has this amount of patients…we're going to those practices that are large and potentially successful in actually taking part in research so that means obviously it reduces the time that we spend, the money that we spend and hopefully increases the success of the trial…and also obviously reduces practice workload... every so often a patient would go into the practice, have their blood pressure taken anyway for us then to ask that patient to come in for an extra research visit when actually we can just pull that data from the medical record, it seems counterintuitive, so making it easier one for practices they don't have to arrange for patients to come in and two for patients, it means they can absolutely take part in this trial without actually having to go into their GP practice.” 

The participant explained that they also used e-consent in the trial when recruiting participant. Consent forms were sent to patients from the GP practises via text message or email. The participant explains the process below.

“In response to COVID and the reduction in face-to-face GP visits we obviously needed to find a work around practices to be able to recruit patients to the trial. So we actually developed a fully remote consent system using REDCap…we used it to send e-consent forms out to patients. So the GP practice puts in some contact details of the patient and that then automatically sends a text message or e-mail to the patient containing a link to their consent form in this sort of electronic survey form. Patients then click on that link in that text message or e-mail it brings up their e-consent form. It provides them with the patient information sheet that they need to complete. Then they then go down and click yes, yes or no to each of the consent criteria. They then fill in their details. 

They're able to electronically sign that with that signature being saved so it matches that, that sort of the NHS criteria for e-consent system we then collect further information, so their e-mail and telephone number.”

The participant explains below the benefits of participants, GP and research team all receiving a copy of the e-consent form.

“…and then once they've completed that it automatically notifies the GP practice that they've completed…the GP practice are then able to sign their section of the consent form and then it also automatically sends a copy of the consent form to the patient and to the practise. So it automatically means that thinking about that good clinical practise the patients gets it, the practise gets it and us as a study team have that as a central repository so it makes it a lot easier for patient to consent… GP's obviously like that nature of being able to send the consent form out and then just being notified to say it's complete as well”.

Other benefits include the ability to send out follow up questions in different languages.

“We're able to use REDCap to automatically send out our follow up questionnaires. So every six months we send quality of life questionnaires that again, patients are able to do on their phone, they're able to do in different languages. And they're able to then that would so complete that automatically comes back us and saves in the REDCap database… it's able to be used by patients and we've always had good response and patients have found it easy to complete and we get really good sort of follow up questionnaire response rate. I think we've got very few that haven't completed their follow up questionnaires”

• Recruitment case study 2

One participant provided a bespoke database for participant recruitment as their most effective tool, described below. 

“It's their sort of patient record system and so this is now across all sites. So, for recruitment of patients…a person comes into the hospital at any point that admission or that contact with hospital will be recorded on the database…So if somebody comes in and they are COVID positive, we can set up alerts on the database to flag up or just to screen for anybody who's got a specific disease or has come in with a specific issue. So that can then help people identify potential people for research. So, another example might people coming in with stroke and you can search and say actually in the last 24 hours, has anybody been admitted with symptoms or query stroke that will then enable research teams to go, ok right, these are the people and look to see and to follow them up then to see if they're eligible for recruitment into any trials that we might have in those areas.

"And it will also enable us if somebody's involved in a trial, we can then label that so that speaks to a way of putting it, but we can put that information into pics, so that then if there's any safety concerns, any safety reporting, SAE reporting, etc.. We can then have alerts as they actually this person has something happen to them. We need to follow it up from a safety point of view for the study and potentially reporting SAE’s so it can be used.. it's a lot quicker and easier to identify these issues, for the delivery teams going forward.” 

The participant explained how previously research staff would have to physically sort through paper files or speak to colleagues in different departments in order to identify a patient with eligibility criteria for a trial. However, in contrast an electronic database puts all of this information in one central place. 

“It would be probably paper records. 

So to identify patients to screen patients, using stroke example, you might go to A&E, you might rely on somebody in A&E going Oh, here's a stroke patient we might need to bleep the stroke research team or you can try and pull out erm you know results if you're looking for somebody with a specific type like the COVID example, erm you might then sort of have gone to pathology to say, actually, can you send us over a list of everybody that's tested positive for COVID? So you can pull it from different systems and depending on the type of study, but it was more understanding the different ways hospital worked before and what information you want and how could you access it. Whereas obviously the database puts it all in one place to access.”

Final summary

This mixed-method study aimed to explore which tools are being used across clinical research as well as staff experiences of digital tool use. Our findings indicate that digital tools are now a fundamental part of clinical research; the COVID-19 pandemic accelerated the use of tools with adoption observed across all research stages. 

Tools were mostly viewed favourably, with increased efficiency cited as a leading benefit for research staff. It was noted that tools can account for individual differences of research participants, (e.g., materials can be easily offered to research participants in different languages and reduce participants need to travel to a physical location), potentially offering research opportunities to a broader sample of participants. 

Despite this, some drawbacks, and barriers to the use of tools exist. In particular, there were concerns about digital exclusion of research participants who may not have access to digital tools, and/or internet connections, or lack familiarity with using these platforms. There was also a concern about limited resources within clinical research; particularly the need for individuals with digital expertise to advise on and develop digital tools.  

One of the insights from this study includes suggestions for overcoming barriers to digital tool use. Recommendations included attempts to reduce digital exclusion by providing training on digital tools in various modalities (e.g., videos, face-to-face) to increase research participants/staff confidence in use and providing a physical location and/or digital tool in order for people to participate in a study. Further insights included issues around digital tool selection; participants proposed the need for a standardised process for selecting the most appropriate tool as well as published case studies of effective tools. 

Indeed, it has been previously reported that staff are often not sure of tool appropriateness for a particular task, research stage or stakeholder group and therefore a formalised selection process is required.  The need for collaboration with clinical staff and digital expertise was also highlighted in the context of developing digital tools in the future, the Management Case Study in section 5.10.2 provided an example of a cost-effective bespoke tool.  

Concerns exist around the security of data when using digital tools, particularly among older age research participants and invitations to participate via text/email. It is important that research participants perceive invitations as authentic and legitimate otherwise this may negatively affect study uptake. In the study, a participant reported conducting research to compare the recruitment rates of a pre-notification text to inform participants they would be shortly receiving a survey link compared with no text. 

However, no difference was found between the methods on participant uptake. This indicates such an approach may not be effective in improving the credibility of the subsequent message. It is critical that participants can trust recruitment invitations for clinical research, with the opportunity for scam messages being minimised, based on the way invitations are distributed.  

This study further highlights several considerations for research staff and organisations when utilising digital tools for research. Research projects should aim to be inclusive for people who may not have access to digital tools or the internet (e.g., providing a physical location/digital tool for people to complete the study). 

Organisations need to consider whether adequate training is provided for both staff and research participants to help address gaps in knowledge, demystify tools and increase confidence. It is apparent that?one size does not fit all, research staff should be willing to accommodate personal preferences of research participants when using digital tools (e.g., the option to keep their video camera off on video conferencing platforms) as well as consider offering a hybrid approach (both paper and digital tool options).

In addition, staff should capitalise on digital expertise both locally in their organisations and nationally via funders and other relevant organisations.  

Strengths of this study include the use of a broad range of stakeholders as participants and the use of mixed methods, enabling exploration of digital tool use from multiple perspectives. 

However, a limitation of this study is not fully exploring the views of participants/carers. Given that our findings alongside previous research suggest that participants/carers are a key user of digital tools, future research should seek to understand their experiences. It is evident that future research should develop a number of resources to help researchers choose appropriate digital tools. 

Our research findings alongside previous studies have recognised requirements to publish case studies of effective digital tools (specific to research stage/stakeholders) as well as develop a standardised process of choosing a tool and/or programme for recommending appropriate tools. A target for the funders and organisations related to clinical research is to develop local and national networks to share and showcase expertise in digital tools. 

Further research avenues should include an evaluation of the effectiveness of tools (specific to the research stage)?and an economic analysis to better understand the cost effectiveness of digital tools (particularly in relation to non-digital approaches) which would help to bolster the argument for use of digital tools in clinical research. 

Institute of Digital Healthcare, WMG, 
University of Warwick
Coventry
UK
CV4 7AL 

Contact: Dr. Mark Elliott 

Email: m.t.elliott@warwick.ac.uk