James Riddle and Willie Muehlhausen Discuss Digital Technology and Virtual Data Collection in Clinical Research

About the Advarra In Conversations With…

The future of healthcare innovation hinges on research and clinical trials. Advarra sits down with leading experts to dig into pressing issues and explore cornerstone solutions. Join us as we discuss topics and trends impacting the healthcare of tomorrow and advancing clinical research to be safer, smarter, faster.

 

 

About This Episode:

In this episode, Advarra’s James Riddle and Willie Muehlhausen, Co-Founder of SAFIRA Clinical Research, discuss the growth of decentralized clinical trial technology and how IRBs evolved to account for technology advancements.

Discussions with:

James Riddle
MCSE, CIP, CPIA, CRQM
VP of Research Services & Strategic Consulting
Advarra

Willie Muehlhausen
Co-Founder
SAFIRA Clinical Research

Episode Transcript:

Chapter 1: Background: What is decentralized clinical trial technology?

James Riddle (00:04): Hello, this is James Riddle, Vice President of Research Services and Strategic Consulting at Advarra. I’d like to welcome you to the 10th installment of the “Advarra in Conversations With…” podcast. Today we’re going to be talking about how IRBs and research ethics boards review virtual trial technologies.

I’m pleased to be joined by Willie from SAFIRA Partners. We’re going to be talking about this really important topic, so Willie, would you mind introducing yourself to the to the podcast listeners?

Willie Muehlhausen (00:32): Sure, I can do that! Thanks for having me today, my name is Willie Muehlhausen, I’m a Co-Founder of SAFIRA Clinical Research and I’m really looking forward to discussing how IRBs deal with screenshot reviews today.

James Riddle (00:44): Today, Willie and I are going to take a dive into how decentralized clinical trial technology is reviewed through the ethics committee and more importantly, how it is developed and how we can make the review process more efficient to move clinical trials along faster. With that in mind, I think that that we should probably go ahead and jump in. Willie are you ready to roll?

Willie Muehlhausen (01:06): I am ready. I’m looking forward to this, it’s been a long time coming, so I’m actually really happy to discuss with you today, thank you.

James Riddle (01:14): Maybe we can ground everybody on the podcast with what we’re really talking about for decentralized clinical trial technology. From the IRB perspective, Advarra has seen a proliferation of virtual trial technologies throughout the last decade and a half. There are some FDA guidance documents that guide how we have traditionally reviewed such technologies.

The guidance documents were written 10-12 years ago, and the technology has advanced substantially just in the last few years. Willie, I know you are really involved in this world of building virtual trial data capture technologies and working with sponsors on their projects, maybe you could tell us just a little bit about what SAFIRA does and how you interact with the sponsors to build these technologies?

Willie Muehlhausen (02:04): Sure, thank you. SAFIRA is a small boutique CRO, is what would I call it, so we have been working with sponsors, CROs, and technology companies basically to develop systems and processes around mostly eCOA capture so basically capturing data from patients when they’re at home or not in the doctor’s office. We’ve been doing this for quite a while. I’ve been in the industry for 23 years doing mostly this for 23 years on various projects and different sponsors and companies that I was involved with. And so, we’ve worked with many different technologies. We use wearable device data we use eCOA so patient diary data on anything in that space, that’s our focus that’s basically what we do at SAFIRA.

James Riddle (02:47): For those who might not necessarily know, what does eCOA stand for?

Willie Muehlhausen (02:51): So COA is Clinical Outcomes Assessment, and there are various different forms of that it was a term coined by the FDA many years ago. Most people will probably refer to that more into ePRO, so Patient-Reported Outcomes Data, but there’s also Observer Clinician and Performance Outcomes data. So, all four together are basically COA and “e” means that’s the electronic implementation of the systems to capture that data from patients.

James Riddle (03:15): In the old days (and if the podcast listeners could see me using air quotes), in the old days these things were captured on paper, it was diaries on paper or a journal that someone filled out. Consent forms were on paper with ink and now a lot of these capture technologies are done on your phone or on an iPad or on your desktop and the way that IRBs review the various collection devices, the collection language, the consent language has certainly evolved over the years.

But I would say that, from the standpoint of IRB review, as I look back into history, there has been a traditional “let’s review the content” and then let’s also review the final how the content will be delivered in the case of like an electronic consent, for instance.

The FDA, many years ago, issued some guidance for IRBs on how IRBs should approach reviewing these virtual collection activities and the guidance was coined around eConsent. It suggested that IRBs should review both the content and the final delivery. Now, what was this actually going to look like when a participant got it in their hands in their iPad or on their iPhone or what have you?

And so many IRBs including Advarra, for many years, had the practice of asking sponsors, to provide for eConsent the final screenshots or access to the final version of the electronic consenting form, and that then ultimately spilled over into e-diaries and ePRO and eCOA and everything else. Where the IRB would ask the sponsors, for: “what is the content you’re going to provide to the participants and then,

after you’re done building your electronic patient diary, “please give us a copy or let us see some screenshots of this, of the final product”, so that the IRB could confirm there wasn’t going to be pictures of people dancing through tulip fields and dropping their crutches after they’ve taken the investigation on medication in the background of the patient diary for instance.

And that’s a very, very over-the-top example, but that was sort of the premise behind it. So, in your work, Willie, over the years, how was that review methodology of content, first, then coming back and having to provide the IRB with a copy of the documentation or a copy of the screenshots, how has that impacted the advancement of these electronic collection modalities?

Willie Muehlhausen (05:55): We have actually moved quite a bit on there’s very little paper anymore in these settings and we have, because the FDA basically, put out a guideline in 2009 on how to develop these questionnaires and how to validate the questionnaires. So, we still see a lot of let’s say exploratory questionnaires or diaries, but we see more standardized questionnaires and many studies.

And there’s a long process involved before they end up in a final version on the screen. You need to get the license from the author, then you have to implement that, then the author has to review the screens, approve them, then they go through another review. And then, if you add translations there are a couple more reruns until you have the translator versions on these devices so there’s a lot of work involved, and takes a long time to get these onto the screens first.

And then, in parallel, generally once you have done all the testing, we want to go live. But in parallel, you have to do IRB submissions, and these, right now in most cases still require screenshot reports, which means that we cannot really provide the screenshots at an early stage to run these two processes in parallel and that always causes problems with the timelines because we can actually really only produce the screenshots at a time when we actually should have provided them weeks ago, or sometimes months ago to not miss an IRB submission. So, this is a big issue and in the whole process, as it is right now. We are moving more and more into BYOD, which means patients use their own hardware and their own device. This makes it easier in many cases for patients, but not every patient wants that, but it makes it easier for a lot of patients.

But that adds on complexity is that we now don’t actually know exactly what these questionnaires will look like on the different devices, different sizes, different operating systems, so they all look slightly different.

So, then, the question is, what do we actually submit to the IRB? Do we submit the tablet version, do we do the mobile phone version, the android versus the iOS version? So, it’s an open minefield, right now, as far as I’m concerned. It opens up an opportunity, though, as well, because we don’t know what the end product will look like in the minute detail now, we do know there’s nobody dancing through tulip fields in the background right so

James Riddle (08:08): That’s a pretty extreme example on my part, but it illustrates the point of why IRBs traditionally asked for screenshots.

Willie Muehlhausen (08:16): It so does so we don’t have that, basically, even with these different versions but we could basically generate some of the screenshots digitally instead of taking them off the actual device. But the question really is what does it add to the safety of the patient by having a standard questionnaire submitted to I don’t know how many dozens of IRBs within one study and then next week, the same ones, to the next IRB or the same IRB. So there’s a lot of work and a lot of delays actually are caused by the submission request basically.

Willie Muehlhausen (08:51): We have seen that the bring your own device (BYOD) has certainly proliferated during COVID and even beforehand, but the COVID stay at home orders, Advarra has seen a lot more remote capture devices, remote capture modalities, collection in the home, collection through Google glasses things of that nature. It really did enlighten our thinking about the way that we approach electronic consent and electronic patient-reported outcomes and e-diaries and the things of that nature.

I can’t speak for every single IRB out there in the world or every single research ethics board in the world, but at least at Advarra, we did change last year, 2021, we did change our approach to how we review eConsent and we switched, and maybe Willie, I’d like to get your thoughts on if you think this will improve the situation.

So, we switched from the traditional “show us your content in a Microsoft word document, we’ll approve the text and then send us a screenshot of how that text manifests itself in the electronic consent.”

When it’s loaded into the tool. Sometimes that would be a day or two later sometimes that might be a month later, depending on how long it took the vendor to load the approved text into the eConsent tool. And then, we would issue a final approval based on the screenshot of the consent in the tool itself or we sometimes had vendors, which would allow us to go in and look at the tool.

What we switched to is what we realized was we really weren’t ever finding anything. We weren’t finding stuff between the approval of the text in the approval of the final screenshots. So we’ve switched, at least at Advarra, to a methodology where the investigator or the sponsor whoever it is that is submitting the research is able to describe for the IRB the process by which the approved text of the IRB is going to make it into the final electronic presentation in the tool and provide some attestation to the IRB things like I promise that the text that the IRB approved will be the same text that is presented to the participant in the electronic tool, then the IRB is looking not necessarily at the final end product, but the process by which the approved text of the IRB consent will be manifested in the electronic tool, how does it get there.

You’ll see that same methodology coming out from Advarra for all of the other electronic content eCOA, ePRO, e-diary, all of those things again with air quotes sometime early next year, but really the IRB wants to look at the content.

The IRB wants to know what information is being presented to the participants and how can the IRB be assured that the electronic tool isn’t messing it up. Right, the text that the IRB approved is actually the text that’s going to appear on the tool. How it appears on the tool is sometimes unknown, as you mentioned, because with bring your own device it might look different on Safari browser than it does on Chrome browser or much different again what it looks like on an Android phone versus a versus my iPhone.

Chapter 2: What are the challenges with advancements in technology and How can IRBs evolve to account for technological advancements?

James Riddle (12:09): And so, I think the IRB at least, Advarra, has come to the conclusion that, we’re not adding a lot of value by asking for all of these screenshots. Hopefully, the rest of the IRBs on the planet will follow suit, we shall see but we’ll maybe you could comment from your perspective, like, will having IRBs change to this “review the content and then review the methodology, by which the content, makes it to the electronic tool” and not ask for the final screenshots, will that fix the issue of the delays and such that you were describing?

Willie Muehlhausen (12:41): I think it has a huge potential to do that and it’s just like you said, you didn’t find anything. When you use a standard questionnaire, I just mentioned one or two like an EQ5P or an SO36. Look, they haven’t changed in years and they probably won’t change for many years to come.

So basically, you wouldn’t find anything there either because it doesn’t really matter who submits that and what technology uses the wording will always be the same. That’s just what the FDA and other states basically by defining these processes that there’s a good standardization amongst many of these questionnaires.

James Riddle (13:15): Willie, you talked about international you talked about translation, so I’m assuming that this is going to be a challenge internationally as well. And maybe you could, for the listeners and we have a lot of listeners of the podcast who are other research ethics board members from around the world. Maybe you could just give an anecdotal example or two of the kinds of delays that you’ve seen happen in clinical trials, because of the way that the research ethics boards and IRBs insist on capturing this end of product screenshot capability in a clinical trial.

Willie Muehlhausen (13:45): So, for example, if we take a translation of a questionnaire that doesn’t exist in the language that you’re looking for. I’m German, so let’s take German as an example. So, if you have a questionnaire and you have it available in English, US English usually is a starting point, and you can then easily program that and submit it to the IRB when you’re ready, that’s the fastest. We can do another take, between 10 and 12 weeks, sometimes you will have a longer but let’s say around 10 to 12 weeks.

If you then have to translate that into 20, 30 different languages and we’ll stick with German, that process can take a couple of weeks, in addition to that, and there’s a very specific process that we have to follow to do what we call linguistic validation so it’s not just a simple forward-backward translation it’s a lengthy process. German is now an easier one to follow, but there are some other more complex languages, where you have less basically availability of translators and patients that you can include in that process that can take 14 weeks 16 weeks the translation process.

So, if we then have to submit screenshots in the local language to the individual IRBs, so you add on the first few weeks for the build, and you add a couple of weeks, probably even months for the translation. Then, it has to be built into the system has to be reviewed by the author, which will take a week or two or three. So, you just add up all these weeks we’re talking about three, four, or five months easily and I’ve seen other studies, where it takes up to nine months basically in delays, because we have to get the screenshots in there now.

The screenshots are not the only problem, in this case, it’s also the translations, but it just exacerbates everything and makes it more difficult and takes longer. So it’s definitely a big issue right now in our industry.

James Riddle (15:30): Unfortunately, that only delays the startup of the trial and is ultimately going to delay on the back end how quickly these cures make it to market. Just because of this, this one particular issue and there are lots of issues with study startup in clinical trials there are lots of areas where there can be efficiency gains, but this seems like a pretty easy win to be able to get trials moving along faster.

From a human subject’s protection standpoint, I’m interested Willie, in your perspective on what happens during the trial when there’s a modification to the protocol and something in the diary has to change? Is there a potential subject safety issue by the IRBs insisting on seeing the final screenshots before a modification to the diaries or you know pill reminder forms, or what have you gotten pushed out to the participants?

Willie Muehlhausen (16:23): There could be potential, yes. Look, we see a lot of protocol changes, but quite often they don’t necessarily affect the eCOA (so don’t affect the patient questionnaire part) and if it does it’s usually that we change the time point when they have to answer a question, they are we add a question or to your questionnaire out, so the changes are not that big in most cases.

James Riddle (16:44): From an IRB perspective, one of the reasons why we made the change in our approach to reviewing eConsent in particular, not only the efficiency of making sure that the trials get started up as efficiently as possible, but also during the trial, for eConsent, we were finding that participants who were being reconsented or provided with additional information on a paper-based form were getting new risk information provided to them faster than the folks who were utilizing an eConsent, which presumably should have been more effective and quicker, but was delayed because the changes to the consent needed to be verified back through the IRB with a final screenshot once the vendor had loaded in the process.

Advarra does provide an electronic consent product, and we have really worked hard to make sure that the delivery of modifications and the changes to the consent are done as rapidly as possible, but it does add an extra step when the IRB insists on seeing a screenshot of the final version before it goes in front of the participant’s nose.

That’s one of the other reasons why we decided to change the consent as well, because we want to make sure that changes are pushed out rapidly so people get new risk information as quickly as we possibly can.

Willie Muehlhausen (18:06): This is the big advantage of these electronic formats right, so we can push out a new version of a questionnaire or a new questionnaire basically with a push of a button right versus having to ship paper forms into 20, 30, 50 different countries and sites and so on, so this is exactly one of the advantages of using electronic means is that we can do that across borders, very quickly and very efficiently. And that’s why I’m always looking, we at SAFIRA, we’re always looking at all kinds of processes around that, trying to improve where we can help to improve where we can.

So, this initiative has been coming for quite a few years. I’m actually very happy that we’re finally getting somewhere, and this makes some changes because it will address one of the major drawbacks and one of the major delays that we have in any clinical trial in this space so it’s actually it’s very good that we see that moving.

Chapter 3: What’s Next for IRB approval of DCT technology?

James Riddle (18:58): I’d like to just kind of think about what comes next, so let’s you know, let’s envision a world where every IRB and research ethics board in the world doesn’t ask Willie for screenshots anymore.

Willie Muehlhausen (19:08): Yeah, look right now we spend a lot of time, effort, and money basically on producing the screenshots. And at this point, from my point of view, a lot of wasted energy wasted effort, because, again, it doesn’t add to the patient safety at all, not in a meaningful way.

So if we can redirect all these efforts and enthusiasm for the screenshots into other areas that we can address the next hurdle is basically I’m sure we’ll find a lot of other things that we can improve as well, but at a minimum, we will basically cut the timelines from submission to basically approval and because we can do the submission area, which means we can start studies already.

Now, there are other things that need to be fixed as well or need to be addressed. But in this case, I think this will have a huge impact on study startup timelines, especially in international clinical trials, where we have 10, 20, 30, 40 different countries and languages and often than this.

This is a huge undertaking and if we can get to a place where we submit with the protocol, the necessary information about data privacy, protection of the patient with regard to the technology, and probably even, how we translated these questionnaires into basically different languages.

So some additional information around that so that IRB can rest assured that what they see as what’s on the device in the end and basically in the patient’s hands, then I think we’re in good shape and can redirect a lot of efforts into other areas.

James Riddle (20:36): That’s encouraging. At Advarra, we are definitely in the mode of making sure that these protocols are run effectively, safely, to get the cures to the end market as quickly as possible.

And also, when we see the advancement of clinical trials really as a patient first issue. These cures need to get through clinical trials, so we can determine if they’re safe and effective.

By if we can eliminate one thing, two things, or five things if we can eliminate a few things to make it go just a little bit faster everybody benefits, and certainly, the way that IRBs review decentralized clinical trial technologies seems like an area that we are at least Advarra is trying to move this forward and eliminate some of the burdens of the study startup that go into reviewing these technologies. I’m interested to know as well; we see a proliferation of these remote capture tools.

One of the benefits we see of decentralized clinical trials is the ability to both include more people who may not necessarily have been able to participate in the past, because some people don’t have time to go to the clinic to fill out their diary for their patient-reported outcomes once a week Monday through Friday from 8 to 5, they’d much rather do it on their iPhone at eight o’clock at night when it’s convenient for them to fill in how are they doing what are they feeling things of that nature and so.

We as a community, when I talked to my other IRB colleagues, one of the concerns that we have sometimes that comes up with these decentralized trial technologies is, is the data being collected the same quality of data or better than what we used to collect on the paper? You’ve been around since paper I’ve been around since paper.

Now we see people you know recording their e-diaries and their patient-reported outcomes on their phone or on their iPad or whatever, rather than filling out of paper-based form and sending that in.

In general, the James Riddle opinion is, we’re probably generating better data through these electronic tools than we did with pencils being checking boxes on paper and then somebody transcribing that into a case report form.

Willie, from your perspective, as we see these technologies proliferating do you have the same view and opinion about the improvement in the quality of the data based on these tools?

Willie Muehlhausen (23:02): Yeah, I do, and I could talk about that all day long.

James Riddle (23:06): But I think it’s a really important topic that if we can get more of these tools in the clinical trials and improve the quality of the data, we really have put the patient, first in advance, these trials, the long and the meaning, in a very meaningful way.

Willie Muehlhausen (23:24): So, on my first day, on my first job in this industry, in March 1998 my first job was double data entry of paper-based diaries from patients into a data management system and, at the time in Berlin. So I saw pretty quickly after a couple of hours that there’s something not right with that approach right, but that was, at the time when we didn’t have smartphones. There was a time in 1998 when in vivo data PhD in others, the already companies were doing some e-diaries.

But there was also the time when there was the first study to prove that basically patients, when they enter data in paper diaries and then somebody, transcribed it afterward. Not only do you introduce errors during the transcription process, but we have that what we called “parking lot syndrome”, so a lot of patients fill in the diary on paper, just the next 10 minutes before they go into the doctor’s office when they have their visit.

And then there’s that proof over many studies that basically when you have a recall bias, when you do it like this, you want to have patience entering the data as close to the event that you’re trying to monitor as possible, especially when you talk about patient daily diaries, for example.

Willie Muehlhausen (24:32): So, there’s a whole host of publication lower last 20 years and my team and we’ve done a lot of work as well on the BYOD side of things, to prove that you can get the same if not better data, better compliance, better engagement from patients. If you have a BYOD approach, so if you let them use their own device and then, as an extension to that in a DCT setting actually let them do it whenever it suits them and not when it suits us, and when we want them to be at the site.

So, there’s more and more data coming out now because, due to a COVID we were forced into accepting more DCT settings basically right.

So now we’re seeing more and more data coming out that’s being published that actually proves the point that, at a minimum that data is no worse. Actually, in cases, the data is better. We get higher engagement rates, we get patients to stay on the studies longer, they provide the data more accurately now we’re always starting small percentages in most cases it’s not like a huge difference, just small percentages, but I think what we’d see over time, we see increase the recruitment rates because we know can reach patients that we couldn’t reach before now, we should feel the patient should fill studies with patients faster than we currently do.

So, all this is ongoing research and ongoing monitoring to look at the numbers and, as you said earlier, there’s so many new vendors out there and so many new technologies out there, that we will see more of that and we will get a confirmation whether we’re right or wrong about this and I’m with you, I think we’ll see better data and the first indications would show that.

But there’s other stuff, for example, that I think is going to be interesting, and in the setting as well. I’m a big fan of voice assistance since I’ve seen that my children use them on a regular basis, I’m still more old, so I like to type on the keyboard and on a screen versus talking to my iPad, for example. But I think that’s huge potential there with the voice assistance, and that brings up another question for IRBs, how do you get this done? So how do you how do review the voice of an SF36, you look at scripts for it, I guess. or you will get scripts are you don’t need to look at that at all, so no you just know it’s as SF36 and we’re good to go.

And there won’t be any Tulips in the back, I want to use voice, maybe something else, some nice music or something.

James Riddle (26:47): I can picture it now. I haven’t seen this in the IRB yet, but I’m sure it’s coming where the participant gets: to go “hey Alexa is it time to take my pill yet?” and somehow that’s built into.

Willie Muehlhausen (26:59): I built systems like that. We already built systems like that, so we’ve done some work and I’m doing some work right now to show because there’s huge interest and again, talking about making studies more accessible. We can reach patients that can’t read or write, but they can still understand listen and answer through a voice assistant, or we can reach patients that have let’s say a vision impairment to various levels. They can still talk and answer questions. I think the voices are currently one of the undervalued modalities that are out there, but I’m absolutely sure that will change over the next few years, which brings different challenges but it’s a huge opportunity.

James Riddle (27:37): Well, that sounds like the next installment of one of our podcast series, we can talk about the integration of personal digital assistants into clinical trials.

I’m just waiting for the next conversation, where the IRB is trying to decide how to consent, artificial intelligence to be able to use the artificial intelligence in the research where the consent form is written to the AI rather than written to the participant, I suspect we will probably get there in our lifetimes.

So, hey Willie, this has been a distinct pleasure getting to talk to you, your organization is really moving this forward, thank you for participating in the podcast. I hope we get to have another conversation like this down the line.

Willie Muehlhausen (28:18): Now look I would love to come back, there’s always a good topic. I’ve enjoyed the conversation. It makes me very happy that Advarra is so forward-thinking and it’s moving the needle in this case so I’m very happy and wish you best of luck with that. I’m sure will do well and I’m happy to come back some other time.

James Riddle (28:35): Everyone, thank you for listening. That’s going to conclude our 10th installment of Advarra In Conversations With. Make sure that if you enjoyed today’s session that you follow us on all of Advarra’s channels: LinkedIn, Twitter, and also come back to Advarra.com to download this podcast and the next podcast or wherever you get your podcast. Look forward to talking to you again soon. Thank you and have an enjoyable rest of the day, or evening or morning depending on when you’re listening to the podcast. Take care!

Back to Resources