exclude: true --- class: middle background-image: url("img/RANZCR_Presentation_ASM_2021_BG.png") background-size: cover <br> # .white-col[Driving innovation through electronic PROMs in Radiation Oncology routine care] ### [RANZCR ASM 2021](https://www.ranzcr2021.com/) ### .white-col[**Thilo Schuler**^1,2, Andrew Kneebone^1,3, George Hruby^1,3, Rebecca van Gelder¹, Julia Hunter¹, Blanca Gallego^2,4, Thomas Eade^1,3] ### .white-col[1.] [NSCC](https://www.nslhd.health.nsw.gov.au/RadiationOncology/Pages/default.aspx), .white-col[2.] [AIHI](https://www.mq.edu.au/research/research-centres-groups-and-facilities/healthy-people/centres/australian-institute-of-health-innovation/aihi-research-centres/health-informatics), .white-col[3.] [USyd](https://www.sydney.edu.au/medicine-health/schools/clinical-schools/northern-clinical-school.html), .white-col[4.] [CBDRH](https://cbdrh.med.unsw.edu.au/) ??? * Hi everyone. Thank you for the opportunity to present on: Driving innovation through electronic PROMS in Radiation Oncology routine care <br> * This talk describes our experience of using ePROMs routinely over the last 2 1/2 yrs including some **EXCITING** initiatives these ePROMs allowed us to start <br> * I would like to acknowledge the whole time including my co-authors --- layout: true background-image: url("img/Precise_NSCC_logo_353_100.jpg") background-size: 200px background-position: 96% 4% --- # Patient-reported Outcome Measures <br> > ###Patient-reported outcome measures capture a person’s perception of their own health through questionnaires (ACSQHC). <center> <div id="htmlwidget-bc56d6a654857488a3a3" style="width:90%;height:300px;" class="grViz html-widget"></div> <script type="application/json" data-for="htmlwidget-bc56d6a654857488a3a3">{"x":{"diagram":"\n\ndigraph {\n\n # a \"graph\" statement\n graph [layout = dot, rankdir = LR, overlap = false]\n\n # several \"node\" statements\n node [shape = square,\n fontname = \"Century Gothic\",\n style = filled,\n color = white,\n fontcolor = white,\n fontsize = 35,\n fixedsize = true,\n width = 3\n ]\n ranksep=5; \n node [fillcolor = steelblue]\n \n node [label =\n\"Research tool\"]\n reprex1\n \n node [label =\n\"Research \nintervention \n(ePRO)\"]\n reprex2\n \n node [fillcolor = coral,\n label =\n\"Routine\ncare\ntranslation\"]\n reprex3\n \n edge [color = grey, penwidth = 5, arrowsize=2]\n reprex1 -> reprex2 \n reprex2 -> reprex3 [style=dashed]\n\n}\n","config":{"engine":"dot","options":null}},"evals":[],"jsHooks":[]}</script> </center> ??? * The Australian Commission on Safety and Quality in Healthcare definition of PROMs reads: Patient-reported outcome measures capture a person's perception of their own health through questionnaires. <br> * PROMs have undergone an <u>**evolution**</u>: 1. Initially they were a **pure research tool** and generally captured on **paper** <br> 2. Over the last decade **research** has demonstrated several **benefits** in clinical practice. However this has required a shift to **electronic PROMs**. <br> 3. Broader translation including implementation research to overcome barriers is ongoing --- # ePRO(M)s .pull-left[ ### ePRO Benefits * Better QoL (symptom control) * Better communication * Survival benefit (Basch et al, JCO 2016) ] .pull-right[ ### ePRO Challenges * Complex healthcare workflows * Culture & Clinician Buy-in * Technology (Integration) ] <br> .pull-left[ ### ePRO Opportunities * A successful ePRO implementation enables innovative routine care approaches... ] ??? * Clinical research has repeatedly shown <u>**ePRO benefits**</u> including: 1. Better symptoms control and QoL 2. Improved communication 2. The seminal RCT by Basch et al showed a survival benefit <br> * However routine care translation remains <u>**challenging**</u> and reasons include: 1. Complexity of healthcare and its workflows 2. Cultural and buy-in issues 3. Integration into existing technology landscape <br> * Once ePROs are in place, they offer exciting opportunties to improve care...In the next slide I will present some of the work that ePROs have allowed us to do before sharing our ePRO experience. --- # ePRO Innovations #### Innovative care pathways * Development of on **ePRO**-driven referral pathway to the community service Canteen demonstrating an **increase in referral rates by 350%** * Slides and video here: [https://thiloschuler.me/project/epro-driven-crisp](https://thiloschuler.me/project/epro-driven-crisp) #### Remote toxicity monitoring * Trend for hypo#/SBRT with significant side effect peaks after the RT course ➡️ remote **ePROs** +/- alerts can capture them and trigger actions. #### Embedded, pragmatic real-world clinical trials * Our ongoing in-house prostate cancer RCT (def. hypo# vs SBRT) has recruited n=188 pts in 2 years * Without **ePROs** this wouldn't have been possible in a small 3 linac department. #### Safe, risk-adapted remote follow-up * Even before COVID-19 we had started to integrate **ePROs** into our phone follow-up program * We are working on an **ePRO**-only approach under certain circumstances ??? #### Innovative care pathways * In collaboration with Canteen we developed an **ePRO**-driven referral pathway to their services which increased our referral rates substantially * ePROs were critical in sustainably achieving this and I am proud of this work as it has made a real difference for some families affected by cancer * Slides and **video** about this project at this link: [https://thiloschuler.me/project/epro-driven-crisp](https://thiloschuler.me/project/epro-driven-crisp) #### Remote toxicity monitoring in an era of increasing hypo#/SBRT * ePROs combined with clinical alerts are attractive given the recent trend to hypo#/SBRT with acute symptom peaks often happening after the RT course #### Embedded, pragmatic real-world clinical trials * Our ongoing in-house prostate cancer RCT (def. hypo# vs SBRT) has recruited almost 200 pts in 2 years * Without **ePROs** this wouldn't have been possible in a small 3 linac department. * Thus, ePRO are a key enabler of embedded, pragmatic clinical trials #### Safe, risk-adapted remote follow-up * Even before COVID-19 we had started to integrate **ePROs** into our phone follow-up program * We are working on an **ePRO**-only approach under certain circumstances to enable safe and risk-adapted follow-up --- exclude: true background-image: url('img/slide4.png') background-size: cover class: left, top ??? * As an example of the <u>**ongoing translation challenges**</u>: <br> Ethan Basch who published the seminal RCT on ePROs in 2016 describes implementation barriers in a 2019 paper. --- background-image: url('img/full_team.png') background-size: cover class: left, top # Clinical GU ePRO team @ NSCC ??? * This is the clinical team behind this effort. Most of you would know Andrew, Tom and George. <br> * Julia Hunter is our GU clinical nurse coordiantor and she has played a critical role as as our administrative ePRO team. --- exclude: true background-image: url("img/Precise_NSCC_logo_353_100.jpg"), url("img/ir_toolkit_identity_300_who.png") background-size: 200px, 300px background-position: 96% 4%, 80% 60% # Journey .pull-left[ ### 2016 * **Onsite ePRO** collection (iPad in waiting room) * Completion rates (35-59%; RANZCR ASM 2017) too low. ### 2017 * **Offsite ePRO**: No commercial vendor * Re-Discovery of **REDCap** as "ePRO survey engine" (survey designer & API) ### 2018 * Re-implementation of in-house clinical research system enabling REDCap **integration** ] ??? This is our <u>**implementation research journey**</u> how we our current approach: <br> 1. Basically onsite ePROs alone had too low completion rates 2. After we couldn't find a suitable commercial system we integrated the well-known REDCap system into our inhouse system as the ePRO engine --- background-image: url('img/ProDB_ePRO.PNG') background-size: cover class: left, top ??? * this is a screenshot from the in-hous clinical research application. <br> * The timeline in the upper part visually displays summary metrics regarding GU and GI toxicity over time: ePRO scores show as little squares and clinician grades as circles. --- background-image: url('img/REDCap_Pt.PNG') background-size: contain background-position: 50% 0% ??? This is how an ePRO survey looks to the patient. --- # REDCap as ePRO Survey Engine <center><img src="img/REDCap_ePRO_survey_engine_1400.png" alt="drawing" width="900"/></center> ??? * As mentioned before we use <u>**REDCap as the "engine"**</u> of the ePRO collection process: 1. The patient gets onboarded usually before the new patient appointment 2. REDCap sents SMS or Emal invitation and if needed 1-2 reminders 3. The patient fills the ePRO suvey on their phone or computer 4. a) The results get automatically integrated and displayed in the clinical rssearch system 4. b) If predefined thresholds are breached an email goes to the clinical team 4. c) The admin team monitors freetext answers and escalates to the clinical team if any concers --- background-image: url("img/Precise_NSCC_logo_353_100.jpg"), url("img/requite_black.png") background-size: 200px, 300px background-position: 96% 4%, 50% 70% # Surveys and Frequencies in our GU Practice <br> #### Superset of international [REQUITE](https://requite.eu/) consortium's questionnaire: * **Baseline** (ePRO prior to new patient appointment) * **Peri-treatment** (abridged; weekly ePRO during and in first month post RT) * **Follow-up** (6 monthly ePRO) ??? * We use the pelvic <u>**questionnaires from the REQUITE consortium**</u> that we have extended with some extra questions. There are 3 survey types: <br> 1. A Baseline survey is completed prior to the new paient appointment 2. A weekly Peri-treatment survey , and 3. A 6monthly Follow-up survey --- background-image: url("img/Precise_NSCC_logo_353_100.jpg"), url("img/completion_plot2.png") background-size: 200px, 900px background-position: 96% 4%, 50% 90% # ePRO Coverage & Completion Between Jan 2019 - June 2021 (30mo) n=3201 invitations (+/- reminders) for separate ePROs were sent as part of NSCC prostate cancer clinic. <table class="table" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:left;"> Baseline </th> <th style="text-align:left;"> Peri-treatment </th> <th style="text-align:left;"> Follow-up </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Coverage (only for 2019) </td> <td style="text-align:left;"> 85% </td> <td style="text-align:left;"> - </td> <td style="text-align:left;"> - </td> </tr> <tr> <td style="text-align:left;"> Completion (Jan 2019 - Jun 2021) </td> <td style="text-align:left;"> 94% (293/311) </td> <td style="text-align:left;"> 91% (1170/1292) </td> <td style="text-align:left;"> 76% (1216/1598) </td> </tr> </tbody> </table> ??? * Between Jan 2019 - June 2021 **more than 3000** invitations plus up to 2 reminders if needed were sent as part of our prostate cancer clinic. <br> * Coverage defined as the proportion of all new patients that were sent an ePRO invitation was 85%. This was only measured in the first year. <br> * Completion rates 94% for the baseline suvey, 91% for the peritreatment survey and 76% for the follow-up survey <br> * This diagram shows these compltion rates over time. This is from an interactive dashbaord that is available online for anybody keen. I will share a link at the end. After the next slide I will also give you a glimpse of this the actual dashboard. --- background-image: url("img/Precise_NSCC_logo_353_100.jpg"), url("img/duration_plot2.png") background-size: 200px, 900px background-position: 96% 4%, 50% 90% # ePRO Duration <table class="table" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> </th> <th style="text-align:left;"> Baseline </th> <th style="text-align:center;"> Peri-treatment </th> <th style="text-align:left;"> Follow-up </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> 1st Quartile </td> <td style="text-align:left;"> 6.0 min </td> <td style="text-align:center;"> 1.8 min </td> <td style="text-align:left;"> 7.7 min </td> </tr> <tr> <td style="text-align:left;"> Median </td> <td style="text-align:left;"> 8.2 min </td> <td style="text-align:center;"> 3.0 min </td> <td style="text-align:left;"> 11.1 min </td> </tr> <tr> <td style="text-align:left;"> 3rd Quartile </td> <td style="text-align:left;"> 12.7 min </td> <td style="text-align:center;"> 5.1 min </td> <td style="text-align:left;"> 16.1 min </td> </tr> </tbody> </table> ??? In terms of ePRO duration: the median time required baseline survey was 8.2 min, the median duration for the peri-treatment survey was 3 min and 11.1 min for the follow-up survey --- background-image: url("img/Precise_NSCC_logo_353_100.jpg"), url("img/qr_dashboard.png"), url("img/qr_crisp.png") background-size: 200px, 200px, 200px background-position: 96% 4%, 22% 90%, 78% 90% class: center, middle # Conclusion ### Routine care ePROs are .highlight[challenging on multiple levels], but definitely .highlight[feasible] and well worth the effort as they open .highlight[exciting opportunities] to improve the care for our patients. .pull-left[ #### Link to **ePRO Dashboard**: [https://thiloschuler.shinyapps.io/prospector-asm21](https://thiloschuler.shinyapps.io/prospector-asm21/) ] .pull-right[ #### Link to ePRO-driven referral process incl **VIDEO**: [https://thiloschuler.me/project/epro-driven-crisp](https://thiloschuler.me/project/epro-driven-crisp) ] <br> ## Thank you <br> 🙏