5th Annual Research Roundtable: Data Analytics in Healthcare

Dr. Saba Vahid, Group Manager, Data and Decision Sciences, Ontario Health; Dr. Tahera Yesmin, Data Scientist, Data and Decision Sciences, Ontario Health; and, Dr. Shabnam Balamchi, Decision Scientist, Data and Decision Sciences, Ontario Health

Titles & Abstracts:

8:10-8:35am

Waiting Experience in Open-Shop Service Networks: Improvements via Flow Analytics & Automation

Presenter:

Manlu Chen, Assistant Professor, School of Business, Renmin University of China

Additional Authors: 

Opher Baron, Professor, Operations Management, Rotman School of Management. 

Avishai Mandelbaum, Professor Emeritus, William Davidson Faculty of Industrial Engineering and Management, Technion in Haifa, Israel. 

Jianfu Wang, Associate Professor, Management Sciences at the College of Business, City University of Hong Kong. 

Galit Yom-Tov, Associate Professor, Technion – Israel Institute of Technology, Industrial Engineering and Management faculty, Co-director of the Service Engineering Enterprise (SEE)-Lab. 

Nadir Arber, Professor, Medicine and Gastroenterology in Tel Aviv Medical Center, Sackler School of Medicine, Tel Aviv University.

Abstract:

Waiting-for-service is a central, typically detrimental, factor in service experiences, and multiple delays will most likely amplify customers' poor impressions of a service. Yet multi-delay experiences are commonly assessed via macro measurements, e.g., overall waiting, as opposed to micro measurements that account for individual delays, e.g., maximal or most-recent delay. Furthermore, the COVID-19 pandemic has exacerbated the challenges of controlling micro measures -- physical distancing has turned short queues and waits into rigid constraints. Our paper, motivated by a health screening clinic, jointly considers macro and micro measurements.

To improve customers' waiting experience, the clinic implemented two information technologies: automated customer routing system (ACRS) and SMS-based customer paging system (SCPS). However, as our empirical study of the clinic revealed, these implementations had no significant impact for three main reasons. First, ACRS reduced system flexibility and thus caused unintended idleness of resources. Second, SCPS improved non-bottleneck activities. Third, the clinic automated its sub-optimal practice, namely station-level first-come-first-served policy, leading to long delays towards the end of a customer's route.  

Nevertheless, these initiatives could facilitate enhanced operations and improve waiting experience. To this end, via a stylized queueing model and a data-driven simulation model of the clinic, we analyze delays in an open-shop service network. Our models reveal that a priority-based buffer strategy, which accounts for both system- and station-level characteristics, improves both macro- and micro-level measurements. In particular, when prioritizing according to shortest expected remaining processing time priority-base buffer policy, performance improves at both micro-level by 41.9% and macro-level by 14.9%.

8:35-9:00am

Exploring the Public Response to COVID’s Technology Challenges in the First Year of the Pandemic: Addressing the deployment of mRNA Vaccinations and Care delivery through Telemedicine

Presenter: 

Abraham (Avi) Seidmann, Professor, Department of Information Systems, Questrom Business School, Boston University; Associate Research Director, Health Analytics and Digital Health, Digital Business Institute, Boston University

Additional author:

Arriel Benis, Faculty of Industrial Engineering and Technology Management, Holon Institute of Technology; Faculty of Digital Technologies in Medicine, Holon Institute of Technology, Holon, Israel

Abstract:

The COVID-19 pandemic challenges almost all of our healthcare services in unexpected ways and has devolved many public health conventions. In addition, this pandemic had a stimulating effect on some innovative technological expansions. Two prime examples are the rapid introduction of novel mRNA-based vaccines, and the rapid deployment of Online Care and of Telemedicine. Policy makers and public health officials were looking for the right ways to encourage broad-based vaccinations as well as telemedicine. We saw are a dramatic increase in the use of mass media and social media communication in trying to swing public support for vaccinations and telemedicine.

In our studies we sought to elucidate the socio-demographic characteristics and the principal reasons of the public intent to take the newly introduced mRNA vaccine(s) against COVID-19, to use telemedicine during the COVID-19 pandemic, and the overall propensity to try and use it thereafter.

Our extensive social media study results clearly indicate that contrary to the prevailing public perceptions in the USA, young audiences have mostly a positive attitude towards COVID-19 vaccination (81.5%). These younger individuals want to protect their families and their relatives (96.7%); they see vaccination as an act of civic responsibility (91.9%) and they have expressed strong confidence in their healthcare providers (87.7%). Another critical factor is the younger population’s fear of personal COVID-19 infection (88.2%). Moreover, the greater the number of children the participants have, the greater was their intent to get the COVID-19 vaccine. When it comes to telemedicine, the initial tremulous ways to success were dramatically shortened. Yet, we were surprised to see that only a third of the participants intended to continue using telemedicine after the COVID-19 pandemic. The evidence is clear that an expected substitution effect, technical proficiency, reduced queueing times, and peer experiences are the four major factors in the overall prospective adoption of telemedicine. The results highlight some novel public health policies for dealing with the successful deployment of these technologies at the current COVID time and well after that.

_______________

9:00am to 9:25am

Developing Pre-Testing Diagnostic Tools for Pandemics Using Predictive Analytics: The Case of COVID-19

Presenters:

Ramy Elitzur, Associate Professor of Accounting, Rotman School of Management, University of Toronto 

Dmitry Krass, Professor of Operations Management and Statistics, Sydney C. Cooper Chair in Business and Technology, Rotman School of Management, University of Toronto

Abstract:

Developing rapid tools for early detection of viral infection is crucial for pandemic containment. This is particularly crucial when testing resources are constrained and/or there are significant delays until the test results are available – as was quite common in the early days of Covid-19 pandemic. We show how predictive analytics methods using machine learning algorithms can be used to accurately predict outcome of the test based on observed previous test results and patients’ symptoms. The resulting predictions can be incorporated into a pre-test screening mechanism, greatly increasing test efficiency (i.e., rate of true positives per test), as well as to allow doctors to initiate treatment before the test results are available. We also apply simple Bayesian techniques to correct for test false positive and false negative rates in both, accurately estimating the infection rate among the tested population and developing an effective pre-screening mechanism.

Our methods are applied to the case of Covid-19 pandemic and the “gold standard” reverse transcriptase polymerase chain reaction (RT-PCR) tests for SARS-COV-2. We analyzed data reported by the Israeli Ministry of Health for RT-PCR tests from March to September 2020. Our results demonstrate that in the context of the Covid-19 pandemic a pre-test probability screening tool with conventional RT-PCR testing could have potentially increased efficiency by 180%.

We also show how our admission policies can be extended to allow multiple testing of high-risk individuals. While multiple testing is never optimal with random admission policy, it can be optimal for a policy driven by a machine learning model. We derive conditions for optimality and a simple linear time algorithm to optimize test efficiency.

9:25am to 9:50am

Capacitated SIR Model with an Application to COVID-19

Presenter:

Chaoyu Zhang, Ph.D. candidate, Rotman School of Management

Additional Authors: 

Ming Hu, Distinguished Professor of Business Operations and Analytics, Rotman School of Management

Ningyuan Chen, Assistant Professor, Department of Management at the University of Toronto, Mississauga

Abstract:

The classical SIR model and its variants have succeeded in predicting infectious diseases' spread. To better capture the COVID-19 outbreak, we extend the SIR model to impose a testing capacity and differentiate the infected people into symptomatic and asymptomatic. Using this capacitated SIR model, we study how to choose the best type of testing method, how to allocate limited testing capacity over time and across symptomatic and asymptomatic people. We use the COVID-19 data and a sliding window method to calibrate our model and point out its public policy implications.

Estimating the Impact of Asymptomatic Carriers on the spread of Infectious Diseases: An interaction-based Model

Presenter:

Yaniv Ravid, Ph.D. candidate, Rotman School of Management

Additional Authors:

Abraham (Avi) Seidmann, Professor, Department of Information Systems, Questrom Business School, Boston University; Associate Research Director, Health Analytics and Digital Health, Digital Business Institute, Boston University.

Abstract:

Epidemiological models are commonly used to predict and describe the spread of a viral outbreak among a population. Many such models use differential equations and transition rates to predict the growth dynamics of the infectious and exposed groups with a greater population. However, no model distinguishes between infectious individuals, making it impossible to account for asymptomatic individuals and their effect on the epidemic. In this paper, we propose a new model that includes asymptomatic carriers while maintaining the structure and assumptions of the classical models. We validate our model and show that it replicates the results found by both the classical epidemiological methodologies and new research models. We develop a data-driven application of our model that can accurately identify asymptomatic individuals among the population.

10:00-10:25am

Interpretable Machine Learning: Application to Triage and Reassessment Guidelines for Ventilator Rationing

Presenter:

Julien Grand-Clément, Assistant Professor, Information System & Operations Management Department, HEC Paris

Additional Authors: 

Carri Chan, Professor, Division of Risk and Operations, Columbia Business School, Columbia University;

Vineet Goyal, Associate Professor, Industrial Engineering and Operations Research Department, Columbia University;

Elizabeth Chuang, Associate Professor, Department of Family and Social Medicine, Albert Einstein College of Medicine

Abstract:

Algorithms for sequential decision-making in healthcare often suffer from a lack of interpretability. Decision trees have gained interest in recent years, due to their performances and their interpretability. We present a model to compute interpretable sequential policies that have a tree structure, called tree policies. We apply our model to learn triage and reassessment guidelines for ventilator allocations to patients affected by Sars-Cov-2. We find that official New York state guidelines, based on risk scores, may not outperform First-Come-First-Served guidelines, because they are not adjusted to the specifics of Sars-Cov-2. Our simple tree policies improve upon First-Come-First-Served guidelines and New York State guidelines by reducing the number of excess deaths associated with various hypothetical levels of ventilator shortage.

10:25-10:50am

Optimizing Inter-Hospital Patient Transfer Decisions During a Pandemic: A Queueing Network Approach

Presenter:

Vahid Sarhangian, Assistant Professor, Department of Mechanical and Industrial Engineering, University of Toronto

Additional Authors: 

Timothy C. Y. Chan, Department of Mechanical and Industrial Engineering, University of Toronto

Frances Pogacar, Ontario Health

Erik Hellsten, Ontario Health

Fahad Razak Division of General Internal Medicine and Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto 

Amol Verma, Division of General Internal Medicine and Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Unity Health Toronto

Abstract: 

Geographical mismatch between demand for care and availability of healthcare resources has been a major challenge during the COVID-19 pandemic. As such, inter-hospital patient transfers have emerged as a key aspect of the pandemic response in many countries. In this work, we propose and investigate inter-facility patient transfer policies with the goal of alleviating hospital congestion and reducing inequities in the distribution of COVID patients across the health system. We propose a queueing network model that captures the salient features of patient flow from acute wards to ICUs within each hospital, and between wards and ICUs of different hospitals in a network. We formulate the problem of determining optimal patient transfer policies between the hospitals as a stochastic control problem and develop a solution method by leveraging a deterministic fluid approximation of the queueing network. Using real data during the pandemic from a network of 21 hospitals in the Greater Toronto Area of Ontario, Canada, we validate our queueing model and conduct a comprehensive case study to examine the value of guiding patient transfers using our proposed approach. Compared to the no-transfer policy, the expected reduction in the number of patient days above a 95% occupancy threshold under our policy can be up to 43.6% in wards and 30.9% in ICUs, and the expected reduction in COVID load inequity can be up to 53.2%. In addition, we find that our optimized transfers outperform the actual transfer decisions made during the periods considered in the case study.

10:50-11:15am

Applied Artificial Intelligence at Mayo Clinic

Presenters: 

Atul Dhanorker, Principal Health System Engineer at Mayo Clinic 

Adam Resnick, Health System Engineer - Strategy Department, Mayo Clinic, Rochester, Minnesota

Abstract:

Artificial intelligence is increasingly impacting the clinical and administrative practices of healthcare. Algorithms across a spectrum of maturity from discovery, translation, and application are under development at Mayo Clinic. This presentation will highlight two case studies of artificial intelligence, discussing the development of algorithms to predict emergency department transfer and to automate breast cancer risk assessments. We will highlight challenges in the development of algorithms from clinical data, results from algorithm development, and lessons learned in the implementation of algorithms into practice.

11:15am to 11:40am

Healthcare Analytics for Managing and Predicting Waits in Practice

Presenters: 

Dr. Saba Vahid, Group Manager, Data and Decision Sciences, Ontario Health 

Dr. Tahera Yesmin, Data Scientist, Data and Decision Sciences, Ontario Health

Dr. Shabnam Balamchi, Decision Scientist, Data and Decision Sciences, Ontario Health

Abstract:

Data and Decision Sciences team at Ontario Health supports clinical and information programs across the organization by providing analytical models for planning, prediction, and evaluation purposes. In this talk we provide an overview and preliminary results for two ongoing projects. The first is focused on cancer radiation therapy wait times and determining a safe “capacity buffer” for long range planning purposes to meet wait times targets. The second project aims to assess the feasibility and accuracy of emergency departments wait times predictions based on data available at triage, using machine learning models.

11:50am to 12:15pm

Treatment Planning of Victims with Heterogeneous Time-sensitivities in Mass Casualty Incidents

Presenter:

Nan Liu, Associate Professor, Carroll School of Management, Boston College

Additional authors: 

Yunting Shi, Ph.D. candidate, Antai College of Economics and Management, Shanghai Jiao Tong University. 

Guohua Wan, Professor, Antai College of Economics and Management, Shanghai Jiao Tong University.

Abstract: 

This research is motivated by operational challenges faced by decision and policy makers in humanitarian response to mass casualty incidents (MCIs), such as the recent 2020 Aegean Sea Earthquake and 2022 Hunga Tonga Eruption and Tsunami. When these large-scale disasters strike, emergency response resources are overwhelmed by a sudden jump in demand, making the rationing of resources inevitable. We obtained a unique timestamps dataset of surgeries operated in a field hospital set up in response to a large-scale earthquake. Analyzing this dataset, to the best of our knowledge, provides evidence for the first time that patient surgical times may depend on wait times in the context of MCI. Informed by this finding, we develop decision-making models to aid treatment planning for MCIs. A distinguishing feature of our modeling framework is to simultaneously consider patient health deterioration and wait-dependent service times in making decisions. We find that sometimes patients with a less critical initial condition should have higher priority than their counterparts, in order to do the greatest good for the greatest number---the priority order depends on patient deterioration trajectories and the resource (i.e., treatment time) availability. This deviates from the current emergency response guidelines which suggest always giving priority of treatment to those patients whose initial health conditions are more critical. A counterfactual analysis based on our data shows that adopting our model would significantly reduce both the total number of clinically deteriorated patients (by 32%) and the surgical makespan (by 8%) compared to using the then-implemented treatment plan; care coordination among surgical teams could further reduce the number of deteriorated. By demonstrating the value of adopting data-driven approaches in MCI response, our research holds strong potentials to improve emergency response and to inform its policy making.

12:15pm to 12:40pm

Does Delay Stimulate Speedup? Evidence from Operating Rooms

Presenters:  

Yiwen Jin, Ph.D. candidate, Sauder School of Business, University of British Columbia

Additional Authors: 

Yichuan (Daniel) Ding, Assistant Professor, Desautels Faculty of Management, McGill University 

Steven Shechter, Associate Professor, Operations and Logistics, Sauder School of Business, University of British Columbia

Abstract: 

Delays and schedules are ubiquitous in all areas. People's responses towards delays are also complex. Using an administrative data set with 8,961 surgery cases from a children's hospital that includes both real surgery duration and the schedule, we investigate the surgical team's reaction to delays in shift schedules. We find that surgical teams expedite in the subsequent surgery when they fall behind the schedule. For ten minutes delay the surgical team will act faster by reducing the surgery duration of focal case by approximately two minutes on average. To deal with the dynamic structure and the following endogeneity issues in the model, we construct a dynamic panel model and utilize the Arellano-Bond GMM estimator. We further explore the heterogeneity of such impact among different departments and we identify the experience of surgeons as a moderator. We find that the surgical teams with senior surgeons tend to expedite while junior surgeons are variable. We further probe into speedup’s impact on surgical quality. Our research provides important guidance for hospital managers to understand the dynamics in operating rooms and to improve the quality of service.

12:40-1:05pm

How can effective patient scheduling help reduce medical procedure backlogs

Presenter: 

Hossein Abouee-Mehrizi, Associate Professor, Department of Management Sciences, University of Waterloo; Canada Research Chair in healthcare Analytics

Abstract: 

Reduced capacity for elective medical procedures during the Covid-19 pandemic has created significant backlogs. The CovidSurg Collaborative (CSC) estimated in June 2020 that more than 28 million elective surgeries would be cancelled or delayed worldwide in the initial 12 weeks of COVID-19. A key question that health policy makers are facing is how to clear this backlog without significant additional costs. The goal of this talk is to provide insights on how effective patient scheduling can help alleviate this backlog. We consider advance scheduling of multi-class patients where different classes have different service durations. We assume that, on any given day, the scheduling calendar is open only for the next fixed number of days, and patients who are not scheduled on the day of their arrivals will be waiting to be scheduled in the future. We formulate the problem in a dynamic setting and derive some properties of the problem and use those properties to propose a dynamic policy for advance patient scheduling. Using Ontario MRI data, we show that the proposed policy outperforms the current practice. We also observe that considering a proper sequencing together with the proposed policy can increase the daily number of procedures performed, and thus, reduce the backlogs to the pre-pandemic level faster than the current practice.

Hosted by: Chair of Scientific Program: Opher Baron, Distinguished Professor of Operations Management, Rotman School of Management

Roundtable Co-Host: Sandra Rotman Centre for Health Sector Strategy

Driving Excellence in Healthcare Management - The Sandra Rotman Centre for Health Sector Strategy is a research, education and policy centre aimed at generating insights for governments, organizations and other key stakeholders facing complex healthcare challenges.

Roundtable Co-host:  TD Management Data and Analytics Lab 

The TD Management Data and Analytics Lab promotes cutting-edge analytic tools in business through teaching and research and is a central source of knowledge and expertise in data science, AI, and machine learning applications.

Questions? Rosemary Hannam: rosemary.hannam@rotman.utoronto.ca