Introduction

Physician burnout and reduced work satisfaction are linked to increased electronic health records (EHRs) usage1,2,3, and specifically within the “inbasket,"4,5 where physicians exchange messages with patients and other healthcare providers. The inbasket, serving as the EHR’s communication hub, is a common functionality provided by EHR vendors. As physicians’ clinical workload increases, so does the amount of inbasket work, creating a burden that is challenging to manage during clinical hours6. To respond, physicians invest additional hours outside their official working hours, known as “work-outside-work"7,8 which exacerbates burnout. The inbasket work has been characterized as “Sisyphean"- an involuntary, ubiquitous, after-hours second job4, that can create a potentially 24/7 work environment for physicians which impacts their physical and mental wellness9. The increasing workload in inbasket may force the physicians to reduce clinical work and the time allocated to patients’ visits10,11, or leave medicine altogether creating further challenges for health services delivery that is already plagued by shortages of physicians12,13.

Physician time is a key resource in healthcare service delivery11. Already a challenge before the pandemic, physicians’ time in inbasket grew due to COVID-19-related disruptions4 that continue to transform how healthcare services are delivered14 and how patients access their routine care15. Patients have increasingly embraced digital health services such as telemedicine, patient portals and messaging applications that provide with greater access to their physicians16,17,18. A surge in telemedicine utilization, initially seen as a potential replacement for in-person visits, has evolved into a co-existing model as per more recent trends15,19, that may further increase physicians’ workload. Moreover, prior research suggests that physicians servicing higher percentage of appointments through telemedicine engage in increased levels of work-outside-work. This further suggests telemedicine, as currently delivered, is not as efficient, and likely to increase physicians’ work burden8. How increased telemedicine use leads physicians to spend more time in inbasket is unknown and requires in-depth investigation of their inbasket action logs.

On the other hand, patient portal messages are a new source of physicians’ asynchronous, electronic, non-visit care (NVC)20. Since the pandemic, patients have significantly increased their digital communication with their physicians16, leading the latter to spend more time in their inbasket21. To support physicians grappling with overloaded inbaskets, healthcare systems have deployed various inbasket management strategies such as, eliminating or suppressing messages that are of low clinical value, delegating, or reroute messages to other team members (e.g., advanced practice providers or APPs) for better inbasket coverage, adopting billing policies that would discourage patients from sending avoidable messages22, and developing smart functionalities (e.g., Macros) that enable physicians to “complete” inbasket messages more efficiently16,23 – each with a varying degree of effect24. While patients send messages to physicians’ inbasket for various reasons, such as prescription refills, appointment requests, and prior communication clarifications, the messages that are to “Get Medical Advice" (patient’s medical advice request, PMAR) often necessitate direct physicians’ review, making existing inbasket management strategies, such as delegating, and eliminating less effective. Despite the prior evidence on increasing PMAR messages in inbasket since pandemic25, existing research has only explored the broad impact of all inbasket messages on physicians’ workloads and burnout5,9. To understand the specific effects of increased PMAR messages on physicians’ time, further research is needed. Moreover, existing research on the impact of inbox messages has predominantly concentrated on primary-care physicians9,26,27. The influence of these messages on non-primary care physicians remains underexplored.

To address the above concerns, this study aims to investigate how the combined increase in PMAR messages and the expansion of telemedicine services, driven by COVID-19 disruptions, has impacted the amount of time physicians spend managing their inbasket. We examine the time spent in inbasket by ambulatory care physicians in one large health system (New York University Langone Health, NYULH) in New York City, and assess three key factors: (1) number of PMAR messages they received, (2) their specialties (whether they are primary care or non-primary care role), and (3) their telemedicine adoption, and the impact these factors have on their work-outside-work.

The study utilized five key measures: clinical load (cl), primary care (pc), PMAR messages received (PMARs), work-outside-work (WOW) and telemedicine intensity (ti). All measures were derived from data collected by Epic Signal, which reports average values per reporting period. Further details are provided in the Methods section.

Results

Descriptive analysis, provider specialties, clinical load

The majority of physicians in our cohort were from internal medicine subspecialties (e.g., cardiology, pulmonology), followed by ambulatory surgery (including general surgery and surgical subspecialists) and general medicine practice (e.g., family medicine). See Table 1 for further details. Among internal medicine subspecialties 289 physicians (16.84%) were practicing in Internal Medicine or Family Medicine department, and were treated as primary care physicians in the rest of the analysis. In total 240 cases involving approximately 4.6% of all physicians, total number of inbasket messages received by them were available, but data specifically on PMARs were missing, which were estimated using multiple imputation.

Table 1 Specialty of physicians included in the study (N=1716)
Full size table

Physicians on average completed 864 visits (min: 117, max: 4008, SD: 342.05) in the four months, with an average of more than 7 visits per day per physician. Majority (85.80%) of them were delivered in-person. The average proportion of total visits per physician that were telemedicine-based was 0.14 (SD: 0.21) (see Fig. 1). A multinominal logistic regression used to evaluate the change in ti over time (i.e., month of reporting) among physicians did not find any evidence of ti significantly changing during the study time period (F1, 6716 = 2.27, p value = 0.11).

Fig. 1
figure 1

Histograms depicting distribution of physicians’ clinical load and telemedicine intensity.

Full size image

Our results showed physicians’ average WOWadj was 9.48 minutes (min: 0.04 mins, max: 12.18 mins, median: 6.33 mins, SD: 10.46 mins). Additionally, the average WOWadj was higher for non-primary care physicians (mean: 9.62mins, median: 6.37 mins) than their primary-care peers (mean: 8.80 mins, median: 6.02 mins). In contrast, primary care physicians, on average, received more PMARs per appointment (mean: 0.61, median: 0.41), than their non-primary care physicians did (mean: 0.52, median: 0.26). Figure 2 shows the distributions of three continuous measures PMARadj, ti, and WOWadj among primary care and non-primary care physicians. One-way analysis of variance (ANOVA) found means of these two groups to be significantly different for all three measures.

Fig. 2: Distributions of key measures among primary care (n = 289) and non-primary care (n = 1427) physicians.
figure 2

a PMARs per appointment (PMARad); b telemedicine intensity (ti); c work-outside-work per appointment (WOWadj).

Full size image

Association between key measures

Prior to investigating the effect of PMARs and telemedicine intensity (ti) on physicians’ work-outside-work (WOW), we examined the association between these three measures using Pearson’s correlation coefficient. We discovered robust and positive correlations between physicians’ PMAR messages and their WOW. This association held true whether we considered unadjusted measures (coef. = 0.26, p value < 0.001, 95% CI [0.24, 0.29]) or measures adjusted (PMARadj, WOWadj) for clinical load (coef. = 0.22, p value < 0.001, 95% CI [0.20, 0.24]). On the other hand, ti displayed a positive correlation with PMARadj (coef. = 0.13, p value < 0.001, 95% CI [0.10, 0.16]), but had a very small negative correlation with WOWadj (coef. = −0.078, p value < 0.001, 95% CI [−0.13, −0.03]). Further investigation through dichotomizing ti based on median split revealed a more nuanced association between ti and WOWadj - that for physicians with high ti had a negative association with the WOWadj (coef. = −0.25, p value < 0.001, 95% CI [−0.33, −0.16]), whereas, for physicians with low telemedicine intensity, ti had a positive association with WOWadj (coef. = 0.23, p value < 0.001, 95% CI [0.18, 0.29]). Overall, the findings suggest that PMARs have a strong, positive relationship with physicians’ WOW, and that telemedicine may have a more nuanced effect on WOW depending on the physicians’ level of telemedicine load.

Effect of PMARs and telemedicine on work-outside-work

Our regression analysis found a statistically significant effect of PMAR volumes on the duration of time that physicians spend in WOW, irrespective of whether we considered the measures unadjusted (F1,1716 = 497, p value < 0.001) or adjusted for both measures at the per-appointment level (F1,1716 = 340, p value < 0.001). Additionally, our investigation also unveiled a significant disparity in the influence of PMAR per appointment (PMARadj) between primary care and non-primary care physicians. As PMARadj increased, non-primary care physicians experienced a more pronounced rise in work-outside-work per appointment (WOWadj) compared to their primary care counterparts (see Fig. 3).

Fig. 3
figure 3

Predicted work-outside-work per appointment (WOWadj) based on primary-care and non-primary care physicians’ PMARs per appointment (PMARadj).

Full size image

To evaluate the impact of telemedicine on PMARs, we investigated the effect of telemedicine intensity (ti), primary care (pc) and their interaction on PMAR per appointment (PMARadj). We found both the main effects of ti (coef: 0.50, Std. error: 0.04, p value < 0.001), and pc (coef: 0.21, std. error: 0.03, p value < 0.001) and their interaction (coef: −0.88, std. error: 0.16, p value < 0.001) to be significant. Our result showed that physicians’ specialties moderated the effect of telemedicine intensity on PMAR volumes. As the telemedicine intensity increased primary care physicians received less PMARs per appointment, whereas physicians in non-primary care or specialties received more PMARs with the increase in their ti (see Fig. 4).

Fig. 4
figure 4

Predicted PMARs per appointment (PMARadj) based on primary-care and non-primary care physicians’ telemedicine intensity (ti).

Full size image

To evaluate the effect on work-outside-work (Fig. 5), we conducted an analysis using a three-way interaction model, wherein the main effects of PMARadj, pc, and ti, along with their interactions were evaluated for their potential impact on WOWadj. The results are summarized in Table 2. We did not find any evidence of the three-way-interaction significantly influencing WOWadj (F1,1716 = 57.14, p value = 0.51). However, the two-way interaction between ti and pc was found to have a significant effect on WOWadj. Further investigating this two-way interaction between with physicians’ ti as both a continuous variable and dichotomized (high or low) based on median split (Fig. 5a), we consistently observed the interaction to have opposing effects based on physicians’ specialty. For primary-care physicians with a high telemedicine load, the WOWadj attenuated, while for non-primary care physicians WOWadj increased with the increase in physicians’ ti (see Fig. 5b).

Fig. 5: Effect of telemedicine service on work-outside-work per appointment (WOWadj) among primary care and non-primary care physicians.
figure 5

a Dichotomized telemedicine intensity (ti); b as a continuous measure.

Full size image
Table 2 Three-way interaction result to estimate the effect of PMAR per appointment (PMARadj), telemedicine intensity (ti), and physicians’ specialties (pc) on their WOW per appointment (WOWadj)
Full size table

Discussion

The study aimed to evaluate the impact of two distinct digital health services on physicians’ workload: first, the level of patients’ digital medical advice request (PMAR) sent to physicians’ inbox (inbasket)), reflecting asynchronous clinical work, and second, physicians’ level of engagement in telemedicine services, intended to substitute some synchronous in-person work. Our findings reveal a significant influence of PMARs on physicians’ work-related hours, evident in their engagement in after-hours clinical work (work-outside-work). Physicians with higher level of PMARs spent more time on work-outside-work, a pattern that persisted after controlling for variations in physicians’ clinical load. The impact of PMARs on work-outside-work varied across physicians’ departments or specialties. Notably, non-primary care physicians or specialists, with increased PMARs, dedicated more time to work-outside-work compared to primary care physicians. Regarding telemedicine, our study shows telemedicine load has a more nuanced effect on physicians’ workload depending on their specialties and the proportion of their clinical time dedicated to telemedicine. In summary, while engagement in telemedicine has a significant (but waning than what previously reported8,28) effect on physicians’ effect on physicians’ work-outside-work, further analysis revealed contrasting effects on primary care and non-primary care physicians. Primary care physicians, with increased telemedicine visits, experience a decrease in PMARs and work-outside-work. Conversely, non-primary care physicians receive more PMARs, and subsequently, spend more time in work-outside-work as the proportion of telemedicine visits increased.

Health institutions have recognized physicians’ burnout concerns due to increasing inbasket workload, and are deploying strategies to address it16,24 e.g., filtering out low-clinical-value messages (elimination), delegation - rerouting messages to Advanced Practice Providers (APPs), and allocating dedicated resources to triage patient messages. Initial investigation suggests a positive effect of implemented strategies with a notable up to 75% reduction in certain message categories in physicians’ inbasket due to the elimination of unnecessary messages. Further research is needed to assess how these strategies have impacted non-primary care physicians and specialists with a particular focus on comparing their experiences with those of primary care physicians. Our overall observations show that non-primary care physicians or specialists dedicate more time to after-hours clinical work. The difference in patient panel among specialists could be a potential reason behind doing additional after-hours clinical work - PCPs typically manage ’fixed’ panel of mostly long-term patients, while specialists have a more dynamic mix of transient and long-term, necessitating additional time for EHR work. Future studies can explore the effect of panel size by comparing patient types between primary-care and non-primary care physicians.

Prior studies have suggested lack of familiarity with telemedicine29, difficulties due to switching frequently from one type of visit to another30, are some of the challenges physicians are facing with telemedicine. Whether these challenges affect specialties more severely than primary care that needs to be investigated. As our institution resumed in-person services post-pandemic waves, primary care practices shifted telemedicine use to less complex tasks that do not require patient’s physical presence, potentially explaining its attenuating effect on primary care physicians’ work-outside-work. Further investigation is needed into the utilization of telemedicine service in specialties to understand what factors contribute to non-primary care physicians providing more telemedicine services to receive more PMARs. Future studies can investigate each of the potential factors independently in multiple cohort studies. Such investigations can inform the health organizations to create policies for optimizing the physicians’ schedule.

Recent work has advocated for a “digital minimalism” approach to addressing physician burden and burnout31. From this lens, healthcare organizations and policymakers need to be thoughtful of how their policies may inadvertently encourage PMARs and the potential impact on physician workload. To reduce the burden of PMARs on physicians, healthcare organizations should consider implementing policies and procedures that nudge patients to more carefully use the PMAR channel with their physicians, divert them to less burdensome tools, limit the number of non-urgent PMARs or provide additional support and resources to help physicians manage their inboxes more efficiently. Potential optimizations might include enhanced digital self-service tools to answer PMARs autonomously and AI/ML based models that automatically classify acuity from patient requests and triage the messages. Recent advancements in generative AI and large language models (LLMs) are being explored to draft responses that physicians could then edit32 for more efficient resolution of patient requests while maintaining quality and empathy in responses. Telemedicine must offer more than just marginal benefits for patients to justify physicians’ investment of time28. Future research should further investigate the relationships between subtypes of PMARs and WOW/burden as well as the impact of the aforementioned optimization efforts.

The study has limitations, including the use of a dataset from a single EHR system, which limits the generalizability of the findings. Considering the widespread adoption of the same EHR vendor, Epic, globally, and the deployment of similar digital health technologies across numerous health system, comparable outcomes can be anticipated in a broad context. Limitations in our Epic Signal data set preclude the ability to review and analyze physician EHR activity with sufficient granularity beyond certain time periods; e.g., time periods more specific than a calendar month or physician activity log data at smaller than 15-minute increments. Our analysis is also constrained by the limited timeframe of data coverage, and the presence of seasonal variations in disease outbreaks, which can affect the results. Lastly, the study did not examine the effect of other factors that could affect physician workload, such as the complexity of patients’ conditions, their social determinants of health, physician characteristics (age, years in job, panel size, part-time vs full-time, writing style etc.) or the physicians’ work environment. Future in-depth investigation can control for these factors.

In conclusion, the study found that PMARs and telemedicine have a significant impact on physicians’ workload, and telemedicine may have a more nuanced effect depending on the physicians’ level of telemedicine load. Our findings suggest that reducing PMAR volumes and improving the management of in-basket messages could help reduce physicians’ work-outside-work. Further research is needed to explore how PMARs, and other digitally induced burdens affect physician workload to support the development of effective mitigation approaches.

Methods

Data source

Data was obtained as part of an EHR-based retrospective cohort study of 1716 ambulatory care physicians continuously practicing (defined as an average of at least one appointment per day in the reporting period) at any New York-based NYULH practice network for four months between November 27, 2022 - March 25, 2023. Non-physician practitioners (e.g., advanced-practice providers) and residents were expected to have different workflows and patient- facing responsibilities and thus, were not included in the study cohort. The data were obtained using Epic Signal, which provides comprehensive data on the time clinicians spent on different EHR interfaces based on user action logs. Additional EHR data on telemedicine visits that occurred during the specified time period and involved same set of physicians were extracted via Epic’s Clarity reporting database, aggregated to provider level and connected with the Signal data based on matching provider identifiers.

The study was deemed part of quality improvement and all data was collected as part of routine care. Ethics review and informed consent were not sought because the study met the criteria for exemption from such review according to NYULH institutional policy.

Study measures

This section defines the key measures used in the analysis.

Clinical load (cl)

Clinical load refers to the amount of clinical or patient care responsibilities that a healthcare provider is responsible for, and is typically measured panel size, percent of appointment slots filled33. Prior research has shown an association between clinical load and work-outside-work burden7,34,35. This study measured clinical load as the number of appointments or visits they completed during the reporting period. This was measured using the average number of appointments per clinical day as obtained from Signal, multiplied by number of clinical days (days with at least one appointment) in the reporting period.

Primary care (pc)

A primary care practice serves as the patient’s entry point into the health care system. Prior research points to the magnifying workload of primary care physicians (PCP) as organizations transitioned to ambulatory care: the PCP is now expected to gain content on all patient encounters (whether initiated by PCP or not) and coordinate effectively36. This study used physicians’ department and specialty information available from Signal data to classify them as primary care and non-primary care. Physicians from internal medicine or family medicine were grouped as primary-care and the rest were treated as non-primary care physicians.

PMAR messages received (PMARs)

This measure refers to the total volume of messages initiated by patients to “Get Medical Advice” that physicians receive exclusively in their inbasket during the reporting period. Messages that were sent to a shared inbasket (pool messages) and managed by multiple clinicians (APPs) were excluded from this count. Signal provides the number of such messages for a physician normalized by each day the physician logged into the system, and we calculated the total volume of PMARs by multiplying the average daily number of such messages by the number of days the physician logged into the system during the reporting period.

Work-outside-work (WOW)

The work-outside-work measure reflects after-hours clinical work of physicians based on the time they spend in the electronic health record (EHR) system outside of their scheduled patient hours7,8. To measure this, we combined the time spent in the system outside of scheduled hours on scheduled days (scheduled hours are determined from Epic Cadence scheduling data plus two 30-minute “buffer" periods added before the start of first appointment and after the end of last appointment) and time spent on unscheduled days (i.e., minutes spent in the system on days with no scheduled patients), and measured the work-outside-work in minutes. Although, Epic’s own metric pajama time (PT: the average number of minutes per day spent in charting activities on weekdays outside a standard 7 AM to 5:30 PM workday and any time on weekends) was considered as an alternate representation of after-hours clinical work, recent studies called into question PT’s appropriateness for this purpose26. Therefore, we did not use PT and instead relied on our own measure of work-outside-work.

Telemedicine intensity (ti)

Telemedicine intensity reflects the proportion of clinical care the physician provides via telemedicine8. To compute telemedicine intensity, we categorized each patient appointment completed by a physician during the reporting period as either telemedicine or in-person using information queried from the EHR. Then, we aggregated all telemedicine visits for each physician and measured the proportion of telemedicine to total visits completed (numerator: number of telemedicine visits, denominator: total no of visits), resulting in a telemedicine intensity measure with a value that could range from 0 to 1.

Analytical approach

In this section, we characterize different methods used in this study for understanding the relationship between key measures listed above, discussing the appropriateness of their respective assumptions. We first estimated clinical load (cl), and telemedicine intensity (ti) for all physicians in the EHR that met our inclusion criteria. Next, we adjusted the WOW and PMAR measures for variations in physicians’ cl and computed WOW per appointment (WOWadj) and PMAR per appointment (PMARadj).

Correlation analysis

We first analyzed the bivariate correlations to measure the association of PMAR and WOW both unadjusted and adjusted, and their association with telemedicine intensity (ti). We hypothesized that physicians’ PMAR would be positively associated with their WOW. The missing data points were addressed prior to this step. We assumed that for any physician with missing data on PMAR message volumes or WOW measures, data were missing at random (MAR)37, and therefore, adopted intent-to-treat analyses using multiple imputations to estimate the missing values38.

Regression analysis

To evaluate the effect of PMAR on WOW, we used multiple linear regression analysis in which WOW per appointment (WOWadj) was the dependent variable, and PMAR per appointment (PMARadj) is the predictor. We further hypothesized (H1) that this effect varies based on the physician’s specialty. To address this, we introduced a binary variable (pc) to distinguish between primary-care and non-primary care physicians. Our analytical approach involved a hierarchical linear regression analysis, in which initially the main effects of PMARadj and pc were examined, followed by assessing their interaction (see Fig. 6a). Next, following the same hierarchical process, we measured the effect of interaction between telemedicine intensity (ti) and pc on PMARadj. Our hypothesis (H2) was telemedicine generates more PMARs and this relationship is moderated by physicians’ specialties (see Fig. 6b). Finally, we used a three-way interaction among PMARadj, pc, and telemedicine intensity (ti) in predicting WOWadj. We hypothesized (H3) that the size or direction of the interaction effect between the PMARadj and pc on WOWadj varies depending on the ti of the physician. In this model, pcc is a primary moderator, and ti is a secondary moderator (see Fig. 6c). All analyses were conducted in R statistical environment.

Fig. 6: Conceptual models showing the relationships tested between key measures.
figure 6

a Interaction effect of the physicians’ specialties (pc) and PMARs per appointment (PMARadj) on their work-outside-work per appointment (WOWadj) in minutes (H1); b interaction effect between telemedicine intensity (ti) and pc on PMARadj (H2); c three-way interaction model with pc as the primary moderator, and ti as the secondary moderator (H3).

Full size image