INTRODUCTION

Clinical genetics is a labor-intensive, time-consuming, and predominantly cognitive service.1,2 Metrics are important to illustrate service inputs/outputs, and hence justify the required resources. Within our organization, Monash Health, the service delivery environment is changing due to expanded access to investigative modalities such as exome sequencing. The statewide strategic plan Genetic and Genomic Healthcare for Victoria 20213 reported a 50% demand increase for clinical genetics services from 2011 to 2016. To improve care and ensure equitable access in the face of such rapidly increasing demand, cohesive data are critical for strategic service development.

Collection of metrics that appropriately capture service provision changes is challenging due to a combination of resource constraints and the absence of standardized metrics. Additionally, a paucity of published contemporary service provision data and the lack of national/international benchmarks make it difficult to quantify changing demand or assess relative performance. This hampers service delivery planning and, ultimately, the ability of a service to optimally employ resources to provide timely and appropriate care.

To establish appropriate criteria for assessing service delivery, it is pertinent to consider both historical and contemporary literature. This article reviews the literature to identify metrics evaluating clinical genetics service delivery and assesses the validity of these metrics. Suitable refinements were considered and applied to local data to demonstrate utility. A broad framework of metrics was produced, suitable for service delivery evaluation and planning.

MATERIALS AND METHODS

Literature search

To investigate the evidence base for metrics evaluating service provision in clinical genetics, a literature search was undertaken. PubMed and Google Scholar searches were conducted of all publications prior to 31 January 2019 using terms related to clinical genetics (e.g., genetics/clinical genetics/medical genetics) and service delivery (e.g., service delivery/workforce/time/benchmarking/metrics). References within articles of interest were explored for further relevant literature. Internet searches for gray literature, health service reports, regional/state strategic reports, and genetic provider training program data were completed. Data obtained from the literature search included metrics from previously published workload studies, workforce reports, and workforce surveys in clinical genetics or comparable specialties.

Data collection

Monash Health provides multisite adult tertiary inpatient (1846 beds), pediatric tertiary inpatient (230 beds, 64 neonatal intensive care unit/special care nursery [NICU/SCN] beds) and adult/pediatric outpatient services to southeastern metropolitan Melbourne and regional Victoria. Monash Health accounted for approximately 36% of general genetics (noncancer) referrals for the state of Victoria in 2017–2018 (unpublished data). The estimated outpatient source population base for the general genetics service is 2.6 million individuals.4 However, there are no defined statewide catchments and other services provide both standard and outreach outpatient services to a variable proportion of this population, limiting the ability to accurately define local full-time equivalent (FTE) per million capita (pMC) figures. The general genetics service is currently staffed by 6 clinical geneticists (2.09 FTE), 1 clinical genetics trainee (0.75 FTE), 8 genetic counselors (4.3 FTE clinical/management, 1 FTE research) and 3.3 FTE administration.

Local data were obtained through audit of the general genetics service; familial cancer genetic services were excluded. Acquired data includes the clinical activity of pediatric, adult, prenatal, and multidisciplinary/subspecialty genetics outpatient clinics, in addition to all inpatient consultations completed at Monash Medical Centre and Monash Children’s Hospital. Due to reporting cycles, referral data comprise January 2015 to September 2018, inclusive. Consultation data commences July 2014 and ends September 2018. Testing data were collated from records dated January 2014 to September 2018, inclusive. At the time of data collection 2018 was incomplete, therefore projections based on the first three quarters of 2018 were used for the comparison.

Data were obtained from Monash Health’s patient management database and our locally hosted genetics database. Single-gene, next-generation sequencing (NGS) panel, exome/genome sequencing, and segregation/cascade testing data was obtained by reviewing internal billing and tracking records. Excluded were the following tests commonly ordered independent of, or prior to, general genetics review: chromosomal microarray analysis, fragile X polymerase chain reaction (PCR), and methylation studies.

Contextual statewide data were provided by the Department of Health and Human Services Victoria.

RESULTS

Literature review

Workload

There is no published consensus regarding which metrics best monitor service delivery in clinical genetics. The following summary of data gathered and metrics employed in prior workload studies is derived from three surveys and four prospective studies published to date (Table 1).

Table 1 Studies of clinical genetics staff time required per new and review patient, excluding administration staff
Full size table

In 1987 and 1989, the mean clinical staff time spent on service delivery for new and returning families ranged from 2 to 4 hours and 2.1 to 3.1 hours, respectively.2,5 Surveys of US clinical geneticists published in 2002 and 2005 reported the mean time spent by clinical geneticists on new referrals was 3.1 hours (range 0.5 to >4 hours) and 2.8 hours (no range), respectively.6,7

McPherson et al.8 completed the first real-time study of workflow in 2008. Consistent with earlier studies, this demonstrated clinical throughput is limited by the time taken to complete direct patient/provider interactions and patient-related activities (PRA), a measure of indirect clinical and administrative requirements. PRA represented an estimated 63% of the overall clinical staff workload. Significantly, time requirements per new patient had increased to 7 hours.

A 2015 multinational survey evaluated the additional burden of new diagnostic approaches in clinical practice.9 Key findings included that only 25–42% of clinical staff time for a new pediatric patient was spent on face-to-face activities. This data confirmed that PRA remains the largest component of workload burden for clinical staff. Williams et al.10 assessed time taken for pretest case review and clinical counseling for genome sequencing, reporting a mean of 7 hours required per case. Importantly, this did not include post-test case resolution requirements. This would suggest that, in implementing genomics clinically, the combined pre- and post-test time requirements far exceed the mean for a new patient assessment reported in 1987 and 2008.2,8

A 2015 survey of American College of Medical Genetics and Genomics (ACMG) members highlighted that, on average, clinical geneticists were seeing 18 patients per week, almost double the 10 per week being seen in 2005.11 The authors found waiting times had increased, with 30% of patients waiting over 3 months, compared with 11% in 2005. There has not, however, been any post-2015 data clarifying the impact of wider genomic testing availability.

Workforce

Table 2 illustrates the varied staff resourcing pMC through time in select countries. Internationally, there is no consensus on the appropriate workforce of clinical geneticists pMC. 2003 survey data showed only 70% of US MD geneticists provided any direct patient care (DPC).12 The 2005 American Board of Medical Genetics and Genomics (ABMGG) workforce data did not capture active DPC FTE, limiting its utility.13 A 2019 report, based on 2015 US survey data, estimated clinical geneticist DPC FTE as just over 2 pMC.11 In 2016, the UK had a clinical geneticist workforce equivalent to ~3.5 pMC, but only 58% were working full-time.14,15

Table 2 Comparison of clinical geneticist and genetic counselor pMC workforces in select countries and jurisdictions 2003–2018
Full size table

In Australia, there was a mean of 1.78 DPC FTE pMC (range 0.8–2.38) in 2005.16 The absolute headcount was slightly increased in 2015 but that data did not clarify DPC FTE.17,18 Recent state data highlight large geographic variation in physician supply.4,19,20,21 In New South Wales22 and Victoria (unpublished data), 79% and 82% of FTE worked in general genetics, respectively.

Data for similarly consultative medical specialties are limited. A 2012 US neurologist workforce study showed staffing was equivalent to 52.14 pMC.23 Adequacy of that workforce was determined using average work relative value units (RVUs), finding there was a 10–20% nationwide FTE shortage. Presently, there is no such standardized method of determining workforce adequacy in clinical genetics.

Although there has not been an internationally accepted workforce benchmark for genetic counselors, the UK-based Association of Genetic Nurses and Counsellors (AGNC) recommend 10 DPC FTE pMC.24 Abacan et al.25 recently reported genetic counselor workforce figures from 28 countries but, crucially, without DPC FTE figures. In the United States, nationally, there are no published FTE figures. In 2016 it was estimated that 65% of genetic counselors worked in DPC.24 On a state level, Utah reported a 2018 workforce in excess of the AGNC recommendation, at 12 DPC FTE pMC.26

In Australia, public sector DPC FTE figures are captured. Workforce FTE has increased substantially in Tasmania27 and Victoria (unpublished data) since 2005; meanwhile Queensland reported a 6.9% reduction on a per capita basis to 2017.4,21 In Victoria, 63.3% of practicing genetic counselors work in general genetics. Australian data did not consider private practice or academic genetic counselors, although such roles were previously reported to constitute 18% of the workforce.28

An important factor in workforce planning is adequate generation and retention of qualified professionals (Table S1). The ABMGG has certified an average of 34 clinical geneticists annually (~0.1 pMC) over the last 10 years.13,29 As of July 2018, Canada had 46 trainees in their Canadian College of Medical Geneticists (CCMG)–accredited residency program, suggesting certification of ~9.2 trainees annually (~0.24 pMC).30,31 The UK had an average annual intake of 11.66 trainees (~0.17 pMC) from 2016 to 2018.15,32

Genetic counselor training program annual certifications are estimated at ~400 in the United States (~1.22 pMC), 20–25 in Canada (~0.54–0.67 pMC), 40 in the UK (~0.67 pMC), and 40 in Australia/New Zealand (~1.33 pMC).4,15,25,29,31 At this rate, it is estimated the United States will reach 10 DPC FTE pMC in 2023–2024.24

Metrics

Previously employed metrics were identified in the literature and appraised. Workload studies (Table 1) addressed clinical and administrative time spent per patient, time spent per patient overall, and time allocated to face-to-face and PRA (indirect) service delivery. However, time expenditure was estimated in several studies and, as McPherson et al. reported, accurately measuring the time dedicated to PRA per patient was not practical.8 Unless the challenges of accurately measuring these metrics can be overcome, their utility is limited. Workforce studies and surveys previously identified the workforce as either absolute clinical staff or clinical staff FTE, neither of which clarify the time available for DPC.

Framework metrics were selected considering the need for simple and inexpensive data collection over time, reliability, and their being consistently applicable across different health systems (Table 3). The metrics chosen measure service inputs and outputs in the context of staffing. The key input is the quantity of referrals received. The key outputs are the quantity of inpatient and outpatient consultations completed and the quantity, and type, of genetic testing completed.

Table 3 Proposed framework of service delivery metrics
Full size table

Application of proposed metrics to local data

Monash Genetics general genetics clinical staff DPC FTE pMC is estimated, based on a 36% share of the state referral base, at 0.9 and 1.85 for clinical geneticists and genetic counselors, respectively.

On an annualized basis, a 58% increase in patient referrals to the general genetics service from 2015 to 2018 has been observed. This combines all outpatient and inpatient referrals. Figure 1 illustrates this trend of increasing referrals over a 45-month period.

Fig. 1
figure 1

Monash Genetics total inpatient and outpatient referrals per quarter 2015–2018 line and markers) and order 2 polynomial trend line (dashed line) with R-squared value of 0.8421.

Full size image

Figure 2 shows the total number of consultations completed, divided into the proportions of new and review patients, set against clinical staff DPC FTE. Review patients have become a larger proportion of consultations completed over the 50-month period, from a low of 13.6% in early 2015 to a high of 33% in early 2018. This trend toward more review consultations as a proportion of total consultation workload is reflected in Fig. S1, although the R-squared value is low, indicating significant variance. The number of new consultations completed does not match the total referrals received due to a combined cancellation and failure-to-attend (FTA) rate averaging 26.7% (range 14.1−45.3%) (Fig. S2), together with an expansion of the clinic waiting times.

Fig. 2
figure 2

Monash Genetics quarterly new and review consultations (inpatient and outpatient) completed overlaid with absolute clinical geneticist and genetic counselor full-time equivalent (FTE).

Full size image

Inpatient consultations completed have increased since 2015; projected at 144% to the end of the 2018 calendar year (Fig. S3). Inpatient consultations, as a proportion of overall consultations, have increased from 2.5% to 4% over the last 4 years. In 2015, 20.3% of consultations completed were for patients seen more than once (Fig. 3). In the first three quarters of 2018 that increased to 59%, a significant change (χ2 = 108.188, df = 12, p value < 0.0001).

Fig. 3
figure 3

The proportion of total inpatient consultations in which a patient was reviewed by either a clinical geneticist or a genetic counselor a given number of times, 2015 vs. 2018. This data does not include genetic provider–patient care team interactions. *Data are for January–September 2018, inclusive

Full size image

From these results staffing ratios were calculated. These showed that new referrals per clinical staff member have increased 11% since 2015, with a 25% increase for genetic counselors (Fig. S4). Review consultations completed have increased 44% per clinical staff member with rises of 24% for clinical geneticists and 65% for genetic counselors (Fig. S5). Inpatient consultations have increased by 75% per clinical staff member, and by 45% for clinical geneticists, who undertake most inpatient consultations (Fig. S6).

Genetic testing requests have risen over the 4.75 years for which data are available (Fig. 4). Total tests ordered have increased 323%. Single-gene testing increased initially but has subsequently decreased, ending 26% higher than in 2014. NGS panel ordering has increased 110% since 2014. Ordering of exome sequencing, which became available to the service in 2014, has increased swiftly to a projected 276 orders in 2018—an annual increase of 42% from 2017. For every consultation completed, exome sequencing was pursued at an increasing rate over the audit period, from 1/860 in 2015 to 1/12 in 2018. Familial variant testing, which encompasses segregation and cascade testing, has increased 514% from 2014 levels with the greatest rise in 2017. This change in testing output, in both volume and type, is a significant change from expected levels (χ2 = 427.376, df = 12, p value < 0.0001).

Fig. 4
figure 4

Monash Genetics testing output 2014–2018 by total tests ordered (1) and each test type ordered (2–5). NGS next-generation sequencing.

Full size image

DISCUSSION

Based on literature review and local service data audit, we have developed a framework of metrics for standardized evaluation and monitoring of clinical genetics services. We have applied these metrics to our own service, demonstrating both absolute and proportionate changes in service demand. This analysis highlights the impact that greater awareness of clinical genetics and the changing availability of genomic testing has had on service delivery. This includes increasing referrals, greater demand for inpatient consultations, and increasing investigational complexity, all of which contribute to greater service demand.

Historical data are consistent with the concept that clinical genetics requires more time to review and diagnose patients than many other medical specialties.8,12 Evolution of increasingly complex diagnostic testing over the past three decades has further increased patient management requirements. The structured literature review conducted found no published peer-reviewed studies subsequent to 2008 that systematically quantified changes in overall clinical workload. Therefore, changes associated with the introduction of novel technologies such as microarray and massively parallel sequencing have not been captured.

As demonstrated, comparison of studies is difficult due to the lack of standardized data collection methods and/or inconsistent use of metrics. This in turn makes evaluation of a service’s changing workload and overall performance difficult, limiting the ability to plan service development and optimize care delivery.

Metrics that together appropriately approximate clinical genetics service delivery and form a framework for service provision evaluation were developed based on literature and review of local data. These metrics were applied to local data to demonstrate their practical use and provide some benchmarking data. Not all metrics were able to be derived from our local data, highlighting areas for possible improvement in data capture going forward.

Evaluation of metrics

Staffing

Prior literature has used FTE inconsistently, making valid comparison of data difficult. “Absolute clinical staff per capita” is not a suitable metric for this purpose, as it does not appropriately take account of partial employment and clinical staff involvement in nonclinical activities. FTE lacks utility as a measure without first defining the percentage of that FTE that is spent on direct patient care. While the FTE assigned by institutions to those providers with a mix of roles may be difficult to establish, an approximation is still worthwhile to enable comparison. Implementing this framework may incentivize health services to more accurately track division of FTE.

The FTE involved in DPC pMC, for each of clinical geneticists and genetic counselors, can be used to robustly assess clinically active staffing. This metric provides a denominator for the calculation of workload to staffing ratios (Fig. S4–S6). Capturing clinical staff FTE is crucial to inform the other proposed metrics and service planning overall. We recommend capturing FTE spent in clinical, research, and administration roles to provide a comprehensive picture of workforce utilization.

If relevant, FTE can be subdivided into metropolitan and regional components. The authors suggest regional telehealth and outreach clinics supported by metropolitan services be considered regional in this context. Quantifying clinical staff FTE in subspecializations, such as neurogenetics, is important to ascertain the relative service provision for each subspecialty and allows for comparison of subspecialties.

Genetic nursing is a well-established and significant component of the existing clinical genetics workforce in some, but not all, countries.33 Training pathways and formal recognition vary considerably across jurisdictions, so data collection around this metric presently requires interpretation in the local context.33,34 Genetic nursing FTE in DPC pMC should be recorded as a metric, consistent with other clinical roles.

Referrals and consultations

Referrals per quarter captures total new inpatient and outpatient intake; this metric measures service demand and is independent of staffing. Consultations completed per quarter per staff member indicates clinical service efficiency and is staffing independent but does not directly capture case complexity or associated PRA. It could be used as an aggregate measure across a service to account for such variation, rather than at the level of individual clinical staff. The ratio of new to review patients indicates caseload complexity, highlighting the need to see larger proportions of patients repeatedly, particularly to organize and discuss the results of complex or sequential investigations.

Quantifying cancellation and failure-to-attend rates captures service inefficiency, as time and resources expended on preparatory work are lost. It is possible some services show differences in cancellation/FTA rates due to systematic, geographic, and societal issues such as clinic/transport costs, poor transport infrastructure, or other socioeconomic challenges. If identified, some of these issues are potentially targetable with changes to service structure, systems, and delivery models.

The number of inpatient consultations completed per annum is important to measure, given inpatient consultations may have a disproportionate effect on workflow and workload, compared with planned and centralized outpatient consultations. Inpatient consultations are typically less time efficient, as reflected by the increased typical times for inpatient evaluation and management service (E/M) Current Procedural Terminology (CPT) codes compared with outpatients.35 No studies exist evaluating this within the field of clinical genetics but a single study of neurology inpatient consultations estimated they take twice as long as outpatient consultations.36 Should this hold true for clinical genetics, increasing inpatient demand would have a disproportionate effect on workload and workflow. The proportion of inpatient and outpatient cases requiring more than a single review reflects a combination of case and investigational complexity. These metrics flag the additional workload associated with communicating complex results to patients and, in an inpatient setting, to the primary treating team.

Testing

Cataloging genetic/genomic test ordering captures a component of PRA. The quantity of genomic tests ordered per consultation indicates the changing density and complexity of investigational ordering. This reflects additional workload associated with ordering complex testing. Across similar populations, this measure may also demonstrate relative under- or overservicing in diagnostic testing.

Time

Direct face-to-face time specifically captures patient–clinician interaction, while PRA shows associated preparatory, investigative, and administrative burden. If local systems permit robust and cost-effective collection of mean time measurements, this information provides an excellent basis for calculating adequate staffing requirements, where service demand is also accurately predicted. Unfortunately, as highlighted by previous studies, time measurements have been historically difficult to collect accurately and/or consistently. None of the published studies on workflow were able to directly measure these data points, instead including at least some estimation in their models.2,5,8,9,10 As McPherson et al. noted in their report of a real-time workflow study, a prior study in their institution “had to be abandoned due to the severe interference recording of specific tasks was having on workflow.”8

Application to dissimilar services

The metrics proposed can be applied in both the general genetics and cancer genetics settings. Differentiating these is important as general and cancer genetics practice may differ due to variations in staffing, case mix complexity, the proportion of inpatient consultations, and relevant testing methodologies. In services where general genetics and cancer genetics services are colocated, separation of metrics capture for each service is suggested so that dissimilar services are not inappropriately amalgamated. This principle could also be applied to other genetics services with different care provision models, such as prenatal genetic services.

Evaluation of local service

The application of these metrics to our local data identifies marked changes in clinical practice over the last four years. They highlight the value of metric collection and benchmarking to adequately resource services to match growing clinical demand.

From 2015 to 2018, the Victorian population has grown by 1.99–2.55% annually,18 but the Monash Genetics referral rate has increased by 10.7–22.6% annually. This change may be due to the advent of research-funded genomics testing, state-sponsored genomics testing, and greater physician and patient awareness of the clinical utility of genomic testing.

Consultations undertaken by Monash Genetics have increased over time but in a stepwise fashion due to intermittent addition of clinical FTE (Fig. 2). Although in absolute terms, the service is providing increasing episodes of care annually, the number of new patients seen stagnated over 2017–2018 due to the increasing proportion of review consultations. This coincides with the increase in genomic testing ordered by the service (Fig. 4) and is consistent with increasingly complex case management, reflected by the inpatient consultation data (Fig. 3).

Staffing ratios for several metrics (Fig. S4–S6) show increased workload over the period of data collection for each of new referrals and review and inpatient consultations completed. An important additional factor affecting workload and service delivery is the FTA and cancellation rates as these influence the service output. Clinical genetics requires crucial, and often lengthy, preparatory time prior to patient consultations. The time required for this, in cases that subsequently cancel or FTA, is not appropriately captured, and thus when reporting service delivery figures, this work is lost to measurement.

Inpatients consultations constituted a growing proportion of total consultations over the audited period. Historically, inpatient consultations typically led to further nonurgent investigation with outpatient follow-up. Recent research initiatives being conducted in Victoria have changed the inpatient testing landscape. In 2016–2017, Stark et al. conducted a rapid genomic diagnosis program that recruited NICU and pediatric intensive care unit (PICU) patients.37 Since 2018, a similar program evaluating trio genomic sequencing with a 5-day test completion target has been ongoing. Our service has participated in both programs, leading to a change in the local investigative pathway for many inpatients. Figure 3 demonstrates a mean annual inpatient consultation volume increase of 41% over a 3-year period, with a significant shift toward multiple consultations, reflecting the impact of rapid genomics. Given the reported clinical and economic benefits of rapid genomic testing,37 it is expected this pattern will persist, shifting further clinical genetics workload from an outpatient to an inpatient setting.

The recent return of single-gene test ordering to near-2014 levels, after increases earlier in the audit period, is likely due to preferential use of genomic testing for first-line investigation. Familial variant testing increased dramatically as singleton genomic tests led to segregation analyses in a large proportion of cases. Notably, trio testing may curtail this rise. Increased absolute and proportionate genomic test ordering is consistent with the increased number of inpatient consultations per patient shown in our data, as these extra reviews are required to organize consent, sampling, and for results disclosure. This could be expected to be replicated for outpatients, if measured. Given the increasing rate at which genomic testing is being pursued, it follows that there is likely to be a profound, if not presently quantifiable, associated increase in workload due to the additional consultations required, together with accompanying administrative and clinical interpretation tasks.

These data together form a strong basis for future service planning and valuable feedback to management and funding bodies. We have selected underlying data that are feasible to collect but still produce robust and representative data. We suggest employing this framework of metrics to produce benchmarks for internal and external service comparison and to allow coordinated health system planning.

Further research

Development of a standardized method of determining workforce adequacy in clinical genetics, based on average work RVUs or similar metrics, would assist in clarifying acceptable international benchmarks. Combining this with modeling on training program outputs, as well as physician retention and retirement timelines, will be important to guide strategic workforce planning. Cancer genetics and subspecialties with significant practice structure variation may require customized frameworks. This framework will capture areas of practice that require modification in response to shifts in technology and workload. Longitudinal data gained by implementation of this framework and publication of the outcomes will assist services worldwide in refining best practices.