Abstract
Study design
Consensus study.
Objectives
The purpose of this study is to define a standardized (network) outcomes set for traumatic spinal cord injury (t-SCI), covering the patient journey from acute to chronic rehabilitation phase, including patient-relevant outcomes, adequate measurement instruments, as well as case-mix and risk factors.
Setting
Acute Spinal Cord Injury (ASCI) Unit Nijmegen, the Netherlands.
Methods
A modified Delphi method was performed, including a multidisciplinary panel of 19 health-care professionals with experience in t-SCI management. Formal consensus was reached after two web-based surveys, a face-to-face meeting, and a final confirmation round (threshold consensus: 70%).
Results
In the first two Delphi rounds, 18/19 invited panelists (94.7%) responded and 10 panelists participated in the final meeting. The prefinal set was confirmed by all panelists. The standard set encompasses the three-tiered outcome hierarchy and consists of patient-reported and clinician-reported outcome domains and measurement instruments. Consensus was reached to include survival, degree of health or recovery, time to recovery, and return to normal activities, disutility of care or treatment process, sustainability of health and nature of recurrences, and long-term consequences of therapy. A measurement schedule was defined as well as for proposed casemix and risk factors, including demographics, clinical status, and treatment process.
Conclusion
A standard set of network outcomes is developed that could be implemented in hospitals and rehabilitation centers involved in the treatment of t-SCI. Using this standard set, comparison of the quality of care is possible and prognostic prediction of outcomes of treatment is feasible, so that each patient receives the right care at the right time in the right place.
Similar content being viewed by others
Introduction
Traumatic spinal cord injury (t-SCI) is an impairing neurological condition with a devastating impact for patients and their families [1]. T-SCI is defined as damage to the spinal cord, caused by an external physical impact resulting in a temporary or permanent neurological deficit [2, 3]. The estimated incidence of t-SCI in the Netherlands is 11.7 per million residents per year, which are about 200 new cases of t-SCI each year [4, 5].
The adagium ‘time is spine’ has been a central theme in the management of t-SCI, assuming there is a critical time window after the primary injury during which the severe effects of the secondary injury mechanisms should be diverted [6]. The management of patients with t-SCI is complex and involves multiple stages of care, which often continues for years after the initial injury. As the incidence of t-SCI is low and the time factor is very important in the acute care, specialized Acute Spinal Cord Injury (ASCI) Units (in Dutch: Acute Ruggenmergletsel Unit, ARU) have been implemented. ASCI units were first developed in 1944 in England and have been implemented in North America [7]. The ARU of Nijmegen was founded in 2015 with the aim to improve the quality of care in the acute and the primary rehabilitation phase of the patient journey of t-SCI, which is in accordance with the aim of ASCI units. These ASCI units enable people with t-SCI to receive the necessary expertise and care immediately after the injury [8]. The ARU of Nijmegen is the first in the Netherlands to provide the full cycle of care in a regional network. Although many studies have been conducted to evaluate new interventions for t-SCI, it appears that little is known about standardization of care in t-SCI. Several studies have described the status of SCI care in other countries and a systematic review by Maharaj mentioned a lack of standardization within ASCI units on a global scale, with significantly different outcomes reported across published studies [9,10,11]. In a recent study by Fransen et al. the authors concluded that a large practice variance exists in pre-hospital and acute t-SCI management among ASCI-units and level 1 trauma centers in the Netherlands [12]. In addition, a study by Nijendijk et al. showed that a significant proportion of young survivors with t-SCI are not referred to specialized rehabilitation centers [4]. On the other hand, the population of elderly patients with SCI has increased and the majority is being referred to nursing homes that are not specialized in SCI.
To gain insight and improve the quality of care, several (inter)national databases exist in which data about t-SCI are collected. For example, the European Multicenter Study about Spinal Cord Injury (EMSCI) database supports a network consisting of 19 European SCI-centers [13]. In this network, systematic and standardized data are collected from patients with acute t-SCI. The core data set of The International Spinal Cord Society (ISCoS Core Data Set) contains recommendations for uniform reporting of basic data regarding SCI to facilitate accurate descriptions of patient populations and for globally meaningful comparisons of results between published studies [14,15,16]. The EMSCI database and the Dutch SCI Database (Nederlandse Dataset Dwarslaesie; NDD) [17] – which is derived from the ISCoS Core Data – are the data sources currently being collected in the Netherlands, covering several separate domains of SCI. However, these data sets do not collect interdisciplinary and collective measurements, derived from different phases in t-SCI treatment. Therefore, there is no uniformity to monitor and evaluate the quality in the full cycle of care of the “patient’s journey”, based on the three different domains (Body Functions and Body Structure, Activity and Participation) of the International Classification of Functioning, Disability and Health [18].
Systematic and continuous outcome monitoring by harmonization and standardization of treatment outcomes is important for assessing the value of provided care [19]. Case-mix adjustment, a statistical process that aims to account for differences in the mix of patient’s attributes across definitive patient cohorts, is required to make comparisons between hospitals with different patient populations [20]. Routine use of patient-reported outcome measures (PROMs) are valuable tools for patients and clinicians for (shared) decision-making, and enable comparisons of providers’ performance facilitating quality improvement [21, 22]. A patient outcomes registry supports outcome monitoring and is expected to contribute to the improvement of the quality of care and contribute to a better understanding of practice variation and treatment outcomes [23]. The data can be used to describe care patterns, including suitability of care and inequalities in care provision. A standard set of outcomes that follows the patient journey in the full cycle of care, from the acute phase of the injury until the discharge after rehabilitation phase, does not exist in the Netherlands. Therefore, the purpose of this study is to develop a standardized (network) outcomes set for t-SCI based on existing data sets, which includes patient-relevant outcome domains, adequate measurement instruments, and case-mix variables and risk factors based on the three domains of the ICF [18]. This standardized outcomes set is to be implemented in our ASCI Unit (ARU Nijmegen) and to be implemented for use nationwide.
Methods
Design
A modified Delphi study was performed, which consisted of a preparatory stage in which a preliminary standard network outcome set, including modifying case-mix and risk factors was developed, and three formal consensus rounds (modified Delphi study). These Delphi rounds consisted of two online surveys and one (final) online face-to-face meeting. This study was registered in the Core Outcome Measures in Effectiveness Trials (COMET) database (ID 1703) [24] and guidelines for the development of a core set of outcomes were applied [25]. The recommendations from the Guidance on Conducting and Reporting Delphi Studies (CREDES) [26] and the Core Outcome Set standards for reporting (COS-STAR Statement) were used to report the study findings [27]. The study was performed between March 2019 (study preparations) and November 2020 (data synthesis and reporting).
Project team
The study was performed by an independent project team, not participating in the Delphi study, consisting of a spine surgeon (MS), a rehabilitation physician (MVvdH), a methodologist (MvH) and a research assistant (TvS). Members of the project team work at the Radboud University Hospital and the Sint Maartenskliniek (rehabilitation center). The project team was responsible for the design and conduct of the full study. They performed the day-to-day activities, design, and key aspects of the study. An independent moderator was added to the project team to lead the third (online) face-to-face meeting.
Expert panel
The expert panel consisted of 19 members from ARU t-SCI network in Nijmegen, The Netherlands, consisting of an academic hospital and a rehabilitation center. Health care professionals involved in the full cycle of care in t-SCI with at least three years of experience were included. The multidisciplinary panel consisted of orthopedic surgeons (n = 3), rehabilitation physicians (n = 3), neurosurgeons (n = 2), physiotherapists (n = 4), occupational therapists (n = 2), specialized nurses (n = 3), a psychologist (n = 1), and a physician assistant (n = 1), with an average of 14 years (range 3–35) of experience in the treatment of traumatic spinal cord injury.
Preparatory stage - preliminary outcome set
The project team drafted a preliminary set of outcomes, casemix and risk factors, and adequate measurement instruments, based on peer-reviewed literature and various (inter)national studies and databases such as the EMSCI database, the ISCoS Core Data Set and the NDD [14,15,16,17, 28]. The standard set included demographics and patient characteristics, admission and discharge date of initial acute and rehabilitation care, cause of trauma, presence of vertebral fractures, spinal surgery, and neurological and respiratory status [14, 15].
The preliminary set was developed based on the three-tiered outcome measures hierarchy as suggested by M. Porter [29]. To measure outcomes for any medical condition, Porter proposed a hierarchy based on three tiers: the achieved or retained health status, the process of recovery, and the sustainability of health. The top tier is most important and the lower tiers are dependent on the higher ones. Each medical condition has its own outcome measures and Porter states that a data set should consist of at least one outcome dimension at each tier [29]. The three-tiered outcome hierarchy was used as a framework to capture full cycle of care, which is endorsed by the International Consortium for Health Outcomes Measurement (ICHOM) [30] and the Organization for Economic Cooperation and Development (OECD) Health Ministers [31]. The initial preliminary set consists of 77 patient-reported and clinician-reported outcome domains, casemix and risk factors and subsequent measurement instruments, covering the acute surgical, subacute and chronic rehabilitation phase, and served as input for the Delphi rounds.
Procedure modified Delphi rounds
The first two rounds were performed through an online survey, and the third and final round was being held during an online face-to-face meeting. A final online round, including all panelists was held to confirm the pre-final set. The panelists were asked to answer “Yes or No” to questions related to outcome domains, measurement instruments (patient-reported and clinician-reported), and potentially relevant case-mix and risk factors. For each question the panelists could provide feedback in open text boxes. They were asked to consider their own professional opinion as well as the evidence provided from the literature and were encouraged to provide free text feedback. After each round the panelists received an anonymized feedback report with an overview of votes and a summary of the panelist’s feedback. This report was used as input for subsequent rounds.
The threshold for consensus was reached if ≥70% of panelists agreed on inclusion of a proposed outcome domain, measurement instrument, and contributing risk factor. Consensus of <30% lead to direct exclusion of the specific item. An agreement between 31% and 70% for the proposed outcome, factor or instrument lead to inclusion for consensus in the second Delphi round [25].
During the first online Delphi round (Delphi 1), formal consensus was sought for proposed outcome domains and measurements, as well as for casemix and risk factors. In the second online Delphi round (Delphi 2) the panelists were asked to vote and re-evaluate items for which no consensus was reached in the previous round and to confirm those items that already reached formal consensus.
The third Delphi round yielded an online face-to-face meeting using a videoconferencing platform (Zoom Video Communications Inc., 2016) with a multidisciplinary subgroup of the expert panel, consisting of 10 panelists, covering all relevant medical domains (orthopedic surgeons [n = 2], rehabilitation specialists [n = 2], physiotherapists [n = 2], occupational therapist [n = 1], psychologist [n = 1], physician assistant [n = 1] and a specialized nurse [n = 1]). The meeting was guided by an experienced, independent moderator, a methodologist who was not familiar with the management of t-SCI. Consensus was reached after discussion, for outstanding inconclusive outcome domains, measurement instruments and case-mix and risk factors, or the timing of data collection. After the online meeting, pre-final draft ‘factsheets’ were compiled of the standard outcomes set to be confirmed by all panelists. These factsheets included: the outcome domains, the appropriate combination of PROM’s, clinician-reported outcomes measures, the case-mix and risk factors, and the recommended timing of the data collection.
Results
Response
18 panelists of the 19 invited panelists responded to the first round of the online survey, achieving a response rate of 18/19 (94.7%). 16 panelists of the 19 invited responded to the second round of the online survey, achieving a response rate of 16/19 (84.2%). 10 panelists participated in the face-to-face meeting forming an expert panel. In Fig. 1 the Delphi process is shown, which includes the flow of agreed outcome domains, casemix and risk factors, and corresponding measurement instruments.
Delphi Round 1 (Figs. 2 and 3)
Consensus was reached on 24/26 outcome domains and on 13/24 measurement instruments. For the casemix and risk factors, consensus was reached on 17/20 items and on 13/19 proposed measurement instruments. A complete overview of the comments per outcome domain, measurement instrument, and casemix and risk factors in Round 1 is provided in the Supplementary Material.
Delphi Round 2 (Fig. 4)
Consensus was reached on 9/10 previously accepted dimensions and on 10/14 outcome domains and measurement instruments that did not reach consensus in Delphi Round 1. The project team reconsidered four items (“Hoffer test and 2 MWT” [5/12; 42%]; “CHART-SF” [8/18; 44%]; “USER-P” [10/18; 56%] “Dural tear/CSF leakage” [10/18; 56%]) from Delphi round 1 that did not reach consensus and suggested to exclude these items based on the feedback of panelists. In Delphi 2 the expert panel unanimously agreed to remove these items from the standard set of outcomes.
The previously accepted dimension “Time to recovery and time to return to normal activities” was discussed. Panelists mentioned that these are not outcome measurements, but rather process indicators. The project team re-formulated these items and proposed a new timing schedule as well (see Supplementary Material). No consensus was reached on dural tear and/or CSF leakage (11/16; 69%), rate of re-operations (11/16; 69%), and the duration of surgery (11/16; 69%). These items returned in Delphi 3.
A complete overview of the comments and discussion in Round 2 is provided in Supplementary Material.
Delphi Round 3 (Fig. 5)
During the final online face-to-face meeting the expert panel discussed the items that did not reach consensus in previous rounds, being the dimension of “Time to recovery and time to return to normal activities” and the outcome domains CSF leakage, rate of re-operations and duration of surgery.
Also, the project team enclosed four additional items based on received feedback on missing items (redefining timing of treatment and recovery, categorization of the rehabilitation program, WISCI II for walking ability, timing of data collection following EMSCI). The expert panel reached consensus on all the items (n = 8). A complete overview of the discussion in Round 3 is provided in Supplementary Material.
Outcome domains
Timepoint start rehabilitation phase
The variability of the timepoint when rehabilitation starts, during or after hospital admission, was discussed. The panel agreed that it is of relevance to identify the exact starting point of the rehabilitation (e.g., rehabilitation physician gave indication rehabilitation and active multidisciplinary rehabilitation had started [physiotherapist and occupational therapist are involved in treatment]). Suggested was to rename the second timepoint (arrival at hospital), and to define it as ‘admission to intervention center’.
The expert panel agreed (9/10 [90%]) on the timing of treatment and recovery as suggested by the project team (see Supplementary Material).
Duration of surgery, rate of re-operations and dural tear and/or CSF leakage
Duration of surgery was not deemed relevant for a standard outcomes set because of the amount of variation caused by other external factors, e.g., surgeon’s skills and difficulty of the procedure. Therefore, ‘duration of surgery’ was removed from the standard set (10/10 [100%]). Re-operations are rare and after discussion the ‘rate of re-operations of the spine’ was removed from the standard set (10/10 [100%]). Unanimous agreement was reached (10/10 [100%]) to exclude dural tear and/or CSF leakage from the standard set.
Casemix and risk factors
Rehabilitation program
The expert panel proposed that the earlier categories (Individual Physiotherapy, Technology Assisted, Group Walking Training, Fitness) should be revised and expanded. New categories should be defined in the future (e.g., based on the treatment frame SCI rehabilitation [in Dutch: ‘Behandelkader Dwarsleasie’]) [32]. The expert panel unanimously agreed (10/10 [100%] that the rehabilitation program should be categorized and included in the standard set.
Measurement instruments
Walking Index for Spinal Cord Injury II (WISCI II)
Issues were raised on including the WISCI II to measure walking ability, which was considered essential to include next to 10MWT and TuG. The 10MWT, TuG, WISCI II are required instruments in the EMSCI database and as such needed for a complete impression of walking ability. Two panelists withheld from voting, and consensus was reached (7/8 [87.5%]) resulting in the inclusion of the WISCI II in the standard set.
Arm and hand function
From a research perspective the relevance of the Graded Redefined Assessment of Strength, Sensibility and Prehension (GRASSP) test is acknowledged [33]. The use of the GRASSP test in daily practice is questioned and as such, it was removed from the standard set (10/10 [100%]).
Time schedule for data collection
The panelists agreed (9/9 [100%]) to include the time schedule for assessments as proposed following the EMSCI guidelines, providing a fixed time schedule in which patients can be tested and documented after SCI consisting of an acute care phase (0-15 days) and a follow-up phase (4 weeks, 12 weeks, 24 weeks and 12 months) [34].
Final standard network outcomes set
The pre-final standard set was confirmed by all panelists (response rate 18/19 [95%]) and is provided in Table 1. The confirmed casemix and risk factors are provided in Table 2.
Discussion
Standardization and systematically monitoring outcome domains is essential for the improvement of quality in clinical practice and multicenter research in t-SCI. This study provides a standard network outcomes set for patients with traumatic spinal cord injury (t-SCI). This set is based on peer-reviewed literature, existing databases as the European Multicenter Study about Spinal Cord Injury (EMSCI) and the Dutch SCI Database (Nederlandse Dataset Dwarslaesie; NDD), and includes patient-relevant outcome domains covering the full cycle of care for patients with t-SCI. The current network core outcomes set slightly differs from other proposed sets (e.g., ICF Core Sets for individuals with spinal cord injury in the early post-acute context) because this set starts from the acute phase of SCI, in contrast to other data or outcomes sets that start measuring outcome domains from the post-acute phase [35, 36]. A recent study by Khosravi et al. mentioned the importance of the Delphi method to establish standards that can be used by all hospitals and rehabilitation centers involved in the care of patients with t-SCI [37]. In this study, consensus was reached among a panel of multidisciplinary experts of a regional Acute Spinal Cord Injury (ASCI) unit, on a network outcomes set consisting of outcome domains (n = 23), casemix and risk factors (n = 15), corresponding measurement instruments, and a schedule for timing of assessments.
To our knowledge, this standard network outcomes set is the first that provides patient-relevant outcome domains for the full cycle of care in form acute to chronic SCI care. This is based on the three domains of the ICF, assessed with patient-reported outcome measures (PROMs) to be used in daily clinical practice in the Netherlands. PROMs are widely used in orthopedic surgery and have the potential to provide information on value-based care, to improve shared decision making and endpoints that patients are interested in [22, 38]. Furthermore, it has been recognized that health care professionals and patients differently appraise certain outcomes. For example, physicians treating patients with t-SCI often assume that walking ability is the most important goal to achieve. However, most patients value improved sexual- or bladder function as most important [39, 40]. It is necessary that these mismatches are identified and resolved to improve the treatment of SCI.
During the Delphi study several items led to discussion among panelists. For example, the timing of the start of the rehabilitation phase was questioned. To resolve this issue guidelines are proposed for the management of SCI [41]. A study by Fehlings et al. suggests that rehabilitation should be offered to patients with SCI when they are medically stable and can tolerate required rehabilitation intensity. The evidence for this suggestion is weak and future studies are needed to explore the timing of rehabilitation after SCI [42]. This could lead to standardization of t-SCI care and in future this might contribute to improvement of the quality of care. Another example is the removal of ‘duration of surgery’ as a risk factor from the standard outcomes set. To our knowledge, no studies exist on the effect of duration of surgery on outcome in patients with t-SCI undergoing surgery. In other medical domains prolonged duration of surgery resulted in an increased risk for infection in total knee arthroplasty (TKA) and total hip arthroplasty (THA) [43,44,45,46,47,48,49]. As yet, duration of surgery was deemed irrelevant by the panelists to include in the standard outcomes set. In future, its relevance could be reconsidered when evidence is available for t-SCI.
Several rehabilitation items elicited discussion as well. Firstly, the item “rehabilitation program in a specialized SCI center” which was obtained in the standard set, needs further elucidation. Although rehabilitation after t-SCI follows (inter)national guidelines, multidisciplinary rehabilitation programs, do not follow a uniform and standardized protocol [41]. Protocolized care is not able to address the large interindividual variation of a patient with SCI, as the consequences of provided care and the patient-related factors determine which needs are most important for the patient. In future, to optimize rehabilitation programs, it is recommended to study specific treatment elements, such as individual physiotherapy, technology assisted training (e.g., exoskeleton) and walking training in group. Optimizing rehabilitation programs based on patient characteristics will ultimately result in achieving improved outcome for these patients.
This study has some limitations. First, we did not include a patient representative in the expert panel, so only health care professionals were involved in the modified Delphi process. Recent guidelines on Delphi procedures have proposed involving patient representatives in the process [50, 51]. In retrospect, it would have been relevant to involve a patient representative, since the patient can certainly contribute to improving the treatment process. In order to include the patients’ perspective, in a following study the current developed network core outcomes set will be validated using a patient sample.
Another limitation is that we did not engage program or data administrators to ensure feasibility of the collection of data. When implementing the data set, we recommend to include data administrators to ensure the feasibility of the data collection.
Finally, selection bias might have occurred. The expert panel was recruited from one ASCI network (ARU Nijmegen, The Netherlands). The ARU network in Nijmegen is a patient-centered network, containing the full cycle of care in the management of t-SCI. The response rates in the Delphi rounds were around 100%, extensive reviews of the literature were performed, and efforts were taken to strictly follow existing methodological guidelines. A next step would be to collaborate with and implement the standard outcomes set in comparable networks.
In this study a standard network outcomes set was developed, which is unique in its form. The standard outcomes set follows the patient journey for patients with t-SCI in the full cycle of care, from the acute phase of the injury until the discharge after rehabilitation phase. This network outcome set encompasses the recommendations of the EMSCI and NDD and can be used to compare casemix and risk-adjusted outcomes across regions, studies, and registries in order to gain insight and improve the quality of care for t-SCI in clinical practice. This network outcomes set will be implemented and evaluated in the ASCI Unit (ARU Nijmegen) and when feasible nationwide implemented. Implementation and use of these standard outcomes serve different purposes: individual evaluation of patient care, continuous evaluation of the quality of the care provided in a defined subgroup of patients, case mix and risk-adjusted benchmark between professionals and institutions, value-related healthcare, research (e.g., comparative effectiveness of procedures). By sharing the acquired knowledge with the patient, this ultimately supports patient-specific decision-making, and ultimately improves outcomes and value of care for t-SCI patients.
Data availability
The data can be found within the published article and its supplementary files.
References
Alizadeh A, Dyck SM, Karimi-Abdolrezaee S. Traumatic spinal cord injury: an overview of pathophysiology, models and acute injury mechanisms. Front Neurol. 2019. https://doi.org/10.3389/fneur.2019.00282.
Jia X, Kowalski RG, Sciubba DM, Geocadin RG. Critical care of traumatic spinal cord injury. J Intensive Care Med. 2013;28:12–23.
Ahuja CS, Wilson JR, Nori S, Kotter MRN, Druschel C, Curt A, et al. Traumatic spinal cord injury. Nat Rev Dis Prim. 2017. https://doi.org/10.1038/nrdp.2017.18.
Nijendijk JHB, Post MWM, van Asbeck FWA. Epidemiology of traumatic spinal cord injuries in the Netherlands in 2010. Spinal Cord. 2014;52:258–63.
Spek J. Zorgstandaard Dwarslaesie, revisie 2017. Dwarslaesie Organisatie Nederland: Nijkerk. 2017.
Badhiwala JH, Ahuja CS, Fehlings MG. Time is spine: a review of translational advances in spinal cord injury. J Neurosurg Spine. 2018. https://doi.org/10.3171/2018.9.SPINE18682.
Parent S, Barchi S, LeBreton M, Casha S, Fehlings MG. The impact of specialized centers of care for spinal cord injury on length of stay, complications, and mortality: a systematic review of the literature. J Neurotrauma. 2011;28:1363–70.
Tator CH, Duncan EG, Edmonds VE, Lapczak LI, Andrews DF. Neurological recovery, mortality and length of stay after acute spinal cord injury associated with changes in management. Paraplegia. 1995;33:254–62.
New PW, Simmonds F, Stevermuer T. Comparison of patients managed in specialised spinal rehabilitation units with those managed in non-specialised rehabilitation units. Spinal Cord. 2011;49:909–16.
Pattanakuhar S, Kammuang-lue P, Kovindha A, Komaratat N, Mahachai R, Chotiyarnwong C, et al. Is admission to an SCI specialized rehabilitation facility associated with better functional outcomes? Analysis of data from the Thai Spinal Cord Injury Registry. Spinal Cord. 2019;57:684–91.
Maharaj MM, Hogan JA, Phan K, Mobbs RJ. The role of specialist units to provide focused care and complication avoidance following traumatic spinal cord injury: a systematic review. Eur Spine J. 2016;25:1813–20.
Fransen BL, Hosman AJ, van Middendorp JJ, Edwards M, van Grunsven PM, van de Meent H, et al. Pre-hospital and acute management of traumatic spinal cord injury in the Netherlands: survey results urge the need for standardisation. Spinal Cord. 2016;54:34–8.
Curt A, Schubert M. European Multicenter Study about Spinal Cord Injury (EMSCI). 2004. https://www.emsci.org/index.php/project/the-project. Accessed 13 April 2021).
DeVivo M, Biering-Sørensen F, Charlifue S, Noonan V, Post M, Stripling T, et al. International spinal cord injury core data set. Spinal Cord. 2006;44:535–40.
DeVivo MJ, Biering-Sørensen F, New P, Chen Y. Standardization of data analysis and reporting of results from the International Spinal Cord Injury Core Data Set. Spinal Cord. 2011;49:596–9.
Biering-Sørensen F, Devivo M, Charlifue S, Chen Y, New P, Noonan V, et al. International Spinal Cord Injury Core Data Set (version 2.0)-including standardization of reporting. Spinal Cord. 2017;55:759–64.
Nachtegaal J, van Langeveld SA, Slootman H, Post MWM. Implementation of a standardized dataset for collecting information on patients with spinal cord injury. Top Spinal Cord Inj Rehabil. 2018;24:133–40.
World Health Organization. International classification of functioning, disability and health: ICF. Vol. 10. World Health Organization: Geneva. 2001.
Porter ME, Larsson S, Lee TH. Standardizing patient outcomes measurement. N Engl J Med. 2016;374:504–6.
Iezzoni L. Risk adjustment for performance measurement. In: Smith PC, Mossialios E, Papanicolas I, Leatherman S, editors. Performance measurement for health system improvement: experiences, challenges and prospects. Cambridge: Cambridge University Press; 2009, p. 251–85.
Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013. https://doi.org/10.1136/bmj.f167.
Rolfson O, Bohm E, Franklin P, Lyman S, Denissen G, Dawson J, et al. Patient-reported outcome measures in arthroplasty registries: Report of the Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries. Part II. Recommendations for selection, administration, and analysis. Acta Orthop. 2016;87:9–23.
van Hooff ML, Jacobs WCH, Willems PC, Wouters MWJM, Kleuver Mde, Peul WC, et al. Evidence and practice in spine registries. Acta Orthop. 2015;86:534–44.
Standard set of network outcomes for traumatic spinal cord injury: a consensus-based approach using the Delphi method: COMET Initiative. 2020. https://www.cometinitiative.org/Studies/Details/1703. Accessed 9 May 2021.
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012. https://doi.org/10.1186/1745-6215-13-132.
Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31:684–706.
Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Devane D, et al. Core outcome Set–STAndards for reporting: the COS-STAR statement. PLoS Med. 2016. https://doi.org/10.1371/journal.pmed.1002148.
Biering-Sørensen F, Alai S, Anderson K, Charlifue S, Chen Y, DeVivo M, et al. Common data elements for spinal cord injury clinical research: a National Institute for Neurological Disorders and Stroke project. Spinal Cord. 2015;53:265–77.
Porter ME. What is value in health care? N Engl J Med. 2010;363:2477–81.
International Consortium for Health Outcomes Measurement (ICHOM) n.d.; 16. 2013. https://www.ichom.org/. Accessed 13 April 2021.
Organization for Economic Cooperation and Development (OECD) n.d. 2011. https://www.oecd.org/. Accessed 13 April 2021.
Nederlands-Vlaams Dwarslaesie Genootschap. Behandelkader Dwarslaesie. 2011.
Kalsi-Ryan S, Curt A, Verrier MC, Fehlings MG. Development of the Graded Redefined Assessment of Strength, Sensibility and Prehension (GRASSP): reviewing measurement specific to the upper limb in tetraplegia. J Neurosurg Spine. 2012;17:65–76.
EMSCI. Time Schedule - European Multicenter Study about Spinal Cord Injury n.d. 2004. https://www.emsci.org/index.php/project/the-project/time-schedule. Accessed April 13, 2021.
Kirchberger I, Cieza A, Biering-Sørensen F, Baumberger M, Charlifue S, Post MW, et al. ICF Core Sets for individuals with spinal cord injury in the early post-acute context. Spinal Cord. 2010;48:297–304.
Scheel-Sailer A, Lampart P, Selb M, Baumberger M, Gmünder HP, Sigrist-Nix D, et al. The Nottwil standard-development and implementation of an international classification of functioning, disability and health-based clinical standard assessment for post-acute rehabilitation after newly acquired spinal cord injury. Front Rehabilit Sci. 2021. https://doi.org/10.3389/FRESC.2021.720395.
Khosravi S, Khayyamfar A, Shemshadi M, Koltapeh MP, Sadeghi-Naini M, Ghodsi Z, et al. Indicators of quality of care in individuals with traumatic spinal cord injury: a scoping review. Glob Spine J. 2022;12:166–81.
Wilson I, Bohm E, Lübbeke A, Lyman S, Overgaard S, Rolfson O, et al. Orthopaedic registries with patient-reported outcome measures. EFORT Open Rev. 2019;4:357–67.
Simpson LA, Eng JJ, Hsieh JTC, Wolfe DL. The health and life priorities of individuals with spinal cord injury: a systematic review. J Neurotrauma. 2012;29:1548–55.
Estores IM. The consumer’s perspective and the professional literature: What do persons with spinal cord injury want? J Rehabil Res Dev. 2003;40:93–8.
Fehlings MG, Tetreault LA, Wilson JR, Kwon BK, Burns AS, Martin AR, et al. A clinical practice guideline for the management of acute spinal cord injury: introduction, rationale, and scope. Global Spine J. 2017. https://doi.org/10.1177/2192568217703387.
Burns AS, Marino RJ, Kalsi-Ryan S, Middleton JW, Tetreault LA, Dettori JR, et al. Type and timing of rehabilitation following acute and subacute spinal cord injury: a systematic review. Glob Spine J. 2017;7:175–94.
Ravi B, Jenkinson R, O’Heireamhoin S, Austin PC, Aktar S, Leroux TS, et al. Surgical duration is associated with an increased risk of periprosthetic infection following total knee arthroplasty: a population-based retrospective cohort study. EClinicalMedicine. 2019;16:74–80.
Cheng H, Chen BPH, Soleas IM, Ferko NC, Cameron CG, Hinoul P. Prolonged operative duration increases risk of surgical site infections: a systematic review. Surg Infect. 2017;18:722–35.
Anis HK, Sodhi N, Klika AK, Mont MA, Barsoum WK, Higuera CA, et al. Is operative time a predictor for post-operative infection in primary total knee arthroplasty? J Arthroplast. 2019;34:331–6.
Surace P, Sultan AA, George J, Samuel LT, Khlopas A, Molloy RM, et al. The association between operative time and short-term complications in total hip arthroplasty: an analysis of 89,802 surgeries. J Arthroplast. 2019;34:426–32.
Ong KL, Lau E, Manley M, Kurtz SM. Effect of procedure duration on total hip arthroplasty and total knee arthroplasty survivorship in the United States medicare population. J Arthroplast. 2008;23:127–32.
Kessler S, Kinkel S, Käfer W, Puhl W, Schochat T. Influence of operation duration on perioperative morbidity in revision total hip arthroplasty. Acta Orthop Belg. 2003;69:328–33.
Duchman KR, Pugely AJ, Martin CT, Gao Y, Bedard NA, Callaghan JJ. Operative time affects short-term complications in total joint arthroplasty. J Arthroplast. 2017;32:1285–91.
Khodyakov D, Grant S, Denger B, Kinnett K, Martin A, Peay H, et al. Practical considerations in using online modified-delphi approaches to engage patients and other stakeholders in clinical practice guideline development. Patient. 2020;13:11–21.
Légaré F, Boivin A, van der Weijden T, Pakenham C, Burgers J, Légaré J, et al. Patient and public involvement in clinical practice guidelines: a knowledge synthesis of existing programs. Med Decis Mak. 2011;31:45–74.
Acknowledgements
The authors thank the panelists (M. van Bilsen, E. Brouwers, A. Derks, P. van den Heuvel, A. Hosman, T. Janssen, K. Kitamura, M. de Kleuver, H. van de Meent, M. van Olst, H. Bongers-Janssen, T. van Diemen, P. Koomen, H. Rijken, C. Tatu, T. Terwindt, S. van der Zanden) who participated in the modified Delphi rounds and provided useful input for the set of network outcomes. The moderator, Prof. Dr. P. van der Wees, is thanked for his efforts in the final Delphi round. Prof. Dr. M. de Kleuver and Drs. H. Bongers-Janssen are thanked for participating in the steering committee and providing useful feedback.
Author information
Authors and Affiliations
Contributions
MP, MVvdH and MvH had the idea for the study. MP and MvH contributed to the study design. TvS prepared and executed the consensus rounds. TvS, MP and MvH analyzed the data. All authors contributed to interpretation of the results. TvS, MP, MVvdH and MvH drafted the manuscript. All authors critically revised the manuscript, provided feedback, and approved the final version.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
About this article
Cite this article
van Schaik, T.J.A., Pouw, M.H., Curt, A. et al. Standard set of network outcomes for traumatic spinal cord injury: a consensus-based approach using the Delphi method. Spinal Cord 60, 789–798 (2022). https://doi.org/10.1038/s41393-022-00792-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41393-022-00792-z
This article is cited by
-
Identifying erosion risk criteria for coastal city sustainability by using a Hyper-Delphi-hierarchy model: a case study of Kuala Terengganu, Malaysia
Environment, Development and Sustainability (2024)