Introduction

Prehospital care plays an essential role in the management of out-of-hospital cardiac arrest (OHCA)1,2,3. Previous studies have identified several components of prehospital management that is highly associated with OHCA patient outcome, including chest compression time, defibrillation, and transport3,4,5,6. However, it is still challenging to make clinical decisions in OHCA settings due to the difficulties in identifying the cause and estimating the patient's resuscitation prognosis.

It is difficult to predict whether return of spontaneous circulation (ROSC) can be achieved on the scene without providing advanced cardiac life support in a hospital, which makes it unclear when to perform “scoop and run” to the hospital for advanced cardiac life support (ACLS)7,8,9. To save time during transport, where resuscitation quality can degrade in a moving ambulance, it would be prudent to perform “stay and play” resuscitation if there is a high likelihood of ROSC occurring at the scene10,11,12,13. However, a hospital would obviously apply more resources and provide optimal management when ROSC is difficult due to unknown medical causes7,9. It makes it more difficult to make decisions during the prehospital phase because, although it is time-sensitive and influenced by the scene environment, less information and resources are available than during the in-hospital phase14,15.

Prior to this study, multiple publications suggested a machine learning-based prediction model of OHCA patient outcomes or analyzed the relevance of each predictor with ROSC16,17,18,19. Some models were built to be used in the prehospital phase for predicting on-scene ROSC20,21. The situation at the scene might keep varying, and is especially time-dependent in the prehospital stage of OHCA15,22. A real-time prediction model might therefore be beneficial for emergency medical service (EMS) prehospital resuscitation.

In this study, we aimed to develop a pre-hospital real-time cardiac arrest outcome prediction model using time-adaptive cohort to provide real-time prediction of ROSC on scene and further prognosis, during the EMS resuscitation attempt.

Methods

This study was approved by the Institutional Review Board (IRB) of the Samsung Medical Center (IRB No. 2022-04-093). The need for informed consent was waived by of the Samsung Medical Center IRB because of the retrospective, observational, and anonymous nature of the study. All the methods were performed in accordance with relevant guidelines and regulations. Because it was inappropriate, it was not possible to involve patients or the public in the design, conduct, reporting, or dissemination plans of our research.

We developed the pre-hospital real-time cardiac arrest outcome prediction (PReCAP) model to predict the ROSC on screen and other patient outcomes using prehospital input variables with time-adaptive cohort.

Study setting and population

The Pan-Asian Resuscitation Outcome Study (PAROS) database, which included OHCA patients transferred by emergency medical service (EMS) in Asia between 2009 and 2018, was used for this study. Seven Pacific Asia nations were included. Patients in the PAROS database who were under the age of 18 were excluded. Patients who were not transported by EMS transport but with private or public transport were also excluded. Patients without an EMS team resuscitation attempt or patients whose arrest occurred after hospital arrival were excluded. Finally, patients who were missing at least one of the essential time information, including the date and time of the arrest incidence and the time of call received at the dispatch center were excluded.

Primary outcome

This research aimed to predict ROSC at the scene. ROSC at the scene refers to ROSC during the prehospital phase, prior to hospital arrival Scene ROSC information was recorded by each country’s EMS and collected from the PAROS database.

Secondary outcome

The patient survival to discharge from the emergency department (ED), the 30-day survival rate, and the final Cerebral Performance Category (CPC) were secondary prediction outcomes in this study. Survival to discharge from the ED refers to whether a patient is admitted, transferred, or discharged for further care in a state of survival. Thirty days of survival refers to whether the patient survived 30 days following cardiac arrest. Whether the patient's final CPC was 1 or 2 was also predicted as an outcome.

Selection of predictor and preprocessing

From variables available in the PAROS database, we selected relevant variables to predict OHCA outcomes. Demographic information, including city, sex, race, and age was collected. Scene environment information, including location type and time, was included. Location type was divided into home residence, health care facility, public building, nursing home, street, industrial area, transport center, place of recreation, ambulance, and others. Prehospital cardiopulmonary resuscitation (CPR) information, including information about witnessed arrest, bystander CPR information, first responder, information about automated external defibrillator (AED) usage by bystander or EMS, first rhythm of patient, prehospital airway method, and prehospital epinephrine administration, was also included (Fig. 1, Table S1 of Multimedia Appendix 1).

Figure 1
figure 1

Overall study flow in the development of PReCAP (Pre-hospital Real-time Cardiac Arrest outcome Prediction model). CPR, cardiopulmonary resuscitation; EMS, emergency medical service; AED, automated external defibrillator; ER, emergency room; ED, emergency department; Dfib, defibrillation.

Full size image

Handling missing data

Patients with missing essential time information, such as time of arrival of the ambulance at the scene and its arrival at the ED were excluded from the study. Missing information about categorical variables were considered as ‘unknown.’ Missing information about numeric variables was not included in the statical analysis. In outcome variable, unknown outcome was considered as negative outcome. Variables with more than 70% of missing values were excluded in model development.

Time definition and time-adaptive cohort development

For real-time prediction, we considered developing time-adaptive cohorts and created prediction models for each time cohort. By dividing the total prehospital CPR time by one minute, a time-adaptive cohort was developed. Total prehospital CPR time was defined as the amount of time between the arrival of the ambulance at the scene and its arrival at the ED. Each minute of ongoing CPR from the time the ambulance arrived at the scene is defined as an adaptive cohort. The PReCAP model for each minute was developed based on each cohort. The time-adaptive cohort was developed by each minute; the training set and test set were divided separately in each cohort. Therefore, the PReCAP prediction model for every minute was also developed independently.

Model development

We used time-adaptive cohort to develop a pre-hospital real-time prediction model, PReCAP. We trained PReCAP with each data set for every minute.The PReCAP consisted of multiple models that were trained with multiple data sets throughout time. CPR duration from the scene to the hospital was not included as a machine learning model’s training feature but used as a dividing criterion of the time-adaptive cohort.

We compare the performances of PreCAP with the non-time adaptive model. The non-time adaptive model would be the same model with the 0 min point of the CPR patient cohort, which includes all patients in PAROS, because the non-time adaptive model refers to a model that predicts scene ROSC and other outcomes based on all patient cohorts with the same variables as PReCAP without using time-adaptive cohorts.

To develop PReCAP, we additionally developed three models, random forest, light gradient-boosting machine (LightGBM), and extreme gradient boosting (XGBoost) as a machine learning algorithm. The implemented software for model development included the Python programming language (version 3.8.5), the Tensorflow framework (version 2.3.1), and scikit-learn (version 0.23.2). For model development, LightGBM, which is a Python package, was used. LightGBM uses a leaf-wise method, which is different from other level-wise methods of tree-based algorithms. Therefore, LightGBM shows faster and better performance in the tuble structured dataset23. There are various algorithms for training machine learning models, but the tree-based ensemble model shows the best performance for classification of structured data. We compared the area under the receiver operating characteristic curve (AUROC) to find the best performance and chose LightGBM as a PReCAP algorithm.

Statistical analysis

In the descriptive analysis of variables of interest, variables were summarized by frequency and percentage. Comparisons between groups (ROSC vs. non-ROSC) were performed using χ2 test at a 0.05 significance level. Multivariable analysis was performed to show odds ratio of input features to ROSC on scene. All statistical analyses were performed using R (version 4.1.1; R Foundation for Statistical Computing, Vienna, Austria).

Results

Population and demographic information

Between 2009 and 2018, the PAROS database had 160,263 OHCA patients from 13 Asian nations (Table S1 of Multimedia Appendix 1). Patients under the age of 18 (2495) and patients without proper time data (114) were excluded from time-adaptive cohort configuration. Finally, 157,654 patients were enrolled in our study (Fig. 2 of Multimedia Appendix 1).

The basic characteristics of the variables considered in building the model is shown in Table 1. Out of a total population of 160,263 patients, ROSC on scene was accomplished in 11,995 individuals.

Table 1 Patient characteristics.
Full size table

Time-adaptive cohorts were created for each minute of CPR performed at the scene. Each time-adaptive cohort was randomly split into training and test sets (Fig. 3 of Multimedia Appendix 1).

Prediction outcome

Table 2 displays the major outcome of PReCAP at the time of arrival of EMS team in scene, 5 min, and 10 min following the EMS team’s arrival at the scene of the arrest. Table 2 presents the PReCAP prediction results for each outcome. In terms of prediction of ROSC on scene, the PReCAP had an AUROC score between 0.85 and 0.87 (Table S3 of Multimedia Appendix 1). The PReCAP had an AUROC score between 0.91 and 0.93 for predicting survived to discharge from ED (Table S4 of Multimedia Appendix 1), and an AUROC score between 0.80 and 0.86 for predicting the 30-day survival (Table S5 of Multimedia Appendix 1). The PReCAP predicted CPC with an AUROC score ranging from 0.84 to 0.91 (Table S6 of Multimedia Appendix 1).

Table 2 PReCAP (Pre-hospital Real-time Cardiac Arrest outcome Prediction) model AUROC by outcome.
Full size table

Figure 4 shows the predicted survival rate by time in the PReCAP model and conventional model, which predicts ROSC on scene without using time-adaptive cohort. PReCAP model shows lower predicted survival rate than conventional model as time goes on (Fig. 2, Table S7 of Multimedia Appendix 1).

Figure 2
figure 2

Study population.

Full size image
Figure 3
figure 3

Population included for each minute from 0 to 60 min for developing time-adaptive cohorts. The PReCAP model was built independently for each minute, with an independent time cohort.

Full size image
Figure 4
figure 4

Predicted scene return of spontaneous circulation ROSC, survival to discharge from the emergency department (ED), the 30-day survival rate, and the final Cerebral Performance Category (CPC). The prediction rate of non-time adaptive model is shown as a blue line while the prediction rate of PReCAP is shown as an orange line.

Full size image

Feature analysis

Figure 5 and Figs. S1–S3 of Multimedia Appendix 1 illustrate the feature importance of ROSC on scene in PReCAP prediction at 0 min, 5 min, and 10 min after EMS CPR. Age, ambulance arrival time, and initial rhythm were the most influential factors in each time cohort. However, the importance of features changed throughout time. In the early 0 min PReCAP model, for instance, the administration of epinephrine was the fourth important feature. After 5 min, in the PReCAP ROSC on scene prediction, the fourth critical aspect was the prehospital airway. People who observed the arrest after 10 min constituted the fourth important feature. As the prediction model was constructed every minute, the value of importance of certain features likewise evolved over time.

Figure 5
figure 5

Feature importance of ROSC on scene in Pre-Hospital Real Time Precision Model (PReCAP) at 0 min, 5 min and 10 min. noflow_time, Estimated Occurrence time to response time; bystander_time, Bystander CPR time; dr_epi, Prehospital Epinephrine injection; firstrhyth, First Rhythm; typeloc, Location type; pre_air, Prehospital advanced airway; arr_witn, witnessed arrest; by_aed, Bystander AED done; def_amb, Defibrillation in ambulance; frstcpr, first CPR initiator; by_cpr, Bystander CPR done; dev_ab, mechanical CPR device used by EMS/private ambulance; pre_def, prehospital defibrillation; def_frp, defibrilation performed by first responder; def_bylay, defibrillation performed by bystander-lay person.

Full size image

In addition, a multivariate analysis was conducted to determine the relationship between ROSC on scene and each feature (Table 3). In a multivariate analysis, the factors with significant differences in the univariate analysis were analyzed.

Table 3 Multivariable logistic regression analysis for each feature.
Full size table

Discussion

PReCAP is a model that can predict OHCA in real-time during prehospital care. Through the development of a time-adaptive cohort and each time-adaptive cohort-based model, we were able to predict ROSC on scene with an AUROC score greater than 0.8. The potential for ROSC at the scene is crucial for patient outcomes3,8. PReCAP study also includes further long-term predictions, including outcomes of ED, 30-day survival, and CPC. PReCAP also had high AUROC scores (more than 0.8) for most minutes. By providing prediction of possible ROSC at the scene and long-term outcomes, this model can support prehospital decisions, such as when to initiate the transfer, and where to transfer.

To the best of our knowledge, PReCAP is the real-time prediction in the prehospital phase. Few machine-learning-based models make predictions in the prehospital phase. The majority of machine-learning-based prediction models for OHCA are made using in-hospital variables and are developed for hospital settings20,21,24,25,26. In addition, only a few of them make predictions in real-time at the prehospital stage of OHCA.

Each time cohort has a different prediction model, and the feature importance was also different among time-adaptive cohorts (Figs. S3–S5 Multimedia Appendix 1). This might refer to the effect of factors that affect the changes in patient prognosis during CPR.

Each variable’s specific relevance to scene ROSC is not explainable in our study. For example, advanced airway shows a negative effect on scene ROSC in multivariable analysis. However, there might be some bias related to the performer’s skill level or situational factors that a retrospective study cannot consider. If chest compression was interrupted or defibrillation was delayed due to performing advanced airway skills, it might result in a bad outcome on the scene. For example, epinephrine injection shows high feature importance in the initial stage of EMS CPR; however, feature importance itself decreases as time goes on. In multivariable analysis, epinephrine injection shows relevance with scene ROSC in our study. However, this study is a retrospective study, and the non-strictly controlled effect of epinephrine injection in the prehospital stage could be discussed in further study.

PReCAP employs a time-adaptive cohort with distinct prediction results for every minute. Consequently, new predictions are made every minute. This may assist the EMS team in making real-time decisions at the arrest scene. In addition, as seen in a previous study, the time-adaptive cohort-based model can predict arrest outcomes similar to the real world24. Our model also shows a similar trend in the real world. As time goes on, the survival rate decreases sharply in our prediction model, which correlates with real-world results (Fig. 2, Table S6 of Multimedia Appendix 1)4,24,27. This is because it excludes patients for each minute and re-predicts the results based on time. This is one aspect wherein PReCAP reflects the real world, as it’s prediction result changes as times changes.

PReCAP is based on the multinational database PAROS, which includes prehospital information of OHCA patient from thirteen countries28. Consequently, this database contains populations with varying characteristics. By utilizing this database, we anticipate that the performance of this model will be comparable to that of the development database, even in the most diverse populations. However, country, and city details were omitted from the prediction input variable when using this database. We omitted these variables to apply this model to countries and cities not included in the PAROS database, hope that this model could be validated in another environment. On the other hand, given that local elements like the EMS system and rules can have an impact on the decision, country and city data may be helpful when applying this approach in the actual world. For each country or region to use this model, it may be necessary to create multiple versions in order to further customize it for use in the real world.

PReCAP makes use of prehospital obtainable variables; thus, it is applicable to real-world settings. Because it is difficult to obtain information regarding presumed causes of cardiac arrest and hospital treatment during the prehospital phase, it is not applied. In the real world, past medical history cannot be precisely defined during the prehospital phase, especially if the arrest occurs in the absence of witnesses. We selected input variables based on the clarity of their definitions, which can be obtained at the scene during the prehospital phase.

However, we did not perform the exact implementation of this model. There are some hurdles to doing this, like how these variables might be entered for prediction in a real-scene environment. Additional interface systems, such as dashboards or application development, are needed for real-world implementation. Further study is needed for the implementation of this model.

If it is possible to apply this model in the real world, after further development, this model might be helpful for decision-making during the prehospital phase, such as transport time and which hospital to transport, by providing the prediction results of scene outcome and long-term outcome.

Limitation

There are several limitations to this study. First, this is a retrospective observational study. Therefore, there might be a selection bias and the model might show decreased performance in different populations. However, by including a large population, we tried to create a model that could be widely applied in a diverse environment. Second, we did not conduct external validation. However, the database used for this model development includes a multinational population in Asia. Therefore, this model was first trained and tested with multinational data. It already includes a variety of populations and environments in model training, which might minimize the limitation that the performance of the prediction model could decrease in an external environment. However, further study is required to prove its performance in other environments or populations. Thirdly, as mentioned above, the real-world implications are not covered by this study. For real-world use, a system for entering data, calculating result, and an interface should also be developed. This is needed for further study. Fourthly, data was reshuffled each minute as the dataset size changed in each progressive time-adaptive cohort. By making separate cohort by each minute, authors made independently separate prediction model for each time cohort, to avoid problem of using same patient in next model.

Conclusion

PReCAP had an AUROC score between 0.8 and 0.95 for predicting ROSC on scene, survival to discharge from ED, 30-day survival, and CPC by each minute, using the PAROS database. As this model used a multinational database, it may be applicable for a variety of environments and populations. By using a time-adaptive model and predicting outcomes by each minute, this model might be beneficial for real-time decision-making in a prehospital environment after additional further study for practical implementation.