Data Driven Order Set Design in Pediatric Appendicitis

Abstract

Electronic order sets are a form of clinical decision support with the potential to save time; centralize and streamline common orders; facilitate adherence clinical guidelines; be highly specific to clinical situations; and influence health outcomes. Little literature exists on the optimal approach to order set creation. We hypothesized that an optimized, data-driven approach to order set creation can facilitate the effectiveness of and provider satisfaction with electronic order sets; and that order sets can improve compliance with clinical standards of care and patient outcomes in pediatric appendicitis patients. To test this, we performed a combined retrospective cohort, implementation, and prospective cohort study at a high-volume quaternary children’s hospital. Patients who underwent laparoscopic appendectomy at our children’s hospital between July 1, 2018, and May 15, 2024, were included in this analysis. Data sources utilized for this study include patient and operational data from the Epic electronic health record; institutional clinical practice guidelines; the OpenAI large language model, ChatGPT; and expert opinion in the form of pediatric surgery providers. 2,077 patients underwent laparoscopic appendectomy prior to data analysis and received a total of 210,533 procedural electronic orders (520 unique types) and 62,503 medication orders (353 unique types). Analysis of data sources yielded 221 electronic order recommendations for the planned order sets. Survey of pediatric surgery providers found order relevance to be highest for recommendations from historical order frequency analysis (100.0%), and mean utility rating to be highest for clinical practice guideline recommendations (mean rating 4.50 out of 5, SD 0.64). ChatGPT had both the lowest relevance (53.1%) and the lowest utility rating (mean 4.11, SD 0.76). Recommendations were used to build and implement two electronic order sets for pediatric appendicitis (pre- and post-operative). Post-implementation patient data analysis showed a shorter time from surgery consult to antibiotics (3.00 hours, SD 3.39, to 2.16 hours, SD 1.83; p = 0.029) and a decrease in patients receiving oral antibiotics prior to discharge (36.3% to 24.3%; p = 0.046) after order set implementation. Here we propose a development process for data-driven order set design based on study findings.