Evaluation of a novel terminology to categorize clinical document section headers and a related clinical note section tagger

Abstract

The aims of this project are to 1) build and evaluate a terminology that provides categorization labels, or tags, for common segments within clinical documents, and 2) to evaluate a tool to parse and label natural-language clinical documents using the terminology. Clinical documents generally contain many sections and subsections, such as “history of present illness,” “physical examination,” and “cardiovascular exam.” The author developed a section header terminology that models common section names, subsection names, and their relationships. This terminology was built using existing standardized terminologies, textbooks, and review of over 9,000 clinical notes. The section tagging tool, named SecTag, identifies terminology matches from clinical documents using a combination of linguistic, natural language processing, and machine learning techniques. The evaluation study focused on recognizing sections in 319 randomly-chosen “history and physical examination” notes that were generated during hospitalizations and outpatient visits. The overall recall and precision were 99% and 96%, respectively, over 16,036 possible sections. Recall and precision for sections not labeled in the document were 97% and 87%, respectively. The system correctly tagged 93% of the section start and end boundaries. SecTag failed to label 160 sections (1%); only 11 were headings that were absent in the terminology and which should be added to it. SecTag and its terminology are important first steps for understanding clinical notes. Future studies are needed to extend the terminology to other clinical note types and to link SecTag to a more in-depth natural language processing system.