Mr. Bhavraaj Singh

Master of Science in Computer Science

Title: Leveraging Large Language Models For Clinical Documentation Improvement

In today’s fast paced healthcare environment, accurate and efficient documentation is crucial for delivering quality care. However, medical professionals often struggle with time consuming paperwork that can lead to errors or incomplete documentation. This problem sparked Bhavraaj’s motivation to investigate AI-driven solutions that could alleviate this burden. His goal was to develop tools that could support clinicians by automating parts of the documentation process, improving both accuracy and efficiency. Inspired by the rapid advancements in natural language processing (NLP), he set out to create AI medical chatbots for textual and infographic data along with an AI-based system that could intelligently categorize and analyse clinical information.

His thesis investigates how LLMs can be utilized to enhance the clinical documentation process. He developed an AI-powered Categorization and Analysis tool designed to help healthcare professionals process clinical information more efficiently. The tool automatically classifies diseases into broader categories and visualizes the data, making it easier for medical professionals to interpret and utilize the information in decision making processes. He also developed two distinct AI powered chatbots – one designed to process textual data and another focused on clinical infographs. The research integrated retrieval augmented generation (RAG) framework, with the system being tested on various clinical datasets to evaluate its performance in interpreting and summarizing complex medical records.

The results of the research were highly promising. The AI Categorization and Analysis tool demonstrated excellent precision in categorizing diseases and generating detailed visual representations. The textual AI chatbot achieved an F1 score of 0.89, significantly outperforming traditional methods. The chatbots, specifically designed for medical queries, correctly answered 90% of the test questions, illustrating their potential for real-world healthcare applications. These tools offer a novel approach to clinical documentation, enabling better information management and quicker decision making for healthcare providers.

The impact of this research has been profound, influencing both industry and academia. The AI

Categorization and Analysis tool, alongside the medical chatbots, holds the potential to revolutionize how clinical data is processed and managed. By automating tedious and time-consuming documentation tasks, these tools allow healthcare professionals to dedicate more time to patient care, improving operational efficiency and reducing human errors in medical records. In the academic realm, this research has paved the way for further exploration into AI’s role in clinical documentation. This work has demonstrated that large language models, when integrated with RAG framework, can be effective in categorizing, visualising and interpreting complex medical data, opening up new research avenues in medical AI. Moreover, it showcases the importance of interdisciplinary collaboration, combining AI, healthcare and data analytics to address pressing challenges in modern medicine. The research bridges the gap between academia and industry, driving innovation in clinical documentation and highlighting the potential of AI to transform healthcare operations. Bhavraaj’s contributions serve as a foundation for future advancements in both the technology and medical sectors.

Alternate Abstract for the Webinar

Clinical documentation is integral to healthcare, providing essential information for patient care, billing, and research purposes. However, the documentation process often encounters challenges such as inaccuracies, inconsistencies, and inefficiencies along with the process being time-consuming and error-prone. Leveraging advancements in Natural Language Processing (NLP) presents a promising solution to enhance clinical documentation practices. 

This research presents novel approaches to improving clinical documentation through the development of a Medical AI Chatbot powered by natural language processing (NLP) and large language models (LLMs), utilizing an advanced AI model, specifically Llama2, for the development of an AI Clinical Documentation Categorization and Analysis Tool and the development of an AI Clinical Infographic Chatbot to address the challenges and improve clinical documentation. 

The methodology involves leveraging NLP techniques, including text categorization, data cleaning, and visualization, to streamline the clinical documentation process. A case study utilizing a custom dataset of patient records is presented to demonstrate the effectiveness of the approach. 

The chatbots utilize an advanced AI model, Zephyr 7b Beta, to interact with clinical data. The medical chatbot interacts with the dataset of patient records, retrieving relevant information, whereas the infographic chatbot interacts with medical infographics to answer queries related to them. The research details the methodologies used to develop the chatbots and the analysis tool, including data preprocessing, embedding generation, model integration, and query processing. 

The results highlight significant improvements in categorization accuracy, data cleanliness and visualization of trends, and the ability to provide accurate and context-specific information to healthcare professionals through the Retrieval Augmented Generation (RAG) technique, underscoring the potential of NLP in driving clinical documentation improvement and enhancing healthcare delivery.

Mr. Bhavraaj Singh, an Indian Computer Science Engineer who graduated from Vellore Institute of Technology with a passion for innovation and technology, has dedicated his academic journey to leveraging artificial intelligence (AI) for real-world impact. As part of his Master’s in Computer Science thesis at California State University, he explored cutting-edge approaches to improving clinical documentation through large language models (LLMs). Having interned as a Senior Engineer, Bhavraaj plans to join Southern California Gas Company as a Data Analytics Engineer. His research has gained academic recognition, with parts of it published in esteemed journals and presented at prestigious conferences, underscoring its significance in healthcare AI.