ICD (The International Statistical Classification of Diseases and Related Health Problems) is a medical classification list by the World Health Organization (WHO). It helps to standardize diagnoses terms and structure health record data. ICD Coding assistant is a tool that recommends ICD codes based on doctor’s records in real-time, to create better health records and benefits process of payment application to NHI (National Health Insurance). Over 16+ hospitals in Taiwan 775+ hospitals in America (cooperate with Olive) using ICD-10 coding assistant.
While cooperating with hospitals for AI solutions, one of our consultants who works in a medical association mentioned her observation with us, ICD coding is difficult for doctors especially between versions transition. This is a valuable opportunity and approved by a crucial member in a medical association, and actual needs that benefit doctors and DRG(diagnosis-related group) payment and the examination of NHI (National Health Insurance) for the health record accuracy. To understand how the ICD codes work in health care, I conducted interviews with doctors realized that ICD would be used while the whole process of a patient’s treatment, but not every doctor needs to do this, some hospitals would be ICD coders to assign ICD code to optimize the payment application and the coding quality of each visit. If doctor can do coding in advance it would benefit the cost management a lot.
There are outpatient, inpatient scenarios, and inpatient can be separated into 2 stages: admission, progress, discharge. The outpatient visit would generate a set of ICD codes during the session to represent the diagnosis. The doctor’s diagnosis would be based on subjective, objective, and assessment. Once the diagnosis is confirmed, the ICD code can be filled in and come out with the treatment plan.
The outpatient is a fast-paced process, the doctor can only spend 2 minutes per patient on average with plenty of patients during their daily session, doctor only remembers the codes used often, they would need to Google or search for unfamiliar diagnoses, most of the time just assign the code ends with .9, means unspecified, which cause the inaccurate issue of the health record. Besides ICD codes, there are some problems doctors occurred while writing outpatient records. For the product to succeed, it needs to solve these issues while doctor’s daily practice, the goal of outpatient tool would be to write a quality record with small effort.
The inpatient process would have 3 stages: admission, progress, discharge. Each stage would need ICD codes, detail might vary in different hospitals. Usually, there’s no ICD code for diagnosis in the progress notes, but some hospitals would need to assign ICD in progress notes for cost optimization. Coder is the role to final review and reassigns codes for cost application. Whether it’s outpatient or inpatient the most important thing is to assign accurate ICD codes for diagnosis and procedure, to ensure the health record quality, and optimize the GRD for profit to keep the hospital running.
To solve the issue and provide more help for writing a record with high quality, I hosted a workshop by sharing the interview result with the team. It is important especially for complex applications, by cooperating with domain experts we can always come out with great ideas, and align everyone with user and product goals. So the main features are, ICD recommendations, spellcheck, and autocomplete. The product roadmap was to land on outpatient first once the product value and flow be validated it would be expanded to inpatient.
After the first few iterations, the usage was still low and received some complaints, so I conducted the user interviews and workshop to came out the iteration goals:
The ICD recommendation AI model is trained by the data from the past, which means the model accuracy is based on hospital data. To refine the model, added some standard data to recommend and raised the weights for the selected codes in the recommendation list and search to increase the accuracy.
Improve the search feature with abbreviations inputs and adjust string matching, so the doctor can find the code by the terms they usually use by type in the diagnosis even with abbreviations and ICD-9 code.
Analyzed the usage data to decide what features are less helpful and needed to be removed to avoid gaining user cognitions and simplified the product flow by providing the essential features only.
After the iterations, converted some users into power users successfully, that these users even recommend the tool to the other doctor proactively by hosted sharing to many sections of the hospital, raised the usage rapidly because of the user ripple effect, thanks to these power users.
After Landing to inpatient, the usage did not meet the expectation, I conducted the interview found out that coders are not using it because they behave differently compared to doctors when giving the ICD code. Coders focus on code combinations and specify the code by indexing codes. So the product provided the coder version to fit their workflow, to customize the record content for a better reading experience, included ICD combination to adjust the recommendation.
The challenge of it is how to raise the adoption, increase motivation to use the new tool at the rapid pace. I think the user research and connection between our users definitely helped a lot, by knowing how TA think and behave, we can improve the feature to align with the user’s intentions and make the feature easy to use without a steep learning curve. During the interview section, we met a doctor, who was defined as an angry user, he can’t complete his work by using our tool. He provided lots of insight in that meeting, and I did express the intention of this interview and how their feedback will help the product let him know what issues this product is facing to let him understand how valuable his feedback is. Although he was not satisfied with the tool he’s very interested in how AI assists in medical practice, so after we first contact, we ask him if he’s willing to help the product as a beta user. And we made these improvements with his help and turned him into a power user. He did several sharing in his hospital after we launched the new iteration and this hospital soon became the highest usage of all hospitals.
I would like to improve user research and communications. When I did the user research, I realized that there are different TA while assigning the ICD code: doctor, coder, sometimes even nurse. And my assumption is that doctor would give the best code with his diagnosis I did validate with doctors but I didn’t validate with coder at that time. Since the product roadmap is to be launched in outpatient first and got successful after iterations. The product owner thinks we can duplicate the mode to inpatient, and I didn’t aware that the most important TA would be coder, since that would be the doctor’s responsibility and the best role to assign the best code during the whole process. If I did more communication to align and communicate the knowledge of user then the issue could’ve been fount before inpatient launched. The goal for coders when assigning codes is to optimize the profit, so the record might not be good for doctor doing research. But I think this is hospital and NHI system issue, and it’s hard to change in a short amount of time, so the coders assigning the final ICD codes is most beneficial for hospital operations.
Contact me via this form
