How are NLP and Computer Vision being used in healthcare?

Vinith Raj
4 min readMay 28, 2023

Healthcare entities are collecting more data than ever before to better understand and serve patients. With an increase in the amount of data available, healthcare providers can now leverage AI programs to assist physicians with their tasks. Natural language processing, computer vision, and other applications of AI can have a significant impact on how doctors can help patients, with many use cases already being adopted in the healthcare industry.

What is NLP?

Natural language processing (NLP) is a subfield of artificial intelligence that allows machines to process and understand human language. This means a program utilizing NLP could understand and process information from medical documents and even audio recordings. In order for NLP to work effectively, data needs to be prepared and cleaned for the algorithm to analyze it. Processes to accomplish this could include tokenization, which breaks text down into smaller units, word removal, where unnecessary words are removed, and part-of-speech tagging. After cleaning the data, an algorithm can process it. Data management in healthcare is complex due to the high number of data sources, such as provider notes, patient reports, and audio recordings. NLP can take unstructured data from these sources and convert them to structured data for further analysis.

Where can NLP be used in?

NLP has many use cases for EHRs. Electronic health records (EHRs) are digital collections of patient healthcare information, such as appointment notes and surgery reports. A lot of information in EHRs is unstructured, but NLP can be used to make more sense of patient information and optimize healthcare delivery and operations. NLP can quickly and efficiently sift through large quantities of notes, lab results, and other important documents and deliver relevant and important information to providers. This will save doctors a significant amount of time, allowing them to spend more time with patients, and improve patient care.

Abridge is a startup that enables physicians to automate clinical notes and medical conversations. The platform uses AI to transcribe and summarize conversations so they may be uploaded straight into the EHR, without having to be manually typed. The application integrates with top EHRs (e.g., Epic), enabling healthcare providers to transfer summarized information, data, and insights accurately into existing medical records. This also allows patients to have easy access to conversations with their providers, allowing improved communication and patient experience. Abridge recently announced a partnership with the University of Kansas Health System, rolling out the platform across over 140 locations. According to the American Medical Association, healthcare systems are under pressure from physician burnout, with almost 63% of physicians reporting burnout, and platforms such as Abridge can decrease pressure on doctors by eliminating administrative tasks and enabling better service for patients.

How can AI be used to improve medical imaging?

AI is being increasingly used by physicians to improve medical imaging. With more and more data being collected, machine learning, specifically computer vision, can enhance the efficiency of diagnoses and help physicians interpret challenging images.

A company that is doing a lot of work in this space is Sirona Medical. They acquired Nines AI, another healthtech startup that specializes in AI algorithms. Sirona is developing an “operating system” for radiology. Their main product, RadOS, incorporates databases of images with image viewing and reporting software to allow radiologists to seamlessly collect, analyze, and report data from their work and more efficiently help their patients. With the inclusion of Nines’ AI algorithms, RadOS can now also measure and highlight patterns in images and assist doctors in making critical decisions based on the images they collect of patients.

According to GE Healthcare and the American College of Radiology, in 2021, only 30% of radiologists were using AI tools in their practices, but 20% of those that didn’t have any AI tools were planning on purchasing some within the next 5 years. Tools like RadOS can have a direct impact on a patient’s care, and there is a growing market for AI healthcare tools as doctors become more comfortable using AI solutions in their day-to-day tasks. Companies like Microsoft and Google are already researching AI algorithms to help with medical image analysis, and there are an increasing number of startups that are trying to do the same thing. As more companies develop AI solutions for radiologists, Sirona will have to adapt and find ways to continue providing great service to their customers, but also figure out how to stay competitive.

Although there will be challenges associated with acquiring usable data for programs, AI can have an incredible impact on healthcare. NLP, computer vision, and even generative AI can allow physicians to provide better care for their patients, and spend less time interpreting data, allowing them to perform their tasks more efficiently.

--

--

Vinith Raj

Sr. Associate @ EY-Parthenon | UIUC Engineering ’21 | Passionate about Healthcare, Technology, and Startups