New research addresses problems that have historically made colonoscopy a challenging application for computer vision, including 3D reconstruction and depth estimation.
Computer vision systems show promise for improving the accuracy of colonoscopy screenings, a procedure that uses a camera to see the inside of the large intestine to detect cancer and other diseases.
Training and assessing computer vision algorithms typically require a properly labeled dataset that contains “ground truth”—information that is verified to be true in realistic conditions. To date, ground-truth datasets have been difficult to acquire for colonoscopy images.
Researchers have now created a dataset of simulated videos, called C3VD, that is more representative of colonoscopy imaging and can help researchers evaluate how their computer vision models will perform in real-world scenarios.
The research appears in Medical Image Analysis.
According to the researchers, this is the first colonoscopy reconstruction dataset that includes realistic colonoscopy videos labeled with registered ground truth. They are hopeful that their efforts will contribute to the development of advanced computer vision algorithms that can aid endoscopists in real time during colonoscopies.
“Artificial intelligence can dramatically improve colonoscopy procedures—for instance by improving lesion detection or ensuring the full colon is screened. But AI models rely on high-quality data. Our dataset includes commercial colonoscope videos of realistic colon phantoms, with pixel-level annotations of the distance of every pixel from the camera. Researchers around the world are now using this dataset to train and test the ability of AI to estimate colon geometry directly from a video,” says Nicholas Durr, an associate professor of biomedical engineering at Johns Hopkins University and senior author of the paper.
To develop the dataset, researchers from the School of Medicine’s art as applied to medicine department first created a digital sculpture of the colon from reference anatomical images and then physically cast it in silicone. Using the 3D colon model, the researchers recorded video sequences with a clinical HD colonoscope to include realistic lighting, sensor gain, gamma, and noise—all factors that play an important role in the performance of computer vision algorithms.
After recording the simulated colonoscopy videos, the researchers needed to virtually align them to the model—a process known as registration—so that they could extract ground-truth information for each frame. They developed a registration algorithm using a combination of artificial intelligence, ray tracing, and optimization. By registering the videos with the models, they created a video dataset with more than 10,000 frames, in which each frame has ground truth normals, optical flow, occlusion, coverage, and poses.
“These are essential components for building reliable computer vision models, making our dataset useful for various potential applications that may improve colonoscopy quality,” says Taylor Bobrow, a graduate student in biomedical engineering and lead author of the paper.
While the study’s primary contribution was the validation dataset, Bobrow says the team also made an interesting finding related to the registration algorithm. Most work on 2D-to-3D image registration has focused on applications in radiological systems via X-rays, whereas the researchers worked with optical technologies. They discovered that transforming optical images to the depth domain for registration resulted in significant improvements in registration accuracy, as compared to directly registering the optical images.
The team hopes the open-source dataset will enable the wider research community to improve the performance of computer vision tools for colonoscopies.
Bobrow believes that C3VD has the potential to minimize the number of lesions that routine colonoscopy procedures might miss, resulting in later diagnoses of colorectal cancer.
“A big focus for researchers is on using AI to develop real-time feedback systems that achieve complete visualization of the colon and alert endoscopists if they miss inspecting a tissue area during a procedure. Our dataset could be used as a tool for validating algorithms that flag missed regions,” says Bobrow.
Source: Johns Hopkins University
