Interpreting medical ultrasound images is a difficult task, requiring a technician to look at 2D images and mentally organize what the tissue will look like into a 3D representation.
To make that job easier, MIT researchers developed a new approach to ultrasound imaging that allows the user to visualize a 3D augmented-reality image of the object being scanned. Using a virtual-reality headset, they can see an accurate 3D digital representation of what the object actually looks like, making it easier to identify and analyze.
This technique will help speed up the training process for ultrasound technicians and other healthcare providers who use ultrasound. It can also be used in hospitals for tasks such as using ultrasounds to place a needle in the right place for a biopsy.
“For practice, it makes ultrasound more intuitive and understandable. On the clinical side, it’s less time-consuming, more accurate, and can give health care providers more peace of mind. They don’t have to be surprised if they miss something,” says Canon Docteviren, senior lecturer in media arts and sciences at MIT.
MIT graduate students Jason Hu and Srihari Viswanath are lead authors of the paper. Nature Communications Engineering. Other authors of the paper include Bowen Wu ’24 and two MIT summer research program students, Cinay Dilibal, a senior at Dartmouth College, and Tanisha Shende, a senior at Oberlin College.
3D representations
Ultrasound imaging works by bouncing high-frequency sound waves off tissues in the body, which are then reflected back to the ultrasound transducer. The transducer converts these sound waves into electrical signals, which are used to create a 2D image of the tissue. Ultrasound technicians are trained to convert these images into a 3D mental representation of the tissue.
“It’s a difficult skill to master, and there are long learning curves,” Hu says. “The hard part is this mental tomography disruption where you’re trained to reconstruct 2D slices in your 3D mental space. It’s a cognitive load that leads to inaccuracies in scanning.”
To reduce that cognitive load, the MIT team thought it would be helpful to combine two technologies: 3D ultrasound imaging and augmented reality (AR).
Three-dimensional ultrasound imaging is occasionally used in fields such as fetal imaging and echocardiography, which is used to image the heart, but most 3D ultrasound imaging systems are expensive and not widely available. For this study, the MIT team used a recently developed real-time 3D system for breast cancer detection.
Their new system consists of an ultrasound probe, slightly smaller than a deck of cards, that transmits information using a encrypted data acquisition system (cDAQ). The probe has an ultrasound array arranged in the shape of a hollow square, a configuration that allows the array to take 3D images of the underlying tissue.
Because the system has fewer ultrasound components than a conventional 3D ultrasound system, it requires less power and costs less to build.
The data collected by the ultrasound probe can then be compressed and streamed into a 3D computer graphics engine called Unreal Engine, which converts the voxel data from the ultrasound image directly into a 3D representation of the object, without loss of information. Wearing an AR/VR headset, the user can view this 3D rendering representing the internal structure superimposed on the object’s actual location, similar to an X-ray view. By tilting their head or approaching from a different direction, the user can see different views of the object, making it easier to identify.
Easy to use
The researchers tested their new technology, which they call AR-VIU (for augmented real-time volumetric imaging in ultrasound), on a group of 18 participants. Nine of the subjects were experts in ultrasound technology (including sonographers and physicians), and nine had never used ultrasound before.
Each user performed identification tasks using four different ultrasound technologies. In one condition, they viewed 2D images on a conventional screen, which is how most ultrasounds are done now. They viewed 3D images on a regular screen, and two augmented reality conditions: one in 2D and one in 3D (AR-VIU).
In one round of the experiment, users were asked to identify an object — such as a spring, ball, or screw — embedded in gelatin inside an opaque container scanned by ultrasound. In the second set, they were asked to use a pen to mark the location of a “tissue phantom” – a gel-like substance designed to mimic human tissue. It simulates the task of finding the correct location of the needle during a biopsy.
The researchers found that the AR-VIU system significantly improved all users’ ability to detect and identify objects. The effect was almost as strong for novices as for experts when using the AR-VIU. When using the traditional 2D imaging method, experts performed better than novices.
“Ultrasound makes it easier for novices to understand by overlaying images with anatomy and providing a 3D visual environment,” Viswanath says.
In interviews after the tests, most novices reported preferring the AR-VIU approach, with many saying it made tasks easier.
“The 3D system imposes less brain drain, it’s more intuitive, and it’s easier to understand what’s going on in the target area,” Docteviren says.
Many experts said they prefer traditional 2D imaging because that’s what they’re used to and trained to use. However, the experts also said the benefits of the AR-VIU system could be seen in certain situations, such as inserting a needle for a biopsy or visualizing the motion of the heart wall during echocardiography.
The researchers are now conducting additional tests to further improve the resolution of the imaging and demonstrate the accuracy of the AR-VIU technology.
This research was funded by the MIT Media Lab Consortium, the National Science Foundation, an MIT Heals Graduate Fellowship, and an MIT-TATA Graduate Fellowship.
#Augmented #reality #system #medical #ultrasounds #easier #interpret