Abstract

Preoperative evaluation of patient-specific anatomical data is a preliminary step in any surgical
procedure. Medical imaging and visualization of patient-specific anatomical data are critical com�ponents of this step and can be presented as 2D images or 3D objects. While 2D images have been
used in traditional systems, 3D visualization of this data provides more comprehensive information
about the organ under treatment, so it is important to note that exploration and comprehension of the
data may be improved when intuitive interaction and visualization methods are used. Meanwhile,
haptic tools and augmented reality in different medical systems have shown promising improve�ments to the overall system performance in terms of enhancing the sense of touch, providing 3D
perspectives, and, allowing for more intuitive interaction techniques.
In this study, we explore the use of 3D visualization of anatomy on screen and in AR and
different interaction techniques in the context of pre-operative planning. Specifically, we compare
planning using the traditional 2D monitor and mouse, a monitor with the 3D Systems Touch X
force feedback haptic device, and an augmented reality head-mounted display (using the HoloLens
2) which uses gesture-based interaction.We performed two user studies, one with novices and one
with experts. The user studies involved interacting with 3D anatomical data in the context of three
surgical planning scenarios: (1) heart valve repair, (2) hip tumor resection, and (3) pedicle screw
placement. Our findings suggest that our users preferred the AR system for the pedicle screw
placement and for the other two cases, and reported that AR provides the best depth perception
compared to the other two platforms. Our novices preferred the haptic system for hip tumor planning
while experts preferred the mouse and keyboard (2D) system.