A Softer Touch
By Kathy Hardy
Radiology Today
Vol. 20 No. 5 P. 10

Photogrammetry provides a platform for accurate soft tissue imaging.

A face, with all its contours, can be considered a roadmap of someone’s life. Accurately recreating facial features, in the event of traumatic injury or disease, requires a technology sensitive enough to capture the nuances of a crinkled nose or the folds of an ear.

Photogrammetry, a 3D mapping process used to make measurements from digital photographs, is emerging as a tool for recreating the unique, fluid path of facial features. The process enables soft tissue modeling, creating more accurate models or actual reproductions of damaged facial features. The challenge is to “freeze” the motion of flesh, muscle, and other soft tissue, in order to capture images that accurately reflect the patient’s missing parts.

3D printing was first used in an experimental fashion at the former Walter Reed Army Medical Center (WRAMC) in 2003. Skin, however, is more fluid than bone. Therein is the problem in obtaining viable images for accurate reproduction of anatomy such as noses and ears.

“You want to match the soft tissue with the rigidity of bone,” says Gerald T. Grant, DMD, MS, a professor in the department of rehabilitative and reconstructive dentistry in the University of Louisville School of Dentistry in Kentucky. “You need to have the ability to stop the motion of skin and soft tissue, in order to create an accurate image.”

While this fixed system is unsuitable for use with patients who are unable to leave their beds, it does check off a few important boxes when it comes to benefits: There is no exposure to radiation—a plus when it comes to working with young patients. There is nothing invasive about the system. All images are taken simultaneously, which saves time and reduces motion. And, although photogrammetry can be an expensive process, the costs are not as high as with some other imaging modalities.

Grant gained his first experiences with 3D printing as a US Navy dental officer and service chief of the 3D Medical Applications Center (3D MAC) at what is now Walter Reed Bethesda in Maryland. He has worked with surgeons in the reconstruction and rehabilitation of craniofacial defects. His cases have included posttrauma and cancer treatment, using 3D technologies to enhance virtual surgical planning, digital imaging, and custom fabrication of devices.

Quick Pics
Photogrammetry, which Grant used at the former National Navy Medical Center (NNMC) and is now introducing at Louisville, consists of inputting images and outputting a map or 3D model based on those images. Images are acquired simultaneously via a stereophotogrammetry device, recovering the exact positions of surface points. Biomedical engineers and technicians at WRAMC and NNMC began using photogrammetry for cases involving facial reconstruction and maxillofacial prosthetics.

“This instantaneous photo imaging process helps to limit facial expression changes,” says Peter Liacouras, PhD, director of service for the 3D MAC at Walter Reed Bethesda. “Using photogrammetry gives you true facial data that can be used for plastic surgery and for the creation of facial prosthetics.”

Since 2006, Liacouras has applied 3D printing to create custom implants, surgical guides, and prosthetic attachments for Walter Reed Bethesda beneficiaries. Along with other engineers and technicians, he has helped to fulfill requests for custom prosthetic attachments, making it easier for amputees to complete everyday activities, such as brushing their teeth.

“You may be working with a patient who is missing a nose, and we’re capable of making a mold for that missing part,” he says. “We also make guides for surgery, to aid the surgeons when rebuilding a nose or ears. The model can be used as a guide.”

The 3D MAC at Walter Reed Bethesda is situated in a lab setting, rather than a radiology suite, adjacent from the dentistry department. Liacouras performs the scans himself or works with a nurse or technologist who is trained to use the equipment.

Liacouras explains that photogrammetry for facial reconstruction is done using multiple digital cameras—black and white as well as color versions—placed strategically around the patient. Black and white images are used for structural purposes, while color images are used as an overlay, what Liacouras refers to as “color map or texture.”

“Before capturing the images, the machine must be calibrated,” he says. “When the system is ready, we then align the patient within the system’s frame. We tell patients, ‘don’t smile,’ because we need to capture a normal facial expression. Once images are taken, we have the patient wait while the files are reconstructed. This takes approximately five to seven minutes, but we want to make sure we have good data before the patient leaves.”

They can also show patients what the reconstruction looks like on the screen. “Sometimes, the patients are fascinated watching a 3D representation of their face grow on the computer screen and by the final reconstruction,” he says.

All data, including color images, are then transferred to files. As Grant notes, file sizes can be rather large, making storage space an issue, especially when dealing with the newer video-based systems.

“You can take as many as 200 images of a patient at one time,” Grant says. “But you don’t need to save them all. Just delete what you don’t need.”

Liacouras explains that after the 3D data files are taken, measurements can be calculated, then the files are combined with other files, depending on what’s needed.

Effective Models
Grant is helping to build a 3D lab at Louisville, located in the School of Dentistry’s division of radiology and imaging science. The lab, where Grant is refocusing his work on research, is a collaborative effort with the maxillofacial radiology and engineering schools. He says that combining a variety of 3D printers with the expertise of engineers, dental professionals, and radiologists is proving beneficial in the university’s advancement of medical modeling.

“My work now is in trying to see what methods and equipment give us the best images for digital design and fabrication for the least amount of money,” Grant says.

The photogrammetry system for facial scanning at Louisville uses multiple cameras that capture 10 frames per second—an upgrade from the single-capture systems he used in the military—for up to 20 seconds at high resolution, generating a continuous moving 3D image. The system captures facial movement but with individual still images that can be measured and registered.

“The images taken with this system are like having images taken in a photo booth that, when you flip through them, look like they are moving,” Grant says.

Outside of the stereophotogrammetry device, Liacouras says CT images can be used, but the quality can be compromised, depending on the patient or the DICOM image properties. Handheld scanners can also be useful for certain views, particularly when capturing a limb image for prosthetic casts or in dentistry for some facial features. However, there are limitations to this modality.

“Children move frequently during the scanning process, so CT doesn’t always capture the best image of them,” Liacouras says. “With the elderly, when a patient is laying down, there are different gravitational effects than when they are in an upright position. Their skin is looser and gravity will cause it to sag or move differently. There is continuous change in the tissue. With contours of the skin, you need the scanning device to fit the exact need of the patient.”

Restraints can help with a CT scan, Liacouras adds, but also can get in the way, particularly if a clinician wishes to obtain soft tissue images of the head.

“With CT, you can obtain a good reconstruction of the ears,” he says. “However, using restraints on the head will distort the ears. CT is a good mechanism to capture the undercuts of the ear, but only if you’re not using head restraints.”

In the end, Liacouras says making photogrammetry work is all about the equipment available at each facility.

“Even newer cell phones are capable of obtaining low-resolution scan images,” he says.

Grant explains that, prior to photogrammetry, creating an image of something such as an ear required making an impression of the opposite ear, which could be a messy and time-intensive process. This is especially difficult when it comes to working with children, who typically have short attention spans and difficulty remaining still for long periods of time.

“Without photogrammetry, a child would require the use of general anesthesia or would need to be placed under sedation to make an impression, and then we would manually create a mold,” he says. “This method can result in the need for multiple modifications to obtain the best duplication of the facial feature you’re trying to capture. The goal is to have a process that requires less contact with the patient and results in a high level of accuracy.”

Process in Progress
Photogrammetry, with its digital cameras, makes it possible to use commercial systems that capture images in fractions of seconds, an ideal speed for use with young, squirming patients.

“Once we capture the digital images and use them to create a 3D model, we can design the prosthesis, create a mold, or directly print the prosthesis,” Grant says. “We can get the data we need in less than a second of capture time.”

He cites the case of a 6-year-old girl who suffered severe facial trauma as the result of an explosion. She lost her entire nose and received damage to the area around her eyes. A full-head digital image was captured using stereophotogrammetry, which was able to capture images within a few seconds. Those images obtained at Walter Reed Bethesda were used for the design and creation of the prosthesis.

Walter Reed Bethesda’s image archive did not include a model nose for a young girl so, during the process of recreating the girl’s nose, a digital image was taken of a staff member’s 6-year-old daughter and combined with the patient’s facial images to assist in the creation of the final prosthetic nose.

Building image archives are an important aspect of capturing images via photogrammetry. Both Liacouras and Grant emphasize the importance of collecting precombat images of military personnel prior to deployment. These images are stored for use in the event of facial injury requiring reconstruction. Grant sees this as an aspect of predictive change, taking medicine from a reactive approach to a proactive solution.

“When you have an injury to one side of the face, you can mirror the reverse side of the face in order to reconstruct the damaged area,” Grant says. “However, if damage is more invasive of the middle of the face, there’s no mirror image. Having a database of 3D images to call on brings added value to this technology.”

An archive of images captured via photogrammetry could also facilitate AI developments in the creation of prosthetics and facial reconstruction, Grant says.

“As we continue to collect more digital data, the more data we have, the better the AI is,” he says.

Stored images showing before-and-after cases can also be used for planning and validation purposes, he adds. This aspect could be of value in plastic surgery and for making surgical guides.

“You can save this data and refer to it over time,” Grant says. “Other users can see how procedures worked, learn, and enhance future cases.”

Using photogrammetry to rebuild a patient’s face or create a facial prosthetic is an evolving, fluid technology. Discovering best methods for capturing images of flexible, soft tissue is a dynamic progression of discovery.

“It’s a motion or movement issue,” Grant says. “Tissue is dynamic, not static. You need to capture the movement of that tissue. We are currently registering images together and comparing images when we should be capturing the dynamic range of the tissue and comparing that to a normal or planned reconstruction.”

— Kathy Hardy is a freelance writer based in Phoenixville, Pennsylvania. She is a frequent contributor to Radiology Today.