Helping Hand or Too Many Cooks?
By Aine Cryts
Radiology Today
Vol. 23 No. 6 P. 24

Determining AI’s Role in Detecting Lung Cancer

The first study that drew a connection between lung cancer and cigarette smoking was published in 1939. Conducted by Franz Hermann Müller at Cologne Hospital in Germany, the study showed that smokers were far more likely than nonsmokers to have lung cancer. Those findings were confirmed in 1943 in a case-control epidemiological study published by Eberhard Shairer and Eric Schöniger, researchers at the Friedrich Schiller University Jena, a public research university in Thuringia, Germany.

Subsequently, in 1954, the American Cancer Society noted “an association between smoking, particularly cigarette smoking, and lung cancer.” The organization joined other entities around the world in taking the charge on advising people to stop smoking as a way to prevent lung cancer.

At almost 25% of such deaths, lung cancer is one of the leading causes of cancer death in the United States, according to the American Cancer Society. In fact, more US residents die as a result of lung cancer than cancers of the breast, colon, and prostate combined. Still, because many people have quit smoking, the number of new lung cancer cases continues its downward trend. Advances in detection and treatment also impact the fall in lung cancer deaths, per the American Cancer Society.

Nodule Evaluation
Pulmonary nodules are abnormal growths that can be found in the lungs, according to the Cleveland Clinic. These growths are rarely cancerous and often don’t require treatment, but that’s not always the case.

According to Anil Vachani, MD, MSCE, a pulmonologist and associate professor of medicine at the Hospital of the University of Pennsylvania, it’s estimated that 1.6 million people in the United States are found to have lung nodules each year. “The reality is that many nodules are pretty easy to manage,” he says. “The very small ones we know are almost always benign, but the ones we get that are larger—particularly over 8 mm in size—they’re a very difficult challenge for pulmonologists.”

Also challenging is determining which nodules are more likely to be benign or malignant, in addition to whether invasive testing is necessary.

“[The experience] creates a lot of patient anxiety,” says Vachani, who also directs the Lung Nodule Program at the Perelman School of Medicine at the University of Pennsylvania. “Any tools that come forward that can potentially help us as a profession and as a medical community manage patients with this problem are exciting to me.”

Vachani is coauthor of a study published in Radiology in September 2022 that evaluated the impact of an AI-based computer-aided diagnosis tool on clinician-indeterminate pulmonary nodules diagnostic performance and agreement, for malignancy risk categories and management recommendations.

The study described indeterminate pulmonary nodules as “rounded opacities 3 cm or less in diameter surrounded by aerated pulmonary parenchyma without clearly benign features.” According to the study, these nodules are typically found on chest CT scans incidentally during routine clinical care. Furthermore, an increasing proportion of such nodules are found during lung cancer screening.

Used for people with the greatest risk of lung cancer, lung cancer screening is often reserved for older adults who smoke or are former smokers. Of those included in screenings, most are heavy smokers, according to the Mayo Clinic. Other candidates for screening are people who were previously treated for lung cancer, in addition to individuals with COPD, a family history of lung cancer, and those exposed to asbestos.

The National Lung Screening Trial, which launched in 2002 and included 53,454 patients, compared low-dose helical CT and standard chest X-ray, according to the National Institutes of Health’s National Cancer Institute. Nearly one-quarter (24.2%) of the low-dose helical CT screens were positive, and 6.9% of the chest X-rays were positive over the three rounds of screening exams, on average. The majority of patients with positive screens for lung cancer went on for additional testing, per the National Cancer Institute.

The primary results of the study, which concluded that the use of low-dose CT reduces mortality from lung cancer, were published in the New England Journal of Medicine in August 2011.

Supporting Recommendations
The study in Radiology for which Vachani served as coauthor found that computer-aided diagnosis tools improved the estimation of indeterminate pulmonary nodule malignancy risk on chest CT scans. It also found improved agreement by the 12 readers—six radiologists and six pulmonologists—regarding risk stratification and patient management. The categories involved for readers were “no action,” “CT surveillance,” and “diagnostic procedure.”

AI, or deep learning, can help clinicians look beyond a nodule’s characteristics—even beyond its size—to answer questions such as “Is it solid? Is it irregularly shaped? Is it smooth?” Vachani says, adding that these tools can help “scan CT images and mammograms, and all sorts of medical imaging to see if there are additional features that can help predict the risk of cancer.

“I see patients quite frequently who had a CT scan for one reason or another,” Vachani continues. “Often times, it’s because they have nonspecific symptoms, like a cough or chest pain, and they get a CT scan in the [emergency department] or by a physician … and the first thing that comes to mind when [the patient hears] about a nodule in the chest or a nodule anywhere is this could be a malignancy.

“It’s a surprising finding. [Patients] are anxious. They’re worried they have cancer, and they have to be evaluated by me. They want to know how likely we think cancer is in this case and what that means for what we should do next?”

The options range from “watchful waiting,” a common strategy in many areas of medicine where there are small and potentially concerning findings, to a biopsy or something more invasive right away.

Anxiety and concern are common patient reactions for a variety of reasons. “Even if we choose to wait with the patient who’s concerned about whether it could be cancer. … If we choose to go forward, of course there are concerns around if [the patient has] a biopsy. … It’s an invasive procedure. It could be painful. There could be complications [and] financial implications,” Vachani says.

While financial implications aren’t at the forefront of his mind with his patients, Vachani says it’s always underlying any potential cancer diagnosis.

Mark Hammer, MD, an assistant professor of radiology in the division of thoracic imaging at Boston’s Brigham and Women’s Hospital and Harvard Medical School, took note of the Radiology study’s findings pertaining to the AI tool’s effect on radiologists’ reports and recommendations.

“That’s something that really needs to be assessed. Because if at the end of the day, the tool gets you a little bit better, but it doesn’t change what you would do, it’s ultimately just a waste of time,” he says.

Hammer also points out that AI tools are likely to improve the performance of general practice radiologists rather than those who are chest trained. “If you’re out in the community, they’re almost always going to be general radiologists,” he says. “Those folks just aren’t as attuned—because they do so many other things—they’re not as attuned to the nuances about which nodule is suspicious vs not.”

AI tools can also potentially add value to the process of examining nodules that aren’t very small as well as those that aren’t very large, Hammer adds. “If it’s a very small nodule, we’re usually pretty sure that it’s nothing. And if it’s a very large mass with spiculations, we’re pretty sure it’s cancer,” he says. “Obviously, there’s a lot in between. And the real challenge is what we do with the in-between cases. There’s a decent number of them. And what we want to avoid doing is unnecessary imaging, unnecessary biopsies, unnecessary surgeries on that patient population.”

Encouraging Performance
In November 2021, Radiology: Artificial Intelligence published a study about AI tools that were used in a $1 million public competition to identify lung cancer on low-dose CT scans. The question explored in the study: Could the AI tools identify lung cancer with a performance that’s similar to radiologists?

The answer to that question: The AI tools reached a performance that was close to that of radiologists.

The study observed that the top-performing algorithms at the public competition, dubbed The Kaggle Data Science Bowl, achieved a performance level “not significantly worse than that of 11 radiologists for estimating lung cancer risk on low-dose CT scans.”

The event was organized by San Francisco–based Kaggle, a subsidiary of Google that coordinates data science competitions. This competition, which included data from the National Lung Cancer Screening Trial, attracted attention from machine learning and AI scientists, according to Colin Jacobs, PhD, an assistant professor in the department of medical imaging of the Radboud University Medical Center in Nijmegen, the Netherlands, and lead author of the study in Radiology: Artificial Intelligence. In total, there were 6,000 people involved in developing algorithms to detect lung cancer from CT scans, he says.

Included in the observer study were 300 chest CT scans, of which 100 were cancer positive and 200 were cancer negative. The coauthors wrote, “Future development of the deep learning models should focus on providing more information to the user (eg, the location of the suspicious pulmonary nodules that have been found by the model).

“This should be feasible to accomplish because winning solutions used an approach in which they first detected lung nodule candidate locations and subsequently used the detected locations to produce a malignancy risk score at the scan level. Subsequently, studies are needed that focus on evaluating how the use of these algorithms can be integrated with the work of radiologists to positively change the follow-up recommendations in a screening program.”

What’s Next?
While he’s curious about the ability of AI platforms to help support radiologists and other physicians, Hammer says physicians should ask tough questions when an AI software salesperson comes to their practice or department, including the following:

• What’s the external validity of the model? A related question is: Has the AI tool been tested on scans done from different hospitals in different settings in different parts of the country?

• How would this be implemented at my practice or in my department? In particular, radiologists should be asking these types of questions: How do I activate this tool? How do I use it? Do I have to click on a lot of buttons, or does it take me a lot of time to do it? Is it just going to add extra time? Is it very easily integrated into the workflow?

But radiologists also need to do their own self-reflection when considering and then using AI tools, according to Hammer. For example, “How should the number generated by the AI tool be reported?”

Finally, it’s important to ask referring colleagues for their perspectives, Hammer says. For example, he’d likely seek out feedback from his pulmonologist and surgeon colleagues.

“I can tell you that our surgeons are very particular, and they have very strong opinions of their own,” Hammer says. “Would they appreciate having a tool tell them whether something is cancer or not? They may say, ‘We prefer to make our own assessment and decide what’s best for the patient; we don’t want any more ‘cooks in the kitchen.’”

Since they’re considered to be medical devices, the next step when deploying the AI tools featured in his study is FDA certification, according to Jacobs. “These are all proprietary algorithms that were developed in a competition—and not medical devices yet. But it shows the potential, and that’s important,” he says.

However, Jacobs expects AI tools to be in common use for detecting lung cancer within the next five years.

— Aine Cryts is a health care writer based in the Boston area.