Like the artist’s composite sketch that slowly reveals the face of a criminal suspect, the image on Xiaolei Huang's
computer screen is gradually zeroing in on America’s No. 1 killer—heart disease.
Lines in the image crisscross to form a net-like pattern called a polygon mesh. They intersect at three-dimensional nodes called vertices.
Huang has used computer graphics to render a geometric model of three poses of an actual heart that was scanned, over time as it was contracting, by magnetic resonance imaging (MRI).
Huang's computer-generated 3-D model of a whole heart becomes a 4-D image over time.
The heart may have its secrets, but the stress and strain that it undergoes as it pumps blood is not one of them, at least not to Huang, who is an assistant professor of computer science and engineering.
Huang writes software programs that enhance the detecting powers of MRI and other medical imaging techniques. Her programs can take 4-D medical images—with time as the fourth dimension—and extract from them the precise geometric shape and motion of the heart.
Huang then builds 4-D computer models that measure the natural changes, or deformations, in the heart’s structure as it expands and contracts to pump blood.
These models will give cardiologists a clearer picture of the differences between the normal deformations that occur in healthy hearts and the abnormal deformations that occur in damaged and diseased hearts. They will also help physicians recognize different types of abnormal deformation and correspond them to various types of disease or damage.
“The clearer the picture we can obtain of the deformation in a normal heart,” says Huang, “the more quickly and reliably we can identify abnormalities in deformation that indicate a diseased or damaged heart. And if we obtain enough examples of different types of abnormalities, we can characterize the commonalities of these abnormalities.”
Tagging and tracking
The heart modeled on Huang’s computer screen was scanned by an MRI technique called SPAtial Modulation of Magnetization. SPAMM tags material points within the heart wall, which can be tracked over time to reveal the 3-D motion of the heart muscle.
The tags enable Huang to outline, or segment, the heart’s structures, including the boundaries separating heart and lungs, as well as those demarcating the heart’s outer wall, left and right ventricles and myocardium.
Three views of a 3-D model of the heart's left ventricle (left) and a 3-D illustration of a tag plane.
Huang then locates the vertices, manipulates her image and tracks the path of each vertex as the heart deforms. Based on the displacement of the vertices, she calculates the stress and strain imposed on the heart muscle.
“The model is no longer just a geometric model,” says Huang. “Now it has mechanical properties. This enables us to perform precise quantifications. It also enables us to partition the heart wall into sub-regions and follow each of these to determine whether the deformation that occurs is normal or not.”
Huang collaborates with Dr. Leon Axel, professor and director of cardiac imaging in the Department of Radiology at the New York University School of Medicine. She says her model will enable diagnosticians to interpret medical images in a fraction of the time they now require.
“It takes hours for a human expert to interpret a sequence of images from one patient,” she says. “Our software tool can do segmentation and build a 4-D model in 26 seconds, while finding more instances of abnormal deformations and structures, especially very localized ones.”
Huang’s goal is to give technicians a “second eye” that automates the interpretation of medical scans while disclosing the often-hidden signs of disease and debility.
“Quantitative computer analysis, when combined with qualitative viewing, deepens our knowledge of the mechanical function of normal and abnormal hearts. It enables diagnosticians to correlate important clinical conditions, such as myocardial ischemia, hypertrophy or failure, with changes in regional heart wall shape, motion and mechanical properties.”
Computerized image analysis will not eliminate the need for human experts, Huang says.
“The tools of computer-aided diagnosis [CAD] are becoming more and more powerful, but they still don’t work as well as human analysts. They can, however, help technicians do their job faster, less subjectively and more reliably. Studies show that if you use CAD to aid a human diagnostician, that person’s performance can improve.”
A dynamic subject
MRI, ultrasound, CT (computed tomography) and x-rays have opened an unprecedented window into the interior of the living body, pinpointing tiny abnormalities in the brain, detecting nodules before they become tumors, and rendering detailed images of soft tissue.
But the human body, says Huang, is a dynamic organism that changes in unexpected ways. Over time, a lesion may or may not turn cancerous. Over time, as it is being radiated, a tumor can shift its position or change shape. Over time, signs of strain or stress in the mechanical performance of the heart can go unnoticed by even the savviest scan reader.
The challenge in using computers to analyze medical images, says Huang, is to enable computers see the world as humans do and to infer 3-D features from 2-D and 3-D images.
“A medical image is stored in the computer as an array of 2-D or 3-D numbers. It is the task of my software to give intelligence to the computer—to recognize structures in the body and to quantify relevant properties for diagnosis and treatment planning.”
One of Huang’s software programs draws information from images acquired with Positron Emission Tomography (PET), an imaging technique used in radiation oncology. PET produces a 3-D image or map of functional processes in the body such as blood glucose metabolism. The map reveals “hot spots” that represent high areas of energy activity in the body—either an organ like the heart or an abnormal site such as a tumor or inflammation.
Huang’s software automates the interpretation of a PET scan by defining the precise boundaries of hot spots and by differentiating between normal hot spots such as the heart or kidneys and abnormal hot spots such as tumors. Both show up brightly in a scan. To differentiate between them, Huang uses segmentation to identify the precise boundaries of organs and hot spots in both the PET scan and its accompanying CT scan. She applies pattern recognition and computer vision methods to identify organs based on their shape and location so that she can “suppress” these normal hot spots and then track the changes over time in abnormal hot spots.
“The precise boundaries of organs and tumors are very valuable,” says Huang. “Oncologists rely on them to make radiation treatment plans in modern Intensity-Modulated Radiation Therapy. IMRT is designed to protect normal organs by exposing them to as little radiation as possible while targeting tumors with high-intensity radiation.”
Last year, Huang was awarded a grant from the Lindback Foundation’s Minority Junior Faculty Award Program to model the geometric and physical properties of the heart, the project on which she is collaborating with Dr. Axel.
She also won a contract from the National Library of Medicine (NLM), part of the National Institutes of Health, to help create and organize a digital archive of images of the uterine cervix. NLM and the National Cancer Institute have collected 2-D color photographs of the cervixes of 60,000 women. Experts, working manually, have identified the boundaries of the cervix and of abnormal regions in 1,000 of the images. Huang has developed automated segmentation methods to detect abnormal regions in the remaining 59,000 images, and to distinguish between different types of lesions, some cancerous and some not.
“Our automated segmentation methods are getting good results,” says Huang. “Two other groups worked on this project before I joined. They made some progress detecting boundaries of the cervix, but our project is the only one that has detected and classified abnormal lesion boundary markers.
“The segmentation techniques I’m developing for this project are quite different from those I’ve developed for the heart. The images of the heart that I’m working with were obtained by MRI scans, which are intensity-based. The cervical images I’m working with are 2-D color photos. There is a lot of variability in these images because of the lighting and content.”
Huang is also classifying the different types of benign and pre-cancerous lesions, and developing indexing methods that help experts retrieve images representing each type of lesion. Ideally, she says, her system can be used as a computer-aided diagnostic tool and as a teaching aid for medical residents.
Huang’s segmentation technique will be compared with the results obtained by the experts who examined the 1,000 cervical images manually. Many of those images were interpreted by two or three experts. Huang is developing an algorithm, or step-by-step computational procedure, that will evaluate and rate the performance level of each expert and make it possible to obtain a more reliable overall analysis of an image examined by multiple experts. The algorithm will also be able to evaluate the performance level of a computer analysis, providing a yardstick that can be used to compare different types of automated segmentation methods.
Huang also develops algorithms for image registration. In printing, registration refers to the correlating of color separations. In medical imaging, it refers to the aligning or matching of scans taken over time of one person. Registration corrects for changes that are caused by different poses or positions a person assumes when a scan is taken. The scans are “corresponded” to allow a diagnostician to identify the remaining changes—the meaningful pathological changes – in the feature being studied.
Image registration is also employed to correspond the images of the same organ or feature that are taken from more than one person; this enables researchers to learn more about variations in the normal morphology, or geometry, of an organ and variations in that organ’s diseased states.
Huang’s software tools possess many layers: They can do geometrical, statistical and mechanical modeling, while taking into account such factors as a patient’s history and an expert’s track record. These assets enable her programs to keep working when unforeseen obstacles are encountered, and they give her programs an advantage over those developed by other researchers.
“The main obstacle to medical image computing is that you have to do it robustly,” Huang says. “Too often, when a software program encounters noise [slight variations in intensity, unwanted disturbance, energy or interference] or artifacts [ranging from something obvious like dental fillings to something more subtle such as distortions in tissue structure], it breaks down. Once this happens, it is difficult for a diagnostician to recover information.
“You have to choose what kind of model you want to use. Our model can be as complex or as simple as we want. We can attach to it as many properties as we want it to have. It can represent geometry or it can be a statistical model. It can encode variations across patients and infer a statistical model from multiple people. Or it can have mechanical properties. This builds in robustness.”
Huang has begun collaborating with biologists and biophysicists at Lehigh whose work requires perturbing the cell and its organelles in order to gain fundamental knowledge of cell motions and functions. She hopes to help these researchers more precisely quantify the dimensions of organelles, the amount of time required for mitosis, and other phenomena.