CURRENT ISSUE | SUBSCRIBE | ADVERTISE | WRITE TO US | CORNELL AUTHORS | PAST ISSUES

Over the past four years, electrical engineer Anthony Reeves and computer scientist Ramin Zabih have each made the commute between Ithaca and New York City more than 100 times. But the two have a lot more in common than the thousands of miles each has logged traveling between the University's main campus and its Medical College in Manhattan. Both researchers are experts in computer vision, creating mathematical algorithms to analyze digitized images.

And through collaborations with clinicians in Weill Cornell Medical College's Department of Radiology, each has used his expertise to enhance the ability of physicians to accurately diagnose their patients.

Radiology is one of the most computerized fields in medicine, yet its practice lags far behind the sophisticated technology available.While the resolution and sheer number of images radiologists can obtain have increased dramatically, assessment has been less formalized. "The standard for measuring lung nodules today is that the radiologist puts calipers on a two-dimensional image, looks at the largest extent, and says, 'That's the size,' " says Reeves, who works in the field of lung cancer detection and diagnosis. "The technology is producing so many more images in so much more detail that the concept of having a human look at them when a machine has so many more advantages is simply impractical. It's a no-brainer." But first the images have to be transformed into numbers. "Once you have something in quantitative form," says Zabih, "then you can look for patterns."

The task of computer vision researchers is deceptively complex: it involves figuring out what the human mind does instantly and intuitively, and translating that process into mathematical language. "If you ask a person, is it easy or hard to do calculus, they say it's hard," says Zabih. "If you ask a person, is it easy or hard to count the number of people in a room, they say it's easy. But for a computer, the opposite is true." Ultimately, whether a computer vision researcher analyzes photographs of crowds, Computed Tomography (CT) scans, or Magnetic Resonance (MR) images matters little; the same strategies apply. In each of their projects, Zabih and Reeves are building an arsenal of what Zabih calls "power tools," algorithms that translate a radiologist's medical knowledge into a computer program to automate such tasks as maximizing the quality of an image or even helping to diagnose such conditions as lung cancer, aneurysms, and breast cancer.

In 1997, Reeves teamed up with Weill Cornell radiologists Claudia Henschke and David Yankelevitz to develop algorithms for analyzing CT scans of the lung. Henschke, the project's principal investigator, has long recognized the need for computeraided techniques in the field. After earning a doctorate in mathematical statistics in 1969, Henschke consulted on clinical trials for the Veteran's Administration and the National Academy of Science. Then she decided to go to medical school. It was 1977, and CT, a technique that was the first to code diagnostic images into numbers, was just being introduced. "My whole idea for going into radiology was because I was a statistician and a computer programmer," says Henschke, now division chief of Weill Cornell's Chest Imaging. "I wanted to do things with CT numbers, and what is now known as CT image analysis."

With the advent of helical CT scanning in the early 1990s, she and Yankelevitz set out to demonstrate its superiority to traditional chest X-ray screening methods. For help, they contacted the Engineering college on the Ithaca campus, where they found Reeves. Their research focuses on automating the detection, measurement, and diagnosis of pre-cancerous nodules. "In our early days," says Reeves, "we pioneered the concept of trying to measure exactly where is the lesion, where is the blood vessel or chest wall attached to it, where is the joining point between the two. Now this is pretty well accepted as the way you would measure these nodules on the computer, and most CT manufacturers now have products that follow that strategy."What made the difference was Reeves's ability to see the problem from an engineer's perspective. "When we got together,my first reaction was, this is a three-dimensional problem," says Reeves. As a result, the algorithm the trio developed treated a CT scan not as a flat image but as a three-dimensional object consisting of a set of two-dimensional images. The result was a method for measuring the size of lung nodules that far surpassed anything available at the time.

The ability to accurately measure the size of tumors holds the potential to improve the cure rates of patients with lung cancer. "To date," says Reeves, "the best predictor of malignancy is rate of growth." A November 2003 study led by Weill Cornell cardiothoracic surgeon Nasser Altorki found that even minute differences in tumor size have a measurable effect on patient survival, and underscored the need for early detection. Yet nodules are usually detected only after they have grown past the point of easy surgical removal, leading to the historically low cure rate for the disease.With accurate nodule measurement, high-risk patients can be screened periodically, allowing radiologists to analyze the growth of a lesion over time and thus assess its malignancy. "Before, I would look at a scan and say, 'Well, this looks a little bit bigger,' and have to make the decision of whether to do a biopsy--an invasive procedure," says Henschke. "But this technique, as it gets better and better, will make us more confident in making that recommendation." Ultimately, the team hopes their algorithm might even supplant the need for biopsy altogether.

Unlike Reeves, who was sought out by clinicians, Zabih relied on serendipity to yield his collaborations.  In April 2000, the day after the computer science department in Ithaca voted to grant him tenure, Zabih phoned Weill Cornell's Department of Radiology and requested permission to spend his sabbatical there. "I figured maybe at the end of the year they'd be interested enough in what I was doing to make some kind of collaboration."

Early on, Zabih met Martin Prince, director of Weill Cornell's MRI service. Despite MRI's spectacular resolution, it is highly prone to motion artifacts, caused both by quirks in the imaging technology and the natural movements of the human body. "Even though it's been around for fifteen or twenty years as an imaging modality, MRI is still relatively new," says Prince. "And the tools that you need in order to figure things out haven't become commercially available."With Prince, Zabih has developed an algorithm that automatically corrects for motion artifacts in MR angiography, a non-invasive technique for visualizing blood vessels, and provides radiologists with significantly superior images. "For very specific task," says Zabih, "it's actually as good as an expert radiologist."

Zabih has also partnered with Dr. Ruth Rosenblatt, director of Women's Imaging at Weill Cornell, to develop strategies to encode the biological properties of human tissue, thereby distinguishing potential tumors from surrounding flesh. After entering the MR scanner, patients are injected with a tiny amount of a harmless gadolinium-based compound that acts as a contrast agent as it passes through the body. While normal tissue releases the gadolinium evenly over time, tumors have "leaky capillaries," which cause the gadolinium to diffuse out of the tissue in a rush. By measuring the rate at which the contrast agent passes through the tissue, a radiologist could potentially differentiate benign from malignant tissue with the sweep of a computer mouse. "It's like having a spell-checker that recognizes a certain pattern," says Rosenblatt.

One of the biggest challenges facing Zabih and his Medical College collaborators has been the 240-mile distance between them. Reeves and Henschke's lung cancer group has minimized the distance by relying on the Internet, but for Zabih's student Ashish Raj, the miles are not virtual, but mind-numbingly real. Like Bill Murray's character in the movie Groundhog Day, who must repeat the events of one day in his life until he gets it right, Raj travels between Ithaca and New York perfecting his image correction algorithm, spending about one week each month in the city. At the Medical College, he works with Prince to identify scans where motion artifacts have occurred, then returns with the images to his home base in Ithaca. After refining the algorithm to improve the images, he goes back to the city, sits down with Prince again, and asks the radiologists to rate a "double-blind" assortment of corrected and uncorrected images. "Every time he gives a score I ask him, 'Why did you give it this score and not that score?' or 'Why did you think this image was better?' " says Raj. "The things he tells me are pretty much common sense: you want more contrast, more detail, less background. It's just that it really does help to sit down with him when he points out what he's looking for in each case."

Ultimately it is the interaction between the clinical and the theoretical that drives the researchers' insights. "We're trying to be scientists," says Rosenblatt, "but we're also clinical. By bringing two different points of view together, one will help the other." Such collaborations really work, says Zabih, when problems are tackled from both ends. A researcher can design a solution, but a clinician has to test it. "They'll push back on you," says Zabih. "They'll say, 'Hmm, it does a pretty good job here, but in the following circumstances it doesn't work. Do you have any ideas?' Then you can say, 'Let me work on it.' "

Paving the Way Scientists lay the groundwork for cross-campus collaborations.

In recent years, Cornell has made fostering cross-campus research ventures a priority, one that ranks high on the agenda of President Jeffrey Lehman. "There's a growing interest in collaborations, not just between scientific disciplines, but also between science and the social sciences, between science and the humanities, public health, and outreach," says Vice Provost Lisa Staiano-Coico, PhD '81, who holds a joint appointment at both the Medical College and the University. "The more we highlight these collaborations, the more we can leverage our expertise and our strengths."

Seventy faculty members currently participate in twentyone cross-campus collaborations, spanning areas of research from robotics and biochemistry to the social aspects of aging, but very few have been clinical in nature. Computer vision experts Anthony Reeves and Ramin Zabih have more than a decade of combined experience working with Medical College professors, and the challenges have been myriad. From harnessing technology to span the distance between researchers to funding graduate students and making joint applications for grants, Reeves and Zabih have consistently had to blaze a trail. "There was nothing there to make it easy," says Reeves. "There was no mechanism, at least when we started seven years ago, for these interactions."

Particularly in bridging the gulf between clinical medicine and basic research, collaborative research depends heavily on proximity. "I'm a technical problem solver," says graduate student Ashish Raj, a member of Zabih's team. "And I can only solve a problem when I know about it. But to do that we need to interact more with the medical community."

Once theoreticians and clinicians have found one another, they must also overcome profound differences between how they approach research questions. "In computer vision there generally isn't one really good answer," says graduate student Amy Gale, another member of Zabih's research team. "If you offer a computer algorithm a picture of a hand and say, ‘How many objects are in this?'--well, is there one hand or are there five fingers?" Computer vision scientists, she says, might conclude that a problem can't be solved and call it a day. "But there's no real point to saying, ‘This is not a solvable problem' in medicine. You have to come up with something. If there's no perfect answer, you have to come up with a very good answer." Zabih, who has held a joint appointment between Ithaca and the Medical College since 2001, puts it another way: "Academics care about impressing other academics. Doctors couldn't care less about anything that doesn't work."

What has undoubtedly helped these collaborations bloom is the common technical backgrounds the New Yorkers and the Ithacans share. "Mathematics helps bridge the gap," says radiologist Claudia Henschke of her work with Reeves. MR specialist Martin Prince, Zabih's collaborator, earned a doctorate in engineering at MIT before going to medical school.

A more logistical challenge in cross-campus research has been getting Ithaca-based graduate students to New York City. While Reeves has developed an extensive Web-based system that has made it feasible for his students to connect with the radiologists electronically, the situation, says Henschke, is not ideal. "Coming and seeing the medical environment and the process gives them a different insight in terms of what is really valuable, how things need to be presented to the patient. What would help a lot is to have a fund so that students can regularly come down and spend a week or maybe a month working with us."

Zabih's students work for a significant amount of time in the city, where they have found themselves in the strange position of having to register as visiting students at a branch of their own university. Gale, for example, alternates semesters, spending the fall in Ithaca and the spring in New York. "I've never really been clear on what my relationship to the system is," she says. "I can read notices on elevator walls, but I'm not really in the loop." While she managed to secure housing in a Medical College dorm last year and was covered by the University's health plan in Ithaca, she wasn't eligible to be treated at the college's student health center.

The hurdles encountered by Zabih's students revealed an administrative gap that might not otherwise have been obvious. "We realized that the success of this collaboration depended heavily on graduate student and faculty exchange, and so we sought to eliminate some of the associated administrative burdens," says Staiano-Coico. This January, the Medical College announced the Cornell University Graduate Student Synergy (CUGSS) Program, which gives students who must spend at least a semester working at Weill Cornell essentially the same status as students in the Graduate School of Medical Science. The students now have access to Weill Cornell's graduate housing pool and student health services, as well as cultural events. They are also able to register and earn credit for courses. Similar plans are in the works for reciprocal exchanges of New York-based graduate students to the Ithaca campus. Staiano-Coico reports that initial feedback from faculty has been positive. "It makes their life easier," she says. "Now they have somebody to contact; they have a formalized process so that it's not reinventing the wheel every single time they want a student to come down."

Speed of Light Group goes online to bridge distance

One of the widest gulfs between Cornell's Ithaca campus and the Medical College is the 240 miles that separate the two. "I can't just go across the street and be in the medical community," says electrical engineering professor Anthony Reeves. "I can't just attend a meeting. Logistics have to come into it." But having surmounted the initial challenge of finding one another, Reeves and Weill Cornell radiologists Claudia Henschke and David Yankelevitz have transformed distance from a hindrance into one of the driving elements of their work. On the long wall of the conference room of Weill Cornell's Lung Cancer Screening Program hangs a map of the world, marked with nearly three dozen colored flags pinpointing locations across the United States and cities around the world--in France and Finland, China and the Philippines. Each represents a clinical trial site in the International Early Lung Cancer Action Project, I-ELCAP. Along with NY-ELCAP, a parallel program running in a network of medical centers throughout New York City, I-ELCAP investigates the benefits of CT scanning for lung cancer detection.

At one of their first meetings, Reeves watched Henschke filling out forms by hand. "We could put them on the Web," he told her. What developed is the group's online management system, a databank that allows the team to match specific scans with information from the clinical trials. So far, it contains more than 20,000 baseline CT images and as many repeat scans, which are compared to investigate lung health and lung cancer development in at-risk patients. Using the ELCAP database, Henschke and her team published a groundbreaking study in the July 1999 Lancet demonstrating that death rates from lung cancer could potentially be reduced from almost 80 percent to 20 percent if at-risk patients (smokers and former smokers) were regularly screened with CT scans.

"One of the great things about the ELCAP project is that we're facilitating the pooling of data," says Reeves. "By gathering information, we're able to address many more questions than we would be able to do with these studies individually." The system also guides the entire protocol by ensuring that procedures are as consistent as possible at trial sites around the world. By maintaining a fully standardized protocol in real time, says Henschke, "we've brought about a whole new paradigm on how studies can be done in the future."

ALLA KATSNELSON '96 is a science writer living in New York City.

Return to top of page