Partner Content: Why Faster Supercomputers Aren’t Enough for Genomics
Veterans’ data, researchers’ responsibility
A million veterans shared genetic data through the Department of Veterans Affairs’ Million Veteran Program without expecting anything in return. Ravi Madduri, a senior scientist at the U.S. Department of Energy’s Argonne National Laboratory, says he feels an obligation to use the information in ways that serve their intentions and may benefit others.
Madduri joined Argonne as a graduate student in 2001. For two decades, he has worked at the intersection of supercomputing and biomedicine, helping build systems for research into conditions including post-traumatic stress disorder, heart disease and cancer.
Why CPUs could not handle it
The team ran the analysis on Summit at the Oak Ridge Leadership Computing Facility and Aurora at the Argonne Leadership Computing Facility. On Summit, the job stalled because the matrices needed for each group grew beyond the CPUs’ available memory.
The code had been written for conventional processors, but most of its work involved multiplying very large grids of numbers. The team, with Alex Rodriguez leading this effort, rewrote that portion to run on graphics processors. A task estimated to require roughly 481,000 CPU-core hours in its original form became tractable.
Separate analyses reveal associations
The research examined genetic data from veterans who had been sequenced through the program. Because ancestral groups differ, researchers could not simply pool everyone’s data: a marker associated with ancestry might otherwise be mistaken for a disease marker.
The team analyzed each group separately and combined the results afterward. The 1,600 associations appeared only in the combined results, across 13,672 regions; they were not all present in any one group’s analysis.
Discovery is only an early step
Madduri describes the work as preclinical discovery: a statistical map showing where researchers might investigate. Turning any of the regions into a treatment would require years of further work, and the team continued its efforts.
He says biomedical research can be incremental, with new datasets analyzed each year and new genetic variants identified across diseases. In his view, limited access to large-scale computing is one reason progress can follow that pattern.
Moving models instead of data
One approach to data-access constraints is federated learning, which trains models where data is stored rather than moving the data between institutions. Each site trains on its own copy and sends back model adjustments, which are combined into an updated model and shared for another round.
Madduri co-developed APPFL, the Argonne Privacy-Preserving Federated Learning framework, to support this approach. The article also identifies permission as a major challenge, but its final sentence is incomplete.
Key points
- The team shifted key calculations from CPUs to graphics processors to make the analysis tractable.
- Researchers analyzed ancestral groups separately, then combined results to identify 1,600 associations across 13,672 regions.
- The findings are preclinical leads, not treatments; further research could take years.
- Federated learning can train models across institutions without moving their underlying data.