This is a story of humans and hardware. What happened when professors, tops in their different fields of energy research, gained campus access to a world-class supercomputer?
The story began four years ago with the institutional vision to bring a supercomputer named Ra to Mines. Dag Nummedal, director of the Colorado Energy Research Institute, and Physics Professor Mark Lusk had been working to acquire a supercomputer, and when Vice President of Research and Technology Transfer John Poate got involved, “the idea resonated campus-wide,” said Lusk.
With the horsepower of Mines leadership behind this well-timed initiative, it became a commitment to much more than hardware. Five years, one supercomputer, 10 new faculty hires, 15 classes, 60 PhD students and 120 journal publications since that original vision, Mines has become a global leader in computationally guided energy science research.
The initiative has made a huge impact on research volume, led to many important discoveries, and catalyzed interdisciplinary collaboration across campus. More than 90 percent of Mines’ academic departments are pursuing projects supported by the Golden Energy Computing Organization. As scientists from different areas come together, ideas begin to cross-fertilize and surprising synergies emerge. As Lusk notes, “Once all these people start working together under one virtual roof, good things happen.”
One of the new supercomputer hires was Amadeu Sum, a professor in the Chemical and Biological Engineering Department and co-director of Mines’ Center for Hydrate Research. He used Ra to explain the nucleation and growth of hydrates, and his work landed on the cover of Science.
On another side of campus, REMRSEC, the Renewable Energy Materials Research Science & Engineering Center, was chosen for funding by the National Science Foundation in part due to the computing power Ra could bring to the table.
Lusk’s solar cell work with REMRSEC on multiple-exciton generation (MEG) was successful. MEG theorizes it is possible for an electron that has absorbed light energy to transfer some of that energy to other electrons, resulting in more electricity from the same amount of absorbed light.
In a cross-fertilizing leap, REMRSEC decided to look at hydrates as a way to store hydrogen. The center provided seed money to Carolyn Koh, a professor in the Chemical and Biological Engineering Department, to lead a combined team of REMRSEC solar energy scientists and Center for Hydrate Research experts. Together they developed a computer analysis to assess the potential of hydrates for hydrogen storage.
Then they began thinking of other materials that can be assembled into the cage-like clathrates. Could they build a silicon clathrate structure to store hydrogen? Using experimental facilities at the National Renewable Energy Laboratory (NREL), they determined the answer was yes.
The synergy continued. What are the photovoltaic properties of these new silicon clathrates? Can they be used to build a better solar cell? The answer, once again, appears to be yes.
In summary, hydrate engineers and solar energy physicists have founded two new facets of energy research because a world-class supercomputing facility came to Mines. “The successes that have come from our original vision have been snapping together a Lego™ at a time,” said Lusk. “We have a cool system going now and there’s no end in sight.”
A New Season
Now plans are underway to purchase a new machine to become the campus flagship for high-performance computing, with Ra maintained as a set of smaller clones for less demanding projects and student training. Requests for bids have gone out to industry, and by autumn 2012 the new machine should be on campus, humming alongside its predecessor.
The next supercomputer will be a radical step forward, with at least five times Ra’s computing power and roughly 16,000 processor cores to Ra’s 2,144. Even so, it will consume just a fraction of Ra’s physical space and electrical power, thanks to technological developments over the past few years.
Ra’s successor will give a boost to many of Mines’ most ambitious efforts, and Lusk predicts the new machine will be the basis for frontier energy research for years to come. “It’s exciting to be part of this vision,” he said. “The campus now fields several big teams that do high performance computing in close collaboration with experimentalists. The original leadership has evolved into some amazing self-assemblies, and I can’t wait to see what advances come out next.”
Clathrate Hydrates
Researchers have achieved the first real insight into the birth and growth of the cage-like structures known as clathrate hydrates. These materials can form naturally —for example, out of natural gas in pi/gas pipelines, where they form an “icy slush” that can accumulate in the pipelines and eventually clog the flow. Using Ra, Mines researchers have been able to simulate for the first time the molecular processes that cause such hydrates to nucleate and grow, adding – atom by atom – to each rigid molecular cage.
It’s not an easy task. Hydrates form out of disordered systems, with atoms starting out adrift and then coming together in precise ways to form a complex network of water molecules enclosing gas molecules. Simulating how that transition happens takes a lot of computing power, said Amadeu Sum. “That’s why we need to use large resources like we have on campus to do these large and long simulations,” he said.
Knowing how hydrates nucleate will help researchers better understand how to prevent/control them from forming and harness them for useful purposes as well. One impactful area for hydrates is the recovery of methane gas from natural hydrate deposits in the permafrost and ocean seafloor, and the utilization of hydrates as an energy storage medium for natural gas and hydrogen.
Oil and Gas
With the need for traditional fossil fuels still great, Associate Professor Paul Sava is using Ra to discover new sources of oil and gas. Active in the Center for Wave Phenomena in the Geophysics Department, Sava specializes in developing new methods for probing the earth’s interior with seismic waves. Doing so requires running simulation after simulation of how quickly waves travel through the earth, then comparing those to real-world observations to see how closely the two match.
So far, Sava’s team has been able to refine a flagship exploration technique used in industry. The Mines scientists simulate how rock from inside the planet can be squeezed (like a fluid, as industry models it) or twisted around (like an elastic) — a difference that affects the speed of passing seismic waves. Updating this knowledge allows oil and gas companies to better predict where a promising prospect might turn into a lucrative discovery. “The information relevant to them requires a big computer like this,” Sava said.
Hydrology
Mines’ dedication to high-performance computing has helped draw high-profile faculty to the university. “When I showed up, Ra was being unboxed,” said Reed Maxwell, a hydrologist who moved from the Lawrence Livermore National Laboratory in California.
In the Department of Geology and Geological Engineering, Maxwell uses Ra to simulate how water flows from deep within the ground to shallower levels, and also from there into the atmosphere. His computer code, dubbed ParFlow, is one of the few such models to integrate this entire hydrologic cycle. Maxwell has used ParFlow to explore all sorts of important questions, such as how agriculture draws down groundwater and how changes in hydrology affect local atmospheric patterns —for example, the wind energy potential over a particular plot of land.
Because his simulations require so much computing power, Maxwell uses not just Ra but also several other supercomputers, including ones at the Oak Ridge National Laboratory in Tennessee and at a facility in Jülich, Germany. “I always envision running on a range of supercomputers,” he said.
Most recently, Maxwell has built a high-resolution hydrological model of the entire continental United States, which covers 6.3 million square kilometers at a resolution of 1 kilometer. This simulation, which he said is one of hydrology’s “grand challenges,” is tied into leading climate models so that Maxwell can, for example, probe how water flow may affect regional climate change in the decades to come.
Materials
From the scale of continents down to the scale of nanoparticles, Ra’s simulations are doing it all. For Cristian Ciobanu, a materials scientist in the Department of Mechanical Engineering, Mines’ computing resources involve the very small. He works to understand the basic chemistry and physics of materials crucial for energy applications, from lithium-ion batteries to biomass to solar cells. “In all these cases, the campus facility is important,” Ciobanu says. “It’s basically a lot of computing power on site, and you can do things faster, closer to real time.”
For instance, he and his collaborators at NREL have shown that using a material as common as quartz can lead to an increase in the capacity of lithium-ion batteries over the first couple hundred cycles of charging and discharging, thus hinting at new ways to prolong battery life. Other simulations have shown that adding nanoparticles of gold or other precious metals to a particular chemical process speeds up the reaction, while also making it yield a desired reaction product, hastening the conversion of biomass into energy. Ciobanu is now running calculation after calculation on Ra to find the best possible shape and composition for nanoparticles to catalyze the biomass conversion reactions. With such information, an experimentalist can make nanoparticles that work efficiently the first time around, without having to run though the trial-and-error of testing particle after particle in real life.
For solar cells, Ciobanu has been testing how to make the perfect nanoparticles out of germanium and tin with the best electronic properties for absorbing light. Such alternative materials might be used in future photovoltaic cells, especially if they can be designed through supercomputer simulations and then tailor-made to fit those designations.
Photovoltaics
Lusk is also using the power of Ra to figure out how to make better photovoltaics. In 2011, he and colleagues discovered one way to beef up the efficiency with which a solar cell transforms sunlight into energy. Supercomputer simulations done at Mines suggest that in a specially designed material, a particle of light (photon) can knock loose not just one electron (its flow creates the electricity that powers solar cells) but two or more, in a process known as multiple exciton generation. The set of excited electrons would turn more of the original solar energy into useful electricity because not as much is lost to generate heat.
The race is now on to make better photovoltaic materials by exploiting multiple exciton generation and other quirky quantum mechanical properties that Lusk’s team has discovered. They use Ra to model how to best design what amounts to a new form of matter composed of quantum dots. These tiny particles, just a nanometer or two across, both help to capture solar energy and to move it through the solar cell to create useful current.
Lusk is also looking to build on nature’s own solar cell — the leaf — by co-opting its photosynthetic tricks. Nature has evolved some very clever ways of harvesting solar energy, but its solar panels don’t last very long. “We’re using the computer to unravel some of the quantum mechanical secrets that are going on all around us. And then we want to use that information to build inorganic solar cells that do the same thing better and without wearing out as easily,” Lusk said.
This article appears in the 2012-13 issue of Energy and the Earth magazine.
