Testimony
of Dr. James F. Decker
Principal Deputy Director, Office of Science,
U.S. Department of Energy
before the U.S. Senate Committee on Energy
and Natural Resources
Subcommittee
on Energy
June 22, 2004
Mr. Chairman and members of
the Committee, I commend you for holding this
hearing - and I appreciate the opportunity
to testify on behalf of the Department of
Energy's (DOE) Office of Science, on a subject
of central importance to this Nation: advanced
supercomputing capability for science.
The Bush Administration has
recognized the need for the U.S. to emphasize
the importance of high-end computing and is
working as a team to address it. The Administration
commissioned an interagency study by the High
End Computing Revitalization Task Force (HECRTF).
The HECRTF report (http://www.itrd.gov/pubs/2004_hecrtf/20040510_hecrtf.pdf)
reinforces the idea that no one agency can
- or should - be responsible for ensuring
that our scientists have the computational
tools they need to do their job, but duplication
of effort must be avoided.
Through the efforts of DOE's
Office of Science and other federal agencies,
we are working to implement the recommendations
of the HECRTF Report by investing in the development
of the next generation of supercomputer architectures,
as well as the networks to enable widespread
access to these new supercomputers.
On May 12th of this
year, Secretary Spencer Abraham announced
that the DOE will grant Oak Ridge National
Lab (ORNL), Argonne National Lab, Pacific
Northwest National Lab and its development
partners, Cray, IBM and SGI, $25 million in
funding to begin to build a new supercomputer
for scientific research. The Department selected
ORNL from four proposals received from its
non-weapon national labs. The Department
is in the final stages of completing this
award and expects to start the project before
the end of this fiscal year.
*
* *
Computational modeling and simulation
rank among the most significant developments
in the practice of scientific inquiry in the
latter half of the 20th century and are now
a major force for discovery in their own right.
In the past century, scientific research was
extraordinarily successful in identifying
the fundamental physical laws that govern
our material world. At the same time, the
advances promised by these discoveries have
not been fully realized, in part because the
real-world systems governed by these physical
laws are extraordinarily complex. Computers
help us visualize, test hypotheses, guide
experimental design, and most importantly
determine if there is consistency between
theoretical models and experiment. Computer-based
simulation provides a means for predicting
the behavior of complex systems that can only
be described empirically at present. Since
the development of digital computers in mid-century,
scientific computing has greatly advanced
our understanding of the fundamental processes
of nature, e.g., fluid flow and turbulence
in physics, molecular structure and reactivity
in chemistry, and drug-receptor interactions
in biology. Computational simulation has even
been used to explain, and sometimes predict,
the behavior of such complex natural and engineered
systems as weather patterns and aircraft performance.
Within the past two decades,
scientific computing has become a contributor
to essentially all scientific research programs.
It is particularly important to the solution
of research problems that are (i) insoluble
by traditional theoretical and experimental
approaches, e.g., prediction of future climates
or the fate of underground contaminants; (ii)
hazardous to study in the laboratory, e.g.,
characterization of the chemistry of radionuclides
or other toxic chemicals; or (iii) time-consuming
or expensive to solve by traditional means,
e.g., development of new materials, determination
of the structure of proteins, understanding
plasma instabilities, or exploring the limitations
of the "Standard Model" of particle
physics. In many cases, theoretical and experimental
approaches do not provide sufficient information
to understand and predict the behavior of
the systems being studied. Computational modeling
and simulation, which allows a description
of the system to be constructed from basic
theoretical principles and the available experimental
data, are keys to solving such problems.
We have moved beyond using computers
to solve very complicated sets of equations
to a new regime in which scientific simulation
enables us to obtain scientific results and
to perform discovery in the same way that
experiment and theory have traditionally been
used to accomplish those ends. We must think
of computation as the third of the three pillars
that support scientific discovery, and indeed
there are areas where the only approach to
a solution is through high-end computation.
Combustion is the key source
of energy for power generation, industrial
process heat and residential applications.
In all of these areas, combustion occurs in
a turbulent environment. Although experimental
and theoretical investigations have been able
to provide substantial insights into turbulent
flame dynamics, fundamental questions about
flame behavior remain unanswered. Current
limitations in computational power do not
allow combustion scientists to address the
range of conditions needed to have environmental
and economic impact. Leadership class computers
should enable us to model more complex fuels
with emission chemistry under conditions typical
of industrial settings. These computations
should make it possible to design new low-emission
burners that could dramatically reduce NOx
emissions.
The Fusion Program must be able
to model an experiment the size of the International
Thermonuclear Experimental Reactor (ITER)
through the duration of a discharge that may
last on the order of hundreds of seconds.
Current codes are able to model a variety
of the physical phenomena that occur in small
experiments operating on a millisecond time
scale. Leadership class computers should enable
scientists to simulate burning plasmas in
ITER and include new physics such as more
realistic treatment of electron dynamics and
multiple species of fusion products such as
high energy alpha particles.
High-end computing must be coupled
with high-performance networks to fully realize
its potential. These networks play a critical
role because they make it possible to overcome
the geographical distances that often hinder
science. They make vast scientific resources
available to scientists, regardless of location,
whether they are at a university, national
laboratory, or industrial setting. We work
with the National Science Foundation and university
consortia such as Internet 2 to ensure that
scientists at universities can seamlessly
access unique DOE facilities and their scientific
partners in DOE laboratories. In addition,
the emergence of high performance computers
as tools for science, just like our light
sources, accelerators and neutron sources,
has changed the way in which science is conducted.
Today and in the future, large multidisciplinary
teams are needed to make the best use of computers
as tools for science. These teams need access
to significant allocations of computer resources
to perform leading edge science. In the Office
of Science we are building on the experience
of the National Nuclear Security Administration's
Office of Advanced Simulation and Computing
program to build and manage these teams.
*
* *
The astonishing speeds of new
high-end machines, including the Earth Simulator,
should allow computation to inform our approach
to science. We are now able to contemplate
exploration of worlds never before accessible
to mankind. Previously, we used computers
to solve sets of equations representing physical
laws too complicated to solve analytically.
Now we can simulate systems to discover physical
laws for which there are no known predictive
equations. We can model physical structures
with hundreds of thousands, or maybe even
millions, of "actors" interacting
with one another in a complex fashion. The
speed of our new computational environment
allows us to test different inter-actor relations
to see what macroscopic behaviors can ensue.
Simulations can help determine the nature
of the fundamental "forces" or interactions
between "actors."
*
* *
The ASCR program mission is
to discover, develop, and deploy the computational
and networking tools that enable scientific
researchers to analyze, model, simulate, and
predict complex phenomena important to the
Department of Energy - and to the U.S. and
the world.
Advanced scientific computing
is central to DOE's missions. It is essential
to simulate and predict the behavior of nuclear
weapons and aid in the discovery of new scientific
knowledge.
As the lead government funding
agency for basic research in the physical
sciences, the Office of Science has a special
responsibility to ensure that its research
programs continue to advance the frontiers
of science. This requires significant enhancements
to the Office of Science's scientific computing
programs. These include both more capable
computing platforms and the development of
the sophisticated mathematical and software
tools required for large-scale simulations.
Existing highly parallel computer
architectures, while extremely effective for
many applications, including solution of some
important scientific problems, are only able
to operate at 5-10% of their theoretical maximum
capability on other applications. For most
vendors, today's high performance computer
market is too small a fraction of the overall
market to justify the level of R&D needed
to ensure development of computers that can
solve the most challenging scientific problems
or the substantial investments needed to validate
their effectiveness on industrial problems.
Therefore, we are working in
partnership with the National Nuclear Security
Administration (NNSA), the National Security
Agency (NSA), and the Defense Advanced Research
Project Agency (DARPA) to identify architectures
which are most effective in solving specific
types of problems; to evaluate the effectiveness
of various different existing computer architectures;
and to spur the development of new architectures
tailored to the requirements of science and
national security applications.
This partnership is working
to ensure the development of computers that
can meet the most demanding Federal missions
in science and national security. We are
also working to transfer the knowledge we
develop to U.S. industry to enable a vibrant
U.S. high performance computing industry,
which can provide the impetus for economic
growth and competitiveness across the nation.
The Office of Science plays a key role in
providing these capabilities to the open science
community to support U.S. scientific leadership,
just as we do with other facilities for science.
Advanced scientific computing
will continue to be a key contributor to scientific
research in the 21st century. Major
scientific challenges in all Office of Science
research programs will be addressed by advanced
scientific supercomputing. Designing materials
atom-by-atom, revealing the functions of proteins,
understanding and controlling fusion plasma
turbulence, designing new particle accelerators,
and modeling global climate change,
are just a few examples.
In fact, in fulfilling its mission
over the years, the Office of Science has
played a key role in maintaining U.S. leadership
in scientific computation and networking worldwide.
Consider some of the innovations and contributions
made by DOE's Office of Science:
- helped develop the Internet;
- pioneered the transition
to massively parallel supercomputing in
the civilian sector;
- began the computational analysis
of global climate change;
- developed many of the computational
technologies for DNA sequencing that have
made possible the unraveling of the human
genetic code.
*
* *
Various computational scientists
have said that discovery through simulation
requires sustained speeds starting at 50 teraflops
to examine a subset of challenging problems
in accelerator science and technology, astrophysics,
biology, chemistry and catalysis, climate
prediction, combustion, computational fluid
dynamics, computational structural and systems
biology, environmental molecular science,
fusion energy science, geosciences, groundwater
protection, high energy physics, materials
science and nanoscience, nuclear physics,
soot formation and growth, and more.
The Office of Science also is
a leader in research efforts to capitalize
on the promise of nanoscale science and biotechnology.
This revolution in science promises a revolution
in industry.
*
* *
To develop systems capable of
meeting the challenges faced by DOE, universities,
and industry, the Office of Science invests
in several areas of computation: high-performance
computing, large-scale networks, and the software
that enables scientists to use these resources
as tools for discovery. The FY 2005 President's
Request for the Office of Science includes
$204 million for ASCR for IT R&D and approximately
$20 million in the other Offices to support
the development of the next generation of
scientific simulation software for SC mission
applications.
As a part of this portfolio the
Office of Science supports basic research
in applied mathematics and the computer science
needed to underpin advances in high performance
computers and networks for science.
In FY 2001 the Office of Science
initiated the Scientific Discovery through
Advanced Computing (www.science.doe.gov/SciDAC/)
effort to leverage our basic research in mathematics
and computer science and integrate this research
into the scientific teams that extend the
frontiers of science across DOE-SC. We have
assembled interdisciplinary teams and collaborations
to develop the necessary state-of-the-art
mathematical algorithms and software, supported
by appropriate hardware and middleware infrastructure,
to use terascale computers effectively to
advance fundamental scientific research at
the core of DOE's mission.
All of these research efforts,
as well as the success of computational science
across SC, depend on a portfolio of high performance
computing facilities and test beds and on
the high performance networks that link these
resources to the scientists across the country.
DOE and the Office of Science have been leaders
in testing and evaluating new high performance
computers and networks and turning them into
tools for scientific discovery since the early
1950s. The Office of Science established
the first national civilian supercomputer
center, the Magnetic Fusion Energy Computer
Center, in 1975. We have tested and evaluated
early versions of computers ranging from the
first Cray 1s to the parallel architectures
of the 1990s to the Cray X1 at ORNL. In many
cases these systems would not have existed
without the Office of Science as a partner
with the vendors. Our current facilities
and test beds include:
· The Center for Computational
Sciences (CCS) at Oak Ridge National Laboratory,
has been testing and evaluating leading edge
computer architectures as tools for science
for over a decade. The latest evaluation
is on a Cray X1 formed the basis for ORNL's
successful proposal to begin developing a
leadership class computing capability for
the U.S. open scientific community. In his
remarks announcing the result of this competition,
Secretary of Energy Spencer Abraham stated,
"This new facility will enable the Office
of Science to deliver world leadership-class
computing for science," and "will serve
to revitalize the U.S. effort in high-end
computing." This supercomputer will be open
to the scientific community for research.
· The National Energy
Research Scientific Computing Center (NERSC)
at Lawrence Berkeley National Laboratory,
which provides leading edge high-performance
computing services to over 2,000 scientists
nationwide. NERSC has a 6,000 processor IBM
SP3 computer with a peak speed of 10 TeraFLOPS.
We have initiated a new program at NERSC,
Innovative and Novel Computational Impact
on Theory and Experiment (INCITE), to
allocate substantial computing resources to
a few, competitively selected, research proposals
from the national scientific community. Last
year, I selected three proposals for INCITE.
One of these has successfully simulated the
explosion of a supernova in 3-D for the first
time.
· The Energy Sciences Network
(ESnet), which links DOE facilities and researchers
to the worldwide research community. ESnet
works closely with other Federal research
networks and with university consortia such
as Internet 2 to provide seamless connections
from DOE to other research communities. This
network must address facilities that produce
millions of gigabytes (petabytes) of data
each year and deliver these data to scientists
across the world.
We have learned important lessons
from these test beds. By sharing our evaluations
with vendors we have enabled them to produce
better products to meet critical scientific
and national security missions. Our spending
complements commercial R&D in IT which
is focused on product development and on the
demands of commercial applications which generally
place different requirements on the hardware
and software than do leading edge scientific
applications.
*
* *
The Office of Science coordinates
with other federal agencies to avoid duplication
of efforts. In the areas where the Office
of Science (DOE-SC) focuses its research --
High End Computing and Large Scale Networking
-- DOE-SC co-chairs the relevant federal coordinating
group. In addition to this mechanism, DOE-SC
has engaged in a number of other joint planning
and coordination efforts.
- DOE-SC participated in the
National Security community planning effort
to develop an Integrated High End Computing
plan.
- DOE-SC and DOD co-chaired
the HECRTF.
- DOE-SC and NSF co-chair the
Federal teams that coordinate the engineering
of Federal research networks and the emerging
GRID Middleware.
- DOE-SC is a partner with
DARPA in the High Productivity Computing
Systems project, which will deliver the
next generation of advanced computer architectures
for critical science and national security
missions through partnerships with U.S.
industry.
- DOE-SC works closely with
NNSA on critical software issues for high
performance computing.
- DOE-SC, DOE-NNSA, DOD-ODDR&E,
DOD-NSA, and DOD-DARPA have developed a
Memorandum of Understanding to jointly plan
our research in high performance computing.
This MOU will enable us to better integrate
our substantial ongoing collaborative projects.
*
* *
High end computing is a key tool
in carrying out Federal agency missions in
science and technology, but the high end computer
market is simply not large enough to divert
computer industry attention from the much
larger and more lucrative commerce and business
computing sector. The federal government
must perform the research and prototype development
on the next generation of tools to meet those
needs. This next generation of computers,
however, might also serve to benefit industry.
Mr. Chairman, high-performance
computing provides a new window for researchers
to understand the natural world with a precision
that could only be imagined a few years ago.
Research investments in advanced scientific
computing will equip researchers with premier
computational tools to advance knowledge and
to help solve the most challenging scientific
problems facing the Nation.
With vital support from this Committee,
the Congress and the Administration, we in
the Office of Science hope to continue to
play an important role in the world of scientific
supercomputing.
Thank you very much.
Appendix:
Office of Science: Who We Are
The Office of Science is the
single largest supporter of basic research
in the physical sciences in the United States,
providing more than 40 percent of total funding
for this vital area of national importance.
It oversees - and is the principal federal
funding agency of - the Nation's research
programs in high-energy physics, nuclear physics,
and fusion energy sciences.
The Office of Science manages
fundamental research programs in basic energy
sciences, biological and environmental sciences,
and computational science. In addition, the
Office of Science is the Federal Government's
largest single source of funds for materials
and chemical sciences, and it supports unique
and vital parts of U.S. research in climate
change, geophysics, genomics, life sciences,
and science education.
The Office of Science manages
this research portfolio through six interdisciplinary
program offices: Advanced Scientific Computing Research,
Basic
Energy Sciences, Biological
and Environmental Research, Fusion
Energy Sciences, and High Energy Physics and
Nuclear
Physics.
The Office of Science also manages
10
world-class laboratories, which often
are called the "crown jewels" of our national
research infrastructure. The national laboratory
system, created over a half-century ago, is
the most comprehensive research system of
its kind in the world. The 10 Office of Science
laboratories are: Ames
Laboratory, Argonne National Laboratory,
Brookhaven National Laboratory, Fermi
National Accelerator Laboratory, Thomas
Jefferson National Accelerator Facility, Lawrence
Berkeley National Laboratory, Oak
Ridge National Laboratory, Pacific Northwest
National Laboratory, Princeton Plasma Physics
Laboratory and the Stanford
Linear Accelerator Center.
The Office of Science oversees
the construction and operation of some of
the Nation's most advanced R&D user
facilities, located at national laboratories
and universities. These include particle and
nuclear physics accelerators, synchrotron
light sources, neutron scattering facilities,
supercomputers and high-speed computer networks.
Each year these facilities are used by more
than 18,000 researchers from universities,
other government agencies and private industry.
The Office of Science is a principal
supporter of graduate students and postdoctoral
researchers early in their careers. About
50 percent of its research funding goes to
support research at 250 colleges, universities,
and institutes nationwide.
For more than half a century,
every President and each Congress has recognized
the vital role of science in sustaining this
Nation's leadership in the world. According
to some estimates, fully half of the growth
in the U.S. economy in the last 50 years stems
from federal funding of scientific and technological
innovation. American taxpayers have received
great value for their investment in the basic
research sponsored by the Office of Science
and other agencies in our government.
Ever since its inception as part
of the Atomic Energy Commission immediately
following World War II, the Office of Science
has blended cutting edge-research and innovative
problem solving to keep the U.S. at the forefront
of scientific discovery. In fact, since the
mid-1940's, the Office of Science has supported
the work of more than 40 Nobel Prize winners,
testimony to the high quality and importance
of the work it underwrites.
Office of Science research investments historically
have yielded a wealth of dividends including:
significant technological innovations; medical
and health advances; new intellectual capital;
enhanced economic competitiveness; and improved
quality of life for the American people.
