HPC Wire

Subscribe to HPC Wire feed
Since 1987 - Covering the Fastest Computers in the World and the People Who Run Them
Updated: 6 hours 8 min ago

SC17 Video Highlights Nobel Prize Winning LIGO Collaboration

Thu, 10/05/2017 - 10:26

To help tell the story of how high performance computing is enabling truly “out of this world” discoveries, SC17 has released a new video that focuses on the role of high performance computers in the Nobel Prize-winning LIGO (Laser Interferometer Gravitational-Wave Observatory) collaboration. The just-announced prize was awarded for the discovery of gravitational waves, originally theorized 100 years ago by Albert Einstein in his general theory of relativity.

LIGO, funded by the National Science Foundation, uses incredibly sophisticated geographically-distributed laser detectors to find the elusive sounds in the universe that prove the existence of gravitational waves.

“We are only now beginning to hear the vibrations of space-time that are all around us—we just needed a better ear.  And when we detect that, we’re detecting the vibrations of everything that has ever moved in the universe. This is real. This is really there, and we’ve never noticed it until now,” said Alan Weinstein Head, Caltech LIGO Laboratory Astrophysics Group and Professor of Physics, Caltech.

Click on the YouTube link below to see the video and please share with your colleagues:

Click the following link for more info on LIGO and the Nobel Prize.

See original article here.

The post SC17 Video Highlights Nobel Prize Winning LIGO Collaboration appeared first on HPCwire.

MareNostrum Supercomputer Allocated 20 Million Hours of Calculation to Nobel Prize-Winning Project

Thu, 10/05/2017 - 10:20

BARCELONA, Oct. 5, 2017 — The MareNostrum Supercomputer, in the Barcelona Supercomputing Center, has allocated 20 million processor hours to make calculations for the project that has been awarded with the Nobel Prize for Physics.

Since 2015, the LiGO project, awarded for having detected gravitational waves as predicted by Einstein, has been using the MareNostrum through Dr Sascha Husa, Theoretical Physics Professor of the University of Illes Balears (Spain) and member of the project.

In this video, Dr Husa explains how he uses MareNostrum for his everyday work: “In my research, I am interested in the study of gravitational wave signals, which are created when two black holes collide; the most violent events in the universe. My particular job is to solve equations and calculate, with very large calculations, what such collisions look like and what exactly are the signals that are given off. Experimentalists in the LiGO collaboration can compare my predictions with the data and find out which systems have created these signals, and for these calculations we have to use very large machines. My home machine is MareNostrum; I log into the machine almost every day and the importance of MareNostrum for our work is very easy to explain: without it we could not do the kind of work we do; we would have to change our direction of research”.

Dr Husa and his team access MareNostrum through different calls which are opened periodically by the European network PRACE (Partnership for Advanced Computing in Europe), to grant access to the most powerful Supercomputers in the EU to European researchers at the service of science. The access to these machines is calculated in “processor hours”, also known as “calculation hours”.

About Barcelona Supercomputing Center

Barcelona Supercomputing Center (BSC) is the national supercomputing centre in Spain. BSC specialises in High Performance Computing (HPC) and its mission is two-fold: to provide infrastructure and supercomputing services to European scientists, and to generate knowledge and technology to transfer to business and society.

BSC is a Severo Ochoa Center of Excellence and a first level hosting member of the European research infrastructure PRACE (Partnership for Advanced Computing in Europe). BSC also manages the Spanish Supercomputing Network (RES).

BSC is a consortium formed by the Ministry of Economy, Industry and Competitiveness of the Spanish Government, the Business and Knowledge Department of the Catalan Government  and the Universitat Politecnica de Catalunya (UPC).

Source: Barcelona Supercomputing Center

The post MareNostrum Supercomputer Allocated 20 Million Hours of Calculation to Nobel Prize-Winning Project appeared first on HPCwire.

New Book Shows How to Leverage Cloud for Science and Engineering

Wed, 10/04/2017 - 10:02

Cloud computing exploded on the scene in the last decade (Amazon Web Services was founded in 2006) shifting the paradigm for how computing is delivered toward a ubiquitous, on-demand model. Cloud’s legacy can be traced to the pioneering work of Ian Foster who along with colleagues Carl Kesselman and Steve Tuecke developed distributed computing approaches that enabled a new era in innovation for data-intensive sciences, such as high-energy physics, environmental science, and biomedicine. The “grid” techniques invented by these collaborators set the stage for IT’s cloudy transformation.

As one of the fathers of grid computing, Foster has brought his knowledge and experience to bear in a new book that addresses the potential for cloud (public and private) to accelerate scientific discovery. Written with co-author Dennis Gannon (Emeritus Professor of Computer Science at Indiana University Bloomington) “Cloud Computing for Science and Engineering” serves as “a guide to cloud computing for students, scientists, and engineers, with advice and many hands-on examples.”

Foster, senior scientist at Argonne National Laboratory and the University of Chicago, provides additional information about the book in a blog post:

Clouds operated by Amazon, Microsoft, Google, and others provide convenient on-demand access to storage and computing. They also provide powerful services for organizing data, processing data streams, machine learning, and many other tasks. Every scientist and engineer needs to understand what these services can and cannot do, and what the emergence of cloud means for their work. This book addresses that need, describing cloud computing and how you may apply it to advantage in science and engineering. It is highly practical, with many hands-on examples of how to use cloud to address specific problems that arise in technical computing. It provides actionable advice on how and when to apply cloud computing in your work.

The book covers cloud services for managing data, and how to program these services; computing in the cloud, from deploying single virtual machines or containers to supporting basic interactive science experiments to gathering clusters of machines to do data analytics; using the cloud as a platform for automating analysis procedures, machine learning, and analyzing streaming data; building your own cloud with open source software; and cloud security. It covers major services provided by the AmazonGoogle, and Microsoft clouds and the research data management capabilities of the Globus cloud service. Two chapters by guest authors cover the Eucalyptus (Rich Wolski) and OpenStack (Stig Telfer) private cloud technologies. See the figure below and the Table of Contents for more information on what the book covers.

The book was positively reviewed by Tony Hey, Chief Data Scientist, Science and Technology Facilities, Rutherford Appleton Lab. He writes, “Cloud computing has changed the corporate world dramatically in a few short years and is now about to play a major role in scientific and engineering applications. This very timely book by Foster and Gannon is both a comprehensive overview and a practical guide to the services offered by the three major public cloud providers, Amazon, Google, and Microsoft. It will be an invaluable resource for scientists and engineers in both industry and academia.”

Cloud Computing for Science and Engineering” was published on Sept. 29, 2017, by MIT Press and is available for purchase at Amazon. The full text (along with associated Jupyter notebooks and other supporting material) is also available online at https://cloud4scieng.org.

The post New Book Shows How to Leverage Cloud for Science and Engineering appeared first on HPCwire.

Bright Computing Announces Tiered Partner Program for Resellers

Wed, 10/04/2017 - 08:44

SAN JOSE, Calif., Oct. 4, 2017 — Bright Computing, a global leader in cluster and cloud infrastructure automation software, today announced that the company has rolled out a tiered partner program for value added reseller partners in North and South America.

The tiered program, launched across EMEA and APAC in January 2017, is designed to recognize and reward loyalty amongst the Bright partner community. The program acknowledges the contribution that the Bright partner community makes to Bright’s business, and provides the tools to improve partner profitability while empowering end customers to build dynamic clustered infrastructures.

The program comprises three participation levels. Partners that invest more in their relationship with Bright will receive higher-value benefits and resources across several categories:

  •  Premier partners who demonstrate significant achievements as a Bright reseller, will be rewarded with a number of exclusive commercial, resource and marketing program benefits.
  •  Advanced partners who have achieved success as a Bright reseller, will be rewarded with a proportional number of program benefits.
  •  All new partners join at the Member tier; this introductory level to the Bright Partner Program enables partners to get up to speed quickly and easily, rewarding them with a number of standard benefits.

Lee Carter, VP Worldwide Alliances at Bright Computing, commented; “So far in 2017, the tiered partner program has proved very successful across EMEA and APAC, which is reflected in the growing impact that our channel partners have on business revenue contribution. As we extend the loyalty program to partners across North and South America, we look forward to seeing increased business activity in the Americas, as partners are rewarded for their commitment to Bright Computing.”

Source: Bright Computing

The post Bright Computing Announces Tiered Partner Program for Resellers appeared first on HPCwire.

NCSA Congratulates Nobel Prize in Physics Laureates

Wed, 10/04/2017 - 08:39

CHAMPAIGN, Ill., Oct. 4, 2017 — The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign, a member of the Laser Interferometer Gravitational-Wave Observatory (LIGO) Scientific Consortium, congratulates the three scientists who were honored for “decisive contributions to the LIGO detector and the observation of gravitational waves” with the 2017 Nobel Prize in Physics by the Royal Swedish Academy of Sciences. Rainer Weiss, Massachusetts Institute of Technology, received one-half of the award and Barry Barish and Kip Thorne, both at the California Institute of Technology, share half.

“All of us at NCSA congratulate the laureates on the recognition of their vision and leadership that led to the historic first gravitational wave detection,” says William “Bill” Gropp, director of NCSA. “It’s exciting to see this recognition of a brilliant experiment that has given us a new way to observe the universe, an experiment that was supported by high-performance computing. NCSA is proud to be a member of the LIGO consortium.”

LIGO is a collaborative project with over one thousand researchers from more than 20 countries. Weiss and Thorne originally proposed LIGO as a means of detecting gravitational waves in the 1980s. Barish is recognized for bringing the project to completion.

The Historic First Gravitational Wave Detection

On Sept. 14, 2015, at 5:51 a.m. Eastern Daylight Time (9:51 UTC) scientists observed for the first-time ripples in the fabric of space-time called gravitational waves, arriving at the earth from a cataclysmic event in the distant universe. This confirmed a major prediction of Albert Einstein’s 1915 general theory of relativity and opens an unprecedented new window onto the cosmos.

Gravitational waves carry information about their dramatic origins and about the nature of gravity that cannot otherwise be obtained. Physicists have concluded that the detected gravitational waves were produced during the final fraction of a second of the merger of two black holes to produce a single, more massive spinning black hole. This collision of two black holes had been predicted but never observed.

The gravitational waves were detected by both of the twin LIGO detectors, located in Livingston, La., and Hanford, Wash. The LIGO Observatories are funded by the National Science Foundation, and were conceived, built, and are operated by Caltech and the Massachusetts Institute of Technology. The discovery, published in the journal Physical Review Letters, was made by the LIGO Scientific Collaboration and the Virgo Collaboration using data from the two LIGO detectors.

Eliu Huerta, a member of the LIGO Scientific Collaboration since 2011 and current leader of the relativity group at NCSA, is a co-author of that paper. “Today’s award is a recognition to a generation of physicists, mathematicians and computer scientists that have promoted gravitational waves as a new observational tool that has revolutionized our understanding of the cosmos. Major scientific discoveries are yet to come,” he says.

NCSA has a long history as a leader in applying supercomputers to black hole and gravitational wave problems and continues to support the most complex problems in numerical relativity and relativistic astrophysics, including working with several groups to simulate gravitational wave sources seen by LIGO in the discovery.

“While most Nobel prizes are awarded for a discovery that happened in the past, this one recognizes the dawn of a new era in physics that will continue for generations of discoveries, says former NCSA Director Ed Seidel, who is also Founder Professor of Physics and professor of astronomy at Illinois. “Our entire community celebrates the achievements of our leaders, friends, and now Nobel laureates Rainer Weiss, Kip Thorne, and Barry Barish, as well as the entire LIGO Scientific Collaboration, who all worked for decades to make this happen.”

Gabrielle Allen, professor of astronomy at Illinois and senior researcher at NCSA, says, “Because of the leadership of the new laureates, today we are seeing multiple gravitational wave events with the 4-km LIGO and VIRGO interferometers.”

Congratulations Rainer Weiss, Barry Barish, and Kip Thorne on your Nobel Prize!

Source: University of Illinois at Urbana-Champaign

The post NCSA Congratulates Nobel Prize in Physics Laureates appeared first on HPCwire.

NCSA Director Featured Twice in DOE’s Top 40 Scientific Milestones of Past 40 Years

Wed, 10/04/2017 - 08:35

CHAMPAIGN, Ill. Oct. 4, 2017 — Two different papers that were coauthored by Dr. William “Bill” Gropp, the Director of the National Center for Supercomputing Applications (NCSA) and the Thomas M. Siebel Chair in the Department of Computer Science at Illinois, have been named in a select group of papers that have “changed the face of science” over the past 40 years. The list, which was put together by the Department of Energy’s Office of Science, celebrates the DOE’s 40th anniversary and features 40 impactful papers from the past 40 years of the DOE to do so.

“A high-performance, portable implementation of the MPI Message Passing Interface” by W. Gropp, E. Lusk, and A. Skjellum, was published in 1996 and outlined MPICH, the first full implementation of the Message Passing Interface (MPI) standard. Using this, programmers are able to develop software that can run on parallel systems of all sizes, from multicore nodes to clusters to the most powerful supercomputers. This flexibility, in turn, assured MPICH’s ability to run on HPC systems both in the present and far into the future. MPICH’s relevance still exists over two decades later, where it has become the most widely used implementation of MPI in the world.

The landmark paper appeared in: Parallel Computing 22, 789-828 (1996).

“Efficient management of parallelism in object-oriented numerical software libraries” by S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith, was published in 1997 and described techniques implemented in the Portable, Extendable Toolkit for Scientific Computation (PETSc) 2.0 package. This was used to simplify the process of programming networks for parallel processing while maintaining high efficiency and portability.  Perhaps one on the most impactful software advancements in the history of scientific computing, PETSc has been used in thousands of models, helping to depict everything from airflow to water movement.

The paper appeared in: Modern Software Tools in Scientific Computing, edited by E. Arge, A. M. Bruaset, and H. P. Langtangen, pp. 163-202 (1997).

Source: University of Illinois at Urbana-Champaign

The post NCSA Director Featured Twice in DOE’s Top 40 Scientific Milestones of Past 40 Years appeared first on HPCwire.

OLCF’s 200 Petaflops Summit Machine Still Slated for 2018 Start-up

Tue, 10/03/2017 - 16:11

The Department of Energy’s planned 200 petaflops Summit computer, which is currently being installed at Oak Ridge Leadership Computing Facility, is on track to be completed in early 2018 according to an OLCF presentation made last week at the HPCXXL meeting. There had been some speculation Summit might be running by SC17. Standing up such massive machines is typically a painstaking process.

OLCF researcher Wayne Joubert told the audience, “We now have all of the cabinets delivered to Oak Ridge and installed. They are empty cabinets – we are still waiting for the processor boards. [Work] on the fiber optic cables is partly done and in progress now. We hope in early 2018 to have the entire system delivered. We’ll start working on it [then], trying out our codes on it and driving to acceptance of the system later in 2018.”

Joubert’s talk (Evaluating functionality and performance of IBM Power8+ systems) highlighted a dedicated OpenPOWER day at the HPCXXL[i] summer meeting held in New York. HPCXXL is a user group for sites which have large installations of IBM or Lenovo equipment. The focus of the HPCXXL group is on large-scale scientific/technical computing using IBM or Lenovo hardware.

Joubert reviewed work being done on the Summitdev machine – a one petaflop ‘prototype’ – in preparation for Summit. IBM (Power CPU) and Nvidia (P100 GPU) are key vendors on the project. Summitdev is an impressive computer on its own (specs below) that outperforms Titan in several areas. Titan was one of the first leadership computers to use heterogeneous (CPU plus GPU) architecture.

OLCF took delivery of Summitdev, a three cabinet 54-node IBM Minsky system, last October. Summitdev is primarily intended for code teams to prepare applications for Summit, a Power9 / Volta system that is among the pre-exascale machines DoE is funding on the path to exascale. Summit is expected to deliver 200 petaflops of double precision performance, and, based on Nvidia announcements, it will provide 3.2 exaflops of mixed precision performance for machine learning applications. “We expect Summit to be the world’s most powerful machine learning system,” said Joubert.


The big difference between Summitdev and Summit – besides sheer size – is use of IBM’s newest processor, the Power9, which has been much anticipated. Summitdev uses IBM Power8+ chips introduced roughly a year ago as the core of IBM’s Minsky PowerAI platform (see HPCwire article, IBM Launches PowerAI Suite Optimized for its Highest Performing Server.) IBM’s high-end server (S822LC for HPC) with Power8+ and NVlink was one of the first systems to ship with Nvidia’s new P100 GPU.

Summitdev was delivered to OLCF last October, underwent the usual testing and was accepted in early December. Since then more work has been done around debugging and evaluating system software and application performance. Joubert’s update covered acceptance work and recent evaluations. Summit, in addition to the new Power9 CPU, will have Nvidia’s new Volta 100 GPU.

OLCF director Arthur “Buddy” Bland declined to say whether OLCF had been working with early Power9 silicon, but said he didn’t think transitioning from the Power8+ to Power9 would be a problem. He also confirmed the Summit rollout schedule.

“Our team does not expect the port from Power8 to Power9 to be difficult. We fully expect that codes will move over with little effort. Of course, as with any system modifications may be needed to take full advantage of the new features. The details of Power9 are still under NDA so I can’t really say any more at this time. As for the dates, we have been saying for a long time that we expect to have the machine installed in 2018 and put early users on in the second half of 2018,” said Bland.

Most Joubert’s talk covered the Summitdev performance testing. The problems encountered, he said were largely typical. There were a few early hardware problems. A wide variety of compilers (XL, PGI, GNU, LLVM) and libraries et. al. were tested. Interestingly, LLVM, perhaps because it is still new, had a couple of glitches.


Application performance was generally encouraging and in line with expectations. The OLCF team not only examined required CORAL apps but also a few other science applications that are heavily used.

You can see from the slide below that Summitdev is already surpassing Titan on select applications such as QMCPACK.

The OLCF update of the Summitdev/Summit project was just one of several IBM-centric presentations made at the OpenPOWER day. Here are links to a few other presentations:

Link to OLCF presentation video: https://www.youtube.com/watch?v=gb9yUI9H2mk

Link to OLCF presentation slides: https://ibm.ent.box.com/v/hpcxxlopenpowerdday2017/file/230502745769

Link to all HPCXXL presentations: https://ibm.ent.box.com/v/hpcxxlopenpowerdday2017

OpenPOWER YouTube channel: https://www.youtube.com/channel/UCYLtbUp0AH0ZAv5mNut1Kcg

[i] HPCXXL is a user group for sites which have large installations of IBM or Lenovo equipment. The focus of the HPCXXL group is on large-scale scientific/technical computing using IBM or Lenovo hardware. Some of the areas we cover are: Applications, Code Development Tools, Communications, Networking, Parallel I/O, Resource Management, System Administration, and Training. We address topics across a wide range of issues that are important to sustained multi-petascale scientific/technical computing on scaleable parallel machines.

The HPCXXL is a self-organized and self-supporting group. Members and affiliates are expected to participate actively in the HPCXXL meetings and activities and to cover their own costs for participating. HPCXXL meetings are open only to members and affiliates of the HPCXXL. HPCXXL member institutions must have an appropriate non-disclosure agreement in place with IBM and Lenovo, since at times both vendors disclose and discusses information of a confidential nature with the group. Beginning in 2018 the spring workshop will be co-located with the HPC Advisory Council Switzerland in Lugano without a vendor dedicated focus. The summer/autum workshop will be traditionally focused on IBM and their partners.

The post OLCF’s 200 Petaflops Summit Machine Still Slated for 2018 Start-up appeared first on HPCwire.

Former Intel CEO Paul S. Otellini Dies at Age 66

Tue, 10/03/2017 - 12:00

SANTA CLARA, Calif., Oct. 3, 2017 – Intel Corporation today announced that the company’s former CEO Paul Otellini passed away in his sleep Monday, Oct. 2, 2017, at the age of 66.

Paul Otellini became Intel’s fifth chief executive officer in 2005. Under his leadership the company made important strategic, technological and financial gains. These included transforming operations and cost structure for long-term growth; assuming a leadership position in the server market segment; and maintaining profitability during the global recession. Other accomplishments included signing on notable new customer engagements, such as winning the Apple PC business, and business partnerships and strategic acquisitions that expanded Intel’s presence in security, software and mobile communications. On the financial front, Intel generated more revenue during his eight-year tenure as CEO than it did during the company’s previous 45 years. In the last full year before he was named CEO, Intel had $34 billion in sales; by 2012, the number had grown to $53 billion.

Intel Corporation announced that Paul Otellini, the company’s former chief operating officer, died Oct. 2, 2017, at the age of 66. (Credit: Intel Corporation)

“We are deeply saddened by Paul’s passing,” Intel CEO Brian Krzanich said. “He was the relentless voice of the customer in a sea of engineers, and he taught us that we only win when we put the customer first.”

Otellini was born in San Francisco on Oct. 12, 1950, and remained a fan of the city all his life. He received a bachelor’s degree in economics from the University of San Francisco in 1972 and an MBA from the University of California, Berkeley in 1974. He joined Intel in 1974 and served in a number of positions, including general manager of Intel’s Peripheral Components Operation and the Folsom Microcomputer Division, and in 1989 as then-CEO Andy Grove’s chief of staff.

From 1990 to 2002, he held various positions at Intel, including executive vice president and general manager of the Intel Architecture Group, responsible for the company’s microprocessor and chipset businesses and strategies for desktop, mobile and enterprise computing, as well as executive vice president and general manager of the Sales and Marketing Group. Otellini also served as chief operating officer from 2002 to 2005.

“Paul’s business acumen, optimism and dedication fueled our growth throughout his tenure as CEO,” Intel Chairman Andy Bryant said. “His tireless drive, discipline and humility were cornerstones of his leadership and live on in our company values to this day.”

Paul and his wife, Sandy, were married for 30 years. He is survived by his wife; his son, Patrick; and his daughter, Alexis.  Since he retired in 2013, Otellini dedicated time to mentoring young people and being involved with several philanthropic and charitable organizations, including the San Francisco Symphony and San Francisco General Hospital Foundation.

Source: Intel

The post Former Intel CEO Paul S. Otellini Dies at Age 66 appeared first on HPCwire.

New ‘Upcycled’ HPC Machine at Durham University Helps Space Science

Tue, 10/03/2017 - 11:48

Oct. 3, 2017 — Researchers specialising in astrophysics and cosmology, particle physics and nuclear physics at Durham University and from across the UK can now take advantage of an extended HPC service. The DiRAC  Data Centric  HPC system installed at Durham University has been enhanced by the deployment of COSMA6, a machine with 8,000 Intel Sandy Bridge cores and 4.3 petabytes of storage ‘upcycled’ from another system previously located at the Hartree Centre in Daresbury. This additional resource was needed to maintain the international competitiveness of the research community served by DiRAC for the next 12 months.  In recognition of this, DiRAC was able to secure funding from the Science and Technology Facilities Council [STFC] to make use of the generous gift from Hartree and transfer and re-install the system at Durham University.

COSMA6 enables researchers to extend large-scale structure simulations of the evolution of the universe, analyse data from gravitational wave detectors, and simulate the Sun and planets in the solar system. COSMA6 is live and operational from April 2017.

The University’s COSMA6 service, combined with its existing COSMA5 service, now contributes over 14,000 cores to the national computing facility, DiRAC 2.5. DiRAC is a facility for theoretical modelling and HPC-based research in particle physics, nuclear physics, astronomy and cosmology, areas in which the UK is world-leading. Since its launch in 2010 DiRAC has received £29 million in funding from the Government.

The upcycled cores of COSMA6 added to the DiRAC Data Centric system installed at Durham University began life as a HPC machine at The Hartree Centre, Daresbury in 2012,  but, as a consequence of  upgrades, the Centre no longer had space to house the machine in its data centre. The staff of the Institute for Computational Cosmology (ICC) at Durham University worked with HPC, storage and data analytics integrator OCF, and the specialist server relocation and data centre migration specialist Technimove to dismantle, transport, and rebuild the HPC machine at its new home.

“We had a very similar setup to The Hartree Centre; we had an IBM machine as did they; both of our older systems arrived in the UK within a day of each other in April 2012; we have a new machine room in Durham so we had the space, cooling and power and we also needed the cores to help boost our contribution to the national DiRAC system,” explains Lydia Heck, Technical Director at the Institute for Computational Cosmology (ICC) based at Durham University.

The rebuilding of the machine and its integration into the current DiRAC systems was an excellent example of collaboration between the customer and the IT partners. As half of the storage runs on DDN’s own Lustre parallel filesystem, Exascaler, and the other half on Intel’s Enterprise Lustre, the storage elements had to be installed in three racks that were re-cabled by OCF (and supported by DDN), and three re-cabled, and supported by the University. This complex integration of the different storage elements has been successful and the users see a seamless storage service.

Heck continues: “While it was quite an effort to bring it to its current state, as it is the same architecture and the same network layout as our previous system, we expect this to run very well.

“The new HPC system at Durham University  is a testament to the skills of all involved in the project who were able to re-install a second hand cluster, add to it new RAM memory and design a solution that will prove invaluable to the research community”, comments Julian Fielden, Managing Director of OCF. “We have a long history of working with Durham University so we’re really pleased to have been involved in such a unique project.”

Source: Durham University

The post New ‘Upcycled’ HPC Machine at Durham University Helps Space Science appeared first on HPCwire.

Mellanox Technologies Schedules Release of 3rd Quarter Financial Results and Conference Call

Tue, 10/03/2017 - 09:45

SUNNYVALE, Calif. & YOKNEAM, Israel, Oct. 3, 2017 — Mellanox Technologies, Ltd. (NASDAQ: MLNX), a leading supplier of high-performance, end-to-end interconnect solutions for data center servers and storage systems, today announced that it will release its financial results for the third quarter 2017 after the market closes on Wednesday, October 25, 2017.

Following the release, Mellanox will conduct a conference call at 2 p.m. Pacific Time (5 p.m. Eastern Time). To listen to the call dial: +1-866-831-8713 (non-U.S. residents: +1-203-518-9713) approximately ten minutes prior to the start time.

The Mellanox financial results conference call will be available, via a live webcast, on the investor relations section of the Mellanox website at: http://ir.mellanox.com.

Interested parties may access the website 15 minutes prior to the start of the call to download and install any necessary audio software. An archived webcast replay will also be available on the Mellanox website.

About Mellanox

Mellanox Technologies (NASDAQ: MLNX) is a leading supplier of end-to-end Ethernet and InfiniBand intelligent interconnect solutions and services for servers, storage, and hyper-converged infrastructure. Mellanox’s intelligent interconnect solutions increase data center efficiency by providing the highest throughput and lowest latency, delivering data faster to applications and unlocking system performance. Mellanox offers a choice of high performance solutions: network and multicore processors, network adapters, switches, cables, software and silicon, that accelerate application runtime and maximize business results for a wide range of markets including high performance computing, enterprise data centers, Web 2.0, cloud, storage, network security, telecom and financial services. More information is available at www.mellanox.com.

Source: Mellanox Technologies

The post Mellanox Technologies Schedules Release of 3rd Quarter Financial Results and Conference Call appeared first on HPCwire.

Galactos Project Solves One of Cosmology’s Hardest Challenges

Mon, 10/02/2017 - 16:13

The nature of dark energy and the complete theory of gravity are two of the central questions currently facing cosmologists. As the universe evolved, the expansion following the Big Bang initially slowed down due to gravity’s powerful inward pull. Presently, dark energy–a mysterious substance that seems to be associated with the vacuum of space itself–is pushing the universe outwards more strongly than gravity pulls in, causing the universe to not only expand but to do so faster and faster.

While dark energy constitutes 72 percent of the universe’s current energy density, its fundamental nature remains unknown. Much of today’s scientific work is trying to understand the interplay between gravity and dark energy in an effort to understand the current state of the universe.

There is an open problem in astronomy and cosmology in computing the anisotropic (direction dependent) and isotropic (direction-averaged) 3-point correlation (3CPF) function which provides information on the structure of the universe. According to Prabhat, Big Data Center (BDC) Director and Group Lead for the Data and Analytics and Services team at Lawrence Berkeley National Laboratory’s (Berkeley Lab) National Energy Research Scientific Computing Center (NERSC), “Cosmologists and Astronomers have wanted to perform the 3-point computation for a long time but could not do so because they did not have access to scalable methods and highly optimized calculations that they could apply to datasets.”

A project called Galactos has made a major breakthrough in successfully running the 3-point correlation calculation on Outer Rim, the largest known galaxy dataset that contains information for two billion galaxies. The Galactos project is part of the Big Data Center collaboration between NERSC, Berkeley Lab and Intel.

“Essentially, we performed the entire 3CPF computation on two billion galaxies in less than 20 minutes and have solved the problem of how to compute the 3-point correlation function for the next decade. This work would not be possible without the use of high performance computer (HPC) systems, efficient algorithms, specialized software and optimizations,” states Prabhat.

Overview of the Galactos Project

According to Debbie Bard, Big Data Architect, Berkeley Lab who will be hosting a poster session on this topic at the Intel HPC Developers Conference just prior to SC17, “The statistics that our team uses to characterize the structure of matter in the universe are correlation functions. Our calculations provide information on how matter is clustered as well as insights about the nature of gravity and dark energy. The 2-point correlation function (2PCF) looks at pairs of galaxies and the distribution of galaxy pairs. The 3-point correlation (3PCF) calculation, which looks at triplets, provides more detail about the structure of the universe, because you’ve added an extra dimension. 3PCF is rarely studied because it is very hard to calculate and it is computationally intensive. We felt if we could solve this problem from an algorithmic and computational point of view, then we would enable scientists to access the extra information about the structure of the universe.”

Figure 1: Over time, the attractive force of gravity and the expansive force of dark energy create a web-like structure of matter in the universe. Courtesy of Lawrence Berkeley National Laboratory. Specifications of Cori HPC System

The Galactos code ran on the NERSC Cori system at Lawrence Berkeley National Laboratory. Cori is a Cray XC40 system featuring 2,388 nodes of Intel Xeon Processor E5-2698 v3 (named Haswell) and 9,688 nodes (recently expanded from 9,304) of Intel Xeon Phi Processor 7250 (named Knights Landing). The team performed all computations on Intel Xeon Phi nodes. Each of these nodes contains 68 cores (each supporting 4 simultaneous hardware threads), 16 GB of on-package, multi-channel DRAM (“MCDRAM”), and 96 GB of DDR4-2400 DRAM. Cores are connected in a 2D mesh network with 2 cores per tile, and 1 MB cache-coherent L2 cache per tile. Each core has 32 KB instruction and 32 KB data in L1 cache. The nodes are connected via the Cray Aries interconnect.

Process used in the 3-point Computation

Figure 2 shows the image of a simulated miniature universe containing 225,000 galaxies in a box, and provides information on how galaxies are grouped together in a structured way rather than randomly distributed. The Galactos computation process involves three major steps. Around a selected primary galaxy, the algorithm first gathers all galaxy neighbors (secondaries) within a maximum distance Rmax and bins them into spherical shells. It then rotates all coordinates so that the line of sight to the primary from an observer is along the z-axis, and transfers all of the secondaries’ separation vectors from the primary to that frame. Then the algorithm expands the angular dependence of the galaxies within each bin into spherical harmonics, a particular set of mathematical functions. This expansion is represented by the shading in the Expand portion of the graphic.

Figure 2: Simulation of process used in the Galactos project. Courtesy of Lawrence Berkeley National Laboratory.

The Galactos algorithm is parallelized across nodes by taking all 2 billion galaxies in the dataset and breaking them into smaller boxes using the highly efficient k-d tree algorithm developed by Intel. There is also a halo exchange component to expand a fixed box by 200 megaparsecs (200 Mpc = 300 million light years) on each face and pull in all of the galaxies that reside within the extended region. Each node has all the galaxies it needs to determine the full 3PCF and need not communicate with any other nodes until the very end.

O(N2) Algorithm Speeds 3CPF Computation

Galactos used a highly scalable O(N2) algorithm originally created by Zachary Slepian, Einstein Fellow at Berkeley Lab, in conjunction with Daniel Eisenstein, Slepian’s PhD advisor and professor of astronomy at Harvard University. In addition, the team used optimized k-d tree libraries to perform the galaxy spatial partitioning. Brian Friesen (Berkeley Lab, HPC Consultant) and Intel worked on optimizing the code to run across all 9,636 nodes of the NERSC Cori supercomputer.

According to Slepian, “Counting all possible triangles formed by a set of objects is a combinatorially explosive challenge: there are an enormous number of possible triangles. For N objects, there are N options for the first choice, N-1 for the second, and N-2 for the third, leading to N(N-1)(N-2) triangles. If the number of objects is very large, this is roughly N^3.

The key advance of our O(N2) algorithm is to reorder the counting process to reduce the scaling to N2. In practice, this means a speed-up of 500X or more over a naive, ‘just-counting’ approach.

The algorithm exploits the fact that, in cosmology, we want our result to be binned in triangle side length. For example, I might report a result for triangles with the first side between 30 and 60 million light years and second side between 90 and 120 million light years. Our algorithm manages to do this binning first, so one never has to compare combinations of three galaxies, but rather, one compares combinations of bins. There are many fewer bins than galaxies, so this is an enormous computational savings.

The algorithm does this is by writing the problem using a particular set of mathematical functions, known as a ‘basis’, that is ideal for the problem. Our basis has the same symmetries as galaxy clustering and can compactly represent the information the clustering contains. Further, this set of functions, called Legendre polynomials, can be split into spherical harmonic factors.”

Optimizations and Vectorization used in the Galactos Project

According to Friesen, “The Galactos optimization consisted of two components including single node and multi-node scaling. The multi-node scaling uses a k-d tree algorithm, which is a multi-node k-d tree with Message Passing Interface (MPI) built in. k-d trees are used to partition a data set so that data elements that are physically near each other are close to each other in memory. In Galactos, the k-d tree helps improves performance when determining which galaxy neighbors are nearby.

“The k-d tree is also important for computational load balance between nodes on the system,” Friesen adds. “The bulk of the computation in Galactos occurs within a node, so there is very little communication between nodes. If there are large load imbalances between nodes, then the algorithm only calculates as fast as the slowest node. The team worked to make the computational load as similar as possible between nodes to increase the speed of the algorithm.”

Enabling vectorization on the Intel Xeon Phi processor required sorting the galaxies, such that pairs of galaxies separated by similar distances were adjacent in memory. This enabled the algorithm to compute the geometric properties of many galaxy pairs simultaneously using vectorization, rather than computing the properties of each galaxy pair individually.

For the Galactos project, Intel optimized performance within a single Intel Xeon Phi node and across the Cray XC40 supercomputer. Intel was earlier involved in computing the 2-point correlation function with Berkeley Lab [1]. “Our optimizations to Galactos included (1) a distributed k-d tree algorithm for partitioning the galaxies and enabling fast computation of nearest neighbors to any galaxy and (2) computation of spherical harmonics around each galaxy locally. Step 2 is the biggest computational bottleneck and was vectorized over the neighbors of a given galaxy with multiple galaxies running in parallel on different threads. We used Intel developer tools optimized for Intel Xeon Phi processors,” states Narayanan Sundaram, Intel Research Scientist. In all Galactos computations, code is compiled using the Intel C++ v17.0.1 compiler with Cray MPI. The team ran the code with one MPI process per Intel Xeon Phi compute node, using 272 threads per node (four threads per physical core).

Galactos Time to Solution Test Results for the Outer Rim Dataset

Galactos testing included performance breakdown of the code running the Outer Rim dataset with 225,000 galaxies on a single node. Its single-node performance has been highly optimized for Intel Xeon Phi processors, reaching 39 percent of peak, with efficient use of vectorization and the full memory hierarchy. Galactos achieves almost perfect weak and strong scaling, and achieves a sustained 5.06 PF across 9636 nodes.

Figure 3: Weak scaling of Galactos code on Cori, using the Outer Rim datasets. Courtesy of Lawrence Berkeley National Laboratory. Figure 4: Strong scaling of Galactos code on Cori, using the Outer Rim datasets. Courtesy of Lawrence Berkeley National Laboratory.

The team ran Galactos over 9,636 available nodes of the Cori system in both mixed and double precision. (In mixed precision, the k-d tree is computed in single precision and everything else is in double precision.) The time to solution to compute the 3PCF for 2 billion galaxies in mixed precision is 982.4 sec (16.37 minutes); in pure double precision, the time to solution is 1070.6 sec (17.84 minutes).

Galactos Aids in Future Cosmology Research

Prabhat states, “As computer scientists, a lot of our achievements are surpassed in a few years because our field changes rapidly. The Galactos project has enabled a previously intractable computation to run on the Cori supercomputer in 20 minutes. When the LSST comes online, the code will be able to process massive datasets in a day or two. This project has been particularly satisfying for our team, because we have not only solved the 3-pt correlation problem for the largest dataset available in 2017, but for the next decade in astronomy. How often do you get to make that claim?”


[1] Galactos: Computing the Anisotropic 3-Point Correlation Function for 2 Billion Galaxies

Brian Friesen, Md. Mostofa Ali Patwary, Brian Austin, Nadathur Satish, Zachary Slepian, Narayanan Sundaram, Deborah Bard, Daniel J Eisenstein, Jack Deslippe, Pradeep Dubey, Prabhat, Cornell University Library, 31 Aug 2017
hyperlink: https://arxiv.org/abs/1709.00086

[2] J. Chhugani et al., “Billion-particle SIMD-friendly two-point correlation on large-scale HPC cluster systems,” High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, Salt Lake City, UT, 2012, pp. 1-11.

[3] Z. Slepian & D.J. Eisenstein, Computing the three-point correlation function of galaxies in O(N^2) time, Monthly Notices of the Royal Astronomical Society, Volume 454, Issue 4, p.4142-4158
hyperlink: https://arxiv.org/abs/1506.02040

About the Author

Linda Barney is the founder and owner of Barney and Associates, a technical/marketing writing, training and web design firm in Beaverton, OR.

The post Galactos Project Solves One of Cosmology’s Hardest Challenges appeared first on HPCwire.

Exascale Computing to Help Accelerate Drive for Clean Fusion Energy

Mon, 10/02/2017 - 13:10

For decades, scientists have struggled to create a clean, unlimited energy source here on Earth by recreating the conditions that drive our sun. Called a fusion reactor, the mechanism would use powerful magnetic fields to confine and compress gases four times as hot as our sun. By using the magnetic fields to squeeze the gases, the atoms would fuse and release more energy than was used to power the reactor. But to date, that has only worked in theory.

Achieving fusion energy production would benefit society by providing a power source that is non-polluting, renewable and using fuels such as the hydrogen isotopes found in seawater and boron isotopes found in minerals.

Early fusion research projects in the 1950s and ‘60s relied on building expensive magnetic devices, testing them and then building new ones and repeating the cycle. In the mid-1970s, fusion scientists began using powerful computers to simulate how the hot gases, called plasmas, would be heated, squeezed and fused to produce energy. It’s an extremely complex and difficult problem, one that some fusion researchers have likened to holding gelatin together with rubber bands.

Using supercomputers to model and simulate plasma behavior, scientists have made great strides toward building a working reactor. The next generation of supercomputers on the horizon, known as exascale systems, will bring the promise of fusion energy closer.

The best-known fusion reactor design is called a tokamak, in which a donut-shaped chamber is used to contain the hot gases, inside. Because the reactors are so expensive, only small-scale ones have been built. ITER, an international effort to build the largest-ever tokamak-in the south of France. The project, conceived in 1985, is now scheduled to have its first plasma experiments in 2025 and begin fusion experiments in 2035. The estimated cost is 14 billion euros, with the European Union and six other nations footing the bill.

Historically, fusion research around the world has been funded by governments due to the high cost and long-range nature of the work.

But in the Orange County foothills of Southern California, a private company is also pursuing fusion energy, but taking a far different path than that of ITER and other tokamaks. Tri Alpha Energy’s cylindrical reactor design is completely different in its design philosophy, geometry, fuels and method of heating the plasma, all built with a different funding model. Chief Science Officer Toshiki Tajima says their approach makes them mavericks in the fusion community.

But the one thing both ITER and similar projects and Tri Alpha Energy have consistently relied on is using high-performance computers to simulate conditions inside the reactor as they seek to overcome the challenges inherent in designing, building and operating a machine that will replicate the processes of the sun here on Earth.

As each generation of supercomputers has come online, fusion scientists have been able to study plasma conditions in greater detail, helping them understand how the plasma will behave, how it may lose energy and disrupt the reactions, and what can be done to create and maintain fusion. With exascale supercomputers that are 50 times more powerful than today’s top systems looming on the horizon, Tri Alpha Energy sees great possibilities in accelerating the development of their reactor design. Tajima is one of 18 members of the industry advisory council for the U.S. Department of Energy’s (DOE) Exascale Computing Project (ECP).

“We’re very excited by the promise of exascale computing – we are currently fund-raising for our next-generation machine, but we can build a simulated reactor using a very powerful computer, and for this we would certainly need exascale,” Tajima said. “This would help us accurately predict if our idea would work, and if it works as predicted, our investors would be encouraged to support construction of the real thing.”

The Tri Alpha Energy fusion model builds on the experience and expertise of Tajima and his longtime mentor, the late Norman Rostoker, a professor of physics at the University of California, Irvine (UCI). Tajima first met Rostoker as a graduate student, leaving Japan to study at Irvine in 1973. In addition to his work with TAE, Tajima holds the Norman Rostoker Chair in Applied Physics at UCI. In 1998, Rostoker co-founded TAE, which Tajima joined in 2011.

In it for the long run

It was also in the mid-1970s, that the U.S. Atomic Energy Commission, the forerunner of DOE, created a computing center to support magnetic fusion energy research, first with a cast-off computer from classified defense programs, but then with a series of ever-more capable supercomputers. From the outset, Tajima was an active user, and still remembers he was User No. 1100 at the Magnetic Fusion Energy Computer Center. The Control Data Corp. and Cray supercomputers were a big leap ahead of the IBM 360 he had been using.

“The behavior of plasma could not easily be predicted with computation back then and it was very hard to make any progress,” Tajima said. “I was one of the very early birds to foul up the machines. When the Cray-1 arrived, it was marvelous and I fell in love with it.”

At the time, the tokamak was seen as the hot design and most people in the field gravitated in this direction, Tajima said, and he followed. But after learning about plasma-driven accelerators under Professor Rostoker, in 1976 he went to UCLA to work with Prof. John Dawson. “He and I shared a vision of new accelerators and we began using large-scale computation in 1975, an area in which I wanted to learn more from him,” Tajima said.

As a result, the two men wrote a paper entitled “Laser Electron Accelerator,” which appeared in Physical Review Letters in 1979. The seminal paper explained how firing an intense electromagnetic pulse (or beam of particles) into a plasma can create a wake in the plasma and that electrons, and perhaps ions, trapped in this wake can be accelerated to very high energies.

TAE’s philosophy, built on Rostoker’s ideas, is to combine both accelerator and fusion plasma research. In a tokamak, the deuterium-tritium fuel needs to be heated and confined at an energy level of 10,000 eV (electron volts) for fusion to occur. The TAE reactor, however, needs to be 30 times hotter. In a tokamak, the same magnetic fields that confine the plasma also heat it to 3 billion degrees C. In the TAE machine, the energy will be injected using a particle accelerator. “A 100,000 eV beam is nothing for an accelerator,” Tajima said, pointing to the 1G eV BELLA device at DOE’s Lawrence Berkeley National Laboratory. “Using a beam-driven plasma is relatively easy but it may be counterintuitive that you can get higher energy with more stability — the more energetic the wake is, the more stable it becomes.”

But this approach is not without risk. With the tokamak, the magnetic fields protect the plasma, much like the exoskeleton of a beetle protects the insect’s innards, Tajima said. But the accelerator beam creates a kind of spine, which creates the plasma by its weak magnetic fields, a condition known as Reverse Field Configuration. One of Rostoker’s concerns was that the plasma would be too vulnerable to other forces in the early stages of its formation. However, in the 40-centimeter diameter cylindrical reactor, the beam forms a ring like a bicycle tire, and like a bicycle, the stability increases the faster the wheels spin.

“The stronger the beam is, the more stable the plasma becomes,” Tajima said. “This was the riskiest problem for us to solve, but in early 2000 we showed the plasma could survive and this reassured our investors. We call this approach of tackling the hardest problem first ‘fail fast’.”

Another advantage of TAE’s approach is that the main fuel, Boron-11, does not produce neutrons as a by-product; instead it produces three alpha particles, which is the basis of the company’s name. A tokamak, using hydrogen-isotope fuels, generates neutrons, which can penetrate and damage materials, including the superconducting magnets that confine the tokamak plasma. To prevent this, the tokamak reactor requires one-meter-thick shielding. Without the need to contain neutrons, the TAE reactor does not need heavy shielding. This also helps reduce construction costs.

Computation Critical to Future Progress

With his 40 years of experience using HPC to advance fusion energy, Tajima offers a long-term perspective, from the past decades to exascale systems in the early 2020s. As a principal investigator on the Numerical Tokamak project in the early 1990s, he has helped build much of the HPC ecosystem for fusion research.

At the early stage of modeling fusion behavior, the codes focus on the global plasma at very fast time scales. These codes, known as MHD codes (magnetohydrodynamics), are not as computationally “expensive,” meaning they do not require as many computing resources, and at TAE were run on in-house clusters.

The next step is to model the more minute part of the plasma instability, known as kinetic instability, which requires more sophisticated codes that can simulate the plasma in greater detail over longer time scales. Achieving this requires more sophisticated systems. Around 2008-09, TAE stabilized this stage of the problem using its own computing system and by working with university collaborators who have access to federally funded supercomputing centers, such as those supported by DOE. “Our computing became more demanding during this time,” Tajima said.

The third step, which TAE is now tackling, is to make a plasma that can “live” longer, which is known as the transport issue in the fusion community. “This is a very, very difficult problem and consumes large amounts of computing resources as it encompasses a different element of the plasma,” Tajima said, “and the plasma becomes much more complex.”

The problem involves three distinct functions:

  • The core of the field reverse configuration, which is where the plasma is at the highest temperature
  • The “scrape-off layer,” which is the protective outer layer of ash on the core and which Tajima likens to an onion’s skin
  • The “ash cans,” or diverters, that are at each end of the reactor. They remove the ash, or impurities, from the scrape-off layer, which can make the plasma muddy and cause it to behave improperly.

“The problem is that the three elements behave very, very differently in both the plasma physics as well as in other properties,” Tajima said. “For example, the diverters are facing the metallic walls so you have to understand the interaction of the cold plate metals and the out-rushing impurities. And those dynamics are totally different than the core which is very high temperature and very high energy and spinning around like a bicycle tire, and the scrape-off layer.”

These factors are all coupled to each other using very complex geometries and in order to see if the TAE approach is feasible, researchers need to simulate the entirety of the reactor in order to understand and eventually control the reactions.

“We will run a three-layered simulation of our fusion reactor on the computer, with the huge particle code, the transport code and the neural net on the simulation – that’s our vision and we will certainly need an exascale machine to do this,” Tajima said. “This will allow us to predict if our concept works or not in advance of building machine so that our investors’ funds are not wasted.”

The overall code will have three components. At the basic level will be a representative simulation of particles in each part of the plasma. The second layer will be the more abstract transport code, which tracks heat moving in and out of the plasma. But even on exascale systems, the transport code will not be able to run fast enough to keep up with real-time changes in the plasma. Instabilities which affect the heat transport in the plasma come and go in milliseconds.

“So, we need a third layer that will be an artificial neural net, which will be able to react in microseconds, which is a bit similar to a driverless auto, and will ‘learn’ how to control the bicycle tire-shaped plasma, Tajima said. This application will be run on top of transport code and it will observe experimental data and react appropriately to keep the simulation running.

“Doing this will certainly require exascale computing,” Tajima said. “Without it we will take up to 30 years to finish – and our investors cannot wait that long. This project has been independent of the government funding, so that our investors’ fund provided an independent, totally different path toward fusion. This could amount to a means of national security to provide an alternative solution to a problem as large as fusion energy. Society will also benefit from a clean source of energy and our exascale-driven reactor march will be a very good thing for the nation and the world.”

Advanced Accelerators are Pivotal

Both particle accelerators and fusion energy are technologies important to the nation’s scientific leadership, with research funded over many decades by the Department of Energy and its predecessor agencies.

Not only are particle accelerators a vital part of the DOE-supported infrastructure of discovery science and university research, they also have private-sector applications and a broad range of benefits to industry, security, energy, the environment and medicine.

Since Toshiki Tajima and John Dawson published their paper “Laser Electron Accelerator” in 1979, the idea of building smaller accelerators, with the length measure in meters instead of kilometers, has gained traction. In these new accelerators, particles “surf” in the plasma wake of injected particles, reaching very high energy levels in very short distances.

According to Jean-Luc Vay, a researcher at DOE’s Lawrence Berkeley National Laboratory, taking full advantage of accelerators’ societal benefits, game-changing improvements in the size and cost of accelerators are needed. Plasma-based particle accelerators stand apart in their potential for these improvements, according to Vay, and turning this from a promising technology into a mainstream scientific tool depends critically on high-performance, high-fidelity modeling of complex processes that develop over a wide range of space and time scales.

To help achieve this goal, Vay is leading a project called Exascale Modeling of Advanced Particle Accelerators as part of DOE’s Exascale Computing Project. This project supports the practical economic design of smaller, less-expensive plasma-based accelerators.

As Tri Alpha Energy pursues its goal of using a particle accelerator (though this accelerator is not related to wakefield accelerators) to achieve fusion energy, the company is also planning to apply its experience and expertise in accelerator research for medical applications. Not only will this effort produce returns for the company’s investors, but it should also help advance TAE’s understanding of accelerators and using them to create a fusion reactor.

The post Exascale Computing to Help Accelerate Drive for Clean Fusion Energy appeared first on HPCwire.

Engility to Provide NOAA with HPC Expertise

Mon, 10/02/2017 - 11:40

CHANTILLY, Va., Oct. 2, 2017 — Engility Holdings, Inc. (NYSE:EGL) secured $14 million in task order awards from NOAA’s Geophysical Fluid Dynamics Laboratory. Engility scientists will conduct HPC software development and optimization, help users gain scientific insights, and maintain cyber security controls on NOAA’s R&D High Performance Computing System. These services assist NOAA GFDL in enhancing and advancing their HPC capability to explore and understand climate and weather.

“As we saw with Hurricanes Harvey and Irma, a deeper understanding of climate and weather are critical to America’s preparedness, infrastructure and security stance,” said Lynn Dugle, CEO of Engility. “Engility has been at the forefront of leveraging HPC to advance scientific discovery and solve the toughest engineering problems. HPC is, and will continue to be, an area of high interest and value among our customers as they seek to analyze huge and ever-expanding data sets.”

Under the two task orders, the Engility team will also provide data analytics, web development, visualization and network support across the entire GFDL organization. The time and materials awards were both made in the third quarter of 2017 and each have a base year with one option year.

Engility is a leader in HPC capabilities, providing solutions that increase system performance and improve workforce capacity to enable scientific advancements. In addition to the weather and climate modeling work for NOAA, the company works with DOD researchers to offer strategic insights and trusted advice through activities such as algorithm development and implementation, code porting and development, and specialized training. For the Food and Drug administration, Engility builds fast, reliable and adaptable HPC architectures that accommodate analyses of complex data sets, such as rapid genomics analyses.

For more information about Engility, please visit www.engilitycorp.com.

About Engility

Engility (NYSE: EGL) is engineered to make a difference. Built on six decades of heritage, Engility is a leading provider of integrated solutions and services, supporting U.S. government customers in the defense, federal civilian, intelligence and space communities. Our innovative, highly technical solutions and engineering capabilities address diverse client missions. We draw upon our team’s intimate understanding of customer needs, deep domain expertise and technical skills to help solve our nation’s toughest challenges. Headquartered in Chantilly, Virginia, and with offices around the world, Engility’s array of specialized technical service offerings include high-performance computing, cybersecurity, enterprise modernization and systems engineering. To learn more about Engility, please visit www.engilitycorp.com and connect with us on Facebook, LinkedIn and Twitter.

Source: Engility

The post Engility to Provide NOAA with HPC Expertise appeared first on HPCwire.

Four Los Alamos Scientists Named 2017 Laboratory Fellows

Mon, 10/02/2017 - 11:22

LOS ALAMOS, N.M., Oct. 2, 2017 — Four Los Alamos National Laboratory scientists have been named 2017 Fellows: Donald Burton, Stephen Doorn, Manvendra Dubey, and Turab Lookman.

“Becoming a Los Alamos National Laboratory Fellow is one of the highest accomplishments in the Lab,” said Laboratory Director Charles McMillan.  “Each of these scientists has demonstrated sustained high-level achievement in programs of importance to the Laboratory and are recognized authorities in their fields. They have made significant contributions to both Los Alamos and the broader scientific community.”

Donald Burton of the Computational Physics division at the Laboratory is the inventor of computational methods that have become standards in the field and are used all over the world daily in hydrodynamic computations. His codes have been central to the Advanced Simulation and Computing (ASC) program since its inception and have enormously impacted both the nation’s nuclear stockpile stewardship program and the broader scientific community. Burton is the leading inventor for the conservative Lagrangian methods in shock wave compression of condensed matter, has written more than 200 papers and reports, and has served as a mentor to numerous students and postdoctoral researchers.

Stephen Doorn of the Laboratory’s Center for Integrated Nanotechnologies is a world leader in the field of carbon nanotube spectroscopy, establishing the first spectroscopic structure assignments that are universally used today and, more recently, pioneering the development of doped carbon nanotubes as tunable and bright quantum emitters in the near-infrared. In addition to authoring or co-authoring 130 publications with more than 6,500 citations, Doorn has also made important contributions to the leadership of nanoscience at Los Alamos and played a critical role as a mentor to young scientists.

Manvendra Dubeyof the Laboratory’s Earth and Environmental Sciences division is internationally recognized for his high-level strategic involvement in climate research that has moved the issue to center stage for Department of Energy program offices and the national laboratory system. The hallmark of Dubey’s work is excellence in conception, execution, analysis, and synthesis; his work has changed the science community’s understanding of aerosol impacts on planetary temperatures. Additionally, Dubey’s work on methane emissions in the Four Corners area has led to new Laboratory programs and highlighted the need for research on methane impacts to the environment.

Turab Lookman of the Laboratory’s Theoretical Division is an expert in the computational physics of materials, complex fluids, and nonlinear dynamics. His recent work on materials design and informatics applies data science to the discovery of materials with new, beneficial properties. Lookman’s work in this field has received enormous worldwide attention. He is co-author of two books and more than 250 publications. Lookman is also the recipient of the 2009 Los Alamos National Laboratory’s Fellows Prize for Outstanding Research and the 2016 Distinguished Postdoctoral Mentor Award. He is a fellow of the American Physical Society.

About Los Alamos National Laboratory

Los Alamos National Laboratory, a multidisciplinary research institution engaged in strategic science on behalf of national security, is operated by Los Alamos National Security, LLC, a team composed of Bechtel National, the University of California, BWXT Government Group, and URS, an AECOM company, for the Department of Energy’s National Nuclear Security Administration.

Los Alamos enhances national security by ensuring the safety and reliability of the U.S. nuclear stockpile, developing technologies to reduce threats from weapons of mass destruction, and solving problems related to energy, environment, infrastructure, health, and global security concerns.

Source: Los Alamos National Laboratory

The post Four Los Alamos Scientists Named 2017 Laboratory Fellows appeared first on HPCwire.

ECP Co-Design Center Achieves Orders of Magnitude Speed-Up in Latest Software Release

Mon, 10/02/2017 - 11:12

Oct. 2, 2017 — Just one year after the U.S. Department of Energy (DOE) began funding projects to prepare scientific applications for exascale supercomputers, the Block-Structured Adaptive Mesh Refinement Co-Design Center has released a new version of its software that solves a benchmark problem hundreds of times faster than the original baseline.

The Block-Structured Adaptive Mesh Refinement Co-Design Center is one of five ECP co-design centers, so named because they are structured to create a close interchange of ideas between hardware technologies, software technologies and applications. The goal of the co-design center’s AMReX software framework is to support the development of block-structured adaptive mesh refinement (AMR) algorithms for solving systems of partial differential equations on next-generation architectures.

AMR allows scientists to focus computing power on the most critical parts of the problem in the most computationally efficient way possible. Applications worldwide rely on AMR for a wide variety of applications. Within the ECP specifically, projects in the areas of accelerator modeling, astrophysics, combustion, cosmology and multiphase flows already rely on AMReX to support their algorithms.

Led by John Bell at Lawrence Berkeley National Laboratory, the AMR Co-Design Center builds on previous block-structured AMR work done by Berkeley Lab’s Center for Computational Sciences and Engineering and the lab’s Applied Numerical Algorithms Group. The groups respectively developed BoxLib and Chombo, two of the leading AMR frameworks used by the U.S. research community.

“The goal of the center is to redesign and re-implement our core block-structured AMR methodology to enable codes using AMReX to run efficiently on exascale computers,” said Bell. “Our latest release of AMReX demonstrates that DOE’s investment in the exascale program is already paying scientific dividends. Researchers using AMReX can now run today’s problems more efficiently.”

Members of the BoxLib and Chombo development teams, as well as developers of FLASH, a DOE-supported application code used in astrophysics and high-energy density physics, have come together with researchers from DOE’s National Renewable Energy Laboratory to build the new framework.

The co-design center released its first version of the AMReX framework in June and this latest release adds the capability to represent solid geometries using a technique known as the embedded boundary technique. This approach can be used to calculate flow past an object inside the domain, such as air over an airplane wing, or can represent the boundaries of a domain, such as the walls of a combustion chamber. To measure the effectiveness of ECP’s investment in AMReX, the team used an existing code to define a baseline, then solved the same problem with AMReX.  After a comprehensive re-design of the core data structures and a new implementation of the embedded boundary methodology, AMReX-based solvers reached the solution almost 200 times faster on the same number of processors.

Volume rendering of gas density inside a domain including many of the geometric features used to control and stabilize combustion. The solution is computed using PeleC, one of the ECP application codes built on AMReX that uses the embedded boundary approach to treat geometry. In this calculation, cold fluid enters from the bottom in the center tube while swirling hot fluid enters through the outer annular ring.

In addition, while the baseline code had limited support for running in parallel, the AMReX code uses a hybrid parallel approach, an effective way take full advantage of systems such as Cori with manycore processors.  

Members of the co-design center are also working with other groups developing exascale applications to make sure that researchers will be able to effectively use AMR algorithms on exascale systems when they are deployed.

As one example, AMReX is a cornerstone of the Pele combustion codes being developed by the “Transforming Combustion Science and Technology with Exascale Simulations” ECP application project led by Jacqueline Chen of Sandia National Laboratories. This project will enable combustion simulations using direct number simulations (DNS) and hybrid DNS/ arge eddy simulations in realistic geometries. Ultimately, the aim of the project is to contribute to the design of high-efficiency, low-emission combustion engines and gas turbines to reduce emissions and improve fuel efficiency.

“The Pele suite of combustion codes is built on top of the block-structured AMR infrastructure that AMReX is providing,” Chen said. “Notably, AMReX is providing performance portability, embedded boundary methods for complex geometry, and a container for treating particle-based spray, soot and radiation physics modules.”

The importance of AMR to DOE’s research missions was recently recognized as part of the 40th anniversary of the founding of the Department of Energy. To mark the anniversary, DOE’s Office of Science selected 40 of the most significant research milestones. Among them was the 1989 research paper “Local adaptive mesh refinement for shock hydrodynamics,” published in the Journal of Computational Physics and written by Marsha J. Berger of New York University and Phillip Colella, who led the development of Chombo.

Over the years AMR has been used to solve increasing complex problems and has been implemented in increasingly sophisticated software frameworks.

“AMR has played an important role in DOE since the ‘90s,” said David Brown, director of Berkeley Lab’s Computational Research Division. “With the development of AMReX, numerous scientific applications will have access to AMR at the exascale, which will have an enormous effect on future scientific productivity.”

The work was funded through the Department of Energy’s Exascale Computing Project through the Office of Advanced Scientific Computing Research in the DOE Office of Science.

About Computing Sciences at Berkeley Lab

The Lawrence Berkeley National Laboratory (Berkeley Lab) Computing Sciences organization provides the computing and networking resources and expertise critical to advancing the Department of Energy’s research missions: developing new energy sources, improving energy efficiency, developing new materials and increasing our understanding of ourselves, our world and our universe.

Source: Lawrence Berkeley National Laboratory

The post ECP Co-Design Center Achieves Orders of Magnitude Speed-Up in Latest Software Release appeared first on HPCwire.

Supercomputing Experiments Help Explain Magnetic Fields, Solar Storms

Mon, 10/02/2017 - 10:34

Oct. 2, 2017 — In July 2012, a powerful solar storm almost struck Earth. Scientists estimate that had the storm, called a coronal mass ejection (CME), hit the planet, the impact would have crippled power grids worldwide, burning out transformers and instruments.

A NASA probe that happened to lie in the CME’s path detected some of the charged particles it contained. Data the satellite collected showed the storm was twice as powerful as a 1989 event that knocked out Quebec’s entire power grid, disrupted power delivery across the United States and made the northern lights visible as far south as Cuba. In fact, the recent storm might have been stronger than the first and most powerful CME known to hit the planet, the Carrington event. That 1859 storm sprayed sparks from telegraph lines, setting fire to telegraph stations. Researchers put the odds of a Carrington-size CME occurring by 2024 – and possibly hitting Earth – at 12 percent.

Such events occur when field lines in the sun’s massive magnetic system snap and reconnect. “Magnetic fields are a reservoir of an enormous amount of energy, and major eruptive events occur in which this energy is liberated,” says Amitava Bhattacharjee, a plasma physicist at the Princeton Plasma Physics Laboratory (PPPL), a Department of Energy facility in Princeton, New Jersey. “Charged particles tend to get tied to magnetic field lines like beads on a wire – when the wire breaks, the beads get thrown off at enormous speeds.”

The phenomenon, known as fast magnetic reconnection, remains a mystery. No one knows how field lines break and rejoin fast enough to expel the billions of tons of material unleashed in a CME, or even in the smaller eruptions of common solar flares. In laboratory experiments and simulations, Bhattacharjee and his colleagues have revealed new mechanisms that help explain fast magnetic reconnection.

Coronal loops on the sun are linked to magnetic fields. Princeton Plasma Physics Laboratory experiments are combining with Oak Ridge National Laboratory supercomputer simulations to illuminate how the fields break apart and reconnect. Photo courtesy of NASA/Solar Dynamics Observatory.

Bhattacharjee has been in pursuit of such mechanisms since graduate school, when he realized that plasma physics is “a beautiful, classical field with wonderful equations that were good things to analyze and do computer simulations with,” he says. At the same time, he saw that plasmas – which constitute 99.5 percent of the visible universe – are also the key to “a very practical and important problem for humanity, namely magnetic fusion energy.”

For decades, nuclear fusion machines, such as doughnut-shaped tokamaks, have promised a virtually limitless supply of relatively clean energy. But a working fusion device is still out of reach, partly because of fast magnetic reconnection. “Magnetic fusion reactors have magnetic fields in them, and these magnetic fields can also reconnect and cause disruptive instabilities within a tokamak fusion plasma,” says Bhattacharjee, professor of astrophysical sciences at Princeton University and head of PPPL’s Theory and Computation Division.

In the present model of reconnection, opposing magnetic fields are pushed together by some external force, such as plasma currents. A thin, flat contact area forms between the two fields, building tension in the field lines. In this thin region, called a current sheet, plasma particles – ions and electrons – collide with one another, breaking field lines and allowing them to form new, lower-energy connections with partners from the opposing magnetic field. But under this model, the lines reconnect only as fast as they are pushed into the current sheet – not nearly fast enough to explain the tremendous outpouring of energy and particles in a fast-reconnection event.

Since this slow reconnection model depends on plasma particle collisions, many research groups have searched for collisionless effects that might account for fast reconnection. Promising explanations focus on the behavior of charged particles in the current sheet, where field strength is close to zero. There, the charged properties of the massive, sluggish ions are suppressed, and the nimble electrons are free to carry the current and whip field lines into new configurations.

For laboratory experiments on hidden mechanisms, Bhattacharjee’s team uses powerful lasers at the University of Rochester’s Omega facility. To develop computer models, the group uses Titan, a Cray XK7 supercomputer at the Oak Ridge Leadership Computing Facility, a DOE Office of Science user facility, through the Office of Science’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. The Office of Science’s Fusion Energy Sciences program and the DOE National Nuclear Security Administration sponsor the experiments.

In an early experiment led by PPPL research physicist Will Fox, the team pointed two intense Omega lasers at materials that yield plasma bubbles under the beams. Each bubble spontaneously generated its own magnetic field through an effect known as the Biermann battery. As happens in the sun and nuclear fusion devices, charged plasma particles lined up on the magnetic field lines. The bubbles plowed into each other, and a current sheet formed between them. The reconnection rate between the fields was fast – too fast for classical theory.

“That’s where we were first establishing the underlying mechanism for reconnection happening in this machine,” Bhattacharjee says. The team now had a model for fast magnetic reconnection, one applicable to earlier pioneering experiments conducted by groups in the United Kingdom and the United States. A simulation on Titan showed that more field lines were crammed together in the current sheet than anyone had realized, a phenomenon called flux pileup. The study showed that, in addition to previously suggested collisionless effects, flux pileup plays a role in fast reconnection.

In later experiments led by Gennady Fiksel, now at the University of Michigan, the team didn’t want to rely solely on spontaneously generated magnetic fields. “We felt we needed greater control on the magnetic fields we were using for the reconnection process,” Bhattacharjee says. “And so we used an external generator called MIFEDS (magneto-inertial fusion electrical discharge system), which produced external magnetic fields we could control.”

To capture changes in this field, the team filled the space with a thin background plasma, generated by a third laser, and imaged it using a beam of protons, which magnetic fields deflect. When two plasma bubbles impinged on the external magnetic field, the team created the clearest image so far of events taking place in the region where field lines reconnect. The new configuration also showed flux pileup, followed by a reconnection event that included small plasma bubbles forming in the region between the bubbles and, finally, abrupt annihilation of the magnetic field.

“The mechanism that we found is that you form this thin current sheet that can then be unstable, in what we call a plasmoid instability that breaks up this thin current sheet into little magnetic bubbles,” Bhattacharjee says. “The plasmoid instability is a novel mechanism for the onset of fast reconnection, which happens on a time scale that is independent of the resistance of the plasma.”

Bhattacharjee and his colleagues are working to understand how their discovery fits into the big picture of solar activity, solar storms and nuclear fusion devices. Once they and the broader community of plasma physicists fully understand reconnection, the ability to predict CMEs and tame some of the plasma instabilities inside tokamak reactors, for example, may be within reach.

Source: Princeton Plasma Physics Laboratory

The post Supercomputing Experiments Help Explain Magnetic Fields, Solar Storms appeared first on HPCwire.

Intel® Xeon® Scalable Processors Demonstrate Superlative Performance

Mon, 10/02/2017 - 01:02
Results Redefine AI, Dense Linear Algebra, and More

The new Intel® Xeon® Scalable Processors are already seeing some impressive results as they tackle some of the most challenging HPC workloads in AI, deep learning, and electromagnetics, and more. Improvements in the processor’s memory and network interfaces are redefining HPC computing and cost effectiveness.

Overall performance improvements

Intel claims the new Intel Xeon Scalable processors deliver a performance increase of 1.63xi on average performance increase for a range of HPC workloads over previous Intel Xeon processors due to a host of microarchitecture improvements that include: support for Intel® AVX-512 wide vector instructions, up to 28 cores and 56 threads per socket, support for up to eight socket systems, an additional two memory channels, support for DDR4 2666 MHz memory, and more.

Powering a “renaissance” in dense linear algebra

As Hatem Ltaief, Senior Research Scientist at KAUST, an Intel® Parallel Computing Center (IPCC) collaborator who received an early system states, “They rock!”

Ltaief and his postdocs have been testing these processors with their HiCMA library, an approximate yet accurate dense linear algebra package for big matrices – think billion rows by billion columns. He refers to the ability to perform matrix operations at this scale a “renaissance” in dense linear algebra. One example of the faster runtime and ability to run larger than current highly-optimized dense linear packages is shown in the figure below.

Figure 1: HiCMA results for various matrix sizes compared to MKL (Courtesy KAUST) Deep learning

Using these new processors, SURFsara (another IPCC collaborator) reports they have established both comparable and newii single-model state-of-the-art accuracy and time-to-model results on some popular deep learning benchmark architectures and data sets. These results surpassed results reported by both Microsoft and IBM by a large margin when training on Places-365 and Imagenet-22k datasets and replicate Intel® Xeon Phi™ processor results.  The authors also posted results on the popular ImageNet-1K dataset, where they achieved convergence in less than 40 minutes, explaining “all our trained models achieve a top-1 validation accuracy greater than 74% on ILSVRC-2012 validation set, and 74.3% on ILSVRC-2014 respectively.”

SURFsara used the 3500 dual-socket Intel Xeon Scalable 8160 processor (Skylake) node MareNostrum 4 supercomputer supplied by Lenovo at the Barcelona Supercomputing Center (BSC).iii Their distributed training scaled nicely they report, “leading to around 90% scaling efficiency when going from 1 to 256 nodes” as seen below.

“We didn’t find a clear upper limit for accuracy yet,” – Valeriu Codreanu (IPCC Primary Investigator, SURFsara)

Figure 2: Resnet-50 on Imagenet-1K distributed deep learning training scaling results on MareNostrum 4

Exploiting the larger memory capacity of the Intel processors compared to accelerators, SURFsara also found they could achieve better results on the same deep learning benchmarks using wide residual networks. Valeriu Codreanu, IPCC primary investigator at SURFsara, reported, “We didn’t find a clear upper limit for accuracy yet – wider models achieve better accuracy”. Thus, the Intel Xeon Scalable processors (along with Intel Xeon Phi processors) are opening the door to new and improved deep learning training.

Joe Curley, Director of HPC Platform and Ecosystem Enabling at Intel, summed up the importance of the SURFsara accomplishment: “Deep learning neural networks are now practical to deploy at scale to solve complex problems faster than ever before”.

“Deep learning neural networks are now practical to deploy at scale to solve complex problems faster than ever before” – Joe Curley (Senior Director of HPC Platform and Ecosystem Enabling, Intel Corporation)

Improved performance across a wide range of EM applications

Computer Simulation Technology (CST) also reports a performance boost across a wide range of electromagnetic applications as compared to previous generation processors. Their results are shown below:

Figure 3: Improved performance across a wide range of applications (Source http://cst-simulation.blogspot.de/2017/09/performance-of-cst-studio-suite-solvers.html ) Integrated Intel® Omni-Path Architecture (Intel® OPA)

CST reports efficient strong scaling of their transient solver when using an external Intel Omni-Path Architecture adaptor.

Figure 4: Reported scaling behavior (Source: http://cst-simulation.blogspot.de/2017/09/support-of-intel-omni-path-hpc.html

These results present a very positive view towards the future as Intel has launched a series of Intel Xeon Platinum and Intel Xeon Gold processor SKUs with Intel OPA integrated onto the processor package: access a fast, low-latency 100Gbps fabric without having to purchase an external Intel OPA interface card!

Figure 5: Some SKUs have an integrated Intel OPA transceiver Fast results from a fast processor

Even after a short period of general availability, reports show that these new processors are delivering superlative performance, hardware efficiency, and performance vs. cost capability.

Learn more about the value Intel Xeon Scalable processors can bring to your HPC environment at http://www.intel.com/ssf.


i Up to 1.63x Gains based on Geomean of Weather Research Forecasting – Conus 12Km, HOMME, LSTCLS-DYNA Explicit, INTES PERMAS V16, MILC, GROMACS water 1.5M_pme, VASP­Si256, NAMDstmv, LAMMPS, Amber GB Nucleosome, Binomial option pricing, Black-Scholes, Monte Carlo European options. Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, compo­nents, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance/datacenter

ii Using a custom Wide Residual Network architecture

iii SURFsara would like to acknowledge PRACE for awarding access to the MareNostrum 4 resource based in Spain at BSC.

The post Intel® Xeon® Scalable Processors Demonstrate Superlative Performance appeared first on HPCwire.

Livermore Labs’ Dona Crawford to Give Discovery Park Distinguished Lecture

Fri, 09/29/2017 - 14:48

Sept. 29 — On Oct. 2, Discovery Park, the College of Science and Department of Computer Science will host Dona Crawford, associate director emeritus of Lawrence Livermore National Laboratory, as part of the Discovery Park Distinguished Lecture Series.

Dona Crawford

In her talk, “#HPCMatters,” Crawford will discuss the role of high-performance computing in solving grand challenges to national security, scientific discovery, and economic competitiveness. Crawford’s talk will also explore current advantages and challenges in high-performance computing and steps necessary to advancing the field.

The lecture will be held 1:30-2:30 p.m. in Lawson Computer Science Building, Room 1142, and is open to the public.

Crawford is currently the president of the Livermore Lab Foundation, whose mission is to promote philanthropic engagement to advance promising scientific research, technology development and educational endeavors at LLNL. She has served on advisory committees for the National Research Council and the National Science Foundation.

The Discovery Park Distinguished Lecture Series is made possible by the Lilly Endowment, with the purpose of bringing experts and thought leaders in the areas of science and technology to Purdue. For more information about Crawford’s lecture, or the Discovery Park Distinguished Lecture Series, contact Maria Longoria-Littleton at mlongori@purdue.edu.

Source: Purdue University

The post Livermore Labs’ Dona Crawford to Give Discovery Park Distinguished Lecture appeared first on HPCwire.

Gartner Reveals the 2017 Hype Cycle for Data Management

Fri, 09/29/2017 - 11:41

STAMFORD, Conn., Sept. 29, 2017 — As data becomes ever more distributed across multiple systems, organizations have to cope with increasingly complex ecosystems and digital business requirements. The Gartner, Inc. Hype Cycle for Data Management helps CIOs, chief data officers (CDOs) and other senior data and analytics leaders understand the maturity of the data management technologies they are evaluating to provide a cohesive data management ecosystem in their organizations.

“Data management continues to be central to the move toward digital business. As requirements change within the architecture of the organization and place greater demands on underlying technology, the maturity and capability of many of the technologies highlighted in the Hype Cycle will advance rapidly,” said Donald Feinberg, vice president and distinguished analyst at Gartner. “Recent years have seen many new additions to the Hype Cycle, including in-memory, cloud, data virtualization, advanced analytics, data as a service, machine learning, graph, non-relational and Hadoop.”

Two technologies are of particular interest, in that they show the impact cloud computing is having on the data management discipline. Hadoop distributions are deemed to be obsolete before reaching the Plateau of Productivity because the complexity and questionable usefulness of the entire Hadoop stack is causing many organizations to reconsider its role in their information infrastructure. Instead, organizations are looking at increasingly competitive and convenient cloud-based options with on-demand pricing and fit-for-purpose data processing options.

As part of the same cloud-led trend, SQL interfaces to cloud object stores have appeared at the Innovation Trigger stage. “We expect these interfaces to represent the future of cloud database Platform as a Service (Paas) and reach the Plateau within two to five years because they are the focus of most cloud vendors and products in this space,” said Mr. Feinberg. “They enable organizations to interact with data stored in the cloud, using a familiar SQL syntax. Object stores are well suited to storing large volumes of multistructured data, typical of data lakes.”

Of the 35 other technologies highlighted on the 2017 Hype Cycle for Data Management, four are judged to be transformational in nature. Two — event stream processing (ESP) and operational in-memory database management system (IMDBMS) — are expected to reach the Plateau of Productivity within two to five years, while both blockchain and distributed ledgers are expected to take five to 10 years.

Event Stream Processing

ESP is one of the key enablers of digital business, algorithmic business and intelligent business operations. ESP technology, including distributed stream computing platforms (DSCPs) and event processing platforms (EPPs), is maturing rapidly. Stream analytics provided by ESP software improves the quality of decision-making by presenting information that could otherwise be overlooked.

Operational In-Memory DBMS

Operational In-Memory database management systems (IMDBMS) technology is maturing and growing in acceptance, although the infrastructure required to support it remains relatively expensive. Another inhibitor to the growth of operational IMDBMS technology is the need for persistence models that support the high levels of availability required to meet transaction SLAs. Nevertheless, operational IMDBMSs for transactions have the potential to make a tremendous impact on business value by speeding up data transactions 100 to 1,000 times.


Public distributed ledgers, including blockchain, continue to have high visibility, although organizations remain cautious about the future of public (permission-less) distributed ledger concepts due to scalability, risk and governance issues. Most business use cases have yet to be proven and extreme price volatility in bitcoin persists. Presupposing the technical and business challenges of distributed ledgers can be overcome; in the short term, organizations are most likely to use distributed ledger for operational efficiency gains via the use of shared information and infrastructure. Longer term, Gartner expects a complete reformation of whole industries and commercial activity as the programmable economy develops and ledgers contribute to the monetization of new ecosystems.

Distributed Ledgers

The requirements for more standards and enterprise-scale capabilities are evolving slowly, but distributed ledgers are still not adoptable in a mission-critical at-scale context. Their value propositions, compared with existing technology, are also not clearly established, making the widespread acceptance of the technology problematic. Private distributed ledger concepts are gaining traction, because they hold the promise to transform industry operating models and overcome some of the issues of scalability, risk management and governance that plague public ledgers. As with blockchain, however, many business use cases are unproven at this time.

Gartner clients can learn more in the report “Hype Cycle for Data Management 2017.” This research is part of the Gartner Trend Insight Report “2017 Hype Cycles Highlight Enterprise and Ecosystem Digital Disruptions.”With over 1,800 profiles of technologies, services and disciplines spanning over 100 Hype Cycles focused on a diversity of regions, industries and roles, this Trend Insight Report is designed to help CIOs and IT leaders respond to the opportunities and threats affecting their businesses, take the lead in technology-enabled business innovations and help their organizations define an effective digital business strategy. Information on further 2017 Hype Cycles covering data and analytics can be found on the Gartner Blog Network.

*Previously titled “Hype Cycle for Information Infrastructure, 2016, “Hype Cycle for Data Management, 2017”covers the broad aspects and technologies that describe, organize, integrate, share and govern data.

About the Gartner Data & Analytics Summits 2017/18

Gartner analysts will provide additional analysis on data and analytics leadership trends at the Gartner Data & Analytics Summits, taking place: November 20-21, 2017 in Frankfurt; February 26-27, 2018 in Sydney; March 5-8, 2018 in Grapevine, Texas; and March 19-21, 2018 in London. Follow news and updates from the events on Twitter using #GartnerDA.

About Gartner

Gartner, Inc. (NYSE: IT) is a leading research and advisory company. The company helps business leaders across all major functions in every industry and enterprise size with the objective insights they need to make the right decisions. Gartner’s comprehensive suite of services delivers strategic advice and proven best practices to help clients succeed in their mission-critical priorities. Gartner is headquartered in Stamford, Connecticut, U.S.A., and has more than 13,000 associates serving clients in 11,000 enterprises in 100 countries. For more information, visit www.gartner.com.

Source: Gartner

The post Gartner Reveals the 2017 Hype Cycle for Data Management appeared first on HPCwire.

NSF Gives Grants to Three Universities to Create Online Data Collaboration Platform

Fri, 09/29/2017 - 09:56

SALT LAKE CITY, Utah, Sept. 29, 2017 — The National Science Foundation has awarded the collective universities of Utah, Chicago and Michigan a $4 million, four-year grant to produce SLATE, a new online platform for stitching together large amount of data from multiple institutions that reduces friction commonly found in multi-faceted collaborations.

When complete, SLATE, which stands for Services Layer At The Edge, will be a local software and hardware platform that connects and interacts with cloud resources. This online platform will reduce the need for technical expertise, amount of physical resources like servers and time demands on individual IT departments, especially for smaller universities that lack the resources of larger institutions and computing centers.

From the cosmic radiation measurements by the South Pole Telescope to the particle physics of CERN, multi-institutional research collaborations require computing environments that connect instruments, data and storage servers. Because of the complexity of the science, and the scale of the data, these resources are often distributed among university research computing centers, national high-performance computing centers or commercial cloud providers. This resource disparity causes scientists to spend more time on the technical aspects of computation than on discoveries and knowledge creation.

A partner of the project, the University of Utah will be contributing to the following aspects of the project. Reference architecture, advanced networking aspects, core design, implementation and outreach to other principal investors from science disciplines and partner universities. Senior IT architect Joe Breen explained the goal is to have one simple platform for the end user.

“Software will be updated just like an app by experts from the platform operations and research teams. The software will need little to no assistance required from their local IT personnel. The SLATE platform is designed to work in any data center environment and will utilize advanced network capabilities, if available.”

The platform

Once installed, central research teams will be able to connect with far-flung research groups allowing the exchange of data to be automated. Software and computing tasks among institutions will no longer burden local system administrators with installation and operation of highly customized scientific computing services. By stitching together these resources, SLATE will also expand the reach of these domain-specific “science gateways.”

SLATE works by implementing “cyberinfrastructure as code,” increasing bandwidth science networks with a programmable “underlayment” edge platform. This platform hosts advanced services needed for higher-level capabilities such as data and software delivery, workflow services and science gateway components.

“A central goal of SLATE is to lower the threshold for campuses and researchers to create research platforms within the national cyberinfrastructure,” said University of Chicago senior fellow Robert Gardner.

Practical applications

Today’s most ambitious scientific investigations are too large for a single university or laboratory to tackle alone. Dozens of international collaborations comprised of scientific groups and institutions must coordinate the collection and analysis of immense data streams. These data streams include dark matter searches, the detection of new particles at the high-energy frontier and the precise measurement of radiation from the early universe. The data can come from telescopes, particle accelerators and other advanced instruments.

Today, many universities and research laboratories use a “Science DMZ” architecture to balance the need for security with the ability to rapidly move large amounts of data in and out of the local network. As sciences from physics to biology to astronomy become more data-heavy, the complexity and need for these subnetworks grows rapidly, placing additional strain on local IT teams.

Since 2003, a team of computation and Enrico Fermi Institute scientists led by Gardner has partnered with global projects to create the advanced cyberinfrastructure necessary for rapidly sharing data, computer cycles and software between partner institutions.

User benefits

“Science, ultimately, is a collective endeavor. Most scientists don’t work in a vacuum, they work in collaboration with their peers at other institutions,” said Shawn McKee, director of the Center for Network and Storage-Enabled Collaborative Computational Science at the University of Michigan. “They often need to share not only data, but systems that allow execution of workflows across multiple institutions. Today, it is a very labor-intensive, manual process to stitch together data centers into platforms that provide the research computing environment required by forefront scientific discoveries.”

With SLATE, local research groups will be able to fully participate in multi-institutional collaborations and contribute resources to their collective platforms with minimal hands-on effort from their local IT team. When joining a project, the researchers and admins can select a package of software from a cloud-based service — a kind of app store — that allows them to connect and work with the other partners.

By reducing the technical expertise and time demands for participating in multi-institution collaborations, the SLATE platform will be especially helpful to smaller universities that lack the resources and staff of larger institutions and computing centers. The SLATE functionality can also support the development of “science gateways” that make it easier for individual researchers to connect to HPC resources such as the Open Science Grid and XSEDE.

Source: University of Utah

The post NSF Gives Grants to Three Universities to Create Online Data Collaboration Platform appeared first on HPCwire.