The gold standard for designing the materials of tomorrow

Scientists from EPFL teamed up with colleagues from over 30 universities and institutes to investigate the accuracy and reproducibility of quantum-mechanical simulations of materials. Their work has led to a new gold standard in this field. Their study appears in this week’s edition of Science.

Over the last few years, materials simulations have seen an incredible amount of effort in the world of science and technology. Simulations bring a paradigm shift to the field of materials: instead of experimenting in a lab, scientists use the laws of physics and computational methods to reproduce in silico the behaviour of materials. But it gets better: starting from the laws of quantum mechanics, which govern the behavior of electrons and atoms, scientists can predict properties that haven’t been measured yet, or materials that haven’t been yet made. Since no physical laboratory or any major infrastructure is required, scientists are only limited by their own ingenuity and the power of their computers. (For a look at the current status of materials modelling, see this week’s "News and Views" in Nature Materials, by Nicola Marzari).

Many initiatives worldwide have sprung up to capitalize on these new capabilities – these include the launch of the Materials Genome Initiative in the United States, the NCCR MARVEL at EPFL, and the creation of several Centers of Excellence in Materials Design at the European level.

The software at the core of these projects involves teams of many scientists working often for decades at their development. Since these tools are so complex, with hundreds of thousands of lines of code, it becomes an impossible task to determine if they actually perform as they should – i.e. if they accurately calculate what they were meant to do in first place. Hence the only solution to break from this impass was to perform a blind test pitting against each other the state-of-the-art worldwide.

Online collaboration brings experts together
Scientists from EPFL therefore joined forces with more than 30 other research groups to crowdsource this problem to a final solution. 40 different methods to describe the influence of pressure in 71 different crystals were used, and due to the international composition of the team, discussions and collaboration were mainly conducted via online tools – similarly to the way people collaborate to write Wikipedia.

Two major camps were present – those of all-electron calculations, that are extremely accurate but overly expensive, and those of pseudopotential calculations, that aim not only at accuracy but at maximum computational efficiency. The results were enlightening, allowing to identify not only major outliers, but finding some dirt in everyone’s courtyard. And, at the end, allowing to define a common gold standard for an entire community, that can act as a seal of quality for any future calculation. Ghent University now acts as a formal repository of all the results ( ) and the EPFL team was proud to deliver what is currently the most accurate set up worldwide for materials modelling using pseudopotentials (the Solid State Pseudopotential Library – and the only one currently available with an accuracy matching all-electron calculations.

Reproducibility does not come easily
It’s a corner stone of science: independent yet identical experiments should produce identical results. However, several recent studies have pointed out that such reproducibility does not always come spontaneously. In scientific areas as diverse as psychology research and genetic research, cases were identified where repeating previous experiments led to very different results. Even predictions by computer codes require caution, since the way in which theoretical models are implemented may affect simulation results. For the study and design of materials, for instance, there were several independent software packages available based on quantum physics, and there were being used increasingly often in an automated fashion with limited human supervision.

Despite the need for reliable property predictions of materials, the reproducibility of quantum simulations had not been investigated before. This is mainly because this was an herculean task where many different flavours existed and many dialects were spoken. Thanks to the works of scientists worldwide, a new standard has been established, opening doors to realizing the best simulations possible.


Reproducibility in density functional theory calculations of solids, Science




NCCR Marvel

MaX Center


Text: Nicola Marzari / Laure-Anne Pessina