Informing geometric deep learning with electronic interactions to accelerate quantum chemistry
Edited by Klavs Jensen, Massachusetts Institute of Technology, Cambridge, MA; received April 1, 2022; accepted June 6, 2022
Significance
The interplay between molecular simulation and artificial intelligence has spurred many insights into chemical discovery, yet the data requirement of machine learning approaches remains a bottleneck. We tackle this challenge by developing a rigorous framework to conserve the symmetries of quantum chemical systems within deep neural networks. Our framework greatly improves the prediction of numerous molecular electronic and energetic properties, even for systems far from the training data. Through tight integration of quantum theory, geometry, and machine learning, our study offers a solution to vastly accelerate the in silico modelling of chemical phenomena in many domains, such as material design and catalysis.
Abstract
Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. However, existing machine learning techniques are challenged by the scarcity of training data when exploring unknown chemical spaces. We overcome this barrier by systematically incorporating knowledge of molecular electronic structure into deep learning. By developing a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simulations and learned mappings to recover high-fidelity physical quantities. OrbNet-Equi accurately models a wide spectrum of target properties while being several orders of magnitude faster than density functional theory. Despite only using training samples collected from readily available small-molecule libraries, OrbNet-Equi outperforms traditional semiempirical and machine learning–based methods on comprehensive downstream benchmarks that encompass diverse main-group chemical processes. Our method also describes interactions in challenging charge-transfer complexes and open-shell systems. We anticipate that the strategy presented here will help to expand opportunities for studies in chemistry and materials science, where the acquisition of experimental or reference training data is costly.
Data Availability
Source data for results described in the text and SI Appendix, the training dataset, code, and evaluation examples have been deposited in Zenodo (https://zenodo.org/record/6568518#.YrtTKHbMK38) (99).
Acknowledgments
Z.Q. acknowledges graduate research funding from Caltech and partial support from the Amazon–Caltech AI4Science fellowship. A.A. and T.F.M. acknowledge partial support from the Caltech DeLogi fund, and A.A. acknowledges support from a Caltech Bren professorship. Z.Q. acknowledges Bo Li, Vignesh Bhethanabotla, Dani Kiyasseh, Hongkai Zheng, Sahin Lale, and Rafal Kocielnik for proofreading and helpful comments on the manuscript.
Supporting Information
Appendix 01 (PDF)
- Download
- 2.09 MB
References
1
C. Ling, A review of the recent progress in battery informatics. npj Comput. Mater 8, 1–22 (2022).
2
J. Vamathevan et al., Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discov. 18, 463–477 (2019).
3
M. H. Baik, R. A. Friesner, Computing redox potentials in solution: Density functional theory as a tool for rational design of redox agents. J. Phys. Chem. A 106, 7407–7412 (2002).
4
B. K. Shoichet, Virtual screening of chemical libraries. Nature 432, 862–865 (2004).
5
M. Cordova, M. D. Wodrich, B. Meyer, B. Sawatlon, C. Corminboeuf, Data-driven advancement of homogeneous nickel catalyst activity for aryl ether cleavage. ACS Catal. 10, 7021–7031 (2020).
6
A. Szabo, N. S. Ostlund, Modern Quantum Chemistry (Mineola, Dover, United Kingdom, 1996), pp. 231–239.
7
W. Kohn, L. J. Sham, Self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133 (1965).
8
A. D. Mackerell Jr., Empirical force fields for biological macromolecules: Overview and issues. J. Comput. Chem. 25, 1584–1604 (2004).
9
J. Behler, M. Parrinello, Generalized neural-network representation of high-dimensional potential-energy surfaces. Phys. Rev. Lett. 98, 146401 (2007).
10
J. Behler, Perspective: Machine learning potentials for atomistic simulations. J. Chem. Phys. 145, 170901 (2016).
11
A. S. Christensen, L. A. Bratholm, F. A. Faber, O. Anatole von Lilienfeld, FCHL revisited: Faster and more accurate quantum machine learning. J. Chem. Phys. 152, 044107 (2020).
12
L. Zhang, J. Han, H. Wang, R. Car, W. E, Deep potential molecular dynamics: A scalable model with the accuracy of quantum mechanics. Phys. Rev. Lett. 120, 143001 (2018).
13
K. Schütt et al., Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. Adv. Neural Inf. Process. Syst. 30, 991–1001 (2017).
14
O. T. Unke, M. Meuwly, Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput. 15, 3678–3693 (2019).
15
Z. Liu et al., Transferable multi-level attention neural network for accurate prediction of quantum chemistry properties via multitask learning. J. Chem. Inf. Model. 61, 1066–1082 (2021).
16
J. Gasteiger, J. Groß, S. Günnemann, “Directional message passing for molecular graphs” in International Conference on Learning Representations (ICLR) (Technical University of Munich, Munich, Germany, 2020).
17
Y. Liu et al., Spherical message passing for 3D graph networks. arXiv [Preprint] (2021). https://arxiv.org/abs/2102.05013v2 (Accessed 13 April 2021).
18
K. T. Schütt, O. T. Unke, M. Gastegger, Equivariant message passing for the prediction of tensorial properties and molecular spectra. arXiv [Preprint] (2021). https://arxiv.org/abs/2102.03150 (Accessed 13 April 2021).
19
R. J. L. Townshend et al., Geometric deep learning of RNA structure. Science 373, 1047–1051 (2021).
20
B. Cheng, G. Mazzola, C. J. Pickard, M. Ceriotti, Evidence for supercritical behaviour of high-pressure liquid hydrogen. Nature 585, 217–220 (2020).
21
D. Folmsbee, G. Hutchison, Assessing conformer energies using electronic structure and machine learning methods. Int. J. Quantum Chem. 121, e26381 (2020).
22
D. Rosenberger, J. S. Smith, A. E. Garcia, Modeling of peptides with classical and novel machine learning force fields: A comparison. J. Phys. Chem. B 125, 3598–3612 (2021).
23
L. Cheng, M. Welborn, A. S. Christensen, T. F. Miller III, A universal density matrix functional from molecular orbital-based machine learning: Transferability across organic molecules. J. Chem. Phys. 150, 131103 (2019).
24
Y. Chen, L. Zhang, H. Wang, W. E, Ground state energy functional with hartree–fock efficiency and chemical accuracy. J. Phys. Chem. A 124, 7155–7165 (2020).
25
H. Li, C. Collins, M. Tanha, G. J. Gordon, D. J. Yaron, A density functional tight binding layer for deep learning of chemical Hamiltonians. J. Chem. Theory Comput. 14, 5764–5776 (2018).
26
Z. Qiao, M. Welborn, A. Anandkumar, F. R. Manby, T. F. Miller III, OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features. J. Chem. Phys. 153, 124111 (2020).
27
L. Li et al., Kohn-sham equations as regularizer: Building prior knowledge into machine-learned physics. Phys. Rev. Lett. 126, 036401 (2021).
28
R. Nagai, R. Akashi, O. Sugino, Completing density functional theory by machine learning hidden messages from molecules. npj Comput. Mater 6, 1–8 (2020).
29
J. Kirkpatrick et al., Pushing the frontiers of density functionals by solving the fractional electron problem. Science 374, 1385–1389 (2021).
30
M. Welborn, L. Cheng, T. F. Miller III, Transferability in machine learning for electronic structure via the molecular orbital basis. J. Chem. Theory Comput. 14, 4772–4779 (2018).
31
S. Dick, M. Fernandez-Serra, Machine learning accurate exchange and correlation functionals of the electronic density. Nat. Commun. 11, 3509 (2020).
32
K. Low, M. L. Coote, E. I. Izgorodina, Inclusion of more physics leads to less data: Learning the interaction energy as a function of electron deformation density with limited training data. J. Chem. Theory Comput. 18, 1607–1618 (2022).
33
K. Karandashev, O. A. von Lilienfeld, An orbital-based representation for accurate quantum machine learning. arXiv [Preprint] (2021). https://arxiv.org/abs/2112.12877v1 (Accessed 24 March 2022).
34
M. M. Bronstein, J. Bruna, T. Cohen, P. Veličković, Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv [Preprint] (2021). https://arxiv.org/abs/2104.13478 (Accessed 25 October 2021).
35
T. Cohen, M. Welling, “Group equivariant convolutional networks” in International Conference on Machine Learning(PMLR) (JMLR, New York, NY, 2016), pp. 2990–2999.
36
M. Weiler, M. Geiger, M. Welling, W. Boomsma, T. Cohen, 3D steerable CNNs: Learning rotationally equivariant features in volumetric data. arXiv [Preprint] (2018). https://arxiv.org/abs/1807.02547 (Accessed 25 October 2021).
37
R. Kondor, Z. Lin, S. Trivedi, Clebsch–Gordan nets: A fully Fourier space spherical convolutional neural network. Adv. Neural Inf. Process. Syst. 31, 10138–10147 (2018).
38
B. Anderson, T. S. Hy, R. Kondor, “Cormorant: Covariant molecular neural networks” in Advances in Neural Information Processing Systems, H. Wallachet al., Eds. (Curran Associates, Inc., 2019), vol. 32, pp. 14537–14546.
39
N. Thomas et al., Tensor field networks: Rotation-and translation-equivariant neural networks for 3D point clouds. arXiv [Preprint] (2018). https://arxiv.org/abs/1802.08219 (Accessed 18 November 2020).
40
F. B. Fuchs, D. E. Worrall, V. Fischer, M. Welling, Se (3)-transformers: 3D roto-translation equivariant attention networks. arXiv [Preprint] (2020). https://arxiv.org/abs/2006.10503 (Accessed 9 December 2020).
41
C. Bannwarth et al., Extended tight-binding quantum chemistry methods. Wiley Interdiscip. Rev. Comput. Mol. Sci. 11, e1493 (2021).
42
L. D. Chen et al., Embedded mean-field theory for solution-phase transition-metal polyolefin catalysis. J. Chem. Theory Comput. 16, 4226–4237 (2020).
43
J. P. Janet, S. Ramesh, C. Duan, H. J. Kulik, Accurate multiobjective design in a space of millions of transition metal complexes with neural-network-driven efficient global optimization. ACS Cent. Sci. 6, 513–524 (2020).
44
A. Dommer et al., #COVIDisAirborne: AI-enabled multiscale computational microscopy of delta SARS-CoV-2 in a respiratory aerosol. bioRxiv [Preprint] (2021). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8609898/ (Accessed 24 March 2022).
45
S. Grimme, C. Bannwarth, P. Shushkov, A robust and accurate tight-binding quantum chemical method for structures, vibrational frequencies, and noncovalent interactions of large molecular systems parametrized for all spd-block elements (Z=1–86). J. Chem. Theory Comput. 13, 1989–2009 (2017).
46
R. Ramakrishnan, P. O. Dral, M. Rupp, O. A. von Lilienfeld, Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1, 140022 (2014).
47
J. Klicpera, S. Giri, J. T. Margraf, S. Günnemann, Fast and uncertainty-aware directional message passing for non-equilibrium molecules. arXiv [Preprint] (2020). https://arxiv.org/abs/2011.14115v2 (Accessed 13 April 2021).
48
R. Ramakrishnan, P. O. Dral, M. Rupp, O. A. von Lilienfeld, Big data meets quantum chemistry approximations: The Δ-machine learning approach. J. Chem. Theory Comput. 11, 2087–2096 (2015).
49
F. A. Faber, A. S. Christensen, B. Huang, O. A. von Lilienfeld, Alchemical and structural distribution based representation for universal quantum machine learning. J. Chem. Phys. 148, 241717 (2018).
50
A. P. Bartók et al., Machine learning unifies the modeling of materials and molecules. Sci. Adv. 3, e1701816 (2017).
51
A. S. Christensen, F. A. Faber, O. A. von Lilienfeld, Operators in quantum machine learning: Response properties in chemical space. J. Chem. Phys. 150, 064105 (2019).
52
M. Veit, D. M. Wilkins, Y. Yang, R. A. DiStasio Jr., M. Ceriotti, Predicting molecular dipole moments by combining atomic partial charges and atomic dipoles. J. Chem. Phys. 153, 024113 (2020).
53
B. Huang, O. A. von Lilienfeld, Quantum machine learning using atom-in-molecule-based fragments selected on the fly. Nat. Chem. 12, 945–951 (2020).
54
W. Yang, W. J. Mortier, The use of global and local molecular parameters for the analysis of the gas-phase basicity of amines. J. Am. Chem. Soc. 108, 5708–5711 (1986).
55
G. L. Silva, V. Ediz, D. Yaron, B. A. Armitage, Experimental and computational investigation of unsymmetrical cyanine dyes: Understanding torsionally responsive fluorogenic dyes. J. Am. Chem. Soc. 129, 5710–5718 (2007).
56
K. Atz, C. Isert, M. N. A. Böcker, J. Jiménez-Luna, G. Schneider, δ-quantum machine learning for medicinal chemistry. Phys. Chem. Chem. Phys. 24, 10775–10783 (2022).
57
A. S. Christensen, O. A. von Lilienfeld, On the role of gradients for machine learning of molecular energies and forces. Mach. Learn. Sci. Technol. 1, 045018 (2020).
58
S. Batzner et al., E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13, 2453 (2022).
59
S. Chmiela et al., Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 3, e1603015 (2017).
60
L. A. Burns et al., The biofragment database (BFDb): An open-data platform for computational chemistry analysis of noncovalent interactions. J. Chem. Phys. 147, 161727 (2017).
61
A. Fabrizio, A. Grisafi, B. Meyer, M. Ceriotti, C. Corminboeuf, Electron density learning of non-covalent systems. Chem. Sci. (Camb.) 10, 9424–9432 (2019).
62
P. B. Jørgensen, A. Bhowmik, Graph neural networks for fast electron density estimation of molecules, liquids, and solids. arXiv [Preprint] (2021). https://arxiv.org/abs/2112.00652 (Accessed 29 March 2022).
63
S. Marqusee, R. L. Baldwin, Helix stabilization by Glu-…Lys+ salt bridges in short peptides of de novo design. Proc. Natl. Acad. Sci. U.S.A. 84, 8898–8902 (1987).
64
J. S. Smith et al., Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning. Nat. Commun. 10, 2903 (2019).
65
P. Zheng, R. Zubatyuk, W. Wu, O. Isayev, P. O. Dral, Artificial intelligence-enhanced quantum chemical method with broad applicability. Nat. Commun. 12, 7022 (2021).
66
J. G. Brandenburg, C. Bannwarth, A. Hansen, S. Grimme, B97-3c: A revised low-cost variant of the B97-D density functional method. J. Chem. Phys. 148, 064104 (2018).
67
S. Grimme, A. Hansen, S. Ehlert, J. M. Mewes, r2SCAN-3c: A “Swiss army knife” composite electronic-structure method. J. Chem. Phys. 154, 064103 (2021).
68
B. K. Rai et al., Torsionnet: A deep neural network to rapidly predict small-molecule torsional energy profiles with the accuracy of quantum mechanics. J. Chem. Inf. Model. 62, 785–800 (2022).
69
C. Bannwarth, S. Ehlert, S. Grimme, GFN2-xTB — An accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and density-dependent dispersion contributions. J. Chem. Theory Comput. 15, 1652–1671 (2019).
70
D. G. Smith, L. A. Burns, K. Patkowski, C. D. Sherrill, Revised damping parameters for the d3 dispersion correction to density functional theory. J. Phys. Chem. Lett. 7, 2197–2203 (2016).
71
T. Risthaus, M. Steinmetz, S. Grimme, Implementation of nuclear gradients of range-separated hybrid density functionals and benchmarking on rotational constants for organic molecules. J. Comput. Chem. 35, 1509–1516 (2014).
72
U. R. Fogueri, S. Kozuch, A. Karton, J. M. Martin, The melatonin conformer space: Benchmark and assessment of wave function and DFT methods for a paradigmatic biological and pharmacological molecule. J. Phys. Chem. A 117, 2269–2277 (2013).
73
L. A. Curtiss, K. Raghavachari, G. W. Trucks, J. A. Pople, Gaussian-2 theory for molecular energies of first-and second-row compounds. J. Chem. Phys. 94, 7221–7230 (1991).
74
A. Studer, D. P. Curran, The electron is a catalyst. Nat. Chem. 6, 765–773 (2014).
75
L. Goerigk et al., A look at the density functional theory zoo with the advanced GMTKN55 database for general main group thermochemistry, kinetics and noncovalent interactions. Phys. Chem. Chem. Phys. 19, 32184–32215 (2017).
76
C. Devereux et al., Extending the applicability of the ANI deep learning molecular potential to sulfur and halogens. J. Chem. Theory Comput. 16, 4192–4202 (2020).
77
K. U. Lao, R. Schäffer, G. Jansen, J. M. Herbert, Accurate description of intermolecular interactions involving ions using symmetry-adapted perturbation theory. J. Chem. Theory Comput. 11, 2473–2486 (2015).
78
A. Karton, S. Daon, J. M. Martin, W4-11: A high-confidence benchmark dataset for computational thermochemistry derived from first-principles w4 data. Chem. Phys. Lett. 510, 165–178 (2011).
79
B. Li, H. Xu, Y. Dang, K. Houk, Dispersion and steric effects on enantio-/diastereoselectivities in synergistic dual transition-metal catalysis. J. Am. Chem. Soc. 144, 1971–1985 (2022).
80
O. Lampret et al., The roles of long-range proton-coupled electron transfer in the directionality and efficiency of [FeFe]-hydrogenases. Proc. Natl. Acad. Sci. U.S.A. 117, 20520–20529 (2020).
81
W. Chen, Z. Yu, Z. Wang, A. Anandkumar, “Automated synthetic-to-real generalization” in International Conference on Machine Learning (PMLR, 2020), pp. 1746–1756.
82
J. J. Sakurai, E. D. Commins, Modern Quantum Mechanics(Addison-Wesley, Reading, MA, revised ed., 1995).
83
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, G. E. Dahl, “Neural message passing for quantum chemistry” in Proceedings of the 34th International Conference on Machine Learning-Volume 70 (PMLR, 2017), pp. 1263–1272.
84
T. H. Dunning, P. J. Hay, Gaussian Basis Sets for Molecular Calculations in Methods of Electronic Structure Theory (Springer, 1977), pp. 1–27.
85
F. Manby et al., entos: A quantum molecular simulation package. ChemRxiv [Preprint] (2019). https://chemrxiv.org/engage/chemrxiv/article-details/60c740e10f50db1f643959db (Accessed 25 October 2021).
86
S. Ioffe, C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift” in International Conference on Machine Learning (JMLR, 2015), pp. 448–456.
87
J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization. arXiv [Preprint] (2016). https://arxiv.org/abs/1607.06450 (Accessed 25 October 2021).
88
Z. Qiao et al., Multi-task learning for electronic structure to predict and explore molecular potential energy surfaces. arXiv [Preprint] (2020). https://arxiv.org/abs/2011.02680 (Accessed 24 March 2022).
89
A. Gaulton et al., The ChEMBL database in 2017. Nucleic Acids Res. 45, D945–D954 (2017).
90
J. S. Smith, O. Isayev, A. E. Roitberg, ANI-1, A data set of 20 million calculated off-equilibrium conformations for organic molecules. Sci. Data 4, 170193 (2017).
91
P. J. Ropp, J. C. Kaminsky, S. Yablonski, J. D. Durrant, Dimorphite-DL: An open-source program for enumerating the ionization states of drug-like small molecules. J. Cheminformatics 11, 14 (2019).
92
A. P. Bento et al., An open source chemical structure curation pipeline using RDKit. J. Cheminformatics 12, 51 (2020).
93
P. Jurečka, J. Šponer, J. Cerný, P. Hobza, Benchmark database of accurate (MP2 and CCSD(T) complete basis set limit) interaction energies of small model complexes, DNA base pairs, and amino acid pairs. Phys. Chem. Chem. Phys. 8, 1985–1993 (2006).
94
Y. S. Lin, G. D. Li, S. P. Mao, J. D. Chai, Long-range corrected hybrid density functionals with improved dispersion corrections. J. Chem. Theory Comput. 9, 263–272 (2013).
95
F. Weigend, R. Ahlrichs, Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence quality for H to Rn: Design and assessment of accuracy. Phys. Chem. Chem. Phys. 7, 3297–3305 (2005).
96
C. Lee, W. Yang, R. G. Parr, Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density. Phys. Rev. B Condens. Matter 37, 785–789 (1988).
97
F. Weigend, Hartree-Fock exchange fitting basis sets for H to Rn. J. Comput. Chem. 29, 167–175 (2008).
98
B. Huang, O. A. von Lilienfeld, Quantum machine learning using atom-in-molecule-based fragments selected on the fly. Nat. Chem. 12, 945–951 (2020).
99
Z. Qiao et al., OrbNet-Equi: Informing geometric deep learning with electronic interactions to accelerate quantum chemistry. Zenodo. https://doi.org/10.5281/zenodo.6568518. Deposited 20 May 2022.
Information & Authors
Information
Published in
Classifications
Copyright
Copyright © 2022 the Author(s). Published by PNAS. This article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).
Data Availability
Source data for results described in the text and SI Appendix, the training dataset, code, and evaluation examples have been deposited in Zenodo (https://zenodo.org/record/6568518#.YrtTKHbMK38) (99).
Submission history
Received: April 1, 2022
Accepted: June 6, 2022
Published online: July 28, 2022
Published in issue: August 2, 2022
Keywords
Acknowledgments
Z.Q. acknowledges graduate research funding from Caltech and partial support from the Amazon–Caltech AI4Science fellowship. A.A. and T.F.M. acknowledge partial support from the Caltech DeLogi fund, and A.A. acknowledges support from a Caltech Bren professorship. Z.Q. acknowledges Bo Li, Vignesh Bhethanabotla, Dani Kiyasseh, Hongkai Zheng, Sahin Lale, and Rafal Kocielnik for proofreading and helpful comments on the manuscript.
Notes
This article is a PNAS Direct Submission.
Authors
Competing Interests
Competing interest statement: A patent application related to this work has been filed. A.S.C., M.W., F.R.M., and T.F.M. are employees of Entos, Inc. or its affiliates. The software used for computing input features and gradients is proprietary to Entos, Inc.
Metrics & Citations
Metrics
Citation statements
Altmetrics
Citations
Cite this article
119 (31) e2205221119,
Export the article citation data by selecting a format from the list below and clicking Export.
Cited by
Loading...
View Options
View options
PDF format
Download this article as a PDF file
DOWNLOAD PDFLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Personal login Institutional LoginRecommend to a librarian
Recommend PNAS to a LibrarianPurchase options
Purchase this article to access the full text.