ON THE DEFINITION AND IMPORTANCE OF INTERPRETABILITY IN SCIENTIFIC MACHINE LEARNING

In the context of scientific machine learning (SciML), the "black box" nature of models involving neural networks makes researchers uneasy. Though neural networks trained on large data sets have been successfully used to describe and predict many physical phenomena, there is a sense that, unlike traditional scientific models-where relationships come packaged in the form of simple mathematical expressions-the findings of the neural network cannot be integrated into the body of scientific knowledge. Critics of machine learning (ML)'s inability to produce human-understandable relationships have converged on the concept of "interpretability" as its point of departure from more traditional forms of science. As the growing interest in interpretability has shown, researchers in the physical sciences seek not just predictive models, but also to uncover the fundamental principles that govern a system of interest. In hopes of ushering in a future where ML models participate in basic scientific discovery, it is now commonplace to view interpretability as a primary goal. However, clarity around a definition of interpretability and the precise role that it plays in science is lacking in the lit