arXiv Open Access 2023

Explainability Techniques for Chemical Language Models

Stefan Hödl William Robinson Yoram Bachrach Wilhelm Huck Tal Kachman

Lihat Sumber

Abstrak

Explainability techniques are crucial in gaining insights into the reasons behind the predictions of deep learning models, which have not yet been applied to chemical language models. We propose an explainable AI technique that attributes the importance of individual atoms towards the predictions made by these models. Our method backpropagates the relevance information towards the chemical input string and visualizes the importance of individual atoms. We focus on self-attention Transformers operating on molecular string representations and leverage a pretrained encoder for finetuning. We showcase the method by predicting and visualizing solubility in water and organic solvents. We achieve competitive model performance while obtaining interpretable predictions, which we use to inspect the pretrained model.

Topik & Kata Kunci

cs.LG cs.AI physics.chem-ph q-bio.QM

Penulis (5)

Stefan Hödl

William Robinson

Yoram Bachrach

Wilhelm Huck

Tal Kachman

Format Sitasi

APA MLA BibTeX

Hödl, S., Robinson, W., Bachrach, Y., Huck, W., Kachman, T. (2023). Explainability Techniques for Chemical Language Models. https://arxiv.org/abs/2305.16192

Akses Cepat

Lihat di Sumber

Informasi Jurnal

Tahun Terbit: 2023
Bahasa: en
Sumber Database: arXiv
Akses: Open Access ✓