Deep Models and on Shaping their Development

Percy Liang, Stanford University Machine Learning Advances and Applications Seminar Date and Time: Monday, March 8, 2021 - 3:00pm to 4:00pm Abstract: Models in deep learning are wild beasts: they devour raw data, are powerful but hard to control. This talk explores two approaches to taming them. First, I will introduce concept bottleneck networks, in which a deep neural network makes a prediction via interpretable, high-level concepts. We show that such models can obtain comparable accuracy with standard models, while offering the unique ability for a human to perform test-time interventions on the concepts. Second, I will introduce prefix-tuning, which allows one to harness the power of pre-trained language models (e.g., GPT-2) for text generation tasks. The key idea is to learn a continuous task-specific prefix that primes the language model for the task at hand. Prefix-tuning obtains comparable accuracy to fine-tuning,