Mechanistic Interpretability

Michael Igorevich Ivanitskiy

2026-06-22

What you’ll learn

  • Some motivation for why you should care about interpretability
  • overview of current techniques, mostly on interp for language models
  • open problems that might make for good projects

Talk Outline

  • Introduction
  • “Classical” Interpretability, where it fails
  • Transformer notation & review
  • mechanistic interpretability

Introduction

The Problem:

  • We don’t “make” LLMs, we find them
  • These models are getting smarter, very fast

Gradient descent finds a set of parameters which minimize the training objective, but it’s just a point in a billions-dimensional space of possible models. We don’t build models the way we build cars or planes.