Reading notes, ideas, and explanations on deep learning, language, vision, and mathematics.
- A note on a note on mechanistic interpretability, variables, and importance of interpretable bases
- Notes on Toy Models of Superposition
- How Does Information Bottleneck Help Deep Learning?
- Masked Language Modeling
- Language Models for Text Classification: Is In-Context Learning Enough?
- Linear Decoding and Deep Nets with Neural Collapse
- Some definitions in Information Theory
- Large Language Models Struggle to Learn Long-Tail Knowledge