Tuesday & Thursdays, 9:30-11am in 56-167
Hadas Kotek
Course Description
This course explores the abilities and limitations of language models, focusing on state of the art tools such as ChatGPT. LLMs possess impressive language abilities, but they also occasionally fail in unpredictable ways. Our goal in this class will be to map the abilities and limitations of these models, focusing on complex reasoning and language abilities. We will attempt to discover systematicity in the models’ failures and to understand how they relate on the one hand to how the prompt is formulated and what we believe the training data and model architecture to be, and on the other hand how humans perform on the same tasks and how children acquire this knowledge. Along the way, we will learn about the development of language technologies and their capacities over time, as well as the state of the art linguistic theories that explain the phenomena of interest. We’ll ask ourselves whether LLMs appear to resemble humans in their approach to language and reasoning, and what this means for how we should understand what LLMs actually do (and how humans can and should interact with them).
The course also aims to develop skills and materials useful for non-academic jobs, such as behavioral study design, fundamentals of prompt engineering, knowledge engineering, and data annotation. Class assignments will build a scaffolding for a final paper or experiment to be presented at the end of the semester. No prior knowledge of programming languages or computational linguistics is expected.