Stanford CS 329Z lecture compares data sources for training AI agents

Stanford's CS 329Z lecture 11, "What Data Do Agents Need?", says AI agents need two kinds of training data: trajectory data for imitation learning and environments for reinforcement learning.

According to a summary posted on X by @shao__meng, the lecture compares five collection strategies: human demonstrations, web mining, synthetic trajectories, synthetic environments, and simulators.

The lecture warns that poorly filtered data can teach reward hacking and overfitting to a single harness, the summary says.