MIT course MAS.S62 "How to AI an AI": Four Layers of Evolution in Prompt Optimization, Harness Evolution, Model Self-Play, and Task Generation
@pliang279's Lectures 4-5 on "Harness evolution" are strongly recommended reading. They cover Harness Evolution (harness components, memory, tool and skill evolution) and Multi-Agent Evolution (planning, task decomposition, multi-agent systems and their evolution).
# Lecture 4: First Dissect the Harness, Then Discuss Making It Evolve
https://mit-mi.github.io/how2ai-course/fall2026/schedule/lec4%20-%20harness.pdf
The course dissects the Harness into five components, a framework that runs through both lectures:
· Context: prompts, memory, retrieved information
· Control: planning, routing, multi-agent orchestration
· Action: tools, skills, RAG, APIs, environments
· Evaluation: verification, safety, human feedback
· Evolution: reflection, updating, learning
Memory section: a three-axis taxonomy plus four representative systems
The lecture notes organize the whole memory research space along three axes: what is stored (feedback/skills/trajectories/semantic facts), how it is represented (text/structured/multimodal/parametric), and where it is stored (context/parameters/external), plus how it is reused (retrieval/adaptation/composition/transfer).
Four systems are then arranged along the "where stored" axis:
· @MemGPT (external memory): analogizes the LLM to an operating system, turning memory reads and writes into the agent's own function calls;
· @mem0ai (external memory): structures memory as a graph of entity-relation triples, supporting four operations: add/update/delete/noop;
· MEM1 (in-context memory, work the instructor composed himself): this is the part the lecture spends the most time on. Addressing the problem that "growing reasoning context causes complexity and forgetting," MEM1 has the model generate, at each step, an internal state that integrates reasoning and memory, then discard all prior history; the context size stays constant, and outcome-reward end-to-end training can teach the model to "compress." Experiments show three emergent behaviors: categorically storing key information for each question, updating memory after answering one question before moving to the next, and learning to skip a question after several fruitless searches. None of these behaviors was explicitly programmed.
· Reflexion / ExpeL (experiential memory): trajectories are stored as short-term memory and self-reflections as long-term memory; ExpeL further stores feedback on whether an experience was useful.
Context Language Models (CLM) bring the memory section to a conclusion: the model treats its entire context as a file, editing it freely with Bash commands, with edits synchronized to the real context; creating or deleting context files instantiates or terminates sub-agents; memory, context management, and multi-agent structure are unified as "file operations."
Tools and Skills section: from Toolformer to Meta-Harness
The lineage runs: Toolformer (the model learns single-step tool calls on its own) → ToolLLM (extended to multi-step calls across 16,000+ APIs) → CodeAct (the key claim: code is a meta-tool; a single action can combine control flow, libraries, and multiple operations; experiments show that using code for tool calls has the highest success rate and the fewest interaction rounds) → the Skills layer: skills can be represented as text or code (the latter being more composable), as abstract classes in code (PolySkill), or distilled from successful and failed trajectories into reusable workflows (Agent Workflow Memory) → Voyager's skill library (automatic curriculum generation + environment feedback + skill induction into the library).
The finale, Meta-Harness, pushes the entire lecture to its peak: existing LLM optimization methods show the optimizer only "brief summaries, scalar scores, and partial candidate histories," whereas the harness is far more complex. Meta-Harness represents the entire harness as a file system, with complete source code, scores, and all execution traces inside it, letting the proposer (Claude Code, reading with grep/cat) trace failures back to specific harness decisions. The feedback carries nearly 400 times more information than prior methods (10 million tokens versus 26,000). Prompts, RAG, skills, tools, memory, and the program loop are all unified as optimizable objects in the file system.
The most important page of the lecture: the abstraction ladder
The final summary page is the essence of the whole lecture notes. Professor Liang points out that the three methods he most admires share the same pattern:
· MEM1: treats memory as internal state within the context, updated autonomously by the LLM
· CLM: treats the entire context as a persistent file, autonomously maintained and rewritten by the LLM
· Meta-Harness: treats the entire harness as a file system, autonomously modified by the LLM
Common traits: simple, no manual design, end-to-end; the objects that can be optimized autonomously keep rising up the ladder from "memory → context → harness"; and the same paradigm can be extended in zero-shot, in-context learning, and fine-tuning modes. This is the most distilled expression of the course's "self-evolving" idea: rather than making AI stronger at one point, keep expanding the boundary of "what can be rewritten by AI itself."
# Lecture 5: From Planning to Multi-Agent, Turning Mysticism into Empirical Science
https://mit-mi.github.io/how2ai-course/fall2026/schedule/lec5%20-%20multi%20agent.pdf
Planning section: three key questions
The notes first clarify the core questions of planning: when is planning useful, and when is it harmful? How should plans be represented? When should a plan be revised? The conclusion is clear: prompt-based planning (CoT, Least-to-Most) suits pure-text, side-effect-free, reversible scenarios; once a plan acts on the real world, changes agent state, or is irreversible, one must "plan-execute-observe-replan." Subsequent cases show a spectrum: division of labor between large and small models (a large model plans while a 7B model executes GUI actions), human-in-the-loop clarification of uncertainty, training planners on synthetic data, sampling multiple plans to estimate uncertainty (calibrated planning), and "Thinking vs Doing": at test time, it is not just thinking, but actually executing, observing feedback, and then retrying.
Multi-agent section: five empirical findings
This section follows Towards a Science of Scaling Agent Systems (Nature Machine Intelligence 2026) as its main thread. Rather than stopping at the popular narrative that "multi-agent is more powerful," the notes present five sober empirical conclusions:
1. Multi-agent helps only for specific task structures. Financial agent tasks are naturally decomposable (revenue/cost/market), and MAS wins 81%; PlanCraft-type tasks have strong sequential dependencies, where splitting actually hurts, and MAS loses 70%. Guideline: parallelizable → centralized; requires exploration → decentralized; sequential dependencies → single agent.
2. Returns vary with the base model: OpenAI models add cost but yield platform-like performance; Google models have high cost and high performance; on Anthropic models, MAS may even harm performance.
3. More agents is not necessarily better: tool-intensive tasks (workflows with 16 tools) are dragged down by coordination overhead, and the penalty compounds with environmental complexity.
4. Heterogeneous mixes can beat a stronger single base model: you do not need to use expensive models everywhere; a "cheap orchestrator + expensive execution sub-agents" is a practical configuration; and if the single-agent baseline already exceeds 45%, MAS actually becomes harmful.
5. Performance is predictable: a scaling law parameterized by model, task, and MAS configuration can predict the optimal architecture with 87% accuracy, moving from "trial-and-error configuration" toward a principled science of agent design.
These five points, together with "Why Do Multi-Agent LLM Systems Fail?" on the reading list, form the course's empirical corrective to the multi-agent hype.
Multi-Agent Evolution: CORAL
Since the design space of multi-agent systems (tasks, base models, topology) is so sensitive, the next step is to let it evolve autonomously. There are three levels of objects that can evolve: individual agents (model/context/harness), system structure (roles/communication/topology), and shared information (public memory/tools/Skills). A representative work is CORAL (the first-author paper of TA Ao Qu in this course): multi-agent specialization, high autonomy, shared memory, and a heartbeat mechanism to sustain continuous operation. It achieves new SOTA on 8 tasks including mathematics, algorithms, and system optimization, and multi-agent evolution brings a 4%–20% improvement over single-agent evolution. Its analysis answers "why is co-evolution useful": 36% of evolution attempts take other agents as parent nodes, and these attempts have an improvement rate of 17% (versus an average of only 9%); cross-agent information transfer is the real source of gains; the exploration directions of the agents have low overlap (Jaccard 0.3–0.4, and more than half of the explorations are unique); contributions are balanced, with no free riders.
Putting the two lectures together: where are the boundaries of autonomous optimization?
Lecture 4 advances along "raising the object": memory → context → the entire harness (files → file system). Each level hands a larger piece of the system over to the LLM for autonomous rewriting.
Lecture 5 advances along "raising the system": planning of an individual agent → multi-agent orchestration → the evolution of the multi-agent system itself (roles/topology/shared memory). Each level hands a larger design space over to autonomous search.
Both lectures maintain the same critical restraint: Lecture 4 stresses that "simple, hand-design-free, end-to-end" is what scales, and Lecture 5 repeatedly reminds us through five empirical findings that "multi-agent systems are not a free lunch." Establishing a rigorous taxonomy and empirical regularities first, and only then handing the remaining design space over to autonomous evolution, is both the teaching structure of these two lectures and the methodological answer this course gives to the meta-question of "how to study self-evolving AI."