Teach-and-Repeat: Accurately Extracting Operational Knowledge from Mobile Screen Demonstrations to Empower GUI Agents

In three linesTeach VLM extracts operational knowledge from mobile screen demonstrations by analyzing visual state transitions and generating natural-language instructions. The Teach-and-Repeat paradigm uses this knowledge to guide downstream GUI execution agents. Evaluation on Android World shows consistent Task Success Rate improvements.

Summary generated by Claude — human-verified