Programming By Demonstration

Overview

Computer applications are difficult to customize for the non-programmer faced with repetitive tasks. Macro recording systems allow users to record sequences of actions and play them back, but many users don't use them. Often the time and effort required to construct and debug macros that work correctly in all cases exceeds the time and effort required to do the repetitive task by hand.

Macro systems are a form of programming by demonstration (PBD), in which users demonstrate their task to the system, which then learns a program that accomplishes the task. The benefits of PBD are that the user doesn't need to learn an arcane programming language in order to automate repetitive tasks, and she demonstrates her programs using an interface with she is already familiar.

However, existing macro recorder systems have many drawbacks. They are not robust to variations in the user's data; if the user's context varies slightly from the context in which the macro was recorded, it could fail to execute, or worse, mangle the hapless user's data. If a particular repetitive task should be performed differently depending on some aspect of the data being processed, simple macro systems---those which record and playback only the sequence of keystrokes or mouse clicks---will fail. Second, existing macro recording systems typically give the user only a single chance to demonstrate the repetitive task. However, the recorded macro will be expected to work in a multitude of different situations. As a consequence, the user must think hard about the different situations that might arise and perform the macro is a sufficiently general way so as to cover all those situations.

This research aims to address these three drawbacks by combining a simple macro recording interface with AI machine learning techniques. Our goal is to apply algorithms from AI to construct a robust, domain-independent programming by demonstration system.

For more information, please visit Tessa's SMARTedit page.

Publications

• Lau, Domingos, and Weld. Version space algebra and its application to programming by demonstration. In Proceedings of the International Conference on Machine Learning (ICML). 2000.
• Lau and Weld. Programming by Demonstration: an Inductive Learning Formulation . . In Proceedings of the 1999 International Conference on Intelligent User Interfaces (IUI 99). 1999.