Process-Oriented Evaluation

Overview

Current methods to avoid overfitting are either data-oriented (using separate data for validation) or representation-oriented (penalizing complexity in the model). In this project we develop process-oriented evaluation, where a model's expected generalization error is computed as a function of the search process that led to it. The project develops the necessary theoretical framework, and applies it to different types of learning.

Publications

• Domingos. The Role of Occam's Razor in Knowledge Discovery. In Data Mining and Knowledge Discovery. 1999.
• Domingos. Process-Oriented Estimation of Generalization Error. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. 1999.
• Domingos. A Process-Oriented Heuristic for Model Selection. In Proceedings of the Fifteenth International Conference on Machine Learning. 1998.
• Domingos. Occam's Two Razors: The Sharp and the Blunt. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining. 1998.