Location: Plant, Soil and Nutrition Research
Title: Ensemble learning with trees and rules: supervised, semi-supervised, unsupervised Authors
|Akdemir, Deniz -|
Submitted to: Intelligent Data Analysis (An International Journal)
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: April 14, 2013
Publication Date: N/A
Interpretive Summary: One approach to prediction involves using predictor variables to develop a large number of rules, called an ensemble, that split observations into different and divergent groups. These rules are then combined in a processing step to obtain a prediction for each individual. In this article, we propose several new approaches for processing a large ensemble of rules to generate accurate predictions and estimate relationships among individuals. We show with various examples that for regression problems with many predictors the models constructed by processing the rules with statistical method called partial least squares regression have significantly better prediction performance than processing using other existing methods. When rule ensembles are used to estimate relationships among individuals and cluster them, measures of cluster validity indicate high quality groupings.
Technical Abstract: In this article, we propose several new approaches for post processing a large ensemble of conjunctive rules for supervised and semi-supervised learning problems. We show with various examples that for high dimensional regression problems the models constructed by the post processing the rules with partial least squares regression have significantly better prediction performance than the ones produces by the random forest or the rulefit algorithms which use equal weights or weights estimated from lasso regression. When rule ensembles are used for semi-supervised and unsupervised learning, the internal and external measures of cluster validity point to high quality groupings.