Ensemble Methods

=Ensemble Methods=

AdaBoost
Robert Schapire did his Phd thesis on boosting. Since then he has done many development in the boosting. He gave a nice introduction to the depth of boosting in the machine learning summer school

In depth tutorial of Adaboost by Robert Schapire

He also has the book Boosting: Boosting Foundation and Algorithms

Boosting can be view as the zero sum game playing using minmax

Gradient Boost
There has been cases where the overfitting will be problem in the adaboost. so the solution is to move in the direction of function gradient. This idea is illustrated in

Gradient Boosting by Mason etal

a tutorial on gradient boosting by Friedman