07-22-2010, 01:25 AM
Q3)
Avoiding over-fitting the data
In principle decision tree algorithm described in Figure 2 can grow each branch of the tree just deeply enough to perfectly classify the training examples. While this is sometimes a reasonable strategy, in fact it can lead to difficulties when there is noise in the data, or when the number of training examples is too small or too large to produce a representative sample of the true target function. In either of these cases, this simple algorithm can produce trees that over-fit the training examples.
Avoiding over-fitting the data
In principle decision tree algorithm described in Figure 2 can grow each branch of the tree just deeply enough to perfectly classify the training examples. While this is sometimes a reasonable strategy, in fact it can lead to difficulties when there is noise in the data, or when the number of training examples is too small or too large to produce a representative sample of the true target function. In either of these cases, this simple algorithm can produce trees that over-fit the training examples.