# Decision tree learning

**decision treesdecision treeClassification and regression treeCARTGini impurityclassification and regression treesclassification treeDecision tree algorithmsmachine learningrecursive partitioning**

In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves).wikipedia

111 Related Articles

### Decision tree

**decision treesdecision rulesRegression trees**

In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves).

Decision trees are commonly used in operations research, specifically in decision analysis, to help identify a strategy most likely to reach a goal, but are also a popular tool in machine learning.

### Data mining

**data-miningdataminingknowledge discovery in databases**

It is one of the predictive modeling approaches used in statistics, data mining and machine learning.

As data sets have grown in size and complexity, direct "hands-on" data analysis has increasingly been augmented with indirect, automated data processing, aided by other discoveries in computer science, such as neural networks, cluster analysis, genetic algorithms (1950s), decision trees and decision rules (1960s), and support vector machines (1990s).

### Recursive partitioning

**recursive-partitioning analysis**

This process is repeated on each derived subset in a recursive manner called recursive partitioning.

Recursive partitioning creates a decision tree that strives to correctly classify members of the population by splitting it into sub-populations based on several dichotomous independent variables.

### Gradient boosting

**boosted decision treeBoosted treesboosted decision trees**

Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees.

### ID3 algorithm

**ID3Iterative Dichotomiser 3**

Used by the ID3, C4.5 and C5.0 tree-generation algorithms.

In decision tree learning, ID3 (Iterative Dichotomiser 3) is an algorithm invented by Ross Quinlan used to generate a decision tree from a dataset.

### Random forest

**random forestsRandom multinomial logitRandom naive Bayes**

Random forests or random decision forests are an ensemble learning method for classification, regression and other tasks that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees.

### AdaBoost

**AdaBoost algorithm**

Every learning algorithm tends to suit some problem types better than others, and typically has many different parameters and configurations to adjust before it achieves optimal performance on a dataset, AdaBoost (with decision trees as the weak learners) is often referred to as the best out-of-the-box classifier.

### Leo Breiman

**Breiman, LeoBreiman**

The term Classification And Regression Tree (CART) analysis is an umbrella term used to refer to both of the above procedures, first introduced by Breiman et al. in 1984.

His most important contributions were his work on classification and regression trees and ensembles of trees fit to bootstrap samples.

### Chi-square automatic interaction detection

**CHAIDChi-squared Automatic Interaction Detection**

Chi-square automatic interaction detection (CHAID) is a decision tree technique, based on adjusted significance testing (Bonferroni testing).

### Bootstrap aggregating

**baggingBootstrap aggregationbagged nearest neighbour classifier**

Although it is usually applied to decision tree methods, it can be used with any type of method.

### Greedy algorithm

**greedygreedilygreedy heuristic**

This process of top-down induction of decision trees (TDIDT) is an example of a greedy algorithm, and it is by far the most common strategy for learning decision trees from data.

### Multivariate adaptive regression spline

**Multivariate adaptive regression splinesHinge functionsMARS**

see the recursive partitioning article for details).

### Predictive analytics

**predictiveCARTpredictive analysis**

Classification and regression trees (CART) are a non-parametric decision tree learning technique that produces either classification or regression trees, depending on whether the dependent variable is categorical or numeric, respectively.

### Information gain in decision trees

**information gain**

Such a sequence (which depends on the outcome of the investigation of previous attributes at each stage) is called a decision tree and applied in the area of machine learning known as decision tree learning.

### Feature selection

**variable selectionfeaturesselecting**

### Decision tree pruning

**pruningprunedPruning (decision trees)**

Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances.

### Statistics

**statisticalstatistical analysisstatistician**

In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves). It is one of the predictive modeling approaches used in statistics, data mining and machine learning.

### Predictive modelling

**predictive modelingpredictive modelpredictive models**

In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves).

### Machine learning

**machine-learninglearningstatistical learning**

It is one of the predictive modeling approaches used in statistics, data mining and machine learning.

### Logical conjunction

**conjunctionANDlogical AND**

Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.

### Decision-making

**decision makingdecisionsdecision**

In decision analysis, a decision tree can be used to visually and explicitly represent decisions and decision making.

### Tree (data structure)

**treetree data structuretrees**

Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.

### Feature (machine learning)

**feature vectorfeature spacefeatures**

For this section, assume that all of the input features have finite discrete domains, and there is a single target feature called the "classification".

### Set (mathematics)

**setsetsmathematical set**

A tree is built by splitting the source set, constituting the root node of the tree, into subsets - which constitute the successor children.