Decision tree learning

decision treesdecision treeClassification and regression treeCARTGini impurityclassification and regression treesclassification treeDecision tree algorithmsmachine learningrecursive partitioning
In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves).wikipedia
111 Related Articles

Decision tree

decision treesdecision rulesRegression trees
In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves).
Decision trees are commonly used in operations research, specifically in decision analysis, to help identify a strategy most likely to reach a goal, but are also a popular tool in machine learning.

Data mining

data-miningdataminingknowledge discovery in databases
It is one of the predictive modeling approaches used in statistics, data mining and machine learning.
As data sets have grown in size and complexity, direct "hands-on" data analysis has increasingly been augmented with indirect, automated data processing, aided by other discoveries in computer science, such as neural networks, cluster analysis, genetic algorithms (1950s), decision trees and decision rules (1960s), and support vector machines (1990s).

Recursive partitioning

recursive-partitioning analysis
This process is repeated on each derived subset in a recursive manner called recursive partitioning.
Recursive partitioning creates a decision tree that strives to correctly classify members of the population by splitting it into sub-populations based on several dichotomous independent variables.

boosted decision treeBoosted treesboosted decision trees
Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees.

ID3 algorithm

ID3Iterative Dichotomiser 3
Used by the ID3, C4.5 and C5.0 tree-generation algorithms.
In decision tree learning, ID3 (Iterative Dichotomiser 3) is an algorithm invented by Ross Quinlan used to generate a decision tree from a dataset.

Random forest

random forestsRandom multinomial logitRandom naive Bayes
Random forests or random decision forests are an ensemble learning method for classification, regression and other tasks that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees.

Every learning algorithm tends to suit some problem types better than others, and typically has many different parameters and configurations to adjust before it achieves optimal performance on a dataset, AdaBoost (with decision trees as the weak learners) is often referred to as the best out-of-the-box classifier.

Leo Breiman

Breiman, LeoBreiman
The term Classification And Regression Tree (CART) analysis is an umbrella term used to refer to both of the above procedures, first introduced by Breiman et al. in 1984.
His most important contributions were his work on classification and regression trees and ensembles of trees fit to bootstrap samples.

Chi-square automatic interaction detection

CHAIDChi-squared Automatic Interaction Detection
Chi-square automatic interaction detection (CHAID) is a decision tree technique, based on adjusted significance testing (Bonferroni testing).

Bootstrap aggregating

baggingBootstrap aggregationbagged nearest neighbour classifier
Although it is usually applied to decision tree methods, it can be used with any type of method.

Greedy algorithm

greedygreedilygreedy heuristic
This process of top-down induction of decision trees (TDIDT) is an example of a greedy algorithm, and it is by far the most common strategy for learning decision trees from data.

see the recursive partitioning article for details).

Predictive analytics

predictiveCARTpredictive analysis
Classification and regression trees (CART) are a non-parametric decision tree learning technique that produces either classification or regression trees, depending on whether the dependent variable is categorical or numeric, respectively.

Information gain in decision trees

information gain
Such a sequence (which depends on the outcome of the investigation of previous attributes at each stage) is called a decision tree and applied in the area of machine learning known as decision tree learning.

Feature selection

variable selectionfeaturesselecting

Decision tree pruning

pruningprunedPruning (decision trees)
Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances.

Statistics

statisticalstatistical analysisstatistician
In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves). It is one of the predictive modeling approaches used in statistics, data mining and machine learning.

Predictive modelling

predictive modelingpredictive modelpredictive models
In statistics, Decision tree learning uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves).

Machine learning

machine-learninglearningstatistical learning
It is one of the predictive modeling approaches used in statistics, data mining and machine learning.

Logical conjunction

conjunctionANDlogical AND
Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.

Decision-making

decision makingdecisionsdecision
In decision analysis, a decision tree can be used to visually and explicitly represent decisions and decision making.

Tree (data structure)

treetree data structuretrees
Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.

Feature (machine learning)

feature vectorfeature spacefeatures
For this section, assume that all of the input features have finite discrete domains, and there is a single target feature called the "classification".

Set (mathematics)

setsetsmathematical set
A tree is built by splitting the source set, constituting the root node of the tree, into subsets - which constitute the successor children.