# Training, validation, and test sets

**training settraining datatest setvalidation setdatasetHoldouttraining datasettraining examplesavailable textcalibration set**

In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data.wikipedia

97 Related Articles

### Machine learning

**machine-learninglearningstatistical learning**

In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data.

Machine learning algorithms build a mathematical model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to perform the task.

### Cross-validation (statistics)

**cross-validationcross validationLeave-one-out cross-validation**

If the data in the test dataset has never been used in training (for example in cross-validation), the test dataset is also called a holdout dataset.

In a prediction problem, a model is usually given a dataset of known data on which training is run (training dataset), and a dataset of unknown data (or first seen data) against which the model is tested (called the validation dataset or testing set).

### Overfitting

**overfitover-fitover-fitted**

Validation datasets can be used for regularization by early stopping: stop training when the error on the validation dataset increases, as this is a sign of overfitting to the training dataset.

For example, a model might be selected by maximizing its performance on some set of training data, and yet its suitability might be determined by its ability to perform well on unseen data; then overfitting occurs when a model begins to "memorize" training data rather than "learning" to generalize from a trend.

### Statistical classification

**classificationclassifierclassifiers**

A training dataset is a dataset of examples used for learning, that is to fit the parameters (e.g., weights) of, for example, a classifier.

In machine learning and statistics, classification is the problem of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known.

### Early stopping

Validation datasets can be used for regularization by early stopping: stop training when the error on the validation dataset increases, as this is a sign of overfitting to the training dataset.

These early stopping rules work by splitting the original training set into a new training set and a validation set.

### Algorithm

**algorithmsalgorithm designcomputer algorithm**

In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data.

### Data

**statistical datascientific datadatum**

In machine learning, a common task is the study and construction of algorithms that can learn from and make predictions on data.

### Mathematical model

**mathematical modelingmodelmathematical models**

Such algorithms work by making data-driven predictions or decisions, through building a mathematical model from input data.

### Data set

**datasetdatasetsdata sets**

A training dataset is a dataset of examples used for learning, that is to fit the parameters (e.g., weights) of, for example, a classifier. The data used to build the final model usually comes from multiple datasets. A test dataset is a dataset that is independent of the training dataset, but that follows the same probability distribution as the training dataset.

### Artificial neural network

**artificial neural networksneural networksneural network**

The model is initially fit on a training dataset, that is a set of examples used to fit the parameters (e.g. weights of connections between neurons in artificial neural networks) of the model. The model (e.g. a neural net or a naive Bayes classifier) is trained on the training dataset using a supervised learning method (e.g. gradient descent or stochastic gradient descent).

### Naive Bayes classifier

**Naive Bayesnaive Bayes classificationNaïve Bayes**

The model (e.g. a neural net or a naive Bayes classifier) is trained on the training dataset using a supervised learning method (e.g. gradient descent or stochastic gradient descent).

### Supervised learning

**supervisedsupervised machine learningsupervised classification**

The model (e.g. a neural net or a naive Bayes classifier) is trained on the training dataset using a supervised learning method (e.g. gradient descent or stochastic gradient descent).

### Gradient descent

**steepest descentgradient ascentgradient**

The model (e.g. a neural net or a naive Bayes classifier) is trained on the training dataset using a supervised learning method (e.g. gradient descent or stochastic gradient descent).

### Stochastic gradient descent

**AdaGradAdaptive Moment Estimationgradient-based learning methods**

### Array data structure

**arrayarraysvector**

In practice, the training dataset often consist of pairs of an input vector (or scalar) and the corresponding output vector (or scalar), which is commonly denoted as the target (or label).

### Feature selection

**variable selectionfeaturesselecting**

The model fitting can include both variable selection and parameter estimation.

### Estimation theory

**parameter estimationestimationestimated**

The model fitting can include both variable selection and parameter estimation.

### Hyperparameter (machine learning)

**hyperparametershyperparameterhyper-parameter**

The validation dataset provides an unbiased evaluation of a model fit on the training dataset while tuning the model's hyperparameters (e.g. the number of hidden units in a neural network ).

### Regularization (mathematics)

**regularizationregularizedregularize**

Validation datasets can be used for regularization by early stopping: stop training when the error on the validation dataset increases, as this is a sign of overfitting to the training dataset.

### Independence (probability theory)

**independentstatistically independentindependence**

A test dataset is a dataset that is independent of the training dataset, but that follows the same probability distribution as the training dataset.

### Probability distribution

**distributioncontinuous probability distributiondiscrete probability distribution**

A test dataset is a dataset that is independent of the training dataset, but that follows the same probability distribution as the training dataset.

### Accuracy and precision

**accuracyprecisionaccurate**

For example, if the most suitable classifier for the problem is sought, the training dataset is used to train the candidate algorithms, the validation dataset is used to compare their performances and decide which one to take and, finally, the test dataset is used to obtain the performance characteristics such as accuracy, sensitivity, specificity, F-measure, and so on.

### Sensitivity and specificity

**sensitivityspecificitysensitive**

For example, if the most suitable classifier for the problem is sought, the training dataset is used to train the candidate algorithms, the validation dataset is used to compare their performances and decide which one to take and, finally, the test dataset is used to obtain the performance characteristics such as accuracy, sensitivity, specificity, F-measure, and so on.

### Precision and recall

**precisionrecallF-measure**

For example, if the most suitable classifier for the problem is sought, the training dataset is used to train the candidate algorithms, the validation dataset is used to compare their performances and decide which one to take and, finally, the test dataset is used to obtain the performance characteristics such as accuracy, sensitivity, specificity, F-measure, and so on.

### Model selection

**statistical model selectionselectingchoose a model**

The basic process of using a validation dataset for model selection (as part of training dataset, validation dataset, and test dataset) is: