Cross-Validation Machine Learning Models with scikit-learn
Cross-validation is a technique for evaluating ML models by training several ML models on subsets of the available input data and evaluating them on the complementary subset of the data. Use cross-validation to detect overfitting, ie, failing to generalize a pattern.
We can solve the overlapping problems via Cross-Validation.
Model validation helps in ensuring that the model performs well on new data, and helps in selecting the best model, the parameters, and the accuracy metrics.
There are several model validation techniques, mentioned below:
- Hold Out Validation
- K-fold Cross-Validation.
- Stratified K-fold Cross-Validation
- Leave One Out Cross-Validation.
- Repeated Random Test-Train Splits
k-fold cross-validation
In k-fold cross-validation, there will be k data sets in my complete data, First time, k-1 data set will be test data and remaining are training dataset. Second time, another k-1 is testing and remaining are training data, it will repeat k times and never repeat any data second time for test data. Never will be overlap data in training and testing data set. Every data set will be pass-through testing and training data sets.
from sklearn import model_selection
from sklearn.model_selection import KFoldkfold = model_selection.KFold(n_splits=10, random_state=100)
model_kfold = LogisticRegression()
results_kfold = model_selection.cross_val_score(model_kfold, fdf_normalized, y1, cv=kfold)
print("Accuracy: %.2f%%" % (results_kfold.mean()*100.0))
The mean accuracy for the model using k-fold cross-validation is 74.05 percent, which is better than the 72(Old) percent we achieved in the holdout validation approach.
That’s it for this time! I hope you enjoyed this post. As always, I welcome questions, notes, comments and requests for posts on topics you’d like to read. See you next time!
