#### Overview of this book

Preface
Section 1:The Methods
Free Chapter
Chapter 1: Evaluating Machine Learning Models
Chapter 2: Introducing Hyperparameter Tuning
Chapter 3: Exploring Exhaustive Search
Chapter 4: Exploring Bayesian Optimization
Chapter 5: Exploring Heuristic Search
Chapter 6: Exploring Multi-Fidelity Optimization
Section 2:The Implementation
Chapter 7: Hyperparameter Tuning via Scikit
Chapter 8: Hyperparameter Tuning via Hyperopt
Chapter 9: Hyperparameter Tuning via Optuna
Chapter 10: Advanced Hyperparameter Tuning with DEAP and Microsoft NNI
Section 3:Putting Things into Practice
Chapter 11: Understanding the Hyperparameters of Popular Algorithms
Chapter 12: Introducing Hyperparameter Tuning Decision Map
Chapter 13: Tracking Hyperparameter Tuning Experiments
Chapter 14: Conclusions and Next Steps
Other Books You May Enjoy

# Discovering LPO cross-validation

LPO cross-validation is a variation of the LOO cross-validation strategy, where the validation set in each fold contains p samples instead of only 1 sample. Similar to LOO, this strategy will ensure that we get all possible combinations of train-validation pairs. To be more precise, there will be number of folds assuming there are n samples on our data. For example, there will be or 142,506 folds if we want to perform Leave-5-Out cross-validation on data that has 50 samples.

LPO is suitable when you have a small number of samples and want to get even higher confidence in the model's estimated performance compared to the LOO method. LPO will result in an exploding number of folds when you have a large number of samples.

This strategy is a bit different from k-fold or LOO in terms of the overlapping between the validation sets. For P > 1, LPO will result in overlapping validation sets, while k-fold and LOO will always result in non-overlapping validation sets. Also, note that LPO is different from k-fold with K = N // P since k-fold will always create non-overlapping validation sets, but not with the LPO strategy:

```from sklearn.model_selection import train_test_split, LeavePOut
df_cv, df_test = train_test_split(df, test_size=0.2, random_state=0)
lpo = LeavePOut(p=2)
for train_index, val_index in lpo.split(df_cv):
df_train, df_val = df_cv.iloc[train_index], df_cv.iloc[val_index]
#perform training or hyperparameter tuning here```

Unlike LOO, we have to provide the `p` argument to LPO, which refers to the p values in LPO.

In this section, we have learned about the variations of the LOO cross-validation strategy. In the next section, we will learn how to perform cross-validation on time-series data.