The main difference between filter and wrapper approaches is that in filter approaches, such as embedded methods, you cannot separate the learning and feature selection parts.
Regularization methods are the most common type of embedded feature selection methods.
In classification problems such as this one, the logistic regression method cannot handle the multi-collinearity problem, which occurs when variables are very correlated. When the number of observations is not much larger than the number of variables of covariates, p, then there can be a lot of variability. Consequently, this variability could even increase the likelihood by simply adding more parameters, resulting in overfitting.
If variables are highly correlated or if collinearity exists, we expect the model parameters and variance to be inflated. The high variance is because of the wrongly specified...
