In the previous section, we set a KNN parameter maximizing the performance. Another tuning option is to define which data we use to build the model. Our table describes the flags using 37 features and we included all of them in the model. However, KNN might perform better including only a subset of them.

The easiest way to select the features is to use a filter (as anticipated in the Ranking the features using a filter or a dimensionality reduction section in Chapter 4, Step 1 – Data Exploration and Feature Engineering) that estimates the impact of each feature and includes only the most relevant features. After ranking all the features on the basis of their relevance, we can define the n parameters specifying how many of them we include in the model. Then, we can maximize the accuracy depending on n, using an approach similar to the previous section.

The first step is defining how to rank the features. We can use the information gain ratio...