There is always room for improvement in the accuracy of any model. In this section, we will talk about some of the parameters that can be tweaked to improve our model accuracy score of 87.5%
obtained from the previous section.
This section walks through the steps to fine-tune the model.
- Define a new logistic regression model with additional parameters for
regParam
andelasticNetParam
as seen in the following script:
logregFT = LogisticRegression( regParam=0.05, elasticNetParam=0.3, maxIter=15,labelCol = "label", featuresCol="features")
- Create a new pipeline configured for the newly created model using the following script:
pipelineFT = Pipeline(stages=[vectorizer, logregFT])
- Fit the pipeline to the trained dataset,
trainDF
, using the following script:
pipeline_model_FT = pipelineFT.fit(trainDF)
- Apply the model transformation to the test dataset,
testDF
, to be able to compare actual versus...