Classifying with BERT
While BERT was put in action previously in the chapter, we didn’t perform any fine-tuning tasks using our dataset. So, this step is part of the current section. Moreover, we need to create a neural architecture consisting of several parts to perform classification. The
build_classifier_model method that follows shows the structure of the specific architecture:
# Method to build the classifier model. def build_classifier_model(): text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text') preprocessing_layer = hub.KerasLayer(tfhub_handle_preprocess, name='preprocessing') encoder_inputs = preprocessing_layer(text_input) encoder = hub.KerasLayer(tfhub_handle_encoder, trainable=True, name='BERT_encoder') outputs = encoder(encoder_inputs) net = outputs['pooled_output'] net = tf.keras.layers.Dropout(0.1)(net) ...