Classifying with BERT

While BERT was put in action previously in the chapter, we didn’t perform any fine-tuning tasks using our dataset. So, this step is part of the current section. Moreover, we need to create a neural architecture consisting of several parts to perform classification. The build_classifier_model method that follows shows the structure of the specific architecture:

# Method to build the classifier model.
def build_classifier_model():
  text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text')
  preprocessing_layer = hub.KerasLayer(tfhub_handle_preprocess, name='preprocessing')
  encoder_inputs = preprocessing_layer(text_input)
  encoder = hub.KerasLayer(tfhub_handle_encoder, trainable=True, name='BERT_encoder')
  outputs = encoder(encoder_inputs)
  net = outputs['pooled_output']
  net = tf.keras.layers.Dropout(0.1)(net)
  ...