Training the model

Now we can start training the model. In this example, we chose to train the model using SGD with a batch size of 64 and 100 epochs. To validate the model, we randomly selected 16 words and used the similarity measure as an evaluation metric:

1. Let's begin training:

   valid_size = 16     # Random set of words to evaluate similarity on.
   valid_window = 100  # Only pick dev samples in the head of the distribution.
   valid_examples = np.array(np.random.choice(valid_window, valid_size, replace=False), dtype='int32')
   
   n_epochs = 100
   n_train_batches = data_size // batch_size
   n_iters = n_epochs * n_train_batches
   train_loss = np.zeros(n_iters)
   average_loss = 0
   
   for epoch in range(n_epochs):
       for minibatch_index in range(n_train_batches):
   
           iteration = minibatch_index + n_train_batches * epoch
           loss = train_model(minibatch_index)
           train_loss[iteration] = loss
           average_loss += loss
   
   
           if iteration % 2000 == 0:
   
             if iteration > 0:
           ...