TensorFlow has some utility functions to warm-start estimators; that is, to initialize some of their layers with pretrained weights. The following snippet tells TensorFlow to use the saved parameters of a pretrained estimator for the new one for the layers sharing the same name:
def model_function():
# ... define new model, reusing pretrained one as feature extractor.
ckpt_path = '/path/to/pretrained/estimator/model.ckpt'
ws = tf.estimator.WarmStartSettings(ckpt_path)
estimator = tf.estimator.Estimator(model_fn, warm_start_from=ws)
The WarmStartSettings initializer takes an optional vars_to_warm_start parameter, which can also be used to provide the names of the specific variables (as a list or a regex) that you want to restore from the checkpoint files (refer to the documentation for more details at https://www.tensorflow.org/api_docs/python/tf/estimator/WarmStartSettings).
With Keras, we can simply restore the pretrained model before...
