Chapter 14 Keep The Best Models During Training With Checkpointing

Question 1

What’s application check-pointing? P 104

Answer 1

Application check-pointing is a fault tolerance technique for long running processes. It is an approach where a snapshot of the state of the system is taken in case of system failure.

Question 2

What do check-points do? P 104

Answer 2

When training deep learning models, the checkpoint captures the weights of the model. These weights can be used to make predictions as-is, or used as the basis for ongoing training.

Question 3

The Keras library provides a checkpointing capability by a…, The … callback class allows you to define where to checkpoint the model weights, how the file should be named and under what circumstances to make a checkpoint of the model. P 104

Answer 3

callback API , ModelCheckpoint

Question 4

The callback API for check-point allows you to specify metrics to monitor, such as loss or accuracy on the training or validation dataset. True/False? P 104

Answer 4

True

Question 5

What does the below code do? P 104

# Checkpoint the weights when validation loss improves
from keras.models import Sequential
from keras.layers import Dense
from keras.callbacks import ModelCheckpoint
import matplotlib.pyplot as plt
import numpy
from sklearn.datasets import load_breast_cancer, load_iris,load_diabetes

dataset =  load_breast_cancer(as_frame=True)
X = dataset.data
Y = dataset.target
model = Sequential()
model.add(Dense(12, input_dim=30,activation= "relu" ))
model.add(Dense(8, activation= "relu" ))
model.add(Dense(1, activation= "sigmoid" ))
# Compile model
model.compile(loss= "binary_crossentropy" , optimizer= "adam" , metrics=[ "accuracy" ])
# checkpoint 
filepath="weights-improvement-{epoch:02d}-{val_loss:.2f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor= "val_loss" , verbose=1, save_best_only=True,
mode= "min" )
callbacks_list = [checkpoint]
# Fit the model
model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, callbacks=callbacks_list, verbose=0)

Answer 5

This is a very simple check-pointing strategy. It may create a lot of unnecessary checkpoint files if the validation loss moves up and down over training epochs. Nevertheless, it will ensure that you have a snapshot of the best model discovered during your run.

Checkpoint Doc

Question 6

How can we create just one file for check-points of a neural network that saves the best weights? P 105

Answer 6

from keras.models import Sequential
from keras.layers import Dense
from keras.callbacks import ModelCheckpoint
import matplotlib.pyplot as plt
import numpy
from sklearn.datasets import load_breast_cancer, load_iris,load_diabetes

dataset =  load_breast_cancer(as_frame=True)
X = dataset.data
Y = dataset.target
model = Sequential()
model.add(Dense(12, input_dim=30,activation= "relu" ))
model.add(Dense(8, activation= "relu" ))
model.add(Dense(1, activation= "sigmoid" ))
# Compile model
model.compile(loss= "binary_crossentropy" , optimizer= "adam" , metrics=[ "accuracy" ])
# checkpoint 
filepath="weights_best.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor= "val_accuracy" , verbose=1, save_best_only=True,
mode= "max" )
callbacks_list = [checkpoint]
# Fit the model
model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, callbacks=callbacks_list, verbose=0)

A simpler checkpoint strategy is to save the model weights to the same file, if and only if the validation accuracy improves. This can be done easily using the same code from above and changing the output filename to be fixed (not include score or epoch information).

Question 7

What’s a callback API in NN? External

Answer 7

A callback is an object that can perform actions at various stages of training (e.g. at the start or end of an epoch, before or after a single batch, etc). You can use callbacks to: Write TensorBoard logs after every batch of training to monitor your metrics. Periodically save your model to disk.

Question 8

How can we load and use a check-pointed model weights? P 106

Answer 8

from keras.models import Sequential
from keras.layers import Dense
from keras.callbacks import ModelCheckpoint
import matplotlib.pyplot as plt
import numpy
from sklearn.datasets import load_breast_cancer, load_iris,load_diabetes
dataset =  load_breast_cancer(as_frame=True)
X = dataset.data
Y = dataset.target
model = Sequential()
model.add(Dense(12, input_dim=30,activation= "relu" ))
model.add(Dense(8, activation= "relu" ))
model.add(Dense(1, activation= "sigmoid" ))
# Compile model
model.compile(loss= "binary_crossentropy" , optimizer= "adam" , metrics=[ "accuracy" ])
# Load weights
model.load_weights("weights.best.hdf5")
print("Created model and loaded weights from file")

scores = model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

Note the abscense of “fit”