在Keras中，可以使用ModelCheckpoint回调来保存模型的权重和参数。这个回调函数可以在每个epoch结束时将模型写入磁盘，并且可以配置为仅保存表现最好的几个epoch。从磁盘恢复已保存的模型后，您可以使用Model.fit来加载预训练的权重并继续训练。
下面是一个完整的示例代码，展示了如何使用ModelCheckpoint回调以及如何保存并加载预训练模型：
from keras.callbacks import ModelCheckpoint
from keras.models import Sequential
from keras.layers import Dense
# 创建简单的神经网络模型
model = Sequential()
model.add(Dense(units=64, activation='relu', input_dim=100))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='sgd')
# 创建ModelCheckpoint回调，以每个epoch保存最佳模型
checkpoint = ModelCheckpoint('weights.{epoch:02d}-{val_loss:.2f}.hdf5',
monitor='val_loss',
verbose=1,
save_best_only=True,
mode='min')
# 模拟一些数据用于训练
import numpy as np
x_train = np.random.random((1000, 100))
y_train = np.random.randint(2, size=(1000, 1))
# 使用回调函数训练模型
model.fit(x_train, y_train,
epochs=50,
batch_size=128,
callbacks=[checkpoint],
validation_split=0.2)
# 加载预训练的权重并继续训练
model.load_weights('weights.10-0.60.hdf5')
model.fit(x_train, y_train,
epochs=100,
batch_size=128,
callbacks=[checkpoint],
validation_split=0.2)
在这个示例中，ModelCheckpoint回调在每个epoch结束时都会检查验证集的性能，并将表现最佳的模型权重保存到磁盘上。然后，我们加载了第10轮的预训练权重，并继续训练模型100轮。
注意，在使用ModelCheckpoint时，文件名中的{ }表示回调函数将使用该参数来格式化保存的模型文件名。这里，我们使用了.epoch和.val_loss变量，它们分别代表当前epoch数和验证集损失函数的值。这样可以让文件名包含有关模型的有用信息，并避免覆盖先前的模型保存。
问题描述
我有一个已经训练了 40 个 epoch 的模型.我为每个时期保留了检查点，并且我还使用 model.save() 保存了模型.训练代码为:
I have a model that I've trained for 40 epochs. I kept checkpoints for each epochs, and I have also saved the model with model.save(). The code for training is:
n_units = 1000
model = Sequential()
model.add(LSTM(n_units, input_shape=(None, vec_size), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(n_units, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(n_units))
model.add(Dropout(0.2))
model.add(Dense(vec_size, activation='linear'))
model.compile(loss='mean_squared_error', optimizer='adam')
# define the checkpoint
filepath="word2vec-{epoch:02d}-{loss:.4f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
# fit the model
model.fit(x, y, epochs=40, batch_size=50, callbacks=callbacks_list)
然而，当我加载模型并再次尝试训练时，它会从头开始，就好像它之前没有训练过一样.损失不是从上次训练开始的.
However, when I load the model and try training it again, it starts all over as if it hasn't been trained before. The loss doesn't start from the last training.
令我困惑的是，当我加载模型并重新定义模型结构并使用 load_weight 时，model.predict() 运行良好.因此，我相信模型权重已加载:
What confuses me is when I load the model and redefine the model structure and use load_weight, model.predict() works well. Thus, I believe the model weights are loaded:
model = Sequential()
model.add(LSTM(n_units, input_shape=(None, vec_size), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(n_units, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(n_units))
model.add(Dropout(0.2))
model.add(Dense(vec_size, activation='linear'))
filename = "word2vec-39-0.0027.hdf5"
model.load_weights(filename)
model.compile(loss='mean_squared_error', optimizer='adam')
然而，当我继续训练时，损失和初始阶段一样高:
However, When I continue training with this, the loss is as high as the initial stage:
filepath="word2vec-{epoch:02d}-{loss:.4f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
# fit the model
model.fit(x, y, epochs=40, batch_size=50, callbacks=callbacks_list)
我搜索并找到了一些保存和加载模型的示例这里 和 此处.但是，它们都不起作用.
I searched and found some examples of saving and loading models here and here. However, none of them work.
更新 1
我看了这个问题，试过了它有效:
I looked at this question, tried it and it works:
model.save('partly_trained.h5')
del model
load_model('partly_trained.h5')
但是当我关闭 Python 并重新打开它，然后再次运行 load_model 时，它失败了.损失与初始状态一样高.
But when I close Python and reopen it, then run load_model again, it fails. The loss is as high as the initial state.
更新 2
我尝试了Yu-Yang 的示例代码，并且有效.但是，当我再次使用我的代码时，它仍然失败.
I tried Yu-Yang's example code and it works. However, when I use my code again, it still failed.
这是原始训练的结果.第二个 epoch 应该从 loss = 3.1*** 开始:
This is result form the original training. The second epoch should start with loss = 3.1***:
13700/13846 [============================>.] - ETA: 0s - loss: 3.0519
13750/13846 [============================>.] - ETA: 0s - loss: 3.0511
13800/13846 [============================>.] - ETA: 0s - loss: 3.0512Epoch 00000: loss improved from inf to 3.05101, saving model to LPT-00-3.0510.h5
13846/13846 [==============================] - 81s - loss: 3.0510
Epoch 2/60
50/13846 [..............................] - ETA: 80s - loss: 3.1754
100/13846 [..............................] - ETA: 78s - loss: 3.1174
150/13846 [..............................] - ETA: 78s - loss: 3.0745
我关闭 Python，重新打开它，用 model = load_model("LPT-00-3.0510.h5") 加载模型，然后训练:
I closed Python, reopened it, loaded the model with model = load_model("LPT-00-3.0510.h5") then train with:
filepath="LPT-{epoch:02d}-{loss:.4f}.h5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
# fit the model
model.fit(x, y, epochs=60, batch_size=50, callbacks=callbacks_list)
损失从 4.54 开始:
The loss starts with 4.54:
Epoch 1/60
50/13846 [..............................] - ETA: 162s - loss: 4.5451
100/13846 [..............................] - ETA: 113s - loss: 4.3835
推荐答案
由于很难弄清楚问题出在哪里，我根据您的代码创建了一个玩具示例，它似乎工作正常.
As it's quite difficult to clarify where the problem is, I created a toy example from your code, and it seems to work alright.
import numpy as np
from numpy.testing import assert_allclose
from keras.models import Sequential, load_model
from keras.layers import LSTM, Dropout, Dense
from keras.callbacks import ModelCheckpoint
vec_size = 100
n_units = 10
x_train = np.random.rand(500, 10, vec_size)
y_train = np.random.rand(500, vec_size)
model = Sequential()
model.add(LSTM(n_units, input_shape=(None, vec_size), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(n_units, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(n_units))
model.add(Dropout(0.2))
model.add(Dense(vec_size, activation='linear'))
model.compile(loss='mean_squared_error', optimizer='adam')
# define the checkpoint
filepath = "model.h5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
# fit the model
model.fit(x_train, y_train, epochs=5, batch_size=50, callbacks=callbacks_list)
# load the model
new_model = load_model(filepath)
assert_allclose(model.predict(x_train),
new_model.predict(x_train),
1e-5)
# fit the model
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
new_model.fit(x_train, y_train, epochs=5, batch_size=50, callbacks=callbacks_list)
模型加载后损失继续减少.(重启python也没问题)
The loss continues to decrease after model loading. (restarting python also gives no problem)
Using TensorFlow backend.
Epoch 1/5
500/500 [==============================] - 2s - loss: 0.3216 Epoch 00000: loss improved from inf to 0.32163, saving model to model.h5
Epoch 2/5
500/500 [==============================] - 0s - loss: 0.2923 Epoch 00001: loss improved from 0.32163 to 0.29234, saving model to model.h5
Epoch 3/5
500/500 [==============================] - 0s - loss: 0.2542 Epoch 00002: loss improved from 0.29234 to 0.25415, saving model to model.h5
Epoch 4/5
500/500 [==============================] - 0s - loss: 0.2086 Epoch 00003: loss improved from 0.25415 to 0.20860, saving model to model.h5
Epoch 5/5
500/500 [==============================] - 0s - loss: 0.1725 Epoch 00004: loss improved from 0.20860 to 0.17249, saving model to model.h5
Epoch 1/5
500/500 [==============================] - 0s - loss: 0.1454 Epoch 00000: loss improved from inf to 0.14543, saving model to model.h5
Epoch 2/5
500/500 [==============================] - 0s - loss: 0.1289 Epoch 00001: loss improved from 0.14543 to 0.12892, saving model to model.h5
Epoch 3/5
500/500 [==============================] - 0s - loss: 0.1169 Epoch 00002: loss improved from 0.12892 to 0.11694, saving model to model.h5
Epoch 4/5
500/500 [==============================] - 0s - loss: 0.1097 Epoch 00003: loss improved from 0.11694 to 0.10971, saving model to model.h5
Epoch 5/5
500/500 [==============================] - 0s - loss: 0.1057 Epoch 00004: loss improved from 0.10971 to 0.10570, saving model to model.h5
顺便说一句，重新定义模型后跟 load_weight() 肯定行不通，因为 save_weight() 和 load_weight() 不起作用保存/加载优化器.
BTW, redefining the model followed by load_weight() definitely won't work, because save_weight() and load_weight() does not save/load the optimizer.
这篇关于Keras:如何保存模型并继续训练?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，WP2