（原）tensorflow使用eager在mnist上训练的简单例子
2024-02-24 20:48:01
转载请注明出处：
https://www.cnblogs.com/darkknightzh/p/9989586.html
代码网址：
https://github.com/darkknightzh/trainEagerMnist
参考网址：
https://github.com/tensorflow/models/blob/master/official/mnist/mnist_eager.py
一 总体流程
tensorflow使用eager时，需要下面几句话（如果不使用第三句话，则依旧可以使用静态图）：
import tensorflow as tf
import tensorflow.contrib.eager as tfe
tfe.enable_eager_execution()
tensorflow使用eager模式后，感觉和pytorch一样方便。使用eager后，不需要tf.placeholder，用起来更加方便。
目前貌似tf.keras.layers和tf.layers支持eager，slim不支持。
总体流程如下：
initial optimizer
for I in range(epochs):
for imgs, targets in training_data:
with tf.GradientTape() as tape:
logits = model(imgs, training=True)
loss_value = calc_loss(logits, targets)
grads = tape.gradient(loss_value, model.variables)
optimizer.apply_gradients(zip(grads, model.variables), global_step=step_counter)
update training_accurate, total_loss
test model
save model
二 创建模型
可以使用下面三种方式创建模型
1. 类似pytorch的方式
先在__init__中定义用到的层，然后重载call函数，构建网络。模型前向计算时，会调用call函数。如下面代码所示：
class simpleModel(tf.keras.Model):
def __init__(self, num_classes):
super(simpleModel, self).__init__() input_shape = [28, 28, 1]
data_format = 'channels_last'
self.reshape = tf.keras.layers.Reshape(target_shape=input_shape, input_shape=(input_shape[0] * input_shape[1],)) self.conv1 = tf.keras.layers.Conv2D(16, 5, padding="same", activation='relu')
self.batch1 = tf.keras.layers.BatchNormalization()
self.pool1 = tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format) self.conv2 = tf.keras.layers.Conv2D(32, 5, padding="same", activation='relu')
self.batch2 = tf.keras.layers.BatchNormalization()
self.pool2 = tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format) self.conv3 = tf.keras.layers.Conv2D(64, 5, padding="same", activation='relu')
self.batch3 = tf.keras.layers.BatchNormalization()
self.pool3 = tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format) self.conv4 = tf.keras.layers.Conv2D(64, 5, padding="same", activation='relu')
self.batch4 = tf.keras.layers.BatchNormalization()
self.pool4 = tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format) self.flat = tf.keras.layers.Flatten()
self.fc5 = tf.keras.layers.Dense(1024, activation='relu')
self.batch5 = tf.keras.layers.BatchNormalization() self.fc6 = tf.keras.layers.Dense(num_classes)
self.batch6 = tf.keras.layers.BatchNormalization() def call(self, inputs, training=None):
x = self.reshape(inputs) x = self.conv1(x)
x = self.batch1(x, training=training)
x = self.pool1(x) x = self.conv2(x)
x = self.batch2(x, training=training)
x = self.pool2(x) x = self.conv3(x)
x = self.batch3(x, training=training)
x = self.pool3(x) x = self.conv4(x)
x = self.batch4(x, training=training)
x = self.pool4(x) x = self.flat(x)
x = self.fc5(x)
x = self.batch5(x, training=training) x = self.fc6(x)
x = self.batch6(x, training=training)
# x = tf.layers.dropout(x, rate=0.3, training=training)
return x def get_acc(self, target):
correct_prediction = tf.equal(tf.argmax(self.logits, 1), tf.argmax(target, 1))
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
return acc def get_loss(self):
return self.loss def loss_fn(self, images, target, training):
self.logits = self(images, training) # call call(self, inputs, training=None) function
self.loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.logits, labels=target))
return self.loss def grads_fn(self, images, target, training): # do not return loss and acc if unnecessary
with tfe.GradientTape() as tape:
loss = self.loss_fn(images, target, training)
return tape.gradient(loss, self.variables)
2. 直接使用tf.keras.Sequential
如下面代码所示：
def create_model1():
data_format = 'channels_last'
input_shape = [28, 28, 1]
l = tf.keras.layers
max_pool = l.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format)
# The model consists of a sequential chain of layers, so tf.keras.Sequential (a subclass of tf.keras.Model) makes for a compact description.
return tf.keras.Sequential(
[
l.Reshape(target_shape=input_shape, input_shape=(28 * 28,)),
l.Conv2D(16, 5, padding='same', data_format=data_format, activation=tf.nn.relu),
l.BatchNormalization(),
max_pool, l.Conv2D(32, 5, padding='same', data_format=data_format, activation=tf.nn.relu),
l.BatchNormalization(),
max_pool, l.Conv2D(64, 5, padding='same', data_format=data_format, activation=tf.nn.relu),
l.BatchNormalization(),
max_pool, l.Conv2D(64, 5, padding='same', data_format=data_format, activation=tf.nn.relu),
l.BatchNormalization(),
max_pool, l.Flatten(),
l.Dense(1024, activation=tf.nn.relu),
l.BatchNormalization(), # # l.Dropout(0.4),
l.Dense(10),
l.BatchNormalization()
])
3. 使用tf.keras.Sequential()及add函数
如下面代码所示：
def create_model2():
data_format = 'channels_last'
input_shape = [28, 28, 1] model = tf.keras.Sequential() model.add(tf.keras.layers.Reshape(target_shape=input_shape, input_shape=(input_shape[0] * input_shape[1],))) model.add(tf.keras.layers.Conv2D(16, 5, padding="same", activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format)) model.add(tf.keras.layers.Conv2D(32, 5, padding="same", activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format)) model.add(tf.keras.layers.Conv2D(64, 5, padding="same", activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format)) model.add(tf.keras.layers.Conv2D(64, 5, padding="same", activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same', data_format=data_format)) model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(1024, activation='relu'))
model.add(tf.keras.layers.BatchNormalization()) model.add(tf.keras.layers.Dense(10))
model.add(tf.keras.layers.BatchNormalization()) return model
三 使用动态图更新梯度
在更新梯度时，需要加上下面的几句话
with tf.GradientTape() as tape:
logits = model(imgs, training=True)
loss_value = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=labs))
grads = tape.gradient(loss_value, model.variables)
optimizer.apply_gradients(zip(grads, model.variables), global_step=step_counter)
第二行得到特征，第三行得到损失，第四行得到梯度，第五行将梯度应用到模型，更新模型参数。
四 保存及载入模型
1. 使用tfe.Saver
代码如下
def saveModelV1(model_dir, model, global_step, modelname='model1'):
tfe.Saver(model.variables).save(os.path.join(model_dir, modelname), global_step=global_step)
def restoreModelV1(model_dir, model):
dummy_input = tf.constant(tf.zeros((1, 28, 28, 1))) # Run the model once to initialize variables
dummy_pred = model(dummy_input, training=False) saver = tfe.Saver(model.variables) # Restore the variables of the model
saver.restore(tf.train.latest_checkpoint(model_dir))
2. 使用tf.train.Checkpoint
代码如下
step_counter = tf.train.get_or_create_global_step()
checkpoint = tf.train.Checkpoint(model=model, optimizer=optimizer, step_counter=step_counter) def saveModelV2(model_dir, checkpoint, modelname='model2'):
checkpoint_prefix = os.path.join(model_dir, modelname)
checkpoint.save(checkpoint_prefix) def restoreModelV2(model_dir, checkpoint):
checkpoint.restore(tf.train.latest_checkpoint(model_dir))
五 具体代码
代码未严格按照总体流程的步骤，仅供参考，见https://github.com/darkknightzh/trainEagerMnist
其中eagerFlag为使用eager的方式，0为不使用eager（使用静态图），1为使用V1的方式，2为使用V2的方式。当使用静态图时，不要加tfe.enable_eager_execution()，否则会报错。具体可参考代码。
最新文章
- JVM学习笔记：虚拟机的类加载机制
- Caffe学习系列(9)：solver优化方法
- java模拟一个简单的QQ
- 游戏 tabpanel
- cocos2dx新建android项目lib拷贝、访问权限等问题集
- 百度的android采访分析
- iOS开发系统版本适配（未完待续。。。）
- iOS 多线程NSThread理解与场景示例
- TransactionTemplate编程式事务管理方式的进阶使用---自定义拓展模板类
- 通过Git Gui Here上传本地项目到GitHub上
- PAT乙级-1037. 在霍格沃茨找零钱（20）
- [Swift]LeetCode765. 情侣牵手 | Couples Holding Hands
- 给有C或C++基础的Python入门 ：Python Crash Course 4 操作列表 4.1--4.3
- AI制作icon标准参考线与多面板复制
- UVA 10382 Watering Grass（区间覆盖，贪心）题解
- tp3.2 支付宝手机网站支付
- lis nlogn算法
- 使用 github Pages 服务建立个人独立博客全过程
- php history.back返回后表单数据丢失的解决办法
- 给HTML初学者的三十条最佳实践