PyTorch之初使用
作者:互联网
- 使用流程
①. 数据准备; ②. 模型确立; ③. 损失函数确立; ④. 优化器确立; ⑤. 模型训练及保存 - 模块介绍
Part Ⅰ: 数据准备
torch.utils.data.Dataset
torch.utils.data.DataLoader
关于Dataset, 作为数据集, 需要实现基本的3个方法, 分别为: __init__、__len__、__getitem__. 示例如下,
1 class TrainingDataset(Dataset): 2 3 def __init__(self, X, Y_, transform=None, target_transform=None): 4 self.__X = X 5 self.__Y_ = Y_ 6 self.__transform = transform 7 self.__target_transform = target_transform 8 9 10 def __len__(self): 11 return len(self.__X) 12 13 14 def __getitem__(self, idx): 15 x = self.__X[idx] 16 y_ = self.__Y_[idx] 17 if self.__transform: 18 x = self.__transform(x) 19 if self.__target_transform: 20 y_ = self.__target_transform(y_) 21 return x, y_
关于DataLoader, 作为数据集封装, 将数据集Dataset封装为可迭代对象. 示例如下,
1 batch_size = 100 2 trainingLoader = DataLoader(trainingData, batch_size=batch_size, shuffle=True)
Part Ⅱ: 模型确立
torch.nn
torch.nn.Module
网络模型由基类Module派生, 内部所有操作模块均由命名空间nn提供, 需要实现基本的2个方法, 分别为: __init__、forward. 其中, __init__方法定义操作, forward方法运用操作进行正向计算. 示例如下,1 class NeuralNetwork(nn.Module): 2 3 def __init__(self): 4 super(NeuralNetwork, self).__init__() 5 self.__linear_tanh_stack = nn.Sequential( 6 nn.Linear(3, 5), 7 nn.Tanh(), 8 nn.Linear(5, 3) 9 ) 10 11 12 def forward(self, x): 13 y = self.__linear_tanh_stack(x) 14 return y 15 16 17 model = NeuralNetwork()
Part Ⅲ: 损失函数确立
torch.nn
常见损失函数有: nn.MSELoss(回归任务)、nn.CrossEntropyLoss(多分类任务)等. 示例如下,1 loss_func = nn.MSELoss(reduction="sum")
Part Ⅳ: 优化器确立
torch.optim
常见的优化器有: optim.SGD、optim.Adam等. 示例如下,1 optimizer = optim.Adam(model.parameters(), lr=0.001)
Part Ⅴ: 模型训练及保存
有效整合数据、模型、损失函数及优化器. 注意, 模型参数之梯度默认累积, 每次参数优化需要显式清零. 示例如下,1 def train_loop(dataloader, model, loss_func, optimizer): 2 for batchIdx, (X, Y_) in enumerate(dataloader): 3 Y = model(X) 4 loss = loss_func(Y, Y_) 5 6 optimizer.zero_grad() 7 loss.backward() 8 optimizer.step() 9 10 11 epoch = 50000 12 for epochIdx in range(epoch): 13 train_loop(trainingLoader, model, loss_func, optimizer) 14 15 16 torch.save(model.state_dict(), "model_params.pth")
- 代码实现
本文使用与Back Propagation - Python实现相同的网络架构及数据生成策略, 分别如下所示,$$
\begin{equation*}
\left\{
\begin{split}
x &= r + 2g + 3b \\
y &= r^2 + 2g^2 + 3b^2 \\
lv &= -3r - 4g - 5b
\end{split}
\right.
\end{equation*}
$$
具体实现如下,1 import numpy 2 import torch 3 from torch import nn 4 from torch import optim 5 from torch.utils.data import Dataset, DataLoader 6 from matplotlib import pyplot as plt 7 8 9 numpy.random.seed(1) 10 torch.manual_seed(3) 11 12 13 # 生成training数据 14 def getData(n=100): 15 rgbRange = (-1, 1) 16 r = numpy.random.uniform(*rgbRange, (n, 1)) 17 g = numpy.random.uniform(*rgbRange, (n, 1)) 18 b = numpy.random.uniform(*rgbRange, (n, 1)) 19 x_ = r + 2 * g + 3 * b 20 y_ = r ** 2 + 2 * g ** 2 + 3 * b ** 2 21 lv_ = -3 * r - 4 * g - 5 * b 22 RGB = numpy.hstack((r, g, b)) 23 XYLv_ = numpy.hstack((x_, y_, lv_)) 24 return RGB, XYLv_ 25 26 27 class TrainingDataset(Dataset): 28 29 def __init__(self, X, Y_, transform=None, target_transform=None): 30 self.__X = X 31 self.__Y_ = Y_ 32 self.__transform = transform 33 self.__target_transform = target_transform 34 35 36 def __len__(self): 37 return len(self.__X) 38 39 40 def __getitem__(self, idx): 41 x = self.__X[idx] 42 y_ = self.__Y_[idx] 43 if self.__transform: 44 x = self.__transform(x) 45 if self.__target_transform: 46 y_ = self.__target_transform(y_) 47 return x, y_ 48 49 50 RGB, XYLv_ = getData(1000) 51 trainingData = TrainingDataset(RGB, XYLv_, torch.Tensor, torch.Tensor) 52 53 batch_size = 100 54 trainingLoader = DataLoader(trainingData, batch_size=batch_size, shuffle=True) 55 56 57 class NeuralNetwork(nn.Module): 58 59 def __init__(self): 60 super(NeuralNetwork, self).__init__() 61 self.__linear_tanh_stack = nn.Sequential( 62 nn.Linear(3, 5), 63 nn.Tanh(), 64 nn.Linear(5, 3) 65 ) 66 67 68 def forward(self, x): 69 y = self.__linear_tanh_stack(x) 70 return y 71 72 73 model = NeuralNetwork() 74 loss_func = nn.MSELoss(reduction="sum") 75 optimizer = optim.Adam(model.parameters(), lr=0.001) 76 77 78 def train_loop(dataloader, model, loss_func, optimizer): 79 JVal = 0 80 for batchIdx, (X, Y_) in enumerate(dataloader): 81 Y = model(X) 82 loss = loss_func(Y, Y_) 83 84 JVal += loss.item() 85 86 optimizer.zero_grad() 87 loss.backward() 88 optimizer.step() 89 90 JVal /= 2 91 return JVal 92 93 94 JPath = list() 95 epoch = 50000 96 for epochIdx in range(epoch): 97 JVal = train_loop(trainingLoader, model, loss_func, optimizer) 98 print("epoch: {:5d}, JVal = {:.5f}".format(epochIdx, JVal)) 99 JPath.append(JVal) 100 101 102 torch.save(model.state_dict(), "model_params.pth") 103 104 105 fig = plt.figure(figsize=(6, 4)) 106 ax1 = fig.add_subplot(1, 1, 1) 107 108 ax1.plot(numpy.arange(len(JPath)), JPath, "k.", markersize=1) 109 ax1.plot(0, JPath[0], "go", label="seed") 110 ax1.plot(len(JPath)-1, JPath[-1], "r*", label="solution") 111 112 ax1.legend() 113 ax1.set(xlabel="$epoch$", ylabel="$JVal$", title="solution-JVal = {:.5f}".format(JPath[-1])) 114 115 fig.tight_layout() 116 fig.savefig("plot_fig.png", dpi=100)
View Code - 结果展示
可以看到, 在training data上总体loss随epoch增加逐渐降低.
-
使用建议
①. 分batch处理训练数据, 可以提升训练初始阶段模型参数收敛速度;
②. 常规优化器推荐Adam, 具备自动步长调节的能力. - 参考文档
①. https://pytorch.org/tutorials/beginner/basics/intro.html
标签:__,.__,nn,self,torch,transform,PyTorch,使用,之初 来源: https://www.cnblogs.com/xxhbdk/p/15551023.html