关于源码还是没看懂
问题：
Q: 对pytorch里GAN更新G的过程疑问？
fake=G(x)
G.zero_gard()
out = D(fake)
loss = cri(out,label)
loss.backward()
opt_G.step()
这里，经过了D网络结构，怎么可以就之更新G了呢？？？
A: 经过了前向传播之后，pytorch就构建了计算图，可以想像G计算图的输出作为了D计算图的输入，当loss.backward的时候虽然计算了所有计算图的梯度，但是由于只更新了G参数的数值，所以只利用了G计算图的梯度，这是可以理解的。之前纠结的点是跳过D更新G,但计算梯度的时候确实是经过了D的。只是GAN这里没有去更新D。
code
敲了敲代码模拟了一下这个过程
import torch
import torch.nn as nn
import torch.optim as optim
net1 = nn.Conv2d(2,1,kernel_size=3)
net2 = nn.Linear(3,1)
x = torch.randn((1,2 ,5, 5), requires_grad=True)
opt_1 = optim.SGD(net1.parameters(),lr=0.01,momentum=0)
opt_2 = optim.SGD(net2.parameters(),lr=0.01,momentum=0)
cri = nn.MSELoss()
#打印参数（ out_channels x in_channels x kernel_size x kernel_size )
for p in net1.parameters():
print(p)
>>>
Parameter containing:
tensor([[[[-0.1586, 0.0571, -0.0606],
[-0.0712, -0.0753, 0.1583],
[-0.1438, 0.0217, 0.2169]],
[[ 0.0830, -0.2124, 0.1965],
[ 0.2107, 0.0655, -0.2229],
[ 0.0809, 0.1761, -0.2056]]]], requires_grad=True)
Parameter containing:
tensor([0.2213], requires_grad=True)
for p in net2.parameters():
print(p)
>>>
Parameter containing:
tensor([[0.1985, 0.0381, 0.3382]], requires_grad=True)
Parameter containing:
tensor([0.1597], requires_grad=True)
#through net1
y = net1(x)
print('y',y)
#through net2
z = net2(y)
print('z',z,z.shape)
>>>
y tensor([[[[ 0.7343, 0.1432, -0.3362],
[ 0.5525, 0.5968, 0.6297],
[ 1.0429, -0.0552, 0.6783]]]], grad_fn=<MkldnnConvolutionBackward>)
z tensor([[[[0.1972],
[0.5050],
[0.5940]]]], grad_fn=<AddBackward0>) torch.Size([1, 1, 3, 1])
#reshape z and compute loss
net1.zero_grad()
new_z = z.reshape(1,-1)
target = torch.tensor([[1,1,0]],dtype=torch.float)
loss = cri(new_z,target)
#before backward
print(net1.bias.grad)
print(net2.bias.grad)
#==========compute gradient======
loss.backward()
#================================
#after
print(net1.bias.grad)
print(net2.bias.grad)
for f in net1.parameters():
print(f.grad.data)
>>>
None
None
tensor([-0.2697])
tensor([-0.4691])
tensor([[[[-0.5155, -0.3812, -0.3589],
[ 0.2562, -0.3376, 0.1517],
[ 0.2322, -0.2214, 0.0431]],
[[ 0.1007, 0.2636, 0.0203],
[ 0.0201, 0.0348, -0.2474],
[-0.0404, 0.0261, 0.0283]]]])
tensor([-0.2697])
for p in net1.parameters():
print(p)
print('manully updata params:')
tmp_p = p.data.clone()
print(tmp_p.sub_(p.grad.data*0.01))#weight = weight-lr*d_p
print('---')
print(p.data)
for p in net2.parameters():
print(p)
Parameter containing:
tensor([[[[-0.1586, 0.0571, -0.0606],
[-0.0712, -0.0753, 0.1583],
[-0.1438, 0.0217, 0.2169]],
[[ 0.0830, -0.2124, 0.1965],
[ 0.2107, 0.0655, -0.2229],
[ 0.0809, 0.1761, -0.2056]]]], requires_grad=True)
manully updata params:
tensor([[[[-0.1535, 0.0609, -0.0570],
[-0.0738, -0.0719, 0.1568],
[-0.1461, 0.0239, 0.2165]],
[[ 0.0820, -0.2150, 0.1963],
[ 0.2105, 0.0652, -0.2204],
[ 0.0813, 0.1758, -0.2059]]]])
---
tensor([[[[-0.1586, 0.0571, -0.0606],
[-0.0712, -0.0753, 0.1583],
[-0.1438, 0.0217, 0.2169]],
[[ 0.0830, -0.2124, 0.1965],
[ 0.2107, 0.0655, -0.2229],
[ 0.0809, 0.1761, -0.2056]]]])
Parameter containing:
tensor([0.2213], requires_grad=True)
manully updata params:
tensor([0.2240])
---
tensor([0.2213])
Parameter containing:
tensor([[0.1985, 0.0381, 0.3382]], requires_grad=True)
Parameter containing:
tensor([0.1597], requires_grad=True)
#upadate
opt_1.step()
for p in net1.parameters():
print(p)
print('=======')
for p in net2.parameters():
print(p)
Parameter containing:
tensor([[[[-0.1535, 0.0609, -0.0570],
[-0.0738, -0.0719, 0.1568],
[-0.1461, 0.0239, 0.2165]],
[[ 0.0820, -0.2150, 0.1963],
[ 0.2105, 0.0652, -0.2204],
[ 0.0813, 0.1758, -0.2059]]]], requires_grad=True)
Parameter containing:
tensor([0.2240], requires_grad=True)
=======
Parameter containing:
tensor([[0.1985, 0.0381, 0.3382]], requires_grad=True)
Parameter containing:
tensor([0.1597], requires_grad=True)
大概就是这么个流程，不过具体的自动求导机制暂时还看不懂源码，之后如果看懂的话，会在这里补充的。
