我正在尝试创建一个模块,它包含nn.Parameters()的某些层。如果我按以下方式初始化该层-
self.W = nn.Parameter(torch.randn(4,4), requires_grad=True).double()然后,该层不显示在模块参数中注册。
但是,这个初始化是有效的-
self.W = nn.Parameter(torch.FloatTensor(4,4), requires_grad=True)完整的例子-
class TestNet(nn.Module):
def __init__(self):
super(TestNet, self).__init__()
self.W = nn.Parameter(torch.randn(4,4), requires_grad=True).double()
def forward(self, x):
x = torch.matmul(x, self.W.T)
x = torch.sigmoid(x)
return x
tnet = TestNet()
print(list(tnet.parameters()))
### Output = [] (an empty list)与-
class TestNet(nn.Module):
def __init__(self):
super(TestNet, self).__init__()
self.W = nn.Parameter(torch.FloatTensor(4,4), requires_grad=True)
def forward(self, x):
x = torch.matmul(x, self.W.T)
x = torch.sigmoid(x)
return x
tnet = TestNet()
print(list(tnet.parameters()))什么指纹-
[Parameter containing:
tensor([[-1.8859e+26, 6.0240e-01, 1.0842e-19, 3.8177e-05],
[ 1.5229e-27, -8.5899e+09, 1.5226e-27, -3.6893e+19],
[ 4.2039e-45, -4.6566e-10, 1.5229e-27, -2.0000e+00],
[ 2.8026e-45, 0.0000e+00, 0.0000e+00, 4.5918e-40]],
requires_grad=True)]那有什么区别呢?为什么torch.randn()版本不能工作?我在文档里或者网上的答案里找不到任何关于这个的东西。
发布于 2022-11-14 08:13:06
打电话给randn是完全可以的。问题是在操作结束时调用了.double():
class TestNet(nn.Module):
def __init__(self):
super(TestNet, self).__init__()
self.W = nn.Parameter(torch.randn(4,4, dtype = torch.double), requires_grad=True)
# self.W = nn.Parameter(torch.randn(4,4).double(), requires_grad=True) # also works
def forward(self, x):
x = torch.matmul(x, self.W.T)
x = torch.sigmoid(x)
return x
tnet = TestNet()
print(tnet.W.dtype)
# torch.float64
print(list(tnet.parameters()))
# [Parameter containing:
# tensor([[-1.9645, -1.5445, 0.2435, 0.4380],
# [ 1.1403, 0.8836, 0.1811, -0.1212],
# [ 1.5983, -0.1854, -0.2626, 0.2881],
# [-1.2364, -0.4802, -0.6038, 0.1164]], requires_grad=True)]现在,代码注册参数。我在dtype = torch.double的初始化中添加了randn,以确保self.W与以前一样包含doubles。
总之,我们不能调用nn.Parameter,然后将其转换为另一种数据类型,作为深度学习系统的神经网络权重。
https://stackoverflow.com/questions/74428429
复制相似问题