我知道如何存储和加载nn.Model,但找不到如何为nn.Parameter创建检查点。我尝试了这个版本,但是优化器在恢复之后没有改变nn.Parameter的值。
from torch import nn as nn
import torch
from torch.optim import Adam
alpha = torch.ones(10)
lr = 0.001
alpha = nn.Parameter(alpha)
print(alpha)
alpha_optimizer = Adam([alpha], lr=lr)
for i in range(10):
alpha_loss = - alpha.mean()
alpha_optimizer.zero_grad()
alpha_loss.backward()
alpha_optimizer.step()
print(alpha)
path = "./test.pt"
state = dict(alpha_optimizer=alpha_optimizer.state_dict(),
alpha=alpha)
torch.save(state, path)
checkpoint = torch.load(path)
alpha = checkpoint["alpha"]
alpha_optimizer.load_state_dict(checkpoint["alpha_optimizer"])
for i in range(10):
alpha_loss = - alpha.mean()
alpha_optimizer.zero_grad()
alpha_loss.backward()
alpha_optimizer.step()
print(alpha)发布于 2021-05-13 01:33:57
问题是优化器仍然引用旧的alpha (在最后一个for循环之前检查id(alpha)与id(alpha_optimizer.param_groups[0]["params"][0]) ),而当您从alpha = checkpoint["alpha"]中的检查点加载它时,将设置一个新的from。
在加载状态之前,您需要更新优化器的参数:
# ....
torch.save(state, path)
checkpoint = torch.load(path)
# here's where the reference of alpha changes, and the source of the problem
alpha = checkpoint["alpha"]
# reset optim
alpha_optimizer = Adam([alpha], lr=lr)
alpha_optimizer.load_state_dict(checkpoint["alpha_optimizer"])
for i in range(10):
alpha_loss = - alpha.mean()
alpha_optimizer.zero_grad()
alpha_loss.backward()
alpha_optimizer.step()
print(alpha)https://stackoverflow.com/questions/67505670
复制相似问题