在numpy true除法中遇到溢出

Question

我正在尝试用python从头开始实现动量梯度下降。在实现指数加权平均的偏差校正时，我遇到了一个运行时警告：“在numpy true除法中遇到溢出”。我正在尝试用一个非常小的浮点数来划分一个numpy数组(在代码部分中提供)。

for i in range(1, len(self.layers)):
            self.dvw['dvw'+str(i)] = (self.beta1*self.dvw['dvw'+str(i)] + (1-self.beta1)*self.dw['dw'+str(i)])
            self.dvb['dvb'+str(i)] = (self.beta1*self.dvb['dvb'+str(i)] + (1-self.beta1)*self.db['db'+str(i)])
            self.dvw['dvw'+str(i)] = self.dvw['dvw'+str(i)].astype('float')
            self.dvb['dvb'+str(i)] = self.dvb['dvb'+str(i)].astype('float')
            self.dvw['dvw'+str(i)] = np.divide(self.dvw['dvw'+str(i)], (1-np.power(self.beta1, t))) # encounterd error in this line
            self.dvb['dvb'+str(i)] = np.divide(self.dvb['dvb'+str(i)], (1-np.power(self.beta1, t))) # and this line aswell

dvw和dvb都是以numpy数组的值作为值的字典。beta1值为0.9，t为实数

Answer 1

尝试通过添加以下内容来提高代码的数值稳定性：

eps=1e-6
for i in range(1, len(self.layers)):
    self.dvw['dvw'+str(i)] = (self.beta1*self.dvw['dvw'+str(i)] + (1-self.beta1)*self.dw['dw'+str(i)])
    self.dvb['dvb'+str(i)] = (self.beta1*self.dvb['dvb'+str(i)] + (1-self.beta1)*self.db['db'+str(i)])
    self.dvw['dvw'+str(i)] = self.dvw['dvw'+str(i)].astype('float')
    self.dvb['dvb'+str(i)] = self.dvb['dvb'+str(i)].astype('float')
    self.dvw['dvw'+str(i)] = np.divide(self.dvw['dvw'+str(i)], (eps + 1-np.power(self.beta1, t)))
    self.dvb['dvb'+str(i)] = np.divide(self.dvb['dvb'+str(i)], (eps + 1-np.power(self.beta1, t)))