简单三层神经网络不能训练

Question

#3 layers neural network
import numpy as np
from __future__ import division
def nonlin(x,deriv=False): #activation function
    if(deriv==True):
        return np.exp(x)/(1+np.exp(x))**2
    return 1/(1+np.exp(-x))

X = np.array([
                [0,0,1],
                [0,1,1],
                [1,1,1],
                [1,0,0],
                [0,1,0],
                [1,1,0]])
Y = np.array([[0,1,0,1,1,0]]).T
np.random.seed(1)
l0 = X
syn0 = 2*np.random.random((3,30))-1
syn1 = 2*np.random.random((30,1))-1

for i in xrange(60000):
    l0 = X
    l1 = nonlin(np.dot(l0,syn0))
    l2 = nonlin(np.dot(l1,syn1))
    l2_error = Y-l2
    l2_delta = l2_error*nonlin(l2,deriv=True)

    l1_error = l2_delta.dot(syn1.T)
    l1_delta = l1_error*nonlin(l1,deriv=True)
    syn0 += l0.T.dot(l1_delta)
    syn1 += l1.T.dot(l2_delta)
print l2

我一直在搅乱https://iamtrask.github.io/2015/07/12/basic-python-network/的神经网络实现

这是代码的输出：

[[1.85572928e-04]
 [9.99755942e-01]
 [5.21248255e-09]
 [9.99767481e-01]
 [9.99963580e-01]
 [2.07334909e-04]]

我希望像Y这样的东西

[[0]
 [1]
 [0]
 [1]
 [1]
 [0]]

这里怎么可能出了问题？

Answer 1

您正在获得的结果如下：

[[1.85572928e-04]  =  0.000185572928         ~ 0
 [9.99755942e-01]  =  0.999755942            ~ 1
 [5.21248255e-09]  =  0.000000000521248255   ~ 0
 [9.99767481e-01]  =  0.999767481            ~ 1
 [9.99963580e-01]  =  0.999963580            ~ 1
 [2.07334909e-04]] =  0.000207334909         ~ 0

这些确实非常接近你的预期结果。您正在计算和预测浮点数，而不是二进制零和1。例如，您可以添加一个简单的规则，将接受值低于阈值为零，高于阈值为1。