用角角逼近行列式

Question

我正在训练一个keras稠密模型来逼近2x2矩阵的行列式。我使用的是30个隐藏层，每个层有100个节点，10E6个矩阵(在间隔[0,100 ]中有条目)。在对测试集(占总数的33.3%)进行预测之后，我计算出MSE的平方根，得到通常不大于100的值。我认为这是一个相当高的错误(虽然我不确定在这种情况下什么是一个很好的错误)，但是除了增加样本数量之外，我不确定如何改进它(已经有10E6似乎是一个很大的数字)。我希望有人能提供一些建议。以下是代码：

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense


### Select number of samples, matrix size and range of entries in matrices
nb_samples = 1000000
matrix_size = 2
entries_range = 100

### Generate random matrices and determinants
matrices = []
determinants = []
for i in range(nb_samples):
    matrix = np.random.randint(entries_range, size = (matrix_size,matrix_size))
    matrices.append(matrix.reshape(matrix_size**2,))
    determinants.append(np.array(np.linalg.det(matrix)).reshape(1,))
matrices = np.array(matrices)
determinants = np.array(determinants)


### Split the data 
matrices_train, matrices_test, determinants_train, determinants_test = train_test_split(matrices,determinants,train_size = 0.66) 

### Select number of layers and neurons
nb_layers = 30
nb_neurons = 100

### Create dense neural network with nb_layers hidden layers having nb_neurons neurons each
model = Sequential()
model.add(Dense(nb_neurons, input_dim = matrix_size**2, activation='relu'))
for i in range(nb_layers):
    model.add(Dense(nb_neurons, activation='relu'))
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')
model.fit(matrices_train, determinants_train, epochs = 10, batch_size = 100, verbose = 0)

#_ , test_acc = model.evaluate(matrices_test,determinants_test)
#print(test_acc) 

### Make a prediction on the test set
determinants_pred = model.predict(matrices_test)

print('''
RMSE: {}
Number of layers: {}
Number of neurons: {}
Number of samples: {}
'''.format(np.sqrt(mean_squared_error(determinants_test,determinants_pred)),nb_layers,nb_neurons,nb_samples))

以下是一个输出：

20.429616387932295

• Number：
• RMSE:层数:32
• 神经元数:32
• 样本数: 1000000

注意:我决定通过反复尝试每个层和100个节点( MSE似乎是围绕这些值的最低值)。

Answer 1

我认为您的网络对于问题的大小来说是巨大的(输入dim =4 output = 1)，而且您没有足够的时间。

我们也可以在这里欺骗一点，因为我们知道计算基本上可以用输入的线性组合的平方表示，我们可以使用x*x自定义激活函数。下面是一个例子，10个神经元，1个隐藏层，自定义激活函数如上，历元= 1000，nsamples = 10000。

RMSE: 0.04413008355924881
Number of layers: 1
Number of neurons: 10
Number of samples: 10000

这是完整的代码，我做了一些小修改

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense


### Select number of samples, matrix size and range of entries in matrices
nb_samples = 10000#00
matrix_size = 2
entries_range = 100

### Generate random matrices and determinants
matrices = []
determinants = []
for i in range(nb_samples):
    matrix = np.random.randint(entries_range, size = (matrix_size,matrix_size))
    matrices.append(matrix.reshape(matrix_size**2,))
    determinants.append(np.array(np.linalg.det(matrix)).reshape(1,))
matrices = np.array(matrices)
determinants = np.array(determinants)


### Split the data 
matrices_train, matrices_test, determinants_train, determinants_test = train_test_split(matrices,determinants,train_size = 0.66) 

### Select number of layers and neurons
nb_layers = 1#30
nb_neurons = 10#0

### Create dense neural network with nb_layers hidden layers having nb_neurons neurons each
model = Sequential()
model.add(Dense(nb_neurons, input_dim = matrix_size**2, activation=lambda x:x*x))
#for i in range(nb_layers):
#    model.add(Dense(nb_neurons, activation='relu'))
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')
model.fit(matrices_train, determinants_train, epochs = 1000, batch_size = 100, verbose = 1)

#_ , test_acc = model.evaluate(matrices_test,determinants_test)
#print(test_acc) 

### Make a prediction on the test set
determinants_pred = model.predict(matrices_test)

print('''
RMSE: {}
Number of layers: {}
Number of neurons: {}
Number of samples: {}
'''.format(np.sqrt(mean_squared_error(determinants_test,determinants_pred)),nb_layers,nb_neurons,nb_samples))