求解Tensorflow/Keras微分方程的自定义损失函数问题

Question

我正在尝试复制1902.05563的结果。我们可以在此链接中找到一个例子，它是求解方程的。4在论文中。但是，他们正在通过变量和占位符手工编写层(请注意，它使用的是tf v1)。我没有从头开始编写网络，而是尝试使用Keras来实现同样的目标；

import numpy      as np
import tensorflow as tf
x1     = tf.keras.Input(name='x_1',shape=(1),dtype=tf.dtypes.float32)
dense1 = tf.keras.layers.Dense(10,activation=tf.nn.sigmoid,name='l1')(x1)
output = tf.keras.layers.Dense(1,activation=None,name='output')(dense1)
model  = tf.keras.Model(inputs=x1,outputs=output)
model.summary()
### output
Model: "functional_42"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
x_1 (InputLayer)             [(None, 1)]               0         
_________________________________________________________________
l1 (Dense)                   (None, 10)                20        
_________________________________________________________________
output (Dense)               (None, 1)                 11        
=================================================================
Total params: 31
Trainable params: 31
Non-trainable params: 0
_________________________________________________________________

我认为它的结构和上面给出的链接相同。在此基础上，构建了损失和优化器；

x_train  = np.linspace(0,2,100,endpoint=True)#Generate 100 points in the [0,2] interval
x_t      = np.zeros((len(x_train),1))
x_t[:,0] = x_train

x = tf.constant(x_t)
with tf.GradientTape(persistent=True) as tape:
    variables = model.trainable_variables
    tape.watch(x)
    tape.watch(variables)
    y_pred  = model(x,training=True)
    #dy_dx = tape.gradient(y_pred,x)
    #print(dy_dx)
y = tf.reshape(y_pred,x.shape).numpy()
dy_dx = tape.gradient(y_pred,x)
A = (1+3*(x**2))/(1+x+x**3)
t_loss = tf.reshape(dy_dx,x.shape) + (x + A)*y - x**3 - 2*x - A * x**2
loss = tf.reshape(tf.reduce_mean(t_loss)+(y[0]-1)**2,())

with tf.GradientTape() as tape2:
    tape2.watch(model.trainable_weights)
    logits  = model(x,training=True)
grads = tape.gradient(loss,model.trainable_weights) # it has been tested with trainable_variables as well

这里，t_loss是简单的eq。4在1902.05563和loss中，边界条件还实现为(y[0]-1)**2。但是，由于我的grads是[None, None, None, None]，我的优化器失败了。

optimizer = tf.keras.optimizers.Adam(0.01)
optimizer.apply_gradients(zip(grads, model.trainable_weights))
### output
ValueError: No gradients provided for any variable: ['l1/kernel:0', 'l1/bias:0', 'output/kernel:0', 'output/bias:0'].

如果有人能告诉我如何编写这个微分方程求解器的工作示例，那就太好了。谢谢!

附加信息

在这条线中也提出了一个类似的问题，我试图用它重写我的方法。因此，修改后的方法如下所示，使用相同的model

def _loss_tensor(y_true,y_pred,x_train):
    x = tf.constant(x_train)
    dy_dx = tf.keras.backend.gradients(y_pred,x)
    lq = (1+3*(x1**2))/(1+x1+x1**3)
    t_loss = (dy_dx+(x+lq)*y_pred-x**3-2*x-lq*x*x)
    return tf.reduce_mean(t_loss)+(y_pred[0]-1)**2
def loss_func(x_train):
    def loss(y_true,y_pred):
        return _loss_tensor(y_true,y_pred,x_train)
    return loss
optimizer = tf.keras.optimizers.Adam(1e-2)
model.compile(loss=loss_func(model.inputs), # tried x_t, model.input as well but all gave same result
              optimizer=optimizer)
model.train_on_batch(x_t)
#also tried
#model.fit(x_t,epochs=5,verbose=1)
### Output
ValueError: No gradients provided for any variable: ['l1/kernel:0', 'l1/bias:0', 'output/kernel:0', 'output/bias:0'].

然而，我仍然得到了完全相同的错误。

Answer 1

结果发现，不可能在模型类中使用Keras来定义丢失函数(或者我无法为它找到解决方案)。因此，我通过低级别的TensorFlow (v2.3+注意到v1的语法非常不同)解决了这个问题；

x_train  = np.linspace(0,2,100,endpoint=True)#Generate 100 points in the [0,2] interval
x_t      = np.zeros((len(x_train),1))
x_t[:,0] = x_train

inputs = tf.keras.Input(name='inputs',shape=(1),dtype=tf.dtypes.float32)
hidden = tf.keras.layers.Dense(10,activation=tf.nn.sigmoid,name='hidden')
output = tf.keras.layers.Dense(1,activation=None,name='output')
model  = tf.keras.Sequential([inputs, hidden, output ])

epochs = 50000
Adam   = tf.keras.optimizers.Adam(1e-2)

x = tf.constant(x_t,dtype=float)
trainable_vars = model.trainable_variables
losses = []
for ix in range(epochs):
    with tf.GradientTape(persistent=True) as tape:
        tape.watch(x)
        tape.watch(trainable_vars)
        y = model(x,training=True)
        dy_dx = tape.gradient(y,x)  
        A = (1+3*x**2)/(1+x+x**3)
        t_loss = (dy_dx + (x + A) * y - x**3 - 2*x - x**2*A)**2
        loss  =  tf.reduce_mean(t_loss)+(y[0]-1)**2
        tape.watch(loss)

    gradients = tape.gradient(loss, trainable_vars)
    Adam.apply_gradients(zip(gradients, trainable_vars))
    losses.append(loss.numpy()[0])