平:通过梯度下降的学习参数不会导致适当的数值和降低学习质量。

Question

我正在尝试实现一个物理信息神经网络。与经典的神经网络相比，损失中的微分部分在未知区域确实带来了一些改进。这个未知的领域实际上是已知的，但我只是从培训和测试数据中删除了它们，以检查PINN与其他工艺的性能。下面是我正在使用的代码：

model = tf.keras.Sequential([
    layers.Dense(units=64, activation='relu', input_shape=(2,)),
    layers.Dense(units=64, activation='relu'),
    layers.Dense(units=1,)
])

optimizer = tf.keras.optimizers.Adam()
objective = tf.keras.losses.Huber()
metric = tf.keras.metrics.MeanAbsoluteError()

w_phys = 0.5
w_loss = 1.0 - w_phys

with tf.device('gpu:0'):
    for epoch in range(epochs):
        cumulative_loss_train = 0.0
        metric.reset_states()
        for mini_batch, gdth in dataset:
            with tf.GradientTape(persistent=True) as tape:
                tape.watch(unknown_area_SOCP_tensor)
                tape.watch(mini_batch)

                # Physics loss
                predictions_unkwon = model(unknown_area_SOCP_tensor, training=True)
                d_f = tape.gradient(predictions_unkwon, unknown_area_SOCP_tensor)

                # Physics part with P #
                dp = tf.convert_to_tensor(1/((K*unknown_area_SOCP_tensor[:,0]+L)**2-4*R*unknown_area_SOCP_tensor[:,1]), dtype = np.float64)
                phys_loss_p = 10*tf.cast(tf.math.reduce_mean(tf.math.square(d_f[:,1]**2 - dp)), np.float32)

                # Traditionall loss #
                predictions = model(mini_batch, training=True)
                loss = objective(gdth, predictions)

                # Compute grads #
                grads = tape.gradient(w_loss*loss + w_phys*(phys_loss_p), model.trainable_variables)

            optimizer.apply_gradients(zip(grads, model.trainable_variables))
            cumulative_loss_train += loss
            metric.update_state(gdth, predictions)
            del tape

到目前一切尚好。K、R、L为固定参数。下一步是假设它们是未完成的，并试图找出我们是否可以学习它们。我首先试了一下，只关注R参数。下面是使用的代码：

with tf.device('gpu:0'):
    for epoch in range(epochs):
        cumulative_loss_train = 0.0
        metric.reset_states()
        for mini_batch, gdth in dataset:
            with tf.GradientTape(persistent=True) as tape:
                tape.watch(unknown_area_SOCP_tensor)
                tape.watch(mini_batch)
                tape.watch(R)

                # Physics loss
                predictions_unkwon = model(unknown_area_SOCP_tensor, training=True)
                d_f = tape.gradient(predictions_unkwon, unknown_area_SOCP_tensor)

                # Physics part with P #
                dp = tf.convert_to_tensor(1/((K*unknown_area_SOCP_tensor[:,0]+L)**2-4*R*unknown_area_SOCP_tensor[:,1]), dtype = np.float64)
                phys_loss_p = 10*tf.cast(tf.math.reduce_mean(tf.math.square(d_f[:,1]**2 - dp)), np.float32)

                # Traditionall loss #
                predictions = model(mini_batch, training=True)
                loss = objective(gdth, predictions)

                # Compute grads #
                grads = tape.gradient(w_loss*loss + w_phys*(phys_loss_p), model.trainable_variables + [R])

            optimizer.apply_gradients(zip(grads, model.trainable_variables + [R]))
            cumulative_loss_train += loss
            metric.update_state(gdth, predictions)
            del tape

但这会导致可怕的结果(比如高损失和糟糕的度量)。更糟糕的是，R的值必须是正的，在训练结束时，R被估计为负值.

我对这个方程很有信心，因为我检查了很多时间，与我正在使用的模拟软件相比，它似乎是准确的。此外，这个等式给学习带来了价值(因为对未完成项目的预测要好得多)。

我错过了什么吗？

谢谢你的帮助！

Answer 1

对于未知的张力损失函数，可以根据实际需要确定正确的形状。

为了检查，a和b的变化在相同的窗格中，但在3D内，您可能需要创建具有共享引用的相关性矩阵，用于直接比较或同一域一次更改几个参数。

示例: ax + b作为会计计算器被乘以替换a中的多少来赢得b的更改。当a不能更改为具有关闭值时，b就会发生变化。

import tensorflow as tf

x = 1.00
y = 1.00

a  = tf.Variable(1.0) #We can optimize them later
b = tf.Variable(100.0)
x_ = tf.Variable(x)   #We don't need placeholders for tf.eager()
y_ = tf.Variable(y)
y_hat_ = a*x_ + b
r_ = y_ - y_hat_
rss = tf.reduce_sum(tf.square(y_ - y_hat_))
print(rss) # tf.Tensor(10000.0, shape=(), dtype=float32)


# Calculate the gradients for given values of a and b
tape = tf.GradientTape()
with tape:
    y_hat_ = a*x_ + b
    r_ = y_ - y_hat_
    rss = tf.reduce_sum(tf.square(y_ - y_hat_))
tape.gradient(rss, [a,b])

# We now use the gradients and feed them into the optimizer to go down one gradient step
#optimizer = tf.train.AdamOptimizer(learning_rate=0.01)
optimizer = tf.keras.optimizers.Adam(
    learning_rate=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False,
    name='Adam'
)

for i in range(1000):
    tape = tf.GradientTape() #Each time a tape needs to be provided
    with tape:
        y_hat_ = a*x_ + b
        r_ = y_ - y_hat_
        rss = tf.reduce_sum(tf.square(y_ - y_hat_))
    grads = tape.gradient(rss, [a,b])
    optimizer.apply_gradients(zip(grads, [a,b]))
    if (i % 100 == 0): #Trick not to print so often
        print(a.numpy(), b.numpy(), rss.numpy())

input("Press any KEY!!!")

输出:请参阅a和b作为订单的优先级。

tf.Tensor(10000.0, shape=(), dtype=float32)
0.9999 99.9999 10000.0
0.9898983 99.98998 9996.017
0.9798994 99.980064 9992.033
0.9699037 99.970146 9988.053
0.959908 99.96023 9984.073
0.9499123 99.95031 9980.094
0.9399166 99.94039 9976.116
0.9299209 99.93047 9972.139
0.9199252 99.920555 9968.162
0.90993106 99.91064 9964.187
Press any KEY!!!