tf_agents dqn初始化失败

Question

尽管tf.agents初始化()不需要输入变量，但这一行

agent.initialize()

产生此错误

TypeError: initialize() missing 1 required positional argument: 'self'

我试过agent.initialize(特工)，因为它显然想要自我传递.很明显，XD不起作用

我怀疑问题可能是这条线

print(type(agent))   

产

<class 'abc.ABCMeta'>

但这可能很正常..。

##################################

我下面的整个脚本都是可复制的

###  for 9 by 9 connect 4 board 
#
import tensorflow as tf
from tf_agents.networks import q_network
from tf_agents.agents.dqn import dqn_agent
import tf_agents
import numpy as np

print(tf.__version__)
print(tf_agents.__version__)

import tensorflow.keras


observation_spec  = tf.TensorSpec(   #   observation tensor = the whole board , ideally 0's, 1's , 2's for empty, occupied by player 1 , occupied by player 2
    [9,9],
    dtype=tf.dtypes.float32,
    name=None
)

action_spec  = tf_agents.specs.BoundedArraySpec(    
    [1],                         ### tf_agents.networks.q_network only seems to take an action of size 1  
    dtype= type(1) ,     #tf.dtypes.float64,
    name=None, 
    minimum=0,
    maximum=2
)
#######################################

def make_tut_layer(size):
    return tf.keras.layers.Dense(
        units= size,
        activation= tf.keras.activations.relu,
        kernel_initializer=tf.keras.initializers.RandomNormal(mean=0., stddev=1.)
                                )

def make_q_layer(num_actions):
    q_values_layer = tf.keras.layers.Dense (        # last layer gives probability distribution over all actions so we can pick best action 
        num_actions ,
        activation = tf.keras.activations.relu , 
        kernel_initializer = tf.keras.initializers.RandomUniform( minval = 0.03 , maxval = 0.03),
        bias_initializer = tf.keras.initializers.Constant(-0.2)
                                        ) 
    return q_values_layer;



############################## stick together layers below

normal_layers = []

for i in range(3):
    normal_layers.append(make_tut_layer(81))
q_layer = make_q_layer(9)

q_net = keras.Sequential(normal_layers + [q_layer])

######################################

agent = dqn_agent.DqnAgent
(
    observation_spec,    ### bonus question, why do i get syntax errors when i try to label variables like ---> time_step_spec = observation_spec, gives me SyntaxError: invalid syntax   on the = symbol       
    action_spec,
    q_net,
    tf.keras.optimizers.Adam(learning_rate= 0.001 )

)
eval1 = agent.policy
print(eval1)
eval2= agent.collect_policy
print(eval2)
print(type(agent))  
agent.initialize()
print(" done ")

并产生输出。

2.9.2
0.13.0
<property object at 0x000001A13268DA90>
<property object at 0x000001A13268DAE0>
<class 'abc.ABCMeta'>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [53], in <cell line: 73>()
     71 print(eval2)
     72 print(type(agent))
---> 73 agent.initialize()
     74 print(" done ")

TypeError: initialize() missing 1 required positional argument: 'self'

我的特工类型还好吗？应该是

为什么我的代理没有初始化？

Answer 1

我想，答案非常简单:不能只将(移动到函数调用的下一行。

你实际上在做的是：

使agent成为dqn_agent.DqnAgent (类)的别名

agent = dqn_agent.DqnAgent

计算表达式并丢弃其结果

(
    observation_spec,
    action_spec,
    q_net,
    tf.keras.optimizers.Adam(learning_rate= 0.001 )
)

这也回答了一个额外的问题--因为它不是函数调用，所以没有命名的参数，表达式中也不允许赋值( python就是这么说的)。

将打开的括号放在dqn_agent.DqnAgent之后，它应该可以工作：

agent = dqn_agent.DqnAgent(
    observation_spec,
    action_spec,
    q_net,
    tf.keras.optimizers.Adam(learning_rate= 0.001 )
)