开放式AI健身房观测空间形状问题

Question

我试图解决一次和永远使用强化学习的Yatzee游戏。可悲的是，当我检查健身房是否符合稳定的基线时，我的观察空间的形状就会受到影响。所以，我在构造函数中放置了一个print语句，它告诉我我的观察空间的形状，只要我创建一个对象。

class YatzeeEnv{
    game_state = np.zeros(19, np.int32)

    def __init__(self):
        self.action_space = gym.spaces.Discrete(19)
        self.observation_space = gym.spaces.MultiDiscrete(19)

        for x in self.game_state_adresses:
            self.game_state[x] = -1
        self.reroll()
        self.game_state[self.reroll_state] = 0
        print(self.game_state.shape)
        print(self.observation_space.shape)
}

a = YatzeeEnv()

可悲的是，它的输出是

np array shape: (19,)
Observation space shape: ()

为什么会这样呢？我认为gym.spaces.MultiDiscrete(19)将观察空间定义为19个值的int数组。

Answer 1

从医生那里..。

This represents the cartesian product of arbitrary :class:`Discrete` spaces.
    It is useful to represent game controllers or keyboards where each key can be represented as a discrete action space.
    Note:
        Some environment wrappers assume a value of 0 always represents the NOOP action.
    e.g. Nintendo Game Controller - Can be conceptualized as 3 discrete action spaces:
    1. Arrow Keys: Discrete 5  - NOOP[0], UP[1], RIGHT[2], DOWN[3], LEFT[4]  - params: min: 0, max: 4
    2. Button A:   Discrete 2  - NOOP[0], Pressed[1] - params: min: 0, max: 1
    3. Button B:   Discrete 2  - NOOP[0], Pressed[1] - params: min: 0, max: 1
    It can be initialized as ``MultiDiscrete([ 5, 2, 2 ])`` such that a sample might be ``array([3, 1, 0])``.
    Although this feature is rarely used, :class:`MultiDiscrete` spaces may also have several axes
    if ``nvec`` has several axes:
    Example::
        >> d = MultiDiscrete(np.array([[1, 2], [3, 4]]))
        >> d.sample()
        array([[0, 0],
               [2, 3]])

如果您只有一个操作空间，则不必使用MultiDiscrete。或者使用MultiDiscrete(19)。