It interacts with the game simply using screen-grab (input) and keypress simulation (output) using some def observe(self): # Get current state s from screen using screen-grab and narrow it down to