Documentation

[filipeabperes]

Hi, I was looking into possibly using this for a personal project involving building an agent to play a Flash game. So I was wondering, is this project functional already? If so, is there any documentation or tutorial to guide usage?

Thanks

[perara]

Hello, yes it is ready for use, hovewer im bot done with documentation. Ill finish it up tomorrow for you!

Have a pleasant christmas!

[filipeabperes]

Cool, thanks! No hurries though, no need to overwork on Christmas (:

[perara]

Okay, ive done some work tidying a few things up on master branch. Run games fine, but does not have the capability of labeling states automatically.

[filipeabperes]

Thanks, I was able to install and run a few games fine. However, I seem to have a problem when a game requires a key to be held down. For example, a jump that has its duration defined by how long you hold the jump key. This type of jump just never happens.

I checked the code for the pyVNC and saw the it releases the key automatically on send_key(). However, this issue happens even when I manually input the commands myself into the game. Held down keys are not processed as such. Any ideas on this?

[perara]

Ill see if i can make an api for this feature in early 2018 ;). Have a nice new year!

[filipeabperes]

Cool, thanks. I could try it myself as well. Do you know where I should look to fix it?

Have a nice new year you too!

[filipeabperes]

Just so you know, I had tried both pressing the button myself on the keyboard, and also implementing the methods:

    def send_press(self, key):
        if key in constants.MODIFIERS:
            self.screen.protocol.key_event(constants.MODIFIERS[key], down=1)
        elif key in constants.KEYMAPPINGS:
            self.screen.protocol.key_event(constants.KEYMAPPINGS[key], down=1)
        elif type(key) == str:
            self.screen.protocol.key_event(ord(key))

    def send_release(self, key):
        if key in constants.MODIFIERS:
            self.screen.protocol.key_event(constants.MODIFIERS[key], down=0)
        elif key in constants.KEYMAPPINGS:
            self.screen.protocol.key_event(constants.KEYMAPPINGS[key], down=0)

in the pyVNC Client and using them in the on_frame loop, but neither worked.

[perara]

Are you sure it does not work?

 def send_key(self, key, duration=0.001):
        if key in constants.MODIFIERS:
            self.screen.protocol.key_event(constants.MODIFIERS[key], down=1)
        elif key in constants.KEYMAPPINGS:
            self.screen.protocol.key_event(constants.KEYMAPPINGS[key], down=1)
        elif type(key) == str:
            self.screen.protocol.key_event(ord(key))

        time.sleep(duration)

        if key in constants.MODIFIERS:
            self.screen.protocol.key_event(constants.MODIFIERS[key], down=0)
        elif key in constants.KEYMAPPINGS:
            self.screen.protocol.key_event(constants.KEYMAPPINGS[key], down=0)

i use the "duration" argument myself, and it seems to work for me. By splitting these like you illustrated should work.hmmm

[filipeabperes]

When I set the time.sleep() duration to a higher value, the game just hung (frozen) for that amount of time, so I couldn't tell if it was working or not.

When experimenting with the split versions I pasted above, It seems to me that between each frame the state of the key might be reset (?) so that even though down=0 was never set, the press only lasted 1 frame (and the sleep duration didn't change that because, as I mentioned above, it was freezing on a single frame).

Now, all this might have been an issue of the game I tried (robot unicorn attack), I'm not sure.

[filipeabperes]

Okay, so I've kinda figured this out. So, first, I realized that the problem wasn't the the keys weren't being released, but that actually what was happening was that, after the first time a key was pressed, it was never being released again (and so couldn't be pressed again).

So I modified the send_key() method as following (added the last 2 lines), in order to add what seemed to be a missing key release.

 def send_key(self, key, duration=0.001):
        if key in constants.MODIFIERS:
            self.screen.protocol.key_event(constants.MODIFIERS[key], down=1)
        elif key in constants.KEYMAPPINGS:
            self.screen.protocol.key_event(constants.KEYMAPPINGS[key], down=1)
        elif type(key) == str:
            self.screen.protocol.key_event(ord(key), down=1)

        time.sleep(duration)

        if key in constants.MODIFIERS:
            self.screen.protocol.key_event(constants.MODIFIERS[key], down=0)
        elif key in constants.KEYMAPPINGS:
            self.screen.protocol.key_event(constants.KEYMAPPINGS[key], down=0)
        elif type(key) == str:
            self.screen.protocol.key_event(ord(key), down=0)

And I also added the same 2 lines into the key_release() method. Now with this change the problem of keys being held forever was fixed, but as a downside the send_key() method stopped working altogether. However, using send_press() and send_release() now works as expected: if you key_press() on a frame and then key_release() a few frames later, the key is held during those frames.

I am not sure why key_press() stopped working, but I think what's happening is that despite the time.sleep() call, both the press and release are running on a same frame, so the game never even perceives the press (or only for a very small fraction of a second), thus never actually detecting it, at least on the game I tried, as I mentioned before.

[filipeabperes]

Forgot to mention, for all of this to work, I had also modified the Environment class so that it has a self.frame_count field which increases at each on_frame() call. Then the call to the frame callback becomes self.frame_callback(state, self.frame_count, state_type, self.vnc). Then, the on_frame() method in the example's main.py has its signature changed to def on_frame(state, frame, type, vnc), so that that information can be used by the example.

[perara]

Could you pull request this? Seems like something that is excellent to work on 👍

[filipeabperes]

Sure, I'll make a pull request both here and on pyVNC for the respective parts

[filipeabperes]

Related to the documentation discussion:

Are there any training loop examples? The mujaffa example always displays the game screen and runs only one thread. Are there any examples that run without displaying the screen, run multiple games, etc?

[perara]

No not currently. I only have outdated examples. I'll see if I can update those

[perara]

I'm finishing up my Master thesis today and was thinking of doing some new examples for FlashRL. Do you have any suggestions on flash games / envrionments that could be a good candidate?

[filipeabperes]

I think you might want to take a look here: https://github.com/openai/universe/blob/master/universe/__init__.py#L319

There's a list of games OpenAI (supposedly) already gave some thought to, to include into their framework, so it might be a good starting point. To name a few, some good examples off the top of my head: the DuskDriver car driving game they cite in the front page for the project, FlappyBird and the derivatives they have there (FlappyX) and N (called N0OfficialWebVersion in the file), which is of higher complexity.