Skip to content

2048 environment for Reinforcement Learning and DQN algorithm

Notifications You must be signed in to change notification settings

YangRui2015/2048_env

Repository files navigation

2048 Environment and DQN Algorithm implementation

Thanks to the author of gym-2048 https://github.com/rgal/gym-2048. The code is easy to understand and runs efficiently. I just made some little changes to make it a better RL environment. And I implemented dqn with many tricks using pytorch:

  • Randomly fill buffer first;
  • Soft target replacing;
  • Epsilon decay;
  • Clip gradient norm;
  • Double DQN;
  • Priority Experience Replay;

Performance of environment

I used random policy to evaluate the performance for 1000 times. We can take random policy as a baseline.
The evaluation main function is in base_agent.py.

(1) with rendering:

average episode time:0.10279795455932617 s;
average step time: 0.7373 ms;
average highest score:106.368;
average total score:1078.252;
average steps:139.417;

(2) without rendering:

average episode time:0.03773710775375366 s;
average step time: 0.2671 ms;
average highest score:108.24;
average total score:1102.088;
average steps:141.288;

some example:
image

Performance of Priority DQN

Training for 45k episodes and the max eval mean score is 7700(eval for 50 episodes). image image

Update

  1. add max steps and max illegal steps of one episode;
  2. add dqn agent and training infomation;
  3. fix bug on the Double Q trick (the issue raised by mythsman);

About

2048 environment for Reinforcement Learning and DQN algorithm

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages