Self Learning Agent in FPS Environment - RL

Simple implementation of Wolfenstein 3D game using Raycasting Algorithm integrated with Proximal Policy Optimization ( PPO ) Deep Reinforcement Learning Technique.

Levels:

1. Basic
2. Defend The Center
3. Deadly Corridor

Modes:

1. Train ( Training The Model )
2. Test ( Testing The Model )

Deadly Skills:

1. Easy → 1
2. Medium → 2
3. Hard → 3

Results:

Basic Level:

Defend Level:

Deadly-Corridor Level w/ Sparse Reward:

Deadly-Corridor Level w/ Reward Shaping & Curriculum Learning:

Plots:

Basic

Fig: Basic Level Episode Length & Reward Plot

Fig: Basic Level Loss Plot

Defend

Fig: Defend Level Episode Length & Reward Plot

Fig: Defend Level Loss Plot

Deadly

Fig: Deadly Level Episode Length & Reward Plot

Fig: Deadly Level Loss Plot

CLI Commands:

Basic & Defend:

1. Train Model:

python3 ./main.py --level { basic | defend } --train

2. Test Model:

python3 ./main.py --level { basic | defend } --test --steps { steps }

Deadly:

Valid Skill Level:

1. Easy
2. Medium
3. Hard

1. Train Model w/o Curriculum Learning:

python3 ./main.py --level defend --train --skill { skill }

2. Train Model w/ Curriculum Learning:

python3 ./main.py --level defend --train --skill { skill } --curr

3. Testing Model:

python3 ./main.py --level defend --test --skill { skill } --steps { steps }

Valid Level Mode Command:

1. --level basic --train
2. --level basic --test --steps { steps }
3. --level defend --train
4. --level defend --test --steps { steps }
5. --level deadly --train --skill { skill }
6. --level deadly --train --skill { skill } --curr
7. --level deadly --test --skill { skill } --steps { steps }

Examples:

1. To train the model on basic level:

python3 ./main.py --level basic --train

2. To test the model trained for 200000 steps on DEFEND level:

python3 ./main.py --level defend --test --steps 200000

3. To train the model on DEADLY level on medium mode w/o Curriculum Learning:

python3 ./main.py --level deadly --train --skill 2

4. To train the model on DEADLY level on hard mode w/ Curriculum Learning:

python3 ./main.py --level deadly --train --skill 3 --curr

5. To test the model trained for 70000 steps on DEADLY level on EASY mode:

python3 ./main.py --level deadly --test --skill 1 --steps 700000

Note: Curriculum Learning is only available if a model has been trained on the previous difficulty mode i.e. Curriculum Learning on Hard mode is only available if a model is available for Medium mode.