Deep Reinforcement Learning Algorithms with PyTorch

This repository contains PyTorch implementations of deep reinforcement learning algorithms and environments. It has been forked from p-christ/Deep-Reinforcement-Learning-Algorithms-with-PyTorch and modified to learn a custom environment for use in the Illini Solar Car's Brizo solar-powered car.

Algorithms Implemented

1. Deep Q Learning (DQN) _{^{(Mnih et al. 2013)}}
2. DQN with Fixed Q Targets _{^{(Mnih et al. 2013)}}
3. Double DQN (DDQN) _{^{(Hado van Hasselt et al. 2015)}}
4. DDQN with Prioritised Experience Replay _{^{(Schaul et al. 2016)}}
5. Dueling DDQN _{^{(Wang et al. 2016)}}
6. REINFORCE _{^{(Williams et al. 1992)}}
7. Deep Deterministic Policy Gradients (DDPG) _{^{(Lillicrap et al. 2016 )}}
8. Twin Delayed Deep Deterministic Policy Gradients (TD3) _{^{(Fujimoto et al. 2018)}}
9. Soft Actor-Critic (SAC) _{^{(Haarnoja et al. 2018)}}
10. Soft Actor-Critic for Discrete Actions (SAC-Discrete) _{^{(Christodoulou 2019)}}
11. Asynchronous Advantage Actor Critic (A3C) _{^{(Mnih et al. 2016)}}
12. Syncrhonous Advantage Actor Critic (A2C)
13. Proximal Policy Optimisation (PPO) _{^{(Schulman et al. 2017)}}
14. DQN with Hindsight Experience Replay (DQN-HER) _{^{(Andrychowicz et al. 2018)}}
15. DDPG with Hindsight Experience Replay (DDPG-HER) _{^{(Andrychowicz et al. 2018 )}}
16. Hierarchical-DQN (h-DQN) _{^{(Kulkarni et al. 2016)}}
17. Stochastic NNs for Hierarchical Reinforcement Learning (SNN-HRL) _{^{(Florensa et al. 2017)}}
18. Diversity Is All You Need (DIAYN) _{^{(Eyensbach et al. 2018)}}

All implementations are able to quickly solve Cart Pole (discrete actions), Mountain Car Continuous (continuous actions), Bit Flipping (discrete actions with dynamic goals) or Fetch Reach (continuous actions with dynamic goals). I plan to add more hierarchical RL algorithms soon.

Environments Implemented

1. SimpleISC

Results

Have a peek at our runs over at Weights & Biases

1. SimpleISC

Usage

The repository's high-level structure is:

DeepRL
    └── agents                    
        ├── actor_critic_agents   
        ├── DQN_agents         
        ├── policy_gradient_agents
        └── stochastic_policy_search_agents 
    ├── environments   
    ├── results             
        └── data_and_graphs        
    ├── tests
    ├── utilities             
        └── data structures