Part 9 – Reinforcement learning libraries

Let’s compare some reinforcement learning libraries (October 2019).

This article is inspired from these posts:

  1. https://github.com/godmoves/reinforcement_learning_collections
  2. https://medium.com/data-from-the-trenches/choosing-a-deep-reinforcement-learning-library-890fb0307092

Main characteristics

RL LibrariesFrameworkTensorboard supportCustom environment interface
Keras-RLKerasNoNo
TensorforceTensorflowYesYes
OpenAI BaselinesTensorflow?No
Stable baselinesTensorflowYesYes
TF AgentsTensorflowYes?
Ray / RllibTensorflow / Pytorch / KerasYesYes
TensorlayerTensorflowYes?
Rllab / GarageTensorflow / Pytorch?Yes

Algorithms implemented

RL
Libraries
DQNDDPGNAF / CDQNCEMSARSADqfDPG / REINFORCEPPOA2CA3CTRPOGAEACERACKTRGAILHERSACTD3ERWRNPOREPSTNPGCMA-ES
Keras-RLXXXXX

















TensorforceXXX

XXXXXXX










OpenAI BaselinesXX




XX
X
XXXX






Stable baselinesXX




XX
X
XXXXXX




TF AgentsXX



XX







XX




Ray / RllibXX



XXXX





XX




TensorlayerXX



XXXXX




XX




Rllab / GarageXX
X

XX

X





XXXXXX

Conclusion

Stable baselines, Ray and Garage are the most complete RL libraries to date.