Part 9 – Reinforcement learning libraries

Let’s compare some reinforcement learning libraries (October 2019).

This article is inspired from these posts:

  1. https://github.com/godmoves/reinforcement_learning_collections
  2. https://medium.com/data-from-the-trenches/choosing-a-deep-reinforcement-learning-library-890fb0307092

Main characteristics

RL LibrariesFrameworkTensorboard supportCustom environment interface
Keras-RLKerasNoNo
TensorforceTensorflowYesYes
OpenAI BaselinesTensorflow?No
Stable baselinesTensorflowYesYes
TF AgentsTensorflowYes?
Ray / RllibTensorflow / Pytorch / KerasYesYes
TensorlayerTensorflowYes?
Rllab / GarageTensorflow / Pytorch?Yes
CoachTensorFlowNo but custom dashboardYes

Algorithms implemented

RL
Libraries
DQNDDPGNAF / CDQNCEMSARSADqfDPG / REINFORCEPPOA2CA3CTRPOGAEACERACKTRGAILSACTD3ERWRNPOREPSTNPGCMA-ESMMCPALTDMRIGSkew-Fit
Keras-RLXXXXX
















TensorforceXXX

XXXXXXX









OpenAI BaselinesXX




XX
X
XXX






Stable baselinesXX




XX
X
XXXXX




TF AgentsXX



XX






XX




Ray / RllibXX



XXXX




XX




TensorlayerXX



XXXXX



XX




Rllab / GarageXX
X

XX

X




XXXXXX
CoachXXXXXXXXXXXX
RlkitXXXXXX

Conclusion

Stable baselines, Ray and Garage are the most complete RL libraries to date.