Part 2 : Evolutionary Algorithms for Reinforcement Learning — Solving OpenAI’s Cartpole

Solving the Cartpole problem with a genetic algorithm

In the first part of this article we discussed why alternative evolutionary algorithms are required for reinforcement learning. We also discussed various steps involved in implementing genetic algorithm for an reinforcement learning. In this article lets implement a genetic algorithm to solve the famous OpenAI’s cartpole problem.

Figure 1 : Cartpole game

We will use the CartPole environment to test our agent. The agent is rewarded by keeping the pole upright, and it can move the cart left or right. We can represent an agent as a neural network that approximates the policy function — it accepts a state and outputs an action, or more typically a probability distribution over actions. The following listing shows an example of a three-layer network.

import numpy as np
import torch
 
def model(x,unpacked_params):
    l1,b1,l2,b2,l3,b3 = unpacked_params          1
    y = torch.nn.functional.linear(x,l1,b1)      2
    y = torch.relu(y)                            3
    y = torch.nn.functional.linear(y,l2,b2)
    y =…