Reinforcement learning, in plain terms

Most machine learning is taught as two flavours: supervised, where you learn from labelled examples, and unsupervised, where you find structure without labels. Reinforcement learning is the third flavour, and it’s the strangest of the family: nobody tells the system the right answer, ever. It acts, the world responds, and it learns from the consequences.

The vocabulary is small. An agent takes actions in an environment and receives rewards, positive or negative. Its job is a policy: a way of choosing actions that maximises reward over time, not just on the next move. That last clause is the entire depth of the field. A chess sacrifice looks terrible by immediate reward and wins the game eight moves later, and connecting outcomes to the decisions that caused them — the credit-assignment problem — is what makes RL genuinely hard.

The classic mental picture is a robot in a maze: bump a wall, small penalty; reach the exit, big reward. Thousands of episodes later, it has a route nobody programmed. And the same loop, scaled up, produced the era’s most famous result: a Go-playing system trained substantially through self-play, learning strategies from consequences alone that had never occurred to human masters in two thousand years of play.

That is easy to say and more convincing to watch, so here is one I actually trained. Tabular Q-learning, four thousand episodes, minus one point for every step until it reaches the exit:

Two panels. The learning curve falls from 408 steps in the first episode, through noisy improvement, to a flat line near 20. The maze alongside shows the route the trained policy takes, which is 20 steps — the same length as the true shortest path.

The first episode took 408 steps of near-random flailing. The finished policy takes 20, which is exactly the shortest path through that maze. Nothing in the reward mentioned walls, corners or distance. It only ever said not yet.

Where does it show up outside the highlight reel? A trading algorithm learns from its own profits and losses, which is why I called it reinforcement learning when it appeared in my AI-versus-ML tour. Robotic arms learn grasps through reward for success. Recommendation and assistant systems fold user reactions back into behaviour. The pattern to spot: wherever a system’s own actions change what it sees next, you’ve left supervised learning’s world and entered this one.

If you want to touch it, the traditional playground is OpenAI’s Gym library and its little MountainCar problem, an underpowered car that must learn to rock backwards to build momentum. The loop, stripped to pseudocode, is honestly this small:

for each episode:
    state = env.reset()
    while not done:
        action = agent.choose(state)          # explore vs exploit
        state, reward, done = env.step(action)
        agent.update(state, action, reward)   # learn from consequence

The line that hides the dragons is choose. Always exploit what you know and you never discover the better strategy; explore too much and you never cash in. Balancing that trade-off is the field’s permanent tension.

One warning from practice, because it’s the part the textbooks underplay: RL systems are champion loophole-finders. Specify the reward even slightly wrong and the agent optimises exactly what you wrote rather than what you meant — racing in circles collecting points instead of finishing the race, in one famous demo. It’s the objective problem in miniature, and it’s why my first question about any RL result is never “how big is the reward?” It’s “what, precisely, was rewarded?”

Discuss a project