Lab 6: Watch a learning agent improve

5DV248 Lecture 1 · Same world as the notebook: 5x5 grid, four dirty squares. The agent starts with no rules at all; only the critic (rewards) and the problem generator (epsilon-exploration) teach it a policy.
The world (blue dot = agent, brown = dirt)

The small letters show the current greedy action for the agent's state (best learned Q-value): ^ v < > move, S = Suck.
Learning curve (steps per episode, lower is better)

Components from slide 48, located in the code: performance element = the greedy Q-policy; critic = the reward function; learning element = the Q-value update; problem generator = epsilon-exploration. Question 4 of the notebook: which Wooldridge & Jennings property does this agent still lack?