Software Agents Laboratory Games

The guide walks you through the two-square world and asks you to act as the agent designer: predict what the default rules will do, change a rule, and use the log to explain the result.

What you should learn

Understand how an agent maps percepts to actions, and how its behaviour is determined by its agent function.

Suggested investigation

1. Predict the default action sequence when both squares are dirty, then run the agent and compare.
2. Change the rule for (A, Clean) to NoOp and explain the changed behaviour.
3. Clean both squares and explain why the agent continues moving.
4. Design one rule set that behaves well and one that behaves badly.
5. Explain whether any assignment of the four rules can make the agent stop once both squares are clean.

Prefer the complete instructions? Open the full student guide (Markdown).

The guide frames you as the designer of the performance measure. You will predict agent rankings, adjust the reward and penalty sliders, and explain why the same agent can be rational under one measure but poor under another.

What you should learn

See that rationality is relative to a performance measure, and that the designer chooses what outcomes count as success.

Suggested investigation

1. Predict the ranking of the four agents under the default measure and evaluate them.
2. Increase the movement penalty until the lazy agent wins; record the settings.
3. Increase the penalty for leaving dirt over time and explain which agent benefits.
4. Compare the reflex and paranoid agents and explain their identical scores.
5. Increase dirt respawn probability and identify the changed environment property.
6. Discuss whether one measure can make all four agents equally good, and who defines success in the GAMA firefighter practical.

Prefer the complete instructions? Open the full student guide (Markdown).

The guide asks you to act as an analyst of a fixed agent function. You compare full and partial observability, inspect the percept and decision log, and classify the environment using the AIMA dimensions.

What you should learn

Make the limitations of simple reflex agents visible: they depend on the current percept and can fail when relevant information is unavailable or actions are stochastic.

Suggested investigation

1. Run with all sensors on and explain which rule fires as the agent moves through the corridor.
2. Predict what happens at the rightmost clean square, then verify using the log.
3. Turn off the location sensor and trace the unknown-location percept through the rules.
4. Enable the flaky motor and explain why the agent cannot detect a failed Suck action.
5. Classify the environment with reliable sensors, then with partial observability and a flaky motor.
6. Propose the piece of internal state that would repair the agent's failure in the fog.

Prefer the complete instructions? Open the full student guide (Markdown).

The guide explains how the agent maintains a belief map using a transition model and a sensor model. It then asks you to audit that memory when the world behaves differently from the model.

What you should learn

Understand internal state as a concrete data structure with an update rule, and recognise that an agent's decisions depend on the quality of the models maintaining its belief.

Suggested investigation

1. Run with reliable movement and predict how the belief map is updated.
2. Identify where internal state, rather than the current percept, determines the agent's action.
3. Explain why the agent can stop when its belief says the world is clean.
4. Enable occasional movement failure and find a run where belief and reality diverge.
5. Use the log to locate where the error entered the state and identify the inaccurate model.
6. Propose an additional percept that could correct the drift, then connect the transition and sensor models to the GAMA firefighter.

Prefer the complete instructions? Open the full student guide (Markdown).

In this laboratory, you will investigate a mountain rescue scenario in which an autonomous robot must find a stranded hiker and bring them safely back to its base. The agent must consider its current state, represent its goal, and plan a sequence of actions to achieve it.

What you should learn

Understand the main idea behind goal-based agents: instead of simply reacting to the current percept, an agent uses information about a desired future state to select actions. You will also investigate how search and planning can be used to find an action sequence leading to that goal.

Suggested investigation

1. Before running the agent, identify its initial state, available actions, and goal condition. What must be true for the mission to count as successful?
2. Select Find a plan. Inspect the explored cells and the planned route. How does BFS explore the environment?
3. Execute the plan step by step. Identify when the robot reaches the hiker and when the rescue action takes place.
4. After rescuing the hiker, explain why the robot must continue moving. Has it achieved its goal yet?
5. Activate obstacle editing and change the terrain. Generate a new plan. How does the route change when obstacles are added or removed?
6. Generate a new terrain and compare the resulting plans. Does the goal change, or only the sequence of actions needed to achieve it?

Prefer the complete instructions? Open the interactive laboratory .

Follow the vacuum cleaner as it observes a room, updates its internal state, predicts action consequences and selects actions to clean the room.

What you should learn

Understand how a goal-based agent considers a desired future state and uses a model of action outcomes to choose what to do.

Suggested investigation

1. Identify the agent's goal and available actions before starting.
2. Use Next agent step and identify the percept received by the sensors.
3. Observe how the agent updates its internal state and uses its model of movement and cleaning.
4. Predict what the agent will do when it is standing on a dirty cell, then run the next step.
5. Run the agent automatically. Explain how each action changes the environment and leads to new observations.
6. Explain why the agent needs a goal and an action model rather than relying only on a condition–action rule.

Act as the designer of a utility function for an autonomous medical rescue drone. All routes aim to achieve the same goal—deliver the medicine—but differ in their possible outcomes. Adjust the utility weights and observe how the agent's preferred action changes.

What you should learn

Understand why goal achievement alone is insufficient when objectives conflict or outcomes are uncertain. Learn how a utility function assigns numerical values to outcomes and how an agent selects the action with the highest expected utility.

Suggested investigation

1. Start with the default utility weights. Predict which route the agent will choose, then inspect the expected-utility calculations.
2. Increase the penalty for an incident or loss. Does the agent change its preferred route? Explain the trade-off.
3. Increase the value of successful delivery and the cost of time. Does the agent become more willing to take the direct ridge route?
4. Set the time and battery costs to zero. Explain which route is preferred and why.
5. Run the mission several times with unchanged utility weights. Can the agent choose the same route but obtain different outcomes?
6. Design two different utility functions that lead to different route choices, while keeping the environment and route probabilities unchanged.

The guide maps the four components of a learning agent onto the running code. You compare an untrained agent with a trained one, then investigate how exploration and learning rate affect what it learns.

What you should learn

Understand how a policy can be acquired from experience, and identify the performance element, critic, learning element, and problem generator in the implementation.

Suggested investigation

1. Reset the Q-table and run an episode before training; describe the initially untrained behaviour.
2. Train repeatedly and compare the learning curve and behaviour before and after training.
3. Increase epsilon, reset and retrain; explain the effect of heavier exploration.
4. Set epsilon to zero and investigate what experiences the agent can no longer obtain.
5. Compare low and high learning rates on fresh agents.
6. Discuss autonomy and the missing social property, relate the components to an LLM-based assistant, and compare the Q-learner with the GAMA BDI firefighter.

Prefer the complete instructions? Open the full student guide (Markdown).

Use the vacuum-cleaner world to investigate the learning-agent architecture from Russell and Norvig. Train the agent, inspect its feedback and Q-value updates, and compare its behaviour before and after learning.

What you should learn

Identify the performance element, critic, learning element and problem generator, and explain how they work together to improve an agent's behaviour through experience.

Suggested investigation

1. Start with the untrained agent. Execute a few actions and observe how the performance element selects them using its initial Q-table.
2. Train the agent for one episode, then for 25 episodes. Compare the rewards, success rate and number of actions.
3. Change the exploration parameter (ε). Explain how the problem generator affects the actions tried during training.
4. Change the learning rate (α) and discount factor (γ). Discuss how these parameters affect Q-value updates.
5. Test the learned policy without exploration. Compare its behaviour with the untrained agent.
6. Identify the critic's feedback and explain how it is used by the learning element to update the Q-table.

Investigate a vacuum-cleaner agent whose decision-making mechanism is a small language model running locally in the browser. Inspect the percept and prompt, ask the LLM to propose an action, and decide whether to execute or discard its proposal.

What you should learn

Understand how an LLM can be embedded as the decision-making mechanism inside the agent program described by Russell and Norvig. Distinguish the model's proposed action from the JavaScript actuator that validates and executes it.

Suggested investigation

1. Inspect the agent architecture diagram. Identify the environment, sensors, agent program and actuators.
2. Display the current percept and explain which information is available to the LLM—and which information is not.
3. Request an action. Compare the prompt, raw model response and parsed action.
4. Execute the proposal and observe how the environment changes and produces a new percept.
5. Try several decisions and record invalid actions, wall collisions, actions taken and task completion.
6. Compare this LLM decision-maker with the explicit goal-based and Q-learning vacuum-cleaner agents from Labs 5B and 10.

Investigate how a knowledge-based agent receives information from its environment, stores relevant facts, applies rules, and recommends an action. The laboratory follows the agent architecture discussed in the lecture and uses a synthetic district-heating plant scenario.

What you should learn

Understand the roles of percepts, the knowledge base, TELL and ASK. Explain how an agent can use a limited set of facts and rules to make a task-specific recommendation without representing every aspect of its environment.

Suggested investigation

1. Change the pressure, sensor reliability, independent confirmation, and temperature settings. Generate a percept and describe what the agent receives from its sensors.
2. Press TELL. Inspect the knowledge base and identify which new facts have been added.
3. Press ASK. Identify the rule used by the agent and explain how it supports the recommended action.
4. Run the four predefined experiments. Compare the recommendations when the pressure sensor is reliable, unreliable, or independently corroborated.
5. Change one environmental condition at a time. Explain which change causes the agent to select a different action.
6. Consider whether the agent needs a complete representation of the plant to answer its current question. Explain what happens when relevant information is missing.