5DV248 · Utility-Based Agents — Medical Rescue Drone

The mission

A remote mountain clinic has requested an urgent medical package. The drone can choose a direct ridge route, a sheltered valley route, or a balanced corridor. Every route may lead to different outcomes: successful delivery, delay, or loss of the package.

Important: all three routes pursue the same goal—deliver the medicine. But achieving the goal is not the only consideration. A utility-based agent compares how desirable the possible outcomes are and chooses the action with the highest expected utility.

What makes it utility-based?

A utility function maps a state (or sequence of states) to a real number that represents its desirability. The agent evaluates possible outcomes, taking their probabilities into account.

EU(action) = Σ P(outcome | action) × U(outcome)

The agent selects the route with the highest expected utility—not necessarily the fastest route or the route with the highest success probability.

1 · Set the utility function

Change the importance of each outcome feature. The agent's route recommendation and expected-utility calculations update immediately.

Value of successful delivery+100

Utility points gained if the medicine reaches the clinic.

Penalty for incident / loss−80

Additional penalty if the drone or package is lost.

Cost per minute−1.0

Penalty for time spent, including unsuccessful trips.

Cost per battery unit−2.0

Penalty for energy consumed during the mission.

2 · Compare the possible actions

The route probabilities and outcome characteristics are part of the scenario model. Your utility settings determine how desirable each outcome is.

Direct ridge Valley detour Sheltered corridor

3 · How the agent evaluates outcomes

Each possible outcome has a probability and a utility. Expected utility is the probability-weighted average.

Adjust the utility weights to see the calculations.

Action	Expected utility	Rank

Agent recommendation

Calculating the best action…

4 · Run the mission

Ask the agent to choose the action with the highest expected utility, or select a route yourself. The environment then samples one actual outcome using that route's probabilities.

Mission not started

Change the utility function, compare the routes, then run the mission.

The actual result is stochastic: the highest expected-utility action can still have an unlucky outcome. Rationality maximizes expected performance; it does not guarantee the best realized outcome every time.

Investigate and reflect

Start with the default utility function. Which route does the agent prefer? Explain why the route with the highest success probability may not be the best choice.
Increase the penalty for an incident. Does the agent change its preferred route? Identify the trade-off.
Increase the value of successful delivery and reduce the cost of time. Does the agent become more willing to take the direct ridge route?
Set the time and battery costs to zero. Which route is preferred now, and why?
Run the mission several times with the same utility settings. Can the agent choose the same route but obtain different outcomes? Explain the role of probability.
Design two different utility functions that lead to different choices, even though the environment and route probabilities remain unchanged.

What should I notice?

The goal is the same for all routes: deliver the medicine. A goal-based description alone does not distinguish a fast but risky route from a slow and safe route. The utility function makes those differences explicit, and expected utility combines desirability with uncertainty.

Why can the agent lose even when it chose rationally?

Expected utility is an average over possible outcomes, weighted by their probabilities. It does not predict which particular outcome will occur on one run. A rational choice can therefore lead to an undesirable realized outcome.