Intelligent Agents: Making Rational Concrete
The previous lesson established why acting rationally is the right target for AI. This lesson delivers the machinery that makes it concrete: what an agent is, what “rational” formally means, how to characterise the problems agents face, and the family of designs we will build the rest of the course around.
By the end you should be able to describe any AI task as a PEAS specification, classify its environment on seven dimensions, match it to an appropriate agent design, and explain why each rung on the design ladder fixes the limitation of the one below it.
Agents and environments
Definition · The core vocabulary
An agent is anything that perceives its environment through sensors and acts on it through actuators.
- Percept — what the sensors are taking in right now.
- Percept sequence — the complete history of everything the agent has ever perceived.
An agent's action at any instant can depend on its built-in knowledge and on its entire percept sequence to date, but not on anything it has not perceived.
The distinction we must not blur: function vs. programme
Definition · Agent function vs. agent programme
The agent function is the abstract mathematical object that maps every possible percept sequence to an action — like an infinite lookup table, a complete external description of behaviour.
The agent programme is the concrete implementation of that function running on physical hardware. It is the actual code.
Intuition · The recipe vs. the cook
The agent function is the specification ("for this input, produce that output") — the abstract recipe, possibly infinitely long. The agent programme is the working cook that realises it within real time and memory. Very different programmes can implement the same function; a tiny programme can stand in for an astronomically large table.
Worked example · The vacuum-cleaner world
Two squares, A and B, each clean or dirty. The agent senses its location and whether dirt is present; it can move left, move right, or suck. A simple agent function: if the current square is dirty, suck; otherwise move to the other square. Tabulated, it maps [A, Dirty] → Suck, [A, Clean] → Right, and so on — an unbounded table. The agent programme is the handful of lines of code that produce the same outputs without storing the table. This tiny world recurs throughout the course as the running test case.
Exam trap · Is the agent function the same as the agent programme?
No. Function = abstract map (possibly infinite); programme = its finite implementation on the architecture. Also remember: agent = architecture + programme.
Good behaviour — rationality, precisely
“A rational agent does the right thing” — but what is “right”? AI adopts consequentialism: judge behaviour by its consequences. The agent’s actions push the environment through a sequence of states; if that sequence is desirable, the agent did well.
Definition · Performance measure
A performance measure evaluates any given sequence of environment states. It is the objective yardstick of success, and crucially it lives in the mind of the designer or user — not the agent.
Key insight · Measure outcomes, not behaviours
Design the performance measure around what you want to be true of the environment, not how you imagine the agent should act. Reward the vacuum for a clean floor over time, not for amount of dirt collected — or a rational agent will dump dirt out and re-collect it forever to game the measure.
Definition · A rational agent — the precise statement
What is rational at a given moment depends on four things:
- the performance measure defining success,
- the agent's prior knowledge of the environment,
- the actions available to the agent,
- the agent's percept sequence to date.
For each possible percept sequence, a rational agent selects the action expected to maximise its performance measure, given the evidence of the percept sequence and its built-in knowledge.
Pitfall · Rational ≠ omniscient, perfect, or all-knowing
Rationality maximises expected performance; omniscience would require knowing the actual outcome in advance, which is impossible. A rational agent can do everything right and still get unlucky. Demanding the perfect (after-the-fact best) action is incoherent unless you have a crystal ball. Do not confuse a rational decision with a fortunate outcome.
Information gathering, learning, and autonomy
The precise definition of rationality has three powerful consequences:
Information gathering is rational. Looking both ways before crossing is not optional politeness — an action taken to improve future percepts raises expected performance, so a rational agent does it. (Crossing on an uninformative percept sequence is irrational precisely because the risk is too high.)
Learning is required. A rational agent must learn as much as it can from what it perceives, updating and augmenting its prior knowledge rather than acting on fixed assumptions.
Autonomy is the goal. An agent that leans only on its designers’ built-in knowledge, rather than its own percepts and learning, lacks autonomy. After enough experience, a truly autonomous agent’s behaviour becomes effectively independent of its initial programming.
Worked example · The dung beetle and the sphex wasp
Both insects run fixed, non-learning routines. The dung beetle will pantomime plugging its nest even after its dung ball is snatched away. The sphex wasp, if its caterpillar is nudged a few inches during its "check the burrow" step, restarts the drag-and-check loop dozens of times in a row. These are agents with rich behaviour but zero autonomy: when reality violates the evolved assumption, they fail without noticing. They are the cautionary tale for why rational agents must learn.
An agent looks both ways, then crosses — and is hit by a freak meteorite. Was it irrational?
No. Rationality is about maximising expected performance given the percept sequence, not guaranteeing a good outcome. The agent gathered the relevant information and made the choice with the highest expected value. A bad outcome from an unforeseeable event does not make the decision irrational; that would require omniscience.
Task environments — the problems agents solve
An environment is the “problem”; a rational agent is the “solution”. Before designing any agent, you specify the task environment as fully as possible. The checklist is PEAS.
Definition · PEAS
Performance measure · Environment · Actuators · Sensors. Specifying all four is always the first step in agent design.
Worked example: PEAS for an automated taxi
| P | E | A | S |
|---|---|---|---|
| Safe, fast, legal, comfortable trip; max profit; minimum impact on others | Roads, traffic, pedestrians, policy, customers, weather | Steering, accelerator, brake, signal, horn, display, speech | Cameras, radar/lidar, speedometer, engine sensors, mics, touchscreen |
Note the performance measures already conflict — speed vs. safety vs. comfort vs. profit — foreshadowing why we will need utility functions later in this course.
The seven dimensions of an environment
Every task environment can be classified along seven axes. This is one of the most heavily examined parts of the chapter. You should be able to classify a novel environment on the spot and explain why each property makes the agent’s job harder or easier.
| Dimension | The poles | Why it matters for design |
|---|---|---|
| Observability | Fully / Partially / Unobservable | If fully observable, the agent needs no internal memory of the world. Partial observability (noisy or incomplete sensors) forces the agent to maintain internal state. |
| Agents | Single / Multi (competitive or cooperative) | Other goal-driven entities change everything: in competitive settings, randomised behaviour can be rational (unpredictability); cooperation invites communication. |
| Determinism | Deterministic / Nondeterministic | If the next state is fixed by current state + action, the agent need not worry about surprises. Most real, complex environments must be treated as nondeterministic. |
| Episodes | Episodic / Sequential | Episodic: each decision is self-contained (e.g. spotting defective parts). Sequential: now affects later (chess, driving) — the agent must think ahead. |
| Dynamism | Static / Dynamic / Semidynamic | A dynamic world changes while you deliberate, so dithering is itself a choice. Semidynamic: the world is fixed but your score is not (chess with a clock). |
| Continuity | Discrete / Continuous | Applies to states, time, percepts, and actions. Chess is discrete; driving sweeps through continuous positions, speeds, and steering angles. |
| Knowledge | Known / Unknown | About the agent’s grasp of the “laws of physics” of the world, not the world’s observability. Unknown → the agent must learn how the world works. |
Pitfall · Three classic mix-ups
Known ≠ fully observable. Solitaire is known (you know the rules) but partially observable (face-down cards). A new video game can be unknown (you do not know what the buttons do) yet fully observable (the whole screen is visible).
Stochastic ≠ nondeterministic. Stochastic = possibilities come with explicit probabilities ("25% chance of rain"). Nondeterministic = possibilities are merely listed, unquantified ("might rain").
Partial observability can masquerade as nondeterminism. A deterministic world you cannot fully see looks unpredictable from the agent's perspective.
Key insight · The hardest environment is the real one
The "easy" end of every axis is: fully observable, single-agent, deterministic, episodic, static, discrete, known. The real world sits at the hard end of all seven at once — partially observable, multi-agent, nondeterministic, sequential, dynamic, continuous, unknown. That is exactly why the next chapter deliberately restricts itself to the easy end first, then later chapters relax the constraints one by one.
Classify chess (with a clock) on all seven dimensions.
Fully observable · multi-agent (competitive) · deterministic · sequential · semidynamic (board is static, but the clock changes your score) · discrete · known. Contrast with taxi driving: partially observable, multi-agent, nondeterministic, sequential, dynamic, continuous, and arguably unknown.
The structure of agents — five designs
All agents share the shape agent = architecture + programme, where the programme maps percepts to actions. The interesting question is how the programme decides. The five designs form a ladder of increasing sophistication — each fixes a limitation of the one below it.
1 · Simple reflex agents
Decide using only the current percept, via condition-action rules (“if front car is braking, then brake”). Fast and simple — but blind to history. In a partially observable world they fail badly: a vacuum that senses only its current square cannot reason about the other one. Randomising actions can rescue a stuck simple reflex agent in some cases (e.g. to escape an infinite loop).
2 · Model-based reflex agents
The fix for partial observability: keep an internal state that tracks the unobserved parts of the world. Maintaining it requires two pieces of knowledge:
Transition model
How the world evolves — both the effects of the agent's own actions and how the world changes on its own.
Sensor model
How the actual world state shows up in the agent's percepts.
Together they let the agent maintain a best guess of “what the world is like now”, even when sensors cannot see everything (e.g. tracking cars hidden behind a truck).
3 · Goal-based agents
Knowing the current state is not enough to act — at a junction the right turn depends on where you are going. A goal describes desirable states; the agent combines its model with the goal to choose actions that achieve it. This requires considering the future (“what happens if I do A, and will that satisfy my goal?”), which is the job of search and planning.
Connection · Why goal-based beats reflex despite being slower
A reflex agent brakes at brake-lights because the rule says so — it has no idea why. A goal-based agent brakes because it predicts that doing so prevents a collision. The knowledge is now explicit and modifiable: change the destination and behaviour adapts automatically, whereas a reflex agent's rules would all have to be rewritten. This explicitness is the bridge into the next chapter.
4 · Utility-based agents
Goals give only a binary happy/unhappy verdict. But many action sequences reach the goal — some faster, safer, cheaper. A utility function assigns a real-valued “how desirable” score to states, an internalisation of the performance measure. A rational utility-based agent chooses the action maximising expected utility (the utility averaged over possible outcomes, weighted by their probabilities).
Utility earns its place in exactly two situations goals cannot handle:
- Conflicting goals (speed vs. safety) — the utility function encodes the right trade-off.
- Uncertain achievement — utility weighs the likelihood of success against the importance of each goal.
Key insight · The deep theorem behind utility
Decision theory shows that any rational agent must behave as if it possesses a utility function it is maximising in expectation. Utility is not just one design choice among many — it is a constraint that falls out of rationality itself. A useful side-benefit: an explicit utility function lets one general algorithm make rational decisions regardless of the specific utilities involved.
5 · Learning agents — the four components
Turing argued it would be easier to build a learning machine and teach it than to program intelligence by hand. Any of the four designs above can be wrapped as a learning agent. A learning agent has four conceptual parts:
| Component | Role |
|---|---|
| Performance element | Selects external actions. This is the entire agent from the earlier designs. |
| Learning element | Responsible for improvements; modifies the performance element using feedback from the critic. |
| Critic | Tells the learning element how well the agent is doing against a fixed performance standard. |
| Problem generator | Suggests exploratory, possibly suboptimal actions that lead to informative experiences. |
Pitfall · Why the critic's standard must be fixed and external
The performance standard must sit outside the agent and stay fixed — otherwise the agent could "learn" how to lower the bar to match its own behaviour and declare victory. A percept alone cannot tell the agent it is succeeding (a chess programme sees "checkmate delivered" but needs the external standard to know that is good). The problem generator is what stops a purely greedy agent from never exploring — it is the scientist running an experiment that may not pay off immediately (Galileo dropping rocks to improve his theory, not to break rocks).
How components represent the world: atomic, factored, and structured
Cutting across all designs is how a state is represented. Richer representations express more but cost more to compute with.
| Representation | A state is… | Example and where used |
|---|---|---|
| Atomic | a single indivisible black box with no internal structure | “currently in Arad”. Used by the search and game-playing algorithms of the next few chapters. |
| Factored | a set of attribute-value pairs (variables) | GPS = …, fuel = …, speed = … Used in CSPs, Bayesian networks, planning, ML. |
| Structured | objects and the relationships between them | “the cow is in the truck, which is to the left of the barn”. Used in databases, first-order logic, NLP. |
What we have established
- Agent = sensors + actuators; behaviour = agent function (abstract map) realised by an agent programme (code on an architecture).
- Rational = for every percept sequence, pick the action maximising expected performance, given prior knowledge. Rational ≠ omniscient. Rationality requires information-gathering, learning, and autonomy.
- Specify any task with PEAS; classify it on the seven dimensions. The real world is the hard pole of all seven.
- Five agent designs: simple reflex → model-based → goal-based → utility-based → learning, each fixing the last one’s limitation.
- Learning agent = performance element + learning element + critic + problem generator (with a fixed external standard).
- States are atomic (Ch. 3), factored, or structured — expressiveness versus cost.
Next in this series: Chapter 3 — Solving Problems by Searching. We give the agent a goal and a model of the world, then ask: how does it find a sequence of actions to reach that goal?