One of the clearest statements of an
argument for a common currency thesis is found early in Peter Shizgal and Kent Conover’s 1996 paper ‘On the neural
computation of utility.’ This is a really
important paper for the common currency topic, and should be required reading
for anyone in the area. It doesn’t only state a version of one of the key
arguments, it also provides an exemplary kind of empirical evidence.
Here is how they state the argument:
In natural settings, the goals competing for
behavior are complex, multidimensional objects and outcomes. Yet, for orderly
choice to be possible, the utility of all competing resources must be represented
on a single, common dimension (Shizgal & Conover 1996).
The second
sentence can be slightly reformulated (making one implicit premise explicit) as
follows:
Premise 1: Orderly
choice is possible.
Premise 2: For
orderly choice to be possible, the utility of all competing resources must be
represented on a single, common dimension.
So
Conclusion:
The utility of all competing resources is represented on a single, common
dimension.
This certainly looks like a valid argument.
But what does it mean?
Well, the suggestion is clearly that (a)
the rewards available to an organism might vary in lots of ways, across
multiple dimensions (consider sex, rest, food of different kinds, grooming,
water, avoiding predators, feeding young, etc.), and (b) making ‘orderly’
decisions between available rewards, the rewards all need to have a simple
one-dimensional representation. (See The (very) Basic Big Idea on this site.)
Shizgal and Conover don’t claim that all
choice is in fact orderly. But they
clearly intend to say that when it is orderly, a common dimension of comparison
is required. But what is it for choice to be orderly? Looking at Shizgal and
Conover’s experiments will help clarify this.
The
research Shizgal and Conover report on concerns ‘brain self-reward’. A famous
paper by Olds and Milner (1954) reported that rats would work, including
learning novel behaviours, for no more reinforcement than pulses of electrical
stimulation to a part of the medial forebrain where an electrode terminated.
Some early reports of brain-self reward – as it came to be called, or (BSR) for
short – focused on ways in which it was unusual. The popular imagination got
excited by the rat that pressed its lever nearly continually for around 20 days
at about one press every two seconds (Valenstein & Beer 1964), or cases
where food was apparently ignored over brain self-reward (Routtenberg &
Lindy 1965). Shizgal and Conover sought, in part, to demonstrate that brain
self-reward is a reinforcer like any other. In a series of experiments they had
rats chose between trains of BSR pulses and infusions of sucrose solution.
The experimental design infused the juice
directly into the rats’ mouths to give the sucrose solution a key property of
BSR, which is that a single action leads
to both procurement and consumption. In addition the swallowed fruit juice
was drained away, reducing postingestive effects so that the infusions shared
another property of BSR, which is absence
of satiation.
 |
| Schematic illustration of experiment 1 from preprint version of the paper. |
Shizgal
and Conover manipulated the strength of reinforcement to each modality (by
changing the duration or number of pulses in a train of BSR, or the size of the
sucrose infusion) and measured the relative allocation of lever presses to
each. Among other things they found that each individual reward modality was
preferred over nothing, more of it preferred to less, and that increasing the
opportunity cost of either (by increasing the reward available from the other
lever) led to less consumption of that reward.
In a
variation on the experiment they manipulated the magnitude of a BSR-only
reward, while the alternative reward was a constant combination of BSR and
sucrose solution. In this condition it took more BSR to make the rat forgo the
compound reward than had been necessary for the sucrose part of the reward
alone.
 |
| Schematic illustration of experiment 2 from preprint version of the paper. |
Their
conclusion is that the results of the first experiment imply, “that on a given trial,
the rat selected the alternative that registered a larger value in a common
system of measurement”, rather than following a categorical rule (such as
‘whenever the size of one reward is above some threshold, choose it’).
Regarding the second experiment with the
combination rewards, they say it implies “that the electrical stimulation
and the sucrose were subjected to a common evaluation”.
(The paper describes two further
experiments, complementary to those I’ve just described. I’ll discuss them in a
future posting.)
So, one way of glossing this is that
Shizgal and Conover demonstrate that rat choices between BSR and sucrose
solution is orderly in two senses:
- Choices between modalities are quantitatively sensitive to opportunity cost.
- Choices between rewards in one modality and two-modality combination rewards are similarly sensitive to the combined opportunity cost.
These results are important. They show (in
terminology that I’ve described elsewhere on this site, but which isn’t used by
Shizgal and Conover) that at least some rat choices have a certain patterning,
conforming to an ultimate common
currency.
Shizgal and Conover also clearly intend the
conclusion that the results about an ultimate
common currency support the view that there is a proximal one:
“We speculate that this common ability arises from a common action of the gustatory and electrical stimuli on a neural system that determines goal selection by signaling the utility of competing goals.”
The other two experiments, and their
ongoing research programme, provide further support for this speculation. But
that’s all I’ve got time for now.
References
Olds, J. & Milner, P. 1954. Positive reinforcement produced by
electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological
Psychology, 47, pp. 419-427.
Routtenberg, A.
& Lindy, J. 1965. Effects of the availability of rewarding septal and
hypothalamic stimulation on bar pressing for food under conditions of
deprivation. Journal of Comparative and
Physiological Psychology, 60, pp. 158-161.
Shizgal,
P. & Conover, K. 1998. On the neural computation of utility, Current Directions in Psychological Science, 5(2), pp. 37-43. [Publisher’s site –may be behind a paywall] [Preprint version at CogPrints] [Google Scholar Citations]
Valenstein, E.S. & Beer, B. 1964. Continuous
opportunity for reinforcing brain stimulation. Journal for the Experimental Analysis of Behavior, 7, pp. 183-184.
Forthcoming attractions
Discussion of Shizgal (1999).
No comments:
Post a Comment