This paper changed my life: Erin Calipari ponders the nuances of rewarding and aversive stimuli

Answers have been edited for length and clarity.

What paper changed your life?

Schedules using noxious stimuli. VI: An interlocking shock-postponement schedule in the squirrel monkey. Kelleher, R.T. and W.H. Morse Journal of the Experimental Analysis of Behavior (1969)

Raymond Kelleher and William Morse showed that under certain conditions, animals will press a lever to shock themselves, a counterintuitive result that demonstrates that positive reinforcement can occur even when the stimulus is a shock. Researchers compared how many times squirrel monkeys would press a lever under an “interlocking schedule,” in which successive lever presses led to a shock occurring earlier in time, versus an “alternative schedule,” in which each lever press resulted in a shock after the same fixed amount of time. The researchers observed that monkeys pressed the lever more often during the interlocking schedule. 

The pattern of behavior the researchers observed during the interlocking schedule closely resembled the positive reinforcement maintained by food reward under the same timing conditions, demonstrating that shocks could actually reinforce behavior like a traditional reward. They examined a range of reinforcement schedules and ultimately showed that lever presses could either be reinforced or suppressed depending on the rules of the task. These findings demonstrate that whether a behavioral response occurs or not depends less on whether outcomes are rewarding or aversive and more on the rules of the tasks and how an animal’s actions can control its experience in an environment.  

When did you first encounter this paper?

I first encountered this paper in graduate school at Wake Forest School of Medicine in a short course in which faculty at Wake Forest and the University of North Carolina came together to teach the principles of behavioral pharmacology. We went through a huge number of papers that established the field and discussed how researchers used this information to operationalize what they were studying—specifically reporting data as what they were measuring (i.e., negative versus positive reinforcement)—rather than inferring an animal’s intention based on what the experimenter thinks the animal is doing. This class shaped my research more than any other class throughout my career. 

Why is this paper meaningful to you?

It changed how I think about linking neural circuit activity to behavioral control. In neuroscience, behavior is often treated as a readout of internal states, and we frequently infer what guides animal behavior using our own intuitions of why they behave in a certain way. When we study reinforcement and reward processing, a common assumption is that behavior is driven by the value of a stimulus. Positive or rewarding stimuli promote approach; negative or painful stimuli, such as shock, promote avoidance. 

This paper shows unequivocally that this assumption is not always true. 

How did this research influence your scientific trajectory?

Early in grad school, I thought that a way to tell if something was rewarding was whether it was reinforcing. Would animals lever press for it? I assumed that aversive stimuli mainly suppressed behavior. But after reading this paper, I started to think more deeply about factors that influence how behavior is generated and maintained. Once I started to use neuroscience techniques in vivo, such as optogenetics, these kinds of studies were always in the back of my mind. This work made me examine what the signals I was recording meant for behavior and challenged me to consider what other aspects of behavior I was missing by using my own intuition to infer why a certain behavior was occurring. 

I spent most of the early part of my career studying circuits that are often described as part of the “reward system.” But understanding the distinction between reward (the subjective value of a stimulus) and reinforcement (how that stimulus controls behavior) is nuanced. 

Just because something is rewarding doesn’t mean it always has to reinforce behavior. Stimuli that are categorized as aversive can drive behavior in ways you wouldn’t expect. This reshapes how we think about the relationship between value and what an animal ultimately does. After reading this paper, I started thinking more broadly about the factors that guide behavior across a diverse range of contexts. Ultimately, this shifted my focus away from assessing whether an experience was “good” or “bad” and more toward understanding how timing, feedback and structure within behavioral tasks influence behavior. Because of this, my lab has spent a great deal of time researching how circuits linked to motivation are driven by factors other than valence, such as salience and novelty.

Is there an underappreciated aspect of this paper you think other neuroscientists should know about?

One of the things I like about this paper is just how focused it is on the specific principles that give rise to behavior. It is a simple paper, but the systematic approach they took had a huge impact on how this field developed.

Another thing that I find particularly interesting about the paper is that the researchers didn’t go into this work trying to prove or disprove a large theory. They were interested in understanding punishment and then made a simple, almost accidental, observation. They didn’t throw out the data because it didn’t meet their expectations of what should be happening under these conditions. Instead, they used this surprising finding to understand behavior on a deeper level. These kinds of papers, even though they are narrower in scope, reshaped our understanding of a field in a way that had a lasting impact.