Rat study sniffs out vasopressin’s role in social behavior

Neuroscientists have discovered a population of cells in the smell-perception area of the rat brain that express the hormone vasopressin. The study, published in Nature, begins to unpack the complicated molecular interactions of the hormone in the brain, which could lead to new autism treatments.

By Virginia Hughes
7 May 2010 | 9 min read
This article is more than five years old.
Neuroscience—and science in general—is constantly evolving, so older articles may contain information or theories that have been reevaluated since their original publication date.

Bulb buds: Engineering every brain cell expressing vasopressin to glow green revealed that one group of the cells sits in the olfactory bulb.

Neuroscientists have discovered a population of cells in the smell-perception area of the rat brain that express the hormone vasopressin. Blocking vasopressin in this region, called the olfactory bulb, impairs the animals’ ability to recognize other rats by smell, the study found.

Vasopressin plays vital roles in many body tissues, such as regulating water absorption in the kidneys and salt content in the blood.

But in the past few years, the hormone has attracted attention for its role in complex social behaviors. For instance, some studies have found that individuals with autism, who have impaired personal relationships, have increased levels of vasopressin in blood plasma compared with healthy controls1.

The new rat work, published 18 March in Nature2, adds to several rodent studies showing that vasopressin and its sister brain chemical, oxytocin, control a range of social and reproductive behaviors, from wooing a mate to caring for pups. After years of striking discoveries in animals, geneticists and neuroscientists are beginning to take a closer look at how the hormones affect human social behavior, according to a review published 25 March in Neuron3.

Vasopressin and oxytocin differ chemically by only two amino acids, and can bind to each other’s receptors. They are both produced primarily in the brain and then travel through blood to the rest of the body. Both hormones are released during stress, and their actions seem inextricably tied: vasopressin elicits an active, fight-or-flight response, whereas oxytocin dials down the resulting anxiety.

A few small genetic and clinical studies have linked both hormones to autism.

For example, a February report described a high-functioning man with autism who is missing both olfactory bulbs4. A couple of other studies have also hinted at a problem with the olfactory system in people with autism5. The researchers hypothesize that because olfactory bulbs contain a high density of oxytocin and vasopressin receptors, damage to this region could severely affect social development.

Other groups have reported this year that inhaling oxytocin improves social behaviors6 in individuals with autism and helps teenage boys with the disorder recognize facial emotions7. But scientists know relatively little about how these hormones affect neural circuits.

The new study begins to unpack the complicated molecular interactions of vasopressin in the brain, which could lead to additional autism treatments, researchers say. Nasal sprays are not ideal, because researchers don’t know how the hormones get into the brain, nor which specific regions they act on.

“Maybe in the future, one way to help would be to activate these vasopressin cells in the olfactory bulb by a trick we don’t know yet,” says lead investigator Mike Ludwig, professor of neurophysiology at the University of Edinburgh.

Receptor roots:

The vasopressin protein fits into the oxytocin receptor, but it binds most strongly to three different receptors, one expressed in the kidneys and two — AVPR1a and AVPR1b — in the brain.  

Human studies have tied the gene that encodes AVPR1a to a wide variety of behaviors, from creative dance performance8 to having troubled romantic relationships. Studying 552 twin pairs and their spouses, one group found that men who carry certain common alleles in AVPR1a are less likely to be married or, if they are married, more likely to have marital problems, than are controls9. Other researchers have reported links between AVPR1a variants and age of first sexual experience10 and altruistic tendencies11

Since 2002, five studies have found that common variants of AVPR1a are more likely to arise in individuals with autism than in healthy controls12. All of these studies had small sample sizes, however, and fingered variants in different parts of the gene.

“It’s somewhat disturbing that none of the studies exactly replicate any of the others,” notes Richard Ebstein, professor of psychology at Hebrew University in Jerusalem. His group has investigated AVPR1a’s role in autism, eating disorders, stress responses and impulse control.

Although these association studies indicate that variants in the vasopressin receptor are not primary causes of autism, they may influence the severity of an individual’s social impairments, Ebstein says.

Because there are few treatments for autism’s core social problems, these hormonal pathways deserve more attention, he adds. “In most cases of autism, vasopressin and oxytocin aren’t major explainers of illness, but they might still be very good targets for pharmacology.”

Animal work suggests that the specific expression patterns of vasopressin receptors in the brain control animals’ social and mating behaviors.

For example, researchers at Emory University compared vasopressin receptor coding and expression in the brains of montane voles, which are polygamous, and prairie voles, which are monogamous. Montane voles carry short sequence repeats in a specific region upstream of the AVPR1a gene, whereas prairie voles carry long repeats, the study found13.

What’s more, animals carrying long repeats have a high expression of AVPR1a in the olfactory bulb, suggesting that vasopressin signaling in this region is crucial for pair bonding.

Local connections:

Scientists had assumed that all of the brain’s vasopressin is produced in the hypothalamus — the nub of cells in the center of the brain — and then travels to the olfactory bulb to regulate social bonding behaviors.

But the new Nature study shows that vasopressin is also produced by a group of cells within the bulb.

“That’s what’s exciting — the finding that there’s a new population of neurons that are producing vasopressin locally, rather than having to diffuse a long distance from the hypothalamus,” says Elizabeth Hammock, instructor of pediatrics at Vanderbilt University, who did the vole work as a graduate student.

Ludwig’s team engineered a line of rats with green fluorescent protein genetically inserted into a vasopressin pathway. After the rats are born, every brain cell that makes vasopressin glows green. The researchers found that, surprisingly, one group of these cells sits in the olfactory bulb.

In subsequent experiments, the team used several methods to selectively block the activity of these cells in the olfactory bulb. This severely disrupts the animals’ social recognition, causing them to sniff a familiar rat as if it were a stranger. Interestingly, blocking these cells does not affect the animals’ ability to recognize objects.

In electrophysiological experiments, the scientists found that these vasopressin cells do not project outside of the olfactory bulb. Instead, they link up to other types of cells inside the bulb, which then project deeper into parts of the limbic system, such as the amygdala and hippocampus, important for processing emotions and forming memories.

Ludwig suspects that when rats smell something new, the odor “primes” the vasopressin cells in the olfactory bulb. The second time the animal smells the odor, vasopressin effectively inhibits connections to these deeper areas, which dampens the rat’s emotional response.

The nose knows:

In people, most sensory input, such as light or touch, gets routed through the brain’s relay center, the thalamus, before reaching the limbic areas. The notable exception is smell: odor molecules take a shortcut, going directly from the olfactory bulb to deeper brain structures for processing.

“It has primitive and privileged access to our emotion regulation,” Hammock says.

Even so, scientists studying human social behaviors have paid little attention to the olfactory system. That’s because, unlike rodents, which are dependent on their smelling ability, the olfactory bulb doesn’t take up a huge portion of the human brain. People rely much more on their eyes.

“We don’t sniff each other to recognize who is who anymore. Humans have evolved a bit further,” Ludwig says.

It’s unclear, he says, whether the activity of vasopressin cells in the rat olfactory bulb is reflective of the hormone’s workings in the healthy human brain.

Ebstein says that vasopressin signaling in the amygdala, rather than in the olfactory bulb, is more likely to be important in autism. A functional magnetic resonance imaging study last year found that individuals carrying various autism-associated variants of AVPR1a have abnormal amygdala activity — some variants leading to over-active, and others to under-active, amygdalae — when performing a task of matching faces14.

In unpublished experiments on college students, Ebstein’s team has found that sniffing vasopressin before giving a speech to a panel of judges can increase heart rate and blood cortisol levels — standard measures of stress response. This increased anxiety doesn’t occur when the students make the speech facing a wall.

Ebstein hypothesizes that in the presence of other people, vasopressin stimulates the amygdala, which then sets off a hormonal cascade that results in cortisol release. “Vasopressin needs an audience — it doesn’t work well unless you’ve got that social stress element,” he says.

Although the hormone’s effect on the brain is uncertain, the mounting genetic and behavioral evidence is prompting more neuroscientists to take note, he says. “Because there are no drugs to treat autism’s core social problems, these hormones deserve as much attention as we can give them.”