In the U.S. Federal court system, many important cases go through three-judge panels. The majority opinion of these panels carries the day, meaning that having a majority is crucial for one side or another to get the rulings they want. So, if two out of three of the judges are appointed by Democrats, it’s safe to assume that most cases will go their way.

But a study of the judicial behavior of the District of Columbia Circuit came to a surprising conclusion: A panel of three GOP-appointed judges was actually considerably more likely to make a conservative ruling than a panel of two GOP appointees and one Democratic appointee. Just one Democratic dissenter appeared to make the difference; the dissenter apparently swayed their colleagues, demonstrating how viewpoint diversity has the power to alter the conclusions of a group.

This court study is among many cited by legal scholar Cass Sunstein in his new book Conformity: The Power of Social Influences, which delves deeply into how and why individuals often follow the opinions and behaviors of groups they belong to.

The upside and downside of conformity

While the book does warn of the downsides of conformity, Sunstein doesn’t declare that conforming is always harmful to society. On the contrary, he reiterates numerous circumstances when society can benefit from it.

For instance, Sunstein notes how conformity helped encourage public smoking laws. One study found that when public smoking bans were enacted in three California cities, compliance was high, and the cities received few reports of violations. Sunstein believes that the law had an impact not because of the threat of state enforcement, but because “the law suggests that most people believe it is wrong to smoke in public places. And if most people think it is wrong to smoke in public places, would-be smokers are less likely to smoke, in part because they do not want to be criticized or reprimanded.” In other words, the power of a popular law is due partly to conformity.

But conformity also carries with it the power to make human beings ignore their own consciences, sometimes to the point of committing atrocities.

The book points to Stanley Milgram’s infamous experiment in which participants were told to deliver a series of electric shocks to another participant (actually an actor working as the researcher’s confederate), slightly increasing the intensity every time. While the experiment was a ruse, the participants didn’t know that. Milgram found that all of the participants were willing to shock the confederate at 300 volts, and two-thirds continued to administer shocks at the very highest level of voltage. The participants were simply willing to trust the instructor that what they were doing was okay.

What drives conformity

In order to understand how conformity works—from fairly banal examples such as public smoking bans all the way up to atrocities committed during World War II—Sunstein breaks it down into its component parts:

Informational signals: Sunstein suggests that participants in Milgram’s experiment were willing to follow orders because they believed the experimenter to be a trusted expert who was assuring them that the shocks were causing no lasting harm. This represents an “informational signal”—a batch of information sent out by a trusted expert or a crowd that can help you decide how you feel or act. Signals from in-groups—people you like, trust, or admire—are far more valuable than information signals from out-groups.

Reputational signals: We may have private qualms about a point of view or given course of action, but because we want to remain in the good graces of our social grouping, we suppress our dissent and eventually fall in line. This is particularly apparent in how social media polarization operates, where people gain prestige and influence when agreeing with their cohort’s biases rather than opposing them.

Social cascades: Sunstein identifies both informational and reputational signals as helping produce social cascades: “large-scale social movements in which many people end up thinking something, or doing something, because of the beliefs or actions of a few early movers.” He identifies everything from the success of Jane Austen novels to the elections of Barack Obama and Donald Trump as cascades.

To demonstrate how a cascade can work, he cites a study by sociologist Duncan Watts, in which study participants were asked to rank a group of seventy-two songs from best to worst. A control group was not given any information other than the songs themselves. But eight other subgroups could see how many people had previously downloaded the songs within their subgroup.

Watts found that the songs the control group had labeled as the worst songs generally ended up toward the bottom, while the ones the control group favored generally ended up toward the top. But for most of the other songs, a burst of popularity based on early downloads predicted how well they did in the rankings. In other words, people gave higher rankings to songs they perceived as popular among their group. Results like these may explain why companies marketing certain products often try to grease the wheels of sales by creating an impression of popularity before the product is actually popular.

How conformity drives polarization

The power of conformity and cascades has deep implications for political polarization. Sunstein notes that “like-minded people go to extremes,” and cites three factors for why this happens: “information, corroboration, and social comparison.”

In homogeneous groups, people tend to deal with a limited pool of information. If you are in a social group whose members tend to be opposed to abortion rights, it’s unlikely that you will ever hear any argument in favor of these rights. With your limited information, you are more likely to move in the direction of opposing abortion rights rather than supporting them.

Corroboration comes into play because people who lack confidence in their views tend to have more moderate opinions. As Sunstein writes, people “who are unsure what they should think tend to moderate their views. It is for this reason that cautious people, not knowing what to do, are likely to choose the midpoint between relevant extremes.” But if you surround yourself with people who share your views, this will end up corroborating your beliefs. In this sort of environment, you will become more confident that you are correct and be more likely to move in an extreme direction.

Social comparison leads us to want to be perceived favorably by members of our group. If our group is strongly in favor of gun control, we will naturally gravitate to that position to win applause from our group.

Thus, these three factors together demonstrate how excessive conformity can drive polarization.

What can we do to lessen conformity’s downsides?

For Sunstein, the downsides of conformity are most concerning in his profession: the law. He believes that conformity can undermine our system of deliberative governance, the courts, and the undergraduate and law school education.

The book argues in favor of the checks and balances that exist in the federal system, where cascades can be broken by a House and Senate that are often at odds, for instance. He also argues that freedom of association provides a safeguard against informational and reputational influences that can lead people to conform without considering the downsides of a point of view or plan of action.

Citing the raft of studies showing that the presence of a dissenting judge on federal panels can significantly change outcomes, Sunstein argues for greater diversity on the federal bench. “My only suggestions are that a high degree of diversity on the federal judiciary is desirable, that the Senate is entitled to pursue diversity, and that without such diversity, judicial panels will inevitably go in unjustified directions,” he writes.

Lastly, Sunstein dives into the debate over affirmative action in higher education. He offers a somewhat nuanced view: Racial diversity—the main topic of many higher education debates—can in some circumstances be important, but is not a cure-all. He ultimately favors “cognitive diversity”—meaning, law school classrooms should have rigorous debates with many points of view represented. To the extent that racial and cultural diversity helps promote those debates, Sunstein appears to be in favor. But he also argues that there are many paths to an ideologically diverse classroom.

In his conclusion, Sunstein again concedes that conformity can sometimes benefit society. “In some settings, conformists strengthen social bonds, whereas dissenters imperil them, or at least introduce tension,” he notes.

But ultimately, he comes down on the side of arguing that we could use a little less conformity.

“Much of the time, it is in the interest of the individual to follow the crowd, but in the social interest for individuals to say and do what they think best,” he writes. “Well-functioning institutions take steps to discourage conformity and to promote dissent, partly to protect the rights of dissenters, but mostly to protect interests of their own.”