In 1831, Charles Darwin made his historic five-week tour of the Galapagos Islands aboard the HMS Beagle, changing the course of scientific history and laying the foundation for modern ideas about the history of life. But he almost didn’t make it on to the ship’s register. The captain of the Beagle was fond of the works of a Swiss poet named Johann Lavater, who adduced that, “An exact relationship exists between the soul and the body, between the internal and the external of man.” And no man with a nose as bulbous as Darwin’s, the captain thought, could possibly be vigorous or intelligent enough to serve as the ship’s naturalist. Darwin wrote later of the captain that “he was afterwards well-satisfied that my nose had spoken falsely.”

The human brain, it’s been said, is a prediction machine, and man has always looked for ways to extract meaning from patterns, to separate the useful information from the static. The desire to gain an informational edge has led to methods of prediction that have ranged from the shamanistic (such as hippomancy, the interpretation of the neighing of horses) to the pseudoscientific (phrenology, the reading of the shape of the skull). In The Tell: The Little Clues that Reveal Big Truths About Who We Are, psychologist Matthew Hertenstein marshals a collection of studies to argue that the mind, all on its own, is a much more accurate predictor than we might have thought. To find the most reliable method of various forms of predicting human behavior (from the macro, such as which CEO will lead a company to the highest profits, to the micro, such as whether someone is more likely to be aggressive or docile) we need look no further then our own intuition.

The prime mover here is survival instinct. In one study that Hertenstein cites, subjects only needed to glimpse images of different men’s faces for a few milliseconds to be able to accurately state which faces correlated with a history of aggressive behavior and which posed less of a threat. It was determined, after controlling for a myriad of variables, that what the subjects were primarily keying off of was the width of the face compared to the height; in short, our minds are conditioned to be more wary of wide-faced men. “Remember that these are merely averages,” Hertenstein hastens to emphasize, and “plenty of beefy-faced men are giant teddy bears,” but the fact remains that facial ratios are a reliable cue for predicting behavior. Hertesntein is not particularly committed to deciding whether intuitions such as these are the product of evolution or learned experience. As he says, “Genetics deals the cards, and environment plays the hand.” But it seems that we can safely assume that we’ve learned, either over the course of millennia or our own life spans, to spot and avoid the tyranny of wide-faced men. (Interestingly, ratios in the faces of women had no correlation with aggressive behavior.)

Avoiding aggression is one way to survive, but so is choosing a strong leader. In another study, a group of Swiss children as young as five were asked to play a game in which they were told that they were going to sail on a boat from Troy to Ithaca. They were then shown pictures of candidates running in French elections, and asked to chose which candidate they would select to lead their ship on its voyage. The children chose the winning candidate with an accuracy of 71 percent. The variable in this case is more mysterious (no reliable ratios here), but it would seem that we make snap judgments of competency based on appearance.

Whether facially projected competency correlates with actually political competency would be much harder to prove, but our intuition can also be used to predict a much more objective element: profit. In a study conducted at Tufts University, subjects were given a glance at the faces of the CEOs of Fortune 500 companies, and asked to rate them based on three criteria: Power, warmth, and leadership. Sure enough, those CEOs who most exuded power and leadership (but not warmth) were shown to be those who led the most profitable companies of the sample. Additionally, MRI readings showed increased brain activity in the subjects while observing the photos of the more successful CEOs. When the researchers looked for the primary variable in the CEOs themselves, the “tell” was, again, a wide face. So although you might want to avoid facial width in dark alleys, it’s a different story in boardrooms. (Although Hertenstein “wouldn’t suggest purchasing stock based on the CEO’s facial breadth.”)

Like most arguments under the tent of evolutionary psychology, “problematic” issues abound, and Hertenstein is reluctant to ever make a jump from correlation to causality. He’s also well aware of how these types of studies can be misinterpreted, or used to justify backwards ideas such as those of phrenologists or outright racists. But it’s made clear that data like this can, and should, be used to highlight and undermine harmful biases rather than support them. When considering the findings, he writes, “One should consider the influence of culture and race.” For instance, the Japanese do not conflate aggression with success to the same degree as Americans, and the black male CEOs who led their companies to the most success were actually shown to project warmth, rather than power, which would perhaps “mitigate stereotypes of black males as threatening.” This is an important point. Although it appears that our brains can accurately predict, at a glance, the success of a CEO, why exactly that CEO is successful is another matter entirely. In other words, perhaps we are sensing not which face conveys the most capacity for effective leadership, but rather which face is most equipped to benefit from prevailing cultural prejudice and societal structure.

Hertenstein bridles at the assertion that psychology is nothing more than common sense, although he concedes that it would be silly to be surprised to hear that, for instance, children who are read to by their parents are more likely to grow up to be readers. But the studies here straddle a fine line between common sense and something much darker: the idea that the physical dictates the moral. Surely we all have had the experience of seeing a face and deciding instantly that it belongs to an unsavory or unintelligent customer. Surely we are right to feel shame in these moments, just as surely as Darwin’s captain would have made a historical blunder if he excluded him. It’s not enough, as Hertenstein says, to “keep reminding yourself—as I try to—that the predictions that science tells us we are capable of making are based on probability and not certainty.” If our brains have held on to once-useful Neolithic prejudices, perhaps it’s time to bid them a fond farewell. And if you see this wide-faced book reviewer on the street, even at night, feel free to say hello.