Jeffrey Goldberg’s column in The Atlantic detailing the Obama administration increasingly frosty relationship with Israeli prime minister Benjamin Netanyahu sent geopolitical tongues wagging. Details of administration officials’ list of private insults to describe Netanyahu include chickenshit, myopic, obtuse, blustering. But one description was abhorrent on a level far above politics: “Aspergery.”

Despite the fact that it is incredibly offensive to people on the autism spectrum and the disability community to use the word “Aspergery,” no one on the right or left has criticized the Obama administration for reportedly using the term. While pundits scramble to analyze international implications, no one seems especially perturbed that the highest executive office allegedly throws around a “disability diagnosis as a pejorative,” as Ari Ne’eman, president of the Autistic Self Advocacy Network, puts it.

No major figure from either party has mustered such seemingly obvious denouncement. Ne’eman is not surprised by the relative radio silence on both sides of the aisle, nor does he specifically blame Obama or the Democrats. “Unfortunately, inappropriate statements about disability are not the province of any party, faction, or political affiliation,” he says. He cited Niall Ferguson’s 2004 Daily Telegraph op-ed called “America Has Asperger’s,” in which he critiqued U.S. foreign policy. He used Asperger’s syndrome as a way to describe America’s “own bad behavior,” arguing that “a Bush re-election will look to the rest of the world like evidence that Asperger’s is no longer a treatable condition in America but has become the national norm.” So much is inaccurate about Ferguson’s analogy that it reveals how deeply ignorant he is about the autism spectrum.

Unfortunately, while we’ve discovered so much more about autism in the 10 years since Ferguson’s column, the latest political use of “Aspergery” shows we are apparently just as insensitive. Regardless of whether you’re red or blue, that lack of criticism is revealing a massive problem with how Americans perceive people with autism “It reflects a broader lack of acknowledgement that autistic people are hurt when the way our brain works is used as a political insult,” says Ne’eman.

Too often in everyday life, people throw around “Aspergery” and autistic as an insult without realizing how disrespectful it is to people who are on the spectrum. Friends refer to bad blind dates as “Aspergery” because they were bad conversationalists. Co-workers refer to their new supervisors as autistic because they seem oblivious to office dynamics. “I’ve heard it used in various professional settings and social settings,” says Ne’eman. “I think you have to think about the role that plays in making it more difficult for autistic professionals to succeed and climb up the ladder.”

In many ways, the current blasé attitude towards using “autistic” or “Aspergery” as an insult is akin to the way “gay” was a socially acceptable insult. “It comes back to looking at marginalized groups in general, people who are never quite in the room,” says Ne’eman. “It’s this broader idea that these concepts [such as “autistic” or “gay”] just exist to insult someone without the acknowledgment that they represent actual people who deserve to be treated with respect.”

The tide against using “gay” as a pejorative has only relatively recently started to turn, at least to the point where a politician would be rightly slammed for it. At least part of the motivation for that change came from people coming out of the closet and putting a human face on the insult.

In contrast, the autism community hasn’t had its big coming out moment, and that’s why people don’t question using “Aspergery” as an insult, says Ne’eman, “When we see a similar acknowledgement of autistic people as co-workers, neighbors, classmates, members of the general population, people will see it as similarly inappropriate,,” says Ne’eman. “It comes back to this idea that autism or disability is just around the corner but never in the room.”

Just because people on the spectrum have not come forward en masse does not mean they are absent from our communities and work spaces—even on Capitol Hill. “I know State Department employees on the autism spectrum. I know political operatives for both sides of the aisle on the autism spectrum,” says Ne’eman, The obvious Catch-22 is that these individuals are less likely to come forward and debunk stereotypes about autism when the president’s office reportedly tosses around their diagnosis like an insult—and worse, no one seems to care.