ROME — There is often a fine line between pity and fear when it comes to the hundreds of thousands of refugees who have landed on European shores in the last year.

On one hand, it’s difficult not to feel an outpouring of sympathy over pictures of babies born on perilous rescue missions, or the bodies of children washed up on the waves. Just as it is almost as difficult not to feel distrust and anger over news that, yet again, alleged jihadists have been found hiding among the legitimate refugees.

This week three would-be terrorists were arrested in the Schleswig-Holstein region of Germany on suspicion they were operatives of the so-called Islamic State. The three men, referred to in German court documents which do not include last names as Mahir al-H., 17, Ibrahim M., 18, and Mohamed A., 18, reportedly came to Europe last year through Turkey to Greece, where they essentially rode the migrant wave all the way to Germany. As Syrians, their asylum applications get preferential treatment, allowing them, essentially, to cut the line at the borders.

But before grouping all refugees into the terrorist camp, it is important to note that the three men would not have been caught at all if it had not been for other refugees who raised the alarm several months ago. According to a spokesman for the BKA, Germany’s counter-intelligence force, the trio were trying to recruit others among the pool of mostly idle men waiting for their papers to be processed and they’d been fingered by some of those they approached. As a result, they had been under surveillance for many months, which allowed authorities to gather valuable information about the wider ring.

It’s also the case that while refugees and migrants are living in camps awaiting word on their status requests, they are often subject to practices that invade their privacy in ways regularized citizens would never accept. Counter-terrorism police in Italy, who are part of the country’s anti-Mafia forces, don’t even try to hide the fact that refugee phones often are tapped and that there are undercover faux refugees at most major camps for the sole purpose of spying on them.

In the Sicilian port of Augusta, a number of former refugees are now working for local authorities to help weed out any extremists. One man who goes by the name Leo told The Daily Beast last year that he can spot a fake refugee within 10 minutes of a conversation. “Either they don’t know the dialect of the area they pretend to be coming from, or they fall immediately into a trap when I start complaining about the West,” he said. His job is to meld into the groups of men that have been separated from the refugee families and just start talking to them.

Italian authorities also have native Nigerians, Afghans, and Iraqis on the payroll, depending on the demographics of the migrant boats.

One key piece of information netted by such covert intelligence work: German intelligence is certain that the three men arrested this week had traveled to Leros, Greece, together on the same boat with two of the suicide bombers who blew themselves up in the deadly Nov. 13, 2015, attacks in Paris. One suicide bomber, whose fingerprints collected from the Paris attacks match those of a man identified in refugee records as M. Al-Mahmod, was in the same Greek camp where the men arrested this week first entered.

At least one of the three just arrested also apparently had connections to the group responsible for the August 2015 attack on a Paris-bound train.

German authorities say all of these dots were connected long before arrests were made, thanks to testimony from legitimate refugees who traveled with these five men. For their cooperation, the German authorities say the key witnesses have been moved on to other countries and their asylum applications have been granted.

What complicates the hunt for terrorists among the real refugees is the cottage industry of document forging spurred by the migratory crisis. Fake documents can make anyone seem legitimate—at least until they’re not. Last month, Europol found a whole cache of fake documents in Greek refugee camps apparently designated for foreign fighters coming in to the continent.

Last spring an Iraqi man was arrested in Italy on suspicion of running guns and falsifying documents for similar purposes.

Authorities also believe that there are a number of human trafficking outfits that cater specifically to would-be terrorists, ensuring safe passage hidden among the legitimate refugees.

“Everything points to the fact that the same smuggler organization behind the Paris attacks also brought the three men to Germany,” Germany’s interior minister Thomas de Maizière said at press conference when the arrests were made this week. He says there are at least 520 “potential fighters” in Germany alone, and described how some fighters were part of “hit teams” that sneak in under orders from ISIS commanders.

Others, he says, are “lone wolves” who are either recruited along the way or once they get to the camps. Still others go to the Middle East for training and then come back to carry out attacks. “Some operatives are working on their own; others are spontaneously inspired by other attacks, then there are returnees from crisis zones.”

Just over 28 percent of the 298,099 migrants and refugees who have arrived in Europe by sea since the start of this year are from Syria, according to the United Nations Refugee Agency UNHCR. The rest are from troubled spots like Afghanistan, Iraq, Nigeria and Eritrea. There are Palestinians, Sudanese and other nationalities among the mix as well. Of the total, 54 percent are men; the rest are women and children.

Europe’s border patrol organization Frontex, which cruises the Mediterranean Sea along with the Italian navy and private rescue organizations, has long warned that terrorists find an easy ride with legitimate refugees, which is why it has argued against the rescue operations like Italy’s Mare Nostrum and those in effect now, which it says create a “pull factor.” Frontex argues the only deterrent is to turn back the boats.

“The Paris attacks in November 2015 clearly demonstrated that irregular migratory flows could be used by terrorists to enter the EU,” Frontex reported in its risk analysis report for 2016. “Two of the terrorists involved in the attacks had previously irregularly entered through Leros and had been registered by the Greek authorities. They presented fraudulent Syrian documents to speed up their registration process.”

It is little wonder that Europe remains divided about how to protect itself and still save lives at sea and provide humanitarian aid to those in need.

“It is wrong to put refugees under general suspicion,” says de Maizière. “But the fact is we do have refugees who come here as potential terrorists or sympathizers.” The rest, of course, do deserve legitimate sympathy.

The hard part is telling the difference while protecting people in one group and protecting Europe from those in the other.