On Tuesday, Judge Richard Leon held that the National Security Agency’s controversial phone records program likely violates the Fourth Amendment’s guarantee against “unreasonable searches and seizures.” But when the inevitable appeal comes, far more than a single surveillance program will be at stake. Whether far higher courts are prepared to embrace Leon’s logic could determine if Americans enjoy any meaningful constitutional protection against government monitoring in the information age.

The NSA program—a massive database that logs, and stores for five years, the time, date, duration, and number dialed for nearly every call placed in the United States—is based on Section 215 of the Patriot Act, which authorizes the government to obtain any records it reasonably believes are “relevant” to a foreign intelligence investigation. But that authority itself depends on the so-called “third party doctrine,” which says that business records held by a “third party” like a phone company aren’t protected by the Fourth Amendment.

If not for the third party doctrine, “relevance” would not be enough: The government would have to satisfy the Fourth Amendment’s far stricter demand to show “probable cause” that records it had “particularly described” would yield evidence of wrongdoing. Under Fourth Amendment standards, a program that involved vacuuming up billions of records in order to fish through them later for suspicious calls would be out of the question—the kind of unlimited “general warrant” the framers of the Constitution were especially concerned to prohibit.

The roots of this cramped reading stretch back to 1979, when the Supreme Court unwittingly dealt a profound blow to American privacy in the case of Smith v. Maryland. With the cooperation of the phone company, police had traced a series of obscene phone calls from Michael Lee Smith to a woman he had earlier robbed. Because they had not first obtained a warrant from a judge, Smith argued that the police had conducted an illegal search, akin to a wiretap.

The Court disagreed: Because Smith should have known, based on the itemized list of calls on his monthly bill, that the phone company kept business records of the numbers he dialed, he had voluntarily abandoned his “reasonable expectation of privacy” in that information—and with it, the protection of the Constitution.

Fast forward to 2013, and mindboggling amounts of information about each of us are now stored indefinitely in an ever-growing number of “third party” digital databases. Every time you make a credit card purchase, carry your cell phone to meet a friend, send an e-mail, make a call, or read a website, you leave a trail of virtual breadcrumbs that, taken together, paint a staggeringly detailed and intimate portrait of your life.

The vast expansion of government surveillance powers over the past 12 years has depended crucially on the logic of Smith—and the “third party doctrine” it birthed—to permit those breadcrumbs to be vacuumed up in bulk, without such legal niceties as “individualized suspicion” or “probable cause.” If it’s in some corporate database, the now-familiar refrain goes, it’s outside the umbrella of the Fourth Amendment.

That’s the theory that supports not just the phone program, but the broader Section 215 power, which also apples to any sort of business records held by your Internet provider, your doctor, your school, your credit card company, you name it.

Then there are National Security Letters, which allow the FBI to obtain many types of records without any advance judicial approval. The Bureau issued more than 15,000 of those last year, affecting the records of more than 6,000 Americans—most of whom, it seems safe to assume, were not terrorists.

The third party doctrine was also the basis for a now-defunct program that, until 2011, appears to have collected e-mail and Internet metadata in bulk from Internet providers. Once again, the court that approved this invoked Smith v. Maryland, even while correctly noting that Internet providers don’t normally keep such information in ordinary business records.

Local law enforcement has gotten in on the act as well, though not on the NSA scale. Because cell phone carriers hold so much juicy information that can be obtained without a warrant—including information about a user’s location—they get more than a million government requests for records each year. At least 9,000 of these requests last year were for “tower dumps,” where police ask for records of every cell phone user who was in a particular area at a specific time, usually in hopes of whittling down a list of suspects.

From a Supreme Court ruling that allowed police to examine a few days worth records about calls to one number, we’ve arrived at secret courts holding that the Fourth Amendment has nothing to say about capturing comprehensive records of everyone’s phone records for years at a time.

The government justifies this, as we’ve learned from newly released court documents, with what we might call a “multiplication by zero” argument. If no constitutional rights are infringed by obtaining one person’s records for a few days, as police did in the Smith case, well… a billion times zero is still zero. By this logic, no Constitutional rights are violated by continuous monitoring of all Americans, as long as it’s done using corporate records.

This is a bit like arguing that if a daily glass of wine with dinner won’t harm your health, downing a few bottles must not either, and fortunately, Judge Leon wasn’t buying it. “When do present-day evolutions in the Government’s surveillance capabilities, citizens’ phone habits, and the relationship between the NSA and telecom companies become so thoroughly unlike those considered by the Supreme Court thirty-four years ago that a precedent like Smith simply does not apply?” asked Leon rhetorically. “The answer, unfortunately for the Government, is now.”

Implicitly, Leon was embracing the “mosaic theory” endorsed by at least five justices in a recent Supreme Court case, United States v. Jones—and also by the court that will likely hear the appeal of Leon’s ruling. The concurring opinion in Jones reasoned that, even though we generally have no right to privacy in the trips we take on public roads, using technology means to constantly monitor hundreds of such trips over a period of months could, cumulatively, become a Fourth Amendment “search.”

This theory certainly raises a lot of thorny questions: How many records is too many? Does it depend on the type of record? Is it only the level of individual intrusion that matters, or should the Fourth Amendment—which read literally, protects a “right of the people,” not just individual “persons”—be read as limiting the scale of monitoring as well as the intrusiveness? Given how many intelligence and law enforcement practices rest on the “third party doctrine”—though none quite as sweeping as the NSA program—this is a can of worms most courts would be reluctant to open.

The alternative, however, is a world in which the Fourth Amendment becomes increasingly irrelevant—protecting you from police bursting into your home, but leaving enormous swaths of all our digital lives an open book to government. Even those who support this specific NSA program should be wary of that outcome.