This week, The New York Times published a shocking front-page article on a study claiming that African women using injectable hormonal contraceptives like Depo-Provera have double the risk of contracting HIV from an infected partner, and nearly triple the risk of passing on the disease to uninfected men as well. The study has caused heated debate among those working in the fields of reproductive health and family planning. If the results of this study are to be believed, hundreds of thousands of women could face the impossible choice between preventing unwanted pregnancies—and all their associated risks—and avoiding the transmission of HIV. Damned if they use birth control, damned if they don’t.

Problem is, the study is hardly a smoking gun that exposes a concrete link between birth-control use and HIV infection. Rather, it is methodologically flawed, and contrary to what the study suggests, the jury is still out about the real risk of using birth control—if there is any risk at all.

It’s crucial to get this information right, because the lives of hundreds of thousands of women hang in the balance. Every year, nearly 500,000 women—mostly in the developing world—die from complications related to childbirth. That’s the equivalent of three 747-airplane crashes each day. For many of these women, health-care facilities are often too far away, too expensive, and too short-staffed to help in the case of pregnancy-related emergencies. In rural Ethiopia, where our research group has spent years working to make pregnancy safer for local women, a mother has a 1-in-40 chance of dying from childbirth in her lifetime. In the United States, that chance would be 1 in 2,000.

That’s why injectable contraceptives like Depo-Provera are so important to the more than 6 million women in sub-Saharan Africa who use them as a crucial tool for pregnancy prevention. By simply getting a shot once every three months, women who live far from health-care facilities have a reliable way to prevent unwanted pregnancies. Also, for women who feel they cannot or do not want to let their male partners know they are on birth control, Depo-Provera is a discreet and appealing option.

Clearly, injectable contraceptives are a critical tool for health providers in sub-Saharan Africa. Then again, no doctor would recommend that a woman place herself at grave danger of contracting HIV. That’s why, as scientists, we are so concerned by the obvious limitations of the new study. Perhaps the most significant drawback to the study is the fact that it wasn’t originally designed to look at the link between HIV and birth control. Rather, it was a randomized controlled trial to see whether herpes medication could help prevent HIV transmission. The trial was carefully designed to balance two groups—couples receiving herpes treatment and those not receiving it—and make sure they were as similar as possible. Whether a woman did or did not use contraceptives was not part of the researchers’ original concern when creating the groups.

Because the study was not designed to answer their question, the researchers could not account for all the factors that might be different about women who use contraception and those who do not. In the absence of randomization, trying to compare those two groups of people is notoriously difficult, especially when there may be crucial differences in things like sexual behavior and, say, condom use—which is known to prevent the spread of HIV. Without complete information about such factors, it would be impossible to conclude that injectable contraceptives, rather than inconsistent condom use, are to blame for the increase in HIV transmission rates. And while the researchers did try to adjust for condom use to eliminate this problem, they relied on participants’ own reports of contraceptive use—which is always unreliable—and never took any objective measures to assess how accurate these reports were.

What’s more, the authors focused on the relative risk of contracting HIV for women on birth control, which simply tells us whether injectable contraceptives and HIV transmission are correlated—a measure that is arguably less relevant than the attributable risk, which could tell us whether injectable contraceptives actually cause higher HIV transmission rates. The authors don’t report the attributable risks, but they’re easy to calculate and actually quite small. For example, if we observed 100 women on injectable birth control like Depo-Provera, and 100 who didn’t use injectables, for one year, we’d see three extra cases of HIV in the women using injectables. That’s a very different thing from “double the risk” and could be within the realm of statistical error. In any case, it’s far less alarming than the study’s original conclusion.

There are other issues that skew the study’s results, too. Overall contraceptive use was very low in the population it examined—as was the total number of new HIV infections. That means there may be some locations that never saw any new HIV infections. But it’s impossible to examine the separate, and potentially important, results from each study site, since the researchers aggregated all their data.

To be fair, some previous studies have suggested a tentative link between hormonal birth control—especially in injectable form like Depo-Provera—and a higher susceptibility to HIV transmission. The current study suggests that this is a link worth investigating—but in a thorough, careful, and rigorous manner. As of now, there’s still no consensus on the matter. And in the world of global public health, where our research has real-life implications for the world’s most vulnerable populations, we must be especially careful not to conflate flawed data with fact.