Study Still Dead After Resurrection Attempt Fails

Published at BMJ Open on October 26.
The version below is longer and uncut.

The August 11 response of Dr. Elard Koch and several co-authors fails to rectify the two major flaws in their study that I exposed in my April 9 rebuttal, and in fact, further confirms those flaws. Their very lengthy response also ignored many of my arguments and repeated many of their disputed points.

This reply explains why those two major errors remain fatal to their study’s conclusions. I then address the question of anti-abortion bias, which I believe played a key role in the flawed design of the study. To be fair, I also address my own biases. Finally, I deal with several ancillary issues arising from Koch et al.’s response (for those interested to read on).

Fatal Flaws Are Still Fatal

My rebuttal had explained that abortion law was not related to abortion practice in Mexico, and that this invalidates the study’s methodology and conclusion. That’s because you can’t hypothesize a possible effect on abortion mortality based on the existence or absence of various legal exemptions across Mexican states, when those exemptions don’t even work. In practice, few Mexican women can access abortions under the legal exemptions. In response, Koch et al. confirmed this flaw by saying: “…between states, legal permissiveness may be the same, but accessibility may differ by multiple unmeasured factors. We do not say nor have we stated that we are using this term as a proxy for greater or less accessibility to pregnancy termination…” But their failure to account for the difference between law and practice is the flaw. Their methodology implicitly assumed that abortion restrictions reflect abortion practice, but since they now concede that’s not the case, their conclusion that maternal mortality is unrelated to the presence or absence of certain abortion restrictions is meaningless.

I had shown that it was arbitrary and erroneous to select, in particular, the legal exemption for abortion in cases of genetic or congenital malformation as the criterion for assigning Mexican states to the categories of “more permissive” or “less permissive” abortion legislation, depending on whether the state had that exemption. The rarity of abortion for fetal abnormality makes this variable useless as a proxy, because the miniscule numbers of maternal deaths that may result would be impossible to detect statistically in order to compare states with and without the exemption. The authors’ only response is to argue that such abortions may be slightly less rare than I claim, based on the low prevalence of Down syndrome at birth in several other countries. But that has nothing to do with Mexico or the fact that abortions for any kind of serious fetal abnormality are in the range of 1% of all abortions in countries where abortion is widely legal. So their error remains, especially since abortions due to fetal abnormality are likely even rarer in Mexico than in western countries because of its restrictive laws, inaccessibility of legal abortion under the exemptions, profound stigma, and physician refusals.

The authors say that exploratory analyses are “valid and valuable tools to avoid an arbitrary categorization.” But somehow, that’s exactly what they ended up with. The authors confirm their flaw by explaining that they selected the legal exemption for fetal abnormality as the main variable because it was the only one that yielded an association with maternal mortality, as well as the only one where the 32 Mexican states had a roughly equal distribution in terms of whether they had the exemption or not. In other words, the variable was arbitrarily selected on the basis of convenience and because it happened to yield an association that I’ve already shown is meaningless. (Correlation does not equal causation.) A random association is easy to find if you conduct a fishing expedition for one amongst a host of variables, but one should not then base an entire study on it and draw conclusions from it.

Author Bias

My April rebuttal in the BMJ Open was incomplete because I removed a section entitled “Credibility of Authors in Question” at the request of the editors. However, the full version had already been published at RH Reality Check and I linked to it in my BMJ Open response. Koch et al. appear to have ignored this version, which also has more links to references.

After reading Koch et al’s reply, it became even more apparent to me that an anti-abortion bias had infected the study’s methodology and conclusion. This bias must be addressed, despite the authors’ efforts to take sanctuary under the mantle of scientific objectivity. In fairness, I will also disclose and attempt to justify my own biases.

Nine out of ten authors of the BMJ Open study had an undisclosed bias because they are signatories to the “Dublin Declaration.” This document denies the need for legal abortion even to save a woman’s life. Further, Koch himself and three of his co-authors were named as “False Witnesses” last year in an investigative series carried out last year by RH Reality Check, because they had “pushed false information designed to mislead the public, lawmakers, and the courts about abortion” in their previous research or public statements. The peer review process for their BMJ Open study failed to catch the substantive errors in the study because neither peer reviewer had expertise in the subject matter, and one (R. Lieva) appears to hold the same anti-abortion bias as the authors.

The study contains repeated citations (over 30) of the authors’ own past research on abortion and lists over a dozen references from Koch and various co-authors (including peer reviewer R. Lieva), as if their work is equally legitimate to mainstream research. The effect is to create a false picture of scientific confusion and conflicting data in the abortion field. This same pattern is repeated in their reply to me, even though several of their studies have been debunked. (Here’s a compendium of rebuttals to their work.)

Koch et al. have gone to great lengths (in this and other studies) to try and show that factors such as access to emergency obstetric care have the greatest apparent impact on maternal mortality, not abortion laws. But their focus on maternal deaths, while important, obscures all the other suffering and harms that criminal abortion laws cause to women, including high complication rates from unsafe abortion (about 159,000 women are hospitalized annually in Mexico). The implication is that we should accept this human cost as long as good health care systems can save women’s lives in the end. Presumably the authors would disagree, and hopefully their implication was unintentional, but it shows what can happen when bias infects a study.

Based on my reviews of the BMJ Open study and previous Koch-led studies on abortion in Latin America, I believe that the underlying, unstated objective of Dr. Koch and at least some of his co-authors is to promote the idea that abortion does not need to be legal. Although Koch et al. pointed out that they made no recommendation in their study on the legal status of abortion, they don’t need to. The anti-abortion movement does it for them by turning their conclusions into political soundbites claiming that legalizing abortion does not save women’s lives. In effect, their studies serve as a dog whistle to anti-choice activists.

Given the overwhelming global and historical evidence of the danger posed to women by criminal abortion laws, any study conducted by Koch et al. that concludes that restrictive abortion laws do not contribute to maternal mortality should be treated with suspicion.

But let me turn now to the question of my own biases, since Koch et al. would presumably claim that I’m the one who’s biased, as well as unscientific. This is apparent from the authors’ near ad hominem treatment of my criticisms, dismissing them as “based largely on personal opinions or speculative assumptions,” and “not scientifically based.” I’m a writer and pro-choice activist, not a scientist. I do not apologize for my impassioned defense of women’s rights and lives, or for my ability to recognize and expose the authors’ co-opting of scientific methods to disguise an anti-abortion agenda. I’ve been monitoring the anti-choice movement for 25 years and Koch’s work for 3 years.

For the record, I did not have “discomfort” with the study’s findings, because I recognized the study methodology and conclusions were deeply flawed. However, I did feel offended by the study. I object to its very premise and even that it was published in a reputable scientific journal where it does not belong. Please allow me to explain.

I have a bias in favour of the belief that women deserve equality, human rights, and dignity. I also have a bias against criminalizing life-saving healthcare that only women need. Should such biases, if expressed, lessen my credibility or weaken my arguments? If so, it must be because it’s still up for debate whether women deserve human rights and freedoms, including the fundamental right to control their fertility. Koch et al. are ultimately relying on the fact that there’s still significant controversy over women’s rights in many countries – especially abortion rights – and it might explain why they were able to publish such a study in the first place.

The clear message of their study – not stated by the authors of course, but spread by the anti-abortion movement – is that it’s not necessary to legalize abortion in order to reduce maternal mortality.

Well, let’s consider this analogy: What if some researchers had conducted a near-identical study with the same methodology, but instead of looking for a random association between restrictive abortion laws and reduced maternal mortality, they looked for a random association between black segregation and improved health outcomes for black people? Would such a study have been published? I think not. But unlike the near-universal opposition to racism in our societies, sexism is still mainstream.

In my opinion, Koch et al’s study in the BMJ Open is not much different from a study that would document good living conditions in black ghettoes in order to leave the impression that it would be justified to keep black people locked up in them. Such a study would be highly offensive, no matter how scientifically conducted. I find it equally offensive when studies do the same thing to support the continued criminalization of women’s healthcare. And I object to reputable journals publishing studies with the premise that depriving women of their rights may not be harmful and could even be beneficial for them. 

Addendum: Ancillary Issues

Unsafe Abortion as a Contributing Factor to Maternal Mortality:

The authors protest my “unsubstantiated and dissociated” suggestion that they are using other contributing factors to maternal mortality as a smokescreen to cover up the effect of unsafe or illegal abortion. Their objection is largely answered by my points above on author bias.

Further, I believe the authors are being disingenuous when they say they want to emphasize the other factors that contribute to women suffering and dying in childbirth from preventable causes. Rather, they appear to be primarily concerned with abortion. Their study’s title begins with “Abortion legislation” as the key factor. The study objective is: “To test whether there is an association between abortion legislation and maternal mortality outcomes after controlling for other factors thought to influence maternal health.” Their press release for the study opens with: “Laws protecting the unborn and therefore, less permissive in regard to abortion, bear a negative …connotation, because induced abortion in clandestinity might increase maternal deaths. However, a new study conducted in 32 Mexican states …challenges this notion…” And most of their other research focuses on abortion.

Of course, many factors contribute to maternal mortality, and many effective ways exist to address it. But as I said in my rebuttal, this range of other factors could swamp the effect of unsafe abortion on maternal mortality rates and make it harder to detect statistically, particularly if a country’s mortality rate from unsafe abortion is relatively low compared to other countries. The study methodology the authors employed was almost guaranteed to find no meaningful association anyway, which allowed them to present their foregone conclusion that abortion laws don’t affect maternal mortality.

Clandestine abortion is still a major cause of maternal mortality in many parts of the world, particularly in Africa and Asia. It is unusual for developing countries with strict bans on abortion to have relatively low maternal mortality rates. In such countries (like Chile and a handful of others in Latin America), other factors indeed play an important role including widespread use of misoprostol – however, maternal mortality would almost certainly be even lower if abortion was legal, safe, and accessible. Regardless, complications from unsafe abortion remain high throughout Latin America and the Caribbean, with over one million women hospitalized annually – 159,000 in Mexico alone during 2009. This demonstrates that restrictive abortion laws continue to pose a great danger to women in Mexico, even if fewer are dying than in the past.

By the way, Koch et al. said they “reanalyzed” the 2012 study by Schiavon et al. that found a significant proportion of maternal mortality in Mexico (7%) was due to unsafe abortion, and claimed it was flawed. But Koch et al. fail to mention that their reanalysis was debunked.

Issues with Vital Statistics for Abortion Deaths:

The authors’ arguments that Mexico does not have a problem with underreporting or misclassification of abortions and associated mortality are unpersuasive. They put a lot of faith in vital statistics from the Mexican government – their only source – but it’s unlikely that government statistics are as robust and accurate as they claim. The authors can only cite their own research as support. I found mentions in several studies (listed below) about the problems of misclassification and underreporting associated with the use of vital statistics when it comes to abortion, even in countries like Mexico that have otherwise high-quality records for maternal mortality.

In settings like Latin America where abortion is mostly illegal and highly stigmatized, women and their families would be more likely to not report or misreport an abortion, and health care personnel would be more likely to misclassify causes of death out of ignorance, fear, or compassion. For example, even with the existence of specific ICD codes for “Abortive Outcomes” (ICD-10 O00 to O08), why should we exclude the possible misclassification of unsafe abortion deaths under the codes for sepsis or hemorrhage in early pregnancy (ICD-10 O20 and O23)?

Vital statistics should not be relied upon alone to determine maternal mortality from unsafe abortion. Reputable studies employ a variety of sources and methods, including studies, hospital data, various types of surveys, and more. For no valid reason, Koch et al. neglect and dismiss these additional methodologies, while their own narrower methodology for estimating abortion incidence has been debunked.

Recent studies on maternal death from unsafe abortion do not support Koch et al’s claim that the rate for Mexico is as low as 3% of all maternal mortality. Before looking at these other figures, it’s important to understand that studies and official statistics on maternal mortality rates and causes do not always distinguish adequately between deaths from induced abortion and spontaneous abortion (miscarriage), or even other “abortive outcomes” such as ectopic or molar pregnancies and other “unspecified” complications. However, we know that maternal deaths from miscarriage, legal abortion, and molar pregnancy are all very rare, at least in settings with reasonable access to health care. While ectopic pregnancies carry a higher risk of death, they are far less common than abortion or miscarriage. Therefore, the majority of officially-recorded abortion deaths in countries with restrictive laws are plausibly due to unsafe induced abortion, including many coded under other “Abortive Outcomes.”

The following studies provide rates of maternal mortality due to abortion in Latin America and/or Mexico. The data for Mexico appear to be sourced from vital statistics only, except for #3, which also cites the WHO Mortality Database (in the study’s Appendix).

1. The Schiavon et al. estimate for Mexico was 7% for 2008.
2. A May 2014 global analysis by the World Health Organization (WHO) provided an estimate of 9.9% for Latin America and the Caribbean. For Mexico, the study shows a rate of almost 8% as denoted on a bar graph in the Appendix, page 37.
3. A September 2014 study by IHME, the Institute for Health Metrics and Evaluation, shows in bar graphs an average rate of about 17.5% for Latin America and the Caribbean in 2013 (Figure 6), and a rate of about 11% for Mexico (Appendix, page 127).

Why are these data from Mexico (7%, 8%, and 11%) higher than Koch et al’s estimate of 3%?  It appears it’s because Koch et al. excluded ICD-10 codes O00, O01, O02, and O08 (respectively: ectopic pregnancy, molar pregnancy, other abnormal products of conception [including missed abortion], and complications following abortion and ectopic and molar pregnancy). Excluding these codes could omit many unsafe abortion deaths that were misreported or misclassified.

Global Rates:  The oft-cited 2008 WHO figure of 13% for global maternal mortality due to unsafe abortion should be retired due to the new WHO analysis (#2 above). That study provides a figure of 7.9% globally for 2003 to 2009, with the reduction due to the use of recent data and improved methods. I would suggest that the reduced figure may also reflect the increased use of safer medical abortion by women in some regions.

Using a somewhat different methodology, the IHME study (#3 above) yields an estimate of about 15% global maternal mortality from abortion in 2013 (see Table 2, page 995). However, that 15% figure includes late maternal deaths (43 days to 1 year after delivery), which the WHO does not include in its unsafe abortion mortality estimate. Excluding late maternal deaths, the IHME figure for global maternal mortality from abortion is actually 18%. The significant variance with the WHO figure of 7.9% is unexplained and it remains unclear which is closer to reality. (The IHME figure is plausible if we hypothesize that the proportion of deaths by unsafe abortion has increased because of far greater reductions in other causes of maternal mortality, compared to unsafe abortion.) An independent review by the Guttmacher Institute (requested by myself) found that the two estimates differed "with respect to data sources used, types of abortions included, analytic methods employed, reference periods to which the estimates refer, and whether adjustments were made for likely underreporting or misreporting of abortion deaths." (Guttmacher Institute, pers.comm, Oct 5, 2015).  [Note: Section in blue added/edited after BMJ Open submission]

Incidence of Unsafe Abortion:  This report was the source for my figures on the significant number of unsafe abortions and complications in Mexico (over a million abortions a year, and 159,000 hospitalizations). These numbers are higher than the estimates from Koch et al. because the latter are based only on vital statistics data, which is insufficient.

Medical Abortion:

The authors state: “It is self-contradictory to say that unsafe abortions have increased substantially over the last decades while observing substantial reductions in deaths from this cause.” Unsafe abortions have increased globally in recent years but not substantially (from 19.7 million in 2003 to 21.6 million in 2008). But in Latin America, unsafe abortions decreased by a third between 1990 and 2008, and maternal mortality went down 70%.

This dramatic reduction in maternal deaths coincides with the widespread availability of medical abortion (using misoprostol) in Latin America over the last two decades. In contrast, maternal mortality from unsafe abortion remains high in regions where misoprostol is not as easily attainable, such as Africa. This suggests that misoprostol plays a significant role in reducing women’s deaths.

Several studies and reports (such as this one) also attribute the reduction in maternal deaths in Latin America to the increased use of medical abortion. This method has largely replaced more dangerous traditional methods, although it can still lead to a high rate of mostly non-lethal complications when used without medical supervision.

Koch et al. are unsatisfied however, and want to see an epidemiologic study with evidence that misoprostol decreases unsafe abortion mortality “independently of other major factors such as access to emergency obstetric care.” This seems to be an unreasonable expectation given the swamping effect those factors would have on the detection of unsafe abortion mortality, especially if the study plugs in artificially low death rates from abortion.

One also wonders at the reluctance of the authors to accept anything but the most rigorous direct evidence for the role of misoprostol in improving the safety of clandestine abortion. Could this be related to Koch et al’s past (discredited) attempts to show that rates of illegal abortion in Latin America have been grossly overestimated? The allegation that the high numbers of illegal abortions are exaggerated or even made-up is common propaganda in the anti-abortion movement. Such a belief would conflict with evidence for a dramatic reduction in maternal mortality due to a switch to a safer clandestine method. Incidentally, it could also motivate anti-abortion researchers to design studies “proving” that restrictive abortion laws don’t increase maternal deaths.