Fix tests

oughtinc · Sep 24, 2022 · a0c22ad · a0c22ad
1 parent 025bb07
commit a0c22ad
Show file tree

Hide file tree

Showing 6 changed files with 39 additions and 6 deletions.
diff --git a/papers/.gitignore b/papers/.gitignore
@@ -1,2 +1,6 @@
 *.pdf
 *.txt
+
+# For tests
+!keenan-2018-tiny.txt
+!keenan-2018.pdf
diff --git a/papers/keenan-2018-tiny.txt b/papers/keenan-2018-tiny.txt
@@ -0,0 +1,24 @@
+Abstract
+
+In this cluster-randomized trial, we assigned communities in Malawi, Niger, and Tanzania to four twice-yearly mass distributions of either oral azithromycin (approxi- mately 20 mg per kilogram of body weight) or placebo. Children 1 to 59 months of age were identified in twice-yearly censuses and were offered participation in the trial. Vital status was determined at subsequent censuses. The primary outcome was aggregate all-cause mortality; country-specific rates were assessed in prespecified subgroup analyses. Among postneonatal, preschool children in sub-Saharan Africa, childhood mortality was lower in communities randomly assigned to mass distribution of azithromycin than in those assigned to placebo, with the largest effect seen in Niger. Any imple- mentation of a policy of mass distribution would need to strongly consider the poten- tial effect of such a strategy on antibiotic resistance. (Funded by the Bill and Melinda Gates Foundation; MORDOR ClinicalTrials.gov number, NCT02047981.)
+
+1 Introduction
+
+Trachoma-control programs have distributed more than 600 million doses of oral azithromycin in an effort to elimi- nate the ocular strains of chlamydia that cause the disease.1,2 Azithromycin has been effective against trachoma, although it has caused gastro- intestinal side effects and selected for macrolide- resistant strains of Streptococcus pneumoniae and Escherichia coli.3-8 Investigators have also noted possible benefits of azithromycin for prevention of a number of infectious diseases including malaria, infectious diarrhea, and pneumonia.9-14 The results of a case–control study and a cluster- randomized trial in an area of Ethiopia in which trachoma is endemic suggested that mass distri- bution of azithromycin might reduce childhood mortality.15,16 Some experts believed that a mor- tality benefit was, indeed, possible, although it would probably be smaller in magnitude than what was found in these studies.17
+
+2 Eligibility
+
+MORDOR (Macrolides Oraux pour Réduire les Décès avec un Oeil sur la Résistance) was a cluster-randomized trial conducted in the Mala- wian district of Mangochi, in the Nigerien dis- tricts of Boboye and Loga, and in the Tanzanian districts of Kilosa and Gairo, with communities as the unit of randomization. The community that served as the randomization unit was a health surveillance assistance area in Malawi, a grappe (i.e., a cluster of households representing the smallest government health unit) in Niger, and a hamlet in Tanzania. None of the districts
+were eligible for mass distributions of azithro- mycin for trachoma on the basis of the most recent mapping, and none of the children who participated in the trial had previously received azithromycin. Enrollment was based on census information available before the trial. Commu- nities with a population between 200 and 2000 inhabitants on the most recent census were eli- gible for enrollment (see the Supplementary Ap- pendix, available with the full text of this article at NEJM.org). Communities remained in the trial even if the population drifted out of this range. All children 1 to 59 months of age (truncated to month) who weighed at least 3800 g were eligi- ble to receive azithromycin or placebo.
+
+3 Randomization and Masking
+
+Lists of communities from the most recent pre- trial census were submitted to the data coordi- nating center at the University of California, San Francisco (UCSF). For each country, communities were randomly assigned in equal proportions to 1 of 10 letters, with 5 letters coded for azithro- mycin and 5 for placebo (more information is provided in the statistical analysis plan in the protocol, available at NEJM.org). Randomization was performed with the sample function in R soft- ware, version 3.1 (R Foundation for Statistical Computing). Only key trial personnel were aware of which letters corresponded to each group. Participants, observers, investigators, and data- cleaning team members were unaware of the group assignments. Centralized randomization and simultaneous assignment of communities facilitated complete concealment of the assign- ments. The placebo contained the vehicle of the oral azithromycin suspension and was bottled and labeled identically to azithromycin. Both placebo and azithromycin were donated by Pfizer, which reviewed the protocol but had no other role in the trial.
+
+4 Intervention
+
+Each child 1 to 59 months of age at the time of the census was offered a single directly observed dose of oral azithromycin or placebo (according to the randomization of their community). Chil- dren were given a volume of suspension corre- sponding to at least 20 mg per kilogram of body weight, calculated with the use of a height stick (see the protocol), in accordance with the coun- try’s trachoma program guidelines, or by weight for children unable to stand. Children who were known to be allergic to macrolides were not given azithromycin or placebo. Azithromycin or placebo was administered at the time of the census or during additional visits in an attempt to achieve at least 80% coverage. Administration of trial medication or placebo was documented in the mobile application for each child, and com- munity coverage was calculated relative to the census. The parents or guardians of the children and the local health posts were instructed to contact a village representative regarding any adverse events noted within 7 days after admin- istration of azithromycin or placebo; the village representative reported the events to the site coor- dinator, who in turn reported the events to UCSF.
+
+5 Primary Results
+
+The results of a 12-month interim analysis for efficacy did not meet the prespecified criterion for early stopping, which was a P value of less than 0.001 for the difference between groups. At the end of the trial, the annual mortality rate for eligible children in the three countries combined was 14.6 deaths per 1000 person-years in com- munities that received azithromycin (9.1 per 1000 person-years in Malawi, 22.5 in Niger, and 5.4 in Tanzania) and 16.5 deaths per 1000 person- years in communities that received placebo (9.6 per 1000 person-years in Malawi, 27.5 in Niger, and 5.5 in Tanzania). Community-level, intention- to-treat analysis showed that over all four inter- censal periods, mortality was 13.5% lower over- all (95% confidence interval [CI], 6.7 to 19.8) in the azithromycin group than in the placebo group (P<0.001). The proportion of children whose census status was recorded as moved or unknown did not differ significantly between the groups (P=0.71 and P=0.36, respectively).
diff --git a/papers/keenan-2018.pdf b/papers/keenan-2018.pdf
diff --git a/tests/test_metrics.py b/tests/test_metrics.py
@@ -69,6 +69,7 @@ async def test_rouge_multiple_hypotheses_and_references_matches_single_score_whe
     assert scores_single == scores_repeated
 
 
+@mark.skip(reason="Requires OUGHT_INFERENCE_API_KEY")
 @mark.slow
 @mark.anyio
 async def test_nubia():

diff --git a/tests/test_paper.py b/tests/test_paper.py
@@ -5,24 +5,27 @@
 
 def test_pdf():
     script_path = Path(__file__).parent
-    paper = Paper.load(script_path / "../papers/abebe-2018.pdf")
+    paper = Paper.load(script_path / "../papers/keenan-2018.pdf")
     sentences = list(paper.sentences())
     assert (
         sentences[0]
-        == "We show that an easy-to-scale intervention that helps young job-seekers to signal their ability to employers can generate large improvements in labor market outcomes."
+        == "We hypothesized that mass distribution of a broad-spectrum antibiotic agent to preschool children would reduce mortality in areas of sub-Saharan Africa that are currently far from meeting the Sustainable Development Goals of the United Nations."
+    )
+    assert (
+        sentences[-1]
+        == "Disclosure forms provided by the authors are available with the full text of this article at NEJM.org."
     )
-    assert sentences[-1] == "A.22 A.25"
 
 
 def test_txt():
     script_path = Path(__file__).parent
-    paper = Paper.load(script_path / "../papers/abebe-2018-small.txt")
+    paper = Paper.load(script_path / "../papers/keenan-2018-tiny.txt")
     sentences = list(paper.sentences())
     assert (
         sentences[0]
-        == "We show that an easy-to-scale intervention that helps young job-seekers to signal their ability to employers can generate large improvements in labor market outcomes."
+        == "In this cluster-randomized trial, we assigned communities in Malawi, Niger, and Tanzania to four twice-yearly mass distributions of either oral azithromycin (approxi- mately 20 mg per kilogram of body weight) or placebo."
     )
     assert (
         sentences[-1]
-        == "Overall, these results illustrate the large equity gains that can be generated by helping young workers to access the labour market through improved signalling."
+        == "The proportion of children whose census status was recorded as moved or unknown did not differ significantly between the groups (P=0.71 and P=0.36, respectively)."
     )
diff --git a/weights b/weights