Open-source environmental data as an alternative to snail surveys to assess schistosomiasis risk in areas approaching elimination

Table 3 Performance metrics for the snail and environmental data models

Performance metrics	Snail survey data models			Open-source environmental data models
	Model 1	Model 2	Model 3	Model 1	Model 2	Model 3
AUC	0.852	0.849	0.843	0.800	0.784	0.798
Accuracy	0.845	0.859	0.845	0.887	0.887	0.887
Accuracy 95% CI	0.74–0.92	0.76–0.93	0.74–0.92	0.79–0.95	0.79–0.95	0.79–0.95
NIR^a	0.859	0.859	0.859	0.859	0.859	0.859
P-Value (Accuracy > NIR)	0.706	0.583	0.706	0.316	0.316	0.316
Kappa^b	0.332	0.365	0.332	0.492	0.492	0.492
Sensitivity	0.400	0.400	0.400	0.500	0.500	0.500
Specificity	0.918	0.934	0.918	0.951	0.951	0.951
Pos Pred Value	0.444	0.500	0.444	0.625	0.625	0.625
Neg Pred Value	0.903	0.905	0.903	0.921	0.921	0.921

^aNo Information Rate
^bDue to the high degree of imbalance between the outcome classes across the study period, the Cohen’s kappa statistic is a useful metric for our models, as it helps to correct bias that results when rewarding the prediction of the majority class. The benchmark values outlined by Landis & Koch (1977) are useful here for determining the relative strength of the predictive models: < 0.00 = Poor; 0.00–0.20 = Slight; 0.21–0.40 = Fair; 0.41–0.60 = Moderate; 0.61–0.81 = Substantial; 0.81–1.0 = Almost Perfect

ISSN: 1476-072X