Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment
Abstract
Crop disease image datasets are increasingly used to build decision-support tools for smallholder farmers in Sub-Saharan Africa, yet dataset representativeness across geography, imaging environment, class balance, image quality, and morphological diversity is rarely documented systematically. We apply a quantitative five-dimension representativeness audit to PlantVillage, iCassava 2019, and iBean, quantify cross-domain transfer penalties, and introduce the AgriAI Bias Card , a six-item documentation checklist tailored to African agricultural AI.
Key Findings & Contributions
- Constructed a quantitative five-dimension representativeness rubric (0-10 points) evaluating geography, imaging environment, Gini class balance (threshold <= 0.10), image quality, and morphological coverage.
- Audited three widely deployed agricultural benchmarks, documenting PlantVillage at 4/10 versus Sub-Saharan field-collected iCassava 2019 (9/10) and iBean (9/10).
- Introduced the six-item AgriAI Bias Card giving researchers and agronomic practitioners a standardized pre-deployment documentation protocol.
Key Empirical Figure
Benchmark Results
Five-Dimension Representativeness Audit Across Agricultural Benchmarks
| Dataset | Collection Regime | Audit Score (/10) | Key Limitation / Strength |
|---|---|---|---|
| PlantVillage (~54,306 imgs) | Laboratory uniform grey/black background | 4 / 10 | High lab volume; low East African field transfer |
| iCassava 2019 (21,397 imgs) | Ugandan smallholder smartphones | 9 / 10 | Ecologically valid field shift; heavy CMD skew |
| iBean (Makerere) | In-field Ugandan bean plots | 9 / 10 | Authentic lighting & field occlusions |
BibTeX
@inproceedings{nyalala2026auditing,
title={Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment},
author={Nyalala, Innocent and Vincent, Patrick and Haji, Khamis},
booktitle={Data Science Africa (DSA) 2026 Workshop},
address={Makerere University, Kampala, Uganda},
year={2026},
month={July}
} Nyalala, I., Vincent, P., & Haji, K. (2026, July). Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment. Data Science Africa (DSA) 2026 Workshop, Makerere University, Kampala, Uganda.