SAAIL Lab SAAIL Lab
DSA 2026 · Oral & Poster Published · DSA 2026 Oral & Poster

Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment

Innocent Nyalala 1,2 , Patrick Vincent Ndowo 1 , Khamis Haji 1
1School of Engineering and Science, Indian Institute of Technology Madras Zanzibar, Tanzania
2Wadhwani School of Data Science and AI (WSDSAI), Indian Institute of Technology Madras, Chennai, India
Data Science Africa (DSA) 2026 Workshop, Makerere University, Kampala, Uganda (Jul 2026)
Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment

Summary: Five-dimension representativeness audit radar comparing PlantVillage, iCassava 2019, and iBean across geography, environment, class balance, image quality, and morphological diversity.

Abstract

Crop disease image datasets are increasingly used to build decision-support tools for smallholder farmers in Sub-Saharan Africa, yet dataset representativeness across geography, imaging environment, class balance, image quality, and morphological diversity is rarely documented systematically. We apply a quantitative five-dimension representativeness audit to PlantVillage, iCassava 2019, and iBean, quantify cross-domain transfer penalties, and introduce the AgriAI Bias Card , a six-item documentation checklist tailored to African agricultural AI.

Keywords: dataset auditingAgriAI Bias CardSub-Saharan AfricarepresentativenessPlantVillageiCassava

Key Findings & Contributions

5 Dims
Audit Rubric (0-10)
Geo, Env, Balance, Quality, Morph
4 / 10
PlantVillage Score
Lab background limitations
9 / 10
iCassava & iBean
Field-collected in East Africa
6-Item
AgriAI Bias Card
Deployment readiness checklist
  1. Constructed a quantitative five-dimension representativeness rubric (0-10 points) evaluating geography, imaging environment, Gini class balance (threshold <= 0.10), image quality, and morphological coverage.
  2. Audited three widely deployed agricultural benchmarks, documenting PlantVillage at 4/10 versus Sub-Saharan field-collected iCassava 2019 (9/10) and iBean (9/10).
  3. Introduced the six-item AgriAI Bias Card giving researchers and agronomic practitioners a standardized pre-deployment documentation protocol.

Key Empirical Figure

Per-dimension representation audit scores comparing PlantVillage (4/10) against iCassava (9/10) and iBean (9/10), illustrating Geographic Origin and Capture Environment gaps.
Per-dimension representation audit scores comparing PlantVillage (4/10) against iCassava (9/10) and iBean (9/10), illustrating Geographic Origin and Capture Environment gaps.

Benchmark Results

Five-Dimension Representativeness Audit Across Agricultural Benchmarks

DatasetCollection RegimeAudit Score (/10)Key Limitation / Strength
PlantVillage (~54,306 imgs)Laboratory uniform grey/black background4 / 10High lab volume; low East African field transfer
iCassava 2019 (21,397 imgs)Ugandan smallholder smartphones9 / 10Ecologically valid field shift; heavy CMD skew
iBean (Makerere)In-field Ugandan bean plots9 / 10Authentic lighting & field occlusions

BibTeX

@inproceedings{nyalala2026auditing,
  title={Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment},
  author={Nyalala, Innocent and Vincent, Patrick and Haji, Khamis},
  booktitle={Data Science Africa (DSA) 2026 Workshop},
  address={Makerere University, Kampala, Uganda},
  year={2026},
  month={July}
}
APA Citation

Nyalala, I., Vincent, P., & Haji, K. (2026, July). Auditing Representation in Crop Disease Benchmark Datasets Intended for Sub-Saharan African Deployment. Data Science Africa (DSA) 2026 Workshop, Makerere University, Kampala, Uganda.