Explainable AI for clove quality grading: Benchmarking post hoc XAI and compositional interpretability under domain shift
Abstract
Reliable agricultural commodity grading requires accuracy and explanations that match regulatory criteria. Using 4,603 expert-graded clove images across four Zanzibar State Trading Corporation (ZSTC) quality grades, we benchmark post hoc explainable artificial intelligence (XAI) methods including Grad-CAM, LIME, and GradientSHAP across eight CNN and Vision Transformer architectures and compare them with a compositional segmentation-classification pipeline. All eight monolithic architectures achieve 99.48% to 99.85% test F1-score, yet their Explanation Evidence Ratio (EER) varies sharply from 33.4% to 82.4%, while domain-shift F1 drops range from 6.7 to 86.5 percentage points. By isolating clove foregrounds prior to attribute grading via U-Net segmentation (Dice 0.9505) and ZSTC mpeta percentage rules, the compositional pipeline achieves 99.45% F1 with 100% EER by design and receives unanimous preference from three certified ZSTC inspectors.
Key Findings & Contributions
- Introduced the Explanation Evidence Ratio (EER) to measure what fraction of a visual attribution heatmap actually falls inside annotated clove foreground masks rather than spurious tray background.
- Benchmarked eight deep architectures (ResNet-18/50, DenseNet-121/201, EfficientNet-B0/B2, DeiT-Base, Swin-Base) across 4,603 ZSTC images, proving that >99.4% clean F1 masks severe EER spread (33.4% DeiT Grad-CAM to 81.9% Swin SHAP) and runtime bottlenecks (up to 26.1 s/image for SHAP).
- Quantified background domain-shift robustness across all eight monolithic backbones, uncovering F1 collapses of 53.7 pp (EfficientNet-B2) to 86.5 pp (EfficientNet-B0) despite near-100% in-domain scores.
- Built and validated a compositional U-Net (Dice 0.9505) + ResNet-18 rule-aligned grading pipeline achieving 99.45% F1, guaranteed 100% EER, and unanimous approval from three certified ZSTC clove graders.
Key Empirical Figure
Benchmark Results
In-Domain Clove Grading Classification Benchmark (N = 4,603 Images, 4 ZSTC Grades)
| Architecture | Params (M) | F1 (%) | Precision (%) | Recall (%) |
|---|---|---|---|---|
| DenseNet-201 | 20.0 | 99.85 | 99.84 | 99.86 |
| Swin-Base | 88.0 | 99.79 | 99.76 | 99.81 |
| DenseNet-121 | 8.0 | 99.76 | 99.77 | 99.75 |
| ResNet-50 | 25.6 | 99.74 | 99.75 | 99.73 |
| DeiT-Base | 86.6 | 99.70 | 99.70 | 99.70 |
| ResNet-18 | 11.7 | 99.58 | 99.58 | 99.59 |
| EfficientNet-B0 | 5.3 | 99.53 | 99.55 | 99.50 |
| EfficientNet-B2 | 9.2 | 99.48 | 99.48 | 99.47 |
| Compositional (U-Net + ResNet-18) | 13.2 | 99.45 | 99.45 | 99.45 |
Post Hoc XAI Faithfulness (EER %) & Inference Time per Image (s)
| Architecture | Grad-CAM EER (%) | Grad-CAM Time (s) | LIME EER (%) | LIME Time (s) | SHAP EER (%) | SHAP Time (s) |
|---|---|---|---|---|---|---|
| Swin-Base | 73.6 | 0.028 | 81.1 | 5.8 | 81.9 | 26.1 |
| DeiT-Base | 33.4 | 0.007 | 82.4 | 5.3 | 65.4 | 24.1 |
| DenseNet-121 | 65.7 | 0.049 | 74.4 | 2.0 | 66.0 | 5.1 |
| ResNet-50 | 71.2 | 0.020 | 71.6 | 2.0 | 64.5 | 4.8 |
| ResNet-18 | 69.4 | 0.009 | 57.9 | 1.7 | 66.8 | 3.2 |
| DenseNet-201 | 62.9 | 0.089 | 56.2 | 2.3 | 70.0 | 6.8 |
| EfficientNet-B2 | 60.3 | 0.037 | 48.5 | 1.9 | 60.4 | 4.5 |
| EfficientNet-B0 | 43.8 | 0.029 | 41.9 | 1.8 | 61.4 | 3.7 |
BibTeX
@article{nyalala2026etri_clove_xai,
title={Explainable AI for clove quality grading: Benchmarking post hoc XAI and compositional interpretability under domain shift},
author={Nyalala, Innocent and Ndowo, Patrick Vincent},
journal={ETRI Journal},
volume={48},
number={5},
pages={911--923},
year={2026},
doi={10.4218/etrij.2026-0225}
} Nyalala, I., & Ndowo, P. V. (2026). Explainable AI for clove quality grading: Benchmarking post hoc XAI and compositional interpretability under domain shift. ETRI Journal, 48(5), 911-923. https://doi.org/10.4218/etrij.2026-0225