Transfer Learning: Backbone Selection and Fine-Tuning for Binary Image Classification

Transfer learning is a practical approach for training vision models when compute and labeled data are limited: start from a backbone pretrained on ImageNet, freeze it to train a small task-specific head, and optionally fine-tune the backbone with a low learning rate. In this short study, we implement a standard Keras transfer learning pipeline for binary classification for custom food detection app and compare multiple backbones and fine-tuning settings. On our CPU-only setup, EfficientNetB0 with fine-tuning achieves the best test accuracy (98.71%), while MobileNetV2 provides an excellent speed/accuracy trade-off (97.89% test accuracy in under 7 minutes). We also observe that feature extraction alone already performs strongly (96.69%), and fine-tuning primarily provides incremental gains near saturation for our use cases.