Learned features versus engineered features for multimedia indexing

Abstract : In this paper, we compare " traditional " engineered (hand-crafted) features (or descriptors) and learned features for content-based indexing of image or video documents. Learned (or semantic) features are obtained by training classifiers on a source collection containing samples annotated with concepts. These classifiers are applied to the samples of a destination collection and the classification scores for each sample are gathered into a vector that becomes a feature for it. These feature vectors are then used for training another classifier for the destination concepts on the destination collection. If the classifiers used on the source collection are Deep Convolutional Neu-ral Networks (DCNNs), it is possible to use as a new feature vector also the intermediate values corresponding to the output of all the hidden layers. We made an extensive comparison of the performance of such features with traditional engineered ones as well as with combinations of them. The comparison was made in the context of the TRECVid semantic indexing task. Our results confirm those obtained for still images: features learned from other training data generally outperform engineered features for concept recognition. Additionally , we found that directly training KNN and SVM classifiers using these features performs significantly better than partially retraining the DCNN for adapting it to the new data. We also found that, even though the learned features performed better that the engineered ones, fusing both of them performs even better, indicating that engineered features are still useful, at least in the considered case. Finally, the combination of DCNN features with KNN and SVM classifiers was applied to the VOC 2012 object classification task where it currently obtains the best performance with a MAP of 85.4%.
Type de document :
Article dans une revue
Multimedia Tools and Applications, Springer Verlag, 2016, 〈10.1007/s11042-016-4240-2〉
Liste complète des métadonnées

Contributeur : Georges Quénot <>
Soumis le : mardi 28 février 2017 - 17:01:37
Dernière modification le : jeudi 11 janvier 2018 - 06:26:20
Document(s) archivé(s) le : lundi 29 mai 2017 - 15:57:45


Fichiers produits par l'(les) auteur(s)




Mateusz Budnik, Efrain-Leonardo Gutierrez-Gomez, Bahjat Safadi, Denis Pellerin, Georges Quénot. Learned features versus engineered features for multimedia indexing. Multimedia Tools and Applications, Springer Verlag, 2016, 〈10.1007/s11042-016-4240-2〉. 〈hal-01479240〉



Consultations de la notice


Téléchargements de fichiers