Large Scale Video Indexing

First, we use a cascade of AdaBoost classifiers that is trained to be invariant to translate up to 25% of the original window size to quickly detect candidate regions with overlap. Second, we use SVM classifiers which reuse the features selected by AdaBoost in the previous stage for robust classification and simple training.

Introduction

• Motivations and Objectives
• Challenges
• Problem Statement
• Contributions
• Thesis Overview

How to organize a large number of the extracted faces into meaningful groups for easy browsing, searching and navigation. Prototypes, interfaces, and demonstrations are presented to illustrate the effectiveness of the news video indexing and retrieval system.

Feature Extraction and Selection

Introduction

In face recognition, the success of systems like those in [49, 91] comes mainly from efficient feature selection methods. Next, we propose two new feature selection methods that can be used in object detection systems in general as well as face detection in particular.

Feature Extraction

• Wavelet-based Features
• Local Binary Patterns
• Edge Orientation Histogram
• Fragment-Based Features
• Feature Extraction Using Principal Component Analysis
• Orientation Features
• Discussion

A 256-bin histogram of labels computed in the region can be used as a texture descriptor. The detection of fragment F adds information and reduces the uncertainty (measured by entropy) of the image.

Feature Selection

• Fast Feature Selection from Huge Feature Sets Using Conditional Mu-
• Efficient Feature Selection Using Principle Components
• Discussion

Conditional Mutual Information (CMI)-based feature selection methods have been proposed to take full advantage of the approaches described above to handle large-scale feature sets. To demonstrate the effectiveness of the proposed feature selection method (CMI-Multi), we compared it with two other feature selection methods, namely forward feature selection (FFS) [95] and a CMI-based method using binary features (CMI- Binary). [23, 90] on the dataset and feature sets described above. A similar result was also shown when the three feature selection methods were tested on Gabor wavelet functions.

To address these issues, we propose a simple but efficient feature selection method, where the main idea is to select features whose corresponding axes are closest to the principal components calculated from the data distribution by PCA. And by using the proposed feature selection method, starting from e1, x1, which is closest to e1, is found. We demonstrated the efficiency of our feature selection method by building a face detector based on SVM.

This figure indicates that, in terms of performance, using our feature selection method is slightly better than not using it. We developed two feature selection methods for building fast and robust face detection systems.

Multi-Stage Approach to Fast Face Detection

Introduction

Using about 10 features from the first two layers, more than 90% of non-face patterns were rejected. AdaBoost is used to select discriminative and meaningful features from a pool of a very large number of features and then construct the classifier. To strongly classify complex patterns, it is necessary to use a larger number of features and layer classifiers.

However, to enable the later layers to robustly classify a smaller number of remaining patterns, many more, approximately 5,660, weak classifiers are needed, making the training task much more complicated. It is time consuming because the training time is proportional to the number of features in the input feature set (which is typically hundreds of thousands) and the number of training examples (which is typically tens of thousands). In practice, to stop training a classifier, at least the following three parameters must be predetermined: minimum detection rate, maximum false positive rate, and maximum number of boost rounds (or the number of weak classifiers of each layer).

To improve speed while maintaining high accuracy, our approach takes advantage of the combination of Haar wavelet features and AdaBoost learning for fast and robust estimation. Third, the training time of AdaBoost classifiers is shortened by using simple sampling techniques to reduce the number of features in the feature set.

Related Work

Second, we investigated how to efficiently reuse the features selected by AdaBoost in the previous stage for last-stage SVM classifiers. Together with the use of multiple SVM classifiers instead of many AdaBoost classifiers in later layers, the total training time was significantly reduced. In the work of Viola and Jones, features are selected based on the discriminative performance of their associated weak classifiers through a boosting procedure.

With Wu et al.'s new proposal, weak classifiers are trained only once and features are selected by the direct feature selection method which directly maximizes the learning objective of the output classifier. 35, 36] proposed a reinforcing chain structure in which subsequent layers use the historical information of the previous layers. These new real-valued weak classifiers can effectively distinguish between face and non-face distributions, so the total number of features used is also dramatically reduced.

However, the main problem with these systems is how to choose the most appropriate number of bins. A small number of bins may not accurately approximate the real distribution, while a large number of bins may cause overlap, increase computation time and garbage storage space.

System Overview

So far, skin color is only used to speed up face detection systems by finding candidate face regions. The 36×36 pixel window is chosen in accordance with the idea stated in [75] that the classifier can be trained to be translation invariant up to 25% of the original window size. With this flexible classifier, the moving step size can be increased up to 12 pixels to dramatically reduce the number of analyzed patterns.

The final stage is a cascade of nonlinear SVM classifiers that reuse features selected by an AdaBoost classifier in the second stage classifier. These feature values ​​are evaluated and scaled to be between 0 and 1 to form a feature vector. In our experiments, only 100 features were used, making classification faster than it would have been using pixel-based SVM classifiers [ 31 , 74 ].

Training Cascaded Classifiers

• AdaBoost Learning
• Cascade of classifiers

In each boost round, the best weak classifier ht with the lowest error ϫt is chosen. After each round, these weights are updated in such a way that the weak student will focus much more on the hard examples in the next round. The main idea of ​​building a cascade of classifiers is to reduce computation time by giving different treatments to different complexities of input windows (Figure 3.6).

Only input windows that have passed through all cascade layers are classified as faces. Training cascade classifiers that can achieve both a good detection rate and a lower computational time is quite complicated: a higher detection rate requires more features, but more. To simplify this, the target detection rate and target false positive rate for each layer are usually set in advance.

Viola and Jones [91] stated that, if the layer classifier has not reached the predefined objectives after 200 features have been used, the training process will stop and a new layer will be added.

SVM classifier

C is a predefined parameter that is a compromise between a wide margin and a small number of failed margins. Compared to AdaBoost classifiers, SVM classifiers perform much slower due to the large number of support vectors and heavy kernel computation. To control the trade-off between the number of support vectors and errors, Scholkopf et al.

They proved that the parameter ν is an upper bound on the fraction of margin errors and a lower bound on the fraction of support vectors.

Experiments

• Experiment Setup
• Simplification of Training the Rejection Stage
• Efficiency of the Cascaded 36 × 36 Classifiers
• Features Selected by AdaBoost for SVM
• Efficiency of SVM classifiers
• Performance Comparison
• Robustness to Face Variations

Non-face patterns from the training and the validating sets of the first layer in the cascade were randomly selected. Non-view patterns of the subsequent layer classifiers are false positives collected by the partially trained cascade on the set of non-view images. They look somehow similar to the features of the first 24×24 layer classifier as shown in Figure 3.10(b).

For comparison of the performance of SVM classifiers, 2,450 face patterns and 7,500 non-face patterns separated from the training set were used (Section 3.6.1). The results shown in Figure 3.13 indicate that with more than 100 features, the performance of the classifiers was comparable. The remaining 32,500 non-face patterns and another 2,450 face patterns were used to compare the accuracy of the classifiers.

By running the cascade of second-stage AdaBoost classifiers on the set of non-face images, 40,000 false positives were collected and used as non-face patterns to train the SVM classifiers. First, the cascade of 36×36 AdaBoost classifiers rejected many non-face patterns extremely quickly, while slow SVM classifiers processed only a very small number of the remaining patterns.

Conclusion

For each data set, we used our face detector in two settings, with and without histogram equalization (HE) on the input image to handle lighting conditions. As shown in Figure 3.21 for the Yale A dataset, our face detector can achieve a high detection rate (92.7% recall and 9.4% precision).

Large Scale Video Indexing and Retrieval Using Human Faces 75

• RSC Clustering Model
• Internal and External Association
• Significance of Association
• Cluster Reshaping
• Clustering Strategy
• SASH-based Similarity Search
• Advantages of the RSC Clustering Model
• Implementation of a News Video Indexing and Retrieval System
• TRECVID Dataset
• Face Extraction and Normalization
• Person Name Extraction
• Performance of RSC clustering
• Faces and Names Association
• Browsing and Navigating Video Contents by Names and Faces
• Finding Important People By Multi-modal Analysis
• Discussion

104 , 103 ], instead of detected faces, used the “body,” an extended facial region (e.g., the neck) for comparison of two faces when detecting anchor persons. The RSC model assesses the internal association of a candidate cluster set A as the average of the correlations between A and all relevant sets of size |A| based on an item from A. Members within a cluster can be reordered in order of their contributionsZ(A, v), increasing the power of the underlying similarity measure.

The selection of cluster candidates can be viewed within the framework of the well-studied family of independent vertex set problems for graphs. Experiments on a variety of large, very high-dimensional datasets (such as text, protein sequences, and images) have shown that the SASH consistently returns a large portion of the true k-nearest neighbor set at speeds about two orders of magnitude faster than sequential search. During build time, half of the data items are randomly selected to form the bottom level set S0 of the SASH.

The model does not depend on the exact value of the base similarity measure, except for the purpose of generating relevant ordered sets. We then used PCA [88] to reduce the number of dimensions of feature vectors for face representation.

Discussion

• Summary
• Future Work
• An example of video annotation
• Variations of face appearance due to (a) illumination, (b) image quality and
• An example of meaningful faces in (left) a news video frame (32x37 pixels)
• An example of rare faces extracted from video
• A general framework for a video retrieval system using human face information. 6
• Parameters of a Haar wavelet feature
• Feature evaluation based on an integral image
• Extended Haar Wavelet Features proposed by Lienhart et al. [73]
• Gabor wavelets. (a) The real part of the Gabor kernels at five scales and
• Gabor wavelet representation (the real part and the magnitude) of a sample
• The basic LBP operator
• Different texture primitives detected by the LBP [68]
• Circularly symmetric neighbor sets. Samples that do not exactly match the
• LBP representation for high resolution face image in face recognition systems
• LBP representation for low resolution face image in face detection systems
• Face representation using LGBPHS proposed by Zhang et al. [106]
• An illustration of local orientation histograms proposed by Levi and Weiss [48]
• Examples of informative fragments used to represent faces and cars as shown
• The mean face and eigenfaces [88]

From small to large: Illumination cone models for face recognition under variable light and pose. Probabilistic visual learning for object representation. IEEE Transactions on Pattern Analysis and Machine Intelligence July 1997. Feature Selection Based on Mutual Information: Maximum Dependency, Maximum Relevance, and Minimum Redundancy Criteria.

Local gabor binary pattern histogram sequence (lgbphs): A new non-statistical model for face representation and recognition. Duy-Dinh Le, Shin'ichi Satoh, Robust object detection using fast feature selection from huge feature sets, in Proc.