• 検索結果がありません。

Contribution estimation

ドキュメント内 局所特徴を用いた動画像中の人間動作認識 (ページ 77-82)

Contribution estimation for human interaction recognition

5.2 Contribution estimation

The main contribution of this work is that a contribution estimation method is proposed to extract major participant for interaction recognition instead of using all participants’ action information. Since contribution estimation shields the interference from passive actions, inter-action recognition accuracy is improved. Comparing with previous algorithms using inter-actions of both two participants, we provide a new way to solve such complex problems. Our method is tested on UT-interaction dataset and the LIMU dataset1 captured by ourselves. Experiment results show its efficiency and out-stand performance.

(a) Single-contribution

(b) Co-contribution

Figure 5.2: Contribution interaction model construction

interaction model construction for the two cases of “co-contribution” are almost the same, which will be described later. In this chapter, we use histograms of local feature occurrence calculated by BoW to represent human actions. The frame sequences of interactions are represented as histograms, which are used to constuct interaction models.

To explain the procedure of model construction clearer, the interaction samples in UT dataset are used as examples in the following. In UT dataset, the two participants in these samples stand at the left and the right sides, respectively. The two participants are defined as the left participant and the right participant according to their positions in videos. Suppose that there are total K interaction categories, and the kth category contains Nk training sam-ples, The action histograms of the left and the right participant in thekth category are recorded

as hkL(i),hkR(i), k = 1, . . . , K, i = 1, . . . , Nk, total 2Nk histograms. We average the his-tograms in training samples to obtain the contribution interaction models, which are recorded as HkL,HkR, k= 1, . . . , K.

One contribution interaction model of “single-contribution” interactions is shown at the bottom of Figure5.2(a). When we create the contribution models, the major actions done by either the left or the right participants in training samples are adopted. As shown in the figures on top of Figure5.2(a), the actions marked by yellow rectangles are major actions. Firstly, we separate the Nk training samples into “the left group” and “the right group:” in the left group the left participants do major actions, while in the right group the right ones do the major actions. We suppose that the left group contains NkL training samples, and the right group contains NkR training samples, here Nk = NkL+NkR. Then we average the histograms of the left participants in the left group by formula (5.1) to obtain the action histogram of the left participant in contribution interaction model, HkL, in the contribution interaction model of interaction category k. Similarly, the right histogram, HkR, is calculated by averaging the histograms of the right participants’ actions in the right group.

HkL =

NkL

i=1

hkL(i). (5.1)

For “co-contribution” interactions, the actions of both two participants in training samples are used to construct contribution interaction models. It should be noted that in some “co-contribution” interaction categories, the actions of two participants are similar, while they are dissimilar in some other “co-contribution” interaction categories.

If the interaction categorykbelongs to a “co-contribution” interaction category, and the two participants do similar actions, one contribution interaction model is calculated for the category by averaging the histograms of the left and the right participants in training samples. The construction procedure is shown in Figure5.2(b). Therefore, two histograms in the contribution interaction model are obtained and recorded as HkL,HkR. We calculate HkLby formula (5.2),

and HkR is calculated similarly using action histograms of the right participants.

HkL =

Nk

i=1

hkL(i). (5.2)

For “co-contribution” interaction categories having dissimilar actions, we creat two con-tribution interaction models for each category to handle its reversibility. Figure5.3 shows two examples that the major action may be led by the left participant (left figure in Figure5.3) or by the right participant (right figure in Figure5.3), thus contribution interaction models should be created for the two situations. If categoryk0 belongs to “co-contribution” interaction with dis-similar actions, the training interaction samples are separated to “left group” and “right group”

according to which participant, the left or the right, leads the progress of interaction. The inter-action models of the two groups are calculated using the same procedure with Figure5.2(b). It is supposed that the left group containsNk0Ltraining samples, and the right group containsNk0R training samples, whereNk0 =Nk0L+Nk0R. The histograms of participants in two contribution interaction models are recorded as HLk0L,HLk0Rand HRk0L,HRk0R. The HLk0Lin the model of the left group is calculated by formula (5.3), and HLk0Ris calculated in the same way. Similarly, The HRk0Lin the model of the right group is calculated by formula (5.4), and HRk0Ris also calculated similarly using training samples in the right group.

HLk0L=

Nk0L

i=1

hk0L(i). (5.3)

HRk0L=

Nk0R

i=1

hk0L(i). (5.4)

Finally, we obtain the histograms in contribution interaction models of all interaction cate-gories, recorded as{H0L,H0R,H1L,H1R,· · · ,HLk0L,HLk0R,HRk0L,HRk0R,· · · ,HKL,HKR}. Sup-pose that totalM(M > K)pairs of histograms, we record the histogram set as H={HmL,HmR, m = 1,· · · , M}. We use the contribution interaction models to test one interaction video be-longing to “co-contribution” or “single-contribution” interaction, and to determine the major action of “single-contribution” interaction.

Figure 5.3: Co-contribution interaction with dissimilar actions

5.2.2 Contribution determination

Both of two participants do major actions in “co-contribution” interactions, while only one participant does major action in one “single-contribution” interaction. As introduced above, the contribution interaction models are composed of two major actions. Therefore, we determine a testing interaction to be “co-contribution” or “single-contribution” by comparing the actions of participants in the testing sample with contribution interaction models of all categories. If both of two participants’ actions in the sample match with the actions in one model, the interaction sample is co-contribution interaction. Otherwise, it is single-contribution, and the matched action is the major action. The estimation procedure is explained in the following.

The histograms of local feature occurrence are calculated for two participants in the testing interaction, recorded as htL,htR. Then we compare the two histograms with histograms in contribution interaction models of all categories. The comparison is realized by calculating the distance between histograms using formula 5.5.

Dmj = htj·Hmj

∥htj∥∥Hmj∥. (5.5)

So we obtain the distance set{Dmj}, herej ∈ {L, R}, m= 1, . . . , M. Then, we perform estimation by the following 3 steps.

Step 1:

Look for the maximum value of distance max(Dmj), and obtain the categoryAm0 corre-sponding tomax(Dmj). Then the distance pair{Dm0L, Dm0R}corresponding to categoryAm0

is found in distance set{Dmj}. Step 2:

If categoryAm0 belongs to “co-contribution,” and max(Dmin(Dm0L,Dm0R)

m0L,Dm0R) < T, the testing interac-tion is determined to be “co-contribuinterac-tion” interacinterac-tion.

If category Am0 belongs to “single-contribution,” and max(Dmin(Dm0L,Dm0R)

m0L,Dm0R) > T, the testing interaction is determined to be “single-contribution” interaction. max(Dm0L, Dm0R)indicates the major participant in the testing interaction.

Step 3:

If{Dm0L, Dm0R}fits neither of the two situations in step 2, the distance pair{Dm0L, Dm0R} are set to 0 in{Dmj}, return to step 1 to select the second largest distance.

The processing continues until the contribution estimation result is given in step 2.

The classification result clearly indicates the contribution class of the testing interaction sample, and determines the major actions in single-contribution interactions.

ドキュメント内 局所特徴を用いた動画像中の人間動作認識 (ページ 77-82)

関連したドキュメント