Algorithm

Here, we explain the algorithm of the NPCA-based method using the K-means type constraint connector matrix. When we set r_bx=r_by, r_cx =r_cy, c_b =r_bx, c_c =r_cx, D_x = I, and D_y = I, objective function g_ccca equals objective function g_npc. Therefore, the NPCA-based method is a special case of using theK-means type constrained connector matrix method.

Proposition 3.13.When we setZ^(x)= [X₁^†Q]⁽¹⁾_J

x,Kx andZ^(y)= [Y₁^†W]⁽¹⁾_J

y,Ky. The update

formula of Bx, By, Cx, and Cy is obtained as follows:

Bx =UbxV_bx^′ (3.58)

B_y =U_byV_by^′ (3.59)

C_x =U_cxV_cx^′ (3.60)

Cy =UcyV_cy^′ (3.61)

where Ubx and Vbx are left and right singular matrixes of Z₂^(x)(C_x⊗I)^′F₂^(x)′. U_by and V_by are left and right singular matrixes of

Z₂^(y)(Cy⊗I)F₂^(y)′. U_cx and V_cx are left and right singular matrixes of

Z₃^(x)(B_x⊗I)F₃^(x)′. Ucy andVcy are left and right singular matrixes of

Z₂^(y)(B_y⊗I)F₃^(y)′.

Proof. First, we explain the update formula of B_x. The term related to B_x is the first term. The first term of objective functiong_ccca is rewritten as follows:

∥X1Q−F₁^(x)(Cx⊗Bx)^′∥² =∥Z₁^(x)−F₁^(x)(Cx⊗Bx)^′∥²

=∥Z₂^(x)−BxF₂^(x)(Cx⊗I)^′∥²

=tr(Z₂^(x)′Z₂^(x))−2tr(Z₂^(x)′B_xF₂^(x)(C_x⊗I)^′) + tr((C_x⊗I)F₂^(x)B_x^′B_xF₂^(x)(C_x⊗I)^′)

=tr(Z₂^(x)′Z₂^(x))−2tr(BxF₂^(x)(Cx⊗I)^′Z₂^(x)′) + tr((Cx⊗I)F₂^(x)′F₂^(x)(Cx⊗I)^′).

Given parameters exceptB_x, we obtain the update formula of B_x by maximizing

tr(B_xF₂^(x)(C_x⊗I)^′Z₂^(x)′). From the TenBerge theorem (ten Berge, 1993), tr(B_xF₂^(x)(C_x⊗ I)^′Z₂^(x)′) ≤tr(D)holds. D is the diagonal matrix whose elements are singular values of V DU^′ =F₂^(x)(C_x⊗I)^′Z₂^′. When we set B_x =U V^′, the equation holds. Therefore, we obtain the update formula. The update formula ofB_y is obtained in the same way asB_x. Next, we explain the update formula of C_x, which is obtained in a very similar way to B_x. The term related to C_x is the first term. The first term of objective functiong_ccca is rewritten as follows:

∥X1Q−F₁^(x)(Cx⊗Bx)^′∥² =∥Z₁^(x)−F₁^(x)(Cx⊗Bx)^′∥²

=∥Z₃^(x)−CxF₃^(x)(Bx⊗I)^′∥²

=tr(Z₃^(x)′Z₃^(x))−2tr(Z₃^(x)′C_xF₃^(x)(B_x⊗I)^′) + tr((B_x⊗I)F₃^(x)C_x^′C_xF₃^(x)(B_x⊗I)^′)

=tr(Z₃^(x)′Z₃^(x))−2tr(BxF₃^(x)(Cx⊗I)^′Z₃^(x)′) + tr((Bx⊗I)F₃^(x)′F₃^(x)(Bx⊗I)^′).

Given another parameter exceptCx, we obtain the update formula of Cx by maximizing tr(CxF₃^(x)(Bx⊗I)^′Z₃^(x)′). From the TenBerge theorem, tr(CxF₃^(x)(Bx⊗I)^′Z₃^(x)′)≤tr(D) holds. D is a diagonal matrix whose elements are singular values ofV DU^′ =F₃^(x)(Bx⊗ I)^′Z₃^′. When we set Cx = U V^′, the equation holds. Therefore, we obtain the update formula. The update formula ofCy is obtained in the same way asCx.

Proposition 3.14. The update formulas of F^(x) and F^(y) are obtained as follows:

F₁^(x) =(X₁^†Q(Cx⊗Bx) +F₁^(y)DyD^′_x)(I+DxD_x^′)⁻¹, (3.62) F₁^(y) =(Y₁^†W(C_y⊗B_y) +F₁^(x)D_xD_y^′)(I +D_yD_y^′)⁻¹. (3.63) Proof. First, we explain about the update formula ofF₁^(x).

∥X₁^†Q−F₁^(x)(Cx⊗Bx)^′∥²+∥F₁^(x)Dx−F₁^(y)Dy∥²

=−2tr(F₁^(x)(C_x⊗B_x)^′Q^′X₁^†′)−2tr(F₁^(x)′F₁^(y)D_yD^′_x)

+ tr(F₁^(x)D_xD_x^′F₁^(x)′) + tr(F₁^(x)(C_x^′C_x⊗B_x^′B_x)F₁^(x)) + const.,

where const. is constant independence fromF₁^(x). Thus, the partial derivative function of gccca with respect toF₁^(x) is obtained as follows:

∂

∂F₁^(x){∥X₁^†Q−F₁^(x)(Cx⊗Bx)^′∥²+∥F₁^(x)Dx−F₁^(y)Dy∥²}

=−2X₁^†Q(Cx⊗Bx)−2F₁^(y)DyD^′_x+ 2F₁^(x)DxD_x^′ + 2F₁^(x).

When we set the partial derivative function of gccca with respect to F₁^(x) as 0, we obtain the following equation:

−2X₁^†Q(C_x⊗B_x)−2F₁^(y)D_yD_x^′ + 2F₁^(x)D_xD_x^′ + 2F₁^(x) = 0

⇐⇒F₁^(x)DxD_x^′ +F₁^(x)=X₁^†Q(Cx⊗Bx) +F₁^(y)DyD_x^′

⇐⇒F₁^(x)= (X₁^†Q(Cx⊗Bx) +F₁^(y)DyD^′_x)(I+DxD_x^′)⁻¹. The update formula ofF₁^(y) is obtained in the same way asF₁^(x).

∥Y₁^†W −F₁^(y)(Cy⊗By)^′∥²+∥F₁^(x)Dx−F₁^(y)Dy∥²

=−2tr(F₁^(y)(C_y⊗B_y)^′W^′Y₁^†′)−2tr(F₁^(y)′F₁^(x)D_xD^′_y)

+ tr(F₁^(y)D_yD^′_yF₁^(y)′) + tr(F₁^(y)(C_y^′C_y⊗B_y^′B_y)F₁^(y)) + const.,

where const. is constant independence fromF₁^(y). Thus, the partial derivative function of gccca with respect toF₁^(y) is obtained as follows:

∂

∂F₁^(y){∥Y₁^†W −F₁^(y)(Cy⊗By)^′∥²+∥F₁^(x)Dx−F₁^(y)Dy∥²}

=−2Y₁^†W(Cy⊗By)−2F₁^(x)DxD_y^′ + 2F₁^(y)DyD_y^′ + 2F₁^(y).

When we set the partial derivative function ofF₁^(y)as 0, we obtain the Following equation:

−2Y₁^†W(C_y⊗B_y)−2F₁^(x)D_xD^′_y+ 2F₁^(y)D_yD^′_y+ 2F₁^(y)= 0

⇐⇒F₁^(y)D_yD_y^′ +F₁^(y)=Y₁^†W(C_y⊗B_y) +F₁^(y)D_xD^′_y

⇐⇒F₁^(y)= (Y₁^†W(Cy⊗By) +F₁^(x)DxD^′_y)(I+DyD^′_y)⁻¹.

Proposition 3.15. The update formula ofq_kxjx is obtained as follows:

q_kxjx =√ I(X_k^†

xjx

′X_k^†

xjx)⁻¹²u^(qx)₁ , (3.64) where u^(qx)₁ is the first dimension left singular vector of

(X_k^†

xjx

′X_k^†

xjx)⁻¹²X_k^†

xjx

′J_n(∑_rcx

ℓ c^(x)_k

xℓF_(ℓ)^(x)b^(x)_j

x ). c^(x)_k

xℓ is (k_x, ℓ) element of C_x. b^(x)_j

x is the j_x-th row vector of B_x. F_(ℓ)^(x) is the matrix corresponding to dimensionℓ of C_x.

The update formula of w_kyjy is obtained as follows:

wkyjy =√ I(Y_k^†

yjy

′Y_k^†

yjy)⁻¹²u^(wy)₁ , (3.65) where u^(wx)₁ is the first dimension left singular vector of

(Y_k^†

yjy

′Y_k^†

yjy)⁻¹²Y_k^†

yjy

′Jn(∑rcy

ℓ c^(y)_k

yℓF_(ℓ)^(y)b^(y)_jy ). c^(y)_k

yℓ is(ky, ℓ) element ofCy. b^(y)_jy is thejx-th row vector ofBy. F_(ℓ)^(y) is the matrix corresponding to dimensionℓ of Cy

Proof. First, we explain about the update formula of q_kxjx. From definition Q, q_kxjx are independent from each other. Thus, the update formula of q_kxjx can be calculated individually. The term that is related toq_kxjx is the first term of g_ccca. The first term of g_ccca is rewritten as follows:

∥X₁^†Q−F₁^(x)(Cx⊗Bx)^′∥²=−2tr(Q^′X^†′₁F₁^(x)(Cx⊗Bx)^′) + const. (3.66) From equation (3.66), we consider the minimization problem as theQ that maximizes tr(Q^′X^†′₁F₁^(x)(C_x⊗B_x)^′). From the definition ofQ, in order to maximize tr(Q^′X^†′₁F₁^(x)(C_x⊗ B_x)^′), we consider each value of q_kxjx. Objective function g^∗ for q_kxjx is obtained as fol-lows:

g^∗(q_kxjx |C_x, B_x, F_x, X^†) = tr(q_k^′_xjxX_k^†

xjx

′∑rcx

ℓ

c^(x)_k

xℓF_(ℓ)^(x)b^(x)_j

x ),

From the constraint on q_kxjx, this objective function g^∗ is very similar to the objective function of canonical correlation analysis. From the constraintX_j^†

xkxq_jxkx =J X_j^†

xkxq_jxkx, X_j^†

xkxq_jxkx is the element of complementary space of 1. Therefore, first, the X_j^†

xkx is projectedJ space. Then, we search the parameters maximizingg^∗. Thus, we change the objective functiong^∗ tog₁^∗ as follows:

g^∗₁(q_kxjx |Cx, Bx, Fx, X^†) = tr(q_k^′_xjxX_k^†

xjx

′J

rcx

∑

ℓ

c^(x)_k

xℓF_(ℓ)^(x)b^(x)_jx ).

When we set q^∗_kxjx = √¹ I(X_k^†

xjx

′X_k^†

xjx)^1/2q_kxjx, equation q_k^∗_xjx^′q^∗_kxjx = 1 holds from the constraint case ofgccca. Therefore, we can rewriteg^∗₁ as follows:

g₁^∗(q_kxjx |C_x, B_x, F_x, X^†) =√

Itr(q^∗_kxjx^′(X_k^†

xjx

′X_k^†

xjx)⁻¹²X_k^†

xjx

′J

rcx

∑

ℓ

c^(x)_k

xℓF_(ℓ)^(x)b^(x)_j

x ).

g₁^∗ is the same as the objective function of the weight matrix in the categorical canonical covariance case. Therefore, we obtain the update formula ofq_k^∗_xjx as

q_k^∗_xjx =u^(qx)₁ , whereu^(qx)₁ is the first dimension left singular vector of (X_k^†

xjx

′X_k^†

xjx)⁻¹²X_k^†

xjx

′J∑_rcx

ℓ c^(x)_k

xℓF_(ℓ)^(x)b^(x)_j

x . From the definition of q_k^∗

xjx, the update for-mula ofq_kxjx is obtained as follows:

q_kxjx =√ I(X_k^†

xjx

′X_k^†

xjx)⁻¹²u^(qx)₁ .

The update formula of w_kyjy is obtained in the same way asq_kxjx.

The update formulas of D_x and D_y are the same as the K-means algorithm.

Proposition 3.16. The update formulas of Dbx, Dby, Dcx, and Dcy are obtained as follows:

d^(bx)_ℓq =





 1

(

ℓ= arg min

ℓ^∗

[F₂^(x)(Dcx⊗I)]ℓ^∗−d^(by)q ′F₂^(y)(Dcy⊗I) ) 0 (otherwise)

(q= 1, 2, . . . , c_b),

(3.67)

d^(by)_ℓq =





 1

(

ℓ= arg min

ℓ^∗

[F₂^(y)(D_cy⊗I)]_ℓ∗−d^(bx)_q ^′F₂^(x)(D_cx⊗I) ) 0 (otherwise)

(q= 1, 2, . . . , c_b),

(3.68)

d^(cx)_ℓq =





 1

(

ℓ= arg min

ℓ^∗

[F₃^(x)(D_bx⊗I)]_ℓ∗−d^(cy)q ′F₃^(y)(D_by⊗I_n) ) 0 (otherwise)

(q= 1, 2, . . . , cc),

(3.69)

d^(cy)_ℓq =





 1

(

ℓ= arg min

ℓ^∗

[F₃^(y)(Dby⊗I)]ℓ^∗−d^(cx)q ′F₃^(x)(Dbx⊗In) ) 0 (otherwise)

(q= 1, 2, . . . , c_c),

(3.70)

where d^(bx)_ℓq , d^(by)_ℓq , d^(cx)_ℓq and d^(cy)_ℓq are the (ℓ, q) element of D_bx, D_by, D_cx, and D_cy, respectively. [A]_ℓ is the ℓ-th column vector ofA.

Proof. From the definition ofgccca, the third term depends on D_bx, D_by, Dcx, and Dcy. We rewrite the third term ofgccca as follows:

∥F₁^(x)D_x−F₁^(y)D_y∥² =∥F₁^(x)(D_cx⊗D_bx)−F₁^(y)(D_cy⊗D_by)∥²

=∥D^′_bxF₂^(x)(D_cx⊗I)−D_by^′ F₂^(y)(D_cx⊗I)∥² (3.71)

=∥D^′_cxF₃^(x)(D_bx⊗I)−D_cy^′ F₃^(y)(D_by⊗I)∥² (3.72) From the constraint of D_bx, D_by, equation (3.71) is equivalent to K-means given other parameters. Thus, the update formulas of D_bx, D_by are obtained in the same way as K-means.

From the constraint ofD_cx, D_cy, equation (3.72) is equivalent toK-means given other parameters. Thus, the update formulas of D_cx, D_cy are obtained in the same way as K-means.

Summarizing the update formulas, we obtain the algorithm of categorical canonical covariance analysis for three-mode three-way data as algorithm 6.

Algorithm 6 Algorithm of the NPCA-based method

Set the number of dimensionsrbx, rby, rcx, rcy, cb, cc, and stop conditionε Set initial valuesBx⁽⁰⁾, Cx⁽⁰⁾, By⁽⁰⁾, Cy⁽⁰⁾, F⁽⁰⁾x , F⁽⁰⁾y , D_bx⁽⁰⁾, D⁽⁰⁾_by,D⁽⁰⁾cx,and Dcy⁽⁰⁾, t←0

S⁽⁰⁾←gccca(F⁽⁰⁾x , F⁽⁰⁾y , Bx⁽⁰⁾, By⁽⁰⁾, Cx⁽⁰⁾, Cy⁽⁰⁾, Q⁽⁰⁾, W⁽⁰⁾, D⁽⁰⁾x , Dy⁽⁰⁾|X^†, Y^†) repeat

t←t+ 1

Update B_x and B_y using C_x^(t−1), F^(t_x⁻¹⁾,C_y^(t−1), F^(t_y⁻¹⁾ Update C_x and C_y using B_x^(t), F^(t_x⁻¹⁾,B_y^(t), F^(t_y⁻¹⁾

Update F_x using B_x^(t), B_y^(t),C_x^(t), C_y^(t), F^(t_y⁻¹⁾,D_x^(t−1), D_y^(t−1) Update F_y using B_x^(t), B_y^(t),C_x^(t), C_y^(t), F^(t)_x ,D_x^(t−1), D^(t_y⁻¹⁾ Update D_bx usingB^(t)_x , B^(t)_y ,C_x^(t), C_y^(t), F^(t)_x ,F^(t)_y , D^(t_cx⁻¹⁾, D^(t_y⁻¹⁾ Update D_cx using B_x^(t), B_y^(t),C_x^(t), C_y^(t), F^(t)_x ,F^(t)_y , D^(t)_bx, D^(t_y⁻¹⁾ Update D_by using B_x^(t), B_y^(t),C_x^(t), C_y^(t), F^(t)_x ,F^(t)_y , D_x^(t), D_cy^(t−1) Update D_by using B_x^(t), B_y^(t),C_x^(t), C_y^(t), F^(t)_x ,F^(t)_y , D_x^(t), D_by^(t)

S^(t)←g_ccca(F^(t)x , F^(t)y , Bx^(t), By^(t), Cx^(t), Cy^(t), Q^(t), W^(t), Dx^(t), Dy^(t)|X^†, Y^†) until |S^(t−1)−S^(t)| ≤ε

Chapter 4

Simulation studies

In this chapter, we compare canonical covariance analysis for three-mode three-way data with that for two-mode two-way data using several evaluations. In the K-means based method case, we compare our proposed method with two-mode two-way canonical covariance analysis and no connector matrixes with the mean squared loss between the true and estimated weight matrixes. In the regression-based method case, we compare our proposed method with two-mode two-way methods and three-mode three-way regression using mean squared loss of prediction. In the quantification method case, we compare our proposed method with two-mode two-way canonical covariance analysis, no connector matrixes, and no quantification method using the mean squared loss between the true and estimated weight matrixes.

4.1 Constrained connector method

In this section, we describe a numerical example for the contained connector method.

The purpose of introducing the constraint for parameters is different between theK-means type and the regression type. Therefore, we separate these two types. In theK-means type situation, we evaluate the squared error of parameters. In the regression type situation, we focus on the prediction error.

ドキュメント内 Dimensional reduction method for three-mode three-way data based on canonical covariance analysis (ページ 50-56)