関連ベクターマシンに基づく非線形回帰モデリング

関連ベクターマシンに基づく非線形回帰モデリング

Nonlinear Regression Modeling via the Relevance Vector Machine

数学専攻 松田和己

Kazuki Matsuda

1 ^はじめに

近年の計算機システムの飛躍的な向上は

,

,

,

,

,

,

,

.

,

,

,

.

関連ベクターマシン

(RVM, Tipping, 2001)

,

,

,

判別問題に広く応用されている

. RVM

,

,

. RVM

,

AIC

BIC

,

.

,

,

.

, RVM

.

大規模な自然災害などは

,

,

.

,

.

, Tateishi and Konishi(2011)

, RVM

.

,

.

, 2011

3

.

1

2 RVM ^{回帰モデリング}

説明変数

y

1

x

n

{ (y _i , x _i ); i =

1, 2, · · · , n }

. RVM

,

として広く利用される基底展開法に対して

,

.

y i = w 0 +

∑ n

j=1

w j exp {

− (x i − x j ) ² 2h ²

}

+ ε i i = 1, 2, · · · , n. (2.1)

ただし

, w j (j = 0, 1, · · · , n)

, h ²

ス型カーネル関数の広がりの程度を調整するパラメータである

.

, ε i (i = 1, 2, · · · , n)

N (0, β ⁻¹ )

, β

.

,

w = (w 0 , w 1 , · · · , w n ) ^T

, ARD(Automatic Relevance Determination)

p(w | α) =

∏ n

j=0

N (w _j ; 0, α ⁻ _j ¹ ) (2.2)

,

0

,

.

で

, α j (j = 0, 1, · · · , n)

w j

.

このとき

,

,

h ²

.

, RVM

,

h ²

,

AIC, BIC

.

,

PIC

,

.

, RVM

,

,

, Multi-Overlapping RVM

.

3 RVM ^{に基づく変化点探索}

大規模な自然現象や企業の倒産

,

,

,

.

,

.

2

Tateishi and Konishi(2011)

, RVM

,

,

.

.

, 2011

3

11

.

, 2011

3

15

10

.

0 20 40 60 80 100 120 140

0500100015002000250030003500

x

y

0 20 40 60 80 100 120 140

0500100015002000250030003500

x

y

図

1

10

(

)

(

)

参考文献

[1] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle, 2nd Inter. Symp. Information Theory (eds. by B. N. Petrov and F. Csaki), Akademiai Kiado, Budapest, 267 ― 281. (Reproduced in Breakthroughs in Statistics, Vol. I, Foundations and Basic Theory (eds. S. Kotz and N. L. Johnson), Springer-Verlag, New York, (1992) 610 ^― 624.) [2] Akaike, H. (1977). On entropy maximization principle, Applications of Statistics (Krishinaiah,

P.R., ed.), North-Holland, 27(41).

[3] Akaike, H. (1978). A Bayesian analysis of the minimum AIC procedure. Annals of the Institute of Statistical Mathematics. 30, 9-14.

[4] Akaike, H. (1979). A Bayesian extension of the minimum AIC procedure of autoregressive model fitting. Biometrika, 66, 237-242.

[5] Ando, T., Konishi, S. and Imoto, S. (2008). Nonlinear regression modeling via regularized radial basis function networks. Journal of Statistical Planning and Inference, 138, 3616 ^― 3633.

[6] Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.

[7] Burnham, K. P. and Anderson, D. R. (2002). Model Selection and Multimodel Inference. A Practical Information-Theoretic Approach, 2nd ed., Springer.

[8] Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation, Numerische Mathe- matik. 31, 377-403.

[9] Davison, A. C. (1986). Approximate predictive likelihood. Biometrika, 73, 323 ^― 32.

[10] de Boor, C. (2001). A Practical Guide to Splines. Springer.

[11] Denison, D. G. T., Holmes, C. C., Mallick, B. K. and Smith A. F. M. (2002). “Bayesian Methods for Nonlinear Classification and Regression”. Wiley

3

[12] Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression (with discussion), Ann. Statist., 32, 407 ^― 499.

[13] Friedman, J., Hastie, T. and Tibshirani, R. (2009). Regularization paths for generalized linear models via coordinate descent, Technical Report, Stanford University.

[14] Gijbels, I., Lambert, A. and Qiu, P. (2007). Jump-preserving regression and smoothing using local linear fitting: a compromise. Annals of the Institute of Statistical Mathematics, 59, 235 ^― 272.

[15] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models, London, Chapman & Hall.

[16] Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning (2nd edition). Springer ^― Verlag, New York.

[17] Imoto, S. and Konishi, S. (2003). Selection of smoothing parameter in B-spline nonparametric regression models using information criteria. Annals of the Institute of Statistical Mathematics, 55, 671 ^― 687.

[18] Kawano, S. and Konishi, S. (2007). Nonlinear regression modeling via regularized Gaussian basis functions. Bull. Inform. Cybern., 39, 83 ― 96.

[19] Kitagawa, G. (1997). Information criteria for the predictive evaluation of bayesian models. 

Communications in Statistics-Theory and Methods, 26, 2223 ^― 2246.

[20] Konishi, S., Ando, T. and Imoto, S. (2004). Bayesian information criteria and smoothing pa- rameter selection in radial basis function networks. Biometrika, 91, 27 ― 43.

[21] Konishi, S. and Kitagawa, G. (1996). Generalised information criteria in model selection.

Biometrika, 83, 875 ^― 890.

[22] Konishi, S. and Kitagawa, G. (2008). Information Criteria and Statistical Modeling. Springer.

[23] Kullback, S. and Leibler, R. A. (1951). On information and sufficiency, The Annals of Mathe- matical Statistics, 22, 79 ^― 86.

[24] Loader, C. R. (1996). Change point estimation using nonparametric regression. Annals of Statis- tics, 24, 1667 ― 1678.

[25] MacKay, D. J. C. (1994). Bayesian methods for backpropagation networks. In E. Domany, J. L.

van Hemmen, and K. Schulten (Eds.), Models of Neural Networks III, Chapter 6, pp. 211254.

Springer.

[26] Muller, H. G. (1992). Change-points in nonparametric regression analysis. Annals of Statistics, 20, 737 ^― 761.

[27] Neal, R. M. (1996). Bayesian Learning for Neural Networks. Springer.

[28] Qiu, P. (2003). A jump-preserving curve fitting procedure based on local piecewise-linear kernel estimation. Journal of Nonparametric Statistics, 15, 437 ^― 453.

[29] Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461 ^― 464.

[30] Speckman, P. L. (1997). Detection of change-points in nonparametric regression. Unpublished manuscript.

[31] Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions (with discus- sion), J. Roy. Statist. Soc. B, 36, 111 ― 147.

[32] Tateishi, S. and Konishi, S. (2011). Nonlinear regression modeling and detecting change points via the relevance vector machine. Computational Statistics, 26, 477-490.

[33] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser.

B, 58, 267 ― 288.

[34] Tipping, M.E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 211 ^― 244.

4