Variable Importance Cloud

Variable Importance Cloud

Dong Rudin Variable Importance Cloud

: 88

!_! !_"

"_# 0.4 0.2

"_$ 0.05 0.3

VIC

• VI : Variable Importance

• [Breiman+, 2001]

Breiman, Leo. "Statistical modeling: The two cultures (with comments and a rejoinder by the author)."Statistical science16.3 (2001): 199-231

VI

:

: Variable Importance Cloud

• :

• Variable Importance Cloud VIC [Dong+, 2019]

•

• : (VI)

• VIC

/

ℎ _" ℎ _# $ _% $ _&

' ₍ 0.4 0.2 ' ₎ 0.05 0.3

VIC

Dong, J. and Rudin, C. Variable importance clouds: A way to explore variable importance for the set of good models. arXiv preprint arXiv:1901.03209, 2019.

• VIC Variable Importance Cloud

•

VIC

ℎ _" ℎ _# $ _% $ _&

' ₍ 0.4 0.2 ' ₎ 0.05 0.3

VIC

Dong, J. and Rudin, C. Variable importance clouds: A way to explore variable importance for the set of good models. arXiv preprint arXiv:1901.03209, 2019.

VIC

1. ! ^∗

2. 1 + % ! ^∗

&

3. ℛ

• TDIDT CART

•

ℛ % = 0.2

ℎ

: 0.11

ℎ _.

: 0.12

/ ₀ / ₁ 2 ₃ 0.4 0.2 2 ₄ 0.05 0.3

VIC

: 0.1

VIC :

[Dong+, 2019] (! = 4)

:

1. !

2.

: !

2 :

1. !

decision fern 2.

:

$ _% , … , $ ₍

$ _% $ _% , … , $ _()*

$ _%

0

$ _* $ _*

1 1 0

0

0

1

0 1

1

ℎ _* : 0, 1, 1, 0

ℎ _/ : 0, 1, 1, 0

ℎ ₁ : 0, 2, 2, 0

VIC :

:

1.

ExtraTrees 2.

: :

: Lawler [Ruggieri+, ICML2017]

1.

2. :

: :

! ∖ ! _# ! ∖ ! _$ ! ∖ ! _%

… … …

RF

(VIC )

• monk1

• Adult Lawler

1. !

ℛ ! = 100

2. ℛ VIC

•

• Permutation Importance

1 :

Adult Lawler

•

•

Lawler

•

Lawler x10~x1000

1:

Lawler

Lawler

0.139 0.139

0.164 0.139

0.175 0.145

•

• ! _" , ! _$ 0.99

• ! _" , ! _% ! _$ , ! _% -0.6

! _" ! _$ ! _& ! _' ! _% ! ₍

! ( ! % ! ' ! & ! $ ! "

! _" ! _$ ! _& ! _' ! _% ! ₍

! ( ! % ! ' ! & ! $ ! "

2 : monk1

:

2 : monk1

monk1 ! _" = ! _$ ∨ ! _& = $ 1 0

• ' ₍ , ' _*

→

• ' ₍ , ' ₊ ' _* , ' ₊

→

VIC

3 : Adult

•

• capital_gain, capital_loss 0.97

• age, marital_status -0.6

• education_num, marital_status 0.8

age edu_num marital sex c_gain c_loss age edu_num marital sex c_gain c_loss

c_ lo ss c_ g a in se x m a ri ta l edu_num age c_ lo ss c_ g a in se x m a ri ta l edu_num age

: Lawler

3 : Adult

• capital_gain, capital_loss

→

• age, marital_status

→

• education_num, marital_status

→

•

• VIC

• VIC

VIC

•

•

•

•

• OSDT[Hu+, 2019]

•

[LAMP, Terada+ PNAS 2013]

Hu, Xiyang, Cynthia Rudin, and Margo Seltzer. "Optimal sparse decision trees."Advances in Neural Information Processing Systems. 2019.

!_! !_"

"# 0.4 0.2

"$ 0.05 0.3

VIC