Representations of materials in the human visual cortex(Summary of Awarded Presentation at the 29th Annual Meeting)

(1)

The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

TheJdpanesefoumatofPs.vchonomicScience

201L VoL 30,Ne.1,135-136

Summary

ofAwarded

Presentation1-325

Representations

of

materials

in

the

human

visual

cortex

Chihiro

HiRAMATsu*-

*2' *3,

_Naokazu

_GoDA*･

*4, _and

_Hidehiko

_KoMATsu*'

*4

National institute

for

PhysiologicatSciences*,KbiotoUitiversity*2,

foPan

Society

fbr

the Promotion

_of

Science*3,

The

Graduate

UhiveJ:sity

for

Advanced

Studies

(SOKELVDAO*4

Our ability torecognize surface _qualitiesand inferthe materials that make up objects allows

us to interactappropriately with the objects, Littleisknown, however, about the mechanisms oi

material representation

in

the

brain.

In

thisstudy, we

investigated

how

information

about various materials

is

processed

in

thebrainusing a combination of multivoxel pattern analysis of

functional

magnetic resonance imaging data and _perceptual and image-based _physical measures of rnaterial

properties. We found that

information

about materials istransforrned from image-based

represen-tatiens

in

early visual areas intoperceptual category representation$ along the ventral _pathway.

Key words: vision, functionaJ magnetic resonance imaging, material perception

Objects

made from various materials

(metal,

wood,

fabriq

etc.) are characterized

by

different

surface

qualities. These characteristics _provide important

information

forobject identificationand

categoriza-tion,shaping affective

impressions

and mediating

our interactionswith the objects.

Previous studies

{Cant

&

Goodale,

2007;

Cavina-Pratesi

et al.2010)

have

shown that surface proper-tiesare encoded near the

fusiform

_gyrus and

collat-eral sulcus

(FG/CoS),

Inthe _present study, we

fur-ther examined representations of materials

in

the hurnan visual cortex.

Methods

Participants. Five adults

_<three

femaLes)with nor-rnal vision _participatcdinthisstudy.

StimulL

Eight

exemplars

for

each of nine material

(metal,

ceramic, _glass,stone, bark, wood,

leather,

fabric,

and fur)were synthesized using

com-puter graphic software

<LightWave

3D;Figure 1).

Psychological experiments.

Perceptual

dissimi-larities

between

images

were assessed using the se-mantic differential

(SD)

method and 12 adjective

pairs that described visual and nonvisual

impres-sions of the surfaces

(e.g.

`matte-glossy', 'soft-hard').

Image statistics. Image-based _physical

dissimi-Iaritieswere assessed using 20 low-levelimage

statis-ticsconsisting of

12

subband statistics

(four

orjenta-tions×three spatia] frcquencies)and eight moments

of CIELAB coordinates

(mean

and standard

devia-tionof

L*,

a*,

b*

and skew and kurtosis of

L*).

Functional rnagnetic resonance

imaging

(fMRI)

acquisition and analysis. During fMRI acquisition,

each image

(7.5e

X7.50)

_in

_the category

blocks

was

*

_Graduate

_School

_of

_Letters,

_Kyoto

_University

Yoshida-honmachL Sakyo-ku, Kyoto 606-8501,

Japan

Figurel.

Example

images

from

the nine

material categories

(colored

stimuli were

used in the experiments). Modified

from

Hiramatsu et al.

_{201

1).

(2)

The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

136 (a)Ceramic

Leather MeteiGtssStafie BarkWoodFainric F-f Ceramic Legther kMetaiGlassstene eerkWoedFahtit Fur

The

Japanese

Journal

of Psychonomic

S

Image-based

MeCeG!SiBaVVoteFnFu

VW2

Figure 2.

each ROI and image-base

MeCeGEStBaWti.eFeFu

{a)

Dissimilarity taeCeG[su･BsWoteFeF" medeGlstBaWotefaFu

Perception

MeCeGIStBaWci.eFeFu

FGiCoS

MeCeGISteaWot.eFaFu

matrices foreach

l･IptOe

g.

･...

m;,/,/iso

illi

g

60

_Q'

Ree

9 w'

eV

(b>

cTence feWe.l･B

sedi

Ig

s?$Ee-ag

#sts･

gs

sm

ee

e

Vol.

30,

No.

1

e.4

Vl!V2

FGfCoS

te --- sl .v'sA"."rtr s::va.7 e ' _rza

g･81

a . _r= a.3a 10 x-"1'?g'l;.f,

"

.,,

S e

o.6 o.g QA e.s Q.s

Neurzal

dissimtlmrity

between

neural

dissimilarities

.

(2011),

analysis.

Cb)

Correlation

d

or _perceptual

dissimilarities.

Modified

from

Hiramatsu et al

in

presented centrally for500 ms and participants were

required

to

fixate

on

the

center of the screen and

press a button when the color of the fixationspot

changed from red to _green. fMRI images

(T2*-weighted gradient-echo echo planar sequence:

TR=

2s, TE=30 rns, voxel size=3 ×3×3mm) were

ac-quired using a

Siemens

Allegra

3T

scanner,

Multivoxel pattern analysis was used to extract

patterns of neural activity

for

each category.

Neural

dissimilaritiesbetween categories were assessed

us-ing _pairwise category classification and linear

sup-portvector machine with activation patterns

for

500

visually responsive voxels within a region of interest

(ROI)

for

each subject.

Regions

of

interest

were

determined by a separate retinotopic mapping

ex-periment and anatomical

landmarks.

Results

Matrices

in

Figure

2(a)

depict

dissimilarities

be-tween categories ineach ana]ysis. Image-based

(up-per

left

panel) and _perceptual

(upper

right _panel)

dissimilaritieswere calculated as

Euclidean

dis-tances

between

the

category _pairsusing rnean image

statistics and mean ratings

from

the

SD

method

across the eight exemplars, respectively. Neural dis-similarity rnatrices

for

early retinotopic areas

(Vlf

V2} and higher ventral areas near the FG/CoS were

obtained

based

on mean _pairwise classification

accu-racy ineach ROI across subjects, The significance of the correlation

between

neural

dissimilarities

with

image-based dissimilaritieswas higher

for

VllV2

(one-tailed

permutation test,P<

O,OOO1)

than

for

FG/

CoS

_(P

<O.029;Figure 2(b).upper _panels),whereas the opposite was observed for the correlation with per-ceptual

dissimilarities

to=O.O16

for

VllV2,

_P=

O,0001 forFG/CoS; Figure 2(b),lower _panels).

Discussion

The

neural

dissimilarities

in

early and

higher

ven-tralvisual areas correlated with image-based and

perceptual dissimilaritiesdifferently,This suggests

that

representations of materials change

from

image-based representations inearly visual areas to

percep-tual

category representations along the ventral

path-way. The results indicate that inforrnation about

multimodal characteristics of materials

is

processed

in

the ventra] cortex near the FG, where the signals can be used tocategorize the materials.