Japan Advanced Institute of Science and Technology

Japan Advanced Institute of Science and Technology

JAIST Repository

https://dspace.jaist.ac.jp/

Title

Author(s)

Citation

Issue Date

Type

Text version

URL

Rights

Description

concerned with sound segregation

Masashi Unoki

Scho ol of Information Science

Japan Institute of Science and Technology

14 January1999

Abstract

The aim of this paper is to construct a computational theory of audition. This work is to

explain the following questions: \what is a purp ose of auditory processing?" and \why must

auditory systemcompute it?",based onresearcheson psychology,physiology, and information

science. This computational theory corresponds to the auditory edition of the computational

theoryofvision proposedbyMarr. Ifthecomputational theoryofaudition canbeconstructed,

it can not only clarify human auditory functions but also contribute to some applications

such as a signal processing, robust speech recognition, and a modeling of psychoacoustical

phenomena. HoweverthecomputationaltheoryofauditioninanalogytoMarr'stheoryhasnot

b een constructedcompletely becausepsychoacousticaland physiologicalknowledgeofaudition

is not sucient toconstruct it inanalogy to Marr's theory.

This paper proposes a computational theory of audition concerned with sound segregation

based on the following approaches in analogy to Marr's theory: constraints on sound waves

and environment conditions are necessary in order to uniquely solve the problem (ill-p osed

inverse problem) of segregating the desired signal from mixed signals. This paper adopts

the following idea as a construction method of the computational theory: psychoacoustical

constraints that auditory system uses to solve the problem of auditory scene analysis, that is

the fourregularitiesproposedby Bregman,canb eusedtouniquelysolvethesignalsegregation

problem as mathematical constraints. This paper focuses on \segregation of two sounds" as

a fundamental auditory function. Therefore, the problem of segregating sounds is set to \the

problem ofsegregatingtwoacousticsources." It issupp osedthat thedesiredsignalis\an AM-

FMharmoniccomplextone"suchasvowelandinstrumentalsound. Moreover,acomputational

theory of audition is dened as a strategy of sound segregation, \how are the problem of

segregating twoacousticsources solveduniquely using the constraints ?"

In this paper, the problem of segregating two acoustic sources based on an amplitude and

phase spectra wasformulated. The four regularities proposed by Bregman were formulatedas

mathematicalconstraints: (i)common onsetandosetforthe comp onentofthe complextone,

(ii) continuitydenedby thepiecewise-polynomialapproximationandthe splineinterp olation,

(iii) harmonicity,and (iv)correlationbetween the amplitudeenvelopes. A methodof segregat-

ing AM{FMharmonic complextonefromthemixed signalusing theconstraintswasprop osed.

This metho d was examined whether it could be segregated the desired signal from the mixed

the sucient constraints. The derivedstrategy is touniquely solve the problem of segregating

two acoustic sources by regarding it as the piecewise linear problem and by constraining the

temp oral uctuations of the amplitude and the phase of the desired signal. Finally, the de-

rivedstrategy was examinedby applying ittheory into tworeal segregation problems: (1) the

problem of segregating the desiredreal speech(vowels) fromnoisyspeechand (2) the problem

of segregating pure tone from masked signal, that is co-modulation masking release (CMR).

These examinations showed the derivedstrategy of sound segregationcan be used tolead the

solution ofthe problems.

This strategy can contributetothe applications such asa preprocessor of the robust speech

recognition system and as a modeling of psychoacoustical phenomena. Moreover, it can also

contributetoanewconstructionmethodof the computationaltheory ofauditioninanalogy of

Marr's computational theory.

Key words: computational theory, auditoryscene analysis, the problem of segre-

gating two acoustic sources, Bregman's four regularities, mathematical constraint

c