Japan Advanced Institute of Science and Technology

Japan Advanced Institute of Science and Technology

JAIST Repository

https://dspace.jaist.ac.jp/

Title

Author(s)

Citation

Issue Date

Type

Text version

URL

Rights

Description

of the Minimum Classication Error Learning

Jun Rokui

School of Information Science ,Shimodaira laboratory,

Japan Advanced Institute of Science and Technology

February 13, 1998

Keywords: minimum classicationerror, generalization, a penalty term ,the objective

function,overlearning.

1 Introduction

Some research tosearch for the most suitable data on recognition model has been stud-

ied. To classify exactlya lot of data in this eld of study is in great demand.Until now,

Maximum Likeliho od extimation is widely used. However,now more highly ecientdis-

criminativelearning isdemanded. SoMinimumClassisicationError hasappeared froma

dierentp ointof learning. In this paper, wetakeupthe problem ofMinimum Classica-

tion Error whichis a highly ecient discriminative learning, and presenta solution.

2 Minimum Classication Error

Minimum ClassicationError is highly ecient discriminative learning published as the

result of Katagiri's and Juang's research work in 1992. All data are learned on MCE,

and every parameters are ajusted to a minimum errors in recognition. This dier from

methodwhichassumesclass formuntilnow,isa methodwhichassumesclass b oundaries

to minimizeclassication error.

MCE givesa good recognition ability for training data, but do esa loweringof recog-

nition ability for generated data. this cause is no information for data distribution in

a learning argorithm of MCE. In this paper, we put information for generalization in a

learning argorithm of MCE.So we expect improvement in recognition ability, and study

possibility.

Copyrightc 1998byJunRokui

We have to deal with questions of generalization separately,on a case-by-case basis. We

dene asgeneralization ability which can recognize data addedtrainigdata changeas it.

Inthispaper,wedeneasaxedindex forgenaralizationabilitytheratioofinputchange

to output change. If we could keep output for input as little as p ossible,generalization

ability would improve. In this pap er, we dene as the new objective function with a

penalty term of a xedindex for generalization ability.

4 Proposal

4.1 Abstract

In this pap er, we introduce a derived function of input and output a xed index for

generalization ability to a learning argorithm of MCE. We dene as the penalty term

a xed index prop osed, and add it the objective function for MCE. This proposal is a

learning argorithm whichminimizes ap enalty term and a objectivefunction.

4.2 MCE Multi-layer Feed-forward Neural Networks

MCE can be appliedto many new classier structure suchas the multilayer perceptron,

learning vectorquantizer. In this pap er,weuse multilayerperceptron. Becausethere are

researches in a area of multilayer perceptron which added a penalty term the objective

function, and some of them are conrmed available.

In this pap er, multi-layer perceptron typ e neural network is employed as a platform

of recognizer toevaluatethe performance.

4.3 Preparation and Experiments

MCEhas aprobremthathasastronginuence forinitialvaluebecauseMCEisamethod

which assumes class boundaries to minimize classication error. To decide suitable ini-

tial value is very important. So it is necessary to decide suitable initial value to give

rough class form before exp erimentation. In this paper,we use back propagetion before

experimentationfor MCE and proposed method.

Twotyp es data, artical data and real world data, were used to evaluatethe perfor-

mance of the learning algorithms.

5 Conclusion

In this paper,wemakea comparativestudy of NN(MSE),MCE, and modication MCE

forarticial data. Asaresult, modicationMCEwasthe bestrecognition ratefor gener-

ateddataof all. Incase oftheexperimentsonreal-worlddata,databasesinUCIMachine

cation MCE was the best recognition rate for generated data of all. Modication MCE

goes better to improving generalization ability by dominateing to decide the sharpness

boundary as apurposeof MCE learning.

The parameter of a sigmoid function as a loss function inuences the generalization

ability. So we have experimented in an inuence of changing the papameter. In case of

the experiment on generated data,mo dication MCE was better than MCE, or as well

as it. As a result, we gave the case the prop osed metho d was available regardless of an

inuence of the parameter.

In this paper, we proposed a novel approach to improve the generalization ability of

MCE. The prop osed method giving a solution to essentially probrem of MCE can be

expected to eects onvariousmetho ds using MCE.