博 士 学 位 論 文 Doctoral Dissertation
内容の要旨 及び 審査結果の要旨
Dissertation Abstract and
Summary of the Dissertation Review Result
第２６号
The Twenty-sixth Issue
平成２７年９月
September, 2015
The University of Aizu
はしがき
博士の学位を授与したので、学位規則（昭和２８年４月１日文部省令第９号）第８条の規定 に基づき、その論文の内容の要旨及び論文審査の結果の要旨をここに公表する。
学位記番号に付した｢甲｣は学位規則第４条第１項（いわゆる課程博士）によるものであるこ とを示す。
Preface
On granting the Doctoral Degree to the individuals mentioned below, abstracts of their theses and the theses review results are herewith publicly announced, in according to the provisions provided for in Article 8 of the Ruling of Degrees (Ministry Of Education Ordinance No.9, enacted on April 1, 1953)
The Chinese character, “甲”, at the beginning of the diploma number represents that an
individual has been granted the degree in accordance with the provisions provided for in
Paragraph 4-1 of the Ruling Of Degrees (what is called “Katei Hakase,” or the Doctoral
Degree granted by the University at which the grantee was enrolled.).
- 1 -
目 次 Contents
掲載順
Order
学位記番号 Diploma No.
学位
Degree
氏名
Name
論文題目
Dissertation Title
頁
Page
１ 甲
CI
博 第49
号博士（コンピュー タ理工学）
李 珍妮
LI, Zhenni
信号の疎表現のための過完備辞書の 効率的学習アルゴリズム
Efficient Learning Algorithms of Overcomplete Dictionaries for Sparse Representation of Signals
2
- 2 - Name
氏名
LI, Zhenni
李 珍妮（リ ゼンニ）
The relevant degree
学位の種類Doctoral degree (in Computer Science and Engineering)
博士（コンピュータ理工学）Number of the diploma of the Doctoral Degree
学位記番号甲
CI
博第49
号The Date of Conferment
学位授与日September 18, 2015
平成27
年9
月18
日Requirements for Degree Conferment
学位授与の要件
Please refer to the article five of “University Regulation on University Degrees”
会津大学学位規程 第５条該当
Dissertation Title
論文題目
Efficient Learning Algorithms of Overcomplete Dictionaries for Sparse Representation of Signals
信号の疎表現のための過完備辞書の効率的学習ア ルゴリズムDissertation Review Committee Members
論文審査委員University of Aizu, Prof. DING, S.
(Chief Referee)University of Aizu, Prof. HAYASHI, T.
University of Aizu, Prof. MORI, K.
University of Aizu,
Senior Associate Prof.MARKOV, K.
会津大学教授 丁 数学（主査）
会津大学教授 林 隆史 会津大学教授 森 和好
会津大学上級准教授 ｺﾝｽﾀﾝﾃｨﾝ ﾏﾙｺﾌ
- 3 -
Abstract
Overcomplete dictionaries have recently become the focus of a wide wealth of research in signal processing, machine learning, statistics and related fields. These great modelling flexibility allows to find sparse representations and approximations of data that is proved to be very efficient in a wide range of applications. Sparse models express signals as linear combinations of a few basis called atoms taken from a so-called dictionary. Finding the optimal dictionary from a set of training signals of a given class is the objective of dictionary learning and the main focus of this dissertation.
The first main contribution of the dissertation is developing nonnegative dictionary learning based on NMF framework for sparse representation of nonnegative signals, since signals and corresponding dictionary have nonnegativity limitations in some applications, e.g., multispectral data analysis. The objective function of dictionary learning has two parts: the approximation error and the sparsity regularizer. We propose to use ℓ1/2-norm as sparsity regularizer for nonnegative dictionary learning.
Furthermore, we propose to use coordinate descent strategy to update the dictionary and the coefficient matrix, which leads to an efficient algorithm of dictionary learning. The proposed algorithm can lead stronger sparsity than state-of-the-art algorithms using ℓ1-norm for sparsity.
The second main contribution is developing incoherent dictionary learning for sparse representation of general signals. We introduce the proximal operator to deal with the nonsmoothness of the constraints.
Specially, we extend the proximal operator and establish the threshold function of the coherence penalty. The methods of dictionary learning we advocate simply perform a decomposition scheme and an alternating optimization that can turn the whole problem into a set of minimizations of piecewise quadratic and univariate subproblems. Although each subproblem is still nonsmooth and even nonconvex, remarkably, it can be solved by the proximal operator and the closed-form solutions are obtained directly and explicitly. Hence, the proposed algorithms update the dictionary and the coefficient matrix in the same manner. The advantages in computational complexity and recovered atoms have been studied by the theoretical analysis and experimental analysis.
Summary of the Dissertation Review Result
The sparse representation is a new and emerging field that can provide many potential applications and its importance has recently received many attentions and become one of hot fields in
neuro-computing, statistical signal processing, image processing. The research topic of the candidate
is mainly on sparse representation of signal or image. The sparse representation has many applications
such as, machine learning, pattern recognition and classification, communication, data compression,
blind source separation, denoising, inpainting, super-resolution. In such applications, an effective
sparse representation requires a good dictionary, sometimes is also called pseudo-basis, which is
usually over complete, that is, there are much more words than the word dimensionality. The
candidate’s research topic is to learn such a dictionary. The candidate worked out several efficient
algorithms for this.
- 4 -
The first main contribution of the dissertation is having developed nonnegative dictionary learning based on NMF framework for the sparse representation of nonnegative signals, since signals and the corresponding dictionary have nonnegativity limitations in some applications, e.g., image processing, multispectral data analysis. The objective function of dictionary learning has two parts: the
approximation error and the sparsity regularization. For the sparsity regularization, the candidate proposed to use ℓ1/2-norm as sparsity constraint which can enforce a strong sparsity. Hence, an efficient algorithm of nonnegative dictionary learning has been developed based on the so called coordinate wise descent strategy for minimization of the cost function. Experiments show the good performance. When applied for denoising of images, comparing state-of-the-art algorithm, better SNR enhancements have been verified.
The second main contribution is having introduced a new concept, incoherence between words, into dictionary and is engaged in developing incoherent dictionary learning algorithms with different sparsity constraints for sparse representation of general signals. These are non-convex and non-smooth optimization problems, the traditional methods are not effective. For solving such a problem, the candidate performs a decomposition scheme and the alternating optimization, so that the problem can be transformed into a set of minimizations of piecewise quadratic and univariate subproblems.
Although each of subproblems is still nonsmooth and even nonconvex, remarkably, the candidate successfully extended the proximal operator to deal with the nonsmoothness and gave a closed-form optimal solution. Iterations of such subsolutions constructed the whole algorithm. The main
advantages of the proposed algorithm are that, as suggested by the analysis and simulation study, it has a lower computational complexity and a higher convergence rate than state-of-the-art algorithms.
Experiments show the good performance. When applied for denoising of images and noise
cancellation of audio signals, better SNR enhancements have been verified. This part has been written as a paper that has been published in Neural Computation, a major journal.
Related to these works, the candidate also presented in several major international conferences. The candidate shows strong knowledge and practical skill of topics related to the candidate’s research.
Dissertation writing and oral presentation are also excellent.
- 5 -
博 士 学 位 論 文
Doctoral Dissertation
内容の要旨 及び 審査結果の要旨
Dissertation Abstract
and
Summary of the Dissertation Review Result
第26号
The Twenty-sixth Issue
平成２７年９月
September, 2015
発行 会津大学
〒965-8580 福島県会津若松市一箕町鶴賀
