JJ II

(1)

volume 2, issue 1, article 5, 2001.

Received 12 April, 2000;

accepted 06 October 2000.

Communicated by:L.-E. Persson

Abstract Contents

JJ II

J I

Home Page Go Back

Close Quit

Journal of Inequalities in Pure and Applied Mathematics

FURTHER REVERSE RESULTS FOR JENSEN’S DISCRETE INEQUALITY AND APPLICATIONS IN INFORMATION THEORY

I. BUDIMIR, S.S. DRAGOMIR AND J.E. PE ˇCARI ´C

Department of Mathematics Faculty of Textile Technology University of Zagreb, CROATIA.

EMail:ivanb@zagreb.tekstil.hr

School of Communications and Informatics Victoria University of Technology

PO Box 14428, Melbourne City MC 8001 Victoria, AUSTRALIA.

EMail:sever.dragomir@vu.edu.au

URL:http://rgmia.vu.edu.au/SSDragomirWeb.html Department of Mathematics

Faculty of Textile Technology University of Zagreb, CROATIA.

EMail:pecaric@mahazu.hazu.hr

URL:http://mahazu.hazu.hr/DepMPCS/indexJP.html

c

2000School of Communications and Informatics,Victoria University of Technology ISSN (electronic): 1443-5756

007-00

(2)

Further Reverse Results for Jensen’s Discrete Inequality and Applications in Information

Theory

I. Budimir,S.S. Dragomirand J. Peˇcari´c

Title Page Contents

JJ II

J I

Go Back Close

Quit Page2of31

J. Ineq. Pure and Appl. Math. 2(1) Art. 5, 2001

http://jipam.vu.edu.au

Abstract

Some new inequalities which counterpart Jensen’s discrete inequality and improve the recent results from [4] and [5] are given. A related result for gener- alized means is established. Applications in Information Theory are also provided.

2000 Mathematics Subject Classification:26D15, 94Xxx.

Key words: Convex functions, Jensen’s Inequality, Entropy Mappings.

1. Introduction

Letf :X →Rbe a convex mapping defined on the linear spaceXandx_i ∈X, pi ≥0 (i= 1, ..., m)withPm :=Pm

i=1pi >0.

The following inequality is well known in the literature as Jensen’s inequality

(1.1) f 1

Pm m

X

i=1

p_ix_i

!

≤ 1 Pm

m

X

i=1

p_if(x_i).

There are many well known inequalities which are particular cases of Jensen’s inequality, such as the weighted arithmetic mean-geometric mean-harmonic mean inequality, the Ky-Fan inequality, the Hölder inequality, etc. For a com- prehensive list of recent results on Jensen’s inequality, see the book [25] and the papers [9]-[15] where further references are given.

In 1994, Dragomir and Ionescu [18] proved the following inequality which counterparts (1.1) for real mappings of a real variable.

Theorem 1.1. Letf :I ⊆R→Rbe a differentiable convex mapping on

◦

I (

◦

I is the interior of I), x_i ∈

◦

I, p_i ≥ 0 (i= 1, ..., n)and Pn

i=1p_i = 1. Then we have the inequality

0 ≤

n

X

i=1

pif(xi)−f

n

X

i=1

pixi

! (1.2)

≤

n

X

i=1

p_ix_if⁰(x_i)−

n

X

i=1

p_ix_i

n

X

i=1

p_if⁰(x_i),

(4)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page4of31

wheref⁰is the derivative off on

◦

I.

Using this result and the discrete version of the Grüss inequality for weighted sums, S.S. Dragomir obtained the following simple counterpart of Jensen’s inequality [5]:

Theorem 1.2. With the above assumptions forfand ifm, M ∈I^◦andm≤x_i ≤ M (i= 1, ..., n), then we have

(1.3) 0≤

n

X

i=1

p_if(x_i)−f

n

X

i=1

p_ix_i

!

≤ 1

4(M −m) (f⁰(M)−f⁰(m)). This was subsequently applied in Information Theory for Shannon’s and Rényi’s entropy.

In this paper we point out some other counterparts of Jensen’s inequality that are similar to (1.3), some of which are better than the above inequalities.

(5)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page5of31

2. Some New Counterparts for Jensen’s Discrete Inequality

The following result holds.

Theorem 2.1. Letf :I ⊆R→Rbe a differentiable convex mapping on

◦

I and x_i ∈I^◦ withx₁ ≤ x₂ ≤ · · · ≤ x_n andp_i ≥ 0 (i= 1, ..., n)withPn

i=1p_i = 1.

Then we have 0 ≤

n

X

i=1

p_if(x_i)−f

n

X

i=1

p_ix_i

! (2.1)

≤ (x_n−x₁) (f⁰(x_n)−f⁰(x₁)) max

1≤k≤n−1

P_kP¯_k+1

≤ 1

4(xn−x1) (f⁰(xn)−f⁰(x1)), whereP_k :=Pk

i=1p_iandP¯_k+1 := 1−P_k.

Proof. We use the following Grüss type inequality due to J. E. Peˇcari´c (see for example [25]):

(2.2)

1 Qn

n

X

i=1

q_ia_ib_i− 1 Qn

n

X

i=1

q_ia_i· 1 Qn

n

X

i=1

q_ib_i

≤ |a_n−a₁| |b_n−b₁| max

1≤k≤n−1

QkQ¯k+1

Q²_n

,

(6)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page6of31

provided that a, b are two monotonic n−tuples, q is a positive one, Q_n :=

Pn

i=1qi >0,Qk:=Pk

i=1qiandQ¯k+1 =Qn−Qk+1.

If in (2.2) we chooseq_i =p_i,a_i =x_i,b_i =f⁰(x_i)(anda_i, b_i will be monotonic nondecreasing), then we may state that

(2.3)

n

X

i=1

p_ix_if⁰(x_i)−

n

X

i=1

p_ix_i

n

X

i=1

p_if⁰(x_i)

≤(x_n−x₁) (f⁰(x_n)−f⁰(x₁)) max

1≤k≤n−1

P_kP¯_k+1 .

Now, using (1.2) and (2.3) we obtain the first inequality in (2.1).

For the second inequality, we observe that PkP¯k+1 =Pk(1−Pk)≤ 1

4(Pk+ 1−Pk)² = 1 4 for allk ∈ {1, ..., n−1}and then

1≤k≤n−1max

P_kP¯_k+1 ≤ 1 4, which proves the last part of (2.1).

Remark 2.1. It is obvious that the inequality (2.1) is an improvement of (1.3) if we assume that the order forx_iis as in the statement of Theorem2.1.

Another result is embodied in the following theorem.

(7)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page7of31

Theorem 2.2. Let f : I ⊆ R→R be a differentiable convex mapping on

◦

I andm, M ∈I^◦ withm ≤ x_i ≤ M (i= 1, ..., n)andp_i ≥ 0 (i= 1, ..., n) with Pn

i=1p_i = 1. IfS is a subset of the set{1, ..., n}minimizing the expression (2.4)

X

i∈S

p_i− 1 2 ,

then we have the inequality 0 ≤

n

X

i=1

p_if(x_i)−f

n

X

i=1

p_ix_i

! (2.5)

≤ Q(M−m) (f⁰(M)−f⁰(m))

≤ 1

4(M −m) (f⁰(M)−f⁰(m)), where

Q=X

i∈S

p_i 1−X

i∈S

p_i

! .

Proof. We use the following Grüss type inequality due the Andrica and Badea [2]:

(2.6)

Q_n

n

X

i=1

q_ia_ib_i−

n

X

i=1

q_ia_i ·

n

X

i=1

q_ib_i

≤(M₁−m₁) (M₂−m₂)X

i∈S

q_i Q_n−X

i∈S

q_i

!

(8)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page8of31

provided that m₁ ≤ a_i ≤ M₁, m₂ ≤ b_i ≤ M₂ for i = 1, ..., n, and S is the subset of{1, ..., n}which minimises the expression

X

i∈S

q_i −1 2Q_n

.

Choosingq_i =p_i,a_i =x_i, b_i =f⁰(x_i), then we may state that 0 ≤

n

X

i=1

pixif⁰(xi)−

n

X

i=1

pixi n

X

i=1

pif⁰(xi) (2.7)

≤ (M −m) (f⁰(M)−f⁰(m))X

i∈S

p_i 1−X

i∈S

p_i

! .

Now, using (1.2) and (2.7), we obtain the first inequality in (2.5). For the last part, we observe that

Q≤ 1 4

X

i∈S

p_i+ 1−X

i∈S

p_i

!2

= 1 4

and the theorem is thus proved.

The following inequality is well known in the literature as the arithmetic mean-geometric mean-harmonic-mean inequality:

(2.8) An(p, x)≥Gn(p, x)≥Hn(p, x),

(9)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page9of31

where

A_n(p, x) : =

n

X

i=1

p_ix_i - the arithmetic mean, G_n(p, x) : =

n

Y

i=1

x^p_iⁱ - the geometric mean, H_n(p, x) : = 1

n

P

i=1 pi

xi

- the harmonic mean,

andPn

i=1p_i = 1 p_i ≥0,i= 1, n .

Using the above two theorems, we are able to point out the following reverse of the AGH - inequality.

Proposition 2.3. Letx_i >0 (i= 1, ..., n)andp_i ≥0withPn

i=1p_i = 1.

(i) Ifx₁ ≤x₂ ≤ · · · ≤x_n−1 ≤x_n, then we have 1 ≤ A_n(p, x)

G_n(p, x) (2.9)

≤ exp

"

(x_n−x₁)²

x₁x_n max

1≤k≤n−1

P_kP¯_k+1

#

≤ exp

"

1

4 ·(xn−x1)² x₁x_n

# .

(10)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page10of31

(ii) If the set S ⊆ {1, ..., n} minimizes the expression (2.4), and 0 < m ≤ xi ≤M <∞(i= 1, ..., n), then

1 ≤ A_n(p, x) Gn(p, x) (2.10)

≤ exp

"

Q· (M −m)² mM

#

≤exp

"

1

4· (M −m)² mM

# .

The proof goes by the inequalities (2.1) and (2.5), choosing f(x) = −lnx.

A similar result can be stated forG_nandH_n.

Proposition 2.4. Letp≥1andx_i >0,p_i ≥0 (i= 1, ..., n)withPn

i=1p_i = 1.

(i) Ifx₁ ≤x₂ ≤ · · · ≤xn−1 ≤x_n, then we have 0 ≤

n

X

i=1

p_ix^p_i −

n

X

i=1

p_ix_i

!p

(2.11)

≤ p(x_n−x₁) x^p−1_n −x^p−1₁

1≤k≤n−1max

P_kP¯_k+1

≤ p

4(x_n−x₁) x^p−1_n −x^p−1₁ .

(ii) If the set S ⊆ {1, ..., n} minimizes the expression (2.4), and 0 < m ≤ x_i ≤M <∞(i= 1, ..., n), then

(2.12) 0≤

n

X

i=1

p_ix^p_i −

n

X

i=1

p_ix_i

!p

(11)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page11of31

≤ pQ(M −m) M^p−1−m^p−1

≤ 1

4p(M −m) M^p−1−m^p−1 .

Remark 2.2. The above results are improvements of the corresponding inequal- ities obtained in [5].

Remark 2.3. Similar inequalities can be stated if we choose other convex func- tions such as: f(x) = xlnx, x > 0 orf(x) = exp (x), x ∈ R. We omit the details.

(12)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page12of31

3. A Converse Inequality for Convex Mappings De- fined on R

ⁿ

In 1996, Dragomir and Goh [15] proved the following converse of Jensen’s inequality for convex mappings onRⁿ.

Theorem 3.1. Letf :Rⁿ →Rbe a differentiable convex mapping onRⁿand (∇f) (x) :=

∂f(x)

∂x¹ , ..., ∂f(x)

∂xⁿ

,

the vector of the partial derivatives,x= (x¹, ..., xⁿ)∈Rⁿ. Ifx_i ∈R^m(i= 1, ..., m),p_i ≥0, i= 1, ..., m,withP_m :=Pm

i=1p_i >0, then

0≤ 1 P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! (3.1)

≤ 1 P_m

m

X

i=1

p_ih∇f(x_i), x_ii −

* 1 P_m

m

X

i=1

p_i∇f(x_i), 1 P_m

m

X

i=1

p_ix_i +

.

The result was applied to different problems in Information Theory by providing different counterpart inequalities for Shannon’s entropy, conditional entropy, mutual information, conditional mutual information, etc.

For generalizations of (3.1) in Normed Spaces and other applications in In- formation Theory, see Mati´c’s Ph.D dissertation [23].

Recently, Dragomir [4] provided an upper bound for Jensen’s difference (3.2) ∆ (f, p, x) := 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! ,

(13)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page13of31

which, even though it is not as sharp as (3.1), provides a simpler way, and for applications, a better way, of estimating the Jensen’s differences ∆. His result is embodied in the following theorem.

Theorem 3.2. Letf :Rⁿ→Rbe a differentiable convex mapping andx_i ∈Rⁿ, i= 1, ..., m. Suppose that there exists the vectorsφ,Φ∈Rⁿsuch that

(3.3) φ ≤xi ≤Φ (the order is considered on the co-ordinates) andm, M ∈Rⁿare such that

(3.4) m≤ ∇f(x_i)≤M

for alli∈ {1, ..., m}. Then for allp_i ≥0 (i= 1, ..., m)withP_m >0, we have the inequality

(3.5) 0≤ 1 P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

!

≤ 1

4kΦ−φk kM −mk, wherek·kis the usual Euclidean norm onRⁿ.

He applied this inequality to obtain different upper bounds for Shannon’s and Rényi’s entropies.

In this section, we point out another counterpart for Jensen’s difference, as- suming that the∇−operator is of Hölder’s type, as follows.

Theorem 3.3. Letf :Rⁿ→Rbe a differentiable convex mapping andx_i ∈Rⁿ, p_i ≥ 0 (i= 1, ..., m) with P_m > 0. Suppose that the ∇−operator satisfies a condition ofr−H−Hölder type, i.e.,

(3.6) k∇f(x)− ∇f(y)k ≤Hkx−yk^r, for allx, y ∈Rⁿ,

(14)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page14of31

whereH >0,r ∈(0,1]andk·kis the Euclidean norm.

Then we have the inequality:

0 ≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! (3.7)

≤ H P_m²

X

1≤i<j≤m

p_ip_jkx_i−x_jk^r+1.

Proof. We recall Korkine’s identity, 1

P_m

m

X

i=1

p_ihy_i, x_ii −

* 1 P_m

m

X

i=1

p_iy_i, 1 P_m

m

X

i=1

p_ix_i +

= 1 2P_m²

n

X

i,j=1

pipjhyi−yj, xi−xji, x, y ∈Rⁿ,

and simply write 1

P_m

m

X

i=1

p_ih∇f(x_i), x_ii −

* 1 P_m

m

X

i=1

p_i∇f(x_i), 1 P_m

m

X

i=1

p_ix_i +

= 1 2P_m²

n

X

i,j=1

p_ip_jh∇f(x_i)− ∇f(x_j), x_i−x_ji.

(15)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page15of31

Using (3.1) and the properties of the modulus, we have

0 ≤ 1

P_m

m

X

i=1

pif(xi)−f 1 P_m

m

X

i=1

pixi

!

≤ 1 2P_m²

m

X

i,j=1

p_ip_j|h∇f(x_i)− ∇f(x_j), x_i−x_ji|

≤ 1 2P_m²

m

X

i,j=1

p_ip_jk∇f(x_i)− ∇f(x_j)k kx_i−x_jk

≤ H P_m²

m

X

i,j=1

p_ip_jkx_i−x_jk^r+1

and the inequality (3.7) is proved.

Corollary 3.4. With the assumptions of Theorem3.3and if

∆ = max1≤i<j≤mkxi−xjk, then we have the inequality

(3.8) 0≤ 1 P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

!

≤ H∆^r+1 2P_m² 1−

m

X

i=1

p²_i

! .

Proof. Indeed, as X

1≤i<j≤m

pipjkxi−xjk^r+1 ≤∆^r+1 X

1≤i<j≤m

pipj.

(16)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page16of31

However, X

1≤i<j≤m

p_ip_j = 1 2

m

X

i,j=1

p_ip_j −X

i=j

p_ip_j

!

= 1 2 1−

m

X

i=1

p²_i

! ,

and the inequality (3.8) is proved.

The case of Lipschitzian mappings is embodied in the following corollary.

Corollary 3.5. Let f : Rⁿ → R be a differentiable convex mapping and xi ∈ Rⁿ, pi ≥ 0 (i= 1, ..., n) with Pm > 0. Suppose that the ∇−operator is Lipschitzian with the constantL >0, i.e.,

(3.9) k∇f(x)− ∇f(y)k ≤Lkx−yk, for allx, y ∈Rⁿ, wherek·kis the Euclidean norm. Then

0 ≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! (3.10)

≤ L



 1 P_m

m

X

i=1

p_ikx_ik²−

1 P_m

m

X

i=1

p_ix_i

2

.

Proof. The argument is obvious by Theorem 3.3, taking into account that for r = 1,

X

1≤i<j≤m

pipjkxi−xjk² =Pm m

X

i=1

pikxik²−

m

X

i=1

pixi

2

,

andk·kis the Euclidean norm.

(17)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page17of31

Moreover, if we assume more about the vectors (x_i)_i=1,n, we can obtain a simpler result that is similar to the one in [4].

Corollary 3.6. Assume thatf is as in Corollary3.5. If

(3.11) φ≤x_i ≤Φ (on the co-ordinates),φ,Φ∈Rⁿ (i= 1, .., m), then we have the inequality

0 ≤ 1

P_m

m

X

i=1

p_if(x_i)−f 1 P_m

m

X

i=1

p_ix_i

! (3.12)

≤ 1

4·L· kΦ−φk².

Proof. It follows by the fact that in Rⁿ, we have the following Grüss type inequality (as proved in [4])

(3.13) 1

P_m

m

X

i=1

p_ikx_ik²−

1 P_m

m

X

i=1

p_ix_i

2

≤ 1

4kΦ−φk², provided that (3.11) holds.

Remark 3.1. For some Grüss type inequalities in Inner Product Spaces, see [7].

(18)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page18of31

4. Some Related Results

Start with the following definitions from [3].

Definition 4.1. Let −∞ < a < b < ∞. ThenCM[a, b]denotes the set of all functions with domain[a, b]that are continuous and strictly monotonic there.

Definition 4.2. Let −∞ < a < b < ∞, and let f ∈ CM[a, b]. Then, for each positive integer n, each n−tuple x = (x₁, ..., x_n), where a ≤ x_j ≤ b (j = 1,2, ..., n), and eachn-tuplep= (p₁, p₂, ..., p_n),wherep_j >0 (j = 1,2, ..., n) andPn

j=1p_j = 1, letM_f(x, y)denote the (weighted) mean f⁻¹

( _n X

j=1

p_jf(x_j) )

.

We may state now the following result.

Theorem 4.1. Let Sbe the subset of{1, ..., n}which minimizes the expression

P

i∈Sp_i−1/2

. Iff, g∈CM[a, b], then sup

x

{|M_f(x, p)−M_g(x, p)|} ≤Q·

f⁻¹⁰ _∞·

f ◦g⁻¹⁰⁰

_∞·|g(b)−g(a)|², provided that the right-hand side of the inequality is finite, where, as above,

Q= X

i∈S

pi

!

1−X

i∈S

pi

! , andk·k_∞is the usual sup-norm.

(19)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page19of31

Proof. Let, as in [3],h=f ◦g⁻¹,n >1,

x= (x₁, x₂, ..., x_n) andp= (p₁, p₂, ..., p_n)

be as in the Definition4.2, andy_j =g(x_j) (j = 1,2, ..., n). By the mean-value theorem, for someαin the open interval joiningf(a)tof(b), we have

M_f(x, p)−M_g(x, p) = f⁻¹ ( _n

X

j=1

p_jf(x_j) )

−f⁻¹

"

h ( _n

X

j=1

p_jg(x_j) )#

= f⁻¹0

(α)

" _n X

j=1

p_jf(x_j)−h ( _n

X

j=1

p_jg(x_j) )#

= f⁻¹⁰ (α)

" _n X

j=1

p_jh(y_j)−h ( _n

X

j=1

p_jy_j )#

= f⁻¹0

(α)

" _n X

j=1

p_j (

h(y_j)−h

n

X

k=1

p_ky_k

!)#

.

Using the mean-value theorem a second time, we conclude that there exists pointsz₁, z₂, ..., z_nin the open interval joiningg(a)tog(b), such that

M_f(x, p)−M_g(x, p)

= f⁻¹0

(α)

p₁{(1−p₁)y₁−p₂y₂− · · · −p_ny_n}h⁰(z₁) +p₂{−p₁y₁+ (1−p₂)y₂− · · · −p_ny_n}h⁰(z₂)

+· · ·

+p_n{−p₁y₁−p₂y₂− · · ·+ (1−p_n)y_n}h⁰(z_n)

(20)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page20of31

= f⁻¹0

(α)

p₁{p₂(y₁−y₂) +· · ·+p_n(y₁−y_n)}h⁰(z₁) +p₂{p₁(y₂−y₁) +· · ·+p_n(y₂−y_n)}h⁰(z₂)

+· · ·

+pn{p1(yn−y1) +· · ·+pn−1(yn−yn−1)}h⁰(zn)

= f⁻¹0

(α) X

1≤i<j≤n

pipj(yi−yj){h⁰(zi)−h⁰(zj)}.

Using the mean value theorem a third time, we conclude that there exists points ω_ij (1≤i < j ≤n)in the open interval joiningg(a)tog(b), such that

f⁻¹0

(α) X

1≤i<j≤n

pipj(yi−yj){h⁰(zi)−h⁰(zj)}

= f⁻¹⁰

(α) X

1≤i<j≤n

p_ip_j(y_i−y_j) (z_i−z_j)h⁰⁰(ω_ij).

Consequently,

|Mf(x, p)−Mg(x, p)|

≤ f⁻¹0

(α)

X

1≤i<j≤n

≤

f⁻¹0

_∞· kh⁰⁰k_∞· X

1≤i<j≤n

p_ip_j|y_i−y_j| · |z_i−z_j|

≤(by the Cauchy-Buniakowski-Schwartz inequality)

(21)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page21of31

≤

f⁻¹0 _∞·

f◦g⁻¹00 _∞·

s X

1≤i<j≤n

p_ip_j|y_i−y_j|²

·

s X

1≤i<j≤n

p_ip_j|z_i−z_j|²

≤ (by the Andrica and Badea result)

≤

f⁻¹0 _∞·

f◦g⁻¹00 _∞·

v u u t

X

i∈S

pi

!

1−X

i∈S

pi

!

|g(b)−g(a)|²

· v u u t

X

i∈S

p_i

!

1−X

i∈S

p_i

!

|g(b)−g(a)|²

= Q

f⁻¹0 _∞·

f◦g⁻¹00

_∞· |g(b)−g(a)|², and the theorem is proved.

Corollary 4.2. Iff, g ∈CM[a, b], then sup

x

{|M_f(x, p)−M_g(x, p)|} ≤Q·

1 f⁰

_∞

·

1 g⁰

f⁰ g⁰

0 _∞

· |g(b)−g(a)|², provided that the right hand side of the inequality exists.

Proof. This follows at once from the fact that f⁻¹0

= 1

f⁰◦f⁻¹

(22)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page22of31

and

f◦g⁻¹⁰⁰

= (g⁰◦g⁻¹) (f⁰⁰◦g⁻¹)−(f⁰ ◦g⁻¹) (g⁰⁰◦g⁻¹) (g⁰ ◦g⁻¹)³ =

1 g⁰

f⁰ g⁰

0

◦g⁻¹.

Remark 4.1. This establishes Theorem 4.3 from [3] and replaces the multiplica- tive factor ¹₄ byQ. In Corollary4.2, we also replaced the multiplicative factor

1 4 byQ.

(23)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page23of31

5. Applications in Information Theory

We give some new applications for Shannon’s entropy H_b(X) :=

r

X

i=1

p_ilog_b 1 p_i,

whereXis a random variable with the probability distribution(p_i)_i=1,r.

Theorem 5.1. Let X be as above and assume that p1 ≥ p2 ≥ · · · ≥ pr or p₁ ≤p₂ ≤ · · · ≤p_r. Then we have the inequality

(5.1) 0≤log_br−H_b(X)≤ (p₁−p_r)² p₁p_r max

1≤k≤r

P_kP¯_k+1 . Proof. We choose in Theorem2.1,f(x) =−log_bx,x >0,x_i = _p¹

i (i= 1, ..., r).

Then we havex₁ ≤x₂ ≤ · · · ≤x_rand by (2.1) we obtain 0≤log_br−H_b(X)≤

1 p_r − 1

p₁

1

−_p¹

r

+ 1

1 p1

!

1≤k≤rmax

P_kP¯_k+1 ,

which is equivalent to (5.1). The same inequality is obtained ifp₁ ≤p₂ ≤ · · · ≤ p_r.

Theorem 5.2. LetX be as above and suppose that p_M : = max{p_i|i= 1, ..., r},

pm : = min{pi|i= 1, ..., r}.

(24)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page24of31

IfS is a subset of the set{1, ..., r}minimizing the expression

P

i∈Sp_i−1/2 , then we have the estimation

(5.2) 0≤log_br−H_b(X)≤Q· (p_M −p_m)² lnb·p_Mp_m. Proof. We shall choose in Theorem2.2,

f(x) =−log_bx, x > 0, x_i = 1 pi

i= 1, r .

Thenm= _p¹

M,M = _p¹

m,f⁰(x) = −_x_lnb¹ and the inequality (2.3) becomes:

0 ≤ log_br−

r

X

i=1

p_ilog_b 1 p_i

≤ Q 1 lnb

1 p_m − 1

p_M

− 1

1 pm

+ 1

1 pM

!

= Q· 1

lnb · (p_M −p_m)² pMpm

,

hence the estimation (5.2) is proved.

Consider the Shannon entropy

(5.3) H(X) :=H_e(X) =

r

X

i=1

p_iln 1 p_i

(25)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page25of31

and Rényi’s entropy of orderα(α∈(0,∞)\ {1})

(5.4) H_[α](X) := 1

1−αln

r

X

i=1

p^α_i

! .

Using the classical Jensen’s discrete inequality for convex mappings, i.e.,

(5.5) f

r

X

i=1

p_ix_i

!

≤

r

X

i=1

p_if(x_i),

where f : I ⊆ R→R is a convex mapping on I, x_i ∈ I (i= 1, ..., r) and (p_i)_i=1,r is a probability distribution, for the convex mapping f(x) = −lnx, we have

(5.6) ln

r

X

i=1

p_ix_i

!

≥

r

X

i=1

p_ilnx_i.

Choosex_i =p^α−1_i (i= 1, ..., r)in (5.6) to obtain ln

r

X

i=1

p^α_i

!

≥(α−1)

r

X

i=1

p_ilnp_i,

which is equivalent to

(1−α)

H_[α](X)−H(X)

≥0.

(26)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page26of31

Now, ifα ∈ (0,1),thenH_[α](X) ≤H(X),and ifα >1thenH_[α](X)≥ H(X).Equality holds iff(pi)_i=1,ris a uniform distribution and this fact follows by the strict convexity of−ln (·).This inequality also follows as a special case of the following well known fact: H_[α](X)is a nondecreasing function of α.

See for example [26] or [22].

Theorem 5.3. Under the above assumptions, given that p_m = min_i=1,rp_i, p_M = max_i=1,rp_i, then we have the inequality

(5.7) 0≤(1−α)

H_[α](X)−H(X)

≤Q· p^α−1_M −p^α−1_m 2

p^α−1_M p^α−1_m , for allα∈(0,1)∪(1,∞).

Proof. Ifα∈(0,1), then

xi :=p^α−1_i ∈

p^α−1_M , p^α−1_m and ifα∈(1,∞), then

x_i =p^α−1_i ∈

p^α−1_m , p^α−1_M

, fori∈ {1, ..., n}.

Applying Theorem 2.2 for x_i := p^α−1_i and f(x) = −lnx, and taking into account thatf⁰(x) =−¹_x, we obtain

(1−α)

H_[α](X)−H(X)

≤











Q p^α−1_m −p^α−1_M

− ¹

p^α−1m + ¹

p^α−1_M

if α∈(0,1), Q p^α−1_M −p^α−1_m

− ¹

p^α−1_M + ¹

p^α−1m

if α∈(1,∞)

(27)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page27of31

=











Q· (^p^α−1^m ^−p^α−1M )²

p^α−1m p^α−1_M if α∈(0,1), Q· (^p^α−1M −p^α−1m )²

p^α−1_M p^α−1m if α∈(1,∞)

=Q· p^α−1_M −p^α−1_m 2

p^α−1_M p^α−1_m

for allα∈(0,1)∪(1,∞)and the theorem is proved.

Using a similar argument to the one in Theorem5.3, we can state the following direct application of Theorem2.2.

Theorem 5.4. Let(p_i)_i=1,r be as in Theorem5.3. Then we have the inequality (5.8) 0≤(1−α)H_[α](X)−lnr−αlnG_r(p)≤Q· p^α−1_M −p^α−1_m 2

P_M^α−1p^α−1_m ,

for allα∈(0,1)∪(1,∞).

Remark 5.1. The above results improve the corresponding results from [5] and [4] with the constantQwhich is less than ¹₄.

Acknowledgement 1. The authors would like to thank the anonymous referee for valuable comments and for the references [26] and [22].

(28)

Theory

Title Page Contents

JJ II

J I

Go Back Close

Quit Page28of31

References

[1] A. RÉNYI, On measures of entropy and information, Proc. Fourth Berkley Symp. Math. Statist. Prob., 1 (1961), 547–561, Univ. of California Press, Berkley.

[2] D. ANDRICAANDC. BADEA, Grüss’ inequality for positive linear func- tionals, Periodica Math. Hung., 19(2) (1988), 155–167.

[3] G.T. CARGO AND O. SHISHA, A metric space connected with general means, J. Approx. Th., 2 (1969), 207–222.

[4] S.S. DRAGOMIR, A converse of the Jensen inequality for convex mappings of several variables and applications. (Electronic Preprint:

http://matilda.vu.edu.au/~rgmia/InfTheory/

ConverseJensen.dvi)

[5] S.S. DRAGOMIR, A converse result for Jensen’s discrete inequality via Grüss’ inequality and applications in information theory, Analele Univ.

Oradea, Fasc. Math., 7 (1999-2000), 178–189.

[6] S.S. DRAGOMIR, A further improvement of Jensen’s inequality, Tamkang J. Math., 25(1) (1994), 29–36.

[7] S.S. DRAGOMIR, A generalisation of Grüss’s inequality in inner product spaces and applications, J. Math. Anal. Appl., 237 (1999), 74–82.

[8] S.S. DRAGOMIR, A new improvement of Jensen’s inequality, Indian J.

Pure Appl. Math., 26(10) (1995), 959–968.