Statement of the problem

TO THE PROBLEM OF A STRONG DIFFERENTIABILITY OF INTEGRALS ALONG DIFFERENT DIRECTIONS

G. LEPSVERIDZE

Abstract. It is proved that for any given sequence (σn, n ∈N) = Γ0⊂Γ, where Γ is the set of all directions inR²(i.e., pairs of ortho- gonal straight lines) there exists a locally integrable functionfonR² such that: (1) for almost all directions σ ∈ Γ\Γ0 the integral R

f is differentiable with respect to the family B2σ of open rectangles with sides parallel to the straight lines fromσ; (2) for every direction σn∈Γ0the upper derivative ofR

fwith respect toB2σnequals +∞; (3) for every directionσ∈Γ the upper derivative ofR

|f|with respect toB2σ equals +∞.

§ 1. Statement of the problem. Formulation of the main result

Let B(x) be a differentation basis at the point x ∈ Rⁿ (see [1]). The family{B(x) :x∈Rⁿ}is called a differentiation basis inRⁿ.

For f ∈ Lloc(Rⁿ) and x ∈ Rⁿ let us denote respectively by DB(f)(x) and D_B(f)(x) the upper and the lower derivative of the integralR

f with respect toB at x[1]. When these two derivatives are equal their common value is denoted byDB(f)(x) and the basisB is said to differentiateR

f if the relationDB(f)(x) =f(x) holds almost everywhere.

LetB2 denote the differentiation basis in Rⁿ consisting of all n-dimen- sional open intervals, and B2(x) be the family of sets fromB2 containing x.

Letσbe the union ofnmutually orthogonal straight lines inRⁿ (n≥2) which intersect at the origin. The set of such unions will be denoted by Γ(Rⁿ). Elements of this set will be called directions. Note that Γ(R²) corresponds in the one-to-one manner to the interval [0,^π₂) (see [2]).

For a fixed directionσwe denote byB2σthe differentiation basis in Γ(Rⁿ) which is formed by alln-dimensional open rectangles with the sides parallel

1991Mathematics Subject Classification. 28A15.

Key words and phrases. Strong derivative of an integral, field of directions, negative result in the theory of differentiation of integrals.

157

1072-947X/98/0300-0157$15.00/0 c1998 Plenum Publishing Corporation

to the straight lines fromσ. IfB2σ differentiatesR

f atx, then the integral Rf is said to be strongly differentiable with respect to σatx.

The following problem was proposed by Zygmund (see [1], Ch. IV): Given a function f ∈ L(R²), is it possible to choose a direction σ such that R

f would be strongly differentiable with respect toσ?

Let W(Rⁿ) (n ≥ 2) denote a class of locally integrable functions on Rⁿ whose strong upper derivatives DB2σ(f)(x) are equal to +∞ almost everywhere along each fixed directionσ. When solving Zygmund’s problem, Marstrand [3] showed that the classW(R²) is not empty, and thus his answer to the above stated problem was negative. A stronger result was obtained by L´opez Melero [4] and Stokolos [5].

In connection with Zygmund’s problem we had the following question [2]: Given a pair of directions σ1 and σ2 differing from each other, does there exist an integrable function f such that the integral R

f is strongly differentiable a.e. with respect toσ1and strongly differentiable with respect toσ2on the null set only? Theorems 1 and 2 from [2] give a positive answer to this question.

It is known ([1], Ch. III) that if R

|f| is strongly differentiable almost everywhere, then the same holds for R

f. Papoulis [6] showed that the converse proposition does not hold in general. Namely, there exists an inte- grable functionf onR² such that the integralR

f is strongly differentiable almost everywhere, whileR

|f|is strongly differentiable on the null set only.

Zerekidze [7] has obtained a stronger result from which it follows that for every function f from W(Rⁿ) there exists a measurable function g such that|f|=|g| andR

g is strongly differentiable almost everywhere along all directions. In other words, changing the sign of the function on some set, we can improve the differentiation properties of the integral in all directions.

There arises a question whether the following alternative holds: Given function f from W(R²), can the differentiation properties of the integral Rf after changing the sign of the function be improved in all directions or they do not improve in none of them?

The following theorem gives a negative answer to this question and strengthens the results of Papoulis [6] and Marstrand [3].

Theorem. Let the sequence of directions(σn)^∞_n=1 be given. There exists a locally integrable functionf onR² such that:

(1)for almost all directions σ (σ6=σn,n∈N), DB2σ(f)(x) =f(x) a.e.;

(2)for every directionσn (n∈N),

DB2σn(f)(x) = +∞ a.e.;

(3)for every directionσ,

DB2σ(|f|)(x) = +∞ a.e.

Remark. If the sequence (σn)^∞_n=1consists of a finite number of directions, then in item (1) instead of ”for almost all directions” it should be written

“for all directions”.

Corollary. There is a functionf ∈Lloc(R²)such that:

(a) the integral R

|f| is strongly differentiable a.e. in none of the direc- tions;

(b)for almost all irrational directions the integral R

f is strongly differ- entiable a.e., while for the rational directions it is strongly differentiable on the null set only.

§2. Auxiliary Assertions. Proof of the Main Result Before passing to the formulation of auxiliary assertions let us introduce some notation and definitions.

For the setG,G⊂R², ∂G is assumed to be the boundary of the setG andGits closure. ByE we denote the unit square inR².

Given a natural number n, let us construct two collections of straight lines: x=en⁻¹and y=en⁻¹,e= 0,1, . . . , n, which define the rectangular net Eⁿ in the unit squareE and divide it into open square intervals E_kⁿ, k= 1,2, . . . , n², with sides of lengthn⁻¹.

For the rectifiable curvec denote byd(c) its length.

The set of measurable functions onRⁿ taking only the values−1 and 1 will be denoted byS(Rⁿ).

For the measurable set G, G⊂R², the number λ, 0< λ < 1, and the direction σ, denote byH^σ(χ_G, λ) the union of all those open rectangles R fromB2σ for which

|R|⁻¹ Z

R

χG(y)dy≥λ,

where χG is the characteristic function of the set G. If, moreover, σ is a standard direction, then the setH^σ(χG, λ) will be denoted byH(χG, λ).

Furthermore, for the intervalI= (0, e1)×(0, e2) and the numbersλand c (0< λ <1, 1< c <∞) we define the intervalQ(I, λ, c) as follows:

Q(I, λ, c) =‚

−cλ⁻¹e1,(1 +cλ⁻¹)e1

ƒ×‚

−e2,2e2

ƒ.

Letσbe a fixed direction and letf ∈Lloc(Rⁿ). In the present work we consider the following maximal Hardy–Littlewood functions:

MB2σf(x) = sup

R∈B2σ(x)|R|⁻¹ Z

R

|f(y)|dy,

M_B^∗_2σf(x) = sup

R∈B2σ(x)|R|⁻¹

ŒŒ

ΠZ

R

f(y)dy

ŒŒ

Œ.

The validity of the following two assertions can be easily verified.

Lemma 1. Let 0 < ε <1 and nε= 9ε⁻¹. Let, moreover,c be a conti- nuous rectifiable curve in the unit squareEand letd(c)<1. Then for every natural number n,n≥nε, the following relation holds:

ŒŒ

Œ ∪

k∈τn

E_kⁿŒŒŒ≤ε,

where τn is a collection of those natural numbers for which the square E_kⁿ from the rectangular netEⁿ intersects with the curvec.

Lemma 2. Letσ1 be a fixed direction fromΓ(R²). LetI^σ1 be a rectangle fromB2σ1 and B be a circle(on the plane). Then:

(1) ŒŒH^σ1(χ_Iσ1, λ)ŒŒ> λ⁻¹ln(λ⁻¹)|I^σ1|, 0< λ <1;

(2)for every directionσ,

ŒŒH^σ(χB, λ)ŒŒ> λ⁻¹ln(λ⁻¹)|B|, 0< λ <2⁻¹.

Lemma 3 (Zerekidze [7]). Let ε > 0. There exists a function s ∈ S(Rⁿ)such that

ŒŒ

ΠZ

γ

s(x)d猌Œ< ε,

whereγis an arbitrary interval inRⁿ anddηis the Lebesgue linear measure onγ.

Lemma 4 ([2]). Let I= (0, e1)×(0, e2)and letσ be an arbitrary non- standard direction fromΓ(R²). There exists a number c(σ),1< c(σ)<∞, such that for everyλ,0< λ <1,

H^σ(χ_I, λ)⊂Q(I, λ, c(σ)).

If, moreover,c(σ)λ⁻¹e1≤e2, then

ŒŒH^σ(χI, λ)ŒŒ≤9c(σ)λ⁻¹|I|.

Proof of the theorem. The proof of the theorem is divided into several parts.

1^◦. We define some auxiliary sets.

For any naturaln≥2 denote Γⁿ=ˆ

σ∈Γ(R²) : 0≤α(σ)<2⁻⁽ⁿ⁺¹⁾·n⁻¹‰

∪

∪ˆ

σ∈Γ(R²) :π2⁻¹−2⁻⁽ⁿ⁺¹⁾·n⁻¹< α(σ)< π2⁻¹‰ ,¹ cn= supˆ

c(σ) :σ∈Γ(R²)\Γⁿ‰ , βn= max

š

exp(cnn²2²ⁿ); 2 ⁿX⁻¹

n1=2

n1βn1+

n−1

X

n1=2

λn1

‘›, (1)

λn= 2 nβn+

nX−1 n1=2

(n1βn1+λn1)‘

. (2)

LetIⁿ = (0, eⁿ₁)×(0, eⁿ₂) be the interval for which eⁿ₂ =cnn2ⁿβneⁿ₁. Assume

Qⁿ =Q(Iⁿ,(n2ⁿβn)⁻¹, cn).

Denote by Q^∗n the interval with the same center of symmetry as Qⁿ but with edges four times larger. Next, for e = 1,2, . . . , n denote by I_eⁿ, Qⁿ_e, andQ^∗_eⁿ those rectangles fromB2σe which are obtained from the intervals Iⁿ, Qⁿ, and Q^∗n by rotation with respect to the center of symmetry (the centers of symmetry of the intervalsIⁿ,Qⁿ, andQ^∗n coincide).

Assume

H_eⁿ=H^σe(χ_In

e, β_n⁻¹)∪Q^∗_eⁿ, e= 1,2, . . . , n, a2= 1/2, an=rn−1(M_mⁿ⁻_n−¹₁)⁻¹, n >2.

The sets H_eⁿ (e = 1,2, . . . , n) are compact. We use Lemma 1.3 from [1] and cover almost the whole unit square E by the sequence of non- intersecting sets H_1jⁿ, j = 1,2, . . ., homothetic to H₁ⁿ such that all sets H_1jⁿ are contained in the unit square and have a diameter less thanan. By applying a similar treatment to the setsH_eⁿ,e= 2, . . . , n, we obtain

ŒŒ

ŒE\ ^∞∪

j=1H_ejⁿŒŒŒ= 0, diam(H_ejⁿ)≤an, e= 1,2, . . . , n, j= 1,2, . . . . (3) LetP_ejⁿ (e= 1,2, . . . , n,j = 1,2, . . .) denote the homothety transforming the setH_eⁿ toH_ejⁿ. Assume

I_ejⁿ =P_ejⁿ(I_eⁿ), Qⁿ_ej=P_ejⁿ(Qⁿ_e) Q^∗_ejⁿ=P_ejⁿ(Q^∗_eⁿ).

1For the direction σ, the number 0 ≤α < ^π₂ is defined as the angle between the positive direction of the axisoxand the straight line fromσlying in the first quadrant of the plane.

Denote bybn the circle with center at the point (2⁻¹,2⁻¹) and of radius rn, 0< rn<1. Choose a numberrnso small that the conditionsrn< rn−1

andH(χ_bn, λ⁻_n¹)⊂Eare fulfilled.

For the direction σ denote the set H^σ(χ_bn, λ⁻_n¹) by hⁿ(σ) and for the standard direction use the notation hⁿ =H(χ_bn, λ⁻_n¹). Let mn be a fixed natural number satisfying the condition

1≤mn|hⁿ| ≤2. (4)

Let M₁ⁿ, M₂ⁿ, . . . , M_mⁿ_n be a collection of natural numbers such that M₁ⁿ < M₂ⁿ < · · · < M_mⁿ_n. Let us consider the rectangular nets E^M1n, E^M2n, . . . , E^Mmnn . Denote byq_kiⁿ (k= 1,2, . . . , mn,i= 1,2, . . . ,(M_kⁿ)²) the homothety transforming the unit square E to the square E^M

n k

i from the rectangular netE^Mkn. Assume (σ∈Γ(R²)),

B_kiⁿ =q_kiⁿ(bn), hⁿ_ki(σ) =qⁿ_ki(hⁿ(σ)), Gⁿ_k(σ) =^(M

n k)² i=1∪ hⁿ_ki(σ), Ω²_k(σ) = ^k∪⁻¹

k1=1G²_k1(σ), k >1, Ωⁿ_k(σ) = ⁿ∪⁻¹

n1=2 mn1

k1∪=1Gⁿ_k1¹(σ)∪ ^k−∪¹

k2=1Gⁿ_k2(σ), n >2, θ_k²=

kX−1

k1=1

(M_k²₁)², k >1,

θ_kⁿ=

nX−1 n1=2

mXn1

k1=1

(M_kⁿ₁¹)²+

k−1

X

k2=1

(M_kⁿ₂)², n >2, w_kⁿ= 2r⁻_n¹M_kⁿ₋₁, k >1,

ω_kⁿ= 2θ_kⁿ sup

σ∈Γ(R²)

ˆ|E\Ωⁿ_k(σ)|⁻¹‰ .

Choose numbers M_kⁿ, k = 1,2, . . . , mn, increasing so rapidly that the relation

M_kⁿ≥maxˆ

2η_n⁻¹;d(∂hⁿ); 9ωⁿ_k;M_mⁿ⁻_n−¹₁;kⁿ‰

(5) is fulfilled.

2^◦. We shall construct the function sought for.

LetB^∗_kiⁿ be the circle with the same center as B_kiⁿ and a twofold larger radius. Assume

Aⁿ₁ =ˆ

1,2, . . . ,(M₁ⁿ)²‰ ,

Aⁿ_k =

š

i:E_i^Mk∗∩

E\ _k−1

k1∪=1 (M_kⁿ₁)²

i1∪=1 B_k^∗₁ⁿ_i1‘‘

6

=∅

›

(k= 2, . . . , m2).

Let Sn ∈ S(R²). The functions ψn, gn, and fn will be defined for n= 2,3, . . ., as follows:

ψn(x) =βn

Xn e=1

Nn

X

j=1

χ_In

ej(x), gn(x) =λnSn(x)

mn

X

k=1

X

i∈Aⁿ_k

χ_Bn

ki(x), fn(x) =gn(x) +ψn(x).

The index function is defined as the seriesf(x) =P_∞

n=2fn(x).

Let us prove thatf ∈L(R²). We havekfk¹≤P_∞

n=2

€kψnk¹+kgnk¹ . First we estimatekψnk1. Using Lemma 2(a) and formula (1), we obtain

kψnk¹≤ln⁻¹(βn) Xn e=1

X∞ j=1

βnln(βn)|I_ejⁿ| ≤

≤ln⁻¹(βn) Xn e=1

ŒŒ

Œ ^∞∪

j=1H_ejⁿŒŒŒ=nln⁻¹(βn)<2⁻ⁿ.

Similarly, applying Lemma 2(b) and relations (1), (2), (4) we have

kgnk¹≤ln⁻¹(λn)X^mn

k=1 (M_kⁿ)²

X

i=1

λnln(λn)|B_kⁿ_i|‘

≤

≤ln⁻¹(λn)mn|hn|<2⁻ⁿ. The two latter relations yield the desired inclusion.

3^◦. Here we shall prove that for almost all directionsσ (σ6=σn, n∈N) the integralR

f is strongly differentiable a.e.

(a) Let us first estimate the maximal function M_B^∗_2σgn. Introduce the sets

Bⁿ = supp(gn) = ^m∪ⁿ

k=1 ∪

i∈Aⁿ_kB_kiⁿ, B^∗n= ^m∪ⁿ

k=1 ∪

i∈Aⁿ_kB_ki^∗n and letρn =Pmn

k=1(M_kⁿ)².

By Lemma 3 we can assume that sup

γ

ŒŒ

ΠZ

γ

sn(x)d猌Œ≤rn(2ⁿ⁺¹λnρnM_mⁿ_n)⁻¹, (6)

whereγis an arbitrary interval of an arbitrary straight line inR²anddηis the Lebesgue measure onγ.

Let us show that for every direction σ and for all x from R²\B^∗n the inequality

M_B^∗_2σgn(x)≤2⁻ⁿ, n= 2,3, . . . . (7) is fulfilled. This inequality will be proved only for the case where σ is a standard direction, since the general case has a similar proved.

Let us fix a natural number n (n ≥2), a point x from R²\B^∗n, and a intervalR fromB2(x). We assume thatR∩Bⁿ6=∅and (k1, i) (1≤k1≤ mn, 1≤i≤(M_kⁿ₁)²) is a pair of natural numbers for which

R∩B_kⁿ_1i1 6=∅. (8)

From the inclusionx∈R²\B^∗n we have

dist(x, Bⁿ_k1i1)≥dist(∂B_k^∗n_1i1, B_kⁿ_1i1)≥rn(M_kⁿ₁)⁻¹≥rn(M_mⁿ_n)⁻¹. Taking also the inclusionx∈Rand (8) into consideration, we get

diam(R)≥rn(M_mⁿ_n)⁻¹.

LetR=R1×R2. It follows from the last relation that at least for one p(p= 1,2) the length of the intervalRp is underestimated as follows:

|Rp|¹≥2⁻¹rn(M_mⁿ_n)⁻¹. (9) Without loss of generality we assume thatp= 1. We have (see (9), (6))

|R|⁻¹ŒŒŒ Z

R

gn(y)dyŒŒŒ≤λn|R|⁻¹

mn

X

k=1

X

i∈Aⁿ_k

ŒŒ

ΠZ

R

sn(y)χ_Bn

ki(y)dyŒŒŒ≤

≤λn|R|⁻¹

mn

X

k=1

X

i∈Aⁿ_k

Z

R2

ŒŒ

ΠZ

R1

sn(y1, y2)χ_R1(y1, y2)χ_Bn

ki(y1, y2)dy1

ŒŒ

Œdy2≤

≤λn|R1|⁻¹ρnsup

γ

ŒŒ

ΠZ

γ

sn(y)d猌Œ≤2⁻ⁿ.

To complete the proof of relation (7) it remains to note that R∈B2(x) and is arbitrary.

Let us now show that |B^∗|= 0, whereB^∗= limn→∞supB^∗n. Indeed, using relations (4) and Lemma 2 (b), we obtain

X∞ n=2

|B^∗n| ≤ X∞ n=2

β⁻_n¹ln⁻¹(βn)

mn

X

k=1 (M_kⁿ)²

X

i=1

λnln(λn)|B^∗_kiⁿ| ≤

≤4 X∞ n=2

β_n⁻¹ln⁻¹(βn)

mn

X

k=1

|hⁿ| ≤8 X∞ n=2

β_n⁻¹ln⁻¹(βn)<∞.

Thus|B^∗|= 0, and hence for everyx∈R²\B^∗ there exists a numberp1(x) such that

x∈R²\B^∗n for n≥p1(x). (10) This and inequality (7) imply that for every directionσ and for allxfrom R²\B^∗ the following relation is fulfilled:

M_B^∗_2σgn(x)≤2⁻ⁿ for n≥p1(x). (11) (b) We will now proceed to the estimation of MB2σψn. Taking into account Lemma 4, we find that each one of the following inclusions are fulfilled:

H^σ(χ_In,(n2ⁿβn)⁻¹)⊂Q(Iⁿ,(n2ⁿβn)⁻¹, c(σ))⊂Qⁿ for σ∈Γ(R²)\Γⁿ.

Without loss of generality, we assume that every directionσn,n∈N, is not standard. Suppose (k= 1,2, . . . , n),

Γⁿ_k =ˆ

σ∈Γ(R²);|α(σ)−α(σk)|<2⁻⁽ⁿ⁺¹⁾n⁻¹‰ .

Since the rotation is a measure-preserving transformation and the centers of symmetry of the intervalsIⁿandQⁿcoincide, from the previous inclusion it follows (by virtue of the homothety properties) that the following inclusions hold:

H^σ(χ_In

ki,(n2ⁿβn)⁻¹)⊂Qⁿ_ki for σ∈r(R²)\_e=1∪ⁿ Γⁿ_e, (12) k= 1,2, . . . , n, j = 1,2, . . . .

The definition of the rectangles Qⁿ_ej andQ^∗_ejⁿ immediately implies that for every directionσthere exists a rectangleE_ejⁿ(σ)∈B2σpossessing the prop- erty

Qⁿ_ej⊂E_ejⁿ(σ)⊂Q^∗_ejⁿ. (13) We have |Q^∗_ejⁿ| = 16|Qⁿ_ej| ≤ 144cnβn2ⁿn|I_ejⁿ|. On the other hand, by Lemma 2 (a) we have

ŒŒH^σe(χ_In

ej, β_n⁻¹)ŒŒ≥βnln(βn)|I_ejⁿ| ≥cnn²2²ⁿβn|I_ejⁿ|. The last two relations imply

X∞ n=2

ŒŒ

Œ ∪ⁿ

e=1

∞∪

j=1Q^∗_ejⁿŒŒŒ≤144 X∞ n=2

n⁻¹2⁻ⁿ Xn e=1

X∞ j=1

|H_ejⁿ|=

= 144 X∞ n=2

n⁻¹2⁻ⁿ Xn e=1

ŒŒ

Œ ^∞∪

j=1H_ejⁿŒŒŒ≤144 X∞ n=2

2⁻ⁿ <∞.

Hence|Q^∗|= 0, whereQ^∗= limn→∞sup ∪ⁿ

e=1

∞∪

j=1Q^∗_ejⁿ.

This in turn implies that for all pointsxfromE\Q^∗there exists a number P2(x) such that

x∈E\ ∪ⁿ

e=1

∞∪

j=1Q^∗_ejⁿ for n≥P2(x). (14) Further we have

ŒŒ

Œ_e=1∪ⁿ Γⁿ_e

ŒŒ

β Xn e=1

|Γⁿ_e|= Xn e=1

n⁻¹2⁻ⁿ= 2⁻ⁿ.

Consequently|Γ|= 0, where Γ = limn→∞sup_e=1∪ⁿ Γⁿ_e.

This implies that for every directionσfrom Γ(R²)\Γ there exists a num- bern(σ) such that

σ∈Γ(R²)\ ∪ⁿ

e=1Γⁿ_e for n≥n(σ). (15) Now let us show that ifσ∈Γ(R²)\Γ and x∈E\Q^∗, then

MB2σψn(x)≤2⁻ⁿ for n≥P2(x, σ), (16) whereP2(x, σ) = maxˆ

P2(x);n(σ)‰ .

Indeed, let us fix a direction σ from Γ(R²)\Γ, a point x from E\Q^∗ and a rectangle R^σ from B2σ(x). Let n be a fixed natural number and n≥P2(x, σ). Sinceσ∈Γ(R²)\Γ andn≥P2(x, σ)≥n(σ), it follows from (12), (13), and (15) that for all e,j (e= 1,2, . . . , n, j= 1,2, . . .) the chain of inclusions

H^σ(χ_In

ej,(n2ⁿβn)⁻¹)⊂Qⁿ_ej⊂E_ejⁿ(σ)⊂Q^∗_ejⁿ (17) is fulfilled. Sincex∈E\Q^∗ andn≥P2(x, σ)≥P2(x), from (14) and (13) we have

x∈E\_e=1∪^{n ∞}∪

j=1Q^∗_ejⁿ⊂E\_e=1∪^{n ∞}∪

j=1Eⁿ_ej(σ).

Let{j1, . . . , jse} (e= 1,2, . . . , n) be a set of natural numbers for which

|R^σ ∩I_ejⁿ_i| > 0, i = 1,2, . . . , se. If we observe that the set R^σ ∩E_ejⁿ_i (i= 1,2, . . . , se, e= 1,2, . . . , n) is a rectangle fromB2σ containing at least one point fromE\H^σ(χ_In

ej,(n2ⁿβn)⁻¹) (see (17)), then we obtain

ŒŒR^σ∩E_ejⁿ_i(σ)ŒŒ⁻¹ Z

R^σ∩Eⁿ

eji(σ)

χ_In

eji(y)dy=

=ŒŒR^σ∩E_ejⁿ_i(σ)ŒŒ⁻¹ŒŒR^σ∩I_ejⁿ_iŒŒ≤(n2ⁿβn)⁻¹,

and consequently

ŒŒR^σ∩I_ejⁿ_iŒŒ≤(n2ⁿβn)⁻¹ŒŒR^σ∩E_ejⁿ_i(σ)ŒŒ, e= 1,2, . . . , n, i= 1,2, . . . , se. Next, since the rectanglesQ^∗_ejⁿ,j= 1,2, . . ., do not intersect for every fixed e(and hence the rectanglesE_ejⁿ(σ),j= 1,2, . . .), we have

ŒŒ

Œ ^s∪^e

i=1

€R^σ∩E_ejⁿ_i(σ)ŒŒŒ≤ |R^σ|, e= 1,2, . . . , n.

The two last relations yield

|R^σ|⁻¹ Z

R^σ

ψn(y)dy=βn|R^σ|⁻¹ Xn e=1

X∞ j=1

ŒŒR^σ∩I_ejⁿ_iŒŒ≤

≤βn|R^σ|⁻¹ Xn e=1

se

X

i=1

(n2ⁿβn)⁻¹ŒŒR^σ∩E_ejⁿ_i(σ)ŒŒ=

=n⁻¹2⁻ⁿ|R^σ|⁻¹ Xn e=1

ŒŒ

Œ ^s∪^e

i=1

€R^σ∩E_ejⁿ_i(σ)ŒŒŒ≤2⁻ⁿ.

To complete the proof of (16), it remains to note thatR^σ∈B2σ(x) and is arbitrary.

(c) Let us show that for almost all directions σ the maximal function M_B^∗_2σf is finite a.e. onR². Suppose

P(x, σ) = max{P1(x);P2(x, σ)}.

Fix a direction σ from Γ(R²)\Γ, a point x from E\(Q^∗∪B^∗), and a rectangleR^σ from B2σ(x).

We have

|R^σ|⁻¹ŒŒŒ Z

R^σ

f(y)dyŒŒŒ≤

PX(x,σ)

n=2

|R^σ|⁻¹ Z

R^σ

|fn(y)|dy+

+|R^σ|⁻¹ŒŒŒ Z

R^σ

X

n=p(x;σ)+1

fn(y)dyŒŒŒ=a1(x, R^σ) +a2(x, R^σ)

and

a1(x, R^σ)≤

P(x,σ)

X

n=2

MB2σψn(x) +

P(x,σ)

X

n=2

MB2σgn(x)≤

≤

P(x,σ)

X

n=2

kψnkL^∞+

P(x,σ)

X

n=2

kgnkL^∞ ≤

P(x,σ)

X

n=2

(nβn+mnλn).

Estimate nowa2(x, R^σ). Using the theorem on the passage to the limit under the integral sign as well as relations (7) and (16), we obtain

a2(x, R^σ)≤ X

n=p(x,σ)+1

|R^σ|⁻¹ŒŒŒ Z

R^σ

fn(y)dyŒŒŒ≤

≤ X∞ n=p(x,σ)+1

(MB2σψn(x) +M_B^∗_2σgn(x))≤2.

Hence

|R^σ|⁻¹ŒŒŒ Z

R^σ

f(y)dyŒŒŒ≤

p(x,σ)X

n=2

(nβn+mnλn) + 2<∞.

Since the right-hand side of this inequality does not depend on a choice of rectangles fromB2σ(x), we get

M_B^∗_2σf(x)<∞, σ∈Γ(R²)\Γ, x∈E\(Q^∗∪B^∗).

Consequently forσ∈Γ(R²)\Γ and forx∈E\(Q^∗∪B^∗) we have

−∞< D_B2σ(f)(x)≤DB2σ(f)(x)<+∞.

Using now the Besicovitch theorem on possible values of upper and lower derivatives (see [1], Ch. V), we obtain (|Γ|=|Q^∗∪B^∗|= 0) and for almost every directionσ(σ6=σn,n∈N) the relationDB2σ(f)(x) =f(x) is fulfilled a.e.

4^◦. It will now be shown that for every direction σs(s∈N) the strong upper derivative of the integral R

f is equal to +∞a.e. onE.

To this end we fix a natural number sand notice that |Js| = 1, where Js= limn→∞sup^N∪ⁿ

j=1H^σs(χ_In

sj, β⁻_n¹). Indeed, 1 =ŒŒŒ lim

n→∞sup^N∪ⁿ

j=1H_sjⁿŒŒŒ≤ŒŒŒ lim

n→∞sup^N∪ⁿ

j=1H^σs(χ_In

sj, β⁻_n¹ŒŒŒ+ +ŒŒŒ lim

n→∞sup^N∪ⁿ

j=1Q^∗_sjⁿŒŒŒ=|Js|+|Q^∗|=|Js|. LetDs=_n=2^∞∩ D_sⁿ, whereDⁿ_s =ˆ

y∈E:DB2,σs(fn)(y) =fn(y)‰ .

Since the basis B2 differentiates the integrals of the bounded functions (see [1], Ch. III), it is evident that|Dⁿ_s|= 1 for everyn= 2,3, . . ..

Let us fix a pointxfromJs∩Ds\(Q^∗∪B^∗) and prove that

DB2σs(f)(x) = +∞. (18)