1Introductionandthemainresult DmitryB.Rokhlin StochasticPerron’smethodforoptimalcontrolproblemswithstateconstraints

ISSN:1083-589X in PROBABILITY

Stochastic Perron’s method for optimal control problems with state constraints

Dmitry B. Rokhlin

Abstract

We apply the stochastic Perron method of Bayraktar and Sîrbu to a general infinite horizon optimal control problem, where the stateXis a controlled diffusion process, and the state constraint is described by a closed set. We prove that the value function vis bounded from below (resp., from above) by a viscosity supersolution (resp., sub- solution) of the related state constrained problem for the Hamilton-Jacobi-Bellman equation. In the case of a smooth domain, under some additional assumptions, these estimates allow to identifyvwith a unique continuous constrained viscosity solution of this equation.

Keywords: Stochastic Perron’s method; State constraints; Viscosity solution; Comparison re- sult.

AMS MSC 2010:93E20; 49L25; 60H30.

Submitted to ECP on June 19, 2014, final version accepted on October 20, 2014.

1 Introduction and the main result

The aim of the paper is to extend the scope of applications of the stochastic Per- ron method, developed by Bayraktar and Sîrbu. This method allows to characterise the value function of a controlled diffusion problem as a viscosity solution of the cor- responding Hamilton-Jacobi-Bellman (HJB) equation, bypassing the dynamic program- ming principle. Instead it requires a comparison result, implying the uniqueness of a viscosity solution of the HJB equation. Previously this method was applied to linear parabolic equations [5], stochastic differential games [7, 25, 26], regular [6, 24] and singular control problems [8].

The method involves the construction of two familiesV₋,V+of functions, bounding the value function from below and above

u≤v≤w, u∈ V₋, w∈ V+.

Elements ofV₋, V+ are called stochastic sub- and supersolutions. By the superposi- tion with the state process,uandwgenerate sub- and supermartingale-like processes.

Similarly to the classical Perron method [14, Sections 2.8, 6.3], the setV− (resp.,V+) is directed upward (resp., downward) with respect to the pointwise maximum (resp., minimum) operation. The essence of the method is to prove that the functions

u₋(x) = sup

u∈V−

u(x), w₊(x) = inf

w∈V₊w(x)

*Institute of Mathematics, Mechanics and Computer Sciences, Southern Federal University, Rostov-on- Don, Russia. E-mail:[email protected]

are respectively viscosity super- and subsolutions of the related HJB equation. If a comparison result, providing the inequalityu₋ ≥w₊, holds true, it follows thatu₋=v= w+is a unique (continuous) viscosity solution. This construction differs from Perron’s method of [17], which is not linked to the value function.

In the present paper we consider the stochastic control problem with state con- straints in the form of [21]. In contrast to [23], where the drift is not assumed to be bounded, and the value function is singular near the boundary, in [21] the problem is

"regular". To achieve the regularity it is assumed that the diffusion coefficient depends on the control and degenerates at the boundary. The same problem was considered in [18, 11]. It was proved that under appropriate assumptions the value function v is a unique continuous constrained viscosity solution of the HJB equation. (The term "con- strained" means, in particular, thatvsatisfies special boundary conditions, which in the deterministic situation were introduced in [27].) Roughly speaking, it is enough to as- sume that for each boundary point there exists a control, which kills the diffusion and directs the drift strictly inside the domain.

An application of the stochastic Perron method to state constrained problems seems rather interesting, since, as it is mentioned in [21], a direct proof of the dynamic pro- gramming principle is not available due to a complicated structure of admissible control processes, retaining a phase trajectory in a predetermined domain. Different penaliza- tion and approximation procedures were used instead in [21, 18, 11, 10].

We turn to the precise statement of our main result (Theorem 1.2). Let Ω be the space C([0,∞),R^m) of continuous R^m-valued functions, endowed with the σ-algebra F^◦ of cylindrical sets, and let P be the Wiener measure on F^◦. So, the canonical processWs(ω) =ω(s)is the standardm-dimensional Brownian motion underP. Denote byF^◦ = (Ft^◦)t≥0 the natural filtration ofW, and letF= (Ft)t≥0 be the correspondent minimal augmented filtration. The extension of the Wiener measure to the completion F ^ofF^◦is still denoted byP.

Letαbe anF-progressively measurable stochastic process with values in a compact setA⊂R^k,0∈A. Consider the system of stochastic differential equations

dXt=b(Xt, αt)dt+σ(Xt, αt)dWt, X0=x. (1.1) We assume that the drift vectorb:R^d×A7→R^d and the diffusion matrixσ:R^d×A7→

R^d×R^mare continuous and satisfy the Lipschitz condition

|b(x, a)−b(y, a)|+|σ(x, a)−σ(y, a)| ≤K|x−y|

with some constantKindependent ofx,y,a. Note, that the linear growth condition

|b(x, a)|+|σ(x, a)| ≤K⁰(1 +|x|)

follows from the continuity ofb, σand compactness of A. Thus, there exist a unique F-adapted strong solutionX^x,αof (1.1) on[0,∞): see [22, Chapter 2, Sect. 5].

LetG⊂R^dbe a closed set with the boundary∂Gand nonempty interiorG^◦. It will be convenient to assume that0∈G^◦. Denote byA(x),x∈Gthe set ofF-progressively measurable control processes αwith values in A and such thatX_t^x,α ∈ G, t ≥ 0 a.s.

Elements ofA(x) are calledadmissible controls for the initial condition x. The cost functionalJ and the value functionvare defined as follows

J(x, α) =E Z ∞

0

e^−βsf(X_s^x,α, α_s)ds, v(x) = inf

α∈A(x)

J(x, α), (1.2) wheref :G×A7→Ris a bounded continuous function.

We assume that for any initial condition x ∈ Gthere exists an admissible control:

A(x)6=∅. In this case the setGis calledviable. A necessary condition for the validity of this property is given in [2] (Theorem 1). Let

NG²(x) =

(p, Y)∈R^d×S^d: lim inf

G3y→x

p·(y−x)

|y−x|² +1 2

Y(y−x)·(y−x)

|y−x|²

≥0

be the second order normal cone. HereS^d is the set of symmetricd×dmatrices. If the setGis viable then for allx∈∂G,(p, Y)∈NG²(x)there exista∈Asuch that

p·b(x, a) +1

2Tr (σ(x, a)σ^T(x, a)Y)≥0. (1.3) See [2, Section 3] for more concrete forms of this condition.

We impose a slightly stronger requirement. For any functionψ:R^d7→Aput

bψ(x) =b(x, ψ(x)), σψ(x) =σ(x, ψ(x)). (1.4) Assumption 1.1.There exist a Borel measurable functionψ:R^d7→Asuch thatbψ,σψ

are globally Lipschitz continuous and p·bψ(x) +1

2Tr (σψ(x)σ^T_ψ(x)Y)≥0, x∈∂G, (p, Y)∈NG²(x).

Under this assumption there exist a unique strong solution of the equation

dX_t=b_ψ(X_t)dt+σ_ψ(X_t)dW_t, X₀=x (1.5) and Xt ∈ G, t ≥ 0 a.s.: see [1, Theorem 3.1]. The correspondent control process αt=ψ(Xt)is admissible forx. Hence,A(x)6=∅,x∈G.

Consider the Bellman operator F(x, r, p, Y) = sup

a∈A

βr−f(x, a)−b(x, a)·p−1

2Tr (σ(x, a)σ^T(x, a)Y)

,

defined onR×R×R^d×S^d. Recall that a bounded upper semicontinuous (usc) function uis called aviscosity subsolutionof the equation

F(x, u, Du, D²u) = 0 (1.6)

on a setE⊂R^{dif for any}ϕ∈C²(R^d)and for any local maximum pointx0ofu−ϕonE the inequality

F(x0, u(x0), Dϕ(x0), D²ϕ(x0))≤0

holds true. In the same way, a bounded lower semicontinuous (lsc) functionwis called aviscosity supersolution of (1.6) onE if for anyϕ∈C²(R^d)and for any local minimum pointx₀ofw−ϕonE we have the inequality

F(x0, w(x0), Dϕ(x0), D²ϕ(x0))≥0.

In these definitions one can assume that the maximum (resp., minimum) pointx0is strict andϕ(x₀) =u(x₀)(resp.,ϕ(x₀) =w(x₀)).

It is convenient to introduce thestate constrainedproblem F(x, u, Du, D²u)≤0 on G^◦,

F(x, u, Du, D²u)≥0 on G. ^(1.7)

We say that a bounded usc (resp., lsc) functionu, defined onG, is viscosity subsolution (resp., supersolution) of the state constrained problem (1.7) ifF(x, u, Du, D²u)≤0on G^◦ (resp., F(x, u, Du, D²u) ≥ 0 on G) in the viscosity sense. A bounded function u is called a viscosity solution of (1.7) (or aconstrained viscosity solution), if its upper semicontinuous envelope u^∗ is a viscosity subsolution, and its lower semicontinuous envelopeu∗is a viscosity supersolution of (1.7).

Denote byΓthe set of pointsx∈∂Gsuch that for someα∈A(x)the solutionX^x,α of (1.1) immediately entersG^◦with probability1:

P(inf{t >0 :X_t^x,α∈G^◦}= 0) = 1.

Theorem 1.2.There exist a viscosity subsolutionw₊ and a viscosity supersolutionu₋ of the state constrained problem (1.7) such that

u₋≤v on G; v≤w₊ on G^◦, andv(x)≤lim sup_G◦3y→xw₊(y),x∈Γ.

The nature ofw+ andu− is not explicitly indicated here. Their construction, which is presented in Sections 2 and 3 respectively, is based on the technique of stochastic semisolutions, developed in [5, 6, 7]. The details are quite similar to [6, 24]. One only should take care of admissibility of controls.

Theorem 1.2 is useful if a sort of comparison result is available, and one can conclude thatw+≤u₋. In Section 4 we consider the case of a smooth domain and, under some additional assumptions, mention that such inequality follows from the known result, concerning the boundary behavior of viscosity subsolutions of linear equations [3], and the comparison result of [21]. In combination with Theorem 1.2 this allows to identifyv with a unique continuous viscosity solution of (1.7). The related result (Theorem 4.1) is not new and is presented only to demonstrate the capabilities of the stochastic Perron method.

2 Stochastic supersolutions

ForF-stopping timesτ,σand a setD∈ Fτ denote by

Jτ, σK={(t, ω)∈[0,∞)×Ω :τ(ω)≤t≤σ(ω)}

the stochastic interval, and by

τD=τ ID+ (+∞)ID^c, D^c= Ω\D

the restriction ofτ onD. Put B_ε(x) ={y ∈R^d : |y−x|< ε} and denote byB_ε(x)the closure of this ball.

Let τ : Ω 7→[0,∞]be a stopping time and take anFτ-measurable random vectorξ such thatξI_{{τ <∞}}is bounded andξ∈Gon{τ <∞}. For anF-progressively measurable processαwith values inAconsider the stochastic differential equation (1.1) with the randomized initial condition(τ, ξ):

Xt=ξI_{t≥τ}+ Z t

τ

b(Xs, αs)ds+ Z t

τ

σ(Xs, αs)dWs, t≥0. (2.1) ByRt

τ(·)we meanRt

0I_{s≥τ}(·). As is known, see [22, Chapter 2, Sect. 5], there exists a pathwise unique strong solutionX^τ,ξ,α of (2.1). The trajectories of the processX^τ,ξ,α are continuous on the stochastic intervalJτ,∞K. Moreover,X^τ,ξ,α= 0onJ0, τJ^and

X_τ^τ,ξ,α= lim

t&τX_t^τ,ξ,α=ξ on {τ <∞}.

Denote byA(τ, ξ)the set of progressively measurable control processesαsuch that α_t ∈Aand X_t^τ,ξ,α ∈G, t∈ [τ,∞)a.s. That is,A(τ, ξ)is the set of admissible controls for a randomized initial condition(τ, ξ). We omit indexτ ifτ= 0. For instance,X^x,α= X^0,x,α,A(x) =A(0, x).

Lemma 2.1.Under Assumption 1.1 the set A(τ, ξ) is non-empty for any randomized initial condition(τ, ξ).

Proof. For anF^◦-stopping timeτ⁰theσ-algebraFτ^◦0is countably generated ([29, Lemma 1.3.3]), and there exists a regular conditional probability distributionP^τ0 = (P^τ0,ω)_ω∈Ω of Pwith respect to Fτ^◦0: see [29, Theorem 1.3.4] or [28, Theorem 9.2.1]. For each B ∈ F^◦ the function ω 7→ P^τ0,ω(B) is Fτ^◦0-measurable, for each ω ∈ Ω the function B7→P^τ0,ω(B)is a probability measure onF^{◦ such that}

P^τ0,ω(B) =E(I_B|Fτ^◦0)(ω) P-a.s., B∈F^◦. Moreover, there exists aP-null setN ∈Fτ^◦0 with the property that

P^τ0,ω(C) =IC(ω) for all ω6∈N, C ∈Fτ^◦0. (2.2) Consider the SDE

X_t=ξI_{t≥τ}+ Z t

τ

b_ψ(X_s)ds+ Z t

τ

σ_ψ(X_s)dW_s, t≥0, (2.3)

whereψsatisfies Assumption 1.1. To work withP^τ0, related to the raw filtrationF^{◦, we} pass fromξI_{t≥τ} to an indistinguishableF^◦-adapted process of the same form. Recall that anyF-stopping time is predictable (see [4, Proposition 16.22]) and the filtrationF^is quasi-left continuous (see [15, Theorem 3.40]), that is,Fτ− =Fτ for any (predictable) F-stopping timeτ. By Theorem IV.78 of [13] there exists anF^◦stopping timeτ⁰such that P(τ⁰ 6=τ) = 0, and for anyB ∈Fτ−=Fτthere existsB⁰∈Fτ^◦such thatP(IB⁰ 6=IB) = 0. It easily follows that the process ξI_{t≥τ} is indistinguishable from an F^◦-adapted processξ⁰I_{t≥τ0}with someF_τ^◦0-measurableξ⁰.

PutZ_t⁰=t−t∧τ,Zt=Wt−Wt∧τ. The processZ is a continuous martingale under P, and we can rewrite equation (2.3) in the form

Xt=Ht+ Z t

0

bψ(Xs)dZ_s⁰+ Z t

0

σψ(Xs)dZs, t≥0, (2.4) whereHt=ξ⁰I_{t≥τ⁰_}.

Recall the pathwise construction of a strong solution, presented in [20] (see also [9, 19]). Denote byD=D([0,∞),R^d)the set of functions from[0,∞)toR^d, which are right continuous and have left limits. There exist a mappingS :D×C([0,∞),R^m)7→D such that ifZis a continuous semimartingale on a filtered probability space(Ω,F,Q,F), whereFsatisfies the usual conditions, and ifHis anF-adapted process with trajectories inD^{, then}

X_t(ω) =S(H_·(ω), Z_·(ω))_t is a strong solution of (2.4).

Takeω∈Ω\N withτ⁰(ω)<∞. Note thatZ is aP^τ0,ω-martingale, andZ is the stan- dardd-dimensionalP^τ0,ω-Brownian motion on[τ⁰(ω),∞). It follows thatX is a strong solution of (2.4) underP^τ0,ω with respect to theP^τ0,ω-augmentation ofF^◦. Moreover, by (2.2) we get

P^τ0,ω({ω:τ⁰(ω) =τ⁰(ω), ξ⁰(ω) =ξ⁰(ω)}) = 1.

Hence, underP^τ0,ω, the processH is indistinguishable from ξ⁰(ω)I_{t≥τ0(ω)}, andX is a strong solution of the SDE with a non-random initial condition:

X_t=ξ⁰(ω) + Z t

τ⁰(ω)

b_ψ(X_s)ds+ Z t

τ⁰(ω)

σ_ψ(X_s)dW_s, t≥τ⁰(ω).

In addition,Xt = 0,t∈[0, τ⁰(ω))P^τ0,ω-a.s. sinceZ0, Z,H are indistinguishable from 0 on[0, τ⁰(ω)).

By Assumption 1.1 the diffusion coefficientsb_ψ,σ_ψsatisfy conditions of Theorem 3.1 of [1]. Since0 ∈Gandξ⁰(ω)∈G, we conclude thatX_t∈G, t≥0P^τ0,ω-a.s. It follows thatGis invariant underP:

P(Xt∈G, t≥0) =E

I_{τ⁰_(ω)<∞}P^τ0,ω(Xt∈G, t≥0)

= 1.

The desired control processα∈A(τ, ξ)is given by the formulaα=ψ(X).

Letwbe a uniformly bounded continuous function:w∈Cb(G). Consider the stochas- tic process

Z_t^τ,ξ,α(w) = Z t

τ

e^−βsf(X_s^τ,ξ,α, αs)ds+I_{t≥τ}e^−βtw(X_t^τ,ξ,α).

Definition 2.2.We say that a control processα∈ A(τ, ξ)isw-suitablefor(τ, ξ)if E(Z_ρ^τ,ξ,α(w)|Fτ)≤Z_τ^τ,ξ,α(w) =e^−βτw(ξ)

for any stopping timeρ≥τ. A functionw∈Cb(G)is called a stochastic supersolution of (1.7) if for any randomized initial condition(τ, ξ)withξ∈G^◦there exists aw-suitable controlα.

The set of stochastic supersolutions is denoted byV⁺. Note that in the above defi- nition the valuesX∞are irrelevant, sinceZ∞=R∞

0 e^−βsf(X_s^τ,ξ,α, αs)ds. We emphasize also that the conditionA(τ, ξ)6=∅for all randomized initial conditions (τ, ξ), ξ∈G^◦ is necessary for the existence of stochastic supersolutions.

A stochastic supersolutionw is an upper bound for the value function (1.2) onG^◦. To see this putτ = 0, ξ =x∈G^◦,ρ =∞and take aw-suitable controlα ∈A(x). By Definition 2.2, with the conventionZ^x,α=Z^0,x,α, we get

v(x)≤J(x, α) =EZ_∞^x,α(w)≤EZ₀^x,α(w) =w(x).

The setV⁺is non-empty and contains sufficiently large constantsc: it is easy to see that

E(Z_ρ^τ,ξ,α(c)|Fτ)≤ce^−βτ =Z_τ^τ,ξ,α(c) for c≥f /β, wheref = sup_(x,a)∈G×Af(x, a).

Lemma 2.3.Ifw1, w2 are stochastic supersolutions thenw = w1∧w2 is a stochastic supersolution.

Proof. Letαⁱ∈A(τ, ξ),i= 1,2bewi-suitable controls for a randomized initial condition (τ, ξ). PutA1={w1(ξ)< w2(ξ)} ∈Fτ,A2=A^c₁:= Ω\A1. We claim that

α=I_A1I_{τ≤t}α¹+I_A2I_{τ≤t}α²

belongs toA(τ, ξ)and that it isw-suitable.

The process Y = P2

i=1X_t^τ,ξ,αiIA_i satisfy the same equation as X^τ,ξ,α. From the pathwise uniqueness property it follows that Y = X^τ,ξ,α. We have X^τ,ξ,α ∈ G, t ≥ τ P-a.s., andαisw-suitable for(τ, ξ):

E(Z_ρ^τ,ξ,α(w)|Fτ) =

2

X

i=1

E(IA_iZ_ρ^τ,ξ,αi(w)|Fτ)≤

2

X

i=1

IA_iE(Z_ρ^τ,ξ,αi(wi)|Fτ)

≤

2

X

i=1

IA_ie^−βτwi(ξ) =e^−βτw(ξ).

The following result was used in [5, 6, 24] (see, e.g., Lemmas 2 and 4 of [24]). Its proof use only the fact thatV⁺ is directed downward, that is, the statement of Lemma 2.3 holds true.

Lemma 2.4.There exists a sequencewn∈ V⁺,wn(x)≥wn+1(x),x∈Gsuch that

n→∞lim wn(x) =w+(x) := inf

u∈V⁺w(x).

The next assertion is the most important part of the stochastic Perron method.

Lemma 2.5.The function

w+(x) = inf

w∈V⁺w(x) is a viscosity subsolution of (1.7).

Proof. If w₊ is not a viscosity subsolution then there existx₀ ∈ G^◦, ϕ∈ C²and ε > 0 such thatw₊(x₀) =ϕ(x₀),w₊< ϕon the setB_ε(x₀)\{0} ⊂G^◦and

F(x₀, ϕ(x₀), Dϕ(x₀), D²ϕ(x₀))>0.

Hence, there exists somea∈Asuch thatβϕ(x0)−(L^aϕ)(x0)−f(x0, a)>0, where (L^aϕ)(x) =b(x, a)Dϕ(x) +1

2Tr σ(x, a)σ^T(x, a)D²ϕ(x) .

By the continuity ofb,σ,f we may assume that

βϕ(x)−(L^aϕ)(x)−f(x, a)>0, x∈Bε(x0)⊂G^◦ (2.5) for someε >0.

Sincew+is upper semicontinuous, we have

w+(x)−ϕ(x)≤ −δ <0, x∈Sε:=Bε(x0)\B_ε/2(x0).

By Lemma 2.4 there exists a decreasing sequencewn∈ V⁺,wn&w+. The sets An={x∈Sε:wn(x)−ϕ(x)≥ −δ⁰}, δ⁰∈(0, δ)

are compact,A_n ⊃A_n+1 and∩^∞_n=1A_n =∅. Thus,∩^N_n=1A_n =∅for someN. This means that there exists a functionw=w_N ∈ V⁺such thatw−ϕ <−δ⁰ onS_ε.

Define the function ϕ^η = ϕ−η, whereη ∈ (0, δ⁰)is such that the inequality (2.5) holds true forϕ^η instead ofϕ. Note that

w−ϕ^η=w−ϕ+η <−δ⁰+η <0 on Sε. We claim that

w^η =

ϕ^η∧w on Bε(x0),

w otherwise

is a stochastic subsolution. This gives a contradiction with the definition of w₊ since w^η(x₀) =ϕ^η(x₀) =w₊(x₀)−η < w₊(x₀).

It is clear thatw^η ∈Cb(G). We only need to construct aw^η-suitable controlαfor a randomized initial condition(τ, ξ),ξ∈G^◦. Put

U ={x∈B_ε/2(x₀) :w(x)> ϕ^η(x)}, H={ξ∈U} ∈Fτ

and define a progressively measurable process

α_t= (aI_H+α⁰_tI_Hc)I_{t≥τ}∈A, whereα⁰is aw-suitable control for(τ, ξ). Furthermore, put

τ₁= inf{t≥τ :X_t^τ,ξ,α6∈B_ε/2(x₀)}, αt=αtI_{t≤τ1}+α¹_tI_{t>τ1},

whereα¹is aw-suitable control for(τ₁, ξ₁),ξ₁=X_τ^τ,ξ,α

1 I_{τ1<∞}. We haveX^τ,ξ,α=X^τ,ξ,α on the stochastic intervalJτ, τ1K ^andX^τ,ξ,α =X^τ1,ξ1,α1 on Jτ1,∞K^{. Thus,} α∈ A(τ, ξ). Note also that forE ={ξ∈Bε/2(x0)}we get

X^τ,ξ,α∈B_ε/2(x0) on JτE,(τ1)EK; X^τ,ξ,α=ξ on JτE^c,(τ1)E^cK.

It remains to show thatαis aw^η-suitable control for(τ, ξ). For a stopping timeρ≥τ putD={ρ > τ1}. We have

Z_ρ^τ,ξ,α(w^η)I_D=I_D Z τ1

τ

e^−βsf(X_s^τ,ξ,α, α_s)ds

+ID

Z ρ

τ₁

e^−βsf(X_s^τ1,ξ1,α1, α¹_s)ds+e^−βρw^η(X_ρ^τ1,ξ1,α1)

≤ID

Z τ₁

τ

e^−βsf(X_s^τ,ξ,α, αs)ds+IDZ_ρ^τ1,ξ1,α1(w). (2.6) By Definition 2.2 we get

E(Z_ρ^τ1,ξ1,α1(w)I_D|F_τ1) =E(Z_ρ^τ1,ξ1,α1

D (w)I_D|F_τ1)≤I_De^−βτ1w(ξ₁)

=I_De^−βτ1w^η(ξ₁). (2.7)

The last equality follows from the fact thatξ16∈B_ε/2(x0)on the set{ρ > τ1}andw=w^η onG\B_ε/2(x0). From (2.6), (2.7) it follows that

E(Z_ρ^τ,ξ,α(w^η)ID|Fτ₁)≤ID

Z τ₁

τ

e^−βsf(X_s^τ,ξ,α, αs)ds+e^−βτ1w^η(ξ1)

=I_DZ_τ^τ,ξ,α

1 (w^η), and we obtain the estimate

E(Z_ρ^τ,ξ,α(w^η)|Fτ) =E(I_{ρ≤τ1}Z_ρ^τ,ξ,α(w^η)|Fτ) +E(I_{ρ>τ1}E(Z_ρ^τ,ξ,α(w^η)|Fτ1)|Fτ)

≤E(I_{ρ≤τ1}Z_ρ^τ,ξ,α(w^η)|Fτ) +E(I_{ρ>τ1}Z_τ^τ,ξ,α₁ (w^η)|Fτ)

=E(Z_ρ∧τ^τ,ξ,α₁(w^η)|Fτ). (2.8)

On the stochastic intervalJτ_H,(τ₁)_HKthe trajectories ofX^τ,ξ,α do not leave the ball B_ε/2(x0). Hence, the estimate w^η(X_ρ∧τ^τ,ξ,α₁)≤ϕ^η(X_ρ∧τ^τ,ξ,α₁)holds true onH and we get the inequality

Z_ρ∧τ^τ,ξ,α₁(w^η) =Z_ρ∧τ^τ,ξ,a₁(w^η)IH+Z_ρ∧τ^τ,ξ,α₁⁰(w^η)IH^c≤Z_ρ∧τ^τ,ξ,a₁(ϕ^η)IH+Z_ρ∧τ^τ,ξ,α₁⁰(w)IH^c. (2.9)

Applying Ito’s formula Z_t^τ,ξ,a(ϕ^η) =

Z t

τ

e^−βsf(X_s^τ,ξ,a, a)ds+e^−βtϕ^η(X_t^τ,ξ,a)

=e^−βτϕ^η(ξ) + Z t

τ

e^−βs

f(X_s^τ,ξ,a, a) + (L^aϕ^η−βϕ^η)(X_s^τ,ξ,a) ds

+ Z t

τ

e^−βsϕ^η_x(X_s^τ,ξ,a)·σ(X_s^τ,ξ,a, a)dW_s. (2.10) on the intervalJτ, ρ∧τ1K, taking the conditional expectation, and using (2.5), we get

E(Z_ρ∧τ^τ,ξ,a₁(ϕ^η)IH|Fτ)≤e^−βτϕ^η(ξ)IH=e^−βτw^η(ξ)IH=Z_τ^τ,ξ,α(w^η)IH. (2.11) Furthermore,

E(Z_ρ∧τ^τ,ξ,α₁⁰(w)|Fτ)I_Hc≤Z_τ^τ,ξ,α0(w)I_Hc=Z_τ^τ,ξ,α(w^η)I_Hc (2.12) by the definition ofα⁰. The combination of (2.11), (2.12) with (2.9) and (2.8) gives the desired inequality

E(Z_ρ^τ,ξ,α(w^η)|Fτ)≤Z_τ^τ,ξ,α(w^η).

To show thatw₊ satisfies the last assertion of Theorem 1.2, we study its behavior near the points ofΓ. Fixx∈Γ. By the definition ofΓthere existsα¹∈A(x)such that

τ= inf{t >0 :X_t^x,α1∈G^◦}= 0 a.s. (2.13) Forε >0consider the predictable set

E={(t, ω) :X_t^x,α1(ω)∈G^◦, t∈(0, ε]}=K0, εK∩ X^x,α1−1

(G^◦)

and its projection:D={ω: (t, ω)∈E for some t∈[0,∞)}. The equality (2.13) means that P(D) = 1. By the section theorem [4, Theorem 16.12] there exist an F^-stopping timeσ^εsuch that

{(σ^ε(ω), ω) :ω∈Ω, σ^ε(ω)<∞} ⊂E, P(σ^ε<∞)≥1−ε. (2.14) PutD_ε={σ^ε≤ε}={σ^ε<∞}. Then (2.14) means that

X_σ^x,αε ¹ ∈G^◦ on D_ε, P(D_ε)≥1−ε.

Let wbe a stochastic supersolution, bounded from above by the constantf /β. Put ξ^ε=ID_εX_σ^x,αε ¹ ∈G^◦ and take aw-suitable controlα²∈A(σ^ε, ξ^ε). Then

α=α¹I_{t<σε}+α²I_{t≥σε}∈A(x).

Taking into account thatσ^ε=∞onD_ε^c, by the definitions ofvandwwe obtain:

v(x)≤E Z σ^ε

0

e^−βtf(X_t^x,α1, α_t¹)dt+E Z ∞

σ^ε

e^−βtf(X_t^{σε,ξε,α2}, α²_t)dt Fσ^ε

! ,

≤E Z σ^ε

0

e^−βtf(X_t^x,α1, α_t¹)dt+e^−βσεw(ξ^ε)

!

It easily follows that v(x)≤ f

β

1−Ee^−βσε

+Ee^−βσεw(ξ^ε)ID_ε+f

β(1−P(Dε)). (2.15)

Moreover, by Lemma 2.4 and the monotone convergence theorem we can changewto w₊in this inequality.

Takeεn such thatP(D_ε^c

n)≤1/2ⁿ. By the Borel-Cantelli lemma for allω in some set Ω⁰withP(Ω⁰) = 1we haveω∈Dε_n for sufficiently largen. Thus,

ID_εn →1, ξ^εn→x, σ^εn→0 on Ω⁰, and from (2.15) we obtain the estimatev(x)≤lim sup_G◦3y→xw+(y).

3 Stochastic subsolutions

Definition 3.1.With the notation of Section 2 we callu∈Cb(G)a stochastic subsolu- tionif

E(Z_ρ^τ,ξ,α(u)|Fτ)≥Z_τ^τ,ξ,α(u) =e^−βτu(ξ) (3.1) for any randomized initial condition(τ, ξ), admissible control processα ∈A(τ, ξ)and stopping timeρ≥τ.

Any stochastic subsolutionuis a lower bound forv: forτ= 0,ξ=x,ρ=∞we have J(x, α) =EZ_∞^x,α(u)≥Z₀^x,α(u) =u(x), α∈A(x).

Putf = inf_(x,a)∈G×Af(x, a).The setV⁻ of stochastic subsolutions is non-empty and contains sufficiently large negative constantsc. Indeed, it is easy to see that

E(Z_ρ^τ,ξ,α(c)|Fτ)≥ce^−βτ for c≤f /β.

Lemma 3.2.Letu₁,u₂be stochastic subsolutions. Thenu₁∨u₂is a stochastic subsolu- tion.

The proof follows from the inequality E(Z_ρ^τ,ξ,α(u₁∨u₂)|Fτ)≥max

i=1,2E(Z_ρ^τ,ξ,α(u_i)|Fτ)≥max

i=1,2Z_τ^τ,ξ,α(u_i) =e^−βτ(u₁∨u₂)(ξ).

Lemma 3.3.There exists a sequenceun ∈ V⁻,un(x)≤un+1(x),x∈Gsuch that

n→∞lim un(x) =u₋(x) := sup

u∈V⁻

u(x).

This lemma is analogous to Lemma 2.4.

Lemma 3.4.The function

u₋(x) = sup

u∈V⁻

u(x)

is a viscosity supersolution of (1.7).

Proof. Ifu₋ is not a viscosity supersolution then there existx0 ∈G,ϕ∈C² andε >0 such thatu₋(x0) =ϕ(x0),u₋> ϕon(Bε(x0)\{0})∩Gand

F(x0, ϕ(x0), Dϕ(x0), D²ϕ(x0))<0.

By the continuity ofF we can assume that

F(x, ϕ(x), Dϕ(x), D²ϕ(x))<0, x∈Bε(x0)∩G. (3.2) Furthermore, by the lower-semicontinuity ofu−we have

u−(x)≥ϕ(x) +δ, x∈Sε:= Bε(x0)\Bε/2(x0)

∩G

for someδ >0. In the same way as in the proof of Lemma 2.5, one can show that there existu∈ V⁻ andδ⁰∈(0, δ)such thatu≥ϕ+δ⁰onS_ε.

Take anη ∈(0, δ⁰)such that (3.2) holds true forϕ^η =ϕ+η instead ofϕ. We have u−ϕ^η ≥δ⁰−η >0onSε.

To get a contradiction it is enough to prove that the function u^η=

ϕ^η∨u on B_ε(x₀)∩G,

u otherwise

is a stochastic subsolution, sinceu^η(x0) =ϕ^η(x0)> u₋(x0), contrary to the definition of u₋.

Clearly u^η ∈ Cb(G), and we only should to verify (3.1) for any randomized initial condition(τ, ξ), control processα∈A(τ, ξ)and stopping timeρ≥τ. Put

τ1= inf{t≥τ:X_t^τ,ξ,α6∈B_ε/2(x0)}, ξ1=X_τ^τ,ξ,α

1 I_{τ1<∞}, E={ξ∈B_ε/2(x0)}.

We have

ξ₁∈∂B_ε/2(x₀)∩G on E∩ {τ₁<∞}; ξ₁= 0 on E∩ {τ₁=∞}; ξ₁=ξ on E^c. Moreover,X^τ1,ξ1,α=X^τ,ξ,αon the stochastic intervalJτ1,∞K^.

PutD={ρ > τ1}. Similarly to (2.6) we get Z_ρ^τ,ξ,α(u^η)ID≥ID

Z τ₁

τ

e^−βsf(X_s^τ,ξ,α, αs)ds+IDZ_ρ^τ1,ξ1,α(u). (3.3) Applying Definition 3.1, we obtain

E(Z_ρ^τ1,ξ1,α(u)I_D|F_τ1) =E(Z_ρ^τ1,ξ1,α

D (u)I_D|F_τ1)≥I_De^−βτ1u(ξ₁) =I_De^−βτ1u^η(ξ₁). (3.4) The last equality follows from the fact thatξ1, restricted to D, takes values in the set G\Bε/2(x0)whereu=u^η.

From (3.3), (3.4) it follows that E(Z_ρ^τ,ξ,α(u^η)I_D|Fτ1)≥I_D

Z τ1

τ

e^−βsf(X_s^τ,ξ,α, α_s)ds+e^−βτ1u^η(ξ₁)

=IDZ_τ^τ,ξ,α₁ (u^η). (3.5)

By (3.5) we have

E(Z_ρ^τ,ξ,α(u^η)|F_τ) =E(I_{ρ≤τ1}Z_ρ^τ,ξ,α(u^η)|F_τ) +E(I_{ρ>τ1}E(Z_ρ^τ,ξ,α(u^η)|F_τ1)|F_τ)

≥E(I_{ρ≤τ1}Z_ρ^τ,ξ,α(u^η)|F_τ) +E(I_{ρ>τ1}Z_τ^τ,ξ,α

1 (u^η)|F_τ)

=E(Z_ρ∧τ^τ,ξ,α₁(u^η)|Fτ). (3.6)

Put

U ={x∈G∩B_ε/2(x0) :ϕ^η(x)> u(x)}, H={ξ∈U} ∈Fτ.

On the stochastic intervalJτH,(ρ∧τ1)HKthe trajectories ofX^τ,ξ,αdo not leave the set B_ε/2(x0)∩G. Hence, we haveu^η(X_ρ∧τ^τ,ξ,α₁)IH ≥ϕ^η(X_ρ∧τ^τ,ξ,α₁)IHand

Z_ρ∧τ^τ,ξ,α₁(u^η)≥Z_ρ∧τ^τ,ξ,α₁(ϕ^η)IH+Z_ρ∧τ^τ,ξ,α₁(u)IH^c. (3.7) Apply Ito’s formula (2.10) on the intervalJτ, ρ∧τ₁K^withαinstead ofa. Taking the conditional expectation and using (3.2), we get

E(Z_ρ∧τ^τ,ξ,α₁(ϕ^η)IH|Fτ)≥e^−βτϕ^η(ξ)IH =e^−βτu^η(ξ)IH=Z_τ^τ,ξ,α(u^η)IH. (3.8)

Furthermore,

E(Z_ρ∧τ^τ,ξ,α₁(u)|Fτ)IH^c ≥Z_τ^τ,ξ,α(u)IH^c =Z_τ^τ,ξ,α(u^η)IH^c, (3.9) and the desired inequality

E(Z_ρ^τ,ξ,α(u^η)|Fτ)≥Z_τ^τ,ξ,α(u^η)

follows from (3.8), (3.9), combined with (3.6), (3.7).

4 The case of a smooth domain

LetGcoincide with the closure ofG^◦, and assume that∂Gis of classC². Then the distance functionρfrom∂G:

ρ(x) = inf{y∈G^c :|y−x|}, x∈G

is of classC² in a neighbourhood of∂G(see [14, Lemma 14.16]). Put−n(x) =Dρ(x), x∈G. Ifx∈∂G,n(x)is the unit outer normal to∂Gat x. It is shown in [1, Example 3.2], [2, Example 1] that condition (1.3) is reduced to the following: for any x ∈ ∂G there existsa∈Asuch that

σ^T(x, a)n(x) = 0, −n(x)·b(x, a) +1

2Tr σ(x, a)σ^T(x, a)D²ρ(x)

≥0.

To get a comparison result we need a stronger condition, presented in the next theorem.

Theorem 4.1.Assume that there exists a Borel measurable functionψ : G7→ Asuch that the functions (1.4) are globally Lipschitz continuous and

σ_ψ(x) = 0, −n(x)·b_ψ(x)>0, x∈∂G. (4.1) Then the value functionv, defined by (1.2), is the unique continuous viscosity solution of the state constrained problem (1.7).

Proof. The viscosity subsolutionw+, specified in Theorem 1.2, satisfies also the linear inequality

βw+(x)−f(x, ψ(x))−(bψ·Dw+)(x)−1

2Tr (σψσ_ψ^TD²w+)(x)≤0, x∈G^◦ (4.2) in the viscosity sense. Consider the function

we+(x) =

( lim sup

G^◦3y→x

w+(y) x∈∂G, w₊(x) otherwise.

Clearly,we+is a viscosity subsolution of (4.2), satisfying all conditions of Theorem 1.2.

Now we use conditions (4.1). By Lemma 4.1 of [3] the function we+ is a viscosity subsolution of (4.2) onG. Furthermore, by Theorem 4.1(ii) of [3], for anyx∈∂Gthere exists a sequencex_k∈G^◦,x_k→xsuch thatwe₊(x) = lim_k→∞we₊(x_k)and

lim sup

k→∞

|x_k−x|

d(xk) <∞, or, equivalently,

lim sup

k→∞

(xk−x)·n(x)

|x_k−x| ≤ −β

for some β ∈ (0,1). This is the nontangential upper semicontinuity property of we₊, which, by the comparison result of [21] (Theorem 2.2), implies that

we+≤u₋ onG. (4.3)

Let us prove that∂G= Γ. Forx∈∂Gdenote byX the solution of the equation Xt=x+

Z t

0

bψ(Xs)ds+ Z t

0

σψ(Xs)dWs, x∈∂G.

Since conditions (4.1) imply the viability, we get an admissible control αt = ψ(Xt): Xt=X_t^x,α∈G,t≥0a.s. Takeε >0such thatρ∈C²(Bε(x))and

inf

y∈Bε(x)∩G

−n(y)·b_ψ(y) +1

2Tr σ_ψ(y)σ^T_ψ(y)D²ρ(y)

>0.

Furthermore, putτ = inf{t≥0 :Xt6∈Bε(x)}. By Ito’s formula we have ρ(X_t∧τ) =ρ(x)−

Z t∧τ

0

n(Xs)·bψ(Xs)ds+1 2

Z t∧τ

0

Tr σψ(Xs)σ_ψ^T(Xs)D²ρ(Xs)

ds+Mt,

whereM is a continuous martingale with M0 = 0. From the representation ofM as a time-changed Brownian motion on an extended filtered probability space (see [16, Theorem 7.2’]) it follows that 0 is a limit point of the set{t > 0 : Mt = 0} a.s. For a sequencetk(ω)→0withMt_k= 0we have

ρ(Xt_k) =ρ(x) + Z t_k

0

−n(Xs)·bψ(Xs)ds+1

2Tr σψ(Xs)σ^T_ψ(Xs)D²ρ(Xs)

ds >0 a.s.

for sufficiently largek. Thus,X immediately entersG^◦:

inf{t >0 :Xt∈G^◦}= inf{t >0 :ρ(Xt)>0}= 0 a.s., and we conclude thatx∈Γand∂G= Γ.

This fact, together with Theorem 1.2 and inequality (4.3), implies that v≤we+≤u₋≤v on G.

Hence,v=we₊=u₋is a continuous function, and it satisfies (1.7) in the viscosity sense.

Note also that the uniqueness of a continuous constrained viscosity solution is a more classical result: see [12, Theorem 7.10].

Theorem 4.1 is similar to Theorem 4.1 of [21]. Although, the second condition (4.1) is presented there in the form

−n(x)·b_ψ(x) +1

2Tr σ_ψ(x)σ^T_ψ(x)D²ρ(x)

≥c >0, x∈∂G,

which is formally not comparable to ours local condition−n(x)·bψ(x)>0,x∈∂G, the result of [21] is more sophisticated. To get the comparison result in Theorem 4.1 we used only the fact that any subsolution, being suitably modified at the boundary points, possesses the nontangential upper semicontinuity property under conditions (4.1). In [21] it is shown that a subsolutionu≥v with this property exists even some diffusion in the tangent direction to∂Gis allowed: see conditions A3 of [21].

Certainly, the stochastic Perron method can be applied in the case of finite horizon as well. However, some work is required to study the parabolic problem, corresponding

to (1.7). In particular, a new boundary condition at the terminal time appears, and the viability notion should be modified. Such a problem was studied in [10] by another methods. We mention a comparison result, ensuring the continuity of the value function, proved under conditions similar to (4.1): see [10, Theorem A.1].

Acknowledgments.The author thanks the anonymous referees for useful remarks and for pointing out an error in the previous proof of Lemma 2.1. The research is supported by Southern Federal University, project 213.01-07-2014/07.

References

[1] M. Bardi and P. Goatin,Invariant sets for controlled degenerate diffusions: a viscosity so- lutions approach, Stochastic Analysis, Control, Optimization and Applications (W.M. McE- neaney, G.G. Yin, and Q. Zhang, eds.), Systems & Control: Foundations & Applications, Birkhäuser Boston, 1999, pp. 191–208. MR-1702960

[2] M. Bardi and R. Jensen,A geometric characterization of viable sets for controlled degener- ate diffusions, Set-Valued Analysis10(2002), no. 2-3, 129–141. MR-1926377

[3] G. Barles and E. Rouy, A strong comparison result for the Bellman equation arising in stochastic exit time control problems and its applications, Comm. Partial Differential Equa- tions22(1998), no. 11-12, 1995–2033. MR-1662164

[4] R.F. Bass, Stochastic processes, Cambridge University Press, Cambridge, 2011. MR- 2856623

[5] E. Bayraktar and M. Sîrbu,Stochastic Perron’s method and verification without smoothness using viscosity comparison: the linear case, Proc. Amer. Math. Soc.140 (2012), no. 10, 3645–3654. MR-2929032

[6] ,Stochastic Perron’s method for Hamilton-Jacobi-Bellman equations, SIAM J. Control Optim.51(2013), no. 6, 4274–4294. MR-3124891

[7] ,Stochastic Perron’s method and verification without smoothness using viscosity com- parison: Obstacle problems and Dynkin games, Proc. Amer. Math. Soc.142(2014), no. 4, 1399–1412. MR-3162260

[8] E. Bayraktar and Y. Zhang,Stochastic Perron’s method for the probability of lifetime ruin problem under transaction costs, Preprint arXiv:1404.7406v1 [math.OC], 24 pages, 2014.

[9] K. Bichteler,Stochastic integration andL^p-theory of semimartingales, Ann. Prob.9(1981), no. 1, 49–89. MR-0606798

[10] B. Bouchard and M. Nutz,Weak dynamic programming for generalized state constraints, SIAM J. Control Optim.50(2012), no. 6, 3344–3373. MR-3024163

[11] D. Buckdahn, M. Goreac, and M. Quincampoix,Stochastic optimal control and linear pro- gramming approach, Appl. Math. Optim.63(2011), no. 2, 257–276. MR-2772196

[12] M. Crandall, H. Ishii, and P.-L. Lions,User’s guide to viscosity solutions of second-order partial differential equations, Bull. Amer. Math. Soc.27(1992), no. 1, 1–67. MR-1118699 [13] C. Dellacherie and P.-A. Meyer, Probabilities and potential, North-Holland Mathematics

Studies, vol. 29, North-Holland, Amsterdam, 1978. MR-0521810

[14] D. Gilbarg and N.S. Trudinger, Elliptic partial differential equations of second order, Springer, Berlin, 2001. MR-1814364

[15] S.W. He, J.G. Wang, and J.A. Yan,Semimartingale theory and stochastic calculus, Science Press, Beijing, New York, 1992. MR-1219534

[16] N. Ikeda and S. Watanabe,Stochastic differential equations and diffusion processes, 2nd ed., North-Holland, Amsterdam, 1989. MR-1011252

[17] H. Ishii, Perron’s method for Hamilton-Jacobi equations, Duke Math. J.55(1987), no. 2, 369–384. MR-0894587

[18] H. Ishii and P. Loreti,A class of stochastic optimal control problems with state constraint, Indiana Univ. Math. J.51(2002), no. 5, 1167–135. MR-1947872

[19] R.L. Karandikar,Pathwise solutions of stochastic differential equations, Sankhya Ser. A43 (1981), no. 2, 121–132. MR-0666376

[20] ,On pathwise stochastic integration, Stoch. Proc. Appl.57(1995), no. 1, 11–18. MR- 1327950

[21] M.A. Katsoulakis,Viscosity solutions of second order fully nonlinear elliptic equations with state constraints, Indiana Univ. Math. J.43(1994), no. 2, 493–519. MR-1291526

[22] N.V. Krylov,Controlled diffusion processes, Springer, New York, 1980. MR-0601776 [23] J.M. Lasry and P.L. Lions, Nonlinear elliptic equations with singular boundary conditions

and stochastic control with state constraints, Math. Ann.283(1989), no. 4, 583–630. MR- 0990591

[24] D.B. Rokhlin,Verification by stochastic Perron’s method in stochastic exit time control prob- lems, J. Math. Anal. Appl.419(2014), no. 1, 433–446. MR-3217159

[25] M. Sîrbu,Stochastic Perron’s method and elementary strategies for zero-sum differential games, Preprint arXiv:1305.5083 [math.OC], 17 pages, 2013. MR-3206980

[26] ,Asymptotic Perron’s method and simple Markov strategies in stochastic games and control, Preprint arXiv:1402.7030 [math.OC], 12 pages, 2014.

[27] H.M. Soner,Optimal control with state-space constraint. I, SIAM J. Control Optim.24(1986), no. 3, 552–561. MR-0838056

[28] D.W. Stroock, Probability theory. an analytic view, 2nd ed., Cambridge University Press, Cambridge, 2011. MR-2760872

[29] D.W. Stroock and S.R.S. Varadhan, Multidimensional diffusion processes, Springer, New York, 1979. MR-0532498

Electronic Communications in Probability

Advantages of publishing in EJP-ECP

• Very high standards

• Free for authors, free for readers

• Quick publication (no backlog)

Economical model of EJP-ECP

• Low cost, based on free software (OJS

)

• Non profit, sponsored by IMS

, BS

, PKP

• Purely electronic and secure (LOCKSS

)

Help keep the journal free and vigorous

• Donate to the IMS open access fund

(click here to donate!)

• Submit your best articles to EJP-ECP

• Choose EJP-ECP over for-profit journals

1OJS: Open Journal Systemshttp://pkp.sfu.ca/ojs/

2IMS: Institute of Mathematical Statisticshttp://www.imstat.org/

3BS: Bernoulli Societyhttp://www.bernoulli-society.org/

4PK: Public Knowledge Projecthttp://pkp.sfu.ca/

5LOCKSS: Lots of Copies Keep Stuff Safehttp://www.lockss.org/