QUADRATIC CONTROL OF AFFINE DISCRETE-TIME, PERIODIC SYSTEMS WITH INDEPENDENT RANDOM PERTURBATIONS

Nova S´erie

QUADRATIC CONTROL OF AFFINE DISCRETE-TIME, PERIODIC SYSTEMS WITH INDEPENDENT

RANDOM PERTURBATIONS

Viorica Mariela Ungureanu

Abstract: In this paper we consider the affine discrete-time, periodic systems with independent random perturbations and we solve, under stabilizability and uniform observability or detectability conditions, the discrete time version of the quadratic control problem introduced in [1].

1 – Introduction

We consider the quadratic control problem for the affine discrete-time, peri- odic systems with independent random perturbations in Hilbert spaces (see [1] for continuous time case). The existence of an optimal control is connected with the behavior of the discrete-time Riccati equation associated with this problem. We study the asymptotic behavior of the solutions of the Riccati equation and we find an optimal control. In 1974 J. Zabczyk [10] treated a similar problem for time homogeneous systems and proved that, under stabilizability and detectability conditions, the Riccati equation (14) has a unique nonnegative bounded solu- tion. In connection with this problem, he also introduced the notion of stochastic observability (which is equivalent, in the finite dimensional case, with the one

Received: March 30, 2004; Revised: October 12, 2004.

AMS Subject Classification: 93E20, 49N10, 39A11.

Keywords: exponential stability; detectability and observability; quadratic control; Riccati equation.

This work was supported by CNCSIS (National Council for Research in High Education) under grant no 349/2004.

considered in this paper). The case of time varying systems in finite dimensions has been investigated by T.Morozan in [6]. He proved that, under uniform ob- servability and stabilizability conditions, the discrete-time Riccati equation has a unique, uniformly positive, bounded on N solution. In this paper we gener- alize the results of T. Morozan. We also establish that, in the stochastic case, the uniform observability does not imply the detectability and, consequently, our result is different from that obtained by J. Zabczyk in the time invariant case.

In [1] G. Da Prato and I. Ichikawa proposed a quadratic control problem for affine periodic systems (for both deterministic and stochastic cases), which is a generalization of the average cost criterion, usually considered for time-invariant systems. They proved that, under stabilizability and detectability conditions, the optimal control is given by a periodic feedback, which involves the periodic solution of the Riccati equation associated to this problem. In [9] we consider differential linear stochastic equations. We replace the detectability condition with the uniform observability property and, under stabilizability condition, we prove that the Riccati equation has a unique, uniformly positive, bounded on R⁺ solution, which is stabilizing for the controlled system. This result can be used to find the optimal control and the optimal cost for the quadratic control problem. We also proved in [9] that, in the stochastic case, uniform observabil- ity does not imply detectability, as in the deterministic case, and our result is different from the one of G. Da Prato. On the other hand, we note that the observability property is easier to verify than the detectability condition, both in the continuous and deterministic cases. So, the results of this paper are (in a certain sense) the discrete-time versions of those obtained in [9] and [1] for the continuous case. They are not obtained by a simple discretization of the results mentioned above (for example the algebraic Riccati equation, involved in the time invariant quadratic control problem, is not the same in the discrete-time (see (32)) and continuous cases (see [1])). There are many technical differences between the discrete time and the continuous cases. For example, in the discrete time case, we used the induction to prove the existence of the solution of the Riccati equation with final condition, while in the continuous case we work with specific properties of the functions, which are continuously time dependent.

2 – Notations and statement of the problem

Let H, V, U be separable real Hilbert spaces and let us denote by L(H, V) the Banach space of all bounded linear operators which transform H into V.

IfH=V we put L(H, V) =L(H). We writeh., .i for the inner product andk.k for norms of elements and operators. IfA∈L(H) thenA^∗ is the adjoint operator ofA. The operatorA∈L(H) is said to be nonnegative and we writeA≥0, ifAis self-adjoint andhAx, xi ≥0 for allx∈ H. We denote byHthe Banach subspace of L(H) formed by all self-adjoint operators, by K the cone of all nonnegative operators ofHand byI the identity operator onH. We also consider the Banach space C_b(H) = {ϕ: H → R, ϕ is bounded and continuous}. Let τ ∈ N, τ >1.

The sequence L_n ∈ L(H, V), n ∈ N is bounded on N if sup

n∈N kL_nk < ∞ and is τ-periodic ifLn=Ln+τ for all n∈N.

Let (Ω,F, P) be a probability space and ξ be a real or H -valued random variable on Ω. We writeE(ξ) for mean value (expectation) of ξ. We will use the notationB(H) for the Borel σ-field ofH.

Let us consider the sequence ξn, n ∈ Z of real independent random vari- ables, which satisfy the conditions E(ξn) = 0 and E|ξn|² = bn < ∞. If Fn is the σ-algebra generated by {ξ_i, i ≤ n−1}, then we will denote by L²_n(H) = L²(Ω,F_n, P, H) the space of all equivalence class ofH-valued random variablesη (i.e.ηis a measurable mapping from (Ω,F_n) into (H,B(H))) such thatEkηk² <

∞. Analogously we define L²(Ω,F, P, H) and we denote it L². We introduce the controlled system

(xn+1 =Anxn+ξnBnxn+Dnun+fn

x_k=x∈H,k∈N (1)

whereAn, Bn ∈L(H), Dn∈L(U, H). The control {u_k, u_k+1, ...} belongs to the classU^ek defined by the property that un,n≥k is anU-valued random variable, F_n-measurable and sup

n≥k

Eku_nk²<∞. For everyx∈H andk∈N, fixed, we will denote by U_k,x the subset of admissible controls from U^e_k with the property that (1) has a bounded solution.

If fn = 0 for all n ∈ N we use the notation {A : D, B} for the system (1).

In the sequel we need the hypotheses:

H0: The sequences An, Bn ∈ L(H), Dn ∈ L(U, H), Cn ∈ L(H, V), Kn ∈ L(U),f_n, b_n, n∈Nare bounded on Nand

K_n≥δI, δ >0 for all n∈N. (2)

H₁: The sequencesA_n, B_n, D_n, C_n, K_n, f_n, n∈N, b_n, n∈Zintroduced above areτ-periodic.

If H₀ (respectively H₁) holds we will use the notation Z^e =sup

n∈NkZ_nk(respec- tivelyZ^e= max

n= 0,1,...,τ−1kZ_nk) for Z =A, B, D, C, F, K, b.

Assuming the hypotheses H₀, H₁ we study the following problem:

For everyk∈Nandx∈H, we look for an optimal control u={u_k, u_k+1, ...}, which belongs to the classU_k,x and minimizes the following quadratic cost

I_k(x, u) = lim

q→∞

1 q−kE

q−1X

n=k

[kC_nx_nk²+< K_nu_n, u_n>], (3)

wherex_n is the solution of (1) for all n∈N,n≥k. (It is clear that if u ∈U_k,x thenI_k(x, u)<∞).

We will establish that under stabilizability and uniform observability (or de- tectability) conditions (see Theorem 26) the optimal cost, given by (28), is ob- tained for the optimal control (29).

3 – Preliminaries

3.1. Properties of the solutions of the linear discrete time systems We associate to (1) the linear stochastic system {A, B}

(x_n+1 =A_nx_n+ξ_nB_nx_n x_k=x∈H, n, k∈N. (4)

The random evolution operator of (4) is the operator X(n, k) n ≥ k ≥ 0, whereX(k, k) =I andX(n, k) = (An−1+ξn−1Bn−1)...(A_k+ξ_kB_k), for all ˙n > k.

Definition 1. A sequence {x_n}, n ∈ Z of H-valued random variables is τ-periodic (τ ∈N, τ >1) if

P{x_n1+τ ∈A₁, ..., x_nm+τ ∈A_m}=P{x_n1 ∈A₁, ..., x_nm ∈A_m}, (5)

for alln₁, n₂, ..., n_m ∈Z and allA_p∈ B(H), p= 1, .., m.

It is known that (5) is equivalent with

Eϕ(x_n1+τ, ..., x_nm+τ) =Eϕ(x_n1, .., x_nm), for allϕ∈C_b(H^m).

Remark 2. Assume thatH₁holds and the sequence{ξ_n}, n∈Zisτ-periodic.

There exist the functions F_n,k : R^n−k → H measurable (B(R^n−k),B(H)) such thatX(n, k)x=F_n,k(ξ_n−1, ..., ξ_k), X(n+τ, k+τ)x =F_n+τ,k+τ(ξ_n−1+τ, ..., ξ_k+τ) andF_n+τ,k+τ =F_n,k. Since the random variablesξ_n, n∈Z are independent and τ-periodic, then it follows that the random variablesX(n, k)xandX(n+τ, k+τ)x have the same distribution function for alln≥k, n, k∈Z.

If xn = xn(k, x) is the solution of the system (4) then it is unique and xn(k, x) =X(n, k)x.

It is not very difficult to see that we have the following lemma:

Lemma 3. X(n, k)is a bounded linear operator fromL²_k(H)intoL²_n(H)and we have

EkX(n, k)(ξ)k² ≤(kA_n−1k²+b_n−1kB_n−1k²)...(kA_kk²+b_kkB_kk²)Ekξk² for alln > k and ξ ∈L²_k(H).

From the above considerations it is clear that (1) has a unique solution x_n(x, k;u). Using the induction it follows thatx_n(x, k;u) satisfies the relation

x_n(x, k;u) =X(n, k)x+

n−1X

i=k

X(n, i+ 1) (D_iu_i+f_i) (6)

forn≥k+ 1. Moreover,xn(x, k;u) isFn-measurable andξn-independent.

Now, we introduce the mappings Un, T(n, k) :H → H U_n(S) = A^∗_nSA_n+b_nB_n^∗SB_n,

T(n, k) = U_kU_k+1...U_n−1,for alln−1≥kand T(k, k) =I, (7)

whereI ∈L(H) is the identity operator. It is easy to see thatUnandT(n, k) are linear and bounded operators.

Theorem 4 ([8]). IfX(n, k) is the random evolution operator associated to (4), thenT(n, k)(K)⊂ K and we have

hT(n, k)(S)x, yi=EhSX(n, k)x, X(n, k)yi (8)

for allS ∈ H,n≥k≥0and x, y∈H. Moreover kT(n, k) (I)k=kT(n, k)k.

Remark 5. If H₁ holds then T(n, k) is τ-periodic that means T(n, k) = T(n+τ, k+τ) for alln≥k≥0.

3.2. Uniform exponential stability and uniform observability

Definition 6. We say that {A, B}is uniformly exponentially stable iff there existβ ≥1,a∈(0,1) andn0 ∈Nsuch that we have

EkX(n, k)xk² ≤βa^n−kkxk² (9)

for all n≥k≥n₀ andx∈H.

If B_n = 0 for all n ∈N, we obtain the definition of the uniform exponential stability of the deterministic system x_n+1 = A_nx_n, x_k = x ∈ H, n ≥ k denoted {A}.

Definition 7 ([4]). The deterministic system{A}is uniformly exponentially stable iff there exist β ≥ 1, a ∈ (0,1) and n₀ ∈ N such that we have kA_n−1A_n−2...A_kk ≤βa^n−k for alln≥k≥n₀.

It is easy to see that if{A, B}is uniformly exponentially stable then (9) holds forn₀ = 0. The following result is known [4] for the finite dimensional case but it is presented for the readers’ convenience.

Proposition 8. If H₁ holds and {A} is uniformly exponentially stable then the system

y_n=A^∗_ny_n+1+f_n (10)

has a uniqueτ-periodic solution.

Proof: Using the condition required by theτ-periodic sequences we can ex- tend the sequencesA_n, f_n for all n∈Z. Let us denote Y(n, k) =A^∗_kA^∗_k+1...A^∗_n−1 ifn 6=k and Y(k, k) =I (the identity operator). If {A} is uniformly exponen- tially stable then it is easy to see that there exist a∈ (0,1) and β >1 such as kY(n, k)^∗k=kY(n, k)k ≤βa^n−k. Hence the series ^P∞

p=nY(p, n)fp converges inH.

It is not difficult to see thaty_n= ^P∞

p=nY(p, n)f_p satisfies (10). FromH₁ it follows Y(p+τ, n+τ) =Y(p, n) for all p≥nand consequently

y_n+τ = X∞ p=n+τ

Y(p, n+τ)f_p+τ = X∞ p=n

Y(p+τ, n+τ)f_p =y_n.

Thusy_nis aτ-periodic solution of (10). Ify_nis anotherτ-periodic solution of (10) we have ^°°y_n+1−y_n+1^°° ≤ kY(n, k)k max

k = 0,..,τ−1 ky_k−y_kk ≤ βa^n−k max

k= 0,..,τ−1

ky_k−y_kk. As k → −∞ we get y_n+1 = y_n+1 for all n ∈ Z and the proof is complete.

Now we consider the discrete time stochastic system{A, B;C}formed by the system (4) and the observation relationz_n=C_nx_n, whereC_n∈L(H, V), n∈N. Definition 9 (see Definition 6 in [6]). We say that {A, B;C} is uniformly observable if there existn0∈Nand ρ >0 such that

k+nX0

n=k

EkC_nX(n, k)xk² ≥ρkxk² (11)

for allk∈N andx∈H.

If the stochastic perturbation is missing, that is B_n= 0 for alln∈N, we will use the notation{A, ;C} for the observed (deterministic) system. We have the following definition of the deterministic uniform observability (see [3] and [2]).

Definition 10. We say that {A, ;C} is uniformly observable iff there exist n₀ ∈ N and ρ > 0 such that ^k+nP0

n=k

kC_nA_n−1A_n−2...A_kxk² ≥ ρkxk² for all k ∈ N andx∈H.

Remark 11. It is not difficult to see that, in the time-invariant, finite di- mensional case, the deterministic system {A, ;C} is uniformly observable iff rank(C^∗, A^∗C^∗, ..., (A^∗)ⁿ⁻¹C^∗) =n, wheren is the dimension ofH.

The following proposition is a consequence of the Theorem 4.

Proposition 12 ([8]).

a) The system (4) is uniformly exponentially stable if and only if there exist β≥1,a∈(0,1)and n₀ ∈Nsuch that we have

kT(n, k)k ≤βa^n−k (12)

for all n≥k≥n₀.

b) The system {A, B;C} is uniformly observable if and only if there exist n₀ ∈Nand ρ >0 such that

k+nX0

n=k

T(n, k)(C_n^∗Cn)≥ρI (13)

for all k∈N.

Conclusion 13. From the above proposition it follows that if the determinis- tic system{A, ;C}is uniformly observable then the stochastic system{A, B;C}

is uniformly observable.

Proposition 14. Assume that H₁ holds, D_n = 0 for all n ∈ N and the sequence {ξ_n}, n ∈ Z is τ-periodic. If {A, B} is uniformly exponentially stable then the system (1) (without initial condition) has a uniqueτ-periodic solution inL².

Proof: As in the proof of the Proposition 8 we consider the system (1) onZ. Let us consider the series ^n−1P

p=−∞

X(n, p+ 1)f_p in the Hilbert spaceL². We have

°°

°°

°°

n−1X

p=−∞

X(n, p+ 1)f_p

°°

°°

°°

L²

≤

n−1X

p=−∞

kX(n, p+ 1)f_pk_L2

=

n−1X

p=−∞

q

EkX(n, p+ 1)fpk²

If T(n, k) is the operator associated to the system {A, B} according to the Theorem 4, we deduce by Remark 5 and Proposition 12 that (12) holds for all n≥k >−∞.

Using Theorem 4 and the above considerations we get

°°

°°

°°

n−1X

p=−∞

X(n, p+ 1)fp

°°

°°

°°_L2 ≤

n−1X

p=−∞

rDT(n, p+ 1)f_p, fp

E

≤

n−1X

p=−∞

β^1/2a^{(n−p−1)/2}f <^e ∞.

Consequently the series converges in L². We denote y_n = ^n−1P

p=−∞X(n, p+ 1)f_p. It is a simple exercise to verify that y_n satisfies (1). Now we will prove that it is a τ-periodic solution of (1). We consider the random variables y_n,m =

n−1P

p=mX(n, p+ 1)f_p and y_n+τ,m+τ = ^n−1P

p=mX(n+τ, p+τ + 1)f_p.

Since X(n, p + 1)f_p and X(n+τ, p+τ + 1)f_p have the same distribution functions for all n ≥ p+ 1 > m it is clear that the distributions of y_n,m and y_n+τ,m coincide.

Thus Eϕ(y_n,m) = Eϕ(y_n+τ,m+τ) for all ϕ ∈ C_b(H). Since ky_n,m−y_nk_L2

m−→−∞−→ 0 we deduce that there exists a subsequencey_n,mk such that y_n,mk con- verges toy_n P.a.s, ask−→ ∞.

Analogously, from ky_n+τ,mk+τ−y_n+τk_L2 −→

mk−→−∞0 it follows that there ex- ists a subsequence yn+τ,m_kh+τ such that yn+τ,m_kh+τ −→

h−→∞yn+τ P.a.s. We con- sider now the last subsequence and we denote y_n+τ,mkh+τ = y_n+τ,me

h+τ. It is clear that both sequencesy_n+τ,me

h+τ, y_n,me

h converges to their limitP.a.s and we deduce that ϕ(y_n,me

h) −→

h−→∞ ϕ(y_n) (respectively ϕ(y_n+τ,me

h+τ) −→

h−→∞ ϕ(y_n+τ)) P.a.s for all ϕ ∈ Cb(H). Using the Bounded Convergence Theorem it follows that Eϕ(y_n,me

h) −→

h−→∞ Eϕ(y_n) (respectively Eϕ(y_n+τ,me

h+τ) −→

h−→∞ Eϕ(y_n+τ)).

From Remark 2 we deduce thaty_n,me

h andy_n+τ,me

h+τ have the same distribution function and Eϕ(y_n,me

h) = Eϕ(y_n+τ,me

h+τ). Hence Eϕ(y_n) = Eϕ(y_n+τ) for all ϕ ∈ C_b(H) and y_n, y_n+τ have the same distribution function. Using the same way of proof it can be shown thaty_n1, y_n2, ..., y_nm andy_n1+τ, y_n2+τ, ..., y_nm+τ for all n1, n2, ..., nm ∈ Z have the same joint distribution functions and it follows thatyn isτ-periodic.

If zn∈L², n∈Z is anotherτ-periodic solution of (1) then we have Eky_n+1−z_n+1k² =EhT(n, k) (I) (y_k−z_k), y_k−z_ki

≤ kT(n, k)k max

p=0,..,τ−1Eky_p−z_pk² ≤β¹²a^n−k2 max

p=0,..,τ−1Eky_p−z_pk²

for alln≥k. Ask→ −∞ we gety_n+1 =z_n+1 P.a.s for all n∈Z and the proof is complete.

The above proposition is the infinite dimensional version of the statement i) of Theorem 3 from [5].

4 – Optimal quadratic control for affine discrete-time systems

In this section we assume that the hypothesis H0 holds.

4.1. The discrete-time Riccati equation of stochastic control and the uniform observability

We consider the transformation

G_n:K → K,G_n(S) =A^∗_nSD_n(K_n+D^∗_nSD_n)⁻¹D_n^∗SA_n,

which is well defined. Let U_n∈L(H) be the linear operator defined by (7).

We consider the following Riccati equation

R_n=U_n(R_n+1) +C_n^∗C_n− G_n(R_n+1) (14)

onK, connected with the quadratic cost (3).

Definition 15. A sequence {R_n}_n∈N, R_n ∈ K such as (14) holds is said to be a solution of the Riccati equation (14).

We need the following definitions (see D.3 from [6]).

Definition 16. A solutionR = (R_n)_n∈N of (14) is said to be stabilizing for {A:D, B} if{A+DF, B}with

Fn=−(Kn+D^∗_nRn+1Dn)⁻¹D^∗_nRn+1An, n∈N (15)

is uniformly exponentially stable.

Definition 17 ([6]). The system {A :D, B} is stabilizable if there exists a bounded onN sequence F = {F_n}_n∈N, F_n ∈L(H, U) such that {A+DF, B} is uniformly exponentially stable.

Proposition 18. The Riccati equation (14) has at most one stabilizing and bounded onNsolution.

Proof: Let R_n,1 and R_n,2 be two stabilizing and bounded onN solutions of equation (14). We introduce the systems

(x_n+1,i= (A_n+D_nF_n,i)x_n,i+ξ_nB_nx_n,i x_k,i =x∈H

(16)

for alln≥k,n, k∈N, whereFn,i=−(Kn+D^∗_nRn+1,iDn)⁻¹D_n^∗Rn+1,iAn,i= 1,2.

If we denoteQn=Rn,1−Rn,2, we get

EhQ_n+1x_n+1,1, x_n+1,2i=EhQ_nx_n,1, x_n,2i, for all n≥k.

It is easy to see that EhQn+1xn+1,1, xn+1,2i=hQ_kx, xi for all n≥k,x∈H.

Since R_n,i, i= 1,2 are bounded onNwe deduce that there existsM >0 such thatkQ_nk ≤M for alln∈N. Thus,

0≤ |hQ_kx, xi| ≤M q

Ekx_n+1,1k²Ekx_n+1,2k².

From the hypothesis and from the Definition 17 it follows that the systems (16) are uniformly exponentially stable and Ekxn+1,ik² →

n→∞ 0, i = 1,2, uniformly with respect tox.

As n→ ∞ in the last inequality, we deduce thatQ_k= 0 andR_k,1 =R_k,2 for allk∈N. The proof is complete.

Letx_n be the solution of system{A:D, B}. ByU_k,M, M ∈N^∗ we denote the set of all finite sequencesu^M_k ={u_k, u_k+1, ...u_M−1}ofU-valued andF_imeasurable random variablesu_i,i=k, ..., M −1 with the property Eku_ik² <∞. Now, we introduce the performance

V(M, k, x, u) =E

MX−1 n=k

[kC_nx_nk²+< K_nu_n, u_n>].

Let us consider the sequence R(M, M) = 0∈ K,

R(M, n) =U_n(R(M, n+ 1)) +C_n^∗C_n− G_n(R(M, n+ 1)) for alln≤M−1.

The following lemma prove that the sequence R(M, n) is well defined for all 0≤n ≤M. It is called the solution of the Riccati equation (14) with the final conditionR(M, M) = 0.

Lemma 19.

a) R(M, n)∈ K for all0≤n≤M;

b) 0≤R(M−1, n)≤R(M, n) for all0≤n≤M−1.

Moreover, if H₁ is satisfied then

R(M +τ, n+τ) =R(M, n),0≤n≤M.

(17)

Proof: We will prove the first assertion by induction. Forn=M,R(M, n) = 0∈ K. Let us assume R(M, n)∈ Kfor all n∈N, k < n≤M.

We will prove R(M, k) ∈ K. Let x_n be the solution of system {A : D, B}

with the initial condition x_k = x and let us denote F_n = −[K_n+D_n^∗R(M, n+ 1)D_n]⁻¹D_n^∗R(M, n+ 1)A_n and z_n=u_n−F_nx_n. We have

EhR(M, n+ 1)x_n+1, x_n+1i=

EhR(M, n)x_n, x_ni −EhC_n^∗C_nx_n, x_ni −EhK_nu_n, u_ni+ Eh(Kn+D_n^∗R(M, n+ 1)Dn)zn, zni.

Now, we consider the last equality for n =k, k+ 1, ..., M −1 and summing, we obtain

V(M, k, x, u) = hR(M, k)x, xi+ E

M−1X

n=k

h(K_n+D_n^∗R(M, n+ 1)D_n)z_n, z_ni (18)

Let x_en be the solution of system

(x_n+1= (A_n+D_nF_n)x_n+ξ_nB_nx_n xk=x∈H , (19)

whereFn was introduced above.

It is clear that xen is also the solution of{A:D, B}withuen=Fnxen,k≤n≤ M−1 and {u_en, k≤n≤M −1} ∈U_k,M.

Thus we obtain, for all 0≤k < M

u∈Umink,M

V(M, k, x, u) =V(M, k, x,u) =_e hR(M, k)x, xi. (20)

We deduce thatR(M, k)≥0 and the induction is complete.

b) Let u^M_k ⁻¹ = {ue_k,ue_k+1, ...,ue_M−2}. It is clear that u^M−1_k ∈ U_k,M−1 and from the definition ofV(M, k, x, u) we get V(M−1, k, x, u)≤V(M, k, x,u).e

If we consider (20) for M−1, we have, for all 0≤k < M hR(M−1, k)x, xi= min

b

u∈Uk−1,M

V(M−1, k, x,u)_b ≤V(M−1, k, x, u).

From (20) and the last inequalities it follows the conclusion. The proof of last statement is trivial.

Proposition 20. Assume that {A:D, B} is stabilizable. Then the Riccati equation (14) admits a bounded onNsolution. IfH₁is satisfied then the solution of the Riccati equation isτ-periodic.

Proof: Since{A:D, B}is stabilizable it follows that there exists a bounded onN sequenceF ={F_n}_n∈N,F_n∈L(H, U) such that{A+DF, B} is uniformly exponentially stable.

Let us consideru_n=F_nx_n, wherex_nis the solution of{A+DF, B} with the initial condition x_k=x. SinceF_nis bounded onN, it is not difficult to see that un∈U^e_k. We have

V(M, k, x, u)≤η X∞

n=k

Ekx_nk²

for allM > k, whereη=C^e2+K^fF^e2. Since{A+DF, B}is uniformly exponentially stable, it is not difficult to see that there existsλ₁>0 such thatV(M, k, x, u)≤ ηλ1kxk² =λkxk², x∈H.

Let R(M, n) be the solution of the Riccati equation (14) withR(M, M) = 0.

Using (20) and the above inequality, we deduce that hR(M, k)x, xi ≤λkxk².

Using Lemma 19 it follows that there exists R(k) ∈ L(H) such that 0 ≤ R(M, k) ≤ R(k) ≤ λI for M ∈ N, M ≥ k and the sequence {R(M, k)}_M∈N,M≥k converges toR(k) in the strong operator topology.

We denote L= lim

M→∞(<Gn(R(M, n+ 1))x, x >−<Gn(R(n+ 1))x, x >) and PM,n=Kn+D_n^∗R(M, n+ 1)Dn,Pn=Kn+D^∗_nR(n+ 1)Dn. If

L₁ = lim

M→∞

°°

°P_M,n^{−1 °°}_°kD_n^∗R(M, n+ 1)A_nx−D^∗_nR(n+ 1)A_nxk · kD_n^∗R(M, n+ 1)A_nx+D^∗_nR(n+ 1)A_nxk and

L2 = lim

M→∞

D³P_M,n⁻¹ −P_n^−1´D_n^∗R(n+ 1)Anx, D_n^∗R(n+ 1)Anx^E,

then

|L| ≤L₁+L₂

SinceP_M,n≥K_n≥δI,δ >0 we deduce that^°°_°P_M,n^{−1 °°}_°≤ ¹_δ for allM ≥n+1≥k and from the strong convergence of{R(M, n)}_{M∈N, M≥n} it followsL₁= 0.

We see that ^°°_°P_M,n⁻¹ x−P_n⁻¹x^°°_° ≤ ^°°_°P_M,n^{−1 °°}_°kP_M,ny−P_nyk, where y = P_n⁻¹x.

Since lim

M→∞kP_M,ny−P_nyk= 0 we get lim

M→∞

°°

°P_M,n⁻¹ x−P_n⁻¹x^°°_°= 0.

Now it is clear that L₂ = 0. Hence L= 0 and

Mlim→∞hG_n(R(M, n+ 1))x, xi=hG_n(R(n+ 1))x, xi.

From the definition of R(M, n) and the above result we deduce that R(n) is a solution of (14). IfH₁ holds then we take M → ∞in (17) and it follows that R(n) isτ-periodic.

Theorem 21. Let us assume that the system{A, B;C}is uniformly observ- able. IfRn is a nonnegative bounded onN solution of (14) then:

a) there exist m > 0 such that R_n ≥ mI, for all n ∈ N (R_n is uniformly positive onN).

b) R_nis stabilizing for (1).

Proof: The main idea is the one of [6] .

Let Rn be a nonnegative, τ-periodic solution of (14) and let X(n, k) be the^e random evolution operator associated to system{A+DF, B} with

F_n=−(K_n+D^∗_nR_n+1D_n)⁻¹D_n^∗R_n+1A_n. (21)

Letn₀ andρ be the number introduced by the Definition 9. We have (see the proof of Lemma 19)

hT_nx, xi = hR_nx, xi −

E^DR_n0+n+1X(n^e ₀+n+ 1, n)x,X(n^e ₀+n+ 1, n)x^E, (22)

where the operatorT_n∈ His hT_nx, xi=

n+nX0

j=n

(E^°°_°C_jX(j, n)x^{e °°}_°²+E^DK_jF_jX(j, n)x, F^e _jX(j, n)x^{e E}).

(23)

From (6), we deduce that for all j≥n+ 1 we have X(j, n)xe =X(j, n)x+

j−1X

i=n

X(j, i+ 1)Diuei,

whereue_i =F_iX(i, n)x^e andX(n, k) is the random evolution associated to{A, B}.

Thus

hT_nx, xi >

n+nX0

j=n+1

E

°°

°°

°°C_jX(j, n)x+C_j

j−1X

i=n

X(j, i+ 1)D_iue_i

°°

°°

°°

2

+1

2kC_nxk²

≥ 1 2(

n+nX0

j=n

EkCjX(j, n)xk²)−C^e2

n+nX0

j=n+1

E

°°

°°

°°

j−1X

i=n

X(j, i+ 1)Diuei

°°

°°

°°

2

.

UsingH0, Lemma 3 and (23) it follows E

°°

°°

°°

j−1X

i=n

X(j, i+ 1)Diuei

°°

°°

°°

2

≤D^e2µn0

n+nX0

i=n

E^°°_°FiX(i, n)xe ^°°_°²≤chTnx, xi,

whereµ_n0 =^³n₀max{1,(A^e2+^ebB^e2)ⁿ⁰}^´ andc= ^De2_δ^µn0.

Since the system {A, B;C}is uniformly observable then we have hTnx, xi> 1

2ρkxk²−C^e2n0chTnx, xi).

From the last equality and from the hypothesis we deduce that there exist M > msuch that

mkxk² ≤ hT_nx, xi ≤ hR_nx, xi ≤Mkxk². (24)

We obtain from (22) and (24)

−m/MhR_nx, xi ≥ − hR_nx, xi+

E^DRn0+n+1X(ne 0+n+ 1, n)x,X(n^e 0+n+ 1, n)x^E.

ThusE^DR_n0+n+1X(n^e ₀+n+ 1, n)x,X(n^e ₀+n+ 1, n)x^E≤qhR_nx, xi for all n∈ Nand x∈H, whereq = 1−m/M, q∈(0,1).

Let T^e(n, k) be the operator introduced by Theorem 4 for the system {A+ DF, B}, where the sequence F_n is given by (21). Then the previous inequality can be written

Te(n₀+n+ 1, n) (R_n0+n+1)≤qR_n.

Since T^e(n, k) is monotone (T^e(n, k)(P) ≤ T^e(n, k)(R) for P ≤ R, n ≥ k) we deduce from (24) thatT^e(n, k)^³T^e(n₀+n+ 1, n) (R_n0+n+1)^´≤qT^e(n, k)(R_n) and T(ne 0+n+ 1, k) (Rn0+n+1)≤qT^e(n, k)(Rn) for all n≥k.

Letn≥karbitrary. Then there existsc, r∈Nsuch thatn−k= (n0+ 1)c+r and 0≤r ≤n0. We obtain by induction:

Te(n, k)(R_n)≤q^cT^e(r+k, k)(R_r).

From (24) and Theorem 4 we get mT^e(n, k)(I)≤M q^c°°_{° e}X(r+k, k)^°°_°²I.

Using Lemma 3 we putG=M max

0≤r≤n0

{(A^e2+^ebB^e2)^r}and we getmT^e(n, k)(I)≤ q^cGI. Now we take a = q^1/(n0+1), b = q^−n0/(n0+1)(G/m) ≥ 1 and it follows T(n, k)(I)e ≤ba^n−kI.

From Theorem 4 we deduce E^°°_{° e}X(n, k)x^°°_°² ≤ ba^n−kkxk² for all x ∈ H and 0≤k≤n,k, n∈N. Therefore R_n is stabilizing for (1). The proof is complete.

Now, we can state the main result of this section.

Theorem 22. Assume that

1) the system{A:D, B} is stabilizable and 2) the system{A, B;C}is uniformly observable.

Then the Riccati equation (14) admits a unique uniformly positive, bounded on N and stabilizing solution. Moreover, if H₁ holds then the solution of the Riccati equation isτ-periodic.

Proof: From the Proposition 20 and the assumption 1) we deduce that (14) admits a nonnegative, bounded on N (or τ-periodic, if H₁ holds) solution.

Now, using the above theorem and 2), we deduce that this solution is stabilizing.

A stabilizing and bounded on N solution of the Riccati equation is unique by Proposition 18. The proof is complete.

The above theorem is proved in [6] for the discrete time stochastic systems in finite dimensional spaces. The continuous case, for stochastic systems on infinite dimensional spaces, is treated in [9].

Definition 23. The system{A, B;C}is detectable if there exists a bounded onN sequence P ={P_n}_n∈N, P_n ∈L(U, H) such that {A+P C, B}is uniformly exponentially stable.

The next result is the infinite dimensional version of Proposition 9 from [7], where we replace the Markov perturbations with independent random perturba- tions. So, it can be proved similarly the following proposition.

Proposition 24. If{A, B;C}is detectable then every nonnegative bounded solution of (14) is stabilizing.

Now, it is clear that if we replace the observability condition in Theorem 22 with the detectability property we deduce that the Riccati equation (14) has a unique nonnegative, bounded on N (τ-periodic, if H1 holds) and stabilizing solution. The obtained result is already known for the time invariant case (see [10]) and for the continuous, time-varying case (see [1]).

We only will prove that observability does not imply detectability and it fol- lows that our result is different to those mentioned above. Before to give the counter-example, which will solve this problem we need the following remarks.

Remark 25. Let us consider the time invariant caseA_n=A,B_n=B,b_n=b, C_n=C and K_n=I. It is not difficult to see that, in the finite dimensional case, the system {A, B;C} is detectable if and only if the controlled system {A^∗ : C^∗, B^∗} is stabilizable. Thus, it follows from Proposition 20 that if {A, B;C}

is detectable then the Riccati equation (14), where we replace the operators A with A^∗, B with B^∗, C with I, and D with C^∗ has a nonnegative bounded on N solution. Using Lemma 3.1 from [11] we deduce that the Riccati equation associated to the above detectable system becomes

R_n=A(R_n+1) (25)

whereA:K → K A(S) =bBSB^∗+I+AS(I+C^∗CS)⁻¹A^∗. By Proposition 20 it follows that if the system {A, B;C} is detectable then the algebraic Riccati equation

R=A(R) (26)

has a nonnegative solution.

The following counter-example prove that the stochastic observability doesn’t imply detectability.

Counter-example

Let us consider the stochastic system {A, B;C}whereH =R², V =R(R² is the real 2-dimensional space),A_n=A=

Ã1 0 0 2

!

, C_n=C=^³1 1^´,b_n= 1 and B_n=B =

Ã1 0 0 0

!

for alln∈N.

Since rank(C^∗, A^∗C^∗) = 2 then the deterministic system {A;C} [3] is ob- servable. Therefore (see Conclusion 13) the stochastic system{A, B;C} is uni- formly observable. It is easy to see that if we look for a solutionK=

Ãx₁ x₂ x₂ x₃

!

of (26), which satisfies the conditions x₁x₃ ≥ x²₂, x₁ ≥ 0, we obtain x²₂ = 3x3x1+ 3x3+x1+ 1≥3x²₂+ 1, that is impossible. Thus the equation (26) has not a nonnegative solution. Then, from Remark 25, we deduce that {A, B;C}

cannot be detectable.

4.2. Optimal quadratic control and the uniform observability The following theorem gives the optimal control, which minimize the cost function (3). LetH₀ and H₁ hold.

Theorem 26. Assume that the hypothesis 1) of the Theorem 22 holds and the system{A, B;C}is either uniformly observable or detectable. LetR_nbe the unique solution of Riccati equation (14). Ifgn is the unique τ-periodic solution of the Lyapunov equation

g_n= (A_n+D_nF_n)^∗g_n+1+R_nf_n−1 (27)

whereF_n is given by (21), then

u∈Umink,x

I_k(x, u) = I(x,u)e

= 1 τ

τ−1X

i=0

[2hg_i+1, f_ii −^°°_°V_i^−1/2D^∗_ig_i+1^°°_°²− hR_i+1f_i, f_ii], (28)

where the optimal control is

ue_n=−V_n⁻¹D^∗_n(R_n+1A_nx_en+g_n+1), (29)

n≥k≥0,xe_nis the solution of the system (1) and V_n=K_n+D^∗_nR_n+1D_n.

Proof: First we note that if the above hypotheses hold, then the Riccati equation (14) has a unique nonnegative, bounded onN and stabilizing solution R_n, according Theorem 22 and Proposition 24. Since the Riccati equation is stabilizing we can apply Proposition 12 to deduce that{A+DF}(see Definition 7) is uniformly exponentially stable, where F_{n, n∈N} is given by (21). Using the Proposition 8 it follows that (27) has a uniqueτ-periodic solution. Letxnbe the solution of the system (1) and let us consider the function

v_n:H→R, v_n(x) =hR_nx, xi+ 2hg_n+1,(A_n+D_nF_n)xi. Arguing as in the proof of Lemma 19 we have

Ev_n+1(x_n+1) = Ev_n(x_n)−E[kC_nx_nk²+hK_nu_n, u_ni]+

EhV_n(u_n−F_nx_n), u_n−F_nx_ni+

2EhD^∗_ng_n+1, u_n−F_nx_ni+ 2hg_n+1, f_ni − hR_n+1f_n, f_ni. If we put a_n=V_n⁻¹D_n^∗g_n+1+u_n−F_nx_nwe have

EhVnan, ani −E^°°_°V_n^−1/2D_n^∗gn+1

°°

°² = EhVn(un−Fnxn),(un−Fnxn)i+ 2EhD_n^∗g_n+1, u_n−F_nx_ni.

Hence we deduce that

Ev_n+1(x_n+1) = Ev_n(x_n)−E[kC_nx_nk²+hK_nu_n, u_ni] +EhV_na_n, a_ni −

°°

°V_n^−1/2D^∗_ng_n+1^°°_°²+ 2hg_n+1, f_ni − hR_n+1f_n, f_ni. (30)

Let xe_n be the solution of system (1), where ue_n = F_nxe_n−V_n⁻¹D_n^∗g_n+1. It is not difficult to see that exn and uen are bounded on N. Thus ue∈U_k,x.

Using (30) we get 1

n−k[v_k(x)−Evn+1(xen+1)] = 1 n−k

n−1X

i=k

E[kCixeik²+hKiuei,ueii]− 2hg_i+1, f_ii+^°°_°V_i^−1/2D^∗_ig_i+1^°°_°²+hR_i+1f_i, f_ii. (31)

Since g_n, R_n areτ periodic and R_n is stabilizing we deduce that there exists P >0 such that Ev_n+1(xe_n+1)≤P for all n∈N.

As n→ ∞ in (31), it follows I_k(x,u) = lime

n→∞

1 n−k

n−1X

i=k

[2hg_i+1, f_ii −^°°_°V_i^−1/2D_i^∗g_i+1^°°_°²− hR_i+1f_i, f_ii]

Thus

u∈Umini,x

I_k(x, u) ≤ I_k(x,u) = lim_e

n→∞

1 n−k

n−1X

i=k

[2hg_i+1, f_ii −

°°

°V_i^−1/2D_i^∗g_i+1^°°_°²− hR_i+1f_i, f_ii].

If u∈U_k,x it is not difficult to deduce from (30) that I_k(x, u)≥I_k(x,u).e Thus min

u∈Ui,x

I_k(x, u) = I_k(x,u). Usinge H₁ we see that for n = pτ +k then I_k(x,u) =e _τ^1τ−1P

i=0

[2hg_i+1, f_ii −^°°_°V_i^−1/2D^∗_ig_i+1^°°_°²− hR_i+1f_i, f_ii] and the conclusion follows.

From the above theorem it follows that the optimal cost does not depend on the initial condition. It is not difficult to see that the conclusions of the above theorem stay true if we consider the initial condition x_k = ξ ∈ L²_k(H). Thus, using Proposition 14 we have the following result:

Proposition 27. If the hypothesis of the Theorem 26 holds and the sequence {ξn}, n∈Zisτ-periodic, then the optimal cost is given by (28) and the optimal control is (29), where xe_n = ^n−1P

p=−∞

X(n, pe + 1)^³f_p−D_pV_p⁻¹D^∗_pg_p+1^´, g_p is the τ-periodic solution of (27) and X(n, k), n^e ≥k is the random evolution operator associated with the system{A+DF, B}considered on Z.

The time invariant case

In this subsection we work under the hypotheses H₀ and

H2: Zn=Z for all n∈N(orn∈Z) and Z=A, B, D, C, F, K, b, f. We consider the algebraic Riccati equation

R=U(R) +C^∗C− G(R), (32)

whereU(R) =A^∗RA+bB^∗RB andG(R) =A^∗RD(K+D^∗RD)⁻¹D^∗RA.

Remark 28. It is easy to see that if the hypotheses 1) and 2) of the Theo- rem 22 hold then

a) the algebraic equation (32) has a unique positive solution;

b) the system (10) has a unique time-invariant solution given by g=

X∞ p=0

(A^∗+F^∗D^∗)^pf.

(33)

Corollary 29. If the hypotheses of the Theorem 26 are verified then the Riccati equation (32) has a unique nonnegative solutionR and the optimal cost is

I(u) = 2_e hg, fi −^°°_°V^−1/2D^∗g^°°_°²− hRf, fi, (34)

whereg and the optimal control ue are given by (33) respectively (29) and V = K+D^∗RD.

REFERENCES

[1] Prato, G. Da and Ichikawa, A. – Quadratic Control for Linear Periodic Sys- tems,Appl. Math. Optim.,18 (1988), 39–66.

[2] Kwakernaak, H. and Sivan, R. –Linear Optimal Control Systems, New York, Wiley-Interscience, 1972.

[3] Ionescu, V. – Sisteme liniare (Linear Systems), Editura Academiei Republicii Socialiste Romˆania, Bucuresti, 1973.

[4] Halanay, A.; Morozan, T. and Tudor, C. – Tracking Discrete Almost- Periodic Signals under Random Perturbations, Int. J. Control,47(1) (1988), 381–

392.

[5] Morozan, T. –Bounded, Periodic and Almost Periodic Solutions of Affine Stochas- tic Discrete-Time Systems,Rev. Roumaine Math. Pures Appl.,32(8) (1987), 711–

718.

[6] Morozan, T. –Discrete time Riccati equations connected with quadratic control for linear systems with independent Random Perturbations,Rev. Roumaine Math.

Pures Appl., 37(3) (1992), 233–246.

[7] Morozan, T. – Stability and Control for Linear Discrete-time systems with Markov Perturbations, Rev. Roumaine Math. Pures Appl., 40(5–6) (1995), 471–

494.

[8] Ungureanu, V. – Uniform Exponential Stability for Linear Discrete Time Sys- tems with Stochastic Perturbations,Analele Universit˘at¸ii din Craiova, Seria Matem- atic˘a-Informatic˘a,xxviii (2001), 194–204.

[9] Ungureanu, V. –Riccati Equation of Stochastic Control and Stochastic Uniform Observability in Infinite Dimensions, in “Analysis and Optimization of Differential Systems”, Kluwer Academic Publishers, 2003, pp. 421–423.

[10] Zabczyk, J. – On Optimal Stochastic Control Of Discrete-Time Systems In Hilbert Space,SIAM J. Control,13(6) (1974), 1217–1234.

[11] Zabczyk, J. – Remarks on the Control of Discrete-Time Distributed Parameter Systems,SIAM J. Control,12(4) (1974), 721–735.

Viorica Mariela Ungureanu, Universitatea “Constantin Brˆancusi”,

B-dul Republicii, nr. 1, Tˆargu -Jiu, jud.Gorj, 210152 – ROM ˆANIA E-mail: vio@utgjiu.ro