2.1 Maximum principle with transversality conditions

curvilinear integral cost

Constantin Udri¸ste

Abstract.Recently we have created a multitime maximum principle gath- ering together some concepts in Mechanics, Field Theory, Differential Ge- ometry, and Control Theory. The basic tools of our theory are variational PDE systems, adjoint PDE systems, Hamiltonian PDE systems, duality, multitime maximum principle, incavity on manifolds etc. Now we jus- tify the multitime maximum principle for curvilinear integral cost using the m-needle variations. Section 1 recalls the multitime control theory and proves the equivalence between curvilinear integral costs and multi- ple integral costs. Section 2 formulates variants of multitime maximum principle using control Hamiltonian 1-forms produced by a curvilinear in- tegral cost and a controlledm-flow evolution. Section 3 refers to original proofs of the multitime maximum principle using simple and multiple mul- titimem-needle control variations. The key is to use completely integrable first order PDEs (controlled evolution and variational PDEs) and their ad- joint PDEs. Section 4 formulates and proves sufficient conditions that the multitime maximum principle be true.

M.S.C. 2010: 49J20, 49K20, 93C20.

Key words: multitime maximum principle, curvilinear integral cost, variational PDEs, adjoint PDEs,m-needle variations.

1 Multitime control theory

Themultitime control theoryis concerned with partial derivatives dynamical systems and their optimization over multitime [15]-[30]. Such problems are well-known also as the multidimensional control problems of Dieudonn´e-Rashevsky type [9], [10], [31], [32], but both techniques and results in these papers are different from ours. We confirm the expectations of Lev Pontryaguin, Lawrence Evans and Jacques-Louis Lions regarding the analogy between optimal control of systems governed by first order PDEsand optimal control of systems containing first order ODEs. The ideas we use were stimulated by the original point of view of Lawrence C. Evans [4] on

Balkan Journal of Geometry and Its Applications, Vol.16, No.1, 2011, pp. 128-149.∗

°c Balkan Society of Geometers, Geometry Balkan Press 2011.

(single-time) Pontryaguin maximum principle [11] and by the papers [1]-[3], [5]-[10], [31], [32].

The multidimensional optimal control arise in the description of torsion of pris- matic bars in the elastic case as well as in the elastic-plastic case. Another instance are optimization problems for convex bodies under geometrical restrictions, e.g., max- imization of the area surface for given width and diameter.

Let (T, h) be anm-dimensionalC^∞Riemannian manifold, (M, g) be ann-dimensional C^∞ Riemannian manifold and J¹(T, M) the associatedjet bundle of first order. Let (U, η, M) be acontrol fiber bundle. The manifoldM is calledstate manifoldand the componentsxⁱ,i= 1, . . . , nof a pointx∈M are calledstate variables. Then (xⁱ, u^a), i = 1, . . . , n; a = 1, . . . , k are adapted coordinates in U, and (t^α, xⁱ, xⁱ_α = ∂xⁱ

∂t^α), i= 1, . . . , n; α= 1, . . . , m are natural coordinates inJ¹(T, M). The components u^a of the pointu∈Ux=η⁻¹(x) are calledcontrols.

LetXα :U →J¹(T, M),Xα=X_αⁱ(x, u) ∂

∂xⁱ be a C^∞ fibered mapping, over the identity in the state manifoldM, which produces a continuous control PDEs system (controlled m-flow)

∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), i= 1, . . . , n; α= 1, . . . , m,

where t = (t¹, . . . , t^m) is the multi-parameter of evolution (multitime). This PDEs system has solutions if and only if the complete integrability conditions

(CIC) ∂X_αⁱ

∂x^j X_β^j+∂X_αⁱ

∂u^a

∂u^a

∂t^β = ∂X_βⁱ

∂x^j X_α^j +∂X_βⁱ

∂u^a

∂u^a

∂t^α are satisfied. These determine the set ofadmissible controls

U ={u(·) :R₊^m→U| u(·) is measurable and satisfies CIC}.

The evolution of the state manifold is totally characterized by the image setS = Im(Xα)⊂J¹(T, M) which is described by the control equations

xⁱ=xⁱ, xⁱ_α=X_αⁱ(x, u), i= 1, . . . , n α= 1, . . . , m.

Remark 1.1. A problem of controlled evolution can be thought as a controlled im- mersion if m < n, controlled diffeomorphism if m = n or controlled submersion if m > n. Particularly, if m = n, taking the trace after the indices i, α, we find a divergence type evolution (conservation laws).

To simplify, we replace the manifolds M and T and the fiber Ux by their local representativesRⁿ, R^m, U ⊂R^k respectively. More precisely, for multitimes we use the orthantR^m₊. Having this in mind, for the multitimes s = (s¹, ..., s^m) and t = (t¹, ..., t^m), we denote s ≤ t if and only if s^α ≤ t^α, α = 1, ..., m (product order).

Then the parallelepiped Ω0t0, fixed by the diagonal opposite points 0(0, ...,0) and t0 = (t¹₀, ..., t^m₀), is equivalent to the closed interval 0≤t≤t0. Givenu(·)∈ U, the statex(·) is the solution of the evolution system

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), x(0) =x0, t∈Ω0t0 ⊂R^m₊.

This multitime evolution system is used as a constraint when we want to optimize a multitime cost functional. On the other hand, the cost functionals can be introduced at least in two equivalent ways:

- either using a curvilinear integral, P(u(·)) =

Z

Γ0t0

Xβ(x(t), u(t))dt^β+g(x(t0)),

where Γ0t0 is an arbitraryC¹curve joining the points 0 andt0, therunning costsω= Xβ(x(t), u(t))dt^βis a closed (completely integrable) 1-form (autonomous Lagrangian 1-form), andg is theterminal cost;

- or using the multiple integral, Q(u(·)) =

Z

Ω0t0

X(x(t), u(t))dt¹...dt^m+g(x(t0)),

where therunning costsX(x(t), u(t)) is a continuous function (autonomous Lagrangian), andgis theterminal cost.

Theorem 1.2. The controlled multiple integral

I(t0) = Z

Ω0t0

X⁰(x(t), u(t))dt¹...dt^m,

withX⁰(x(t), u(t))as continuous function, is equivalent to the controlled curvilinear integral

J(t0) = Z

Γ0t0

X_β⁰(x(t), u(t))dt^β,

whereω =X_β⁰(x(t), u(t))dt^β is a closed (completely integrable)1-form and the func- tionsX_β⁰ have partial derivatives of the form

∂

∂t^α, ∂

∂t^α∂t^β(α < β), ... , ∂^m−1

∂t¹...∂tˆ^α...∂t^m. Thehat symbolposed over∂t^αdesignates that∂t^α is omitted.

Proof. The multiple integralI(t₀) suggests to introduce a new coordinate x⁰(t) =

Z

Ω0,t

X⁰(x(t), u(t))dt¹...dt^m, t∈Ω0t0, x⁰(t0) =I(t0).

Taking

X_α⁰(x(t), u(t)) = ∂x⁰

∂t^α(t), we can writex⁰(t) as the curvilinear integral

x⁰(t) = Z

γ0t

X_α⁰(x(s), u(s))ds^α, x⁰(t0) =J(t0),

whereγ0t is an arbitraryC¹ curve joining the points 0 andtin Ω0t0. Also

∂^m−1X_α⁰

∂t¹...∂tˆ^α...∂t^m = ∂^mx⁰

∂t¹...∂t^α...∂t^m.

Conversely, the curvilinear integralJ(t0) suggests to define a new coordinate by x⁰(t) =

Z

γ0t

X_α⁰(x(s), u(s))ds^α, x⁰(t0) =J(t0),

where γ0t is an arbitrary C¹ curve joining the points 0 and t in Ω0t0, and ω = X_α⁰(x(s), u(s))ds^αis a completely integrable 1-form. SinceX_α⁰ = ∂x⁰

∂t^α, we can define X⁰= ∂^m−1X_α⁰

∂t¹...∂tˆ^α...∂t^m = ∂^mx⁰

∂t¹...∂t^α...∂t^m. Then the new coordinate can be written as

x⁰(t) = Z

Ω0t

X⁰(x(t), u(t))dt¹...dt^m, t∈Ω0,t0, x⁰(t0) =I(t0).

¤ In this paper, we shall develop the optimization problems using cost functionals as path independent curvilinear integrals and constraints asm-flows, where the complete integrability conditons are piecewise satisfied.

Remark 1.3. New aspects of control theory are developed in the papers [9], [10], [31]

and [32] using weak derivatives instead of usual partial derivatives.

Remark 1.4. We can extend the holonomic controlled evolution to a nonholonomic controlled evolution, using the Pfaff system dxⁱ =X_αⁱ(x, u)dt^α. In the noholonomic case, the dimension of evolution is smaller thanm.

2 Maximum principle for multitime control theory based on a curvilinear integral cost

Acurvilinear integral costand amultitime flowwere introduced in theoptimal control theoryby our papers [12]-[30].

Multitime optimal control problem. Find maxu(·) P(u(·)) =

Z

Γ0t0

X_β(x(t), u(t))dt^β+g(x(t₀)) subject to ∂xⁱ

∂t^α(t) =X_αⁱ(t, x(t), u(t)), i= 1, ..., n, α= 1, ..., m, u(t)∈ U, t∈Ω0t0, x(0) =x0, x(t0) =xt0.

Themultitime maximum principle(necessary condition) will assert the existence of a costate vector function(p^∗₀, p^∗)(·) = (p^∗₀(·), p^∗_i(·)) which, together with the optimal

m-sheetx^∗(·), satisfies an appropriate PDEs system and a maximum condition. All these conditions can be written using the appropriatecontrol Hamiltonian 1-form

Hα(x, p0, p, u) =p0X_α⁰(x, u) +piX_αⁱ(x, u).

Theorem 2.1. (multitime maximum principle) Suppose u^∗(·) is optimal for (P),(P DE)and that x^∗(·) is the corresponding optimal m-sheet. Then there exists a function(p^∗₀, p^∗) = (p^∗₀, p^∗_i) : Ω0t0→Rⁿ⁺¹ such that

(P DE) ∂x^∗i

∂t^α(t) =∂Hα

∂pi (x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)),

(ADJ) ∂p^∗_i

∂t^α(t) =−∂Hα

∂xⁱ (x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)) and

(M) Hα(x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)) = max

u∈Ux

Hα(x^∗(t), p^∗₀(t), p^∗(t), u), t∈Ω0τ^∗. Also, the functionst→Hα(x^∗(t), p^∗₀(t), p^∗(t), u^∗(t))are constants.

(t0) p^∗₀(t0) =a0, p^∗_i(t0) = ∂g

∂xⁱ(x^∗(t0)) are satisfied.

We callx^∗(·) thestateof the optimally controlled system and (p^∗₀, p^∗(·)) thecostate vector.

Remark 2.2. (P DE)means the identities

∂x^∗i

∂t^β (t) =X_βⁱ(x^∗(t), u^∗(t)), β= 1, . . . , m; i= 1, . . . , n, (controlled evolution PDEs).

Remark 2.3. (ADJ)means the identities

∂p^∗_i

∂t^β (t) =− Ã

p^∗₀(t)∂X_β⁰

∂xⁱ +p^∗_j(t)∂X_β^j

∂xⁱ

!

(x^∗(t), u^∗(t))

(adjoint PDEs).

Remark 2.4. The relations (M)represent the maximization principle and the rela- tion(t0) means the terminal (transversality) condition.

Remark 2.5. The multitime maximum principle states necessary conditions that must hold on an optimalm-sheet of evolution.

Example. Consider the problem of a mine owner who must decide at what rate to extract a complex ore from his mine. He owns rights to the ore from two-time date 0 = (0,0) to two-time dateT = (T, T). Atwo-time datecan be the pair (date, useful component frequence). At two-time date 0 there isx0= (xⁱ₀) ore in the ground, and the instantaneous stock of orex(t) = (xⁱ(t)) declines at the rateu(t) = (uⁱ_α(t)) the mine owner extracts it. The mine owner extracts ore at costqiuⁱ_α(t)²

xⁱ(t) and sells ore at a constant pricep= (pi). He does not value the ore remaining in the ground at time T (there is no ”scrap value”). He chooses the rateu(t) of extraction in two-time to maximize profits over the period of ownership with no two-time discounting.

Solution (continuous two-time version). The manager want to maximizes the profit (curvilinear integral)

P(u(·)) = Z

γ0T

Ã

piuⁱ_α(t)−qiuⁱ_α(t)² xⁱ(t)

! dt^α

subject to the law of evolution ∂x

∂t^γ(t) =−uγ(t). Form the Hamiltonian 1-form Hα=piuⁱ_α(t)−qiuⁱ_α(t)²

xⁱ(t) −λi(t)uⁱ_α(t), differentiate and write the equations

∂Hα

∂uβ = µ

pi−2qiuⁱ_α xⁱ −λi

¶

δ^β_α= 0, no sum after the index i;

∂λi

∂t^α(t) =−∂Hα

∂xⁱ =−q_i

µuⁱ_α(t) xⁱ(t)

¶₂

, no sum after the index i.

As the mine owner does not value the ore remaining at timeT, we haveλi(T) = 0.

Using the above equations, it is easy to solve for the differential equations governing u(t) andλi(t):

2qiuⁱ_α(t)

xⁱ(t) =pi−λi(t), ∂λi

∂t^α(t) =−qi

µuⁱ_α(t) xⁱ(t)

¶2

no sum after i

and using the initial and turn-T conditions, the equations can be solved numerically.

Free multitime, fixed endpoint problem. Given a controlu(·)∈ U, the state x(·) is the solution of the initial value problem

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), x(0) =x0, t∈Ω0t0 ⊂R^m₊.

Suppose that a target pointx1 is given. Then we consider the cost functional (path independent curvilinear integral)

(P) P(u(·)) =

Z

γ0τ

X_β⁰(x(t), u(t))dt^β,

whereγ0τis an arbitraryC¹curve joining the points 0 andτin Ω0t0, forτ =τ(u(·))≤

∞, τ = (τ¹, ..., τ^m) as the first multitime the solution of (P DE) hits the target point x¹. We ask to find an optimal controlu^∗(·) such that

P(u^∗(·)) = max

u(·)∈UP(u(·)).

The control Hamiltonian 1-form is

Hα(x, p0, p, u) =p0X_α⁰(x, u) +piX_αⁱ(x, u).

Theorem 2.6. (multitime maximum principle) Suppose u^∗(·) is optimal for (P),(P DE)and that x^∗(·) is the corresponding optimal m-sheet. Then there exists a function(p^∗₀, p^∗) = (p^∗₀, p^∗_i) : Ω0τ^∗→Rⁿ⁺¹ such that

(P DE) ∂x^∗i

∂t^α(t) =∂Hα

∂pi

(x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)),

(ADJ) ∂p^∗_i

∂t^α(t) =−∂Hα

∂xⁱ (x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)), and

(M) Hα(x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)) = max

u∈Ux

Hα(x^∗(t), p^∗₀(t), p^∗(t), u), t∈Ω0τ^∗. Moreover,

Hα(x^∗(t), p^∗₀(t), p^∗(t), u^∗(t))|Ω_0τ∗ = 0,

whereτ^∗ denotes the first multitime them-sheetx^∗(·)hits the target point x1. We callx^∗(·) thestateof the optimally controlled system and (p^∗₀, p^∗(·)) thecostate matrix.

Remark 2.7. A more careful statement of the multitime maximum principle is: there exist the constantp^∗₀ and the function (p^∗_i) : Ω0t^∗→Rⁿ such that (P DE), (ADJ), and(M)hold. The vector functionp^∗(·)is a Lagrange multiplier, which appears owing to the constraint that the optimal m-sheetx^∗(·)must satisfy (P DE).

Remark 2.8. If the number p^∗₀ is0, then the control Hamiltonian 1-form does not depend on the corresponding running costsX_α⁰and in this case the maximum principle must be reformulated. Such a problem will be called abnormal problem.

Remark 2.9. The previous theory can be extended to the nonautonomous case

(P) P(u(·)) =

Z

γ0τ

X_β⁰(t, x(t), u(t))dt^β and

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(t, x(t), u(t)), t∈R^m₊,

using the idea of reducing to the previous case by introducing new variables x^α = t^α, α= 1, ..., m.

2.1 Maximum principle with transversality conditions

We look again at the dynamics

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), t∈R^m₊,

when the initial pointx0belongs to the subsetM0⊂Rⁿ and the terminal pointx1is constrained to lie in the subsetM1⊂Rⁿ. In other word, we must choose the starting pointx0∈M0in order to maximize

(P) P(u(·)) =

Z

γ0τ

X_β⁰(x(t), u(t))dt^β,

whereγ0τ is an arbitraryC¹curve joining the points 0 andτ in Ω0t0, forτ=τ(u(·)) as the first multitime we hitM1.

Assumption. The subsets M0 and M1 are smooth submanifolds of Rⁿ. In this context, we can use the tangent spacesTx₀M0andTx₁M1.

Theorem 2.10. (more transversality conditions)If the functionsu^∗(·)andx^∗(·) solve the previous problem, withx0=x^∗(0), x1=x(τ^∗), then there exists the function p^∗(·) : Ω0τ^∗→Rⁿ such that(P DE),(ADJ)and(M)hold fort∈Ω0τ^∗. Also, (t0) the vector p^∗(τ^∗)is orthogonal to Tx1M1,

the vector p^∗(0)is orthogonal to Tx0M0.

Remark 2.11. Let Γ0t0 be an arbitraryC¹ curve joining the diagonal points 0 and t0. According the terminal/transversality condition, fort0>0 and

P(u(·)) = Z

Γ0t0

X_β⁰(x(t), u(t))dt^β+g(x(t0)),

the condition(t0)means p^∗_i(t0) = ∂g

∂xⁱ(x^∗(t0)).

Remark 2.12. Suppose M1 = {x|fk(x) = 0, k = 1, ..., `}. Since the normal space (orthogonal complement of the tangent space Tx1M1) is generated by the vec- tors ∂fk

∂xⁱ(x1), k = 1, ..., `, we must have p^∗_i(τ^∗) = λ^k∂fk

∂xⁱ(x1), for some parameters (constants, Lagrange multipliers)λ¹, ..., λ^`.

2.2 Maximum principle with state constraints

Let us return to the problem

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), x(0) =x0, t∈Ω0t0 ⊂R^m₊.

(P) P(u(·)) =

Z

γ0τ

X_β⁰(x(t), u(t))dt^β,

forτ=τ(u(·))≤ ∞, τ = (τ¹, ..., τ^m) as the first multitime withx(τ) =x¹, andγ0τ

an arbitrary C¹ curve joining the points 0 and τ in Ω0t0. In this sense, we have a fixed endpoint problem.

State constraints. Suppose our multitime dynamics remains in the submanifold N = {x ∈ Rⁿ|f(x) ≤ 0}, where f : Rⁿ → R is a differentiable function. The functionsf andX_αⁱ define new functions

cα(x, u) = ∂f

∂xⁱ(x)X_αⁱ(x, u).

Ifx(t)∈∂N fort∈Ωs0s1, thencα(x(t), u(t)) = 0.

Theorem 2.13. (maximum principle for state constraints)Supposeu^∗(·), x^∗(·) solve the previous control theory problem, and thatx^∗(t)∈∂N fort ∈Ωs0s1. Then there exist the costate vector function p^∗(·) : Ωs0s1 → Rⁿ and there exist λ^∗γ_β(·) : Ω_s0s1 →R such that

(P DE) ∂x^∗i

∂t^α(t) =∂Hα

∂pi

(x^∗(t), p^∗₀(t), p^∗(t), u^∗(t)),

(ADJ⁰) ∂p^∗_i

∂t^β (t) =−∂Hβ

∂xⁱ (x^∗(t), p^∗(t), u^∗(t)) +λ^∗γ_β(t)∂cγ

∂xⁱ(x^∗(t), u^∗(t)) (M⁰) Hβ(x^∗(t), p^∗(t), u^∗(t)) = max

u∈Ux

{Hβ(x^∗(t), p^∗(t), u)|cα(x^∗(t), u) = 0}

hold, for multitimest∈Ωs0s1.

Remark 2.14. Let q1, ..., qs be differentiable functions on U which determine the subset

A={u∈U|q1(u)≤0, ..., qs(u)≤0}

in the control set (the condition m ≤ s is necessary). In this case, instead of the relation(M⁰) appear(M⁰⁰):

∂Hβ

∂u (x^∗(t), p^∗(t), u^∗(t)) =λ^∗γ_β(t)∂cγ

∂u(x^∗(t), u^∗(t)) +µ^∗r_β(t)∂qr

∂u(u^∗(t)).

The functionsλ^∗γ_β(·) are those appearing in (ADJ⁰). If x^∗(t)lies in the interior of N, for say the multitimes t∈Ω0s0, then the ordinary multitime maximum principle holds.

Remark 2.15. (Jump conditions)Let s0be a multitime thatp^∗ hits the boundary

∂N. Then p^∗(s0−0) =p^∗(s0+ 0). This means that there is no jump inp^∗ when we hit∂N. However,

p^∗(s1+ 0) =p^∗(s1−0)−λ^∗β_β(s1)∂f

∂x(x^∗(s1)),

i.e., there is possibly a jump inp^∗ when we leave ∂N. Of course, these statements are true when the gluing (contact) sheets, i.e., the unconstrained evolution sheet and the evolution boundary∂N have the same dimension.

3 Proofs of the multitime maximum principle

The proofs of multitime maximum principle relies either on control variations similar to those in variational calculus (for interior solutions, see also [15]-[30]) or on control m-needle variations. These give rise to variations of a referencem-sheet. In spite of the fact that these variations are not equivalent, they assert similar statements.

Here we use them-needle variations. To explain their meaning, we consider a can- didate optimal controlu^∗(·), the corresponding optimalm-sheetx^∗(·), and a multitime pointsofapproximate continuityfor the functionsXα(x^∗(·), u^∗(·)) and u(·)∈ U. An m-needle variationis a family of controls u²(·) obtained replacingu^∗(·) withu(·) on the set Ω0s\Ω0s−ε. Anym-needle variation gives rise to a gradient variation yα of an m-sheet x(t) satisfying the variational PDE ∂yⁱ_α

∂t^β(t) = y^j_α(t)∂X_βⁱ

∂x^j (x(t), u(t)), in the classical sense, only after the multitimes. Of course, the last PDEs satisfies the complete integrability conditions.

3.1 Simple controls variations

The response x(·) to a given control u(·) is the unique solution of the completely integrable PDEs system

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), x(0) =x0, t∈Ω0t0 ⊂R^m₊.

Let us find how the m-needle changes in the control affect the response. In this sense, we fix the multitime s = (s¹, ..., s^m), s^α > 0, α = 1, ..., m, and a control v(·)∈ U. We select ²= (²¹, ..., ²^m), ²^α>0 with the property 0< s^α−²^α < s^α and define the modified control

u²(t) =

½ v(t) ift∈Ω0s\Ω0s−²

u^∗(t) otherwise,

which is called a simple m-needle variation of u^∗(·). We denote by x²(·) the corre- sponding response of our system

(22) ∂xⁱ_²

∂t^α(t) =X_αⁱ(x²(t), u²(t)), x²(0) =x0, t∈Ω0t0 ⊂R^m₊.

Let us try to understand how the choices ofsandv(·) causex²(·) to differ fromx(·), for small||²||>0.

Lemma 3.1. (changing initial conditions)Ifx²(·)is a solution of the initial value problem

∂xⁱ_²

∂t^α(t) =X_αⁱ(x²(t), u(t)), x²(0) =x0+²^αyα0+o(²), t∈R^m₊,

thenx²(t) =x(t) +²^αyα(t) +o(²)as²→0, uniformly fortin compact subsets ofR^m₊, whereyα= (yⁱ_α) =

µ∂xⁱ_²

∂²^α|²=0

¶

is the solution of the initial value variational problem

∂y_αⁱ

∂t^β(t) =y_α^j(t)∂X_βⁱ

∂x^j (x(t), u(t)), yα(0) =yα0, t∈R^m₊.

Coming back to the dynamics (22) and simple controlm-needle variation, we find Lemma 3.2. (dynamics and simple control variations) If u²(·) is a simple variation of the controlu^∗(·), then

x²(t) =x(t) +²^αyα(t) +o(²)as²→0

uniformly fortin compact subsets ofR^m₊, where yα(t) = 0, t∈Ω0s and (23) ∂yⁱ_α

∂t^β(t) =y_α^j(t)∂X_βⁱ

∂x^j (x(t), u^∗(t)), yα(s) =yαs, t∈R^m₊ \Ω0s, for

yαs=Xα(x(s), v(s))−Xα(x(s), u^∗(s)).

Proof. For simplicity, let us drop the superscript∗. First we remark thatx²(t) =x(t) fort∈Ω0s−² and hence yα(t) = 0 fort∈Ω0s−². For the multitimet∈Ω0s\Ω0s−², we can use the curvilinear integral

x²(t)−x(t) = Z

Γs−²t

(Xα(x(s), v(s))−Xα(x(s), u(s)))ds^α=o(²),

where Γs−²tis aC^∞ curve joining the pointss−²andt. Hence we can putyα(t) = 0 fort∈Ω0s\Ω0s−².

We set t=s. Since the curvilinear integral is independent of the path, we select Γ as being the straight line joining the pointss−²ands, i.e.,

Γ :t^α=s^α−²^α+²^ατ, α= 1, ..., m, τ ∈[0,1].

Consequently,

x²(s)−x(s) = (Xα(x(s), v(s))−Xα(x(s), u(s)))²^α+o(²).

On the multitime boxR^m₊\Ω0s, the functionsx(.) andx²(.) are solutions for the same PDEs, but with different initial conditions: x(0) =x0andx²(s) =x(s)+²^αyαs+o(²), foryαs defined by (23). The Lemma of changing initial conditions shows that

x²(t) =x(t) +²^αyα(t) +o(²)

foryα(·) solving (23) and fort∈R^m₊ \Ω0s. ¤

3.2 Free endpoint problem, no running cost

Statement. Let us consider again the multitime dynamics

(P DE) ∂xⁱ

∂t^α(t) =X_αⁱ(x(t), u(t)), x(0) =x0, t∈Ω0t0⊂R^m₊ together the terminal cost functional

(P) P(u(·)) =g(x(t0)),

which must be maximized with respect to the control. We denote byu^∗(·) respectively x^∗(·) the optimal control and the optimalm-sheet of this problem. Since the running cost is zero, the control Hamiltonian 1-form is

Hα(x, p, u) =piX_αⁱ(x, u).

It remains to find a vector functionp^∗= (p^∗_i) : Ω0t₀ →Rⁿ such that

(ADJ) ∂p^∗_i

∂t^α(t) =−∂Hα

∂xⁱ (x^∗(t), p^∗(t), u^∗(t)), t∈Ω0t0, and

(M) Hα(x^∗(t), p^∗(t), u^∗(t)) = max

u∈U Hα(x^∗(t), p^∗(t), u).

For simplicity, let us drop the superscript∗. Also, we take into account the control variationu_²(.).

The costate. We define the costate p : Ω0t0 → Rⁿ, p = (pi), as the unique solution of the terminal value problem

(24) ∂pi

∂t^β(t) =−pj(t)∂X_β^j

∂xⁱ (x(t), u(t)), t∈Ω0t0, pi(t0) = ∂g

∂xⁱ(x(t0)).

The solutions of the Cauchy problems (23)+(24) determine an 1-form of components piy_βⁱ. The PDEs (24) are calledadjoint equationssince we can verify by computation that the components (scalar products) piy_βⁱ are first integrals of the PDEs system (23)+(24). The costate will be used to calculate the variation of the terminal cost.

Lemma 3.3. (variation of terminal cost)The partial variations of the terminal cost are

∂

∂²^βP(u²(·))|²=0=pi(s)¡

X_βⁱ(x(s), v(s))−X_βⁱ(x(s), u(s))¢ .

Proof. Since P(u²(·)) = g(x(t0) +²^βyβ(t0) +o(²)), where y(·) satisfies the previous Lemmas, we find

∂

∂²^βP(u²(·))|²=0= ∂g

∂xⁱ(x(t0))yⁱ_β(t0).

Since the componentsp_iyⁱ_βare first integrals of the PDEs system (23)+(24), we obtain

∂g

∂xⁱ(x(t0))yⁱ_β(t0) =pi(t0)y_βⁱ(t0) =pi(s)y_βⁱ(s), ∀s∈Ω0t0.

Finally, the functionsyβ(s) =Xβ(x(s), v(s))−Xβ(x(s), u(s)) give the desired formula.

¤

We restore the superscript∗and we formulate the next

Theorem 3.4. (multitime maximum principle) There exists a function p^∗ : Ω0t0 →Rⁿ satisfying the adjoint dynamics (ADJ), the maximization principle (M) and the terminal (transversality) condition(t0).

Proof. The adjoint dynamics and the terminal condition appear in the Cauchy prob- lem (24). To prove (M), we fixs∈int Ω0t0 andv(·)∈ U, as above. Since the function

²→P(u²(·)), ²∈Ω0t0,has a maximum at²= 0, we must have 0≥ ∂P

∂²^β(u²(·))|²=0=p^∗_i(s)¡

X_βⁱ(x^∗(s), v(s))−X_βⁱ(x^∗(s), u^∗(s))¢ . Consequently

Hβ(x^∗(s), p^∗(s), v(s)) =p^∗_i(s)X_βⁱ(x^∗(s), v(s))

≤p^∗_i(s)X_βⁱ(x^∗(s), u^∗(s)) =Hβ(x^∗(s), p^∗(s), u^∗(s)),

for eachs ∈ int Ω0t0 and v(·)∈ U. Since the function v(·) is arbitrary, so it is the valuev(s) and therefore

Hα(x^∗(t), p^∗(t), u^∗(t)) = max

u∈U Hα(x^∗(t), p^∗(t), u).

¤

3.3 Free endpoint problem with running costs

Let us consider that the cost functional include a running cost, i.e.,

(P) P(u(·)) =

Z

Γ0t0

X_β⁰(x(t), u(t))dt^β+g(x(t₀)),

where Γ0t0 is an arbitraryC¹curve joining the points 0 andt0, therunning costdx⁰= X_β⁰(x(t), u(t))dt^βis a closed (completely integrable) 1-form, andgis theterminal cost.

In this case thecontrol Hamiltonian 1-frommust have the form H_α(x, p₀, p, u) =p₀X_α⁰(x, u) +p_iX_αⁱ(x, u),

under the condition that we can built a costate functionp^∗(·) = (p^∗₀(·), p^∗_i(·)) satis- fying (ADJ), (M) and (t0).

Adding a new variable. Introducing a new variable x⁰, we convert the theory to the previous case.

Letx⁰: Ω0t0→Rbe the solution of initial problem

(25) ∂x⁰

∂t^α(t) =X_α⁰(x(t), u(t)), x⁰(0) = 0, t∈Ω0t0, wherex(·) = (xⁱ(·)) is the solution of (P DE). Introduce

x= (x¹, ..., xⁿ), x= (x⁰, x), x0= (0, x0), x(·) = (x⁰(·), x(·)), and

Xα(x, u) = (X_α⁰(x, u), Xα(x, u)), g(x) =g(x) +a0x⁰. Then (P DE) and (25) give the dynamics

(P DE) ∂x

∂t^α(t) =Xα(x(t), u(t)), x(0) =x0, t∈Ω0t0.

Consequently, the actual control problem transforms into a new control problem with no running cost and the terminal cost

(P) P(u(·)) =g(x(t0)).

We apply the previous theorem of multitime maximum principle to obtainp^∗: Ω0t0→ Rⁿ⁺¹, p^∗= (p^∗_i) satisfying (Mα) for the control Hamiltonian 1-form

Hα(x, p, u) =p_iXⁱ_α(x, u).

The adjoint equations (ADJ) hold for the terminal transversality condition (t0) p^∗_j(t0) = ∂g

∂x^j(x(t0)), j= 0,1,2, ..., n.

SinceXαdo not depend upon the variablex⁰, the 0-th equation in the adjoint equa- tions (ADJ) is ∂p0

∂t^β = 0. On the other hand, the relation ∂g

∂x⁰ =a0 impliesp0 =a0. ConsequentlyHβ(x, p, u) andp^∗(·) = (p^∗_i(·)) satisfy (ADJ) and (M).

3.4 Multitime multiple control variations

To formulate and prove the multitime maximum principle for the next fixed endpoint problem, we need to introduce a multiple variation of the control.

Let us find how multiple changes in the control affect the response. We fix the multitimes sA = (s¹_A, ..., s^m_A), A = 1, ..., N, with 0 < s1 < ... < sN, the control parametersvA(·)∈ U and strictly positive numbersλ^A,A= 1,2, ..., N. Select² >0 so small that the domains Ω0sA\Ω0sA−λ^A²do not overlap. Define the modified control

u²(t) =

½ vA(t) ift∈Ω0s_A\Ω_0,sA−λA², A= 1, ..., N u^∗(t) otherwise,

which is called amultiplem-needle variationof the controlu^∗(·). We denotex²(·) the corresponding response of the Cauchy problem

(26) ∂xⁱ_²

∂t^α(t) =X_αⁱ(x²(t), u²(t)), x²(0) =x0, t∈R^m₊.

Let us try to understand how the choices ofsA and vA(·) cause x²(·) to differ from x(·), for small² >0. Firstly, we setyα(t) =Yα(t, s)yαs, t∈Ωs∞, for the solution of the Cauchy variational problem (linear PDE system)

∂yⁱ_α

∂t^β(t) =y_α^j(t)∂X_βⁱ

∂x^j (x(t), u(t)), yα(s) =yαs, t∈Ωs∞, where the pointsyαs∈Rⁿ are given andYα(t, s) is thetransition matrix.

We define

yαsA=Xα(x(sA), vA(s))−Xα(x(sA), u(sA)), A= 1,2, ..., N.

and we replace the Lemma of dynamics and simple control variations with

Lemma 3.5. (dynamics and multiple control variations)Ifu²(.)is a multiple variation of the controlu(·), then

x²(t) =x(t) +²^αyα(t) +o(²)as²→0 uniformly fortin compact subsets ofR^m₊, where





yα(t) = 0 t∈Ω0s1

yα(t) =P_P

A=1λ^AYα(t, sA)yαsA t∈Ω0sA+1\Ω0sA, P = 1,2, ..., N−1 yα(t) =P_N

A=1λ^AYα(t, sA)yαsA t∈ΩsN∞.

Definition 3.1. (cones of variations) Let 0 < s1 ≤ s2 ≤ ... ≤ sN < t and yαsA ∈Rⁿ, A= 1, ..., N. For eachα, the set

Kα(t) ={ XN

A=1

λ^AYα(t, sA)yαsA|N = 1,2, ...;λ^A>0}.

is called thecone of variations at multitimet.

We remark that each Kα(t) is a convex cone inRⁿ, consisting in all changes in the statex(t) (up to order ²) we can make by multiple variations of the controlu(·) (see the previous Lemma). To study the geometry ofKα(t), we need the following topological Lemma:

Lemma 3.6. (zeroes of a vector field)LetSbe a closed, bounded, convex subset of Rⁿ andp∈IntS. IfY :S→Rⁿ is a continuous vector field satisfying||Y(x)−x||<

||x−p||, ∀x∈∂S, then there exists a pointx∈S such that Y(x) =p.

Proof. For the general case, we assume after a translation that p= 0, and 0∈IntS.

We mapSontoB(0,1) by a radial dilation, and mapY by rigid motion. This process convert the general case to the next case.

Suppose thatS is the unit ballB(0,1) andp= 0. The inequality in hypothesis is equivalent to (Y(x), x)>0, ∀x∈∂B(0,1). Consequently, for smallt, the continuous mappingZ(x) =x−tY(x) mapsB(0,1) into itself. According Brouwer Fixed Point Theorem, the mappingZ has a fixed point, let sayZ(x^∗) =x^∗, and henceY(x^∗) = 0.

¤

3.5 Fixed endpoint problem

The fixed endpoint problem is characterized by the constraint x(τ) = x1, where τ =τ(u(·)) is the first multitime thatx(·) hits the target point x1. In this context, the cost functional is

P(u(·)) = Z

γ0τ

X_β⁰(x(t), u(t))dt^β.

Adding a new variable. We define againx⁰: Ω0t0 →Ras the solution of initial problem

∂x⁰

∂t^α(t) =X_α⁰(x(t), u(t)), x⁰(0) = 0, t∈Ω0τ,