**AND NONCONSERVATION OF ENTROPY** **FOR DISCRETE-TIME DYNAMICAL SYSTEMS**

WASSIM M. HADDAD, QING HUI, SERGEY G. NERSESOV, AND VIJAYSEKHAR CHELLABOINA

*Received 19 November 2004*

We develop thermodynamic models for discrete-time large-scale dynamical systems.

Specifically, using compartmental dynamical system theory, we develop energy flow mod-
els possessing energy conservation, energy equipartition, temperature equipartition, and
entropy nonconservation principles for discrete-time, large-scale dynamical systems. Fur-
thermore, we introduce a*new*and dual notion to entropy; namely,*ectropy, as a measure*
of the tendency of a dynamical system to do useful work and grow more organized, and
show that conservation of energy in an isolated thermodynamic system necessarily leads
to nonconservation of ectropy and entropy. In addition, using the system ectropy as a Lya-
punov function candidate, we show that our discrete-time, large-scale thermodynamic
energy flow model has convergent trajectories to Lyapunov stable equilibria determined
by the system initial subsystem energies.

**1. Introduction**

Thermodynamic principles have been repeatedly used in continuous-time dynamical sys-
tem theory as well as in information theory for developing models that capture the ex-
change of nonnegative quantities (e.g., mass and energy) between coupled subsystems
[5,6,8,11,20,23,24]. In particular, conservation laws (e.g., mass and energy) are used
to capture the exchange of material between coupled macroscopic subsystems known as
compartments. Each compartment is assumed to be kinetically homogeneous; that is,
any material entering the compartment is instantaneously mixed with the material in the
compartment. These models are known as*compartmental*models and are widespread in
engineering systems as well as in biological and ecological sciences [1,7,9,16,17,22].

Even though the compartmental models developed in the literature are based on the first law of thermodynamics involving conservation of energy principles, they do not tell us whether any particular process can actually occur; that is, they do not address the second law of thermodynamics involving entropy notions in the energy flow between subsys- tems.

The goal of the present paper is directed towards developing nonlinear discrete-time compartmental models that are consistent with thermodynamic principles. Specifically,

Copyright©2005 Hindawi Publishing Corporation Advances in Diﬀerence Equations 2005:3 (2005) 275–318 DOI:10.1155/ADE.2005.275

since thermodynamic models are concerned with energy flow among subsystems, we develop a nonlinear compartmental dynamical system model that is characterized by en- ergy conservation laws capturing the exchange of energy between coupled macroscopic subsystems. Furthermore, using graph-theoretic notions, we state three thermodynamic axioms consistent with the zeroth and second laws of thermodynamics that ensure that our large-scale dynamical system model gives rise to a thermodynamically consistent en- ergy flow model. Specifically, using a large-scale dynamical systems theory perspective, we show that our compartmental dynamical system model leads to a precise formula- tion of the equivalence between work energy and heat in a large-scale dynamical sys- tem.

Next, we give a deterministic definition of entropy for a large-scale dynamical sys- tem that is consistent with the classical thermodynamic definition of entropy and show that it satisfies a Clausius-type inequality leading to the law of entropy nonconservation.

Furthermore, we introduce a*new*and dual notion to entropy; namely,*ectropy, as a mea-*
sure of the tendency of a large-scale dynamical system to do useful work and grow more
organized, and show that conservation of energy in an isolated thermodynamically con-
sistent system necessarily leads to nonconservation of ectropy and entropy. Then, using
the system ectropy as a Lyapunov function candidate, we show that our thermodynami-
cally consistent large-scale nonlinear dynamical system model possesses a continuum of
equilibria and is*semistable; that is, it has convergent subsystem energies to Lyapunov sta-*
ble energy equilibria determined by the large-scale system initial subsystem energies. In
addition, we show that the steady-state distribution of the large-scale system energies is
uniform leading to system energy equipartitioning corresponding to a minimum ectropy
and a maximum entropy equilibrium state. In the case where the subsystem energies
are proportional to subsystem temperatures, we show that our dynamical system model
leads to temperature equipartition, wherein all the system energy is transferred into heat
at a uniform temperature. Furthermore, we show that our system-theoretic definition
of entropy and the newly proposed notion of ectropy are consistent with Boltzmann’s
kinetic theory of gases involving an*n-body theory of ideal gases divided by diathermal*
walls.

The contents of the paper are as follows. In Section 2, we establish notation, defi- nitions, and review some basic results on nonnegative and compartmental dynamical systems. InSection 3, we use a large-scale dynamical systems perspective to develop a nonlinear compartmental dynamical system model characterized by energy conservation laws that is consistent with basic thermodynamic principles. Then we turn our attention to stability and convergence. In particular, using the total subsystem energies as a candi- date system energy storage function, we show that our thermodynamic system is lossless and hence can deliver to its surroundings all of its stored subsystem energies and can store all of the work done to all of its subsystems. Next, using the system ectropy as a Lyapunov function candidate, we show that the proposed thermodynamic model is semistable with a uniform energy distribution corresponding to a minimum ectropy and a maximum en- tropy. InSection 4, we generalize the results ofSection 3to the case where the subsystem energies in large-scale dynamical system model are proportional to subsystem tempera- tures and arrive at temperature equipartition for the proposed thermodynamic model.

Furthermore, we provide an interpretation of the steady-state expressions for entropy and ectropy that is consistent with kinetic theory. InSection 5, we specialize the results of Section 3to thermodynamic models with linear energy exchange. Finally, we draw con- clusions inSection 6.

**2. Mathematical preliminaries**

In this section, we introduce notation, several definitions, and some key results needed for developing the main results of this paper. LetRdenote the set of real numbers, letZ+

denote the set of nonnegative integers, letR* ^{n}*denote the set of

*n*

*×*1 column vectors, let R

^{m}

^{×}*denote the set of*

^{n}*m*

*×*

*n*real matrices, let (

*·*)

^{T}denote transpose, and let

*I*

*n*or

*I*denote the

*n*

*×*

*n*identity matrix. For

*v*

*∈*R

*, we write*

^{q}*v*

*≥≥*0 (resp.,

*v*0) to indicate that every component of

*v*is nonnegative (resp., positive). In this case, we say that

*v*is

*nonnegative*or

*positive, respectively. Let*R+

*andR*

^{q}*+denote the nonnegative and positive orthants of R*

^{q}*; that is, if*

^{q}*v*

*∈*R

*, then*

^{q}*v*

*∈*R

*+and*

^{q}*v*

*∈*R

*+are equivalent, respectively, to*

^{q}*v*

*≥≥*0 and

*v*0. Finally, we write

*·*for the Euclidean vector norm,(M) andᏺ(M) for the range space and the null space of a matrix

*M, respectively, spec(M) for the spectrum of*the square matrix

*M, rank(M) for the rank of the matrixM, ind(M) for the index ofM;*

that is, min*{**k**∈*Z+: rank(M* ^{k}*)

*=*rank(M

^{k}^{+1})

*}*,

*M*

^{#}for the group generalized inverse of

*M, where ind(M)*

*≤*1,∆E(x(k)) for

*E(x(k*+ 1))

*−*

*E(x(k)),*Ꮾ

*ε*(α),

*α*

*∈*R

*,*

^{n}*ε >*0, for the open ball centered at

*α*with radius

*ε, andM*

*≥*0 (resp.,

*M >*0) to denote the fact that the Hermitian matrix

*M*is nonnegative (resp., positive) definite.

The following definition introduces the notion of*Z-,M-, nonnegative, and compart-*
mental matrices.

*Definition 2.1*[2,5,12]. Let*W**∈*R^{q}^{×}* ^{q}*.

*W*is a

*Z-matrix*if

*W*(

*i*,

*j*)

*≤*0,

*i,j*

*=*1,

*...,q,i*

*=*

*j.*

*W*is an*M-matrix*(resp., a*nonsingularM-matrix) ifW*is a*Z-matrix and all the principal*
minors of*W*are nonnegative (resp., positive).*W*is*nonnegative*(resp.,*positive) ifW*(*i*,*j*)*≥*
0 (resp.,*W*(*i*,*j*)*>*0),*i,j**=*1,*...,q. Finally,W* is*compartmental*if*W* is nonnegative and
_{q}

*i**=*1*W*(i,j)*≤*1,*j**=*1,*...,q.*

In this paper, it is important to distinguish between a square nonnegative (resp., posi- tive) matrix and a nonnegative-definite (resp., positive-definite) matrix.

The following definition introduces the notion of nonnegative functions [12].

*Definition 2.2.* Let*w**=*[w1,*...,w** _{q}*]

^{T}:ᐂ

*→*R

*, whereᐂis an open subset ofR*

^{q}*that con- tainsR*

^{q}*+. Then*

^{q}*w*is

*nonnegative*if

*w*

*(z)*

_{i}*≥*0 for all

*i*

*=*1,

*...,q*and

*z*

*∈*R

*+.*

^{q}Note that if*w(z)**=**Wz, whereW**∈*R^{q}^{×}* ^{q}*, then

*w(*

*·*) is nonnegative if and only if

*W*is a nonnegative matrix.

Proposition2.3 [12]. *Suppose that*R* ^{q}*+

*⊂*ᐂ. ThenR

*+*

^{q}*is an invariant set with respect to*

*z(k*+ 1)

*=*

*w*

^{}

*z(k)*

^{},

*z(0)*

*=*

*z*0,

*k*

*∈*Z+, (2.1)

*wherez*0*∈*R* ^{q}*+

*, if and only ifw*:ᐂ

*→*R

^{q}*is nonnegative.*

The following definition introduces several types of stability for the discrete-time
*nonnegative*dynamical system (2.1).

*Definition 2.4.* The equilibrium solution*z(k)**≡**z*eof (2.1) is*Lyapunov stable*if, for every
*ε >*0, there exists *δ**=**δ(ε)>*0 such that if *z*0*∈*Ꮾ*δ*(ze) *∩* R* ^{q}*+, then

*z(k)*

*∈*Ꮾ

*ε*(ze)

*∩*R

*+,*

^{q}*k*

*∈*Z+. The equilibrium solution

*z(k)*

*≡*

*z*e of (2.1) is

*semistable*if it is Lyapunov stable and there exists

*δ >*0 such that if

*z*0

*∈*Ꮾ

*δ*(ze)

*∩*R

*+, then lim*

^{q}

_{k}

_{→∞}*z(k) exists and*corresponds to a Lyapunov stable equilibrium point. The equilibrium solution

*z(k)*

*≡*

*z*e

of (2.1) is*asymptotically stable*if it is Lyapunov stable and there exists*δ >*0 such that if
*z*0*∈*Ꮾ*δ*(ze) *∩* R* ^{q}*+, then lim

_{k}

_{→∞}*z(k)*

*=*

*z*e. Finally, the equilibrium solution

*z(k)*

*≡*

*z*eof (2.1) is

*globally asymptotically stable*if the previous statement holds for all

*z*0

*∈*R

*+.*

^{q}Finally, recall that a matrix*W**∈*R^{q}^{×}* ^{q}*is

*semistable*if and only if lim

_{k}

_{→∞}*W*

*exists [12], while*

^{k}*W*is

*asymptotically stable*if and only if lim

_{k}

_{→∞}*W*

^{k}*=*0.

**3. Thermodynamic modeling for discrete-time systems**

**3.1. Conservation of energy and the first law of thermodynamics.** The fundamental
and unifying concept in the analysis of complex (large-scale) dynamical systems is the
concept of energy. The energy of a state of a dynamical system is the measure of its abil-
ity to produce changes (motion) in its own system state as well as changes in the system
states of its surroundings. These changes occur as a direct consequence of the energy flow
between diﬀerent subsystems within the dynamical system. Since heat (energy) is a funda-
mental concept of thermodynamics involving the capacity of hot bodies (more energetic
subsystems) to produce work, thermodynamics is a theory of large-scale dynamical sys-
tems [13]. As in thermodynamic systems, dynamical systems can exhibit energy (due to
friction) that becomes unavailable to do useful work. This is in turn contributes to an
increase in system entropy; a measure of the tendency of a system to lose the ability to do
useful work.

To develop discrete-time compartmental models that are consistent with thermody-
namic principles, consider the discrete-time large-scale dynamical systemᏳ shown in
Figure 3.1 involving*q* interconnected subsystems. Let *E**i*:Z+*→*R+ denote the energy
(and hence a nonnegative quantity) of the*ith subsystem, letS** _{i}*:Z+

*→*Rdenote the ex- ternal energy supplied to (or extracted from) the

*ith subsystem, letσ*

*:R*

_{i j}*+*

^{q}*→*R+,

*i*

*=*

*j,*

*i,j*

*=*1,

*...,q, denote the exchange of energy from the*

*jth subsystem to theith subsystem,*and let

*σ*

*ii*:R

*+*

^{q}*→*R+,

*i*

*=*1,

*...,q, denote the energy loss from theith subsystem. Anenergy*

*balance*equation for the

*ith subsystem yields*

∆E*i*(k)*=*
*q*
*j**=*1,*j**=**i*

*σ**i j*

*E(k)*^{}*−**σ**ji*

*E(k)*^{}*−**σ**ii*

*E(k)*^{}+*S**i*(k), *k**≥**k*0, (3.1)

or, equivalently, in vector form,

*E(k*+ 1)*=**w*^{}*E(k)*^{}*−**d*^{}*E(k)*^{}+*S(k),* *k**≥**k*0, (3.2)

*S*1

*S**i*

*S**j*

*S**q*

Ᏻ1

Ᏻ*i*

Ᏻ*j*

Ᏻ*q*

.. .

.. .

*σ*11(*E*)

*σ**ii*(*E*)

*σ**j j*(E)

*σ**qq*(E)
*σ**i j*(*E*) *σ**ji*(*E*)

Figure 3.1. Large-scale dynamical systemᏳ.

where *E(k)**=*[E1(k),*...,E**q*(k)]^{T}, *S(k)**=*[S1(k),*...,S**q*(k)]^{T}, *d(E(k))**=*[σ11(E(k)),*...,*
*σ**qq*(E(k))]^{T},*k**≥**k*0, and*w**=*[w1,*...,w**q*]^{T}:R* ^{q}*+

*→*R

*is such that*

^{q}*w**i*(E)*=**E**i*+
*q*
*j**=*1,*j**=**i*

*σ**i j*(E)*−**σ**ji*(E)^{}, *E**∈*R* ^{q}*+

*.*(3.3)

Equation (3.1) yields a conservation of energy equation and implies that the change of
energy stored in the*ith subsystem is equal to the external energy supplied to (or extracted*
from) the*ith subsystem plus the energy gained by theith subsystem from all other sub-*
systems due to subsystem coupling minus the energy dissipated from the*ith subsystem.*

Note that (3.2) or, equivalently, (3.1) is a statement reminiscent of the*first law of thermo-*
*dynamics*for each of the subsystems, with*E** _{i}*(

*·*),

*S*

*(*

_{i}*·*),

*σ*

*(*

_{i j}*·*),

*i*

*=*

*j, andσ*

*(*

_{ii}*·*),

*i*

*=*1,

*...,q,*playing the role of the

*ith subsystem internal energy, energy supplied to (or extracted*from) the

*ith subsystem, the energy exchange between subsystems due to coupling, and*the energy dissipated to the environment, respectively.

To further elucidate that (3.2) is essentially the statement of the principle of the con-
servation of energy, let the total energy in the discrete-time large-scale dynamical system
Ᏻ be given by*U***e**^{T}*E,* *E**∈*R* ^{q}*+, where

**e**

^{T}[1,..., 1], and let the energy received by the discrete-time large-scale dynamical systemᏳ (in forms other than work) over the discrete-time interval

*{*

*k*1,

*...,k*2

*}*be given by

*Q*

^{}

^{k}

_{k}^{2}

_{=}

_{k}_{1}

**e**

^{T}[S(k)

*−*

*d(E(k))], whereE(k),*

*k*

*≥*

*k*0, is the solution to (3.2). Then, premultiplying (3.2) by

**e**

^{T}and using the fact that

**e**

^{T}

*w(E)*

*≡*

**e**

^{T}

*E, it follows that*

∆U*=**Q,* (3.4)

where∆U*U(k*2)*−**U(k*1) denotes the variation in the total energy of the discrete-time
large-scale dynamical systemᏳover the discrete-time interval*{**k*1,...,k2*}*. This is a state-
ment of the first law of thermodynamics for the discrete-time large-scale dynamical sys-
temᏳand gives a precise formulation of the equivalence between variation in system
internal energy and heat.

It is important to note that our discrete-time large-scale dynamical system model does
not consider work done by the system on the environment nor work done by the envi-
ronment on the system. Hence,*Q*can be interpreted physically as the amount of energy
that is received by the system in forms other than work. The extension of addressing work
performed by and on the system can be easily handled by including an additional state
equation, coupled to the energy balance equation (3.2), involving volume states for each
subsystem [13]. Since this slight extension does not alter any of the results of the paper, it
is not considered here for simplicity of exposition.

For our large-scale dynamical system modelᏳ, we assume that *σ**i j*(E)*=*0,*E**∈*R* ^{q}*+,
whenever

*E*

_{j}*=*0,

*i,j*

*=*1,...,

*q. This constraint implies that if the energy of the*

*jth sub-*system ofᏳis zero, then this subsystem cannot supply any energy to its surroundings nor dissipate energy to the environment. Furthermore, for the remainder of this paper, we as- sume that

*E*

*i*

*≥*

*σ*

*ii*(E)

*−*

*S*

*i*

*−*

_{q}*j**=*1,*j**=**i*[σ*i j*(E)*−**σ**ji*(E)]*= −*∆E*i*,*E**∈*R* ^{q}*+,

*S*

*∈*R

*,*

^{q}*i*

*=*1,

*...,q.*

This constraint implies that the energy that can be dissipated, extracted, or exchanged by
the*ith subsystem cannot exceed the current energy in the subsystem. Note that this as-*
sumption implies that*E(k)**≥≥*0 for all*k**≥**k*0.

Next, premultiplying (3.2) by**e**^{T}and using the fact that**e**^{T}*w(E)**≡***e**^{T}*E, it follows that*
**e**^{T}*E*^{}*k*1

*=***e**^{T}*E*^{}*k*0

+

*k*1*−*1
*k**=**k*0

**e**^{T}*S(k)**−*

*k*1*−*1
*k**=**k*0

**e**^{T}*d*^{}*E(k)*^{}, *k*1*≥**k*0*.* (3.5)
Now, for the discrete-time large-scale dynamical systemᏳ, define the input*u(k)S(k)*
and the output*y(k)d(E(k)). Hence, it follows from (3.5) that the discrete-time large-*
scale dynamical systemᏳis*lossless*[23] with respect to the*energy supply rater(u,y)**=*
**e**^{T}*u**−***e**^{T}*y*and with the*energy storage functionU(E)***e**^{T}*E,E**∈*R* ^{q}*+. This implies that (see
[23] for details)

0*≤**U*a

*E*0

*=**U*^{}*E*0

*=**U*r

*E*0

*<**∞*, *E*0*∈*R* ^{q}*+, (3.6)
where

*U*a

*E*0

*−* inf

*u*(*·*),*K**≥**k*0

*K**−*1
*k**=**k*0

**e**^{T}*u(k)**−***e**^{T}*y(k)*^{},

*U*r

*E*0

inf

*u*(*·*),*K**≥−**k*^{0}+1
*k*0*−*1
*k**=−**K*

**e**^{T}*u(k)**−***e**^{T}*y(k)*^{},

(3.7)

and*E*0*=**E(k*0)*∈*R* ^{q}*+. Since

*U*a(E0) is the maximum amount of stored energy which can be extracted from the discrete-time large-scale dynamical systemᏳat any discrete-time instant

*K*, and

*U*r(E0) is the minimum amount of energy which can be delivered to

the discrete-time large-scale dynamical systemᏳto transfer it from a state of minimum
potential*E(**−**K)**=*0 to a given state*E(k*0)*=**E*0, it follows from (3.6) that the discrete-
time large-scale dynamical systemᏳcan deliver to its surroundings all of its stored sub-
system energies and can store all of the work done to all of its subsystems. In the case
where*S(k)**≡*0, it follows from (3.5) and the fact that*σ** _{ii}*(E)

*≥*0,

*E*

*∈*R

*+,*

^{q}*i*

*=*1,...,

*q, that*the zero solution

*E(k)*

*≡*0 of the discrete-time large-scale dynamical systemᏳwith the energy balance equation (3.2) is Lyapunov stable with Lyapunov function

*U(E) corre-*sponding to the total energy in the system.

The next result shows that the large-scale dynamical systemᏳis locally controllable.

Proposition3.1. *Consider the discrete-time large-scale dynamical system*Ᏻ*with energy*
*balance equation (3.2). Then for every equilibrium stateE*e*∈*R* ^{q}*+

*and everyε >*0

*andT*

*∈*Z+

*, there exist*

*S*e

*∈*R

^{q}*,α >*0, and

*T*

^{}

*∈ {*0,...,T

*}*

*such that for every*

*E*

^{}

*∈*R

*+*

^{q}*with*

*E*

*−*

*E*e

*≤*

*αT, there exists*

*S*:

*{*0,

*...,T*

^{}

*} →*R

^{q}*such that*

*S(k)*

*−*

*S*e

*≤*

*ε,k*

*∈ {*0,

*...,T*

^{}

*}*

*, and*

*E(k)*

*=*

*E*e+ ((

*E*

^{}

*−*

*E*e)/

*T*

^{})k,

*k*

*∈ {*0,

*...,T*

^{}

*}*

*.*

*Proof.* Note that with*S*e*=**d(E*e)*−**w(E*e) +*E*e, the state*E*e*∈*R* ^{q}*+is an equilibrium state of
(3.2). Let

*θ >*0 and

*T*

*∈*Z+, and define

*M*(θ,T) sup

*E**∈*Ꮾ1(0),*k**∈{*0,*...*,*T**}*

*w*^{}*E*e+*kθE*^{}*−**w*^{}*E*e

*−**d*^{}*E*e+*kθE*^{}+*d*^{}*E*e

*−**kθE*^{}*.*
(3.8)
Note that for every*T**∈*Z+, lim_{θ}* _{→}*0

^{+}

*M(θ,T)*

*=*0. Next, let

*ε >*0 and

*T*

*∈*Z+ be given, and let

*α >*0 be such that

*M*(α,

*T) +α*

*≤*

*ε. (The existence of such anα*is guaranteed since

*M*(α,

*T)*

*→*0 as

*α*

*→*0

^{+}.) Now, let

*E*

^{}

*∈*R

*+be such that*

^{q}*E*

*−*

*E*e

*≤*

*αT. WithT*

^{}

*E*

*−*

*E*e

*/α*

*≤*

*T, where*

*x*denotes the smallest integer greater than or equal to

*x, and*

*S(k)**= −**w*^{}*E(k)*^{}+*d*^{}*E(k)*^{}+*E(k) +* *E*^{}*−**E*e

*E**−**E*e*/α*^{}, *k**∈ {*0,...,*T*^{}*}*, (3.9)
it follows that

*E(k)**=**E*e+ *E**−**E*e

*E**−**E*e*/α*^{}*k,* *k**∈ {*0,*...,T*^{}*}*, (3.10)
is a solution to (3.2). The result is now immediate by noting that*E(T)*^{} *=**E*and

*S(k)**−**S*e*≤*
*w*

*E*e+ *E**−**E*e

*E**−**E*e*/α*^{}*k*

*−**w*^{}*E*e

*−**d*

*E*e+ *E**−**E*e

*E**−**E*e*/α*^{}*k*

+*d*^{}*E*e

*−* *E**−**E*e

*E**−**E*e*/α*^{}*k*^{}_{}+*α*

*≤**M(α,T*) +*α*

*≤**ε,* *k**∈ {*0,...,*T*^{}*}**.*

(3.11)

It follows fromProposition 3.1that the discrete-time large-scale dynamical systemᏳ
with the energy balance equation (3.2) is*reachable*from and*controllable*to the origin in

R* ^{q}*+. Recall that the discrete-time large-scale dynamical systemᏳwith the energy balance
equation (3.2) is reachable from the origin inR

*+if, for all*

^{q}*E*0

*=*

*E(k*0)

*∈*R

*+, there exist a finite time*

^{q}*k*i

*≤*

*k*0and an input

*S(k) defined on*

*{*

*k*i,...,

*k*0

*}*such that the state

*E(k),k*

*≥*

*k*i, can be driven from

*E(k*i)

*=*0 to

*E(k*0)

*=*

*E*0. Alternatively,Ᏻis controllable to the origin in R

*+if, for all*

^{q}*E*0

*=*

*E(k*0)

*∈*R

*+, there exist a finite time*

^{q}*k*f

*≥*

*k*0and an input

*S(k) defined on*

*{*

*k*0,...,kf

*}*such that the state

*E(k),*

*k*

*≥*

*k*0, can be driven from

*E(k*0)

*=*

*E*0to

*E(k*f)

*=*0.

We letᐁr denote the set of all admissible bounded energy inputs to the discrete-time
large-scale dynamical systemᏳsuch that for any*K**≥ −**k*0, the system energy state can
be driven from*E(**−**K)**=*0 to*E(k*0)*=**E*0*∈*R+* ^{q}* by

*S(*

*·*)

*∈*ᐁr, and we letᐁc denote the set of all admissible bounded energy inputs to the discrete-time large-scale dynamical systemᏳsuch that for any

*K*

*≥*

*k*0, the system energy state can be driven from

*E(k*0)

*=*

*E*0

*∈*R

*+to*

^{q}*E(K*)

*=*0 by

*S(*

*·*)

*∈*ᐁc. Furthermore, letᐁbe an input space that is a subset of bounded continuousR

*-valued functions onZ. The spacesᐁr,ᐁc, andᐁare assumed to be closed under the shift operator; that is, if*

^{q}*S(*

*·*)

*∈*ᐁ(resp.,ᐁcorᐁr), then the function

*S*

*K*defined by

*S*

*K*(k)

*=*

*S(k*+

*K*) is contained inᐁ(resp.,ᐁcorᐁr) for all

*K*

*≥*0.

**3.2. Nonconservation of entropy and the second law of thermodynamics.** The non-
linear energy balance equation (3.2) can exhibit a full range of nonlinear behavior in-
cluding bifurcations, limit cycles, and even chaos. However, a thermodynamically consis-
tent energy flow model should ensure that the evolution of the system energy is diﬀusive
(parabolic) in character with convergent subsystem energies. Hence, to ensure a ther-
modynamically consistent energy flow model, we require the following axioms. For the
statement of these axioms, we first recall the following graph-theoretic notions.

*Definition 3.2*[2]. A*directed graphG(Ꮿ) associated with theconnectivity matrix*Ꮿ*∈*R^{q}^{×}* ^{q}*
has

*vertices*

*{*1, 2,...,q

*}*and an

*arc*from vertex

*i*to vertex

*j,i*

*=*

*j, if and only if*Ꮿ(

*j*,

*i*)

*=*0.

A*graphG(Ꮿ) associated with the connectivity matrix*Ꮿ*∈*R^{q}^{×}* ^{q}* is a directed graph for
which the

*arc set*is symmetric; that is,Ꮿ

*=*Ꮿ

^{T}. It is said that

*G(Ꮿ) isstrongly connected*if for any ordered pair of vertices (i,

*j),i*

*=*

*j, there exists apath*(i.e., sequence of arcs) leading from

*i*to

*j.*

Recall thatᏯ*∈*R^{q}^{×}* ^{q}*is

*irreducible; that is, there does not exist a permutation matrix*such thatᏯis cogredient to a lower-block triangular matrix, if and only if

*G(Ꮿ) is strongly*connected (see [2, Theorem 2.7]). Let

*φ*

*i j*(E)

*σ*

*i j*(E)

*−*

*σ*

*ji*(E),

*E*

*∈*R

*+, denote the net energy exchange between subsystemsᏳ*

^{q}*i*andᏳ

*j*of the discrete-time large-scale dynamical systemᏳ.

Axiom1. *For the connectivity matrix*Ꮿ*∈*R^{q}^{×}^{q}*associated with the large-scale dynamical*
*system*Ᏻ*defined by*

Ꮿ(*i*,*j*)*=*

0 *ifφ** _{i j}*(E)

*≡*0,

1 *otherwise,* *i**=**j,i,j**=*1,*...,q,*
Ꮿ(*i*,*i*)*= −*

*q*
*k**=*1,*k**=**i*

Ꮿ(*k*,*i*), *i**=**j,i**=*1,...,*q,*

(3.12)

rankᏯ*=**q**−*1, and forᏯ(*i*,*j*)*=*1,*i**=**j,φ** _{i j}*(E)

*=*0

*if and only ifE*

_{i}*=*

*E*

_{j}*.*

Axiom2. *Fori,j**=*1,...,q,(E_{i}*−**E** _{j}*)φ

*(E)*

_{i j}*≤*0,

*E*

*∈*R+

^{q}*.*

Axiom3. *Fori,j**=*1,...,q,(∆E*i**−*∆E*j*)/(E_{i}*−**E** _{j}*)

*≥ −*1,

*E*

_{i}*=*

*E*

_{j}*.*

The fact that*φ** _{i j}*(E)

*=*0 if and only if

*E*

_{i}*=*

*E*

*,*

_{j}*i*

*=*

*j, implies that subsystems*Ᏻ

*i*and Ᏻ

*j*ofᏳare

*connected; alternatively,φ*

*(E)*

_{i j}*≡*0 implies thatᏳ

*i*andᏳ

*j*are

*disconnected.*

Axiom 1implies that if the energies in the connected subsystemsᏳ*i*andᏳ*j*are equal, then
energy exchange between these subsystems is not possible. This is a statement consistent
with the*zeroth law of thermodynamics*which postulates that temperature equality is a
necessary and suﬃcient condition for thermal equilibrium. Furthermore, it follows from
the fact thatᏯ*=*Ꮿ^{T} and rankᏯ*=**q**−*1 that the connectivity matrixᏯis irreducible
which implies that for any pair of subsystemsᏳ*i*andᏳ*j*,*i**=**j, of*Ᏻ, there exists a sequence
of connected subsystems ofᏳthat connect Ᏻ*i* andᏳ*j*.Axiom 2implies that energy is
exchanged from more energetic subsystems to less energetic subsystems and is consistent
with the*second law of thermodynamics*which states that heat (energy) must flow in the
direction of lower temperatures. Furthermore, note that*φ** _{i j}*(E)

*= −*

*φ*

*(E),*

_{ji}*E*

*∈*R

*+,*

^{q}*i*

*=*

*j,*

*i,j*

*=*1,...,q, which implies conservation of energy between lossless subsystems. With

*S(k)*

*≡*0, Axioms1and2along with the fact that

*φ*

*i j*(E)

*= −*

*φ*

*ji*(E),

*E*

*∈*R

*+,*

^{q}*i*

*=*

*j,i,j*

*=*1,

*...,q, imply that at a given instant of time, energy can only be transported, stored, or*dissipated but not created and the maximum amount of energy that can be transported and/or dissipated from a subsystem cannot exceed the energy in the subsystem. Finally, Axiom 3implies that for any pair of connected subsystemsᏳ

*i*andᏳ

*j*,

*i*

*=*

*j, the energy*diﬀerence between consecutive time instants is monotonic; that is, [E

*i*(k+ 1)

*−*

*E*

*(k+ 1)][E*

_{j}*(k)*

_{i}*−*

*E*

*(k)]*

_{j}*≥*0 for all

*E*

_{i}*=*

*E*

*,*

_{j}*k*

*≥*

*k*0,

*i,j*

*=*1,

*...,q.*

Next, we establish a Clausius-type inequality for our thermodynamically consistent energy flow model.

Proposition3.3. *Consider the discrete-time large-scale dynamical system*Ᏻ*with energy*
*balance equation (3.2) and assume that Axioms1,2, and3* *hold. Then for allE*0*∈*R* ^{q}*+

*,*

*k*f

*≥*

*k*0

*, andS(*

*·*)

*∈*ᐁ

*such thatE(k*f)

*=*

*E(k*0)

*=*

*E*0

*,*

*k*^{f}*−*1
*k**=**k*^{0}

*q*
*i**=*1

*S** _{i}*(k)

*−*

*σ*

_{ii}^{}

*E(k)*

^{}

*c*+

*E*

*i*(k+ 1)

*=*

*k*f*−*1
*k**=**k*0

*q*
*i**=*1

*Q**i*(k)
*c*+*E**i*(k+ 1)* ^{≤}*0,

(3.13)

*wherec >*0,*Q**i*(k)*S**i*(k)*−**σ**ii*(E(k)),*i**=*1,...,q, is the amount of net energy (heat) re-
*ceived by theith subsystem at thekth instant, andE(k),k**≥**k*0*, is the solution to (3.2) with*
*initial conditionE(k*0)*=**E*0*. Furthermore, equality holds in (3.13) if and only if*∆E*i*(k)*=*0,
*i**=*1,*...,q, andE**i*(k)*=**E**j*(k),*i,j**=*1,...,q,*i**=**j,k**∈ {**k*0,...,kf*−*1*}**.*

*Proof.* Since*E(k)**≥≥*0,*k**≥**k*0, and*φ** _{i j}*(E)

*= −*

*φ*

*(E),*

_{ji}*E*

*∈*R

*+,*

^{q}*i*

*=*

*j,i,j*

*=*1,...,

*q, it fol-*lows from (3.2), Axioms 2 and 3, and the fact that

*x/(x*+ 1)

*≤*log

*(1 +*

_{e}*x),*

*x >*

*−*1

that

*k*^{f}*−*1
*k**=**k*^{0}

*q*
*i**=*1

*Q** _{i}*(k)

*c*+

*E*

*i*(k+ 1)

^{=}*k*^{f}*−*1
*k**=**k*^{0}

*q*
*i**=*1

∆E*i*(k)*−*_{q}

*j**=*1,*j**=**i**φ**i j*
*E(k)*^{}
*c*+*E**i*(k+ 1)

*=*

*k*f*−*1
*k**=**k*0

*q*
*i**=*1

∆E*i*(k)
*c*+*E**i*(k)

1 + ∆E*i*(k)
*c*+*E**i*(k)

*−*1

*−*^{k}

f*−*1

*k**=**k*0

*q*
*i**=*1

*q*
*j**=*1,*j**=**i*

*φ*_{i j}^{}*E(k)*^{}
*c*+*E** _{i}*(k+ 1)

*≤*
*q*
*i**=*1

log_{e}

*c*+*E*_{i}^{}*k*f

*c*+*E*_{i}^{}*k*0

*−*^{k}

f*−*1

*k**=**k*0

*q*
*i**=*1

*q*
*j**=*1,*j**=**i*

*φ*_{i j}^{}*E(k)*^{}
*c*+*E** _{i}*(k+ 1)

*= −*

*k*f*−*1
*k**=**k*0

*q**−*1
*i**=*1

*q*
*j**=**i*+1

*φ**i j*

*E(k)*^{}
*c*+*E**i*(k+ 1)^{−}

*φ**i j*

*E(k)*^{}
*c*+*E**j*(k+ 1)

*= −*^{k}

f*−*1

*k**=**k*0

*q**−*1
*i**=*1

*q*
*j**=**i*+1

*φ*_{i j}^{}*E(k)*^{}*E** _{j}*(k+ 1)

*−*

*E*

*(k+ 1)*

_{i}^{}

*c*+

*E*

*(k+ 1)*

_{i}^{}

*c*+

*E*

*(k+ 1)*

_{j}^{}

*≤*0,

(3.14)

which proves (3.13).

Alternatively, equality holds in (3.13) if and only if^{}^{k}_{k}^{f}_{=}^{−}_{k}^{1}_{0}(∆E*i*(k)/(c+*E**i*(k+ 1)))*=*0,
*i**=*1,*...,q, andφ** _{i j}*(E(k))(E

*(k+ 1)*

_{j}*−*

*E*

*(k+ 1))*

_{i}*=*0,

*i,j*

*=*1,

*...,q,i*

*=*

*j,k*

*≥*

*k*0. Moreover,

_{k}_{f}

_{−}_{1}

*k**=**k*0(∆E*i*(k)/(c+*E**i*(k+ 1)))*=*0 is equivalent to∆E*i*(k)*=*0,*i**=*1,*...,q,k**∈ {**k*0,...,*k*f*−*
1*}*. Hence,*φ** _{i j}*(E(k))(E

*(k+ 1)*

_{j}*−*

*E*

*(k+ 1))*

_{i}*=*

*φ*

*(E(k))(E*

_{i j}*(k)*

_{j}*−*

*E*

*(k))*

_{i}*=*0,

*i,j*

*=*1,

*...,q,*

*i*

*=*

*j,k*

*≥*

*k*0. Thus, it follows from Axioms1,2, and3that equality holds in (3.13) if and only if∆E

*i*

*=*0,

*i*

*=*1,

*...,q, andE*

*j*

*=*

*E*

*i*,

*i,j*

*=*1,

*...,q,i*

*=*

*j.*

Inequality (3.13) is analogous to Clausius’ inequality for reversible and irreversible
thermodynamics as applied to discrete-time large-scale dynamical systems. It follows
fromAxiom 1and (3.2) that for the*isolated*discrete-time large-scale dynamical systemᏳ;
that is,*S(k)**≡*0 and*d(E(k))**≡*0, the energy states given by*E*e*=**αe,α**≥*0, correspond to
the equilibrium energy states ofᏳ. Thus, we can define an*equilibrium process*as a process
where the trajectory of the discrete-time large-scale dynamical systemᏳstays at the equi-
librium point of the isolated systemᏳ. The input that can generate such a trajectory can
be given by*S(k)**=**d(E(k)),k**≥**k*0. Alternatively, a*nonequilibrium process*is a process that
is not an equilibrium one. Hence, it follows fromAxiom 1that for an equilibrium pro-
cess,*φ**i j*(E(k))*≡*0,*k**≥**k*0,*i**=**j,i,j**=*1,...,q, and thus, byProposition 3.3and∆E*i**=*0,
*i**=*1,*...,q, inequality (3.13) is satisfied as an equality. Alternatively, for a nonequilibrium*
process, it follows from Axioms1,2, and3that (3.13) is satisfied as a strict inequality.

Next, we give a deterministic definition of entropy for the discrete-time large-scale dynamical system Ᏻ that is consistent with the classical thermodynamic definition of entropy.

*Definition 3.4.* For the discrete-time large-scale dynamical systemᏳwith energy balance
equation (3.2), a function:R* ^{q}*+

*→*Rsatisfying

^{}*E*^{}*k*2

*≥*^{}*E*^{}*k*1

+

*k*^{2}*−*1
*k**=**k*1

*q*
*i**=*1

*S** _{i}*(k)

*−*

*σ*

_{ii}^{}

*E(k)*

^{}

*c*+*E**i*(k+ 1) , (3.15)
for any*k*2*≥**k*1*≥**k*0and*S(**·*)*∈*ᐁ, is called the*entropy*ofᏳ.

Next, we show that (3.13) guarantees the existence of an entropy function forᏳ. For
this result, define, the*available entropy*of the large-scale dynamical systemᏳby

a
*E*0

*−* sup

*S*(*·*)*∈*ᐁc,*K**≥**k*0

*K**−*1
*k**=**k*0

*q*
*i**=*1

*S**i*(k)*−**σ**ii*
*E(k)*^{}

*c*+*E**i*(k+ 1) , (3.16)
where*E(k*0)*=**E*0*∈*R* ^{q}*+and

*E(K)*

*=*0, and define the

*required entropy supply*of the large- scale dynamical systemᏳby

r

*E*0

sup

*S*(* _{·}*)

*ᐁr,*

_{∈}*K*

*≥−*

*k*0+1

*k*

^{0}

*−*1

*k*

*=−*

*K*

*q*
*i**=*1

*S** _{i}*(k)

*−*

*σ*

_{ii}^{}

*E(k)*

^{}

*c*+*E** _{i}*(k+ 1) , (3.17)
where

*E(*

*−*

*K)*

*=*0 and

*E(k*0)

*=*

*E*0

*∈*R

*+. Note that the available entropya(E0) is the minimum amount of scaled heat (entropy) that can be extracted from the large-scale dynamical systemᏳin order to transfer it from an initial state*

^{q}*E(k*0)

*=*

*E*0 to

*E(K*)

*=*0.

Alternatively, the required entropy supplyr(E0) is the maximum amount of scaled heat
(entropy) that can be delivered toᏳto transfer it from the origin to a given initial state
*E(k*0)*=**E*0.

Theorem3.5. *Consider the discrete-time large-scale dynamical system*Ᏻ*with energy bal-*
*ance equation (3.2) and assume that Axioms2* *and3hold. Then there exists an entropy*
*function for*Ᏻ. Moreover,a(E),*E**∈*R* ^{q}*+

*, and*r(E),

*E*

*∈*R

*+*

^{q}*, are possible entropy functions*

*for*Ᏻ

*with*a(0)

*=*r(0)

*=*0. Finally, all entropy functions(E),

*E*

*∈*R

*+*

^{q}*, for*Ᏻ

*satisfy*

r(E)*≤*(E)*−*(0)*≤*a(E), *E**∈*R* ^{q}*+

*.*(3.18)

*Proof.*Since, byProposition 3.1,Ᏻis controllable to and reachable from the origin inR

*+, it follows from (3.16) and (3.17) thata(E0)*

^{q}*<*

*∞*,

*E*0

*∈*R

*+, andr(E0)*

^{q}*>*

*−∞*,

*E*0

*∈*R

*+, respectively. Next, let*

^{q}*E*0

*∈*R+

*and let*

^{q}*S(*

*·*)

*∈*ᐁbe such that

*E(k*i)

*=*

*E(k*f)

*=*0 and

*E(k*0)

*=*

*E*0, where

*k*i

*≤*

*k*0

*≤*

*k*f. In this case, it follows from (3.13) that

*k*f*−*1
*k**=**k*^{i}

*q*
*i**=*1

*S**i*(k)*−**σ**ii*
*E(k)*^{}

*c*+*E**i*(k+ 1) * ^{≤}*0, (3.19)

or, equivalently,

*k*0*−*1
*k**=**k*i

*q*
*i**=*1

*S**i*(k)*−**σ**ii*
*E(k)*^{}
*c*+*E**i*(k+ 1) ^{≤ −}

*k*f*−*1
*k**=**k*0

*q*
*i**=*1

*S**i*(k)*−**σ**ii*
*E(k)*^{}

*c*+*E**i*(k+ 1) *.* (3.20)