Measures of consciousness from the
viewpoint of information geometry
Masafumi Oizumi
RIKEN BSI and Monash University
Masafumi Oizumi, Naotsugu Tsuchiya, Shun-ichi Amari (2016) Unified Framework for Information Integration Based on Information Geometry, PNAS, 113, 14817-14822.
Outline
• Brief review of Integrated Information Theory
of consciousness (IIT)
• How to mathematically define integrated
information
• Relationships between integrated information
and other quantities
Extrinsic vs intrinsic information
Stimulus: s Response: r
Extrinsic information = Information for an ideal external observer Mutual (Shannon) information between r and s:
Intrinsic information = Information for the system itself
Information that the system can exploit for itself, which does not depend on an external observer but only depend on causal relationships in the system.
Concept of integrated information
Brain
Network of neurons
Digital camera
Collection of photodiodes Integrated information quantifies the difference
(information loss) after causal influences between parts are cut .
If nothing changes (no information is lost) as in a digital camera, integrated information is 0. In the case of the brain, the difference will be a lot.
Exclusion – Boundary of consciousness
Ho a co scious ess e ist?
(A) (B)
Only an entity that generates the local maximum of exists. (i.e., it cannot be partitioned into more integrated parts.)
Outline
• Brief review of Integrated Information Theory
of consciousness (IIT)
• How to mathematically define integrated
information
• Relationships between integrated information
and other quantities
How to quantify integrated information
Original network Full model
Integrated information:
Difference D
Disconnected network Disconnected model
. Defi e the operatio of cutti g causal i flue ces.
2. Define the difference between probability distributions.
3. Minimize the difference.
Kullback-Leibler di erge ce IIT . , Earth o er s dista ce IIT . (Oizumi et al., 2015; Tegmark, 2016)
Dynamical system
x
1x
2y
1y
2time
Example: Gaussian distribution past present Joint probability distribution of
a system (full model)
A: connectivity matrix
E: Gaussian random variables : interactions at the same time : interactions across time
causal influences
Cutti g a causal i flue ce
from one unit to another
x
1x
2y
1y
2time time
x
1x
2y
1y
2The operation of cutting a causal interaction from x2 to y1
Full model Disconnected model
Total amount of information
x
1x
2y
1y
2time
x
1x
2y
1y
2time Difference
Full model Disconnected model
Constraints
Minimize the distance between p and q,
under the constraint,
The closest point q* is the orthogonal projection of p to the submanifold. p, q*, and q form an orthogonal triangle and the following Pythagorean relation holds.
submanifold
Interpretation from information geometry
Mutual information between X and Y The KL-divergence is minimized when
Total amount of information
x
1x
2y
1y
2time
x
1x
2y
1y
2time Difference
Minimized difference between p and q
Full model Disconnected model
Mutual information between X and Y Constraint
Causal interaction from one unit to another
x1
x2
y1
y2
time time
Difference
x1
x2
y1
y2
Constraint
Full model Disconnected model
Minimized difference between p and q
Transfer entropy
Total amount of causal interactions:
Integrated information
x
1x
2y
1y
2time time
Difference
x
1x
2y
1y
2Constraints
Our measure of Integrated information
Integrated information is smaller than
the mutual information
The distance minimized in larger subspace is always smaller.
Mutual information
Integrated information
Transfer entropy
x1
x2
y1
y2
x1
x2
y1
y2
x1
x2
y1
y2 x1
x2
y1
y2
Full model:
Disconnected model:
time
Unified framework based on minimization of the
KL-divergence
Difference
x1
x2
y1
y2
Full model:
Disconnected model:
time
Interpretation of other quantities
Difference
x1
x2
y1
y2
Mutual information
x1
x2
y1
y2
Integrated information
x1
x2
y1
y2
Stochastic interaction
(Ay, 2001, 2015; Barrett & Seth, 2011)
x1
x2
y1
y2
Total correlation
(Watanabe, 1960)
Conclusions
• We proposed a novel measure of integrated
information from a unified framework.
• It will be interesting to derive integrated information
from physics viewpoint (Tegmark, 2015).
1. Define the operation of cutting causal influences.
2. Quantify the difference between the full model p and disconnected model q. 3. Minimize the difference.
Kullback-Leibler di erge ce IIT . , Earth o er s dista ce IIT .
