We propose a model of a fully connected layer consisting of 5 layers. The first layer consists of 256 nodes. Consequently, this layer requires 1,048,832 parameters because the output from the pre-trained model has 4,096 nodes. Let the output of the pre-trained model be y(37)∈R4096. Thus, the output of layer p1 is
y(p1)=w(p1)y(37)+w(p(0)1)
wherey(p1), w(0)(p1) ∈R256 and w(p1)∈M256×4096. The output of layerp2 is y(i)(p2)=max
y(p(i)1),0
wherei= 1,2, . . . ,256. The Dropout in layerp3 has factor 0.2 which means 20% of the output of this layer in each channel is deactivated randomly. The output of layerp3 is
y(i)(p3)=
y(i)(p2) if node-iis activated 0 if node-iis deactivated wherei= 1,2, . . . ,256. Layerp4 have the output as follows,
y(p4)=w(p4)y(p3)+w(p(0)4)
where y(p3) ∈ R256 is the output of layer p3, y(p4), w(0)(p4) ∈ Rn and w(p1) ∈ Mn×256 for the number of categories, n.
The last layer is LogSoftmax, an activation function, where the output of this layer p5
is
y(i)(p5) =log
exp y(p(i)4)
Pn
j=1exp y(j)(p4)
for i= 1,2, . . . , n. We propose two CNN models. The first model is of n= 2 and the second model is of n= 9. The summary of the second approach is given in Figure 3.3.
Chapter Three 23
Figure 3.2: The first 37 layers of the model of VGG16.
Layer (type) Output shape # Parameters
Conv2d-1 [64,224,224] 1,792
ReLU-2 [64,224,224] 0
Conv2d-3 [64,224,224] 36,928
ReLU-4 [64,224,224] 0
MaxPool2d-5 [64,112,112] 0
Conv2d-6 [128,112,112] 73,856
ReLU-7 [128,112,112] 0
Conv2d-8 [128,112,112] 147,584
ReLU-9 [128,112,112] 0
MaxPool2d-10 [128,56,56] 0
Conv2d-11 [256,56,56] 295,168
ReLU-12 [256,56,56] 0
Conv2d-13 [256,56,56] 590,080
ReLU-14 [256,56,56] 0
Conv2d-15 [256,56,56] 590,080
ReLU-16 [256,56,56] 0
MaxPool2d-17 [256,28,28] 0
Conv2d-18 [512,28,28] 1,180,160
ReLU-19 [512,28,28] 0
Conv2d-20 [512,28,28] 2,359,808
ReLU-21 [512,28,28] 0
Conv2d-22 [512,28,28] 2,359,808
ReLU-23 [512,28,28] 0
MaxPool2d-24 [512,14,14] 0
Conv2d-25 [512,14,14] 2,359,808
ReLU-26 [512,14,14] 0
Conv2d-27 [512,14,14] 2,359,808
ReLU-28 [512,14,14] 0
Conv2d-29 [512,14,14] 2,359,808
ReLU-30 [512,14,14] 0
MaxPool2d-31 [512,7,7] 0
Linear-32 [4096] 102,764,544
ReLU-33 [4096] 0
Dropout-34 [4096] 0
Linear-35 [4096] 16,781,312
ReLU-36 [4096] 0
Dropout-37 [4096] 0
Total params: 134,260,544 Trainable params: 0
Non-trainable params: 134,260,544
1
Chapter Three 24
Input the color image VGG16
output-4096 ... output-6 output-5 output-4 output-3 output-2 output-1
Layer-p1
...
Layer-p2
...
Layer-p3
...
Layer-p4
...
Layer-p5
...
Ouput
Figure 3.3: The flow chart of the second approach. The proposed model is in the dashed rectangle.
The loss function that we choose in this research is the Average Negative Log Likelihood.
Since the final layer score is already the LogSoftmax-layer, then multiplying by negative one is enough. We define T as the set of targeted output. LetM ∈Z+ be the number of images in training, y(p5,m) ∈ Rn be the output of the layer p5 for mth-image, and tm ∈Tbe the targeted output of mth-image. Since the final layer is LogSoftmax, then the negative log-likelihood is defined as
L(y(p5),t,w, M) = 1 M
M
X
m=1
−y(p(i)5,m)
i=argmaxt
m
wherey(p5)is the set of the output of layerp5for allMimages,tis the set of the targeted output for allM images,wis the set of trainable and non-trainable parameters.
For the first CNN model, the number of categories is two. Thus, the set of the targeted output is T = {(1,0),(0,1)} where (1,0) belongs to the first category and (0,1) be-longs to the second category. The second CNN model has nine categories. Thus, the targeted output for the category-i is t(i) ∈ R9 where the ith-component is 1 and the other components are 0.
Adam method is implemented to find the minimizer of L with respect to the trainable parameters. This method computes individual adaptive learning rates for different pa-rameters from estimates of first and second moments of the gradients [11]. The name of this method is derived from the adaptive moment estimation.
The output of the CNN model is the score for each category. This score means how close the input image to the training data of each category. Increasing the number of training images might get better results. From this score, we determine that the input image is classified to the highest score category. We also state that the image is classified
Chapter Four 25 correctly if the highest score category is the same as the label of that image. We define average accuracy by
mc
M ×100%,
where mc is the number of images which classified correctly, and M is the number of input images. Finally, all of the setup the CNN in this research is implemented in Python and Torch.
Chapter 4
Application on the Image of the Capillaries
The main goal in this chapter is how to classify the capillary into two categories, that are straight and wiggly. There are two different approaches to this classification. First is the partial differential equation approach. The second is the convolutional neural network approach.
4.1 Partial Differential Equation Approach
We directly look at the image of the real capillaries. We provide three images as in Figure 4.1 to be evaluated by the partial differential equation approach. We decided and then labeled by our own naked eyes that the first and third cases are the straight capillary. The wiggly capillary is the second case.
Figure 4.1: The three selected images of capillary taken by a microscope.
26
Chapter Four 27 First, the active contour method is applied to the brightness image of Figure 4.1 to detect the edge of the capillaries. The results are shown in Figure 4.2. The red curve in the first row is given. Then its evolution is seen from the upper to the below. At the last row, the red curve lies on the edge of the capillary. However, since the image is not clear enough, then there are some noises which are detected by the red curve. Hence, the longest continuous curve needs to be selected carefully.
Let consider the longest continuous curve. It is the red curve on each image of the first column in Figure 4.3, Figure 4.4, and Figure 4.5. Then the curvature of this curve is calculated and shown in the middle column. The last column shows the result of the Fourier transform of the curvature in the middle column.
The curvature of the curve is approximated numerically by implementing finite difference in (2.6). Since there exists the second derivative in this formula, then three different points on the curve are needed to approximate the curvature. The most difficult is how to choose these three points. If these three points are too close to each other, then the sharp noise information is quite dominant. When a curve has a sharp oscillation, this gives very high positive or negative values of the curvature. As a consequence, important information from the curvature vanishes. On the other hand, if the three points are too far from each other, then important information is lost. Therefore, choosing the appropriate or intermediate enough of the distance between the three points is essential and mandatory to extract important information from the curvature.
In each Figure 4.3, Figure 4.4, there are three rows indicating the different distance among the three points which are used to calculate the curvature. From the upper to below shows the distance is increasing. It can be seen how the effect of changing the distance between the three points on the curvature. The effect is clearly seen in Figure 4.4. At the first row of the second column, the curvature at around 0.6 of the horizontal axis is almost -0.4. It is gone as the distance is increasing, see the second and the third row of the middle column.
Next, the Fourier transformation is applied to the curvature. The magnitude is shown in the last column of each Figure 4.3, Figure 4.4. The horizontal axis is the frequency, while the vertical axis is the magnitude. We remind the reader that in this study, we define the magnitude as the absolute value of the result of Fourier transformation by (2.8). There is a conclusion that can be figured out easily from these results. The peak magnitude occurs at the zero frequency if the curvature is dominantly positive as in the second column of Figure 4.3 and Figure 4.5. Otherwise, the peak magnitude occurs at the nonzero frequency as in Figure 4.4.
Chapter Four 28
Figure 4.2: The evolution of the given curve (red) on the given three different images to detect the edge of the capillary. From top to below images show the iteration:
0,20,70,150 except the lowest middle image which is the result of iteration 10,000.
This image needs more iteration since the curve keep evolving at iteration 150.
Chapter Four 29
0 5 10 15 20
0.000 0.002 0.004 0.006
0 5 10 15 20
0.000 0.002 0.004 0.006
0 5 10 15 20
0.000 0.002 0.004 0.006
0.0 0.2 0.4 0.6 0.8 1.0
0.08 0.04 0.00 0.04 0.08
0.0 0.2 0.4 0.6 0.8 1.0
0.08 0.04 0.00 0.04 0.08
0.0 0.2 0.4 0.6 0.8 1.0
0.08 0.04 0.00 0.04 0.08
Figure 4.3: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
0 5 10 15 20
0.000 0.005 0.010 0.015
0 5 10 15 20
0.000 0.005 0.010 0.015
0 5 10 15 20
0.000 0.005 0.010 0.015
0.0 0.2 0.4 0.6 0.8 1.0
0.4 0.2 0.0 0.2 0.4
0.0 0.2 0.4 0.6 0.8 1.0
0.4 0.2 0.0 0.2 0.4
0.0 0.2 0.4 0.6 0.8 1.0
0.4 0.2 0.0 0.2 0.4
Figure 4.4: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
Chapter Four 30
0 5 10 15 20
0.000 0.002 0.004 0.006 0.008
0 5 10 15 20
0.000 0.002 0.004 0.006 0.008
0 5 10 15 20
0.000 0.002 0.004 0.006 0.008
0.0 0.2 0.4 0.6 0.8 1.0
0.02 0.01 0.00 0.01 0.02
0.0 0.2 0.4 0.6 0.8 1.0
0.02 0.01 0.00 0.01 0.02
0.0 0.2 0.4 0.6 0.8 1.0
0.02 0.01 0.00 0.01 0.02
Figure 4.5: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
Finally, when the curvature of the straight capillary is dominantly positive (or domi-nantly negative), then the peak magnitude of the Fourier transform of this curvature occurs at the zero frequency. On the other category, the curvature of the wiggly capillary oscillates. This makes the nonzero frequency is the maximizer of the magnitude of the Fourier transform of this curvature.
We also applied this approach to the image of capillaries which were the main object of [9]. There were two images that consist of representation from a wiggly and straight.
Due to the copyright, those images are not shown in this study. Hence, we present the outer edge of those capillaries in the first column of Figure 4.6 and Figure 4.7.
The outer edge is quite smooth because the image of the capillary is very clear. It seems the image is captured by a very special device rather than a microscope. The curvature is shown in the middle column. For the wiggly capillary, the oscillation of the curvature is easy to be seen. For the straight capillary, there is only one significant hill or mount.
There is oscillation, but it is relatively small compared to the big hill.
The result of the Fourier transformation shown in Figure 4.6 and Figure 4.7 is as the expectation. For the straight capillary, the peak magnitude occurs at the zero frequency.
For the wiggly capillary, the peak magnitude is at a frequency around 4.
Chapter Four 31
0 5 10 15 20
0.000 0.003 0.006 0.009
0 5 10 15 20
0.000 0.003 0.006 0.009
0 5 10 15 20
0.000 0.003 0.006 0.009
0.0 0.2 0.4 0.6 0.8 1.0
0.12 0.06 0.00 0.06 0.12
0.0 0.2 0.4 0.6 0.8 1.0
0.12 0.06 0.00 0.06 0.12
0.0 0.2 0.4 0.6 0.8 1.0
0.12 0.06 0.00 0.06 0.12
Figure 4.6: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
0 5 10 15 20
0.000 0.002 0.004 0.006
0 5 10 15 20
0.000 0.002 0.004 0.006
0 5 10 15 20
0.000 0.002 0.004 0.006
0.0 0.2 0.4 0.6 0.8 1.0
0.06 0.03 0.00 0.03 0.06
0.0 0.2 0.4 0.6 0.8 1.0
0.06 0.03 0.00 0.03 0.06
0.0 0.2 0.4 0.6 0.8 1.0
0.06 0.03 0.00 0.03 0.06
Figure 4.7: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
Chapter Four 32
0 5 10 15 20
0.000 0.002 0.004 0.006
0 5 10 15 20
0.000 0.002 0.004 0.006
0 5 10 15 20
0.000 0.002 0.004 0.006
0.0 0.2 0.4 0.6 0.8 1.0
0.06 0.03 0.00 0.03 0.06
0.0 0.2 0.4 0.6 0.8 1.0
0.06 0.03 0.00 0.03 0.06
0.0 0.2 0.4 0.6 0.8 1.0
0.06 0.03 0.00 0.03 0.06
Figure 4.8: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
0 5 10 15 20
0.000 0.005 0.010 0.015 0.020
0 5 10 15 20
0.000 0.005 0.010 0.015 0.020
0 5 10 15 20
0.000 0.005 0.010 0.015 0.020
0.0 0.2 0.4 0.6 0.8 1.0
0.4 0.2 0.0 0.2 0.4
0.0 0.2 0.4 0.6 0.8 1.0
0.4 0.2 0.0 0.2 0.4
0.0 0.2 0.4 0.6 0.8 1.0
0.4 0.2 0.0 0.2 0.4
Figure 4.9: The red curve (first column) is the outer edge of the capillary while the blue star is the first three points to calculate the curvature. From upper to below, the distance of these points is longer. The middle and the third column shows the curvature
and the magnitude of the Fourier transformation.
Chapter Four 33 We also present the results which are the patterns of the capillaries are abnormal. Both capillaries in Figure 4.8 and Figure 4.9 does not have meaningless meandering. However, there exists a jump and the branch. The existence of this jump makes the peak magni-tude occurs at a nonzero frequency. For the existence of the branch of the capillary in Figure 4.9, it also makes a nonzero frequency become the maximizer of the magnitude.
From these results, if the given input is classified into wiggly, then no further information can be added.