A Study on Image Processing Method for Crack Inspection of Real Concrete Surfaces

(1)

A Study on Image Processing Method for Crack Inspection of Real Concrete Surfaces

コンクリート表面のひび割れ検査のための画像処理手法の研究

February 2008

Major in Pure and Applied Physics

Graduate School of Science and Engineering Waseda University

Tomoyuki Yamaguchi

(2)

i

Introduction

1.1 Social background

In recent years, the ecology movement has brought about the expansion of life spans of existing structures beyond the simple age of “scrap and build” in civil and construction engineering. Gen- erally, concrete is one of the industrial-strength materials, and then it is frequently used for buildings. Nevertheless, the concrete degradation is a one weak point when using for long time.

This degradation effect is caused by several factors: earthquakes, frost damage, salt erosion, rain water, and dry shrinkage. There are many kinds of concrete degradation. In particular, the cracks on the concrete surface are the earliest and simplest indications of the concrete degradation.

Therefore, cracks can be a crucial factor for construction diagnosis and a signal flag for the first step of the concrete degradation investigation.

The conventional crack inspection method is to sketch the cracks detailed manually which measuring the concrete condition simultaneously. After received the cracks details, the crack patterns are estimated. Finally, the part of maintenance and repair are diagnosed. Figure 1.1 illus- trates the distribution map of crack inspection. Especially in the detection and measurement steps, this manual approaches analysis is depending on the specialist’s knowledge, experience and lack objectivity in the quantitative analysis. Therefore, the image processing technique for visual inspection has been increasing for utilizing as an alternative manually sketches in construction area.

Also the visual inspection is highly sought for non-destructive and safety management construction measurement. For practical use, it is necessary to detect all of the cracks and to measure must more than 0.2 [mm] width, in the detection and measurement steps.

(5)

2

Fig. 1.1 Flowchart of crack inspection.

1.2 Academic background

Some methods for crack detection and measurement by means of visual inspection have been proposed by going with development of recent information technology. The crack detection is a first step in the process of crack inspection. The accuracy of its detection affects after processes which are crack measurement and pattern estimation. Therefore, many papers about crack detection have been reported [1]. Abdel-Qader et al. [2] suggested a comparison of the effectiveness of crack detection in the images of a bridge surface by using the wavelet transform, Fourier transform, Sobel filter, and Canny filter. They concluded that the wavelet transform is significantly more reliable than the other methods. Hutchinson et al. [3] used a Canny filter and the wavelet transform for crack detection and estimated the parameters using the receiver operating characteristic (ROC) analysis. Ito et al. [4] and Tanaka et al. [5] have also proposed an automatic visual inspection system using images captured by a digital camera. This system can extract and analyze cracks on the concrete surface by combining several image processing techniques including the wavelet transform, shading correction, and binarization. Kawamura et al. [6] proposed a method using a genetic algorithm for the semiautomatic optimization of image processing parameters for precise crack detection. However, these methods do not consider the essential characteristics of cracks such as its connectivity. Miwa et al. used watershed segmentation to detect crack lines on a tunnel [7]. However, this method does not detect cracks with high precision since it mainly focuses on the watershed for the region separation. Also these methods use global image processing method such as the wavelet transform by focusing on the characteristics of the entire image.

Some approaches employ local image processing for crack detection. The local image processing is necessary to extract the typical local characteristics of cracks as the direction and connectivity.

Roli [8] proposed a method utilizing conditional texture anisotropy for crack detection in granite 1. Crack detection

Drawing up the sketch of the cracks 2. Crack measurement

Measurement of width, length, direction and location of cracks by using the sketch 3. Classification of crack patterns

Categorization of cracks based on its patterns 4. Diagnosis of concrete wall

Diagnosis of the extent of deterioration of the crack, and judgment the presence or absence of the repair based on the measurement result and the pattern

(6)

3

slabs. This method uses the orientation feature in the local window. Fujita et al. [9] proposed two preprocessing methods using the subtraction method and the Hessian matrix. Since the local window is fixed, these methods cannot be flexibly applied to different widths.

The images of real concrete surfaces contain some noises such as concrete blebs, stains, and shadings in several sizes. The crack is difficult to distinguish from the image with noises by the conventional methods which do not use the characteristic of cracks. Moreover, the methods whose window size is fixed are inadequate to extract accurate cracks, because the length and width of cracks are different on the real concrete surfaces. Therefore, the abovementioned conventional methods tend to miss cracks while tend to regard noises as cracks.

In crack measurement for practical use, the accuracy of measurement for crack width is re- quired sub-pixel order. Dare at el. [10] noted that an image is represented by the discrete array of pixel. The measurement with sub-pixel order is performed by the bi-linear interpolation. How- ever, the unit of measurement does not correspond “mm” but “pixel”. Chen et al. [11], [12] improved upon the method of Dare et al. [10], and then their method of sub-pixel measurement bases on DOG filter and quadratic curve interpolation. This method uses a test specimen. The crack measurement with the unit of mm is done by using the size of the specimen and the image size. The test specimen is a rectangular solid composed of the reinforcing bar and concrete. It is used for the load and vibration test. The measurement of crack width to the test specimen is mainly used to evaluate durability to the load and vibration. Sohn et al. [13] also similarly showed the measurement result of the crack width by the exact size, however, they used the test specimen. Commonly-observed things on these report is that the object is a test specimen, and the size of the object is already known. Therefore, these reports can be done the measurement with the exact size by corresponded the image size. Consequently, if the object is not a test specimen, it is impossible to apply these methods to measure with the exact size.

Furthermore, Sano et al. [14] proposed another measurement method with the exact size that two markers fixed on the concrete surface, and it measured the width of cracks between two markers in an image. However, the range of the measurement is limited because this method only measures on cracks between markers, and inefficiency. Schutter [15] measured the width of the crack with the exact size by using a video microscope. However, the detail of this method is not described, and this equipment is special hardware, and the range of the measurement is also narrow. Therefore, these methods do not use the advantage of visual inspection that can examine a wide range of structure surfaces in one shot by taking a picture of the desired areas.

In this way, there are the various problems to achieve the practical use of the visual inspection.

(7)

4

1.3 Objective

In this paper, we achieve four topics as follows.

1. Formulation of novel image processing method which takes into account the connectivity among neighborhoods.

We propose a novel local processing method based on a percolation model. It is considered that a feature of the percolation takes into account the connectivity among the pixels. The feature of our method is to evaluate the central pixel in a local window according to a cluster formed using the percolation process. This method employs a scalable window image processing that changes the termination condition of the percolation process; this makes the method flexible and robust. Therefore, it can work for crack detection even if the width or the length of the crack is different.

2. Application for crack detection to realize highly accurate and robust method for real concrete surfaces.

The images of real concrete surfaces contains some noise and different contrast, however, our method can detect cracks with high-accuracy by improved percolation-based image processing.

In order to detect the cracks with high fidelity, we assume that they are comprised of thin interconnected textures and describe our percolation model that extracts a continuous texture by refer- ring to the connectivity of brightness and the shape of the percolated region, depending on the length criterion of the scalable local image processing techniques.

3. Development of an efficient image processing for possible application of practical use.

The computation time is also important for practical use, because the size of the digital image reaches 10-mega pixels. On the other hand, the pixels that correspond to the crack are very few in a concrete surface image compared with the total pixel count of image.

Generally in local window processing, small window size would normally be used in reducing computational burden. Here, the percolation-based image processing is scalable local processing.

We developed an efficient algorithm for the computational burden. Our method can take small window size when the pixel is regarded as background. Only if the focal pixel is a crack pixel, the window size will be the large size.

4. Implementation of highly accurate crack measurement for concrete walls of real structures.

Crack width is important data in the inspection and diagnosis for safety management of concrete structures. The detection accuracy of the crack width requested for practical use is 0.1 [mm]

(8)

5

real size. In this study, we propose an image measurement method of crack width for the application to real concrete structures whose size is unknown, and clarify the efficient condition of our method for practical use. In order to measure the exact size of crack width, it is necessary to know correct size of the test specimens. However, it is difficult to know the size of the real object in advance when considering practical uses. Therefore, we attached a crack scale to the concrete surface during the step of image acquisition, because the crack scale has already been written the size per scale is clear. For the measurement of correct crack width, we use the brightness of width per the crack scale and compared it with one of the detected cracks.

The validity of the proposed technique is investigated through experiments with images of the real concrete surfaces.

1.4 Definition of cracks

Figure 1.2 shows examples of cracks and noises on a concrete surface. Cracks appear in the area represented by dash line. The noise patterns are indicated by rectangular areas. In this study, we assume that the cracks possess the following two characteristics: (1) their shape is thinner than the shapes of other textural patterns and (2) their brightness is darker than the background one.

Cracks with dark colors are easily detected, while cracks with unclear colors are difficult to detect since their brightness is similar to that of the background. The shape information is extremely effective for detecting the unclear cracks. Moreover, Figure 1.3 shows the enlarged view of the part of cracks. As you can see, to measure width of the crack with high accuracy, we should be use the sub-pixel information because width of the crack is also unclear.

Fig. 1.2 Cracks and similar noises.

(dashed line: cracked part; black rectangle: noisy part)

(9)

6

Fig. 1.3 Enlarged view of crack width.

1.5 Organization of the dissertation

In Chapter 2, we explain a novel local processing method based on a percolation model. The percolation model is used to represent the natural phenomenon of the permeation of liquid. The percolation takes into account the connectivity among the neighborhoods. In the proposed method, a cluster formation by the percolation process is performed first. Then, feature extraction from the cluster is carried out. Therefore, this method is a type of scalable window processing for realizing a robust and flexible feature extraction. We show three applications which are crack detection, noise reduction and edge detection to verify the effectiveness of our method.

In Chapter 3, we describe appropriate crack detection methods that include the specialized percolation processing technique. When the percolation processing is applied to crack detection, the calculation cost and accuracy are problems. However, these problems are trade-off relationship. Therefore, we propose two techniques which are the speeding-up method and highly-accurate method for crack detection based on the percolation-based image processing. Firstly, we explain the improved percolation process based on the shapes and brightness with noise reduction. The percolation model can realize robust and highly accurate crack detection by introducing the circularity of the percolated region as a sort of shape information. Additionally, the percolation processing technique uses the length criterion for scalable local processing. Next, we describe a speeding up method by utilizing the ideas of the sequential similarity detection algorithm (i.e. SSDA) and active search for the basic percolation processing. According to the concept of SSDA, the percolation process is terminated by calculating the circularity on the way.

Moreover, percolation processing can be skipped next pixel according to the circularity of neighborhood pixels. By using these techniques, the computation cost becomes low.

In Chapter 4, we explain an image measurement method of crack width for the application to real concrete structures. Our method attached crack scale on the concrete surface in the image

Crack width

(10)

7

acquisition. Crack width is measured by the brightness of crack scale areas on the image with sub-pixel order accuracy. In the real world, the lighting condition is different in various environ- ments. For its difference, we show clarification of the efficient condition in our method for the practical use.

In Chapter 5, we show the experimental results proving the effectiveness of our methods. The experiments include two experiments which are (1) evaluation of precision, (2) evaluation of computation cost for crack detection, and one experiment which is (3) crack measurement. Fi- nally, the conclusions are given in Chapter 6.

(11)

Percolation-based image processing

In Chapter 2, we explain a novel local processing method based on a percolation model. The percolation model is used to represent the natural phenomenon of the permeation of liquid. The percolation takes into account the connectivity among the neighborhoods. In the proposed method, a cluster formation by the percolation process is performed first. Then, feature extraction from the cluster is carried out. Therefore, this method is a type of scalable window processing for realizing a robust and flexible feature extraction. We show three applications which are crack detection, noise reduction and edge detection to verify the effectiveness of our method.

2.1 Motivation

Image processing methods can be roughly classified into three types; point processing, local processing, and global processing [16]. In point processing methods such as contrast enhance- ment and gamma correction, the brightness of the focal pixel is used to determine the resultant brightness. In local processing methods, the resultant brightness of the focal pixel is determined according to the brightness of the vicinity. Methods such as smoothing and sharpening fall into this category. Global processing like Fourier transformation involves the use of the whole image.

Using the general framework of the local processing method, we set the focal pixel in the center of the local window. Based on the information within the window, local processing is applied to change the focal pixel value. For example, the Sobel filter and Prewitt filter are used for the edge detection, and mean filter and median filter are used for the image restoration [18]. More- over, a variety of local processing methods exist for different purposes such as line detection [18], [19] and circle detection [20]. In these methods, the size of the window is fixed and therefore they have limitations with regard to the size of the feature that is to be detected. In order to solve this problem, several techniques that involve the use of variable window sizes have been proposed [21-23]. However, these techniques have problems with regard to window size selection.

(12)

9

We focus on a percolation model to apply a novel local processing method. We consider that the feature of percolation takes into account the connectivity among the pixels. The some applications of the percolation model to image processing have been attempted previously. Hussain et al. [24] applied a bond percolation model to the parameter estimation of Gibbs-Markov random fields for image segmentation. Craciunescu et al. [25] applied percolation to an MRI image for the analysis of tumor blood perfusion. The former is an example of the indirect use of the percolation model for determining the parameters of an existing image processing method. The latter is a type of analysis method for detecting the global blood flow in an image. Since these methods are designed for specific types of images, they cannot be applied to general image processing.

The concept of the seeded region growing algorithm for image segmentation [26] is similar to the percolation approach. However, it is not suitable to apply this method to regions with larger brightness variations, e.g., regions with impulse noises, because this process involves the selection of neighboring pixels that are only slightly different from the seeded region in term of brightness.

The feature of our method is to evaluate the central pixel in a local window according to a cluster formed by using the percolation process. Furthermore, this method employs a scalable window image processing by changing the termination condition of the percolation process; this makes the method flexible and robust. A variety of local processing tasks can be realized by ar- ranging the cluster classification criterion.

2.2 What is percolation?

The percolation [27], [28] is a physical model based on the natural phenomenon of the permeation of liquid. The percolation model is effective for describing various phenomena such as the spreading of epidemics, fires in orchards, ferromagnetism, and disordered electrical networks.

Figure 2.1 shows the examples of the percolation.

In this section, we consider a percolation model that is typified by a slow capillary displacement of fluid (e.g., oil) from a random porous medium due to another immiscible invading fluid (e.g., water) [29]. This percolation model has been used to describe the phenomena encountered during oil drilling. The percolation begins at an initial point, and then it spreads to the surrounding according to the probability p, which is a measure of the ease of percolation. The point in the nearest neighborhood that exhibits the largest value of p is percolated. By a repetition of this process, the region spreads in the surroundings and reaches the boundary. Therefore, it can be said that the percolation model takes into account the connectivity between each component in a certain system.

(13)

10

(a) Permeation of liquid

(b) Fires in orchards

Fig. 2.1 Examples of percolation.

Figure 2.2(a) shows an example of an image including the feature of linear connectivity. The probability p in the above explanation is replaced with the brightness of each pixel of the image.

When percolation occurs in the local region, as shown in Figure 2.2(b), it can be expected that only the connected pixels that are similar terms brightness are percolated and form a cluster.

(a) Whole image (b) Local image

Fig. 2.2 An example of an image with linear connectivity.

Figure 2.3 shows examples of the percolation processing for two images; one does not have linear feature. Another has linear feature. If the focal pixel belongs to the connective region, the percolated region grows linearly. On the other hand, if the focal pixel belongs to the background which does not have linear feature, the percolated region grows two-dimensionally. Thus, the shape of the percolated region depends on the characteristics of the focal point.

(14)

11

(a) not linear feature

(b) linear feature

Fig. 2.3 Examples of the percolation processing.

2.3 Algorithm of percolation-based image processing

In the proposed method, the percolation model is applied to evaluate the central pixel in the local region. This section describes our algorithm in the case of the line detection as an example of the typical image processing. In this study, we use the images with 256 gray-scale levels. For the values of brightness, we set 0 to represent black and 255 for white.

2.3.1 Basic percolation processing

The proposed method is based on a percolation model of a two-dimensional square lattice. The square lattice of the general percolation model considers only four neighbor-hoods; however, in this study, we adopt eight neighbor-hoods. Although the percolation model is usually applied to the entire region in physical applications, the model in the proposed image processing method is only applied to a local region in order to examine the central pixel according to the local information. In the percolation process, two different cases (Case 1 and Case 2) are considered based on the relation between the object on which the system user focuses and the background. In Case 1, the brightness of the focal pattern is lower than that of the background. In Case 2, the brightness of the focal pattern is higher than that of the background. In this study, our method is applied to impulse noise reduction. Therefore, we consider both the abovementioned cases because impulse noises include both white and black noises. The case is determined according to the relation between the focal pixel and the local region described in Section 2.4.2.

The basic percolation process consists of four steps, as given below. In this process, the second and the third steps of the two cases are different.

(i) First, the size of the local window is fixed as N × N, and we set the initial pixel in the center of the local window. The initial pixel belongs to the percolated region D_p, and the initial

(15)

12

threshold is set to the value of the initial pixel brightness.

(ii) The threshold T is updated as follows:

Case 1:

max(max( ( )), )

p Dp

T V p T w

=

∈

+

(2.1)

Case 2:

min(min( ( )), )

p Dp

T V p T w

=

∈

−

(2.1')

where V(p) is the brightness of the pixel p and the acceleration parameter w is used to accelerate the percolation.

(iii) Case 1:

D_c is defined as the region consists of the eight neighbor pixels of D_p. In D_c, the pixels whose brightness is below the threshold T are regarded as a part of D_p. If there is no such pixel, then the pixel with the lowest brightness in D_c is regarded as a part of D_p.

Case 2:

In D_c, the neighborhood region of D_p, the pixels that have a brightness above the threshold T are regarded as a part of D_p. If there is no such pixel, then the pixel with the highest brightness in D_c is regarded as a part of D_p.

(iv) When D_p reaches the boundary of the local window, the percolation process is terminated.

Otherwise, the process returns to step (ii).

Thus, the final D_p is defined, and a cluster is formed by the percolation process. We can then evaluate the focal pixel whether the pixel belongs to the interest region or not by characterizing the cluster. Figure 2.4 shows the flowchart of the basic percolation process.

Fig. 2.4 Flowchart of the basic percolation process.

End condition Update threshold T

Boundary pixel is percolated No Yes

Set the window size N × N and set the threshold T with the focal pixel

The neighborhood pixel is percolated

(16)

13

2.3.2 Scalable percolation processing

In Section 2.3.1, we proposed fixed window processing according to the percolation model.

However, since the size of the window is fixed, the previous approach cannot adapt to scalable features, including lines of several sizes in the same image. For example, when we are only in- terested in the line features, a window with a fixed size cannot be used to detect lines with variable sizes. In other words, small windows can be used to detect short and narrow lines, but not long or wide lines. Therefore, in such cases, it is essential that the window be scalable. This section describes scalable window processing by a modification of the basic percolation processing explained in the previous Section 2.3.1. In this part, we had considered only Case 1. The scalable window processing algorithm is described below.

(i) N is set to the initial value N₀, and the basic percolation process described in Section 2.3.1 is performed. N is first incremented into N + 2.

(ii) The process is the same as that in step (ii) of Section 2.3.1.

(iii) In D_c, the neighborhood region of D_p, the pixels that have a brightness below the threshold T, are regarded as a part of D_p. If there are no such pixels of this type, the percolation process is terminated. Otherwise, N is incremented into N + 2.

(iv) If N is larger than the maximum window M, the process is terminated. Otherwise, it returns to the step (ii).

N starts from an initial value of N₀, which is the minimum size of the window, and increments until percolation ceases or N₀ reaches the maximum size M.

Figure 2.5 shows the flowchart of the scalable window process based on this percolation model.

Fig. 2.5 Flowchart of the scalable window process.

Increase the window size N→N+2

End condition Update threshold T

No Yes

Set the initial size as N₀×N₀and the maximum size as M×M

N > M

The neighborhood pixel is percolated Yes No

Basic percolation process

(17)

14

2.3.3 Pre-experiment 1

In this section, we examine the scalable window processing for line detection and compare it with the basic percolation processing. In the present experiment, we estimate the circularity F_circle as a characteristic of the percolated cluster; it is expressed by the equation

2

4 _count

circle

max

F C

π

C

= ⋅

⋅ ^(2.2)

where C_count is the number of pixels in the cluster and C_max is the diameter of the circumcircle in the percolated cluster that is formed by the proposed scalable window processing. The F_circle value ranges from 0 to 1. For example, in Figure 2.3(a), F_circle value of the upper image is close to 1 because the cluster shape is nearly circular. On the other hand, the F_circle value of the bottom image is close to 0 because the shape is linear, and different from a circle. We can distinguish between the lines by using F_circle. Percolation processing is carried out at each pixel. Further, the brightness of the focal pixel in the output image is associated with the F_circle value by setting it to F_circle × 255.

(a) Test image (b) Fixed window of 5×5

(c) Fixed window of 21×21 (d) Scalable window Fig. 2.6 Pre-experiment 1.

(18)

15

Figure 2.6(a) shows the test image with several lines with sizes of 1 × 10, 3 × 25, and 5 × 25;

some of these lines exhibit gradation. The image size is 84 × 68 [pixel]. Both the basic percolation processing and the scalable one were applied to this image. For the basic percolation processing, the fixed window size is 5 × 5 and 21 × 21. For the scalable processing, the minimum window size is 3 × 3 and the maximum window size is 67 × 67.

The results are shown in Figures 2.6(b), (c), and (d). The 5 × 5 window detected only the short lines, while the 21 × 21 window detected only the long lines. On the other hand, it was possible to detect every line clearly by scalable window processing.

2.3.4 Pre-experiment 2

The actual images normally include many types of noises. This section verifies that our proposed method can be applied to images with noises.

Figure 2.7(a) shows the image including the Gaussian noise with variance σ²=4 and the impulse noise at a rate of 2 [%]. Figure 2.7(b) shows the result of line detection for Figure 2.7(a) using scalable window processing, where N₀ is set to 3 and w = 0. In this case, it is sensitive to the noise and the lines cannot be detected clearly because the percolation process is interrupted by impulse noise, as shown in Figure 2.8(b), and it is dispersed by Gaussian noise, as shown in Figure 2.9(b).

Figures 2.8(c) and 2.9(c) show the results of the cluster formation with the parameters N₀ and w set at 5 and 1, respectively. The cluster is expanding without being limited by impulse noise because of the setting of N₀, and it is not dispersed by Gaussian noise because of the setting of w.

In this manner, we realized robust image processing for an image with noises by the optimal setting of parameters, namely, the initial size N₀ and acceleration w. Figure 2.7(c) shows the results obtained for line detection with the parameter adjustment.

(a) Noisy image (b)Result of N0 = 3, w = 0 (c)Result of N0 = 5, w = 1 Fig. 2.7 Pre-experiment 2.

(19)

16

(a) Original (b) No - adjustment (c) Adjustment Fig. 2.8 Problem part 1.

(a) Original (b) No - adjustment (c) Adjustment Fig. 2.9 Problem part 2.

2.4 Applications

2.4.1 Crack detection

We conducted an experiment with images of real surfaces in order to study the proposed scalable window processing with the percolation model. In this section, our method is applied to crack detection in 25 images of concrete surfaces. Figure 2.10(a) shows one of the original images (Image 1) used in the experiment. The image size is 500 × 500 [pixel], corresponding to an area of approximately 250 × 250 [mm] on the actual surface. The parameters of the scalable window processing are configured as follows: initial window size N₀ = 21, maximum window size M = 41, and acceleration parameter w = 1. The size of the initial window is determined according to the average length of the cracks. In order to determine the size of the initial window, we performed a statistical analysis of the crack length in a digital image of the concrete surface by using the previous measurement system [4]. In the statistical analysis, we used an image with a size of 3040 × 2008 [pixel], corresponding to an area of approximately 1.5 × 1.0 [m]. Next, we measured 114 cracks and obtained an average length of 20.4 [pixel]. The maximum window size M is determined as 41, which is twice the average length. The acceleration parameter w is determined ex-

(20)

17

perientially. The feature extraction parameter is calculated by Eq. (2.2). For comparison, we carried out fixed window processing with window sizes of 21 × 21 and 41 × 41, an acceleration parameter w of 1, and the previous method [15], which uses a combination of several image processing techniques, including shading correction and binarization. Figures 2.10 (b), (c), and (d) show the experimental results. The value of each result gives an image with 256 gray-scale levels, in which the dark-colored regions represent cracks. Figure 2.11 shows the binarization results with an optimal threshold for different methods. (a) and (b) are obtained by using a fixed window size of 21 × 21 and 41 × 41, (c) is obtained by the previous method [4], (d) is obtained by the scalable window size, and (e) is a reference image created manually by a human observer.

(a) Image (b) Fixed window of 21 × 21

(c) Fixed window of 41 × 41 (d) Scalable window Fig. 2.10 Crack detection (Image 1).

(21)

18

(a) Fixed window of 21 × 21 (b) Fixed window of 41 × 41

(c) Previous method [15] (d) Scalable window

(e) Reference image

Fig. 2.11 Binarization results (Image 1).

For the quantitative evaluation of crack detection, we defined the precision P and recall rate R as follows:

p r

R N

= N

(2.3)

p total

P N

= N

(2.4)

where N_p is the number of pixels classified correctly in the crack region of the binary image, N_r is the number of pixels in the crack region of the reference image, and N_total is the total number of pixels detected as cracks. Table 1.1 shows the R and P results of 25 images for each method.

In Figure 2.11, (a) can detect cracks, but the background noises are not eliminated; (b) can elimi-

(22)

19

nate the background noises, but the short crack is not detected clearly, as seen in the dashed circle; (c) can include the background noises, and the cracks are disconnected; and (d) can detect only cracks without noise.

In Table 1.1, the proposed scalable window processing shows higher average R values than the other methods. The scalable window shows lower average P values than the fixed window of 41

× 41. When the size of the window is larger, the miss-detection decreases to a spot noise. Because the size is not 41 in the scalable window, the noise part might be detected. In a few cases, the scalable window might be inferior to the fixed window of 21 × 21. This is because the percolation process continues to expand the percolated region when the contrast in the crack region is very low and w is relatively large. We should attempt to make the parameter w scalable.

Table 1.1 Quantitative analysis.

P R P R P R P R

1 0.9583 0.9722 0.9580 0.9678 0.8513 0.9599 0.7915 0.6459

2 0.8416 0.9726 0.8839 0.9726 0.4512 0.9738 0.5161 0.6825

3 0.4701 0.8985 0.5841 0.8824 0.1584 0.8663 0.3425 0.4653

4 0.5122 0.9955 0.6319 0.9955 0.1833 0.8643 0.1925 0.7256

5 0.9649 0.8412 0.9130 0.7655 0.8624 0.8949 0.7889 0.6992

6 0.9770 0.8163 0.9874 0.7965 0.8554 0.8095 0.5958 0.6716

7 0.8057 0.9074 0.8530 0.8656 0.7744 0.9623 0.4354 0.8262

8 0.9667 0.8601 0.9034 0.8528 0.8489 0.8584 0.6607 0.8316

9 0.6895 0.7232 0.7762 0.6918 0.7053 0.7845 0.3573 0.6388

10 0.9486 0.6915 0.9703 0.6715 0.8667 0.6098 0.7632 0.6298

11 0.7366 0.9553 0.7851 0.9543 0.7285 0.9325 0.2387 0.5243

12 0.9606 0.8842 0.9603 0.8783 0.8435 0.7634 0.9432 0.5844

13 0.9236 0.9789 0.9664 0.9573 0.8483 0.9773 0.7032 0.9434

14 0.5867 0.9674 0.5895 0.9653 0.4456 0.9565 0.3561 0.6101

15 0.7916 0.9834 0.8632 0.9654 0.7714 0.9834 0.5120 0.9578

16 0.7335 0.9807 0.7557 0.9785 0.7281 0.9919 0.6367 0.9244

17 0.9800 0.9035 0.9812 0.9035 0.8741 0.9220 0.7361 0.8755

18 0.9802 0.9376 0.9833 0.9227 0.9259 0.9062 0.8280 0.7427

19 0.9585 0.9575 0.8351 0.9181 0.9635 0.9452 0.8151 0.7742

20 0.8868 0.7661 0.8686 0.7570 0.8247 0.7556 0.3478 0.4805

21 0.9354 0.9700 0.9461 0.9692 0.8118 0.9795 0.4734 0.6814

22 0.9521 0.9758 0.9652 0.9758 0.8308 0.9611 0.7028 0.8223

23 0.9166 0.7892 0.9274 0.7892 0.9080 0.7846 0.5065 0.7537

24 0.9059 0.8143 0.9227 0.7991 0.9043 0.8108 0.7031 0.8014

25 0.3548 0.9656 0.4227 0.9060 0.2304 0.8872 0.2670 0.7582

ave 0.8493 0.8976 0.8671 0.8832 0.7486 0.8856 0.5811 0.7205

Image No. Scalable window Fixed window 41 Fixed window 21 Previous [15]

2.4.2 Noise reduction

In this section, our method is applied to noise reduction for six test images with impulse noise.

These images are well known images such as “Bridge,” “Boat,” “Airplane,” “Barbara,” “Cam- eraman,” and “Woman”. The parameters are given as follows: N₀ = 3, M = 5, and w = 0. In this

(23)

20

case, we use both types of percolation processing; they are used according to the following rules:

1 ( ) 2 ( )

ave ave

Case if V p l Case if V p l

⎧ <

⎨ ≥

⎩

^(2.5)

where l_ave is the average brightness of 3 × 3 of the initial window size. We use the feature of the neighborhood region D_c. The extracted feature F_a is given as follows.

max(max( ), min( )) ( ) min(min( ), max( )) ( )

p c ave

a

p c ave

D D if V p l

F D D if V p l

⎧ <

= ⎨ ⎩ ≥

^(2.6)

where, the brightness of the focal pixel is set to F_a.

The two test images are shown in Figures 2.12(a) and 2.13(a). Images corrupted by impulse noise with a probability of 1 [%] are shown in Figures 2.12(b) and 2.13(b). The image size is 256

× 256 [pixel]. The impulse noise is applied with a probability of 0, 1,…, 10 [%]. For comparison, we use the fixed window percolation processing, median filter, and PSM (progressive switching median) filter [30]. These are fixed with a 3 × 3 window. In addition, the predefined threshold value T_I is 40 and the iteration number N_D is 3 for the PSM. The results for the images with the 1 [%] impulse noise are shown in Figures 2.12 and 2.13.

The mean square error (MSE) defined below is used to evaluate the restoration performance.

2 1

( ) MSE

K

i i

i

u v K

=

−

= ∑

(2.7) where K is the total number of pixels in the image and u_i and v_i are the pixel values at position i in the original and the resultant images, respectively. Figure 2.14 shows results of MSE for the six images.

For the median filter, there is a problem with regard to defocusing, as shown in Figures 2.12(e) and 2.13(e). The PSM filter has two problems associated with it. 1) If there are many pixels with the brightness similar to that of the impulse noise, then the impulse noise is not detected, as shown in Figure 2.12(f). 2) In fixed window processing, the line loses its linearity, as shown in Figure 2.13 (region A). In the case of the first problem, the preservation of the proposed scalable window processing is comparable to PSM filter. With regard to the second problem, our method preserved the linearity of the line, as shown in Figure 2.13(d). This is because our method takes into account the connectivity. As shown in Figure 2.14, the proposed method is not superior for cases in which the noise exceeds 3 [%]; this is because some noises are connected. At this point, it is confirmed that the MSE of our method is very small when impulse noise is not present. This characteristic is not seen in the other filter processing methods.

(24)

21

(a) Original image (b) Noisy image (1 [%]) (c) Fixed window

(d) Scalable window (e) Median filter (f) PSM filter Fig. 2.12 Noise reduction for “Bridge”.

(a) Original image (b) Noisy image (1[%]) (c) Fixed window

(d) Scalable window (e) Median filter (f) PSM filter Fig. 2.13 Noise reduction for “Boat”.

A A

(25)

22

(a) Bridge (b) Boat

(c) Airplane (d) Barbara

(e) Cameraman (f) Woman

Fig. 2.14 Robustness for the variation of the probability.

(◆: Fixed window, □: Scalable window, ×: PSM, : Median)

2.4.3 Edge detection

In this section, our method is applied to edge detection for three artificial images and the test image “Barbara.” As in Case 1, the parameters are given as N₀ = 3, M = 5, and w = 0. The extracted feature F_d is given by

d ave

( )

F = C − V p

(2.8)

where C_ave is the average brightness of the percolated cluster. The brightness of the focal pixel is set to F_d. For comparison, we use a 3 × 3 Sobel filter, which is often used as a simple detector of

(26)

23

the horizontality and verticality of edges based on 1st derivation and a 3 × 3 Prewitt filter that calculates the maximum response of a set of eight convolution kernels to determine the local edge orientation for each pixel.

The three artificial test images which are binary images are shown in Figure 2.15. The image sizes are 64 × 64 [pixels] and 128 × 128 [pixels]. The test image “Barbara” is shown in Figure 2.16. The results are presented in Figures 2.15 and 2.16. As is evident in these figures, the edge detection feature of the proposed method is clearly better than that of the other methods, even when the edge interval variation is large. The reason is that the proposed technique performs edge detection based on the differences in the features of the cluster, while the other methods detect the brightness differences among the pixels in a region.

Original Sobel Prewitt Proposed (a) Vertical striped (64 × 64)

Original Sobel Prewitt Proposed (b) Hatched line (64 × 64)

Original Sobel Prewitt Proposed (c) Mixed line (128 × 128)

Fig. 2.15 Artificial images.

(27)

24

(a) Original image (b) Proposed method

(c) Sobel filter (d) Prewitt filter Fig. 2.16 Edge detection for “BARBARA” (256 × 256).

2.5 Discussion

In this Chapter, we describe a method of scalable window processing based on a percolation model that takes into account the connectivity between the focal pixel and the neighborhoods.

The proposed method consists of the optimization of the window size by percolation and the characterization of the percolated region. The effectiveness of our method is verified by experiments on crack detection, noise reduction, and edge detection. With regard to crack detection, we presented a quantitative analysis of the performance of our method. The proposed method can also preserve the connectivity of the cracks in addition to detecting them. With regard to noise reduction, we showed that the MSE of our method is comparable with that of conventional methods. Our method was successful in preserving the original image because percolation processing takes into account the connectivity. Hence, our method was able to detect the fine edges clearly in comparison with the existing methods.

For the task of crack detection, the percolation-based image processing is regarded as the suitable crack detection method. However, problems of our method are remained, which are reduction the computation time and set the parameter w scalable for high-accurate detection. In the hereafter, we describe the percolation processing described in this chapter as the basic percolation processing.

(28)

Application for Crack Detection

In Chapter 3, we describe appropriate crack detection methods that include the specialized percolation processing technique. When the percolation processing is applied to crack detection, the calculation cost and accuracy are problems. However, these problems are trade-off relationship.

Therefore, we propose two techniques which are the speeding-up method and highly-accurate method for crack detection based on the percolation-based image processing. Firstly, we explain the improved percolation process based on the shapes and brightness with noise reduction. The percolation model can realize robust and highly accurate crack detection by introducing the circularity of the percolated region as a sort of shape information. Additionally, the percolation processing technique uses the length criterion for scalable local processing. Next, we describe a speeding up method by utilizing the ideas of the sequential similarity detection algorithm (i.e.

SSDA) and active search for the basic percolation processing. According to the concept of SSDA, the percolation process is terminated by calculating the circularity on the way. Moreover, percolation processing can be skipped next pixel according to the circularity of neighborhood pixels.

By using these techniques, the computation cost becomes low.

3.1 Summary of percolation-based crack detection

In this section, we summarize the method of the percolation-based image processing for crack detection. The method uses 256-level grayscale images. We set black to 0 and white to 255 for representation. The percolation process for crack detection is described below.

(i) At first, the size of the initial window is fixed as N × N and the maximum window size is defined as M × M. The pixel located at the center of the local window is set as the initial pixel p_s for percolation and included in the percolated region D_p. Further, the percolation threshold T is set to the value of the initial pixel brightness I (p_s).

(ii) The threshold T is updated as follows:

(29)

26

max(max( ( )), )

p Dp

T I p T w

=

∈

+

(3.1)

where w is an acceleration parameter used to accelerate the percolation.

(iii) The eight neighboring regions of D_p are defined as the candidate region D_c. In D_c, the pixels whose brightness is lower than the threshold T are percolated and included in D_p. If there are no such pixels, the darkest pixel in D_c is included in D_p.

(iv) When D_p reaches the boundary of the N × N window, N is incremented to N + 2 and the percolation process proceeds to step (v). Otherwise, the process goes back to step (ii).

(v) The threshold T is updated in the same manner as that in step (ii).

(vi) In the neighboring regions D_c of D_p, the pixels whose brightness is lower than T are included in D_p. If there are no such pixels, the percolation process is terminated.

(vii) When Dp reaches the boundary of the N x N window, N is incremented to N + 2. Otherwise, the process goes back to step (v).

(viii)If N is larger than the maximum window M, the process is terminated. Otherwise, the process goes back to step (v).

Conclusively, we obtain the final D_p as the resultant region of the percolation process, as shown in Figure 3.1. Then, we can evaluate the focal pixel to determine whether it belongs to a crack by characterizing D_p. We estimate the circularity F_c as a characteristic of D_p by using Equa- tion (2.2).

The F_c value ranges from 0 to 1. For example, the F_c value of the image is close to 1 because the shape of D_p is nearly circular, as shown in Figure 3.1 (a). On the other hand, the F_c value of the image is close to 0 because the shape of D_p as a crack is linear, and it is completely different from a circle, as shown in Figure 3.1 (b).

(a) background

(b) crack

Fig. 3.1 Examples of the percolation process.

(30)

27

Consequently, the brightness of the focal pixel in the output image is associated with the F_c value by setting it to F_c × 255. Percolation processing is carried out for every pixel in the input image. We distinguish whether the pixels are included in the cracks or not by using the value of F_c × 255.

3.2 Crack detection method with high accuracy

The percolation processing is a problem that it is difficult to detect the unclear cracks accurately.

In Section 2.4, we observed that this algorithm requires improvements to make the parameter w scalable. When the contrast in the cracked region is very low (i.e. unclear), this parameter is often too large to form an accurate crack cluster as the percolation is progressing, since the previous percolation model depended only on the brightness. Therefore, in this situation, the circularity becomes closed to 1, and then unclear cracks are regarded as backgrounds.

This section describes an improved percolation processing technique for crack detection. The improved features of the percolation model are as follows: (1) the parameter w is modified in the percolation process and (2) the criterion size for the percolation process is replaced by the length of the percolated cluster. Additionally, we explain a noise reduction technique for cracks based on the percolation model.

3.2.1 Utilizing shape information

The parameter w can be modified according to the percolated cluster. In the improved process, we use F_c , which is calculated every time the percolation process executes step (ii) described in Sect. 3.1. F_c is calculated by Equation (2.2) using D_p of the immediate shape at each iteration.

The improved parameter w' is obtained by using the following equation:

'

_c

w = F w ⋅

(3.2)

Then, Equation (3.1) is improved as follows.

max(max( ( )), ) '

p Dp

T I p T w

=

∈

+

(3.1')

where, when the percolated cluster forms a thin shape (i.e. F_c approaches 0), the parameter w' rapidly decreases. When the percolated cluster forms a circle (i.e. F_c approaches 1), the parameter w' is almost constant. Therefore, the improved percolation processing technique uses Equation (3.1') instead of Equation (3.1) in step (ii) of Sect. 3.1.

A further improvement is that we employ the length of the percolated region and not the window size. This improvement is simple, since the proposed percolation processing is a scalable window processing technique. In Figure 3.2, the percolated cluster can be evaluated using the same criterion, even if the initial points are the front or intermediate points of the cracks.

(31)

28

(a) window size (b) cluster length Fig. 3.2 Length criterion.

Fig. 3.3 Flowchart of the improved percolation process.

The improved descriptions are modified as follows: the initial window size N × N is replaced with length N, the maximum window size M × M becomes length M, and step (iv) and (vii) described in Sect. 3.1 are modified as follows:

(iv') When N is greater than C_max, N is incremented to N + 2 and the percolation process proceeds Set the initial length N, the maximum

length M and parameter w. Further, set the threshold T using the focal pixel.

Update T using Equation (3.1')

Calculate C_max of D_p

The neighboring pixel is percolated using T.

Cmax > N Increment N→N+ 2

The neighboring pixel is percolated Update T using Equation (3.1')

N > M

The darkest pixel in Dc

is included in Dp.

Calculate Fc value using Equation (2.2) No

Yes No

Yes

No Yes

No

Yes

Calculate Cmax of Dp

Cmax > N Increment N→N+ 2 No

Yes N

N N

N

(32)

29

to step (v). Otherwise, the process goes back to step (ii).

(vii') When N is greater than C_max, N is incremented to N + 2 and the percolation process proceeds to step (viii). Otherwise, the process goes back to step (v).

Figure 3.3 shows the flowchart of the improved percolation processing. The improved parts are represented as the dashed boxes. The update process of T is improved. To use the length as criterion of the percolation process, the calculation process of C_max is added instead of the window size.

3.2.2 Verification

In this section, we verify the effectiveness of parameter w’ compared with previous parameter w.

Figure 3.4 shows the original image and the results obtained by using the two methods for three types of cracks: “dark crack,” “background,” and “unclear crack,” respectively. A dark crack is noticeably darker than the background. On the other hand, an unclear crack is brighter than ordi- nary cracks. The parameters of the improved percolation processing are set as follows: initial length N = 21, maximum length M = 41, and parameter w = 1.

In the dark crack and the background cases, the results reveal percolated regions having almost comparable shapes. On the other hand, in the case of an unclear crack, the two methods yield percolated regions having different shapes. The previous method expands the shape of the percolated region; however, the improved method can maintain a percolated region with a thinner shape. The improved method thus detects unclear cracks in a manner similar to that for dark cracks. Moreover, Figures 3.5 (a) and (b) show the alteration of parameter w’ and threshold value T in these situations. From Figure 3.5 (a), the parameter w’ closes to 0 in the crack case, while the previous parameter w is fixed as 1. In the background case, the parameter w’ does not close to 0.

From Figure 3.5 (b), the improved method can be verified by the alteration of T. In the situation of dark and unclear crack, T stays almost constant. On the other hands, in the previous method, T gradually increases in all situations. Therefore, the unclear cracks are regarded as background.

(33)

30

(1) original (2) previous (3) proposed (a) dark crack

(1) original (2) previous (3) proposed (b) background

(1) original (2) previous (3) proposed (a) unclear crack

Fig. 3.4 Differences in the percolated regions.

(×: focal point for percolation; white part: percolated region).

(34)

31

(a) alteration of parameter w’

(b) alteration of threshold value T

Fig. 3.5 The effectiveness of the improved method.

3.2.3 Noise reduction for crack detection

In this section, a noise reduction method based on the percolation model is introduced. When noise reduction is performed after the binarization, the resultant noise reduction depends on the threshold value used in the binarization procedure. Therefore, we apply the noise reduction method, which is based on the percolation model, before the binarization.

The image produced by the improved percolation processing is represented by F_c × 255 (i.e.

circularity). One of the features of the cracks is their thin shape. When the focal pixel is regarded as a crack, the F_c of the neighboring pixels has a value similar to the F_c of the focal pixel, and these pixels are interconnected by a “constant length.” Therefore, the noise reduction method is

0 20 40 60 80 100 120 140 160 180

1 3 5 7 9 11 13 15 17 19 21 23

Number of iteration

Threshold value: T

dark crack ( w' ) dark crack ( w ) unclear crack ( w' ) unclear crack ( w ) background ( w' ) background ( w ) 0

0.2 0.4 0.6 0.8 1 1.2

1 3 5 7 9 11 13 15 17 19 21 23

Number of iteration

Paramter w'

dark crack unclear crack background

(35)

32

applied to the images using the F_c value. In this process, the constant length is then set to N.

Consequently, the resultant value of F_c on the focal pixel after the noise reduction depends on its connective length among the neighborhoods.

The improved percolation method can use the length of the percolated cluster. Then, in the percolation processing, N, M, and w are set to N, N, and 0, respectively. Further, the feature of the noise reduction is extracted as the maximum value F_m in the percolated region as follows.

max( ( ))

m p Dp

F I p

=

∈

(3.3)

Figure 3.6 shows an example of the results of the noise reduction. In this example, the parameter N is set to 21.

(a) image using Fc × 255 (b) after noise reduction Fig. 3.6 Example of the proposed noise reduction technique.

3.3 Speeding up method of crack detection

In this section, we introduce an efficiently high-speed method for crack detection employing a percolation-based image processing. In order to reduce the computation time, we consult the ideas of the sequential similarity detection algorithm (SSDA) and active search. According to the concept of SSDA, the percolation process is terminated by calculating the circularity on the way.

Moreover, percolation processing can be skipped next pixel according to the circularity of neighborhood pixels.

3.3.1 Termination procedure

Barnea et al. [11] introduced a class of sequential algorithms for the fast template matching, it will be referred to as SSDA. They were designed to reduce computation cost in matching proce- dures using similarity measures like the sum of the absolute differences. The matching process can be terminated on the way, if the similarity is monotonically decreasing and less than the cer-

A Study on Image Processing Method for Crack Inspection of Real Concrete Surfaces