[] S.Y. Chen, J. Zhang, Q. Guan, S. Liu, "Detection and amendment of shape distortions based on moment invariants for active shape models", IET Image Processing, Vol. 5, No.3, April 2011, pp. 273-285.

* This file is only for search engines. Please obtain the formally published article from the respective publisher or databases.

Detection and amendment of shape distortions based on moment invariants for active shape models

Detection and amendment of shape distortions based on moment invariants for active shape models

S.Y. Chen, J. Zhang, Q. Guan, S. Liu

Abstract: The active shape model (ASM) is an ever-increasingly important method for object modeling, shape recognition, and target localization. During the process of shape fitting, however, distortions and displacements often happen when the target is not clear or with defects, and there is a lack of effective amendment strategies in ASM. In this paper, under the inspiration of physics, the boundary moment invariants are employed to resolve this difficulty. It is the first time to introduce moment invariants into ASM for distortion detection and shape amendment. Using the proposed strategy, distortions are effectively avoided and the accuracy of fitting result is obviously increased with little extra time. Finally, the results of our practical implementation prove its satisfactory work.  

 


1  Introduction

The Active Shape Models (ASM), proposed by Cootes et. al. [1], is a well-known method for image registration, segmentation, and recognition. It represents the shapes by using a number of landmark points and shape variations by principal component analysis (PCA), it forms the point distribution model (PDM). By using different training sets, the ASM can be fitted any predefine shape. Since it was proposed, many improvements have been presented which lead to great increase in accuracy and reduction of time cost in the fitting process [2-5,13]. However, distortions still occur if the targets are not clear, for instance, when the medical image has weak boundaries or the initial model is placed far from the target image in the situation that the initial position can not be mended manually.

During the fitting process, the constraint of the shape transformation is defined by equation (1) according to the original ASM:

,

(1)

where X stands for the new shape,  is the mean shape computed by the PDM, p means the principal components which are deduced from the PDM, and the elements of vector b are the weights of each principal component. The similarity of the new shape to the initial model shape depends on the vector b.

The element values of the vector b are limited within certain ranges (e.g.  according to [1]) which are determined from the training sets. However, the distortion still occurs. For instance, there is an extreme example in Figure 1. In this example, we reduce the model from the 116 dimensions to 21 by applying PCA but holding on 95% of principal components. Anyway, a poor choice of PDM parameters results in a much distorted model. Although they do not exceed the limitations (which are showed in the second row), the new shapes are still distorted. Even though many literatures have reported good results using ASM in, e.g. face recognition, medical image segmentation, there are still some distorted results which cause fitting failures [4, 12, 13]. If we consider the application of firmer constraints, it will limit the transforming ability of the model. Thus, it is necessary to employ an effective strategy which applies relatively loose constraints and can detect and adjust the distortion.

 

In this paper, an effective detecting and amending strategy is proposed on the basis of the boundary Moment Invariants. The boundary moment invariants can be used to represent the geometrical feature of a model which is aggregated by landmarks. Firstly, we study the various moment invariants and find out the suitable moment invariants which can exhibit the exterior features of a shape in detail. Secondly, some subsets are separated from the model shape and the statistics of the boundary moment invariants of each subset are obtained from the image in the training set and the mean shape. According to these statistical values, we develop an automatic detection strategy. Thirdly, an amendment strategy is developed to effectively solve the distortion problem. Finally, the experiments are carried out on anklebone X-ray images and cardiac MRI (Magnetic Resonance Imaging) images to test the accuracy and reliability of the detection and amendment strategy for the model distortion during the fitting process.

The remainder of the paper is organized as follows. Section 2 outlines the standard ASM procedure and the method choosing the suitable boundary moment invariants for our purpose. The strategy of automatic detection and amendment is explained in Section 3 and Section 4. Practical experiments are shown in Section 5 and a conclusion is followed in Section 6.

 

2  Formulation

2.1  The Active Shape Model

ASM is important for image fitting, shape recognition, and shape localization. Because the models are built from the training set, any instances of an ASM can only be deformed principally in the ways found in its training set. Besides, the local features of structures are considered through modeling the grey-level information around each landmark.

Firstly, a shape model called Point Distribution Model (PDM), consisting of n landmarks that are the important points to represent the particular part of the object or its boundary of the image, is built to describe both typical shapes and typical variations by disposing a training set. The collection of these landmarks is used to define a shape vector, and they must be aligned with respect to a set of axes. The Principal Components Analysis (PCA) is applied to reduce the dimensions of the model and variations. Thus the vector b of t dimensions is obtained from Eq. (1). ASMs then use grey-level values to direct the model to find the new positions of landmark. A new model is formed by new positions of landmarks which are restricted by the parameters b in the equation  as in (1). In this way, we hope the new shape will not bring too large distortion to represent the object shape. However, in the standard ASM, there are still many situations where unexceptional distortion occurs due to some uncertain defects in arbitrary images, and such a distortion will lead to a fault result in the fitting process [12].

2.2  Selection of moment invariants

Some years ago, Hu proposed the moment invariants by applying the theory of algebraic invariant to the normalizing center moments [6]. These moment invariants are invariant to the scale, position and rotation of any objects [8-11,14].

In particular, there are seven moment invariants that are found useful as features for shape recognition. However, they are computed over all pixels in a region of interest, which is time-consuming. A new set of boundary moment invariants is proposed by Chen [7], which is computed only on the basis of the shape boundary.

The definition of those seven moment invariants are the same as that by Hu. Obviously, these new boundary moment invariants are computed much faster. In this paper, the later method is used to obtain the shape features of the model. Since a shape is formed by landmarks, we only need to compute the moments of these landmarks. To accurately detect the change of a shape, we need to investigate different boundary moment invariants and choose the suitable ones. All boundary moment invariants will change when the shape is transforming, but only those boundary moment invariants which vary obviously following shape changes are imperceptible and can meet our requirements. Since they are sensitive to shape variation, they can detect the geometrical distortion effectively. For obtaining the suitable boundary moment invariants, there are 14 moment invariants investigated in [15].

According to statistics we find that, in general, the movement of each landmark is mainly in the range of 0-100 pixels during the fitting process. Therefore we design the following method to select the suitable moment invariants. All vertexes of standard shapes (in our work they are triangle, square, pentagon and hexagon) are randomly moved in the limitation of 0-100 pixels, forming new shapes which are used to compute the moment invariants. Then the statistical information of the changes of different moment invariants is calculated. We use the following two rules to select the suitable moment invariants. (1) The differences of selected moment invariants are larger under the same limitation. Figure 2 illustrates an example of distribution of differences of moment invariants. (2) The increasing values of selected moment invariants are faster along with the increasing limitation. Table 1 lists an example. For the same limitation of the shape changing, the differences of moment invariants between the changed shape and the standard shape are different. There are some moment invariants whose differences are still large when the shape changes little. For distortion detection, this is necessary for them to detect the little change of shape. The moment invariants that feature large differences just meet this requirement. We therefore have the first rule: The differences of selected moment invariants between the changed shape and the standard shape are larger under the same limitation. Thus these moment invariants are more sensitive to the change of shape.

With the increasing of limit of shape changing, the increasing rates of difference of moment invariants between the changed shape and the standard shape are different. The rates of those moment invariants satisfied the first rule are larger than that of those moment invariants not satisfied it. The larger rates mean that the corresponding moment invariants can be more sensitive to shape distortion since they will get larger differences between the changed shape and the standard shape. That comes out the second rule. Thus these moment invariants can be more aware of the large change. It means that the distortion of shape can be detected more easily. From fourteen moment invariants [15], we select following six moment invariants which satisfied both two rules.

(2)

In these equations, means the i+j order normalized centre moment and means the moment invariants. According to the geometrical moment theory, low order moments contain the global information of a shape and high order moments denote the local details. By observing these boundary moment invariants, one can find that they are all combined by the high order moment. Therefore, these selected boundary moment invariants are suitable to represent the details of the shape exterior. For example, Table 1 shows the average differences of boundary moment invariants between the standard hexagon and the transformed hexagons. Each column shows the different transformed extent.  As an example, from Table 1 we can find that, (1) the values of moment invariants in the row of diff2 are larger than that in the row of diff1 in each of limitation; (2) along with the increment of limitations, the differences of moment invariants between two adjacent column in the row of diff2 are also larger than that in the row of diff1. Therefore it can be proved from both theory and experiments that the selected six moment invariants are suitable to assess the change of shape.

 

Table 1 Average differences of boundary moment invariants between a standard hexagon and transformed hexagons. Different columns mean different limitations of transformed extent. Diff1 means the average differences computed by the unselected boundary moment invariants. Diff2 means the average differences computed by the selected boundary moment invariants.

Limitation

10

20

30

40

50

60

70

80

90

100

Diff1

0.21

0.45

0.66

0.92

1.11

1.31

1.59

1.68

1.93

2.22

Diff2

10.33

12.72

18.72

22.65

27.28

29.72

31.19

36.55

38.40

39.98

 

Table 2 shows the average differences of the selected six boundary moment invariants of all four kinds of shapes. These transformed shapes are in different conditions with the range of random change limited in 10 to 100 pixels, and under each condition every kind of shapes is transformed 100 times. Along with the transforming to an enlarging extent, the differences of boundary moment invariants of four shapes between the standard shapes and the transformed shapes have a gradual increment. When shapes are transformed into a larger extent, they are more possible to distort and so the boundary moment invariants have much changes. On the other hand, if shapes are similar, their boundary moment invariants are also closed. Therefore, it is satisfied to make use of the difference of boundary moment invariants as measures of shape varying or shape distortion.

 

Table 2. The average difference of boundary moment invariants between standard shapes and transformed shapes.

Range of change

(pixels)

triangle

square

pentagon

hexagon

10

9.87

10.51

11.03

10.33

20

11.55

13.01

13.89

12.72

30

16.73

18.56

19.02

18.72

40

21.93

22.88

23.18

22.65

50

27.39

28.01

29.05

27.28

60

29.58

30.67

30.94

29.72

70

31.69

32.04

32.83

31.19

80

36.12

37.30

38.07

36.55

90

37.98

39.91

40.25

38.40

100

39.30

40.34

41.08

39.98

3  The active detection strategy

This section firstly describes a method to separate a whole model to several subsets by landmark stability weights. Then the statistics of boundary moment invariants are obtained by K-mean cluster method. At last, the detection strategy of shape distortion is proposed based on the statistical values.

3.1  Subset separation

When aligning all shapes in a training set, one can obtain a weighting matrix which is with more significance to the landmark points that tend to be stable. A stable point moves less with respect to other points in the shape [12]. Therefore, landmark points with the similar weights have the similar stability. That is, when the model is deforming to fit a new object, these landmarks with similar weights have the similar movements with respect to other points in the model. In the proposed method the subsets consist of those points that have similar weights, even if these points are disconnected, the relationship of their positions is more stable than that of those points just selected from local area. Therefore, the distribution of normal moment invariants of subset is more narrow, which makes the proposed method to easier to distinguish distorted subsets from normal subsets. According to this, we separate the whole model into a certain number of subsets. Each subset is composed of the landmark points which have the similar weights. Compared with the whole shape, these subsets are more stable in keeping their profiles since they are separated by stability weights. And if the change of the subset is great, the distortion is more likely to happen. For calculating the boundary moment invariants of a subset, the number of points in each subset must not be less than three. If the number of points in a subset is more than six, the weights of these points can not keep their similarity and the profile of the subset may be not stable. Here are the steps to separate subsets.

1) Sort the weights by their values from maximum to minimum.

2) Choose the maximal weight and select 3-6 points whose weights are most similar to the maximal weight to form the first subset. Note that these points are not continuous. Actually, we can use the following pseudocode to represent the algorithm for obtaining the subset:

文本框: Initialization: Denoting the first seven points as pw1, pw2, … pw7. Computing differences of weights between two adjacent points and denoting them as dw1, dw2, …, dw6. Sorting these differences from the max to min.
For (from the max difference to the min difference)
  If the number of current difference– i –is larger than 2
choosing pw1, pw2, …, pwi to form subset;
break;
  End
End

3) Choose another weight which is not in the first subset and select 3-6 points which weights are most similar to the maximal weight to form the second subset.

4) Deal with other points with the same way.

After separating the model, we obtain a certain number subsets, which are not overlap and not continuous.

3.2  Computation of the Clustering Values of the boundary moment invariants

During the fitting process, when the shape is transformed step by step, its boundary moment invariants change as well. From what has been discussed in subsection 2.2, the extent of shape transformation can be measured by the sum of the six boundary moment invariants. To begin with, the boundary moment invariants of each subset in the training set can be obtained. Because the fitting results are transformed from the model shape, we also need to calculate the boundary moment invariants of the random transformation of the model shape under a limited extent. In our study, at last 150 shapes are calculated. Figure 3 displays the histograms of these boundary moment invariants of each subset.

In Figure 3, the X-axis represents the values of boundary moment invariance of each subset and Y-axis represents the amount of subsets where the boundary moment invariance falls in the same range. From the figure, it is obvious that all the boundary moment invariants of subsets fall into certain ranges. For example, the third subset in Figure 3 shows that the scope of boundary moment invariants of the corresponding subset locates in the ranges of 0-5, 44-48 and 65-67, although there are only seven values of boundary moment invariants in the range of 65-67. The number is so few related to the amount of all subsets that we can ignore this range. All boundary moment invariants in the histogram are obtained from the manually labeled objects in training set and the normally transformed model. Therefore, if the fitting result of the third subset in a new image is normal, according to statistics, the boundary moment invariants of its third subset will have the same distribution, which means that its boundary moment invariant quite possibly locates in one of the two ranges (i.e. the range of 0-5 and the range of 44-48). On the other hand, if ASM fails to fit the object, the result model will be distorted and its boundary moment invariants will beyond the two ranges. Hence, the first step of distortion detection is to obtain the distribution ranges of boundary moment invariants of all subsets. In this paper, a kind of modified K-mean algorithm is employed to cluster these boundary moment invariants. According to the Figure 3, four samples are chosen as the initial cluster centers. We then use K-means to cluster the distribution. Firstly, we use the original K-means clustering method to cluster samples by setting the initial cluster number as 4. After clustering, the cluster whose number of samples is lower than a threshold will be merged into the nearest cluster and the numbers of clusters are 4, 4, 3, 4, 4 and 4 corresponding to each subset respectively. However, merging clusters cannot exclude some outliers from samples. Therefore we also check the amount of each cluster. If the rate of a cluster computing by diving the amount of this cluster to the amount of all samples is less than a threshold (e.g., 0.1 as in our experiments), this cluster is ignored. By using this method, the final numbers of clusters are obtained, which are 3, 3, 2, 2, 3 and 2. For each cluster, the mean-value, variance, maximal value and minimal value (they affect the range of this cluster) are calculated. According to these parameters, each new result transformed by ASM can be judged whether it is normal or distorted. As an example, the mean-values and variances of one group of our experiments are listed in Table 3.

 

Table 3 The clustering centre of each subset’s boundary moment invariants

 

Subset1

Subset2

Subset3

Subset4

Subset5

Subset6

mean

var

mean

var

mean

var

mean

var

mean

var

mean

var

2.53

1.23

0.85

0.49

1.73

0.47

0.96

0.51

1.26

0.49

1.6

0.73

53.29

2.81

55.68

1.67

54.15

0.53

46.58

0.55

49.84

2.21

44.96

1.64

71.65

2.74

70.87

1.03

 

 

 

 

67.03

1.76

 

 

 

In our practical strategy, all these parameters are obtained with statistical information. Actually we do not directly determine whether each subset in the fitting result is normal or distorted but use the distortion probability to describe the similarity between subsets of the result and those of the original model. Suppose that the boundary moment invariants in all locating ranges of each subset have a Gaussian distribution. The variance is Cij and mean is Mij. Then, the distribution function of the j-th clustering range of the i-th subset is

(3)

where σ is the square root of Cijequals to Mij, and Xi is the sum of boundary moment invariants of the i-th subset.

Therefore, the probability of the distortion of a new transformed subset can be computed as follows

 

(4)

where m is the number of the subsets, and n is the number of the clustering value of the subset.

3.3 Distinguish between shape distortions and shape classes

Objects in a same class have the similar shapes, and thus their boundary moment invariants should be very close. However, big difference of boundary moment invariants can be caused by shape distortion in the same class. On the other hand, the boundary moment invariants of objects in different classes can also have big difference. To distinguish them, it is necessary to train the objects of each class for the corresponding ASM and boundary moment invariants. In practice, before the process of fitting, an image classification algorithm can be applied to determine which the class of objects is in the image, e.g. [16]. By using this classification method, different object images (e.g. the cardiac images and the anklebone images) can be classified and the corresponding model is selected automatically. Usually the image statistical information (e.g. histograms of blocks) or feature vectors are extracted for such classification. Then a corresponding model is selected to fit the shape. Through such a pretreatment, the difference of boundary moment invariants is only caused by shape distortion.

3.4  Detection of distortions

The following steps are employed to detect the distortion during a fitting process. Firstly, the new transformed shapes are separated into several subsets according to the way discussed in Section 3.1. Secondly, the probabilities of the distortion of these subsets are computed. Finally, the minimum Dij is chosen as the probability of the distortion of current subset, which is defined as

 

(5)

According to the detection strategy, it is only possible to detect whether a subset is distorted or not. It is unknown whether one point in the subset is distorted or not. Hence, the amendment strategy will modify the whole distorted subset, which is described in the next section.

4  Active amendment strategy

During the fitting process of the ASM, when the shape has imperceptible changes, the boundary moment invariants of the shape also change slightly. However, if the shape is evidently distorted, the boundary moment invariants must be changed relatively largely. In our practical experiments, the boundary moment invariants of each transformed shape are calculated. Figure 4 shows the results of the boundary moment invariants where there are no distorted changes. Figure 5, by contrast, illustrates the result when distortion takes place, in which Figure 5(c) shows the shape transformation when the boundary moment invariants change largely. From them, we know that after the boundary moment invariants of each transformed shape are computed, the amendment strategy is necessary to perform only when the shape change is detected.

4.1  The condition to trigger the amendment strategy

During the fitting process, the boundary moment invariants of each subset at certain iteration are denoted as IMij, where i is the iterative time and j is the number of the subset. The differences among IMij are calculated as:

(6)

When dIMij is bigger than a certain threshold, the amendment strategy will be triggered. The threshold is set to distinct the sharp change of the boundary moment invariants of the shape. It can also be adjusted/adapted in different situations.

4.2 Amendment of a Shape

During the process of amendment, it is important to prevent the result from generating a new distortion. We investigate the two amendment strategies. (1) After a distorted subset is detected, it is compared with the corresponding subset in the previous iteration for finding out the landmarks which caused the distortion. Then these landmarks are modified independently without influencing other landmarks in these subsets. (2) After a distorted subset is detected, we adjust the whole subset in accordance with the distortion degree. In considering the first strategy, other landmarks still stay at the original positions when the distorted landmarks are modified, it will result in destruction of the entirety of this subset and easily cause new distortion. In contrast, if we adjust the whole subset according to the distorted degree and the position of this subset at previous iteration, the entirety of the subset can be maintained and thus the possibility of new distortion is decreased effectively. Figure 6 illustrates the difference between the two strategies. From it, the amendment to the whole sub-area can keep the stableness of the whole sub-area. The change to one point, on the other hand, maybe breaks the stableness and brings to new distortions. Therefore, in this paper, the first amendment strategy to the whole sub-area is mostly considered.

By comparing the model, result (d) is a better than (c). Furthermore, in the amendment strategy the whole model will be modified related to the amendment degree of distorted subsets. Thus, it can lead to a more accurate amendment result. Finally, the whole strategy used to modify the abnormal area is described in detail as follows.

1.       The movement of each point in the abnormal area comparing the corresponding point in the previous shape is calculated.

2.       The new point’s coordinates are modified according to the distortion probability of the subset which is obtained by the way discussed in section 3.

 

(7)

where xn and yn are the coordinates of the point in the new shape, and xp and yp are the coordinates of the corresponding point in the previous transformed shape, x'n and y'n are the new coordinates of the point and need to be revised. Di is the distortion probability of the subset. The modified distance of each point is denoted as dn'.

3.       Other points of the shape will be influenced by this modified point dn', as the equation (8) defined:

(8)

where xno and yno are the coordinates of point Po which is one of other points in the new shape, and xpo and ypo are the coordinates of the corresponding point of Po in the previous shape. dn is the movement of Po. dom is the distance between Po and Pn and is modified at step 2. The mdm is the maximum distance between the Pn and other points in the shape. According to this principle, when a distorted point is amended, other points should also be affected, and the new distortion probably arisen since the amendment can be avoided to a great extent.

4. After all points in abnormal areas being modified, a new shape without distortion is generated.

In the process of image fitting, at each iteration, it is necessary to calculate boundary moment invariants and revise the distorted model.

5  Experiments and results

5.1  Data Set

To evaluate the proposed method, 150 sets of X-ray anklebone images and 150 sets of MRI heart images are used to build two PDMs in experiments. The X-ray images are obtained at 50kv voltage and 20mA current. Each image is 512 x 512 with pixel size 0.37 x 0.37 mm2. The MRI image acquisition protocol is: TE = 90–120 ms, TR = 3500–6500 ms, Magnetic Field Strength=15000, slice thickness = 8 mm. Each slice is 256 x 256 with Pixel Spacing=1.56x1.56. To each set, 50 images are used to test the proposed method. We labelled the object with 30 and 38 landmarks for anklebone images and cardiac images, respectively. Figure 7 illustrates these landmarks and a sub-set of images. The four hundred images consist of normal and abnormal X-ray anklebone images and MRI heart images which come from a hospital with different ages and different sexes.

5.2  Results

In the experiments, we adopt the following strategy to evaluate the performance more accurately and sufficiently: 150 images in the training set are used to build PDMs and to get the statistics of boundary moment invariants at one experiment and the left 50 images are used to test. To evaluate the performance of our strategies under different conditions, both the training set and testing set consist of normal and abnormal images randomly selected from the database. These images are also manually labeled by experienced people for accurate measurement of the fitting results. The experiments of each set performed four times totally.

Figures 8 and 9 illustrate some search results of the new strategy and the traditional ASM. The initial position of the model is given in this way: first, the initial shape is placed at the center of the image for fitting; second, we manually adjust the scale, translate and rotation parameters to make the model close to the true position until the summary of distances of each pair of corresponding points is less than 50. The fitting process will then start. When we evaluate the original ASM and our ASM, the initial parameters of model are the same. In these figures, (a) is the initial position of the model, (b) is the fitting result by standard ASM, (c) is the result by our distortion-sensitive ASM, (e)~(i) are the minimum difference of the boundary moment invariants of each subset and the mean value of boundary moment invariants of each subset which mentioned in Section 3.2 at each iteration. The difference of the i-th subset at the k-th iteration is calculated as following equation:

                             (9)

 

where IMik means the boundary moment invariance of the i-th subset at the k-th iteration and Cij means the mean-value of the j-th clustering range of the i-th subset. By this equation we can measure the differences between the transformed model and the standard model. According to the meaning of boundary moment invariance, if two shapes are more similar, the differences between their boundary moment invariance are less. Therefore, if dik decreases gradually, the similarity between the transformed subsets and the standard one will increase. The solid line is the boundary moment invariants according to the new strategy, and the dash line is the boundary moment invariants according to the standard ASM. From the solid line, it can be found that the difference converges to zero, and the boundary moment invariants converge to the clustering value. The converged difference indicates that each subset of transformed model is close to the standard subset and the possibility of distortion in the transformed model is declined at very low level. Therefore, in the fitting result, the transformed shape can register the object well, but the dash line does not converge so well that the results are distorted.

The accuracy of these results is further evaluated by comparing the boundary moment invariants of the fitting result and the shape labelled manually. Table 4 shows the compared results. It can be found that the result obtained by the proposed distortion-sensitive ASM gets the closer difference of boundary moment invariants than one by the standard ASM. Because the more similar shapes have the closer boundary moment invariants, the fitting result by this strategy is more similar to the manual labeled object. At the same time, the accuracy of fitting results is defined as the following equation:

                               (10)

where  is the boundary moment invariants of fitting result and  is the boundary moment invariants of the manually labeled objects. The improvement by the strategy is defined as follows:

                          (11)

where  is the accuracy of fitting results by the strategy and  is that accuracy by the standard ASM. In our four groups of experiments, the highest accuracy of fitting results by both the standard ASM and our strategy are 99.5%. This indicates that the proposed method does not cause new distortion when the standard ASM can obtain a good fitting result. The correct fitting cases by the proposed strategy are 184 with the average accuracy of 86.6% and correct fitting cases by the standard ASM are 156 with the average accuracy of 77.3%. The increasing of average accuracy shows that the strategy is quite effective.


 

Table 4 Comparison of the boundary moment invariants among the result in Figure 8 and Figure 9


Row

sum of boundary moment invariants calculated from manual labelled model

sum of boundary moment invariants calculated from standard ASM

sum of boundary moment invariants calculated from distortion-sensitive ASM

improvement

1

163.21

180.26

169.31

6.7%

2

120.332

140.364

123.58

13.9%

 

For the further evaluation of accuracy, we employ the average error [4] to calculate the average distance between the manually labeled shapes and the fitting results.

                                  (12)

where N is the number of test images in one group of experiment, and n is the number of landmarks. Pij is the j-th landmark point in the manually labeled shape of the i-th test image manually labeled, Pij' is the j-th landmark point in the resulting shape for the i-th test image. Table 5 shows the average errors of the results fitting by the standard ASM and the distortion-sensitive ASM and the improvements of our proposed strategy. The first four rows are the data obtained by anklebone images and the second four rows are the data obtained by heart images.

Table 5 Comparison of the accuracy between the fitting results of standard ASM and distortion-sensitive ASM

Experiment group

Standard ASM

Average error

Standard ASM

variance of error

Distortion-sensitive ASM

Average error

Distortion-sensitive ASM

variance of error

Improvement

1

3.26

17.21

2.88

6.21

11.6%

2

3.45

19.75

3.03

8.33

12.2%

3

3.51

15.63

3.19

5.8

9.1%

4

2.97

17.38

2.53

6.72

14.8%

5

2.19

15.42

1.71

5.24

21.9%

6

2.03

13.66

1.72

4.97

15.3%

7

2.08

12.53

1.68

7.36

19.2%

8

2.34

16.18

1.92

4.98

17.9%

 

Furthermore, we compare the time cost of fitting process between the standard ASM and the presented strategy. The average time cost is found to be the 1.935 seconds and 2.126 seconds, respectively. This means the proposed strategy needs a little extra time cost to deal with the distortion detection and amendment. Nevertheless, considering the accuracy improvement, this little cost is worthy.

 


6  Conclusion

In this paper, to enhance the robustness and accuracy of image fitting and to avoid the distortion of the fitting result, we proposed an effective strategy to detect and amend the distorted shape which is completely active and automatic and based on boundary moment invariants. It is the first time that it was introduced into ASM for shape distortion detection, forming a distortion-sensitive ASM. Because the boundary moment invariants we selected are suitable to represent the details of the exterior shape, it is valid to detect the distortion of shape. With the statistics of boundary moment invariants of the shape in the training set and some other shapes which are randomly and slightly transformed from the model shape, the boundary moment invariants of a normal shape can be computed. Then, the shape distortion can be found by comparing the boundary moment invariants of transformed shape during the fitting process with that of a normal shape.

When the shape is identified with distortion, the amendment strategy is triggered for the abnormal subsets which are separated by the weights calculated in standard ASM. After the abnormal subsets are amended, other points in the shape would also be influenced by the effective strategy. In this way, new distortion which might be brought about from the adjustment is avoided and the distortion problem is solved satisfactorily.

 

Acknowledgment

The work is supported by the National Natural Science Foundation of China and Microsoft Research Asia (NSFC No. 60870002, 60802087), NCET, and ZJDST (2010R10006, 2009C21008, 2010C33095), and Program for New Century Excellent Talents in University. The authors thank the anonymous referees who provided many useful comments that helped to improve the paper.

 

8  References

1  Cootes, T.F., Taylor, C.J., Cooper, D., and Graham, J.: Active shape models--their training and application’, Computer vision and image understanding, 1995, 61(1), pp38-59

2  de Bruijne, M., van Ginneken, B., Niessen, W.J., and Viergever, M.A.: ‘Active shape model segmentation using a non-linear appearance model: application to 3D AAA segmentation’, Tech. Rep. UU-CS-2003-013, Institute of Information and Computing Sciences, Utrecht University, 2003.

3  Wan, K.W., Lam, K.M., Ng, K.C.: ‘An accurate active shape model for facial feature extraction’, Pattern Recognition Letters, 2005, 26, pp. 2409-2423

4  Wang, W., Shan, S.G., Gao, W., Cao, B., and Yin, B.C.: ‘An Improved Active Shape Model for Face Alignment’, Fourth IEEE International Conference on Multimodal Interfaces, Pittsburgh, USA, Oct. 2002, pp. 523-528.

5  Li, H.Q., and Chutatape, O.: ‘Boundary detection of optic disk by a modified ASM method’, Pattern Recognition, 2003, 36, pp. 2093 – 2104.

6  Hu, M.K.: ‘Visual pattern recognition by moment invariants’, IEEE Trans. on Inf. Theory, 1962, 12, pp. 179-187

7  Chen, C.C.: ‘Improved moment invariants for shape discrimination’, Pattern Recognition, 1993, 26, pp. 683-686

8  Yu, Z.Y., and Li, X.J., ‘Research on Stored—grain Microbe S Recognition Based on Moment Invarian’, Control and Automation, 2006, 6, pp. 251-253

9  Du, P., Zhang,Y.K., and Liu, C.Q.: ‘A Face Recognition Method Based on Moment Invariants’, Computer Simulation, 2002, 19, pp.78-81

10  Wang, Z.L., Mu, Zh.C., Wang, X.Y., and Mi, H.T.: ‘Ear Recognition Based On Moment Invariants’, Pattern Recognition and Artificial Intelligence, 2004, 17, pp. 502-505

11  Muharrem, M., Kayhan; G., and Tarik Veli, M.: ‘Real object recognition using moment invariants’, Academy Proceedings in Engineering Sciences, 2005, 30, pp. 765-775

12  Hamarneh, G.: ‘Active shape models, modeling shape variations and gray level information and an application to image search and classification’, Technical Report R005/1998, Dept. of Signals and Systems, Chalmers Univ. of Technology, Sweden, 1998.

13  Li, Y.Zh., and Wataru, I.: ‘Robust active shape model using adaBoosted histogram classifiers and shape parameter optimization’, IEICE Transactions on Information and Systems, 2006, E89-D, pp. 2117-2123

14      Chalechale A., Mertins A., and Naghdy G.: ‘Edge image description using angular radial partitioning’, IEE Proceedings: Vision, Image and Signal Processing, 2004, 151, pp. 93-101

15      Mukundan, R.: ‘Moment functions In Image Analysis’, World Scientific Publishing Co. Ltd. 1998

16      Ozonat, K.M.; Gray, R.M.: ‘Fast Gauss mixture image classification based on the central limit theorem’, IEEE 6th Workshop on Multimedia Signal Processing, 2004, pp. 446-449


 

 


       

Fig.1 The transformed models under different constraints. An extreme example which shows the distorted shape under the constraint of vector b. The line with diamond dots is the initial model. The lines with triangle and square dots are the distort shapes which are transformed from the initial shape by setting different search parameters. The red circles indicate the distorted parts.

 


 

 

 


Fig.2 An example of the distribution of differences of moment invariants. Each row means the differences under different limitation of shape changing. Each column means different shapes (from left to right they are triangle, rectangle, pentagon and hexagon).The horizontal axis of each graph means the value of difference of moment invariants. The vertical axis means of each graph means the amount of movement of each point of shape. The points marked by ‘*’ illustrate the values that come from those moment invariants to be selected. The points marked ‘o’ illustrate the values that come from those moment invariants to be not selected. It is obvious that most of ‘*’ points locate at the area with high values and all of ‘o’ points locate at the area with low values.


 

文本框: Amount of subsets     

(a) The points of a subset and the corresponding values of boundary moment invariance

 

文本框: Amount of subsets     

(b) The points of a subset and the corresponding values of boundary moment invariance

 

文本框: Amount of subsets    

(c) The points of a subset and the corresponding values of boundary moment invariance

 

文本框: Amount of subsets     

(d) The points of a subset and the corresponding values of boundary moment invariance

 

文本框: Amount of subsets     

(e) The points of a subset and the corresponding values of boundary moment invariance

 

文本框: Amount of subsets    

(f) The points of a subset and the corresponding values of boundary moment invariance

 

Fig.3 The left column is the six sub-sets in a model shape. The green boxes in each sub-figure indicate the selected points. The right column is the boundary moment invariants of corresponding sub-sets which are clustered at some finite ranges.


 

    

(a)                                   (b)

Fig.4 The boundary moment invariants of the shape at each iteration and the fitting result (satisfied)

a the boundary moment invariants of the shape at each iteration

b the fitting result

 


 

 

  

(a)                              (b)                              (c)

Fig.5 The boundary moment invariants of the shape at each iteration and the fitting result (not satisfied). The landmark points in circle are distorted.

a the boundary moment invariants of the shape at each iteration. It is obvious that there is a sharp change at the fifth iteration

b the fitting result

c a distortion shape (the landmark points in circle) is occurred when the boundary moment invariants are changed sharply

 



 

(a)                                            (b)

(c)                                 (d)

Fig.6 Comparison of amending distortion shapes by different strategies

a is the model shape,

b is the distorted shape, and the area formed by point 1, point 2 and point 3 is the distorted area detected by our distortion-sensitive method and the point 2 is the distorted point.

c is the result that only the distorted point 2 is revised.

d is the result that all the points in distorted area are revised.

 


 

    

     

    

    

Fig. 7 The sub-set of experimental images (left) and the landmark points on the image (right).

 

 


 

 

                                        (a)                                                                                            (b)

(c)                                                (d)

                            (e)                                                                      (f)                                                                                (g)

                            (h)                                                                      (i)                                                                                (j)

Fig.8 The fitting results and the minimum difference of the boundary moment invariants of each subset and the clustering value of each subset

a initial shape

b fitting result by standard ASM after 160 iterations

c fitting result by distortion-sensitive ASM after 117 iterations

d the manually labeled shape

e-j the minimum difference of the boundary moment invariants of each subset and the clustering value of each subset


                                          (a)                                                                                           (b)

                       (c)                                                      (d)

                            (e)                                                                      (f)                                                                                (g)

                            (h)                                                                      (i)                                                                                (j)

Fig.9 the fitting results and the minimum difference of the boundary moment invariants of each subset and the clustering value of each subset

a initial shape

b fitting result by standard ASM after 132 iterations

c fitting result by distortion-sensitive ASM after 146 iterations

d the manually labeled shape

e-j the minimum difference of the boundary moment invariants of each subset and the clustering value of each subset


 



© The Institution of Engineering and Technology 2009

doi:10.1049/iet-ipr:2009xxxx

Paper first received 21st June 2007 and in final revised form 15th November 2009.

S.Y. Chen, Q. Guan and S. Liu are with the College of Computer Science, Zhejiang University of Technology, 310023 Hangzhou, China. J. Zhang is with the Dept of Informatik, University of Hamburg, Germany. The work is supported by the National Natural Science Foundation of China and Microsoft Research Asia (NSFC No. 60870002, 60605013), NCET, and ZJDST (2010R10006, 2009C21008, 2010C33095).

E-mail: sy@ieee.org, sychen@theiet.org