A method of automatic shape depiction and information extraction for oceanic edd

ZHENG Ping, CHONG Jin-song, WANG Yu-hang

(1. National Key Laboratory of Science and Technology on Microwave Imaging, Beijing 100190, China;2. Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China;3. University of Chinese Academy of Sciences, Beijing 100049, China)

Abstract： Synthetic aperture radar (SAR) provides a large amount of image data for the observation and research of oceanic eddies. The use of SAR images to automatically depict the shape of eddies and extract the eddy information is of great significance to the study of the oceanic eddies and the application of SAR eddy images. In this paper, a method of automatic shape depiction and information extraction for oceanic eddies in SAR images is proposed, which is for the research of spiral eddies. Firstly, the skeleton image is got by the skeletonization of SAR image. Secondly, the logarithmic spirals detected in the skeleton image are drawn on the SAR image to depict the shape of oceanic eddies. Finally, the eddy information is extracted based on the results of shape depiction. The sentinel-1 SAR eddy images in the Black Sea area were used for the experiment in this paper. The experimental results show that the proposed method can automatically depict the shape of eddies and extract the eddy information. The shape depiction results are consistent with the actual shape of the eddies, and the extracted eddy information is consistent with the reference information extracted by manual operation. As a result, the validity of the method is verified.

Key words： SAR image; ocean eddies; shape depiction; information extraction

0　Introduction

Ocean eddies are a common phenomenon in the sea and widely distributed in the global ocean. It plays an important role in the energy exchange and the material distribution of sea. It has always been a very popular research topic in marine science. Synthetic aperture radar (SAR) with the characteristics of all-weather, all-time, high resolution and lager coverage provides a large amount of image data for oceanic eddy observation and research.

At present, a great deal of research on oceanic eddies has been done using SAR images both at home and abroad. The statistical study of eddies was carried out in Refs.[1-6], and the information of eddy in the statistical process was extracted by manual operation. Min, et al.[7]obtains the logarithmic spirals by using manual tracing and curve fitting to depict the eddy shape, and then extracted the eddy information based on the depiction results. However, in the above mentioned research, the information extraction requires manual operation, which is time-consuming and laborious. With the accumulation of SAR image data, there is an urgent need for the method of automatic extraction of eddy information from SAR images. At present, there is no report about the method of automatic extraction of eddy information in SAR images.

In this paper, a method of automatic shape depiction and information extraction for oceanic eddies in SAR images is proposed. Due to the discontinuity of spiral eddies in SAR images and the interference of other sea surface features in image, firstly, the skeleton image is got by the skeletonization of SAR image. Secondly, based on the skeleton image, the spiral shape characteristics of oceanic eddies are used to automatically depict the shape of eddies. Finally, the eddy information is extracted based on the shape depiction results. The sentinel-1 SAR eddy images in the Black Sea area were used for the experiment in this paper. The experimental results show that the proposed method can automatically depict the shape of eddies and extract the eddy information.

1　Automatic shape depiction and information extraction for oceanic eddies in SAR images

The proposed method of shape depiction and information extraction for eddies is divided into three steps: skeletonization, automatic shape depiction and eddy information extraction.

The schematic diagram of the proposed method is shown in Fig.1.

Fig.1　Schematic diagram of proposed method

1.1　Skeletonization

Skeletonization is a common image processing method, which can greatly reduce the amount of data in the image and keep the shape and position of region in the image unchanged. In the depiction of the eddy shape of the SAR image, only the shape of eddies and their position in image are concerned. For this reason, the skeleton of SAR image is first got.

The steps of skeletonization include image preprocessing, adaptive threshold segmentation, interference area removal and thinning, and so on. This section takes a sentinel-1 SAR eddy image (Fig.2a) for an example to introduce all steps. The results of each step are shown in Figs.2(b)-(f), respectively.

Fig.2　Results of each step of SAR image skeletonization: (a) Sentinel-1 SAR eddy images, November 25 2016, UTC: 15:26; (b) Result of image preprocessing; (c) Result of segmentation; (d) Result of interference region removal; (e) Thinning result; (f) Drawing skeletons in SAR image

1.1.1Image preprocessing

In order to enhance the oceanic eddies in SAR image and filter out a certain speckle noise, a statistical filtering method is used to preprocess the image[8].

The details of method are as follows:

1) Select a pixel (m) in the image, whose gray value isf(m), and extract itsn×nneighborhoodW(n,n);

2) Calculate mean gray value (μ) and standard deviation (σ) forW;

3) Find the homogeneous gray value set (A) with respect toμandσas defined below

A={F∶μ-σ≤f≤μ+σ,∀f∈W}.

(1)

4) Find the minimum value of all the gray levels inA, and replace the pixel (m) by the minimum

f(m)=min(A).

(2)

5) Traverse all pixels in the image, repeat steps 1-4, complete the statistical filtering.

After the statistical filtering, a high-boost filter is used to preserve the edges of eddies[9], and the image contrast is enhanced by grayscale linear stretching technique.

The result of image preprocessing is shown in Fig.2(b).

1.1.2Adaptive threshold segmentation

The segmentation threshold is adaptively determined by the Otsu method[10], and the corresponding region of eddies is got by the threshold segmentation. In order to avoid the influence of image brightness difference on the segmentation result, the image is divided into blocks. Each image block is segmented using separate threshold, and the binary image of oceanic eddy is obtained by combining the segmentation results. Based on the principle that the regions of eddy in the SAR image occupy less area of image[8], the adaptive threshold segmentation method is as follows:

1) Divide SAR image into blocks, image blockFn(n=1,2,3,…,N);

2) Select an image blockFn, initialize parameters, initial regionR0=Fn, segmentation thresholdt0=255, area ratiop0=1, the number of iterationsi=1;

3) Use Otsu method to determine the segmentation thresholdtiofRi-1, then segment to getRi;

4) Calculatepi

(3)

5) Ifpimeets the conditions

pi>pi-1or |pi-pi-1|≤ε,

(4)

go to step 6; elsei=i+1, go to step 3;

6) Taketi-1as threshold to segment image block;

7) Traverse all image blocks, repeat steps 2-6, combine the segmentation results to get a binary image.

The result of adaptive threshold segmentation is shown in Fig.2(c). The white area in the figure corresponds to the oceanic eddies, marked as 1; the black area is the background, marked as 0.

1.1.3Interference region removal

Due to the noise influences, the binary image obtained by segmentation has interference regions. As shown in Fig.2(c), they are small holes or small area regions. The interference regions are removed using the area thresholdTa.

1) Small area region removal: The area of each white region in the binary image is calculated, that is, the number of pixels marked as 1 in the region. When the area of white region is less than the thresholdTa, the region is removed.

2) Small hole filing: The mark of all pixels of binary image is reversed to get a new binary image,

fn=1-f1,

(5)

wheref1is the pixel mark of the original binary image, andfnis the mark of the same pixel after reversed. The small holes are converted to the small area regions of the new binary image. Then the small area regions of the new image are removed by the method of small area region removal. Finally, the pixel marks are reversed again to achieve small holes filing.

The result of interference region removal is shown in Fig.2(d).

1.1.4Thinning

Thinning is the process of converting the image region into a single pixel width curve, and the curve obtained is the skeleton of image. In this paper, the thinning method proposed in Ref.[11] is used to get the skeleton of eddy in SAR image.

There is the interference of burrs on the skeleton. In order to remove the burrs, the points on the skeleton are divided into leaf nodes, intermediate nodes and connected nodes[8]. Then setting the length thresholdlm, if the length from leaf node to connection node is less thanlm, this skeleton segment is removed. For multi-layer burr structure, they are removed by using multiple cycles and gradually increasing the threshold.

The discontinuity of eddy will lead to the fracture of the skeleton. The method based on the proximity and orientation is used to link the skeleton[12].

The result of thinning is shown in Fig.2(e). For ease of observation, the skeletons are drawn on the SAR image, as shown in Fig.2(f).

1.2　Automatic shape depiction

As seen from Fig.2(f), in addition to the eddy, there are other sea features in the skeleton image, so it can not be used to directly depict the shape of eddy. Most of oceanic eddies in the SAR images are spiral, while other sea features generally do not have spiral shape. Therefore, using the spiral shape characteristics of eddy, the spiral lines are detected from the skeleton image to depict the eddy shape. There are many kinds of spiral lines, of which the logarithmic spiral is similar to the spiral eddy in SAR images, and it is often used to depict the shape of eddy[7,13-14]. The logarithmic spiral equation under polar coordinates is

r=asexp(bsθ),

(6)

whereris polar radius;θis polar angle;asandbsare the parameters of logarithmic spiral.

1.2.1Polar coordinate transformation

In order to detect the logarithmic spiral, the polar transformation is performed on the points of the skeleton. When the origin of the polar coordinate system is (x0,y0), polar transformation of any point (x,y) under Cartesian coordinates is

(7)

The origin (x0,y0) is the center of the logarithmic spiral, corresponding to the center of eddies. For the moment, the exact center of eddy is unknown. According to the concept of typhoon eye area[15], the region surrounded by the intersection of spiral stripes in the SAR image is selected as the central area of eddy. Traversing each point in the central area, as the origin of the polar coordinate system, the polar transformation is performed on the skeleton points to obtain multiple sets of transformation results.

In order to improve the efficiency, an intervalstis selected when traversing the central area. As a result, the distance between two selected central points isst. Therefore the rough center of logarithmic spiral is got. And the exact center of logarithmic spiral is obtained by traversingst×stneighborhood of the rough center.

1.2.2Hough transform detection line

Hough transform[16]is a curve detection method in image processing. Using the corresponding relationship between the image domain curves and the parameter domain points, the problem of curve detection in the image domain is transformed into the statistical problem of local peak of accumulator in the parameter domain[17]. Hough transform is based on the local metric to calculate the parameters, thus it has better fault tolerance and robustness for the interruption caused by the interference. Hough transform is used to detect logarithmic spiral to avoid the impact of skeleton fracture.

The logarithm of both sides of Eq.(6) is

lnr=lnas+bsθ.

(8)

The logarithmic spiral equation is converted into a linear equation, and it is easy to detect the straight line using Hough transform. Therefore, the polar diameter of point is taken logarithm first, and the Hough transform is used to detect the straight line in the coordinate system of (θ,lnr), then the straight linear parameter is transformed into the logarithmic spiral parameter to realize the logarithmic spiral detection.

Many sets of results are obtained after polar transformation on the skeleton, and the polar diameter of point in each result is taken logarithm, then the Hough transform is used to detect the straight lines. A set of transformation results only detects the parameters of one straight line, and the parameters of multiple straight lines are detected from multiple sets of transformation results. The numberNlof points on each detected straight line is calculated to select the best detected results. When the distance from the point to the straight line is less than the thresholdd, the point is considered to be on the straight line. The parameters of straight line which has the most points on the line is selected to output, and thus the Hough transform detection is completed.

Because of the quantization of parameter space, Hough transform reduces the accuracy of the detection parameters. In order to improve the accuracy, the straight linear parameters are got by least square fitting. The linear parameters obtained by the least square fitting are calculated by

(9)

wherenis the number of points on the straight line. In order to avoid the influence of some large deviation points on the fitting results, thep-least square method is used to get linear parameters[18].

1.2.3Detected results screening

The detected results are screened by the thresholdTpointof the numberNlof points on the line and the thresholdTbsof the parameterbs. The results will be retained if they meet the threshold requirements.

1) ThresholdTpoint: To avoid the interference from the detection result of short line, the thresholdTpointof the number of points on the line is set.Tpointis adaptively determined by

Tpoint=max(T0,pdNs),

(10)

whereT0is the lower limit of the number of points;Nsis the total number of points on the skeleton;pdis a multiple. WhenNl>Tpoint, the next threshold is judged, otherwise the detection cycle is exited.

2) ThresholdTbs: Other sea features in SAR images may also be detected by Hough transform and satisfy the thresholdTpointrequirement. However, they generally do not have spiral shape, and the corresponding parameterbsis smaller, while the corresponding parameterbsof eddy is larger. In order to exclude the false alarm detection results caused by other sea features, the thresholdTbsis set. Whenbs>Tbs, the detected result is retained, otherwise, the detected result is discarded.

1.2.4Detected skeleton removal

In order to achieve the center positioning of logarithmic spiral, the straight linear detection method proposed in this paper does not have ability to detect multiple straight lines. However, the shape depiction of oceanic eddy may need to detect multiple straight lines. To this end, a loop detection method is designed to detect straight line, and in order to avoid repeated detection, the detected skeleton corresponding to the detected line is removed from the skeleton image. In this way, different lines can be detected in each cycle, so that multiple detected straight lines will be got.

Through the combination of cycle detection and skeleton removal, multiple straight lines are detected, and it is mainly to avoid that the peaks of accumulation in parameter domain caused by other sea features destroy the original peak distribution, which could affect the detection of effective straight lines. Although this method reduces the efficiency, the reliability of detected parameters is improved.

1.2.5Logarithmic spiral drawing

The detected logarithmic spirals are drawn on the SAR image to visually depict the shape of eddy.

The parameter of logarithmic spiral is converted from the linear parameter, and the conversion formula is

bs=bs,as=exp(lnas).

(11)

The center of the logarithmic spiral is the origin of the polar coordinate system corresponding to the detected straight line, and the angle range of the logarithmic spiral is determined by the angle range of the point on the detected straight line.

The result of the logarithmic spiral drawing is shown in Fig.3. It visually depicts the shape of eddy.

Fig.3　Result of logarithmic spiral drawing. Red curve is detected logarithmic spiral, and red point is its center

1.3　Eddy information extraction

Based on the results of the eddy shape depiction, eddy information such as eddy center and eddy radius is extracted.

1.3.1Eddy center extraction

The eddy center is used for eddy positioning. The weighted sum of the centers of the detected logarithmic spirals is used as the eddy center (x0,y0), and the calculation expressions is

(12)

whereNis the number of detected logarithmic spirals;nis the total number of points on all logarithmic spirals;niis the number of points on theith logarithmic spiral; and (xi,yi) is the central coordinates of theith logarithmic spiral.

1.3.2Eddy radius extraction

The eddy radius is often used to characterize the size of the eddy. The length from the eddy center to the outer endpoint of the logarithmic spiral is the eddy radiusR. When there areNlogarithmic spirals,Nradius will be got, and the maximum of them is chosen as the eddy radius, that is

R=max(R1,R2,…,RN).

(13)

Based on the result of the eddy shape depiction in Fig.3, the extracted eddy information is shown in Fig.4.

The extracted eddy center and eddy radius are shown in Table 1.

Table 1　Extracted eddy information

Fig.4　Result of eddy information extraction. Red square point is eddy center position; Red dotted line is eddy radius

2　Experimental results and analysis

In this section, three sentinel-1 SAR eddy images in the Black Sea area were selected for the experiment, as shown in Figs.5(a)-(c), respectively.

In Fig.5(a), there is only one spiral stripe in the eddy, and one logarithmic spiral is required to depict the shape of eddy. In Figs.5(b) and (c), there are multiple spiral stripes in the eddy, so the eddy shape depiction needs multiple logarithmic spirals. The results of the eddy shape depiction are shown in Figs.6(a)-(c), respectively.

Fig.5　Sentinel-1 SAR eddy images for experiment: (a) November 04 2015, UTC: 15:51; (b) December 19 2015, UTC: 15:26; (c) July 16 2016, UTC: 03:33

Fig.6　Results of oceanic eddy shape depiction: (a) One logarithmic spiral is detected to depict the shape of oceanic eddy of Fig.5(a); (b) Three logarithmic spirals are detected to depict the shape of eddy of Fig.5(b); (c) Three logarithmic spirals are detected to depict the shape of eddy of Fig.5(c)

Analyzing Fig.6, it can be seen that: 1) After detecting single or multiple logarithmic spirals, the proposed method realizes the eddy shape depiction in SAR image. 2) The results of shape depiction are consistent with the actual shape of eddy in SAR image. By drawing the detected logarithmic spirals on SAR image, the spiral shape of eddy is directly visualized. 3) The proposed method can eliminate the interference of other features on the eddy shape depiction. 4) The proposed method can avoid the influence of the discontinuity of the oceanic eddy and get a more complete eddy shape. The results of the shape depiction in Fig.6 are used to extract the eddy information, and the results are shown in Fig.7.

Fig.7　Results of information extraction. Red square point is eddy center, and the dotted line is eddy radius

The extracted eddy center and eddy radius are shown in Table 2.

Table 2　Extracted eddy information

In order to verify the accuracy of the extracted information, the eddy information extracted manually by the method in Ref.[7] is used as a reference, and the difference between information extracted by the two methods is calculated. The SAR eddy image in Fig.5(c) is selected as an example, and the extracted reference information and comparison results are shown in Table 3. It can be seen that the eddy information extracted by this method is basically the same as the reference information extracted manually.

Table 3　Comparison of extracted eddy information

Through the above experiment and analysis, the validity of the proposed method is verified.

3　Conclusion

In order to meet the needs of automatic processing of SAR eddy images, a method of automatic shape depiction and information extraction for oceanic eddies in SAR images is proposed in this paper. In this method, the logarithmic spiral detected from the eddy skeleton image is drawn on SAR image to depict the shape of oceanic eddies, and the eddy information is extracted based on the result of shape depiction. The sentinel-1 SAR eddy images in the Black Sea area were selected for the experiment. The experimental results show that the proposed method can automatically depict the shape of oceanic eddies and extract the eddy information. The shape depiction results are consistent with the actual shape of the oceanic eddies, and the extracted information is consistent with the reference information extracted manually. The proposed method still has some shortcomings. For example, the efficiency of the method is not high, and the logarithmic spiral is not suitable for depicting the shape of non-spiral eddy. The next work mainly focuses on how to improve the efficiency of the method, as well as the shape depiction and information extraction of the non-spiral eddies.