Next Article in Journal
Monitoring the Impact of Land Cover Change on Surface Urban Heat Island through Google Earth Engine: Proposal of a Global Methodology, First Applications and Problems
Previous Article in Journal
In-Flight Calibration and Performance of the OSIRIS-REx Visible and IR Spectrometer (OVIRS)
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Convolutional Neural Networks for Automated Characterization of Arctic Ice-Wedge Polygons in Very High Spatial Resolution Aerial Imagery

1
Department of Natural Resources and the Environment, University of Connecticut, Storrs, CT 06269, USA
2
Eversource Energy Center, University of Connecticut, Storrs, CT 06269, USA
3
Department of Geography, University of Connecticut, Storrs, CT 06269, USA
4
Water and Environmental Research Center (WERC), University of Alaska, Fairbanks, AK 99775-5860, USA
*
Author to whom correspondence should be addressed.
Submission received: 25 July 2018 / Revised: 29 August 2018 / Accepted: 14 September 2018 / Published: 18 September 2018

Abstract

:
The microtopography associated with ice-wedge polygons governs many aspects of Arctic ecosystem, permafrost, and hydrologic dynamics from local to regional scales owing to the linkages between microtopography and the flow and storage of water, vegetation succession, and permafrost dynamics. Wide-spread ice-wedge degradation is transforming low-centered polygons into high-centered polygons at an alarming rate. Accurate data on spatial distribution of ice-wedge polygons at a pan-Arctic scale are not yet available, despite the availability of sub-meter-scale remote sensing imagery. This is because the necessary spatial detail quickly produces data volumes that hamper both manual and semi-automated mapping approaches across large geographical extents. Accordingly, transforming big imagery into ‘science-ready’ insightful analytics demands novel image-to-assessment pipelines that are fueled by advanced machine learning techniques and high-performance computational resources. In this exploratory study, we tasked a deep-learning driven object instance segmentation method (i.e., the Mask R-CNN) with delineating and classifying ice-wedge polygons in very high spatial resolution aerial orthoimagery. We conducted a systematic experiment to gauge the performances and interoperability of the Mask R-CNN across spatial resolutions (0.15 m to 1 m) and image scene contents (a total of 134 km2) near Nuiqsut, Northern Alaska. The trained Mask R-CNN reported mean average precisions of 0.70 and 0.60 at thresholds of 0.50 and 0.75, respectively. Manual validations showed that approximately 95% of individual ice-wedge polygons were correctly delineated and classified, with an overall classification accuracy of 79%. Our findings show that the Mask R-CNN is a robust method to automatically identify ice-wedge polygons from fine-resolution optical imagery. Overall, this automated imagery-enabled intense mapping approach can provide a foundational framework that may propel future pan-Arctic studies of permafrost thaw, tundra landscape evolution, and the role of high latitudes in the global climate system.

1. Introduction

Polygonal topography typical of the Circum-Arctic permafrost region is associated with ice wedges, developed by repeated frost (or thermal contraction) cracking and formation of ice veins as a result of filling cracks with meltwater in the spring time; this process occurs over hundreds to thousands of years and leads to formation of large massive-ice bodies [1,2,3,4,5,6,7,8,9,10,11,12,13,14]. Wedge ice is the most common type of massive ground ice in the permafrost region. While the vertical extent of wedge-shaped epigenetic ice wedges does not exceed the depth of frost cracking (usually 2 to 4 m), syngenetic wedges that have formed during continuing sedimentation may have irregular shapes and can be very large. For example, ice wedges in yedoma (ice-rich syngenetic permafrost formed in the late Pleistocene within unglaciated areas of Eurasia and North America) can reach up to 10 m in width and more than 40 m in vertical extent [11,15,16].
Frost cracking and subsequent development of ice wedges create a network of polygons that forms archetypal polygonal patterned tundra occupying a large portion of the Arctic. Leffingwell [17] described two major types of ice-wedge polygons (IWP): polygons with depressed blocks and polygons with elevated blocks. Later, these types of polygons were renamed to low-centered (LCP) and high-centered (HCP) polygons, correspondingly, and since at least the 1950s, many authors have used these terms to distinguish major types of polygons [2,14,18,19,20,21,22,23,24,25,26,27]. In some publications, a third type—flat-centered (FC) polygons—has been added [28,29,30,31]. Drew and Tedrow [32] and Mackay [33] suggested more complicated classifications of IWP based on their morphology; both classifications include six types of polygons. Some authors have described “mixed” polygons; first of all, polygons transitional from LCP to HCP [22,23,34]. In this study, we focus on the two major types—LCP and HCP polygons—widely represented within the Circum-Arctic permafrost region.
Low- and high-centered polygons have very distinctive features that can be relatively easily detected in satellite imagery and aerial photos. LCP polygons are framed by elevated rims that develop above actively growing ice wedges; sometimes the centers of the LCP polygons contain shallow ponds. HCP polygons have elevated (dry) centers and well-developed troughs over ice wedges, where the troughs often are filled with water. LCP polygons are typical of aggrading stages of ice-wedge formation, while HCP polygons usually indicate partly degraded ice wedges [20,35]. LCP polygons are commonly observed within relatively young terrain units (e.g., young drained-lake basins, floodplains) with actively growing ice wedges and are usually are of a larger diameter than HCP polygons, which prevail primarily within older terrain units (e.g., yedoma, old drained-lake basins) [36]. The microtopography associated with the IWPs governs many aspects of permafrost [34,35,37,38], vegetation [22,25,39,40] and hydrologic dynamics [27,41,42], and Arctic ecosystem in general [30,32,35,38] at plot-to-local scales (1–100 m), landscape (100 m–10 km), and regional scales (10–1000 km), mainly due to the role of polygon type on the flow and storage of water [27,41].
Through satellite imagery, aerial photos, and ground observations, large-scale ice-wedge degradation was observed across the Arctic, and in many places, this degradation has resulted in transformation of LCP polygons into HCP polygons in less than a decade [27,29,37,38,43,44,45,46,47,48,49,50,51,52,53,54,55,56]. In most cases, ice-wedge degradation, which is extremely hazardous for both environment and infrastructure, has been triggered by climatic fluctuations [27,38], wildfires [57], human activities [48], or any other factors that lead to increase in the active-layer thickness. Degradation of ice wedges is a quasi-cyclic process, which includes five main stages: Undegraded wedges—Degradation-initial—Degradation-advanced—Stabilization-initial—Stabilization-advanced [34,37,38]. In the continuous permafrost zone, accumulation of organic matter in the troughs developing on top of degrading wedges eventually leads to decrease in the active-layer thickness and formation of the ice-rich soil layer, protecting ice wedges from further degradation. However, under certain conditions ice-wedge degradation may result in complete melting of ice wedges and formation of large thermokarst lakes [34,48]. The probability of such transformation is higher in areas with warmer permafrost, like the Seward Peninsula in Alaska [58]. Understanding of spatiotemporal dynamics behind the evolution of ice-wedge polygonal tundra demands for objective and detailed maps consolidating the ice wedge extent and successional stage [56,59,60].
Vast geographical extent, remoteness, logistical challenges, and high cost retard field-based mapping of IWPs on large scales. Remote sensing (RS) provides transformational opportunities to observe, monitor, and measure the Arctic polygonal landscape at multiple spatial scales and varying temporal windows [59,61,62,63]. The diameter of most IWPs ranges from 5 m to 30 m [9,36,64] and their troughs range from 0 to >1 m depth [34,38,65]; thus, IWPs are difficult to detect in any RS imagery with spatial resolution greater than 4 m [66,67]. High-resolution RS images or Light Detection and Ranging (LiDAR) data are commonly used in IWPs mapping applications. While the imagery supports the general delineation of polygon boundaries, LiDAR-based digital terrain models (DTM) are able further thematically discern the polygon types as HCP, LCP, or flat centered IWPs [67]. In a recent study that coupled ground-based surveys and imagery-based interpretations, Lousada et al. [60] reported that ice-wedge polygonal networks could be accurately delineated and characterized exclusively from very high spatial resolution (VHSR) RS imagery.
Over the years, a considerable number of studies have been conducted to map and characterize IWPs from RS data coming from different sensor platforms, such as unmanned aerial system (UAS), manned-aerial, and satellite. (1) UAS sensor platform: Muster et al. [66] deployed a UAS augmented with a helium-filled dirigible to image IWPs at Samoylov Island in the Lena River Delta in the Russian Arctic at 0.18 m resolution in the visible (VIS) and near-infrared (NIR) ranges. Fraser et al. [68] utilized UAS-based imagery for vegetation classification in the low-Arctic and reported an overall accuracy of 82% for 11 target classes. (2) Manned-aerial sensor platform: Ulrich et al. [69] extracted geomorphometric parameters and topographical properties of polygons from Spitsbergen (Norway) and Mars using 0.2 m four-band (VIS and NIR) aerial imagery from HRSC-AX sensor and 0.3 m imagery from NASA HiRISE mission, respectively. Haltigin et al. [70] collected 0.15 m resolution aerial imagery to investigate the evolution of polygonal terrain networks of Axel Heiberg Island and Devon Island in the Canadian High Arctic. Necsoiu et al. [43] investigated the temporal transformation of IWPs using historical aerial and satellite images and revealed significant ice-wedge degradation in Kobuk Valley National Park, Alaska from 1978 to 2005. Dafflon et al. [71] acquired VHSR images from a kite-based imaging system and coupled with a digital surface model for extracting characteristics of and analyzing the relationship between ice wedges and polygon troughs in Barrow on the Alaskan Arctic. (3) Satellite sensor platform: Lewkowicz and Duguay [72] explored the use of SPOT satellite imagery to detect permafrost features in the Fosheim Peninsula of northern Canadian Ellesmere Island and reported the promising usage of SPOT imagery for discerning large and well-developed HCP and large LCP ponds. Günther et al. [49] segmented polygon morphology based on ground surveyed centroid of thawed IWPs in the Muostakh, Russia using terrain-corrected satellite imagery. Lara et al. [29] conducted polygonal tundra geomorphic classification in the Barrow Peninsula, Alaska with an overall accuracy of 0.75 using hybrid procedures (pixel-based classification, object-based image analysis, and threshold method) in conjunction with images from Landsat-7 (30 m resolution) and Quickbird (0.6 m resolution). Based on high-resolution GeoEye-1 satellite imagery, Perreault et al. [55] assessed the extent of high Arctic wetlands combining normalized difference vegetation index and a threshold-based classification method. Remote sensing-based ice-wedge polygon mapping effort to date follow manual or semi-automated approaches, which quickly constrain the geographical extent of the application.
Due to IWPs’ varying spectral and morphometric characteristics (e.g., irregular shape and trough spacing) [73], visual inspection and manual digitization has so far been the most widely adopted and promising method to delineate polygons from LiDAR or VHSR imagery [13,26,56,65,69,70,74]. Despite the accuracy, the intense-workload of manual digitization constrains the size of a study area. Some studies have proposed semi-automated approaches to leverage human-augmented IWP extraction from imagery. For example, Skurikhin et al. [73,75] proposed a semi-automated hierarchical segmentation method to classify Arctic wetland including ice-wedge polygon ponds in a WorldView-2 satellite image. Ulrich et al. [74] presented a method for calculating wedge-ice volume using Thiessen polygons automatically created in ArcGIS based on polygon-center points. Wainwright et al. [30] semi-automatically delineated IWPs from LiDAR-derived digital elevation models. Chen et al. [64] developed a semi-automated procedure to map IWPs as one of the land cover types in the Arctic coastal plain using high-resolution Pleiades satellite imagery and achieved over 90% producer’s and user’s accuracies. In addition to the efforts spent characterizing IWPs on Earth, Pina et al. [76] developed an algorithm to segment terrain polygons on the surface of Mars using the dynamics of watershed contours with higher spatial resolution images acquired by the Mars Orbiter Camera and Global Surveyor. Pina et al. [77] and Bandeira et al. [78] further used the geometrical and topological information in Pina et al. [76] to analyze the polygonal networks.
To the best of our knowledge, no study has so far developed a fully automated and scalable method, which is capable of accurately detecting and characterizing IWPs (polygon type: LCP and HCP) from VHSR imagery over large geographical areas (e.g., landscape to pan-Arctic) in an operational context. Developing automated methods to map ice-wedge polygonal tundra across the pan-Arctic requires a thinking beyond manual/conventional image interpretation approaches and, instead, exploiting ‘out-of-the-box’ sophisticated image processing techniques. Recently, deep learning (DL) [79] has shown great potential for object instance segmentation in detecting and delineating each distinct object in an image [80] of common objects from everyday pictures. While DL is securing a solid maturity in computer visions applications, its potentials are still at the exploratory stage in the context of remote sensing image classification. Previous studies have used DL with RS imagery in order to extract geographical information. For example, Penatti et al. [81] conducted the first study to test the transferability of CNN approaches from everyday object detection to remote sensing image classification and found that pre-trained CNN models can conceivably repurpose over remote sensing scenes. The underlying rationale is that low-level and mid-level descriptors of both everyday objects and geo-objects inherit quite similar behaviors, even though they finally attain drastically different class labels. Marmanis et al. [82] proposed a Convolution Neural Network (CNN)-based semantic segmentation method for VHSR aerial image with output as boundaries between classes and tested on ISPRS Potsdamsemantic labeling dataset. Ding et al. [83] developed an improved VGG16-Net-backbone Faster RCNN frame for object detections in Google Earth optical remote sensing images. Gevaert et al. [84] applied a fully convolutional network (FCN) to distinguish non-ground and ground objects from VHSR imagery. Accordingly, automated computer vision techniques have reached a stage that, if adopted, could meaningfully inform the natural sciences. Simultaneously, high-resolution (>0.25 m) imagery collected by DigitalGlobe has been available for over five years to the polar science community through a license with the U.S. Federal Government [85].
Designing an automated ice-wedge polygon mapping tool is the first step to allow the Arctic science community to characterize the transforming continuous tundra landscape. The current study builds on our ongoing Arctic permafrost research project that aims to understand the complex and interlinked processes responsible for the evolution of the pan-Arctic ice-wedge polygon tundra landscape. While polar geoscience stands at the precipice of revolution enabled by the petabytes of very high spatial resolution satellite imagery, the distribution of ice-wedge polygons and their status are still largely unknown. Most of the large image repositories are still used for reconnaissance and manual interpretation purposes. This reflects a prevalent methodological vacuum between ‘big imagery’ and Arctic science products. Given the sheer data volumes interlaced with scene complexities, conventional remote sensing image classification methods fail to fill this vacuum. Our overarching goal is to capitalize on ‘image-to-assessment’ pipelines that are catalyzed by deep learning and high-performance computational resources. Recently, He et al. [86] presented that the Mask R-CNN algorithm outperforms state-of-the-art instance segmentation methods such as MNC [87] and FCIS [88] (MNC and FCIS won the COCO 2015 and 2016 segmentation challenges respectively). Although the Mask R-CNN has shown superior performances in everyday object detection in computer vision applications, only limited studies/projects have applied the Mask R-CNN in RS domain. For example, Pešek [89] developed an open-source GIS tool which allows users to train their own network to produce vector masks from RS imagery. In 2018, Remillard launched a project “Images to OpenStreetMap (OSM)” focus on delineating baseball, soccer, tennis, football, and basketball fields from the Microsoft’s Bing imagery and OSM imagery and adding those delineated objects to OSM (https://github.com/jremillard/images-to-osm). In the same year, crowdAI started a “Mapping Challenge” aimed at translating RS imagery into thematic maps (https://github.com/crowdAI/crowdai-mapping-challenge-mask-rcnn). The main goal of this study is to explore the potential of a state-of-the-art DL CNN method (Mask R-CNN) to characterize the tundra ice-wedge polygon landscape. We systematically assessed the performance and transferability of the Mask R-CNN in detection, delineation, and classification of IWPs by repeating the same inference procedure to image scenes of four different spatial resolutions (0.15, 0.25, 0.5, and 1 m) over two study sites near the community of Nuiqsut, Northern Alaska. We conducted a three-step quantitative assessment and detailed visual inspections to assess the correctness and completeness of classification results. To assess the transferability of the Mask R-CNN, we trained the Mask R-CNN only using annotation from VHSR imagery for the first study site (Nuiqsut), with a spatial resolution of 0.15 m, and then used the trained Mask R-CNN to make instance segmentations on resampled VHSR imagery with 0.5 m resolution for Nuiqsut, resampled VHSR imagery with 1 m resolution for Nuiqsut, and VHSR imagery with 0.25 m resolution for the second study site (Crea Creek).

2. Methodology

2.1. Study Area and Image Data

Our study encompasses two areas near the northern Alaska community of Nuiqsut, which is located within the National Petroleum Reserve-Alaska and about 25 km inland from the Beaufort Sea [90] (Figure 1). The two study areas are referred to here as ‘Nuiqsut’ (42 km2) and ‘Crea Creek’ (92 km2). The study areas are generally characterized by low-gradient Arctic tundra and include large features such as lakes and vegetated drained thaw lake basins along with the typical LCPs and HCPs. The VIS-NIR aerial images and LiDAR point clouds were acquired in September 2013 using a fixed-wing aircraft [91,92,93]. The original orthorectified images are in the resolutions of 0.15 m (Nuiqsut) and 0.25 m (Crea Creek) in NAD_1983_StatePlane_Alaska_4_FIPS_5004_Feet_and NAD83_Alaska_Albers coordinate system, respectively. We projected the images into the polar stereographic coordinate system. We synthesized two additional image scenes from the original scenes of Nuiqsut by downsampling following the conventional nearest neighbor resampling technique. The final data set collection contained: (1) the (original) 0.15 m resolution Nuiqsut scene (62,584 rows by 63,103 columns); (2) the (resampled from the original) 0.5 m resolution Nuiqsut scene (19,076 rows by 19,234 columns); (3) the (resampled from the original) 1 m resolution Nuiqsut scene (9538 rows by 9617 columns); and (4) the 0.25 m (original) resolution Crea Creek scene (69,185 rows by 72,539 columns).

2.2. Annotated Data

We selected 340 subsets, each with the dimension of 600 × 600 pixels (i.e., 90 m × 90 m), from the false-color composite of the Nuiqsut image (Figure 2) for manual annotation purposes. We chose the dimension of annotated data as 600 × 600 pixels based on two considerations: (1) to maximize the number of IWPs per subset; and (2) to minimize the error from manual annotation process as a too small or too large subset can be difficult for annotation. To avoid a class balancing problem, we roughly annotated an even number of objects for HCP and LCP polygons (3728 and 3764 polygonal objects). We used the “VGG Image Annotator” web tool (http://www.robots.ox.ac.uk/~vgg/software/via/via.html) to annotate training samples for object instance segmentation and saved the training data in the format of JavaScript Object Notation (.json), which is also the data format being used in some other machine learning training data collection, such as COCO (http://cocodataset.org/#home). Finally, we randomly split the annotated 340 subsets into three sub datasets based on an 80:10:10 split, which entailed: training dataset (272 subsets (Figure 2a)), validation dataset for minimizing overfitting (33 subsets (Figure 2b)), and test dataset for evaluating the performance of the trained DL algorithm (35 subsets (Figure 2c)).

2.3. General Workflow

Our automated imagery-based ice-wedge polygon extraction workflow rests on four key steps (Figure 3): (1) division of VHSR imagery into overlapping patches; (2) object instance segmentation of input patches; (3) mask-to-polygon conversion; (4) eliminate duplicate polygons and compose unique polygons. Two block sizes of 600 × 600 pixels and 360 × 360 pixels were used to partition the VHSR scenes of Nuiqsut and Crea Creek with an overlap of 20% using the Geospatial Data Abstraction Library (GDAL, http://www.gdal.org/). It is worth noting that the two block sizes were selected separately in order to match the scale of the annotated data (i.e., 90 × 90 m). The DL algorithm performed the object instance segmentation with outputs as predicted binary mask with classification information. Finally, we cleaned (e.g., remove duplicates) the output polygons using scikit-image (http://scikit-image.org/) and GDAL.

2.4. Deep Learning Algorithm

We chose the state-of-the-art Mask R-CNN method [86] to implement the object instance segmentation due to its simplicity and effectiveness [94]. The Mask R-CNN method is an extended method for object instance segmentation and is built on the Faster R-CNN (a fast and effective algorithm for object detection [95]) by including a function for predicting masks for distinct objects [86]. Methodologically, the Mask R-CNN is a two-stage algorithm: (1) the Mask R-CNN generates proposals (i.e., candidate object bounding boxes) after scanning the image; (2) the Mask R-CNN predicts the class, bounding box, and binary mask for each region of interest (RoI) [86]. In terms of structure (Figure 4), the Mask R-CNN mainly consists of (1) backbone architecture Residual Learning network (ResNet) [96] for feature extraction; (2) Feature Pyramid Network (FPN) [97] for improving representation of objects at multiple scales; (3) Region Proposal Network (RPN) for generating RoI; (4) RoI Classifier for class prediction of each RoI; (5) Bounding Box Regressor (BBR) for refining RoI; (6) FCN [98] with RoIAlign [86] and bilinear interpolation for predicting pixel-accurate mask. A deeper discussion on the Mask R-CNN algorithm is beyond the scope of this study; thus, we refer readers to He et al. [86] for a detailed discussion on the mathematical basis of the algorithm.

2.5. Accuracy Assessment

We conducted a three-step assessment for the trained Mask R-CNN. First, we assessed the mean average precision (mAP: the mean of average precision values of each class) of the trained Mask R-CNN with the hold-out test dataset. Second, we randomly selected additional 30 subsets (Figure 5) for each study site (including non-IWP samples and excluding previous selected 340 subsets for Nuiqsut) and manually validate the accuracies of detection, delineation, and classification. We evaluated the accuracies of detection, delineation, and classification based on the following criteria: a positive detection indicates a IWP is correctly detected by the Mask R-CNN, and a negative detection indicates a IWP is not correctly detected by the Mask R-CNN; likewise, a positive delineation indicates the Mask R-CNN successfully inferences outline of an IWP based on interpreter’s judgement; a positive classification indicates the Mask R-CNN inferences the correct type of IWP of a detected IWP. Our manual validation included the following steps: in Step (1) we created 30 random square polygons (100 × 100 m) for the each study area with 12 attributes in shapefile database (true-positive, false-positive, true-negative, and false-negative for each of detection, delineation, and classification respectively) to assess the commission and omission errors; In Step (2) we manually counted all IWPs within or crossing the boundaries of the validation square polygons; and in Step (3) we filled the number for the each attribute based on the manual counting. Third, we corroborated qualitative assessments with detailed visual inspections by coupling imagery and LiDAR DTMs.

2.6. Implementation

We implemented the Mask R-CNN method using an open-source package built on Keras and Tensorflow developed by the team of Mask-RCNN on Github [99]. The codes are available on Github (https://github.com/matterport/Mask_RCNN). We conducted experiments on a customized server equipped with an Intel i5 CPU, 16 GB RAM, a GeForce GTX 970 graphic card, and a GeForce GTX 1080ti graphic card. In the training process, a graphics processing unit (GPU) (GeForce GTX 1080ti graphic card) was used to train the adopted ResNet-101 (101 layers) backbone Mask R-CNN in the package with a mini-batch size of 2 images, 272 steps per epoch, learning rate of 0.001, learning momentum of 0.9, and weight decay of 0.0001. The accessibility of this open-source package was the main reason we chose the ResNet-101 rather than ResNet-152 backbone Mask R-NN. We modified the loading dataset function for our customized training data and left other parameters in default settings. For the full description of parameter settings of the used submodels of the Mask R-CNN (e.g., ResNet, RPN, BBR etc.), we refer readers to the model part of the Mask R-CNN repository on Github (https://github.com/matterport/Mask_RCNN). Instead of building the Mask R-CNN from scratch, we trained our model using the pre-trained weights of the Mask R-CNN for COCO because COCO has a large amount of training data for the Mask R-CNN to learn common and discriminative features (known as transfer learning). To minimize overfitting, we used the validation dataset to decide the most generalized Mask-RCNN. Additionally, random horizontal flips augmentation was used to introduce variety in the training data. In the inference process, we set a detection confidence threshold as 70% (i.e., detections with confidence less than 70% were ignored). Two GPUs were used to accelerate the process of inferencing input patches with 20% overlapping by using a data processing queue. As a result, the aerial imagery (63,103 × 62,584 pixels) of Nuiqsut with a spatial resolution of 0.15 m, a total of 8134 patches (600 × 600 pixels), was able to infer approximately to within 21 min (~6.3 fps).

3. Results

3.1. Model Optimization and Accuracy

We optimized the Mask R-CNN model during the training process and evaluated the accuracy of the optimized model prior to its application in the case studies. We adopted an early stopping strategy to minimize overfitting with the hold-out validation dataset. We trained the Mask R-CNN with a predefined 100 epochs so that we could record the full learning curve of the model. As seen in Figure 6a,c–f, the validation loss values reached their lowest and then rebounded, but the training loss values continued descending, which indicates that after the 8th epoch, the model tended to “memorize” the training data, rather than “learn” to generalize the features of IWPs. Similarly, Figure 6b presents that the Mask R-CNN bounding box refinement stopped improving after the 8th epoch. Overall, around the 8th epoch (cyan vertical lines in Figure 6) the validation loss value reached its lowest. This elucidates that the Mask-RCNN tended to be overfitting after the 8th epoch, although all training losses decayed continuously (see Figure 6). Therefore, we selected a Mask R-CNN trained for 8 epochs. The convergence at the 8th epoch can be explained by two main factors. First, we trained our model using the pre-trained weights of the Mask R-CNN for COCO instead of training a Mask R-CNN from randomized weight. Most learning processes for the weights of low-level features were skipped. Therefore, our training goal was focused on adjusting the Mask R-CNN to recognize IWPs. Second, the relatively small size of our training data directly resulted in this early convergence because of the depth of the used network. Finally, we evaluated the Mask R-CNN trained for 8 epochs with the hold-out test dataset (35 subsets including 799 IWPs (i.e., 385 HCPs and 414 LCPs)) (Figure 3). It reported mean average precisions (mAPs) of 0.695 and 0.601 when intersection-over-union (IoU: the ratio between the intersection and the union of the inferred outline and the ground truth outline of an object) thresholds are greater than 0.5 and 0.75.

3.2. Case Studies

We conducted four case studies to systematically assess the performance and transferability of the Mask R-CNN by repeating the same inference procedure of the Mask R-CNN over the four image scenes: (1) VHSR imagery with 0.15 m resolution for Nuiqsut; (2) resampled VHSR imagery with 0.5 m resolution for Nuiqsut; (3) resampled VHSR imagery with 1 m resolution for Nuiqsut; and (4) VHSR imagery with 0.25 m resolution for the Crea Creek. The Mask R-CNN was only trained on the 0.15 m resolution imagery to support each of the four case studies.

3.2.1. Case Study 1: 0.15 m Resolution Image Scene

The total numbers of delineated LCPs and HCPs from the result of 0.15 m image for Nuiqsut are 32,299 and 11,449. Figure 7 presents the enlarged examples of delineation and classification, and geographical distribution of automatically mapped IWPs from the 0.15 m resolution VHSR image of Nuiqsut. Table 1 provides the accuracies of detection, delineation, and classification with precision, recall and overall accuracy (OA). The OA of the detection is 79% and the precision and recall are 0.98 and 0.8, respectively. This indicates that approximately 80% of IWPs can be automatically detected from the test subset. Within the detected bounding boxes, the OAs of the delineation and classification are around 97%. Based on the observation of the enlarged examples, IWPs with clear rims, troughs, or other linear boundaries (Figure 7d,f,h,l,n) are easier to be detected than incomplete or disjoint IWPs (Figure 7b,j,p). The Mask R-CNN can delineate the boundaries of most IWPs even though they exhibit different shape and contextual characteristics.

3.2.2. Case Study 2: 0.5 m Resolution Image Scene

The total numbers of delineated LCPs and HCPs from the result of 0.5 m image for Nuiqsut are 25,037 and 7757. The OA of detection has decreased from 79% to 54% after downsampling the Nuiqsut image from 0.15 m to 0.5 m (Table 2). This is likely to mainly be because the Mask R-CNN was trained based on the 0.15 m resolution samples; thus, it fails to discern some of the key low-level features, which are imperative to defining IWPs at 0.5 m resolution. However, the precision of detection improved by 0.01 compared to the case study 1. Similar to case study 1, the Mask R-CNN can make accurate delineation and classification regardless of resampling issue. In the resampled imagery, linear features between incomplete or disjoint IWPs prone to disappear (Figure 8a,g,i). In fact, fewer IWPs are detected in the 0.5 m resolution scenes with incomplete or disjoint IWPs (Figure 8b,h,j). The effect of resampling does not drastically affect the performance of the Mask R-CNN in the scenes with clearly visible troughs or rims (Figure 8d,f,l,n).

3.2.3. Case Study 3: 1 m Resolution Image Scene

The total numbers of delineated LCPs and HCPs from the result of 0.5 m image for Nuiqsut are 10,763 and 1318. Case 3 is based on the resampled VHSR imagery in a 1 m resolution for Nuiqsut. The increase of pixel size from 0.15 m to 1 m results in substantial disappearance of IWPs’ edge information (Figure 9a,g,i). Therefore, compared the case studies 1 and 2, fewer IWPs are detected using the same Mask R-CNN (Table 3). The OA and recall of detection are 0.15, which is more than 5 times lower than the case study 1 (0.15 m resolution). For the LCPs with wide-span rims and troughs, the approach is still able to detect, accurately delineate, and classify IWPs when the widths of rims and troughs are greater than 1 m (Figure 9d,f,l). The OAs of the delineation and classification are 88% and 100%, respectively. In addition, the number of detected HCPs decreases dramatically as the pixel size of the imagery increases, even though the troughs between HCPs are still visible to the human eye at 1 m resolution (Figure 7h,p, Figure 8h,p, and Figure 9h,p).

3.2.4. Case Study 4: 0.25 m Resolution Image Scene

The total numbers of delineated LCPs and HCPs from the result of 0.25 m image for Crea Creek are 77,874 and 45,183. The OAs of detection, delineation, and classification are 72%, 97%, and 95%, respectively (Table 4). Based on our manual validation, around 72% of total IWPs can be detected with an approximate precision of 0.98. Very similar to the previous case studies, the OAs of delineation and classification of IWPs are high, even in this new image scene with slightly coarser resolution, particularly for IWPs with clear edges (Figure 10b,f). The Mask R-CNN approach is missing incomplete or disjoint IWPs (Figure 10h,l,n), which is reasonable because those IWPs are not distinct objects. Disjoint IWPs might be detected and classified more effectively using scene classification or semantic classification.

3.3. Transferability

Case studies 1–3 depict the degree of transferability of the Mask R-CNN with respect to the spatial resolution and scene content of the input imagery (Figure 11 and Table 5). We trained the Mask R-CNN using annotation from VHSR imagery for Nuiqsut with a spatial resolution of 0.15 m. The OA of the Mask R-CNN decreases as the pixel size of the imagery increases (Figure 12). The total numbers of delineated LCP and HCPs decrease with increasing imagery resolution (Table 5). HCP and LCP had various degrees of reaction to the increasing pixel size. The numbers of missed LCP and HCPs with 0.15–0.5 m resampling are much lower than with 0.5–1 m resampling. Table 5 also shows that HCPs are more sensitive to the resampling than LCPs. For example, 83% of delineated HCP in the 0.5 m imagery was missed in the 1 m imagery, compared to 57% for LCPs. This is mainly because LCPs have more distinguishable features than HCPs, such as a wet center and a dry rim. Case study 4 demonstrates the transferability of the Mask R-CNN in terms of microtopography. The two study areas (Nuiqsut and Crea Creek) are geographically close to each other, but they still manifest fine-scale variations in the microtopography. The trained Mask R-CNN based on the VHSR imagery for Nuiqsut can still achieve 72% OA in a new area and with a different spatial resolution.

3.4. Ice-Wedge Polygons’ Convolutional Feature Maps

The advent of CNN has led to breakthroughs in image processing [79]. Similarly to other DL applications in computer vision, the backbone architecture of the Mask R-CNN relies on convolutional feature map (CFM) from CNN. A CFM is the output layer using a given filter (also known as kernel) to the previous layer. During the process of computing CFMs, sliding-window operations translate current window-size pixel information to a stack of the same number of CFMs by computing and generating the dot product (Figure 13 and Figure 14) with an optimized number of selected filters (such as horizontal/vertical filters, sum/average pooling etc.). The ResNet-101 of the used Mask R-CNN has five stages (a stage is used to name a layer in the ResNet depending on its position in the network): c1, c2, c3, c4, and c5. Figure 13 depicts four CFMs from each of the last output layers of the Mask R-CNN stage from 1 to 5 for HCPs. Each stage in the ResNet consists of a certain number of residual blocks, and each residual block has three layers. For example, Figure 14 (4th row) presents four feature maps from the last layer of the w block (i.e., 23th) in the 4th stage. A 101-layer ResNet includes 23 residual blocks in its 4th stage. Through the proper use of CFMs, the Mask R-CNN can detect the locations of HCPs (Figure 13 (4th row)) and LCPs (Figure 14 (4th row)) based on the high-level features, which then allows delineation and classification of HCPs and LCPs using other representative features from CFMs.

3.5. Visual Inspection

We qualitatively compared enlarged views for automatically detected IWPs to LiDAR DTM (1 m spatial resolution) of polygons in the Crea Creek area (Figure 15, Figure 16 and Figure 17). The LiDAR DTM study included a Canny edge raster analysis, which is a widely used edge detector, and terrain profiles in the XX’ and YY’ directions. The LiDAR was compared to a false color composite VHSR image for HCP, LCP, and incomplete/disjoint IWP (Figure 15a, Figure 16a, and Figure 17a). The troughs of HCPs are indistinguishable in the 1m VHRS image because of the surrounding mixed pixels (Figure 15a). HCP becomes more visible with the help of LiDAR DTM and derived Canny edge from LiDAR (Figure 15b,c). We highlighted the breakpoints where the automated delineation spotted the boundaries in the microtopography to depict a successful agreement between automated delineation and Canny edge information from LiDAR (Figure 15c–e). The exact boundary for a LCP is almost invisible in optical imagery at 1 m resolution (Figure 16a). LiDAR DTM and derived Canny edge from LiDAR can only detect the “inner circle” edge in a LCP; therefore, the slight disagreement with the automated delineation (Figure 16b,c). In contrast, the automatic method captures the actual boundary (Figure 16d,e). Incomplete/disjoint IWPs are difficult to detect because they usually do not show distinct edges. The Canny edge raster for incomplete/disjoint IWP presents highly disorganized edges across the scene (Figure 17). However, the Mask-RCC was able to detect small proportions of incomplete/disjoint IWPs (Figure 17a,d,e), despite the image lacking distinct and organized boundaries.

4. Discussion

Deep learning leverages computational models to learn representations of data with multi-level abstractions. Choosing the right computational model and data is important in deep learning applications. In this exploratory study, we selected the Mask R-CNN, which is an object instance segmentation method, to conduct automated characterization of ice-wedge polygons. The simplicity, effectiveness, and the method’s proven success in processing everyday images have previously been reported by He et al. [86] and Liu et al. [94]. To the best of our knowledge, this is the first study that used the Mask R-CNN method (or any other object instance segmentation methods such as PANet [94] and MaskLab [100]) to detect, delineate, and classify IWPs from VHSR remote sensing imagery. However, future efforts should include a systematic experiment of Mask R-CNN performance to other CNN-driven semantic segmentation methods in IWP mapping. Also, we need to compare how well CNN methods perform compared to well-established knowledge-driven paradigms such as object-based image analysis (OBIA [101]) and other conventional classification methods (e.g., pixel-based). It would also be valuable to conduct a detailed analysis on the CNN model optimization procedures to improve the processing time by examining a variety of combinations of hyperparameters. Optimization is considered to be a critical part of choosing the right computational model, because there are many parameters (e.g., regularization and training schedule) that control the performance of the overall model. In this study, besides all other default regularization settings, we only chose early stopping strategy to minimize overfitting. Without proper implementation, the model could either not function or only function with the training data. The quality of model outputs pivot to the quality of annotated data. Production of accurate and stable annotated data sets is time-, labor-, and cost-intensive, with manual mapping representing one extreme. Therefore, it is advantageous to conduct a comprehensive study on the minimum requirements and quality of the training datasets, which intend to use in the Mask R-CNN training purposes. Here, we only used a training dataset in 0.15 m resolution, while the automated mappings in the coarser resolutions may have benefitted from a training dataset in a comparable resolution.
Mask R-CNN was a successful method in characterizing the Arctic ice-wedge polygonal landscape from VHSR images (Figure 12). The algorithm successfully utilized the multi-level feature representations learned from the training data for detection, delineation, and classification of targets of interest (Figure 13 and Figure 14). However, there is a negative correlation between the overall accuracies of detection and (the coarser) spatial resolution of input imagery. This is likely to be due to the resolution dependency of the training dataset (0.15 m) in the Mask R-CNN approach. This means that the consistency of spatial resolution of both training data and predicting data could significantly impact the Mask R-CNN’s performance. Sub-meter resolution commercial satellite imagery (<0.25 m) is available across the pan-Artic domain. Thus, a Mask R-CNN derived product, with a relaxation to spatial resolution, could potentially be of high value for science applications that can benefit from a pan-Arctic ice-wedge polygon map.
The case studies with their OAs of delineation and classification elucidate the robustness of the model to the spatial resolution of the training data. We envision two possible ways to improve the detection ability of the Mask R-CNN: (1) enriching training data from the aspects of size and source such as training data from multiple spatial resolutions (e.g., UAS, aerial, and satellite imagery, because only relatively small amount of annotated data were prepared for the Mask R-CNN due to the costly annotation process; or (2) lower the confidence threshold for object detection. In this study, we deliberately selected the two extremes of the ice-wedge polygon types—low-centered and high-centered polygons—to investigate the preliminary insights related to the feasibility and appropriateness of the Mask R-CNN. We expect to progressively enhance the thematic depth of the model to discern other polygon classes, such as flat-centered, transitional, and mixed polygons, and conical mounds (i.e., thermokarst mounds or baydzherakhs that form as a result of deep melting of large ice wedges, predominantly in yedoma regions). It is not surprising that all the OAs of the IWP classification are above 95% (Figure 12). Increasing thematic complexity by considering additional types of IWPs (such as types A–F [6]) could potentially lower the OAs of classification.
So far, human-augmented ice-wedge polygon mapping has been the most widely used method in Arctic science applications. Mapping ice-wedge polygons can be challenging. In many instances, flat- and low-centered polygons exhibit ill-developed polygon boundaries. Traditional methods coupled with shallow classifiers (such as nearest-neighbor, support vector machine, and random forest) are incompetent in coping with subtle scene variations, especially in the context of regional-scale mapping applications. Edge detectors, such as Canny, can decode some of the hidden discontinuities in the LiDAR DTM. For example, Canny edge detector can effectively outline the boundary of a HCP polygon (Figure 15c). Based on our observations, the Mask-RCNN can still successfully detect a small proportion of incomplete/disjoint IWP with an inconspicuous edge, which is believed to be the contribution of the feature maps from CNN (Figure 13 and Figure 14). Scene classification might be a solution to detect incomplete/disjoint IWP as a complement to the Mask R-CNN. Here, we systematically tasked the Mask R-CNN algorithm with the original and spatially synthesized imagery as a stress test to observe the behavior against lost spatial detail. Future efforts could expand the stress test by coupling spatial artifacts with spectral, radiometric, and structural artifacts. Such experiments could unravel the susceptibility of the automated algorithm to scene content variations, which are common in remote sensing imagery due to multitude drivers, such as sensor characteristics (e.g., spatial resolution, spectral widths, bit depth), acquisition parameters (e.g., low-angle imaging), seasonal variations (e.g., early Summer versus late Summer), and pre-processing steps (e.g., pansharpening and data compression).
Human interpreters are highly capable of seeking high-level semantics while suppressing unrelated natural variations and process-induced artifacts in an image scene. We ideally expect the automated algorithm to respond in the same manner. Otherwise, the scene dependency could greatly impede the scalability of the mapping task over a large domain with multiple sensors and varying imaging conditions. When the trough network is pronounced and well connected, ice-wedge polygons exhibit bona fide [102] object characteristics in which the object boundaries are crisp and can be demarcated clearly and unambiguously from their respective environment. In some instances, ice-wedge polygons lack a distinct physical border and blend with the landscape. This forces the model to realize an indeterminate (fuzzy) border zone (or boundary-like region), rather than a crisp discontinuity delimiting the object of interest. High-centered polygons circumscribe a spectrally/texturally homogenous entity resembling the real-world representation (Figure 15). In contrast, low-centered polygons sometimes exhibit a repulsion to fidelity. Inundated polygon centers and the dissipating wetness along their boundaries are the readily available cues that support a positive detection of an IWP and a thematic recognition as a low-centered IWP (Figure 16). However, its exact physical boundary is largely undefined in the imagery. The detection would be far more challenging in the absence of water at the polygon center. Vogt et al. [103] coupled spatio-structural properties with functional dispositions and temporal developments when deciding the crispness of the object boundaries. Tundra vegetation is closely linked to moisture status [104], which in turn is closely linked to microtopography [105]—both which can vary dramatically within just a meter. An overly coarse image, such as the 1 m VHRS in the LCP scenario (Figure 16c), blurs the contrasts and spectrally homogenizes the center-rim and trough polygon features, which are actually functionally heterogeneous entities in the landscape. Albrecht et al. [106] experimentally showed the variability in manual object delineations of scale and generalization and emphasized the necessity for degrees of freedom related to fuzziness in the object boundaries. This conceptually aligns with Vogt et al. [103], who argued that indeterminacy of object boundaries comes in various degrees of abruptness and, thus, discontinuity is a matter of scale and granularity. Accordingly, a slight geometrical disagreement between automatically detected polygon boundaries from the imagery and DTM-derived polygon edges would probably even persist in a manual delineation.
Very high spatial resolution remote sensing imagery is radically transforming our ability to understand the most remote, logistically challenging, and ecologically fragile landscapes in Polar Regions, at greater details than ever before [107]. These image data products and their derivatives (e.g., stereo DEMs) result in unprecedented opportunities to map, model, and forecast biological, geological, and hydrological functioning in Arctic and Antarctic in an intensive fashion at extensive spatial scales. The general notion of ‘topography is stationary with respect to climate change’ does not hold true for the Arctic, because microtopographic changes in polygonal tundra due to increasing permafrost temperatures are now visible at sub-decadal scale [41]. Thawing of ice-rich permafrost can be visible in RS imagery because of subsequent ground subsidence and development of troughs in ice-wedge polygon tundra. The entire Arctic has been imaged by commercial satellites in sub-meter resolution four times, on average, in the last six years, some places every month, and the collection is set to continue for at least another three years [85]. Arctic researchers now have access to approximately 2 petabytes of VHSR commercial satellite image repositories, such as Polar Geospatial Center (PGC) at University of Minnesota. Moreover, we now have an opportunity to observe permafrost thaw and disturbance across the Arctic region in a near real-time fashion due to recent and increasing number of launches of SmallSats.
However, scientific opportunities of ‘big imagery’ have so far been poorly explored. Shifting from megabytes to petabytes of imagery demands novel computational approaches to interpretation coupled with efficient use of high-performance parallel and distributed computing resources [108] and reliance on an ecosystem of integrated digital resources and services [109], such as National Extreme Science and Engineering Discovery Environment (XSEDE, https://www.xsede.org/). These cyberinfrastructure platforms can support data discovery and access. Parallel to the revolution of the spatial resolution of commercial satellite imagery, we are experiencing a radical valence towards computer vision approaches that successfully link low-level motifs with high-level semantics. Deep learning approaches excel at identifying higher-order texture features and contextual characteristics critical for uncovering complex data representations in satellite imagery. These methods are promising compared to standard remote sensing classification approaches (e.g., object-based image analysis) at the expense of greater computational complexity. For example, Witharana and Lynch [107] systematically showed the conceptual and implementation caveats of the OBIA framework in autonomous censusing of Antarctic wildlife based on sub-meter satellite imagery using continental scale, repeatable mapping applications.
The efficient use of GPUs is essential for the success of the CNN [79]. Conducting pan-Arctic scale characterization ice-wedge polygons from commercial satellite imagery demands a computationally scalable and efficient solution. Here, the performance of I/O and the Mask R-CNN prediction must be optimized and the image data streamed directly from the data repositories to computing clusters in order to minimize the I/O bound issue. Strategies such as the Message Passing Interface (MPI, a distributed memory parallel model) may allow the implementation of the Mask R-CNN predictions on a large number of available GPUs (Figure 18). A master node would control data flow, such as images distribution and result collection using job queue. Each GPU-node would then receive an assigned image and execute prediction process across the whole image and return the final result in the master node.

5. Conclusions

Imagery-based IWP mapping is still largely constrained to time- and labor-intensive human-augmented workflows. The rapid influx of sub-meter resolution commercial satellite imagery to the polar science community demands high-performance image analysis workflows to meet the ever-increasing demand for imagery-enabled science products. We applied the Mask R-CNN to automatically detect and delineate ice-wedge polygons and identify their type. Four case studies from two study sites were used to assess the performance and transferability of the Mask R-CNN for the mission of characterizing the tundra ice-wedge polygon landscape. Our results report that: (1) the Mask R-CNN can detect up to 79% of IWPs in study sites with a VHSR imagery pixel resolution of 0.15 m and around 72% of IWPs with the imagery with a pixel resolution of 0.25 m; (2) besides promising performance in detection, the Mask R-CNN can delineate and classify detected IWPs accurately; (3) the pressure test of the Mask R-CNN on resampled imagery shows the flexibility and potential in automatically mapping IWPs in coarser RS images. Findings of this study provide an extensible framework for imagery-enabled intense mapping of ice-wedge polygons at extensive spatial and temporal scales.
While the Mask R-CNN presents promising ability in automatically characterizing IWPs, further studies are necessary to fully understand the use of deep learning-driven object instance segmentation in characterizing IWPs. Our future work will focus on four main directions: (1) a comprehensive comparison study on how well the Mask R-CNN performs compared to other methods (e.g., OBIA, PANet, and MaskLab etc.); (2) a full analysis on model optimization procedures by examining a variety of combinations of hyperparameters; (3) a detailed analysis on the minimum requirements and quality of the training datasets, which intend to use in the Mask R-CNN training purposes; (4) a stress analysis by coupling spatial artifacts with spectral, radiometric, and structural artifacts.

Author Contributions

W.Z. designed and conceived the study, collected/preprocessed the data and performed the experiments, and contributed to the manuscript writing and revision. C.W. lead the overall project, contributed to the study designing and conceiving, the data collection, the manuscript writing and revision. A.K.L and M.K. advised in Arctic science during the study design and contributed to the manuscript writing and revision.

Funding

This work was supported by the U.S. National Science Foundation’s Office of Polar Programs [grant No. 1720875].

Acknowledgments

The imagery was supported by Alaska EPSCoR U.S. National Science Foundation’s award #OIA-1208927 and the State of Alaska. We also thank the Department of Geography at University of Connecticut for providing GPU devices and the team of Mask-RCNN for sharing codes on Github (https://github.com/matterport/Mask_RCNN).

Conflicts of Interest

The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

References

  1. Leffingwell, E.D.K. Ground-ice wedges: The dominant form of ground-ice on the north coast of Alaska. J. Geol. 1915, 23, 635–654. [Google Scholar] [CrossRef]
  2. Black, R.F. Permafrost: A review. Geol. Soc. Am. Bull. 1954, 65, 839–856. [Google Scholar] [CrossRef]
  3. Lachenbruch, A.H. Mechanics of Thermal Contraction Cracks and Ice-Wedge Polygons in Permafrost. Spec. Pap. Geol. Soc. Am. 1962, 70. [Google Scholar] [CrossRef]
  4. Dostovalov, B.N.; Popov, A.I. Polygonal systems of ice-wedges and conditions of their development. In Proceedings of the Permafrost International Conference, Lafayette, IN, USA, 11–15 November 1963. [Google Scholar]
  5. Mackay, J.R. The world of underground ice. Ann. Assoc. Am. Geogr. 1972, 62, 1–22. [Google Scholar] [CrossRef]
  6. Vtyurin, B.I. Podzemnye l’dy SSSR [Ground Ice of the USSR]; Nauka: Moscow, Russia, 1975; p. 214. (In Russian) [Google Scholar]
  7. Black, R.F. Periglacial features indicative of permafrost: Ice and soil wedges. Quat. Res. 1976, 6, 3–26. [Google Scholar] [CrossRef]
  8. Romanovskii, N.N. Formirovanie poligonal’no-zhil’nykh struktur [Formation of Polygonal-Wedge Structures]; Nauka: Novosibirsk, Russia, 1977; p. 215. (In Russian) [Google Scholar]
  9. Black, R.F. Ice-wedge polygons of northern Alaska. In Glacial Geomorphology; Coates, D.R., Ed.; Springer: Dordrecht, The Netherlands, 1982; pp. 247–275. ISBN 78-94-011-6493-1. [Google Scholar]
  10. French, H.; Shur, Y. The principles of cryostratigraphy. Earth Sci. Rev. 2010, 110, 190–206. [Google Scholar] [CrossRef]
  11. Murton, J.B. Ground ice and cryostratigraphy. In Treatise on Geomorphology; Schoder, J.F., Giardino, R., Harboor, J., Eds.; Academic Press: San Diego, CA, USA, 2013; Volume 8, pp. 173–201. ISBN 9780123983503. [Google Scholar]
  12. Oliva, M.; Vieira, G.; Pina, P.; Pereira, P.; Neves, M.; Freitas, M.C. Sedimentological characteristics of ice-wedge polygon terrain in Adventdalen (Svalbard)-environmental and climatic implications for the late Holocene. Solid Earth 2014, 5, 901–914. [Google Scholar] [CrossRef]
  13. Templeton, M. Quantifying Ice Wedge Volumes in the Canadian High Arctic. Master’s Thesis, McGill University, Montreal, QC, Canada, April 2017. [Google Scholar]
  14. French, H.M. The Periglacial Environment, 4th ed.; John Wiley and Sons Ltd.: Chichester, UK, 2018; p. 515. ISBN 9781119132783. [Google Scholar]
  15. Schirrmeister, L.; Froese, D.; Tumskoy, V.; Grosse, G.; Wetterich, S. Yedoma: Late Pleistocene ice-rich syngenetic permafrost of Beringia. In Encyclopedia of Quaternary Science, 2nd ed.; Elias, S., Mock, C., Murton, J., Eds.; Elsevier: Amsterdam, The Netherland, 2013; pp. 542–552. ISBN 978-0-444-53643-3. [Google Scholar]
  16. Kanevskiy, M.; Shur, Y.; Strauss, J.; Jorgenson, T.; Fortier, D.; Stephani, E.; Vasiliev, A. Patterns and rates of riverbank erosion involving ice-rich permafrost (yedoma) in northern Alaska. Geomorphology 2016, 253, 370–384. [Google Scholar] [CrossRef] [Green Version]
  17. Leffingwell, E.d.K. The Canning River region, Northern Alaska. U.S. Geological Survey Professional Paper 109. Available online: https://pubs.usgs.gov/pp/0109/report.pdf (accessed on 5 March 2018). [CrossRef]
  18. Britton, M.E. Vegetation of the Arctic tundra. In Arctic Biology: 18th Biology Colloquium; Hansen, H.P., Ed.; Oregon State University Press: Corvallis, OR, USA, 1957; pp. 26–61. [Google Scholar]
  19. Hussey, K.M.; Michelson, R.W. Tundra relief features near Point Barrow, Alaska. Arctic 1966, 19, 162–184. [Google Scholar] [CrossRef]
  20. Péwé, T.L. Quaternary Geology of Alaska; US Government Printing Office: Washington, DC, USA, 1975; Volume 835.
  21. Washburn, A.L. Geocryology: A Survey of Periglacial Processes and Environments; Blackburn Press: London, UK, 1979. [Google Scholar]
  22. Billings, W.D.; Peterson, K.M. Vegetational change and ice-wedge polygons through the thaw-lake cycle in Arctic Alaska. Arct. Alp. Res. 1980, 12, 413–432. [Google Scholar] [CrossRef]
  23. Everett, K.R. Landforms. In Geobotanical Atlas of the Prudhoe Bay Region, Alaska; CRREL Report 80-14; Walker, D.A., Everett, K.R., Webber, P.J., Brown, J., Eds.; U.S. Army Corps of Engineers, Cold Regions Research and Engineering Laboratory: Hanover, NH, USA, 1980; pp. 14–19. [Google Scholar]
  24. Minke, M.; Donner, N.; Karpov, N.; de Klerk, P.; Joosten, H. Distribution, Diversity, Development and Dynamics of Polygon Mires: Examples from Northeast Yakutia (Siberia). Available online: http://pimdeklerk-palynology.eu/polygon_mires_PI__Minke_et_al_2007_.pdf (accessed on 13 March 2018).
  25. Minke, M.; Donner, N.; Karpov, N.; de Klerk, P.; Joosten, H. Patterns in vegetation composition, surface height and thaw depth in polygon mires in the Yakutian Arctic (NE Siberia): A microtopographical characterisation of the active layer. Permafr. Periglac. Process. 2009, 20, 357–368. [Google Scholar] [CrossRef]
  26. Gangodagamage, C.; Rowland, J.C.; Hubbard, S.S.; Brumby, S.P.; Liljedahl, A.K.; Wainwright, H.; Wilson, C.J.; Altmann, G.L.; Dafflon, B.; Peterson, J.; et al. Extrapolating active layer thickness measurements across Arctic polygonal terrain using LiDAR and NDVI data sets. Water Resour. Res. 2014, 50, 6339–6357. [Google Scholar] [CrossRef] [PubMed]
  27. Liljedahl, A.K.; Boike, J.; Daanen, R.P.; Fedorov, A.N.; Frost, G.V.; Grosse, G.; Hinzman, L.D.; Iijma, Y.; Jorgenson, J.C.; Matveyeva, N.; et al. Pan-Arctic ice-wedge degradation in warming permafrost and its influence on tundra hydrology. Nat. Geosci. 2016, 9, 312–318. [Google Scholar] [CrossRef]
  28. Mackay, J.R.; Black, R.F. Origin, composition, and structure of perennially frozen ground and ground ice. In Permafrost: North American Contribution [to the] Second International Conference; National Academy of Science: Washington, DC, USA, 1975; Volume 2, pp. 185–192. [Google Scholar]
  29. Lara, M.J.; McGuire, A.D.; Euskirchen, E.S.; Tweedie, C.E.; Hinkel, K.M.; Skurikhin, A.N.; Romanovsky, V.E.; Grosse, G.; Bolton, W.R.; Genet, H. Polygonal tundra geomorphological change in response to warming alters future CO2 and CH4 flux on the Barrow Peninsula. Glob. Chang. Biol. 2015, 21, 1634–1651. [Google Scholar] [CrossRef] [PubMed]
  30. Wainwright, H.M.; Dafflon, B.; Smith, L.J.; Hahn, M.S.; Curtis, J.B.; Wu, Y.; Ulrich, C.; Peterson, J.E.; Torn, M.S.; Hubbard, S.S. Identifying multiscale zonation and assessing the relative importance of polygon geomorphology on carbon fluxes in an Arctic tundra ecosystem. J. Geophys. Res. Biogeosci. 2015, 120, 788–808. [Google Scholar] [CrossRef] [Green Version]
  31. Fritz, M.; Wolter, J.; Rudaya, N.; Palagushkina, O.; Nazarova, L.; Obu, J.; Rethemeyer, J.; Lantuit, H.; Wetterich, S. Holocene ice-wedge polygon development in northern Yukon permafrost peatlands (Canada). Quat. Sci. Rev. 2016, 147, 279–297. [Google Scholar] [CrossRef] [Green Version]
  32. Drew, J.V.; Tedrow, J.C.F. Arctic soil classification and patterned ground. Arctic 1962, 15, 109–116. [Google Scholar] [CrossRef]
  33. Mackay, J.R. Thermally induced movements in icewedge polygons, western Arctic coast: A long-term study. Geographie Physique et Quaternaire 2000, 54, 41–68. [Google Scholar] [CrossRef]
  34. Kanevskiy, M.; Shur, Y.; Jorgenson, T.; Brown, D.R.N.; Moskalenko, N.G.; Brown, J.; Walker, D.A.; Raynolds, M.K.; Buchhorn, M. Degradation and stabilization of ice wedges: Implications for assessing risk of thermokarst in northern Alaska. Geomorphology 2017, 297, 20–42. [Google Scholar] [CrossRef]
  35. Shur, Y.; Jorgenson, M.T. Cryostructure development on the floodplain of the Colville River Delta, northern Alaska. In Proceedings of the Seventh International Conference on Permafrost, Yellowknife, NT, Canada, 23–27 June 1998. [Google Scholar]
  36. Kanevskiy, M.Z.; Shur, Y.; Jorgenson, M.T.; Ping, C.-L.; Michaelson, G.J.; Fortier, D.; Stephani, E.; Dillon, M.; Tumskoy, V. Ground ice in the upper permafrost of the Beaufort Sea Coast of Alaska. Cold Reg. Sci. Technol. 2013, 85, 56–70. [Google Scholar] [CrossRef]
  37. Jorgenson, M.T.; Shur, Y.L.; Pullman, E.R. Abrupt increase in permafrost degradation in Arctic Alaska. Geophys. Res. Lett. 2006, 33. [Google Scholar] [CrossRef] [Green Version]
  38. Jorgenson, M.T.; Kanevskiy, M.; Shur, Y.; Moskalenko, N.; Brown, D.R.N.; Wickland, K.; Striegl, R.; Koch, J. Role of ground ice dynamics and ecological feedbacks in recent ice wedge degradation and stabilization. J. Geophys. Res. Earth Surf. 2015, 120, 2280–2297. [Google Scholar] [CrossRef]
  39. Walker, D.A.; Webber, P.J. Vegetation. In Geobotanical Atlas of the Prudhoe Bay Region, Alaska; CRREL Report, 80-14, Walker, D.A., Everett, K.R., Webber, P.J., Brown, J., Eds.; U.S. Army Corps of Engineers, Cold Regions Research and Engineering Laboratory: Hanover, NH, USA, 1980; pp. 24–34. [Google Scholar]
  40. Wolter, J.; Lantuit, H.; Fritz, M.; Macias-Fauria, M.; Myers-Smith, I.; Herzschuh, U. Vegetation composition and shrub extent on the Yukon coast, Canada, are strongly linked to ice-wedge polygon degradation. Polar Res. 2016, 35, 27489. [Google Scholar] [CrossRef] [Green Version]
  41. Liljedahl, A.K.; Hinzman, L.D.; Schulla, J. Ice-wedge polygon type controls low-gradient watershed-scale hydrology. In Proceedings of the Tenth International Conference on Permafrost (TICOP), Salekhard, Russia, 25–29 June 2012. [Google Scholar]
  42. Helbig, M.; Boike, J.; Langer, M.; Schreiber, P.; Runkle, B.R.K.; Kutzbach, L. Spatial and seasonal variability of polygonal tundra water balance: Lena River Delta, northern Siberia (Russia). Hydrol. J. 2013, 21, 133–147. [Google Scholar] [CrossRef]
  43. Necsoiu, M.; Dinwiddie, C.L.; Walter, G.R.; Larsen, A.; Stothoff, S.A. Multi-temporal image analysis of historical aerial photographs and recent satellite imagery reveals evolution of water body surface area and polygonal terrain morphology in Kobuk Valley National Park, Alaska. Environ. Res. Lett. 2013, 8, 025007. [Google Scholar] [CrossRef] [Green Version]
  44. Shiklomanov, N.I.; Streletskiy, D.A.; Little, J.D.; Nelson, F.E. Isotropic thaw subsidence in undisturbed permafrost landscapes. Geophys. Res. Lett. 2013, 40, 6356–6361. [Google Scholar] [CrossRef] [Green Version]
  45. Frost, G.V.; Macander, M.J.; Liljedahl, A.K.; Walker, D.A. Regional patterns of ice-wedge degradation since the mid-20th century across northern Alaska. In Proceedings of the Arctic Change 2014 Conference, Ottawa, ON, Canada, 8–12 December 2014; p. 58. [Google Scholar]
  46. Frost, V.G.; Christopherson, T.; Jorgenson, M.T.; Liljedahl, K.A.; Macander, J.M.; Walker, A.D.; Wells, F.A. 2018. Regional Patterns and Asynchronous Onset of Ice-Wedge Degradation since the Mid-20th Century in Arctic Alaska. Remote Sens. 2018, 10, 1312. [Google Scholar] [CrossRef]
  47. Kokelj, S.V.; Lantz, T.C.; Wolfe, S.A.; Kanigan, J.C.; Morse, P.D.; Coutts, R.; Molina-Giraldo, N.; Burn, C.R. Distribution and activity of ice wedges across the forest-tundra transition, Western Arctic Canada. J. Geophys. Res. Earth Surf. 2014, 119, 2032–2047. [Google Scholar] [CrossRef]
  48. Raynolds, M.K.; Walker, D.A.; Ambrosius, K.J.; Brown, J.; Everett, K.R.; Kanevskiy, M.; Kofinas, G.P.; Romanovsky, V.E.; Shur, Y.; Webber, P.J. Cumulative geoecological effects of 62 years of infrastructure and climate change in ice-rich permafrost landscapes, Prudhoe Bay Oilfield, Alaska. Glob. Chang. Biol. 2014, 20, 1211–1224. [Google Scholar] [CrossRef] [PubMed]
  49. Günther, F.; Overduin, P.P.; Yakshina, I.A.; Opel, T.; Baranskaya, A.V.; Grigoriev, M.N. Observing Muostakh disappear: Permafrost thaw subsidence and erosion of a ground-ice-rich island in response to arctic summer warming and sea ice reduction. Cryosphere 2015, 9, 151–178. [Google Scholar] [CrossRef] [Green Version]
  50. Jorgenson, M.T.; Marcot, B.G.; Swanson, D.K.; Jorgenson, J.C.; DeGange, A.R. Projected changes in diverse ecosystems from climate warming and biophysical drivers in northwest Alaska. Clim. Chang. 2015, 130, 131–144. [Google Scholar] [CrossRef] [Green Version]
  51. Pollard, W.; Ward, M.; Becker, M. The Eureka Sound lowlands: An ice-rich permafrost landscape in transition. In Proceedings of the 7th Canadian Permafrost Conference, Quebec City, QC, Canada, 20–23 September 2015. [Google Scholar]
  52. Abolt, C.J.; Young, M.H.; Caldwell, T.G. Numerical modelling of ice-wedge polygon geomorphic transition. Permafr. Periglac. Process. 2016, 28, 347–355. [Google Scholar] [CrossRef]
  53. Christiansen, H.H.; Matsuoka, N.; Watanabe, T. Progress in understanding the dynamics, internal structure and palaeoenvironmental potential of ice wedges and sand wedges. Permafr. Periglac. Process. 2016, 27, 365–376. [Google Scholar] [CrossRef]
  54. Swanson, D. Stability of Ice-Wedges in Kobuk Valley National Park and the Noatak National Preserve, 1951–2009. Natural Resource Report NPS/ARCN/NRR—2016/1248. Available online: https://irma.nps.gov/DataStore/DownloadFile/553217 (accessed on 2 April 2018). [CrossRef]
  55. Perreault, N.; Lévesque, E.; Fortier, D.; Gratton, D.; Lamarque, L.J. Remote sensing evaluation of High Arctic wetland depletion following permafrost disturbance by thermo-erosion gullying processes. Arct. Sci. 2017, 3, 237–253. [Google Scholar] [CrossRef] [Green Version]
  56. Steedman, A.E.; Lantz, T.C.; Kokelj, S.V. Spatio-Temporal Variation in High-Centre Polygons and Ice-Wedge Melt Ponds, Tuktoyaktuk Coastlands, Northwest Territories. Permafr. Periglac. Process. 2017, 28, 66–78. [Google Scholar] [CrossRef]
  57. Jones, B.M.; Grosse, G.; Arp, C.D.; Liu, L.; Miller, E.A.; Hayes, D.J.; Larsen, C. Recent arctic tundra fire initiates widespread thermokarst development. Sci. Rep. 2015, 5, 15865. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  58. Shur, Y.; Kanevskiy, M.; Jorgenson, T.; Dillon, M.; Stephani, E.; Bray, M. Permafrost degradation and thaw settlement under lakes in yedoma environment. In Proceedings of the Tenth International Conference on Permafrost, Salekhard, Russia, 25–29 June 2012; Hinkel, K.M., Ed.; The Northern Publisher: Salekhard, Russia, 2012. [Google Scholar]
  59. Jorgenson, M.T.; Grosse, G. Remote sensing of landscape change in permafrost regions. Permafr. Periglac. Process. 2016, 27, 324–338. [Google Scholar] [CrossRef]
  60. Lousada, M.; Pina, P.; Vieira, G.; Bandeira, L.; Mora, C. Evaluation of the use of very high resolution aerial imagery for accurate ice-wedge polygon mapping (Adventdalen, Svalbard). Sci. Total Environ. 2017, 615, 1574–1583. [Google Scholar] [CrossRef] [PubMed]
  61. Duguay, C.R.; Zhang, T.; Leverington, D.W.; Romanovsky, V.E. Satellite remote sensing of permafrost and seasonally frozen ground. In Remote Sensing in Northern Hydrology: Measuring Environmental Change; Duguay, C.R., Pietroniro, A., Eds.; American Geophysical Union: Washington, DC, USA, 2005; pp. 91–118. ISBN 9780875904283. [Google Scholar]
  62. Kääb, A. Remote sensing of permafrost-related problems and hazards. Permafr. Periglac. Process. 2008, 19, 107–136. [Google Scholar] [CrossRef]
  63. Westermann, S.; Duguay, C.R.; Grosse, G.; Kääb, A. Remote sensing of permafrost and frozen ground. In Remote Sensing of the Cryosphere; Tedesco, M., Ed.; John Wiley & Sons: Hoboken, NJ, USA, 2014; pp. 307–344. ISBN 9781118368855. [Google Scholar]
  64. Chen, Z.; Pasher, J.; Duffe, J.; Behnamian, A. Mapping Arctic Coastal Ecosystems with High Resolution Optical Satellite Imagery Using a Hybrid Classification Approach. Can. J. Remote Sens. 2017, 43, 513–527. [Google Scholar] [CrossRef]
  65. Mora, C.; Vieira, G.; Pina, P.; Lousada, M.; Christiansen, H.H. Land cover classification using high-resolution aerial photography in adventdalen, svalbard. Geogr. Ann. Ser. A Phys. Geogr. 2015, 97, 473–488. [Google Scholar] [CrossRef]
  66. Muster, S.; Langer, M.; Heim, B.; Westermann, S.; Boike, J. Subpixel heterogeneity of ice-wedge polygonal tundra: A multi-scale analysis of land cover and evapotranspiration in the Lena River Delta, Siberia. Tellus B Chem. Phys. Meteorol. 2012, 64, 17301. [Google Scholar] [CrossRef]
  67. Paine, J.G.; Andrews, J.R.; Saylam, K.; Tremblay, T.A. 2015. Airborne LiDAR-based wetland and permafrost-feature mapping on an Arctic Coastal Plain, North Slope, Alaska. In Remote Sensing of Wetlands: Applications and Advances; Tiner, R.W., Lang, M.W., Klemas, V.V., Eds.; CRC Press: Boca Raton, FL, USA, 2015; pp. 413–434. ISBN 9781482237351. [Google Scholar]
  68. Fraser, R.H.; Olthof, I.; Lantz, T.C.; Schmitt, C. UAV photogrammetry for mapping vegetation in the low-Arctic. Arct. Sci. 2016, 2, 79–102. [Google Scholar] [CrossRef] [Green Version]
  69. Ulrich, M.; Hauber, E.; Herzschuh, U.; Härtel, S.; Schirrmeister, L. Polygon pattern geomorphometry on Svalbard (Norway) and western Utopia Planitia (Mars) using high-resolution stereo remote-sensing data. Geomorphology 2011, 134, 197–216. [Google Scholar] [CrossRef] [Green Version]
  70. Haltigin, T.W.; Pollard, W.H.; Dutilleul, P.; Osinski, G.R. Geometric evolution of polygonal terrain networks in the Canadian High Arctic: Evidence of increasing regularity over time. Permafr. Periglac. Process. 2012, 23, 178–186. [Google Scholar] [CrossRef]
  71. Dafflon, B.; Hubbard, S.; Ulrich, C.; Peterson, J.; Wu, Y.; Wainwright, H.; Kneafsey, T.J. Geophysical estimation of shallow permafrost distribution and properties in an ice-wedge polygon-dominated Arctic tundra region. Geophys. 2016, 81, WA247–WA263. [Google Scholar] [CrossRef]
  72. Lewkowicz, A.G.; Duguay, C.R. Detection of permafrost features using SPOT panchromatic imagery, Fosheim Peninsula, Ellesmere Island, NWT. Can. J. Remote Sens. 1999, 25, 34–44. [Google Scholar] [CrossRef]
  73. Skurikhin, A.N.; Gangodagamage, C.; Rowland, J.C.; Wilson, C.J. Arctic tundra ice-wedge landscape characterization by active contours without edges and structural analysis using high-resolution satellite imagery. Remote Sens. Lett. 2013, 4, 1077–1086. [Google Scholar] [CrossRef]
  74. Ulrich, M.; Grosse, G.; Strauss, J.; Schirrmeister, L. Quantifying Wedge-Ice Volumes in Yedoma and Thermokarst Basin Deposits. Permafr. Periglac. Process. 2014, 25, 151–161. [Google Scholar] [CrossRef] [Green Version]
  75. Skurikhin, A.N.; Wilson, C.J.; Liljedahl, A.; Rowland, J.C. Recursive active contours for hierarchical segmentation of wetlands in high-resolution satellite imagery of arctic landscapes. In Proceedings of the 2014 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), San Diego, CA, USA, 6–8 April 2014; IEEE: Piscataway, NJ, USA, 2014. [Google Scholar] [CrossRef]
  76. Pina, P.; Saraiva, J.; Bandeira, L.; Barata, T. Identification of Martian Polygonal Patterns Using the Dynamics of Watershed Contours. In Proceedings of the ICIAR 2006: International Conference Image Analysis and Recognition, Póvoa de Varzim, Portugal, 18–20 September 2006; Campilho, A., Kamel, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
  77. Pina, P.; Saraiva, J.; Bandeira, L.; Antunes, J. Polygonal terrains on Mars: A contribution to their geometric and topological characterization. Planet. Space Sci. 2008, 56, 1919–1924. [Google Scholar] [CrossRef]
  78. Bandeira, L.; Pina, P.; Saraiva, J. A multi-layer approach for the analysis of neighbourhood relations of polygons in remotely acquired images. Pattern Recognit. Lett. 2010, 31, 1175–1183. [Google Scholar] [CrossRef]
  79. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
  80. Romera-Paredes, B.; Torr, P.H.S. Recurrent instance segmentation. In Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer: Cham, Switzerland, 2016. [Google Scholar] [CrossRef]
  81. Penatti, O.A.; Nogueira, K.; dos Santos, J.A. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; IEEE: Piscataway, NJ, USA, 2015. [Google Scholar] [CrossRef]
  82. Marmanis, D.; Schindler, K.; Wegner, J.D.; Galliani, S.; Datcu, M.; Stilla, U. Classification with an edge: Improving semantic image segmentation with boundary detection. ISPRS J. Photogramm. Remote Sens. 2018, 135, 158–172. [Google Scholar] [CrossRef] [Green Version]
  83. Ding, P.; Zhang, Y.; Deng, W.J.; Jia, P.; Kuijper, A. A light and faster regional convolutional neural network for object detection in optical remote sensing images. ISPRS J. Photogramm. Remote Sens. 2018, 141, 208–218. [Google Scholar] [CrossRef]
  84. Gevaert, C.M.; Persello, C.; Nex, F.; Vosselman, G. A deep learning approach to DTM extraction from imagery using rule-based training labels. ISPRS J. Photogramm. Remote Sens. 2018, 142, 106–123. [Google Scholar] [CrossRef]
  85. Morin, P. The high resolution arctic: The ubiquity of sub-meter imagery in the American sciences. In Proceedings of the AGU Fall Meeting 2014, San Francisco, CA, USA, 15–19 December 2014. B54F-05. [Google Scholar]
  86. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; IEEE: Piscataway, NJ, USA, 2017. [Google Scholar] [CrossRef]
  87. Dai, J.; He, K.; Sun, J. Instance-aware semantic segmentation via multi-task network cascades. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; IEEE: Piscataway, NJ, USA, 2016. [Google Scholar] [CrossRef]
  88. Li, Y.; Qi, H.; Dai, J.; Ji, X.; Wei, Y. Fully convolutional instance-aware semantic segmentation. arXiv, 2016; arXiv:1611.07709. [Google Scholar]
  89. Pešek, O. Mask R-CNN v prostředí GRASS GIS. Master’s Thesis, Czech Technical University in Prague, Prague, Czech Republic, 2018. [Google Scholar]
  90. Jones, B.M.; Arp, C.D.; Whitman, M.S.; Nigro, D.; Nitze, I.; Beaver, J.; Gädeke, A.; Zuck, C.; Liljedahl, A.; Daanen, R.; et al. A lake-centric geospatial database to guide research and inform management decisions in an Arctic watershed in northern Alaska experiencing climate and land-use changes. Ambio 2017, 46, 769–786. [Google Scholar] [CrossRef] [PubMed]
  91. Alaska EPSCoR. 2013 Aerial Orthoimagery—Nuiqsut. Available online: http://epscor.portal.gina.alaska.edu/catalogs/11477-2013-aerial-orthoimagery-nuiqsut (accessed on 1 May 2018).
  92. Alaska EPSCoR. 2013 Aerial Orthoimagery—Crea Creek Watershed. Available online: http://epscor.portal.gina.alaska.edu/catalogs/11476-2013-aerial-orthoimagery-crea-creek-watershed (accessed on 1 May 2018).
  93. Alaska EPSCoR. LiDAR Data: Nuiqsut. Available online: http://epscor.portal.gina.alaska.edu/catalogs/11474-lidar-data-nuiqsut (accessed on 1 May 2018).
  94. Liu, S.; Qi, L.; Qin, H.; Shi, J.; Jia, J. Path aggregation network for instance segmentation. In Proceedings of the 2018 Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; IEEE: Piscataway, NJ, USA, 2018. [Google Scholar]
  95. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems 28, Montréal, QC, Canada, 7–12 December 2015; Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R., Eds.; Neural Information Processing Systems Foundation, Inc.: La Jolla, CA, USA, 2015. [Google Scholar]
  96. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; IEEE: Piscataway, NJ, USA, 2016. [Google Scholar] [CrossRef]
  97. Lin, T.Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature pyramid networks for object detection. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; IEEE: Piscataway, NJ, USA, 2017. [Google Scholar] [CrossRef]
  98. Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; IEEE: Piscataway, NJ, USA, 2015. [Google Scholar] [CrossRef]
  99. Abdulla and Waleed. Mask R-CNN for Object Detection and Instance Segmentation on Keras and TensorFlow. Available online: https://github.com/matterport/Mask_RCNN (accessed on 1 April 2018).
  100. Chen, L.C.; Hermans, A.; Papandreou, G.; Schroff, F.; Wang, P.; Adam, H. MaskLab: Instance segmentation by refining object detection with semantic and direction features. arXiv, 2017; arXiv:1712.04837. [Google Scholar]
  101. Blaschke, T. Object based image analysis for remote sensing. ISPRS J. Photogramm. Remote Sens. 2010, 65, 2–16. [Google Scholar] [CrossRef]
  102. Smith, B. On drawing lines on a map. In Proceedings of the International Conference on Spatial Information Theory, Laurel Highlands, PA, USA, 15–18 October 1995; Springer: Berlin/Heidelberg, Germany, 1995. [Google Scholar]
  103. Vogt, L.; Grobe, P.; Quast, B.; Bartolomaeus, T. Fiat or bona fide boundary—A matter of granular perspective. PLoS ONE 2012, 7, e48603. [Google Scholar] [CrossRef] [PubMed]
  104. Engstrom, R.; Hope, A.; Kwon, H.; Stow, D.; Zamolodchikov, D. Spatial distribution of near surface soil moisture and its relationship to microtopography in the Alaskan Arctic Coastal Plain. Hydrol. Res. 2005, 36, 219–234. [Google Scholar] [CrossRef]
  105. Webber, P.J. Vegetation and Production Ecology of an Alaskan Arctic Tundra; Springer: New York, NY, USA, 1978; pp. 37–112. ISBN 978-1-4612-6309-8. [Google Scholar]
  106. Albrecht, F.; Lang, S.; Hölbling, D. Spatial accuracy assessment of object boundaries for object-based image analysis. In Proceedings of the GEOBIA 2010: Geographic Object-Based Image Analysis, Ghent, Belgium, 29 June–2 July 2010; Addink, E.A., Van Coillie, F.M.B., Eds.; ISPRS: Beijing, China. [Google Scholar]
  107. Witharana, C.; Lynch, H.J. An object-based image analysis approach for detecting penguin guano in very high spatial resolution satellite images. Remote Sens. 2016, 8, 375. [Google Scholar] [CrossRef]
  108. Ma, Y.; Wu, H.; Wang, L.; Huang, B.; Ranjan, R.; Zomaya, A.; Jie, W. Remote sensing big data computing: Challenges and opportunities. Future Gener. Comput. Syst. 2015, 51, 47–60. [Google Scholar] [CrossRef] [Green Version]
  109. Youn, C.; Nandigam, V.; Phan, M.; Tarboton, D.; Wilkins-Diehr, N.; Baru, C.; Crosby, C.; Padmanabhan, A.; Wang, S. Leveraging XSEDE HPC resources to address computational challenges with high-resolution topography data. In Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment, Atlanta, GA, USA, 13–18 July 2014; ACM: New York, NY, USA, 2014. [Google Scholar] [CrossRef]
Figure 1. Geographical setting of two study areas. The areas are located near the town of Nuiqsut, Northern Alaska (a) with the ‘Nuiqsut’ scene including the community of Nuiqsut near the Colville River and to the west is the scene ‘Crea Creek’ (bd). The candidate aerial images scenes are presented in false color composites.
Figure 1. Geographical setting of two study areas. The areas are located near the town of Nuiqsut, Northern Alaska (a) with the ‘Nuiqsut’ scene including the community of Nuiqsut near the Colville River and to the west is the scene ‘Crea Creek’ (bd). The candidate aerial images scenes are presented in false color composites.
Remotesensing 10 01487 g001
Figure 2. The 340 subsets of images used in manual annotation. Annotated areas (red outline) from the Nuiqsut scene for (a) training (272 subsets), (b) validation (33 subsets), and (c) testing (35 subsets).
Figure 2. The 340 subsets of images used in manual annotation. Annotated areas (red outline) from the Nuiqsut scene for (a) training (272 subsets), (b) validation (33 subsets), and (c) testing (35 subsets).
Remotesensing 10 01487 g002
Figure 3. General workflow of automated detection, delineation and classification of ice-wedge polygons.
Figure 3. General workflow of automated detection, delineation and classification of ice-wedge polygons.
Remotesensing 10 01487 g003
Figure 4. A simplified schematic showing the implementation architecture of the Mask R-CNN.
Figure 4. A simplified schematic showing the implementation architecture of the Mask R-CNN.
Remotesensing 10 01487 g004
Figure 5. The thirty validation sites (red dots) selected from Nuiqsut (a) and Crea Creek (b) image scenes.
Figure 5. The thirty validation sites (red dots) selected from Nuiqsut (a) and Crea Creek (b) image scenes.
Remotesensing 10 01487 g005
Figure 6. Loss graphs for Mask R-CNN optimization. (a) Smooth-L1 loss; (b) Mask R-CNN bounding box refinement loss; (c) Mask R-CNN classifier loss; (d) mask binary cross-entropy loss; (e) RPN bounding box loss; (f) RPN anchor classifier loss.
Figure 6. Loss graphs for Mask R-CNN optimization. (a) Smooth-L1 loss; (b) Mask R-CNN bounding box refinement loss; (c) Mask R-CNN classifier loss; (d) mask binary cross-entropy loss; (e) RPN bounding box loss; (f) RPN anchor classifier loss.
Remotesensing 10 01487 g006
Figure 7. Example delineation and classification results using the 0.15 m Nuiqsut image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Nuiqsut as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Figure 7. Example delineation and classification results using the 0.15 m Nuiqsut image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Nuiqsut as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Remotesensing 10 01487 g007
Figure 8. Example delineation and classification results using the 0.5 m Nuiqsut image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Nuiqsut as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Figure 8. Example delineation and classification results using the 0.5 m Nuiqsut image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Nuiqsut as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Remotesensing 10 01487 g008
Figure 9. Example delineation and classification results draped using the 1 m Nuiqsut image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Nuiqsut as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Figure 9. Example delineation and classification results draped using the 1 m Nuiqsut image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Nuiqsut as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Remotesensing 10 01487 g009
Figure 10. Example delineation and classification results draped over the corresponding 0.25 m Crea Creek image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Crea Creek as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Figure 10. Example delineation and classification results draped over the corresponding 0.25 m Crea Creek image. (b,d,f,h,j,l,n,p): detected, delineated, and classified IWPs by the Mask R-CNN where the IWP type (low- or high-centered) is given a number 1 (LCP) or 2 (HCP) in the polygon center; (a,c,e,g,i,k,m,o): corresponding portions of enlarged VHSR Imagery for Crea Creek as visual references; (q): their locations in the study area; and (r) geographical distribution of all detected, delineated, and classified IWPs including both LCP and HCPs.
Remotesensing 10 01487 g010
Figure 11. Visual comparison of geographical distribution of delineated LCPs (upper panel) and HCPs (lower panel) using VHSR imagery at different spatial resolutions. Here, the results represent imagery at 0.15 m (a,d), 0.5 m (b,e), and 1 m resolution (c,f) near the community of Nuiqsut.
Figure 11. Visual comparison of geographical distribution of delineated LCPs (upper panel) and HCPs (lower panel) using VHSR imagery at different spatial resolutions. Here, the results represent imagery at 0.15 m (a,d), 0.5 m (b,e), and 1 m resolution (c,f) near the community of Nuiqsut.
Remotesensing 10 01487 g011
Figure 12. Summary of overall accuracies for detection (a), delineation (b), and classification (c) for Nuisqut and Crea Creek.
Figure 12. Summary of overall accuracies for detection (a), delineation (b), and classification (c) for Nuisqut and Crea Creek.
Remotesensing 10 01487 g012
Figure 13. Four selected convolutional feature maps (CFM) from the last output layer of the Mask R-CNN stage 1 to 5 for HCPs. Each feature map includes learned features detected across the image. The third CFM in the 4th row provides the geospatial location features for HCPs.
Figure 13. Four selected convolutional feature maps (CFM) from the last output layer of the Mask R-CNN stage 1 to 5 for HCPs. Each feature map includes learned features detected across the image. The third CFM in the 4th row provides the geospatial location features for HCPs.
Remotesensing 10 01487 g013
Figure 14. Four selected convolutional feature maps (CFM) from the last output layer of the Mask R-CNN stage 1 to 5 for LCPs. Each feature map includes learned features detected across the image. The fourth CFM in the 4th row provides the geospatial location features for LCPs.
Figure 14. Four selected convolutional feature maps (CFM) from the last output layer of the Mask R-CNN stage 1 to 5 for LCPs. Each feature map includes learned features detected across the image. The fourth CFM in the 4th row provides the geospatial location features for LCPs.
Remotesensing 10 01487 g014
Figure 15. Qualitative assessments of HCP extent derived from automated optical imagery mapping (red lines and arrows) to LiDAR data. (a) False color composite of VHSR image in 1 m resolution; (b) LiDAR-derived DTM at 1 m resolution; (c) Canny edge raster from LiDAR DTM; and terrain profiles of LiDAR derived XX” (d) and YY’ (e). The number “2” inside each polygon represent an automated classification as a HCP. The yellow arrows indicate the degree of agreement between automatic delineation and Canny edge.
Figure 15. Qualitative assessments of HCP extent derived from automated optical imagery mapping (red lines and arrows) to LiDAR data. (a) False color composite of VHSR image in 1 m resolution; (b) LiDAR-derived DTM at 1 m resolution; (c) Canny edge raster from LiDAR DTM; and terrain profiles of LiDAR derived XX” (d) and YY’ (e). The number “2” inside each polygon represent an automated classification as a HCP. The yellow arrows indicate the degree of agreement between automatic delineation and Canny edge.
Remotesensing 10 01487 g015
Figure 16. Qualitative assessments of LCP extent derived from automated optical imagery mapping (red lines and arrows) to LiDAR data. (a) False color composite of VHSR image in 1 m resolution; (b) LiDAR-derived DTM at 1 m resolution; (c) Canny edge raster from LiDAR DTM; terrain profile of XX” (d) and YY’ (e). The number “1” inside each polygon represent an automated classification as a LCP. The yellow arrows indicate the degree of agreement between automatic delineation and Canny edge.
Figure 16. Qualitative assessments of LCP extent derived from automated optical imagery mapping (red lines and arrows) to LiDAR data. (a) False color composite of VHSR image in 1 m resolution; (b) LiDAR-derived DTM at 1 m resolution; (c) Canny edge raster from LiDAR DTM; terrain profile of XX” (d) and YY’ (e). The number “1” inside each polygon represent an automated classification as a LCP. The yellow arrows indicate the degree of agreement between automatic delineation and Canny edge.
Remotesensing 10 01487 g016
Figure 17. Qualitative assessments of incomplete/disjoint IWP extent derived from automated optical imagery mapping (red lines and arrows) to LiDAR data. (a) False color composite of VHSR image in 1 m resolution; (b) LiDAR-derived DTM at 1 m resolution; (c) Canny edge raster from LiDAR DTM; terrain profile of XX” (d) and YY’ (e). The number “1” inside each polygon represent an automated classification as a LCP.
Figure 17. Qualitative assessments of incomplete/disjoint IWP extent derived from automated optical imagery mapping (red lines and arrows) to LiDAR data. (a) False color composite of VHSR image in 1 m resolution; (b) LiDAR-derived DTM at 1 m resolution; (c) Canny edge raster from LiDAR DTM; terrain profile of XX” (d) and YY’ (e). The number “1” inside each polygon represent an automated classification as a LCP.
Remotesensing 10 01487 g017
Figure 18. A tentative high-performance computational solution for characterizing IWPs from commercial satellite imagery across the pan-Arctic domain.
Figure 18. A tentative high-performance computational solution for characterizing IWPs from commercial satellite imagery across the pan-Arctic domain.
Remotesensing 10 01487 g018
Table 1. Accuracy assessment for the original (0.15 m) Nuiqsut image.
Table 1. Accuracy assessment for the original (0.15 m) Nuiqsut image.
PredictionPrecisionRecallOverall Accuracy (%)
PositiveNegative
Ground TruthDetectionTrue6131510.980.8079
False100
DelineationTrue597160.980.9796
False100
ClassificationTrue60670.980.9997
False100
Table 2. Accuracy assessment for the resampled (0.5 m) Nuiqsut image.
Table 2. Accuracy assessment for the resampled (0.5 m) Nuiqsut image.
PredictionPrecisionRecallOverall Accuracy (%)
PositiveNegative
Ground TruthDetectionTrue4193450.990.5554
False50
DelineationTrue400200.990.9594
False40
ClassificationTrue41450.990.9998
False50
Table 3. Accuracy assessment for the resampled (1 m) Nuiqsut image.
Table 3. Accuracy assessment for the resampled (1 m) Nuiqsut image.
PredictionPrecisionRecallOverall Accuracy (%)
PositiveNegative
Ground TruthDetectionTrue1136501.000.1515
False00
DelineationTrue100141.000.8888
False00
ClassificationTrue11401.001.00100
False00
Table 4. Accuracy assessment for the original (0.25 m) Crea Creek image.
Table 4. Accuracy assessment for the original (0.25 m) Crea Creek image.
PredictionPrecisionRecallOverall Accuracy (%)
PositiveNegative
Ground TruthDetectionTrue5822210.980.7272
False90
DelineationTrue567150.990.9797
False40
ClassificationTrue559220.990.9695
False70
Table 5. The effect of spatial resolution on the performances of Mask R-CNN in the Nuiqsut study site to identify low- and high-center ice-wedge polygons. The rate of change represents the change in the total number of IWPs compared to the original 0.15 m resolution image.
Table 5. The effect of spatial resolution on the performances of Mask R-CNN in the Nuiqsut study site to identify low- and high-center ice-wedge polygons. The rate of change represents the change in the total number of IWPs compared to the original 0.15 m resolution image.
Pixel Size of Image (m)Number of Delineated LCPRate of Change (%)Number of Delineated HCPRate of Change (%)
0.1532,299-11,449-
0.525,037−227757−32
110,763−571318−83

Share and Cite

MDPI and ACS Style

Zhang, W.; Witharana, C.; Liljedahl, A.K.; Kanevskiy, M. Deep Convolutional Neural Networks for Automated Characterization of Arctic Ice-Wedge Polygons in Very High Spatial Resolution Aerial Imagery. Remote Sens. 2018, 10, 1487. https://0-doi-org.brum.beds.ac.uk/10.3390/rs10091487

AMA Style

Zhang W, Witharana C, Liljedahl AK, Kanevskiy M. Deep Convolutional Neural Networks for Automated Characterization of Arctic Ice-Wedge Polygons in Very High Spatial Resolution Aerial Imagery. Remote Sensing. 2018; 10(9):1487. https://0-doi-org.brum.beds.ac.uk/10.3390/rs10091487

Chicago/Turabian Style

Zhang, Weixing, Chandi Witharana, Anna K. Liljedahl, and Mikhail Kanevskiy. 2018. "Deep Convolutional Neural Networks for Automated Characterization of Arctic Ice-Wedge Polygons in Very High Spatial Resolution Aerial Imagery" Remote Sensing 10, no. 9: 1487. https://0-doi-org.brum.beds.ac.uk/10.3390/rs10091487

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop