Gap filling

From Open Foris Wiki

Jump to: navigation, search

An Exercise how to fill missing scan-lines is provided under oft-gapfill

This HowTo provides an Example of filling Landsat image gaps caused by clouds, cloud shadows, etc

1. Take 2 atmospherically corrected Landsat images from same path/row. The aim is to have one good image (so called anchor) with as few problematic areas as possible and then another which is from same season (as close a date as possible) and has clouds in different locations (so called filler). Note 1: It would be preferable to run the brdf-correction for the imagery first. Note 2: If the atmospheric correction is poor, consider working with original LS imagery or brdf-corrected original imagery.

2. Create a 14-band stack and a mask of your images using script oft-stack_mask2images.bash:
 oft-stack_mask2images.bash anchor_folder filler_folder 

3. Evaluate the mask visually in Qgis. If clouds and cloud shadows are missing, digitize them into a shapefile. The ones in the anchor should get value 1 and the ones in the filler value 3. Also, water affects the models unfavourably and should be removed, i.e. set to value 3 in mask. A simple water threshold (R-SWIR)/(R+SWIR) can be used to detect water pixels. Large burnt areas may also appear, and can be detected similarly with another index (NIR-SWIR)/(NIR+SWIR). For both types, select the appropriate threshold after visually inspecting the output and set the water or burnt areas into 3 in another oft-calc run.

4. Combine the new bad areas into the existing mask with script oft-combine-masks.bash:

oft-combine-masks.bash maskXXXX.img clouds.shp 2 

5. Use program oft-gapfill to fill the gaps in the anchor with the filler, when possible:

oft-gapfill -la 8 -nolocal -sd 4 -um combined_mask.img 14band_composite.img filled.img 

6. Test with different numbers of large area windows (-la) and sampling densities, if you are not satisfied with the result.

7. If you need to use several fillers, here is one way to do it:

- Repeat the process above, with the original anchor and the second filler
- Combine the results of the first fill and second fill, and both masks into one composite image. Note that in order to to work with multiband images, it needs to be the new version -o stacked_fills.img -separate firstfill.img secondfill.img mask1.img mask2.img 
- Fill pixels in first fill with pixels in second fill with oft-calc, IF the first fill was empty at that location (value 3 in the first mask), but the second fill was successful (value 1 in the second mask)
oft-calc stacked_fills.img two_fills.img
#15 3 = #16 1 = * #1 #8 ?
#15 3 = #16 1 = * #2 #9 ?
#15 3 = #16 1 = * #3 #10 ?
#15 3 = #16 1 = * #4 #11 ?
#15 3 = #16 1 = * #5 #12 ?
#15 3 = #16 1 = * #6 #13 ?
#15 3 = #16 1 = * #7 #14 ?


A trial with very cloudy area 166/063 (Tanzania)

  1. One about suitable anchor image was found (day 20 in 2011). It had only few cloud clusters and, of course, the stripes with nodata
  2. Day 36 in 2011 was seasonally close and very nicely had the stripes in different location. It was thus used as the first filler
  3. No brdf-correction was carried out, results from this kind of process will be reported later.
  4. The clouds/shadow mask from ledaps-process did not cover all clouds and especially not all shadows. Gapfilling with this base mask gave results where the filled stripes were significantly different from the anchor in some parts of the image (due to working in several large area windows, the errors do not affect all locations of the image). Using more windows helped in some locations, as well as lowering sampling density (increasing -sd number) in hope of missing the bad pixels in the model, if they are few.
  5. The clouds and shadows were digitized manually from both anchor and filler, and added to the base mask. Image resulting from the oft-gapfill was far better.
  6. However, as the sea area was not completely masked off by the ledaps mask, there were problematic areas by the coast. The sea was masked off manually, as well. Again, improved results, but there were some narrow bays that were difficult to remove manually without getting rid of usable pixels - and those water pixels still affected the result. The threshold above was applied and these areas were successfully removed. Now, no problems.
  7. Then, a second filler was taken into consideration. It was 2010, day 33, i.e. again from nearby seasonal conditions, but from previous year.
  8. It was partly very cloudy, and the ledaps-mask was incomplete. It was considered too tedious to manually digitize all the clouds and shadows. Thus, an opposite approach was taken, and the large cloud-free portion was digitized as "good" area. Then, it was combined with the base-mask from ledaps, so that "if 2 in self-made mask, put base mask value, otherwise put 3". Programs used: and oft-calc
  9. The original anchor was used in the second fill process, not the filled image. Thus, again all possible gaps were filled. But when combining the 2nd fill into the 1st fill, only pixels not filled in the 1st fill were considered.
  10. Further 3 cloudy images were used similarly as fillers, i.e. the usable parts were extracted and rest of image area set as "do nothing" in the mask.
  11. The sample images below are from this process.
Original Landsat 7 After first fill After second fill


Please follow the link oft-gapfill

Back to Open Foris Toolkit Main Page

Back to Tools & Exercises

Personal tools