Satellite Imagery
Working with and analyzing satellite imagery as it relates to PhenoCam imagery.
The Landsat program’s long history, freely available moderate resolution imagery, and wide use by the remote sensing community make it an excellent source of satellite imagery with which to compare to PhenoCam observations.
Conveniently, surface reflectance data derived from from Landsat satellites 4-5, 7, and 8 imagery are now available for download. The Landsat surface reflectance product includes a data quality image in addition to imagery that has been atmospherically corrected. This data quality image allows one to screen individual pixels for clouds, cloud shadows, snow, and water.
Landsat satellites 4-8 cover the period between July 1982 through the present (see here for more information on the Landsat Missions Timeline). Given that the earliest PhenoCam put in operation occurred in 2002, Surface Reflectance data should be available for the entire period of every operational PhenoCam.
Nevertheless, there are differences between the Landsat satellites that one must be aware of when using Surface Reflectance data from more than one platform (e.g., one may be interested in obtaining all Landsat 7 and 8 images for a given area). The bands of one satellite may not, for example, exactly match those of another (e.g., Landsat 7 band 1 most closely matches Landsat 8 Band 2). The characteristics of the Landsat satellites can be found here.
Landsat imagery can be downloaded from the USGS's Earth Explorer web data-portal. Note that a full tutorial of Earth Explorer is available online. The following should serve only as an introduction to acquiring Landsat imagery for comparison to PhenoCam image data.
New users will need to register for a free account before use. Once logged into the system, one first specifies search criteria that will be used to filter datasets to those relevant to the PhenoCam site(s) of interest.
The first step is to specify an area of interest. One may either draw an area of interest directly onto the web map, or add a shapefile representing the area of interest.
To draw an area of interest directly onto the map, simply click within the map. Each click will place a coordinate that defines the edge of a polygon. If any mistakes are made, one can either clear all coordinates by clicking the Clear Coordinates button, or click the red 'X' next to the coordinates one wishes to delete.
A Shapefile containing a linestring or polygon can be uploaded by selecting the "Shapefile" tab. Shapefiles should be uploaded as a single compressed folder containing all of a shapefile's component files (i.e., .prj, .sbx, .shp, etc.).
The Earth Explorer also has options for searching by address, Landsat Path/Row coordinates, State, County, congressional district, and prominent landmarks.
After an area of interest has been defined, one should choose a date range that matches the time frame of interest. If one is interested in comparing Landsat imagery to PhenoCam data from a site that has collected imagery between July 2012 and the present, for example, one would wish to search for landsat images fromm July 2012 to the present date.
Once the area of interest and date range have been set, clicking the Data Sets button or tab will bring you to a data set selection menu. To search for Landsat surface reflectance data sets, expand the "Landsat CDR" option and checkmark the Landsat surface reflectance products one wishes to search for.
When one is done selecting datasets, one may either select additional criteria, or click the Results button to view results that match the area, time, and dataset parameters previously set.
The Additional Criteria tab includes options for searching for Landsat Scene ID, WRS path and row, sensor type, data type level, percent cloud cover, and acquisition time of day. One may wish to search only for Landsat scenes with less than 70% cloud cover to help remove scenes that have clouds obscuring the PhenoCam area of interest. Given the relatively large area covered by the Landsat image footprint, however, a higher percent cloud cover may not matter if the small footprint of the PhenoCam of interest is mostly cloud free.
Once one has reached the Results tab, a list of Landsat scenes matching previously set search parameters is populated. If more than one data set was selected in the Data Sets tab (e.g., Landsat 7 and Landsat 8 Surface Reflectance), the list of results will only include scenes from one data set at a time. One may use the dropdown under 'Data Set' to switch data sets.
Each listed result includes options for viewing the image footprint or a browse overlay of the image in the map window, a table of metadata for the scene, and the choice of either adding the scene to one's cart for ordering, or excluding the scene.
One may wish to use the image browse option to screen images that have full cloud cover over the PhenoCam area of interest, and exclude those results. Those results one wishes to order can either be individually selected by pressing the Order Scene button, or the Show Results Controls dropdown near the top of the Search Results tab can be expanded to choose the 'Add All results From Current Page to Order.' This option will add all results from the current page that have not been marked for exclusion to one's cart.
Once all scenes from each data set have been been reviewed, and those one wishes to download have been added to the cart, click on the "Item Basket" option at the upper right hand of the screen. An Order page will load, allowing one to review selected items and submit the order.
After an order has been submitted, an email will be sent to the user with confirmation information. Large order requests may take several hours to process. When one's order is ready for download, an email will be sent to the user.
If one is downloading more than a few images at a time, it may be desirable to use a tool to assist with bulk downloading the images.
There are two options for bulk downloading 'High Level Science Data Products' such as Landsat Surface Reflectance images:
- The Firefox Browser's DownloadThemAll utility.
- The ESPA Bulk Download Client
Each downloaded Landsat Surface Reflectance product is bundled as a zipped archive. A user should extract the contents of this zipped archive to a folder in order to view the available data.
One may either use a standard archive extraction tool, such as winzip to extract data from an archive, or use the unpack_landsat() function currently included in unpack_landsat.py:
from unpack_landsat import unpack_landsat
path_to_archive = '/path/to/archive.tar.gz'
path_to_output_directory = '/path/to/output/directory/'
# Extract all contents from archive
unpack_landsat(path_to_archive, path_to_output_directory)
# Extract only Landsat bands 1,2,3, and 4
unpack_landsat(path_to_archive, path_to_output_directory, bands=[1,2,3,4])
# Extract only bands 1, 2, 3, 4 and the cfmask band
unpack_landsat(path_to_archive, path_to_output_directory, bands=[1,2,3,4], clouds=True)Alternatively, unpack an entire directory of Landsat archives:
from unpack_landsat import unpack_dir
path_to_dir = '/path/to/directory/containing/archives/'
path_to_outdir = '/path/to/output/directory/'
# Unpack all archives
unpack_dir(path_to_dir, path_to_outdir)Complete details about the Landsat surface Reflectance products and the files included in each archive can be found here for Landsat 4-7 and here for Landsat 8.
Each Landsat archive contains some or all of the following files, where SCENEID is replaced by the Landsat scene ID in question:
- SCENEID.xml: An XML formatted text file containing scene metadata
- SCENEID_cfmask.tif: A .tif image with each pixel characterized as either Clear, Water, Shadow, Snow, or Cloud. Note that this band is considered to be more accurate than the other cloud bands listed below. More details follow.
- SCENEID_sr_adjacent_cloud_qa.tf: Binary image showing pixels adjacent to clouds
- SCENEID_sr_band#.tif: The image of the given band number (#).
- SCENEID_sr_cloud_qa.tif: Binary image showing pixels covered by clouds
- SCENEID_sr_cloud_shadow_qa.tif: Binary image showing pixels contaminated by cloud shadow.
As one can see, each band of the Landsat scene is separate from the others. If one wishes to create a composite image for display, this must be using image processing software (ArcGIS, ENVI, ERDAS), or using the stack_layers() function included with unpack_landsat.py:
from unpack_landsat import stack_layers
in_dir = '/directory/to/unzipped/landsat/archive/'
out_path = '/path/to/save/stacked/image.tif'
# Stack layers 2, 3, and 4:
stack_layers(in_dir, out_path, bands=[2,3,4])It may be desirable to clip landsat images to the PhenoCam's field of view. Indeed, a single Landsat scene with all bands included can be in excess of 1GB in size. One may clip a raster image to a polygon representing the PhenoCam field of view using image processing software such as ArcGIS, ENVI, or ERDAS. Alternatively, the clip_raster() function included with unpack_landsat.py may be used:
from unpack_landsat import clip_raster
path_to_raster = '/path/to/full/raster.tif'
path_to_out_ras = '/path/to/save/clipped/raster.tif'
path_to_shapefile = '/path/to/shapefile.shp'
# Clip the raster to the shapefile
clip_raster(path_to_raster, path_to_out_ras, path_to_shapefile)Clipping the Landsat raster images can save space and significantly improve processing time.
The cfmask band included with the Landsat surface reflectance product provides information about the status of each pixel in the Landsat scene. Depending on a pixel's conditions, a pixel will be assigned one of the following numerical values:
- 255: Fill (no data)
- 0: Clear
- 1: Water
- 2: Shadow
- 3: Snow
- 4: Cloud
The cfmask may therefore be used to mask unwanted pixels during analysis. One may wish, for example, to calculate mean NDVI over the PhenoCam's field of view for only pixels that have a value of 0 (clear). This is especially important for Landsat scenes including clouds.
One can create masks in most image processing software. Alternatively, a mask can be generated with unpack_landsat.py's cfmask_to_mask function, which can then be used with numpy's maskedarray:
import gdal
import numpy as np
import greenness as grn
from unpack_landsat import cfmask_to_mask
# Open the cfmask band
cfmask_raster = gdal.Open('/path/to/landsat/cfmask.tif')
# Convert to a binary numpy array:
mask = cfmask_to_mask(cfmask_raster)
# Open a landsat 7 stack with bands 1,2,3, and 4
# (note the landsat raster and cfmask raster must be the same size.
# If landsat raster was clipped, clip cfmask with the same shape)
landsat_raster = gdal.Open('/path/to/stacked/image.tif')
# Extract the RGB and IR components from the stacked image:
# Read the raster as an array
landsat_raster = landsat_raster.ReadAsArray()
# Swap the axes so [rows, cols, bands]
landsat_raster = np.swapaxes(np.swapaxes(landsat_raster, 0, 2), 0, 1)
# Separate RGB and IR
rgb = landsat_raster[:, :, 0:3]
ir = landsat_raster[:, :, 3]
# Calculate mean NDVI
mean_ndvi = grn.mean_ndvi(rgb, ir, roi=mask)
# Calculate mean GCC
mean_gcc = grn.mean_gcc(rgb, roi=mask)For more about masking and masked arrays, see Masking an image