Welcome to the Harris Geospatial documentation center. Here you will find reference guides and help documents.


  >  Docs Center  >  Using ENVI  >  Preprocessing AVIRIS Data Tutorial

ENVI

Preprocessing AVIRIS Data Tutorial

Preprocessing AVIRIS Data Tutorial

This tutorial demonstrates how to calibrate an AVIRIS image to surface reflectance, while teaching the following concepts:

  • Why you should preprocess hyperspectral data
  • How to use FLAASH® to correct for atmospheric effects
  • How to plot radiance and reflectance spectra
  • How to remove bad bands from reflectance data, which removes artifacts from the plotted spectra.

You must have a separate license for the Atmospheric Correction Module: QUAC® and FLAASH® to use this tutorial.

This is the first in a series of tutorials that demonstrate how to work with hyperspectral data. Here are the other tutorials:

This tutorial can be used with ENVI 5.1 or later.

Files Used in this Tutorial


Tutorial files are available from our website or on the ENVI Resource DVD in the hyperspectral directory.

The image used in this exercise was collected by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) sensor. AVIRIS data files are courtesy of NASA/JPL-Caltech. The sample image covers the Cuprite Hills area of southern Nevada, an area with diverse mineral types. The scene was collected from an ER-2 aircraft on August 8, 2011. The full original scene is available from NASA/JPL.

File

Description

CupriteAVIRISSubset.dat

AVIRIS radiance image and header file, spatially subsetted and saved to ENVI image format.

AVIRIS11_gain

Text file with multiplication factors used for calibration to spectral radiance

Background


Hyperspectral sensors, also called imaging spectrometers, should be spectrally and radiometrically calibrated before you analyze their data. NASA/JPL has already processed the AVIRIS data to remove geometric and radiometric errors associated with the motion of the aircraft used during data collection. However, the data should be further corrected for atmospheric effects and converted to surface reflectance prior to scientific analysis. Before you begin the tutorial, you should understand some basic concepts about the different types of information stored in the pixel values of your imagery.

Raw DN Data

Pixel values are also called digital numbers, or DN values. They have not yet been calibrated into physically meaningful units; they are 2-byte (16-bit) signed integer data that include the effects of surface reflectance, the solar irradiance curve, scattering, atmospheric gas absorptions, variation in illumination due to topography, and instrument response.

Radiance

Radiance is the amount of radiation coming from an area. It includes radiation reflected from the surface, from neighboring pixels, and from clouds. Calibration to radiance compensates for variations in detector sensitivity. To derive a radiance image from an uncalibrated image, you must multiply the raw data by a series of gain values, one for each band. Resulting pixel values are typically in units of W/(m2 * sr * µm).

Data vendors usually provide gains/offsets in the metadata that accompanies the imagery. NASA/JPL documented the gain values for 2011 AVIRIS data in the file AVIRIS11_gain, which you will use later in this tutorial. By applying the gains from this file, the AVIRIS radiance image will be in units of µW/(cm2 * sr * nm).

Radiance images include the radiation effects of the sun; in fact, a spectrum of a radiance pixel closely matches that of the solar irradiance curve, also called the solar spectrum. For hyperspectral data analysis, you should remove the effects of solar irradiance by calibrating the data to reflectance.

Reflectance

Reflectance is the proportion of the radiation reflected off a surface to the radiation striking it. In hyperspectral data analysis, materials are identified by their reflectance spectra. So calibrating the data to reflectance is the first step toward identifying materials from an image. When analyzing imagery from multispectral sensors (such as Landsat or GeoEye), atmospheric effects are of lesser concern because the channels are designed to accommodate atmospheric gas absorption features. However, hyperspectral sensors cover all of the visible and near-infrared spectrum, including absorption features. So it is important to compensate for these effects in hyperspectral imagery.

Other advantages of using reflectance data include:

  • Spectral features are more apparent in the imagery
  • The shape of the image spectra are mostly influenced by chemical and physical properties of the surface materials, rather than by atmospheric and solar effects.

An atmospheric correction tool such as FLAASH can remove the effects of atmospheric scattering and gas absorptions to produce reflectance data. With hyperspectral data analysis, calibrating imagery to apparent surface reflectance is sufficient; the imagery may still have variations in illumination due to topography. With reflectance data, pixel values typically range from 0 to 1 but are often scaled by some factor to yield integer data. FLAASH automatically scales reflectance values by 10,000.

We will demonstrate these concepts in the steps that follow.

Display the Uncorrected Data


  1. Start ENVI.
  2. From the menu bar, select File > Open and select the file CupriteAVIRISSubset.dat. ENVI displays the image with an approximate true-color representation.
  3. Double-click the filename in the Layer Manager to open the Metadata Viewer. Here, you can see more information about the file. For example:
    • Click the Map Info category to see the pixel size and units.
    • Click the Extents category to see the geographic coordinates of the spatial extent.
    • Click the Spectral category to see the wavelengths defined for each of the 224 bands. The FWHM column lists the full-width-half-maximum values that NASA/JPL provided for the AVIRIS file.
  4. Click the Close button to close the Metadata Viewer.
  5. In the middle part of the Status bar (at the bottom of the application), right-click and select Raster Data Values. When you move the cursor around the image, the Status Bar shows the pixel values for the red, green, and blue channels. These are 2-byte signed integer data.

  1. From the menu bar, select Display > Profiles > Spectral.
  2. In the toolbar, click the Crosshairs button .
  3. Click inside of the image to move the crosshairs to a different location. The Spectral Profile dialog updates with the spectrum of that pixel location. Notice how the shape of the radiance curve changes slightly as you move the crosshairs to different materials within the scene.

The spectral profile, in general, shows some of the common atmospheric features often seen in hyperspectral data:

The FLAASH tool that you will use next will convert the radiance data to reflectance. Because this is already a radiance image, you do not need to use the ENVI Radiometric Calibration tool to calibrate it to radiance. Normally you would do that step first before running FLAASH.

Start FLAASH


FLAASH is a model-based radiative transfer program to convert radiance data to reflectance. Developed by Spectral Sciences, Inc., FLAASH uses MODTRAN4 radiation transfer code to correct images for atmospheric water vapor, oxygen, carbon dioxide, methane, ozone absorption, and molecular and aerosol scattering. To run FLAASH, you must purchase a separate license for the Atmospheric Correction Module: FLAASH and QUAC from Harris Geospatial Solutions.

  1. In the Toolbox search field, type FLAASH. The FLAASH Atmospheric Correction tool appears in the search results.
  2. Double-click the tool name to start it. The FLAASH Atmospheric Correction Model Input Parameters dialog appears.

Select Input File and Define Scale Factors

  1. Click the Input Radiance Image button.
  2. Select the file CupriteAVIRISSubset.dat, and click OK. The Radiance Scale Factors dialog appears.
  3. The input radiance image consists of 2-byte signed integer values. The original image contained floating-point data, which consumes a lot of disk space. Most hyperspectral data providers prefer to scale the data to integer format to conserve disk space. NASA/JPL used a set of gain values to scale the data into integer values in units of µW/(cm2 * sr * nm). In the next few steps, you will point FLAASH to the gain values that were used to scale the data.

  4. Keep the default selection of Read array of scale factors (1 per band) from ASCII file, and click OK.
  5. In the file selection dialog, select the file AVIRIS11_gain, and click Open. The Input ASCII File dialog appears.
  6. The top part of this dialog shows the first several records from the gains file. This file contains two columns: one with scale factors, and another with the corresponding channel numbers. The purpose of the scale factors is to boost the signal from each channel, especially in the shortwave-infrared spectrum where the solar irradiance signal is much lower. This results in higher levels of radiance. In this particular file, Bands 1-110 have a scale factor of 300, Bands 111-160 have a scale factor of 600, and Bands 161-224 have a scale factor of 1200. When each spectrum is divided by the factors in this file, the 16-bit integers will be converted to radiance in units of µW/(cm2 * sr * nm).

  7. Change the Scale Column value to 1.
  8. Accept the other default selections in this dialog, and click OK.

Define Output Files

  1. In the Output Reflectance File field, type the full path of the directory where you want to write the output reflectance file. For the filename, type CupriteReflectance.dat.
  2. In the Output Directory for FLAASH Files field, type the full path of the directory where you want to write all other FLAASH output files. These include a column water vapor image, cloud classification map, journal file, and (optionally) a template file.
  3. In the Rootname for FLAASH Files field, type a root name that will be added as a prefix to the FLAASH output files.

Select Scene and Sensor Options

  1. FLAASH automatically determines the scene's center geographic coordinates, so you do not need to enter these values.
  2. From the Sensor Type drop-down button, select Hyperspectral > AVIRIS.
  3. Fill out the other fields as follows:
    • Sensor Altitude (km): 20
    • Ground Elevation (km): 0.6
    • Pixel Size (m): keep the default value of 15.500.
    • Flight Date: Aug 6, 2011
    • Flight Time (GMT): 19:20:00

The Sensor Altitude field is typically 20 km for the ER-2 aircraft collecting AVIRIS data. The Ground Elevation value was estimated using the Google Earth™ mapping service. (Another option is to view the GMTED2010 DEM file that is installed with ENVI.) The flight date and time were recorded in an *.info file that NASA/JPL provided with the AVIRIS imagery.

While more accurate values are best, none of the fields require precise data. They are only used to determine the approximate geographic location and solar elevation angle at the scene, for the atmospheric correction model.

Select Atmospheric Model Options

Hyperspectral images typically include enough spectral information needed to estimate water vapor and aerosols in the atmosphere. You will select to retrieve water vapor and aerosols in the following steps.

  1. From the Atmospheric Model drop-down button, select U.S. Standard.
  2. Click the Water Retrieval toggle button to select Yes.
  3. Accept the default value of 1135 nm for Water Absorption Feature. If you select 1135 nm or 940 nm, and the feature is saturated due to an extremely wet atmosphere (unlikely for summer in southern Nevada), then the 820 nm feature would be used in its place.
  4. Accept the default value of Rural for Aerosol Model. This is a good option for the location of this scene in southern Nevada, where aerosols are not strongly affected by urban or industrial sources. The choice of model actually is not critical in this case, as the visibility is typically greater than 40 km.
  5. Accept the default values for all remaining fields.
  6. The settings available under the Hyperspectral Settings button at the bottom of the FLAASH dialog are only needed if you are working with a hyperspectral sensor that is not widely recognized. You would use these settings to choose how bands are selected for water vapor and/or aerosol retrieval. Since our data are from a named sensor (AVIRIS), you do not need to define these settings.
  7. Click Apply.
  8. FLAASH processing may take several minutes. When processing is complete, the FLAASH Atmospheric Correction Results dialog appears with a summary of processing results. FLAASH creates several output files in the directory that you defined: a cloud mask image, water vapor image, journal file with processing results, template file with the parameters you defined, and the reflectance file.

  9. Close both FLAASH dialogs.

Display the Reflectance Image


  1. From the menu bar, select File > Open.
  2. Navigate to the output folder for FLAASH files, and select CupriteReflectance.dat. Click Open.
  3. Click the Data Manager button in the toolbar.
  4. Scroll down the list of files until you come to CupriteReflectance.dat.
  5. Right-click on CupriteReflectance.dat, and select Load True Color. The reflectance image appears in the display.
  6. With the radiance and reflectance images both selected in the Layer Manager, click the Cursor Value button in the toolbar.
  7. The On demand updates button in the Cursor Value dialog is enabled by default. Click it to turn off the red probe.
  8. Move the cursor around the display, and compare data values from both images. The following screen capture shows an example of a pixel location over a black area of the image. Look for the "Data" row in each image.

FLAASH automatically scaled the reflectance values by 10,000. (For future reference, you can change this setting under the Advanced Settings button of the FLAASH parameters dialog.) In this example, the pixel value for the red band of the reflectance image is 2031, or 0.2, if divided by 10,000. Reflectance values range from 0 to 1.0.

View Spectral Profile


  1. Click the Spectral Profile button in the toolbar.
  2. Click anywhere in the image. The Spectral Profile shows the spectrum for the selected pixel location.
  3. Note the shape of the reflectance curve and how it generally peaks near 1400 nanometers (nm). FLAASH determined that some bands were "bad" according to the strength of the reflectance signal, and the pixel values from these bands were omitted from the spectral profile. These appear as gaps in the reflectance curve. FLAASH wrote these bad bands to the ENVI header file that accompanies the reflectance image. In the header file, bad bands are indicated by a 0 under the bbl field. Reviewing the header file shows that Bands 108-112 and 155-166 were designated as bad.

  4. In the Go To field in the toolbar, enter pixel coordinates 326,290 and press the Enter key. Note the reflectance curve for this pixel location: it has two spikes centered over 1300 nm and 1500 nm:

Conversion of radiance data to reflectance using a model-based program such as FLAASH can introduce artifacts into the spectra. These artifacts can result from any of the following factors:

  • Mismatches in the spectral calibration of the hyperspectral data set and the spectral radiative transfer calculations
  • Errors in the absolute radiometric calibration
  • Errors in the radiative transfer calculations

In addition, the 1400 nm and 1900 nm water vapor bands typically give very noisy reflectance results. These noisy values result from the low radiance values in these portions of the spectrum and can cause undesirable spikes. The following plot shows an example of this:

FLAASH attempts to correct for these artifacts by applying an EFFORT-based spectral polishing method on the data, if you set the Spectral Polishing toggle button to Yes. But it does not correct every artifact, especially with the noisy water vapor bands.

You can remove these artifacts by adding even more bands to the "bad bands" list in the header file for this AVIRIS scene.

Remove Bad Bands

  1. In the Toolbox search window, type Edit ENVI. The Edit ENVI Header tool name appears.
  2. Double-click the Edit ENVI Header tool name to start it.
  3. In the Edit Header Input File dialog, select CupriteReflectance.dat and click OK. The Header Info dialog appears.
  4. From the Edit Attributes drop-down list, select Bad Bands List. The Edit Bad Bands List dialog appears. All good bands are initially selected by default.
  5. Scroll to the right to see the wavelength centers (in nanometers) for each band. In the next step, you will remove the following wavelength ranges, which should remove the artifacts around 1300, 1400, 1500, and 1900 nm.
    • 1260 to 1560 nm (Bands 98-128)
    • 1760 to 1960 nm (Bands 148-170)
  6. Recall from Step 2 that FLAASH already added Bands 108-112 and 155-166 to the bad bands list. You will overwrite that selection by expanding the range of bands to further exclude. Press the Ctrl key, then unselect the following bands (they should turn from blue to white):
    • Bands 98 to 107
    • Bands 113 to 128
    • Bands 148 to 154
    • Bands 167 to 170

    The total number of remaining selected items should be 170.

  7. Click OK, then click OK in the Header Info dialog. The reflectance image is removed from the display. You will need to display it again.
  8. In the Data Manager, scroll down until you see CupriteReflectance.dat. Right-click on the filename and select Load True Color.
  9. Click the Spectral Profile button in the toolbar.
  10. Click anywhere inside of the display, and notice how the spectral profile curve appears. The following is an example:

Gaps in the curve correspond to the bands that you just removed. Out of the original 224 AVIRIS bands, you are now working with 170. This is still sufficient for hyperspectral image analysis and material identification. See the "Basic Hyperspectral Analysis" tutorial to learn more about these concepts.

QUAC and FLAASH are registered trademarks of Spectral Sciences, Inc.



© 2019 Harris Geospatial Solutions, Inc. |  Legal
My Account    |    Store    |    Contact Us