Thomas Harris is an independent consultant working in big data analytics for applications in Earth observation. You can find his musings on Twitter @t_harris.
Author: Thomas Harris
Thomas Harris (@t_harris) here, reporting in from the ESIP Summer Meeting in Frisco, CO.
The Federation of Earth Science Information Partners (ESIP) is an open networked community that brings together science, data, and information technology practitioners. ESIP is a rapidly growing organization, doing great work promoting a vibrant community around Earth observation.
This week, I’ve been presenting a handful of projects VIS has been working on with collaborators in the Earth observation community, applying ENVI & IDL Services Engine to solve problems in data access and usability by providing online ENVI & IDL analytical capabilities as web services. The diversity of these projects speaks to the power and flexibility of ENVI & IDL Services Engine.
All of these applications highlight the potential value of ENVI & IDL Services Engine to organizations managing large amounts of Earth observation data. Existing IDL code, that may have been developed over decades in support of specific EO missions, can easily be deployed as web services, providing advanced algorithms for working with specific data types to a global audience.
A great example comes from James Goodman at HySpeed Computing who’s been using the ENVI & IDL Services Engine to develop the HICO Online Processing System, a cloud computing system that will provide online, on-demand, scalable remote sensing image processing capabilities. The HICO Online Processing System is funded by the Center for the Advancement of Science in Space (CASIS), and uses imagery from The Hyperspectral Imager for the Coastal Ocean (HICO), an imaging spectrometer that is flying on the International Space Station (ISS), and that's optimized for acquisition of aquatic targets. HICO data is archived and distributed by the Ocean Color team at NASA’s Goddard Space Flight Center. HySpeed Computing was able to easily create web services for a collection of coastal remote sensing algorithms that had been written over a number of years in IDL for deriving information on water properties, water depth, and habitat characteristics. The ENVI & IDL Services Engine allowed HySpeed to deploy the IDL applications to a global community of HICO data users via the web. Be sure to contact me (Thomas.email@example.com) if you’re interested in beta testing the HICO Online Processing System when it goes live via a site hosted on the Amazon Cloud. Click below to check out a poster describing the system.
Joe Lee from the HDFGroup built an on-line processing engine for working with and visualizing NASA Hierarchical Data Format (HDF) data. The HDF Group provides a unique suite of technologies and supporting services that make possible the management of large and complex data collections. The HDFGroup mission is to advance and support Hierarchical Data Format (HDF) technologies, ensuring long-term access to HDF data. The HDFGroup developed NASA HDF-EOS data visualization web services from existing IDL codes, and deployed the services within the the HDFGroup web enterprise. The ENVI & IDL Services Engine works behind the scenes to execute the NASA HDF-EOS visualization application, and creates a visualization that can be displayed on a desktop, across an enterprise, or on any device that has access to the World Wide Web. Check out Joe’s presentation by clicking on the image below.
Martin Landsfeld and Bruce Caron from New Media Research created an open source Drupal module for interfacing with ENVI & IDLServices Engine. Their work provides a way for users of ENVI & IDL Services Engine to easily build a Drupal client interface that can be used to present IDL visualizations to users across the web. New Media Research is pointing the way forward for all those researchers who have IDL-based algorithms for working with specific data types, and who want to share those algorithms with a broader user community via the web. For example, the web site implemented by Marty and Bruce uses data from the Ozone Monitoring Instrument (OMI) on NASA’s Aura spacecraft, and provides users on the World Wide Web a way to interact with these data. If you’re interested in building a web client interface for interacting with ENVI & IDL Services Engine, consider the Drupal framework and extending the IDL/ENVI Bridge Module (1.0) from Science on Drupal Central. Check out a PDF of their poster by clicking on the image below.
Categories: ENVI Blog | Imagery Speaks
Tags: IDL, ENVI, NASA, ENVI Services Engine, HDF, Visualization, earth observation
Some of the best examples showcasing the power of remote sensing involve mineral mapping with imaging spectrometers. Exelis VIS is proud to support the U.S. Geological Survey (USGS) in their efforts to assess the natural resources of Afghanistan, as part of their project, "Oil and Gas Resources Assessment of the Katawaz and Helmand Basins."
Imaging spectrometers have the unique capability to measure light across the reflected portion of the electromagnetic spectrum, from the visible, to the near- and shortwave-infrared portions of the spectrum. An imaging spectrometer can acquire images in hundreds of registered, contiguous spectral channels, such that for each pixel, it's possible to derive a complete reflectance spectrum (image below). These types of data provide a powerful technique for mapping the distribution and relative abundance of minerals on the Earth's surface.
In their recent USGS Data Series Publication, "Surface Mineral Maps of Afghanistan Derived from HyMap Imaging Spectrometer Data," Ray Kokaly, Trude King and Todd Hoefen describe their audacious project to map the majority of Afghanistan, over 438,000 square kilometers, with unprecedented spectral and spatial resolution, using a HyMap imaging spectrometer. You can their report, along with their well-described project scope and methodology HERE.
A major part of any project is data preparation and preprocessing, allowing for higher-order analytics that can answer questions. In this paper, Kokaly, et al. provide a helpful and detailed description of their workflow, from raw data acquisition, to georeferencing and reflectance conversion.
It's the downstream analytics that are most interesting. The USGS team created a spectral analytics algorithm called the 'Material Identification and Characterization Algorithm'(MICA), written in the Interactive Data Language (IDL), and deployed as a module within the USGS PRISM (Processing Routines in IDL for Spectroscopic Measurements) software. PRISM is a free add-on to the ENVI remote sensing software, and is available for download HERE.
The image below shows one of the results from the project, a highly detailed map of iron-bearing minerals and other materials. This figure is a subsampled representation of the entire coverage that shows less than 1/600th the detail in the full resolution data. (Full resolution digital maps suitable for incorporation in a GIS can be downloaded HERE.) Iron ores can be found in association with other valuable deposits like emeralds and silver, so these maps are being applied to natural resource development, mapping previously unknown occurrences of minerals that suggest mineralization and potential targets for resource development. The USGS study is an initial step in realizing the potential offered by cutting-edge remote sensing technology in defining mineral distributions to assist in the economic revitalization of Afghanistan.
Tags: ENVI, Remote Sensing, USGS, hyperspectral imagery, mineral deposit, Geospatial Analysis
Over the last decades, nations around the world have built sophisticated observation systems that can monitor changes to the earth system with high confidence and precision. For example, NASA's EarthObserving System (http://eospso.gsfc.nasa.gov/) is a coordinated series of satellites that monitor long-term global observations of the land surface, biosphere, solid Earth, atmosphere, and oceans, and provides humanity with the baseline observations that permits monitoring of the global climate system.
But as powerful as our observation capability is, there are many unanswered questions related to climate change and the impacts of a warming world across the Earth system. Take for example global precipitation. With a warming global climate, how has global precipitation changed over the last decades? How are local changes in precipitation tied to the larger global climate system? How will precipitation change in the future? How will changes in precipitation affect human society?
To monitor global precipitation, and to help answer these big questions, NASA is working with the Japan Aerospace Exploration Agency (JAXA) to launch a new satellite for monitoring global precipitation, Global Precipitation Measurement (GPM). GPM is an international satellite mission that will provide next-generation observations of rain and snow worldwide every three hours and will be the 'core' observatory for linking precipitation-related observations from a constellation of current and planned satellites to produce next-generation global measurements of rainfall and snowfall from space.
GPM, to be launched on the from the Tanegashima Space Centre, Tanegashima Island, Japan, on February 27, will extend the capability to acquire detailed, near real-time measurements of rain and snowfall on a global scale. One of the most exciting capabilities will be mapping the interior structure of storms in 3 dimensions, extending the capabilities of existing aging satellite systems like Tropical Rainfall Measurement Mission (TRMM).
For example, the TRMM imaging radar has 5 km (3.1 mile) horizontal resolution and 250 meter (820 foot) vertical resolution, which can clearly resolve the structure within storm clouds that contains raindrops and ice large enough to fall as precipitation. The image (above) was acquired by TRMM of Typhoon Bopha as it moved toward Mindanao, the second largest major island in the Philippines, in December of 2012, and reveals details like a double eyewall, two concentric rings of intense storm cells that reach 12 km in altitude. The Dual-frequency Precipitation Radar (DPR) launching with GPM, is expected to be more sensitive than its TRMM predecessor, especially in the measurement of light rainfall and snowfall.
Apart from the impressive capabilities for observing individual precipitation events, a huge benefit from GPM will be in extending the 15-year precipitation record created by TRMM and helping answer some of the big questions around the Earth system and humanity's impact on global climate.
Thanks to Owen Kelley at NASA GSFC (owen dot kelley at nasa dot gov) for the image of Bopha.
Categories: ENVI Blog | Imagery Speaks, IDL Blog | IDL Data Point
Tags: IDL, NASA, earth observation, TRMM, GPM
Exelis is proud to support NASA's DEVELOP intern program. This program pairs student interns with NASA and partner science advisors to work on environmental research projects. Exelis supports this mentor program through access to ENVI and IDL, our software solutions that are the standard platform for working with and gaining insights from geospatial Earth observation data.
Each semester, DEVELOP participants create a 'Virtual Poster' to present their work and a competition is held to determine the winner. The 2013 competition included 21 projects conducted by 96 participants from around the world. The 2013 grand prize was awarded to a team of four interns form DEVELOP’s node at the University of Georgia for “Making the Connection: Reforestation in the Bellbird Corridor of Costa Rica”. The team was able to maximize conservation efforts for the Pájaro Campana Biological Corridor, one of the most biologically diverse hotspots in the world, through the use of Earth-observing satellites like Landsat and RapidEye.
This work is important for its implications for improving biodiversity and habitat conservation in Costa Rica. Costa Rica is one of the most biologically diverse countries, and the Pájaro Campana Biological Corridor (PCBC) stretches from the cloud forests of Monteverde to the mangroves off the Gulf of Nicoya within Costa Rica, providing habitat for 83 mammalian species and nearly 500 bird species. Much of the land within the biological corridor is privately owned and utilized for agricultural activities like ranching. The team at University of Georgia utilized spatial statistics to analyze satellite-derived maps to find areas within the biological corridor that could provide the most return through focused reforestation efforts. Watch a YouTube video explaining the project.
Each member of the winning team will receive a one-year trial version of ENVI and ArcGIS® software, provided by competition co-sponsors Exelis and Esri.
Tags: IDL, ENVI, NASA, Esri, ArcGIS, Landsat, environmental monitoring, earth observation
Since support for NPP VIIRS appeared in ENVI (5.0 SP3), I've worked with a number of ENVI users who were confused about VIIRS data support. In this post I'll clarify sources of confusion for folks currently using VIIRS in ENVI, and give an overview of the upgraded VIIRS support that will be available in ENVI 5.1, coming soon.
VIIRS is a highly complex sensor system that collects imagery in 22 separate visible and infrared channels which can be delivered with a number of different ancillary data records. Most of the confusion I've encountered around supported data is related to data access and supported data types.
One source of confusion is data access...There are several different web portals where VIIRS imagery can be found for download,such as from various NASA-sponsored Distributed Active Archive Centers (DAACs). ENVI only supports VIIRS data products accessed via NOAA's Comprehensive Large Array-Data Stewardship System (CLASS). CLASS is the definitive source for VIIRS data, providing data products in HDF5 format. Other files acquired from sources other than CLASS are not supported in ENVI.
Another source of confusion for users of VIIRS imagery in ENVI is related to supported data types. ENVI 5.0 Service Pack 3 currently reads a subset of the total collection of data types available through CLASS,specifically the following:
A huge upgrade for ENVI is coming soon in ENVI 5.1, when Environmental Data Records (EDRs) will be supported. These are the EDR products that you'll be able to work with directly in ENVI:
Georeferencing for your VIIRS data is only available if the file you selected has Latitude and Longitude bands. Note that individual VIIRS data granules include geolocation latitude and longitude bands by default, so geolocation shouldn't be an issue unless the option to 'Package Geolocation with NPP Data Products' is deselected via the 'Advanced Options' in the 'Shopping Cart'.
See this tutorial on Data Access from CLASS by NOAA NCDC.
Tags: ENVI, NASA, NOAA, NPP
Sign Up for News & Updates: Stay informed with the latest news, events, technologies and special offers.