28
Oct
2016
Author: Daryl Atencio
A recent project I worked on required custom polygons – Controlled by my application – to be added to the ENVI display. The following code defines an object class that allows the user to place a polygon in the ENVI display using window coordinates. To run the application:
;------------------------------------------------------------------------------
;+
; Lifecycle method for destroying the object
;-
pro my_polygon::Cleanup
compile_opt idl2, logical_predicate
self->Destruct
end
; Cleans up the member variables of the object class
pro my_polygon::Destruct
self->IDLmiManipGraphicOverlay::Cleanup
self->IDLmiManipLayer::Cleanup
self->IDLgrPolygon::Cleanup
; Lifecycle method for initializing the class
;
; :Returns:
; 1 if the object initializes successfully and 0 otherwise
; :Params:
; xy: in, optional, type="int"
; a [2,n] array containing the [x,y] points in window coordinates
; :Keywords:
; POLYGONS: in, optional, type="int"
; An integer array of one or more polygon descriptions. See IDL help for
; IDLgrPolygon for more information
; _REF_EXTRA: Used to set properties of the inherited IDLgrPolygon
function my_polygon::Init, xy, $
POLYGONS=poly, $
_REF_EXTRA=refExtra
void = self->IDLmiManipGraphicOverlay::Init(_EXTRA=refExtra)
if ~void then begin
return, 0
endif
self->InitializeDataspace
self->SetupManipulatorGraphics
self->InitializeGraphics
if n_elements(xy) then begin
self->SetProperty, DATA=xy, POLYGONS=poly
if n_elements(refExtra) then begin
self->SetProperty, _EXTRA=refExtra
return, 1
; This method initializes the data space used by the graphics layer
pro my_polygon::InitializeDataspace
e = envi(/CURRENT)
eView = e->GetView()
eView->GetProperty, _COMPONENT=ecfViewGroup
oDS = ecfViewGroup->GetDescendants(BY_TYPE='DATASPACE', /FIRST_ONLY)
self._oTargetDS = oDS
; Initializes the graphics components of the class
pro my_polygon::InitializeGraphics
void = self->IDLgrPolygon::Init(COLOR=[255,0,0], /PRIVATE, THICK=2)
self._oGrOverlay->IDLmiContainer::Add, self
; This method is for setting class properties
pro my_polygon::SetProperty, $
DATA=data, $
if n_elements(data) then begin
self->SetData, data, POLYGONS=poly
self->IDLgrPolygon::SetProperty, _EXTRA=refExtra
self->IDLmiManipLayer::SetProperty, _EXTRA=refExtra
; This method maps the points from window coordinates to map coordinates and
; adds the mapped points to the IDLgrPolygon.
; xy: in, required, type="int"
; A [2,n] array of points (In window coordinates) to be added to the polygon
pro my_polygon::SetData, xy, $
POLYGONS=poly
self._oTargetDS->WindowToVis, reform(xy[0,*]), reform(xy[1,*]), xVis, yVis
self->IDLgrPolygon::SetProperty, DATA=transpose([[xVis],[yVis]]), $
; Class structure definition
pro my_polygon__define
void = {my_polygon $
, inherits IDLmiManipGraphicOverlay $
, inherits IDLmiManipLayer $
, inherits IDLgrPolygon $
}
pro envi_polygon_example
if ~isa(e, 'envi') then begin
e = envi()
file = FILEPATH('qb_boulder_msi', ROOT_DIR=e.ROOT_DIR, SUBDIRECTORY=['data'])
eRaster = e->OpenRaster(file)
eLayer = eView->CreateLayer(eRaster)
xy = [[470,140],[560,140],[560,230],[470,230], $
[750,115],[1200,115],[1200,665],[750,665]]
conn = [4,0,1,2,3,4,4,5,6,7]
oPolygon = obj_new('my_polygon', xy, LINESTYLE=5, POLYGONS=conn, STYLE=1)
Categories: IDL Blog | IDL Data Point
Tags: IDL, ENVI, custom polygons
27
Author: Daniel Platt
One of the greatest features of the Geospatial Services Framework (GSF), in my opinion, is the fact that it is built upon Node.js. There are many reasons why this is great, but I wanted to talk about one in particular which is the amount of customization this provides.
Below I will show a simple but powerful example of this customization, but before I get there, I want to give a quick overview of both GSF and Node.js.
We have improved ENVI Service Engine by adding GSF – a lightweight, but powerful framework based on Node.js that can provide scalable geospatial intelligence for any size organization. I like to describe it as a “messenger” system that provides a way to communicate between the web and our various products or “engines” that provide analytics such as ENVI, IDL and more.
GSF by design has its Node.js code exposed. This allows the product to be customized infinitely to fit whatever architecture that it needs to reside in. It is a modular system, and has different modules that can be toggled on/off or even duplicated and extended upon. This makes customization easy and safe.
One of these modules is called a Request Handler. This module serves up endpoints for GSF. This can be simply REST based calls, a webpage or even better, both.
While developing an ENVI web client demo that takes Amazon S3 hosted raster data and passes it to GSF to run analytics on, I found that I didn’t have a way to simply list what data is available in my S3 storage. While exploring ways to solve this problem I came to a realization that I can simply use the power of Node.js to accomplish this task.
After importing the aws-sdk package that is already installed with GSF into my request handler, I just wrote a simple function to use that package to list any .dat files in my S3 storage data and return that information to my front end web application to be ingested and displayed.
Here is the slightly modified request handler code with comments explaining each part.
//Import express, used by node.js to setup rest endpoints var express = require('express'); //Import AWS, used to interact with Amazon Web Services including S3 storage var aws = require('aws-sdk'); //Extra tools that should be included in request handlers var extend = require('util')._extend; var defaultConfig = require('./config.json'); /** * Dynamic Request Handler */ function DynUiHandler() { var s3Bucket, s3; //Setup config options var config = {}; extend(config, defaultConfig); //Grab information for S3 from config.json s3Workspace = config.S3Root; s3Bucket = config.S3Bucket; // Setup S3 s3 = new aws.S3(config); /** * List files in the s3 bucket. * @param {object} req - The Express request object. * @param {object} res - The Express response object. */ function listS3Data(req, res) { //Take the parameter passed from the REST call and set that as the bucket to be accessed in the call to s3. var params = { Bucket: req.params.bucket }; //Call the S3 package's built in listObjects function to return what is avaiable in the specified bucket s3.listObjects(params, function(err, data) { //check if error occured and if so, halt and report it to client. if (err) { var code = err.code || 500; var message = err.message || err; res.status(code).send({ error: message }); } // If no error, push every object found in the response with a '.dat' extension to an array that will be returned to client else { var files = []; //Look at each file contained in data returned by s3.listObjects() data.Contents.forEach(function(file) { //Searches for files with .dat in the bucket requested if(file.Key.endsWith('.dat')){ //If found, store that file information in the files array. files.push(file); } }); } //send the files array containing metadata of all .dat files found. res.send(files); }); }; //Initialize request handler, run when GSF starts up. this.init = function(app) { // Set up request handler to host the html subdirectory as a webpage. app.use('/dynui/', require('express').static(__dirname + '/html/')); // Set up a rest call that runs listS3Data and supplies the accomponying bucket parameter. app.get('/s3data/:bucket', listS3Data); }; } // module.exports = DynUiHandler;
After restarting the server I was able to hit my rest endpoint by pasting the following in my browser: http://localhost:9191/s3data/mybucket
This would return me JSON of the contents of “mybucket” Using this information I was able to give a user of my application real time information right inside of the interface about which data they have available to them.
This is a simple example of how I customized a stock request handler to make my web demo more powerful. However, the ability to modify the source code of such a powerful system in a modular fashion provides a safe way to mold GSF into the exact form and shape you need to fit into your own architecture.
It might just be me but I much prefer that over restructuring my own existing architecture to fit closed source code I know nothing about.
Categories: ENVI Blog | Imagery Speaks
Tags: Cloud, ENVI Services Engine, customization, GSF
25
Author: Joey Griebel
If all goes according to plan, next month the GOES-R weather satellite will be launching and the next generation of weather forecasting, solar activity monitoring, and lightening detection will be here. This advanced satellite will change how quickly and accurately we are able to monitor and predict hazardous weather and help give those in harm’s way the time needed to prepare and evacuate. The GOES-R satellite will include:
Advanced Baseline Imager (ABI) – an advanced imager that has 3 times more channels, 4 times better resolution , and 5 times faster than before. All of this leads to better observation of severe storms, fire, smoke, aerosols, and volcanic ash.
Geostationary Lightening Mapper - The lightning mapper will allow mapping of lightning strikes on ground, as well as lightning in the atmosphere. Researchers have found that an increase in lightning activity may be a sign of tornadoes forming, thus providing the data to detect tornadoes faster.
Space Weather Observation - GOES-R will work with NOAA instruments to gather information on radiation hazards from sun that can interfere with communication and navigation systems, damage satellites, threaten power utilities.
Harris Corporation has been supporting NOAA with some aspects on the GOES-R Satellite construction and will be providing the Ground System Support, as well as the 16.4 Meter triband antenna needed to stay in touch with it.
Now what does this have to do with ENVI? Once GOES-R is operational and collecting data, ENVI will be working to support the Harris Weather groups WXconnect systems for data validation and visualization, supporting the ABI data directly in ENVI, as well as working with NOAA to help continue to create advanced products in the future. It is an exciting time for NOAA with this milestone launch and an exciting time to be working with ENVI to get to work the advanced data that will be coming down!
Below are the baseline products, as well as some future products that can be expected.
BASELINE PRODUCTS
FUTURE PRODUCTS
Advanced Baseline Imager (ABI)
Absorbed Shortwave Radiation: Surface
Aerosol Detection (Including Smoke and Dust)
Aerosol Particle Size
Aerosol Optical Depth (AOD)
Aircraft Icing Threat
Clear Sky Masks
Cloud Ice Water Path
Cloud and Moisture Imagery
Cloud Layers/Heights
Cloud Optical Depth
Cloud Liquid Water
Cloud Particle Size Distribution
Cloud Type
Cloud Top Height
Convective Initiation
Cloud Top Phase
Currents
Cloud Top Pressure
Currents: Offshore
Cloud Top Temperature
Downward Longwave Radiation: Surface
Derived Motion Winds
Enhanced "V" / Overshooting Top Detection
Derived Stability Indices
Flood/Standing Water
Downward Shortwave Radiation: Surface
Ice Cover
Fire/Hot Spot Characterization
Low Cloud and Fog
Hurricane Intensity Estimation
Ozone Total
Land Surface Temperature (Skin)
Probability of Rainfall
Legacy Vertical Moisture Profile
Rainfall Potential
Legacy Vertical Temperature Profile
Sea and Lake Ice: Age
Radiances
Sea and Lake Ice: Concentration
Rainfall Rate / QPE
Sea and Lake Ice: Motion
Reflected Shortwave Radiation: TOA
Snow Depth (Over Plains)
Sea Surface Temperature (Skin)
SO2 Detection
Snow Cover
Surface Albedo
Total Precipitable Water
Surface Emissivity
Volcanic Ash: Detection and Height
Tropopause Folding Turbulence Prediction
Geostationary Lightning Mapper (GLM)
Upward Longwave Radiation: Surface
Lightning Detection: Events, Groups & Flashes
Upward Longwave Radiation: TOA
Space Environment In-Situ Suite (SEISS)
Vegetation Fraction: Green
Energetic Heavy Ions
Vegetation Index
Magnetospheric Electrons & Protons: Low Energy
Visibility
Magnetospheric Electrons & Protons: Med & High Energy
Solar & Galactic Protons
Magnetometer (MAG)
Geomagnetic Field
Extreme Ultraviolet and X-ray Irradiance Suite (EXIS)
Solar Flux: EUV
Solar Flux: X-ray Irradiance
Solar Ultraviolet Imager (SUVI)
Solar EUV Imagery
Tags:
20
Author: Austin Coates
Recently, I was given the chance to practice some spectroscopy and in preparation for the project, I realized that I did not have a simple way to visualize the variations in different absorption features between very discreet wavelengths. The method that I elected to employ for this task is called continuum removal (Kokaly, Despain, Clark, & Livo, 2007). This method allows you to compare different spectra and essentially normalize the data so that they can be more easily compared with one another.
To use the algorithm, you first select the region that you are interested in (for me this was between 550 nm and 700 nm -this is the region of my spectra that deals with chlorophyll absorption and pigment). Once the region is selected then a linear model is fit between the two points and this is called the continuum. The continuum is the hypothetical background absorption feature, or artifact or other absorption feature, which acts as the baseline for the target features to be compared against (Clark, 1999). Once the continuum has been set then the continuum removal process can be performed on all spectra in question using the following equation (Kokaly, Despain, Clark, & Livo, 2007).
RC is the resulting continuum removed spectra, RL is the continuum line and, Ro is the original reflectance value.
Figure 1: Original spectra of two healthy plants. The dotted line denotes the continuum line. The x axis shows wavelengths in nm and the y axis represents reflectance.
Figure 2: The continuum removal for wavelengths 550 nm - 700 nm.
The resulting code gives you a tool that will take in two spectral libraries, with one spectra per library, and return two plots similar to what is shown in Figure 1 and Figure 2.
pro Continuum_Removal
compile_opt IDL2
Spectra_File_1 =
Spectra_File_2 =
; Find the bounds for the feature
FB_left = 550
FB_right =700
; Open Spectra 1
oSLI1 = ENVISpectralLibrary(Spectra_File_1)
spectra_name = oSLI1.SPECTRA_NAMES
Spectra_Info_1 = oSLI1.GetSpectrum(spectra_name)
; Open Spectra 2
oSLI2 = ENVISpectralLibrary(Spectra_File_2)
spectra_name = oSLI2.SPECTRA_NAMES
Spectra_Info_2 = oSLI2.GetSpectrum(spectra_name)
; Get the wavelengths
wl = Spectra_Info_1.wavelengths
; Create Bad Bands List (this removes some regions of the spectra associated with water vapor absorption)
bb_range = [[926,970],[1350,1432],[1796,1972],[2349,2500]]
bbl = fltarr(n_elements(wl))+1
dims = size(bb_range, /DIMENSIONS)
for i = 0 , dims[1]-1 do begin
range = bb_range[*,i]
p1 = where(wl eq range[0])
p2 = where(wl eq range[1])
bbl[p1:p2] = !VALUES.F_Nan
endfor
;Plot oSLI1 / oSLI2
p = plot(wl, Spectra_Info_1.spectrum*bbl, xrange = [min(wl, /nan),max(wl, /nan)],$
yrange=[0,max([Spectra_Info_1.spectrum*bbl,Spectra_Info_2.spectrum*bbl], /nan)], thick = 2, color = 'blue')
p = plot(wl, Spectra_Info_2.spectrum*bbl, /overplot, thick = 2, color = 'green')
; create the linear segment
Spectra_1_y1 = Spectra_Info_1.spectrum[where( wl eq FB_left)]
Spectra_1_y2 = Spectra_Info_1.spectrum[where( wl eq FB_right)]
pl_1 = POLYLINE([FB_left,FB_right], [Spectra_1_y1, Spectra_1_y2], /overplot, /data, thick = 2, LINESTYLE = '--')
Spectra_2_y1 = Spectra_Info_2.spectrum[where( wl eq FB_left)]
Spectra_2_y2 = Spectra_Info_2.spectrum[where( wl eq FB_right)]
pl_2 = POLYLINE([FB_left,FB_right], [Spectra_2_y1, Spectra_2_y2], /overplot, /data, thick = 2, LINESTYLE = '--')
; Get the equation of the line
LF_1 = LINFIT([FB_left,FB_right], [Spectra_1_y1, Spectra_1_y2])
LF_2 = LINFIT([FB_left,FB_right], [Spectra_2_y1, Spectra_2_y2])
; Get the values between the lower and upper bounds
x_vals = wl [ where(wl eq FB_left) : where(wl eq FB_right)]
; Compute the continuum line
RL_1 = LF_1[0] + LF_1[1]* x_vals
RL_2 = LF_2[0] + LF_2[1]* x_vals
; Perform Continuum Removal
Ro_1 = Spectra_Info_1.spectrum[ where(wl eq FB_left) : where(wl eq FB_right)]
RC_1 = Ro_1 / RL_1
Ro_2 = Spectra_Info_2.spectrum[ where(wl eq FB_left) : where(wl eq FB_right)]
RC_2 = Ro_2 / RL_2
; Plot the new Continuum Removal Spectra
pl_RC_1 = plot(x_vals, RC_1, color = 'Blue', xrange = [min(x_vals, /NAN), max(x_vals, /NAN)] )
pl_RC_2 = plot(x_vals, RC_2, color = 'Green', /overplot)
Kokaly, R. F., Despain, D. G., Clark, R. N., & Livo, K. E. (2007). Spectral analysis of absorption features for mapping vegetation cover and microbial communities in Yellowstone National Park using AVIRIS data.
Clark, R. N. (1999). Spectroscopy of rocks and minerals, and principles of spectroscopy. Manual of remote sensing, 3, 3-58.
17
Author: James Lewis
In most cases as a geospatial analyst I use ENVI and IDL to look at the world in a rather flat and two dimensional aspect when it comes to data analytics. However, LiDAR, drones, advancements in photogrammetry, and temporal analytics within ENVI have given me the ability to analyze data in new ways that were previously too time consuming, difficult, or simply not available.
I can use tools like the photogrammetry suite to create dense point clouds over areas that it would be difficult to attain the same fidelity, or even fly over with airborne platforms. As small satellites come online they will give us coverage and revisit times unlike anything we’ve had before!
What will we do with all that data?
Utilizing the spatiotemporal tools in ENVI, we can gain information from days, weeks, months, or years’ worth of collects that classic change detection analytics would not support.
Point cloud drone collect Temporal analysis
LiDAR data has become more available for analyst, and we need to be able to exploit it beyond looking at a bunch of points. Using the ENVI LiDAR tools, you can quickly and easily build robust elevation models to be used for everthing from creating an ortho-mosaic to a line-of-sight analysis. We can also extract 3D models for modeling and simulation, or building scenes for decsion making.
Collada models from ENVI LiDAR
DSM and building vectors from LiDAR point cloud
I can also take advantage of IDL to build out custom tools that enable me to become a more efficient analyst. For example, I can create a tool to ingest a point cloud that gives me multiple products, where before I would have needed to create them individually. I also won’t have to rewrite my code over again when I’m ready to move to the enterprise thanks to the new ENVI task system.
Custom tool combing multiple task
ENVI is not only allowing analyst to respond to various problem sets more efficiently, but also to ENVI enables analysts to do things that just weren’t possible a few years ago. Though the exploitation and types of data is forever changing at a rapid pace, ENVI is leading the charge to remain an industry standard for analytics.
Base 60 encoding of positive floating point numbers in IDL
First Valley Shadow Detection
Machine Learning Training for Automatic Target Detection
Sign Up for News & Updates: Stay informed with the latest news, events, technologies and special offers.