19

Jan

2017

Extending ENVI Extensions

Author: Zachary Norman

One of my favorite parts of being able to extend ENVi with IDL is that I have the ability to add custom buttons into ENVI's toolbox. These buttons are called extensions and I have made many of them during my time with Harris Geospatial. I have created buttons for our tradeshow demos, examples for customers, and even for the Precision Ag Toolkit. Extensions in ENVI can you anything that you want them to. Most of the time this is using ENVI's dynamic UI for creating simple widgets that allow a user to select all the inputs and outputs for a task, but you can also just call IDL directly and instantiate some other GUI based application if you want.

While extensions are very easy to make (I just copy/paste old code I had and tweak it), they can be cumbersome when you have many extensions to create all at once. In addition to this, if you have some addition that you want added to all your extensions, then you have to go through each one by hand to change them. This is where getting creative with IDL programming comes in really handy because IDL can be used to automate the generation of buttons in ENVI. For this example, I decided to finally take a crack at dynamically creating X number of buttons from ENVI tasks without needing to write a separate extension for every one.

The basic idea is that I wanted a dynamic extension that would allow me to pass in any task, create a button, run the right task when I press the button, and offer the ability to display any and all results from the task that are EVNIRasters, ENVIROIs, or ENVIVectors (or shapefiles). If you're unfamiliar how to add extensions to ENVI, then here is a link to the docs which can provide some background information

http://www.harrisgeospatial.com/docs/ENVI__AddExtension.html

Following the example in the docs, the way that I decided to proceed was to use the UVALUE for the buttons to pass in information about which task I wanted to create a dynamic UI for. I decided to use an orderedhash for my data structure with: the key is the task name, the value is a two element string array that represents the folder that we want the task to appear in the name of the button that you will see in ENVI's toolbox. Here is an example of what the data structure looked like:

;create an orderedhash of buttons that we want to create
buttons = orderedhash() 
buttons['ROIMaskRaster'] = ['/Regions of Interest', 'ROI Mask Raster']

At this point it just came down to looping over the hash correctly so that the buttons would be made in the right space. I chose to use the foreach loop because it makes it easy to get the key and value from hashes or orderedhashes all at the same time. Here is what that code looks like:

foreach val, buttons, key do begin
  e.AddExtension, val[1], 'better_extensions', $
    PATH = '/Better Extensions' + val[0], $
    UVALUE = key 
endforeach

That short code block will dynamically go through every entry in my orderedhash and then create the buttons with the right placement and task to be executed when clicked on. Note that I created a subfolder called 'Better Extensions' that would contain all the fancy buttons that the extension would create. Once I had this code figured out, I just needed to look at the actual procedure which would run the extension (I called it "better_extensions")

You can see the complete code for better_extensions below, but there are a few key points to mention about the code:

  • To get the task name for the button that was clicked, when creating the extension I used the UVALUE keyword to store the name. Then, in the better_extensions procedure, I get the event as an argument and can get the uvalue for the button using widget_control.

  • To create the task UI all I needed to do was the following:

        ;create a dialogue for the rest of the task
        ok = e.UI.SelectTaskParameters(task)
    
  • After that, I just needed to go through each task parameter, check for INPUT or OUTPUT and, if it was output, check for ENVIRaster, ENVIVector, or ENVIROI. I then saved these and displayed them in ENVI's current view if they were found in the output.

The main goal here was to show that, with a little bit of IDL programming, you can take your ENVI analytics to the next level. The code for the extension can be found below. Cheers!

Also, keep an eye out for my next blog where I'm going to talk about giving an old extension (from the online extensions library) a much needed update!

; Add the extension to the toolbox. Called automatically on ENVI startup.
pro better_extensions_extensions_init

  ; Set compile options
  compile_opt idl2

  ; Get ENVI session
  e = envi(/CURRENT)
  
  ;create an orderedhash of buttons that we want to create
  buttons = orderedhash() 
  
  ;sample ROI tools
  buttons['ROIMaskRaster'] = $
    ['/Regions of Interest', 'ROI Mask Raster']
  bttons['ROIToClassification'] = $
    ['/Regsions of Interest', 'Classification Image from ROis']
  
  ;classification tools
  buttons['ISODataClassification'] = $
    ['/Classification/Unsupervised Classification', 'ISOData Classification']
  buttons['ClassificationAggregation'] = $
    ['/Classification/Post Classification', 'Classification Aggregation']
  buttons['ClassificationClumping'] = $
    ['/Classification/Post Classification', 'Clump Classes']
  buttons['ClassificationSieving'] = $
    ['/Classification/Post Classification', 'Sieve Classes']
  buttons['ClassificationSmoothing'] = $
    ['/Classification/Post Classification', 'Classification Smoothing']
  buttons['ClassificationToShapefile'] = $
    ['/Classification/Post Classification', 'Classification to vector']
  
  ;radiometric correction tools
  buttons['DarkSubtractionCorrection'] = $
    ['/Radiometric Correction', 'Dark Subtraction']
  buttons['ApplyGainOffset'] = $
    ['/Radiometric Correction', 'Apply Gain and Offset']

  ;filters
  buttons['BitErrorAdaptiveFilter'] = $
    ['/Filters', 'Bit Errors Filter']
  buttons['GaussianHighPassFilter'] = $
    ['/Filters', 'Gaussian High Pass Filter']
  buttons['GaussianLowPassFilter'] = $
    ['/Filters', 'Gaussian Low Pass Filter']
  buttons['HighPassFilter'] = $
    ['/Filters', 'High Pass Filter']
  buttons['LowPassFilter'] = $
    ['/Filters', 'Low Pass Filter']
  
  ;add the buttons
  foreach val, buttons, key do begin
    e.AddExtension, val[1], 'better_extensions', $
      PATH = '/Better Extensions' + val[0], $
      UVALUE = key 
  endforeach

end

; ENVI Extension code. Called when the toolbox item is chosen.
pro better_extensions, event

  ; Set compile options
  compile_opt idl2

  ;Get ENVI session
  e = envi(/CURRENT)
  if (e eq !NULL) then begin
    e = envi()
  endif

  CATCH, Error_status
  IF Error_status NE 0 then begin
    catch, /CANCEL
    help, /LAST_MESSAGE, output = err_txt
    p = dialog_message(err_txt)
    return
  ENDIF

  ;get the directory that our extension lives in
  WIDGET_CONTROL, event.id, GET_UVALUE = taskName
  
  ;create the task object
  task = ENVITask(taskname)
  
  ;add parameter to ask if we want to displayt he results
  displayParam = ENVITaskParameter(NAME='DISPLAY_RESULT', $
    DISPLAY_NAME = 'Display Result',$
    DEFAULT = !FALSE,$
    TYPE='bool', $
    DIRECTION='input', $
    REQUIRED = 1)
  ;replace old parameter
  task.AddParameter, displayParam

  ;create a dialogue for the rest of the task
  ok = e.UI.SelectTaskParameters(task)

  ;user selected OK
  if (ok eq 'OK') then begin
    ;get rid of dummy result
    display = task.DISPLAY_RESULT
    task.RemoveParameter, 'DISPLAY_RESULT'

    ;run the task
    task.execute

    ;check if we want to display things
    if display then begin
      ;things to display
      rasters = list()
      rois = list()
      shapefiles = list()
      
      ;check for output datatypes that are rasters, vectors, or rois
      foreach paramName, task.ParameterNames() do begin
        param = task.parameter(paramName)
        
        if (param.direction eq 'OUTPUT') then begin
          if (param.TYPE eq 'ENVIRASTER') then begin
            e.data.add, param.VALUE
            rasters.add, param.VALUE
          endif
          if (param.TYPE eq 'ENVIROI') then begin
            e.data.add, param.VALUE
            rois.add, param.VALUE
          endif
          if (param.TYPE eq 'ENVIVECTOR') then begin
            e.data.add, param.VALUE
            shapefiles.add, param.VALUE
          endif
        endif
      endforeach
      
      ;get envi's view
      View1 = e.GetView()
      
      ;disable refresh
      e.refresh, /DISABLE
      
      ;check for rasters and ROIs
      if (n_elements(rasters) gt 0) then begin
        
        ;display each raster in the view
        foreach raster, rasters do begin
          rasterLayer = View1.CreateLayer(raster)
        endforeach
        
        ;only display roi is there is a rasterlayer
        foreach roiarr, rois do begin
          ;handle arrays of rois
          foreach roi, roiarr do begin
            roilayer = rasterlayer.AddROI(roi)
          endforeach
        endforeach
      endif
      
      ;check for vectors
      ;only display vectors if ROI is present
      foreach vector, shapefiles do begin
        ; Create a vector layer
        vectorLayer = view.CreateLayer(vector)
      endforeach
      
      ;refresh display again
      e.refresh
    endif
  endif
end

Comments (0) Number of views (33) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

12

Jan

2017

New in IDL 8.6: IDLTasks

Author: Brian Griglak

ENVITasks have been around for a few years now, but they didn’t help those of you who only have IDL and not ENVI as well.  In IDL 8.6 we are happy to announce that the ENVITask concept has been carried over to work in pure IDL without ENVI.  Many of the features are the same, but there are slight differences that I want to point out in this post.

The first and most obvious is how you construct an IDLTask.  Rather than launch ENVI and then call the ENVITask()function, you can call the IDLTask()function at any point in time.  As the help points out, you can pass in either a scalar string that is the name of the task you want or its filename, or you can pass in a Hash object that is the task definition (what you would get if you called JSON_Parse() on a task file).  If you pass in a string that isn’t a filename (relative or absolute), then it is treated as a task name.  The function will then look at !PATH and search for all .task files in each folder listed there.  It catalogs all the .task files it finds, but if there are multiple folders with the same base name only the first is recognized(just like how IDL handles multiple .pro or .sav files with the same base name found in !PATH).  The list of .task files is filtered to those with the correct IDLTask schema, which currently is only “idltask_1.0”.  This way we don’t accidentally pick up an ENVITask files and cause confusion.  If a.task file with the same base name as the requested task name is found, it is used as the task definition.  If no exact match is found, but partial matches exist, then helpful error messages are returned telling you about the name(s) that partially match, so you can correct your code.  I should point out that the current working directory (which can be retrieved by calling CD with the CURRENT keyword) is searched before any of the folders in !PATH, so that can affect the behavior of IDLTask().

The “idltask_1.0” task schema used for IDLTasks in IDL 8.6 is very similar to the “envitask_3.0” schema used by ENVITasks in ENVI 5.4.  The notable exception is that the TYPE property of your parameters won’t understand ENVI class types like ENVIRaster.  But all the basic datatypes available in IDL are supported by IDLTasks – strings, Booleans, and numbers, as well as List, Hash, OrderedHash, and Dictionary.

Another difference is how you interact with IDLTasks on GSF as opposed to ENVITasks.  The service endpoint for IDLTasks will be http://hostname:port/ese/services/IDL,while the ENVITasks use http://hostname:port/ese/services/ENVI.  The different endpoints are used to discriminate between the requests that should use the IDLTask() function vs the ENVITask() function to load the requested task.

Easy GSF deployment is one of the primary reasons you would want to build IDLTasks in the first place. If you have IDL functions or procedures that you are used to calling directly, then you are probably wondering why you would want to wrap them in an IDLTask.  As a C++ developer in a previous life, I appreciated the type safety that C++ requires, so I also appreciate the parameter validation that IDLTasks provide.  When developing your custom IDLTask, you will have to spend some time thinking about what the inputs and outputs are for your code, but once you do that you won’t need to worry about writing lots of input validation code, the IDLTask framework will take care of that for you.  The IDLTasks are also self-documenting like ENVITasks, so if someone else hands you a .task file and .sav file, you can load the task and then learn all about the parameter names, their types, cardinalities, and hopefully even descriptions. All of this information makes it possible to deploy your algorithms on GSF for running in the cloud, with all the same introspection capabilities over the REST endpoint.  Alternatively, you can set up batch processing using some sort of folder watch capability to spawn IDLTaskEngine instances to automatically run your code on each file that appears on your system.

 

Comments (0) Number of views (317) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

15

Dec

2016

Interactive Vector GUI

Author: Austin Coates

With holidays rapidly approaching, I thought it might be nice to present a light hearted demonstration of capabilities; something that you can delight friends and family with during cold winter nights.  This week’s post shows how to use a simple widget_window to create an interactive vector animation.  The code is broken up into four pieces VectorBlog, VectorBlog_Event, VectorBlogMouseEvent, and VectorBlogVectorBuilder. The VectorBlog creates the widget itself and initially displays the vector information.  VectorBlog_Event runs the events for the base widget.  This is mainly the closing of the widget. VectorBlogMouseEvent is the mechanism by which the mouse locations are recoded by the widget window. Last but not least VectorBlogVectorBuilder builds the two matrices containing the horizontal and vertical components of the vector.  When the following code is run the cursor location is recorded, a distance weighted vector diagram is created, and the resulting image is then displayed in the widget window.  The only tricky bits in this code relate to the fact that there are two event handlers, one for the base widget and one for the mouse events in the widget window.  The object reference for the image displayed must also be maintained and manipulated each time the cursor is move.

 

Figure 1: Example Vector Plot

 

;+
; Build the vecort components
; This is distance weighting
;-
pro VectorBlogVectorBuilder, dims = dims, pt = pt, u = u, v = v
  compile_opt IDL2

  ; Get all the x locations
  xs = findgen(dims[0])
  xs = xs#replicate(1,dims[1])

  ; Get all the y locations
  ys = findgen(dims[1])
  ys = transpose(ys# replicate(1,dims[0]))

  ; Build the horizontal components
  u = pt[0] - xs

  ; Build the vertical components
  v =  pt[1] - ys
end


;+
; The event handler called when the mouse is moved.
;-
FUNCTION VectorBlogMouseEvent, win, x, y, keymods
  COMPILE_OPT idl2

  ; Set the window
  win.SetCurrent

  ; Set the scale factor
  sc = .05

  ; Get the size of the window
  dims = win.DIMENSIONS * sc

  ; Get the vecotr information
  VectorBlogVectorBuilder, dims = dims, pt = [x*sc,y*sc], $
    u = u , v = v

  ; Build the vecotr image
  vector = vector(u,v,xrange=[0,dims[0]], yrange=[0,dims[1]], /current, AUTO_COLOR=1,  $
    RGB_TABLE=73, AXIS_STYLE = 0, LENGTH_SCALE=1.5, ARROW_THICK =3)

  ; Delete the old image
  win.uvalue.delete

  ; Display the new image
  win.uvalue = vector



END


;+
; The event handler for the widget
;-
pro VectorBlog_Event, ev
  compile_opt IDL2


end


pro VectorBlog
  compile_opt IDL2

  ; Setup the widget
  tlb = widget_base()
  wwindow = widget_window(tlb, MOUSE_MOTION_HANDLER='VectorBlogMouseEvent', $
    XSIZE=500, YSIZE=500)
  widget_control, tlb, /REALIZE

  ; Set the scale factor
  sc = .05

  ; Get infomation about the widget window
  WIDGET_CONTROL, wwindow, GET_VALUE=w
  dims = w.DIMENSIONS * sc

  ; Build the vecotr infomation
  VectorBlogVectorBuilder, dims = dims, pt = [(dims[0])/2.,(dims[1])/2.], $
    u = u , v = v

  ; Display the vectors
  vector = vector(u,v,xrange=[0,dims[0]], yrange=[0,dims[1]], /current, AUTO_COLOR=1,  $
    RGB_TABLE=73, AXIS_STYLE = 0, LENGTH_SCALE=1.5, ARROW_THICK =3)

  ; Record the vector object
  w.uvalue = vector
  WIDGET_CONTROL, tlb, SET_UVALUE=w

  ; Start the task manager
  XMANAGER, 'VectorBlog', tlb

end

Comments (1) Number of views (430) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

7

Dec

2016

Maintaining Backward Compatibility in IDL 8.6 - Keep Calm And Read Your Release Notes

Author: Jim Pendleton

One of the "benefits" of being a Harris Geospatial Solutions insider is access to pre-release candidates of the commercial products developed by our engineering group.

We in Custom Software Solutions are sometimes the canaries in the coal mine, learning on occasion that an "undocumented feature" that we had used benignly or even to our advantage has been removed from the language in a newer release. Generally, these changes are justifiable. 

Where to Find Release Notes

Sometimes it's not enough to look at the documentation center's What's New help page to learn of all these changes. An additional source of information, generally prepared after the "What's New" documentation has gone to press, is located in a place other than your IDL or ENVI installation's documentation subdirectory.

This file includes information about supported platforms and potential backward-compatibility issues.

If you have received a Harris product installation DVD, check the info subdirectory on the DVD itself for the release notes files.

If your installation was downloaded from the Harris Geospatial Solutions Download and License Center, you or your site's designated license administrator will need to retrieve the release notes from a link that is separate from the product installer. 

Downloading Release Notes

After logging in, select the "Harris" link near the bottom of the web page, under "Browse My Software and Documentation".

Select the "IDL" link, that pops up in a new list in the "Product Lines" column.

Select the "IDL" link in the "Current Releases" tab.

On the "Product Download" page, select the appropriate item for the Release Notes document.

An Example Backward Compatibility Note

Recently, I discovered some of my routines were using an admittedly illegal syntax involving the "_REF_EXTRA" keyword passing mechanism.  The code wasn't so much illegal, as it was ignored. And one could argue the compiler should have complained about it from the time that _REF_EXTRA was added to the language. For example,

function MyRoutine, _REF_EXTRA=extra

    MySubroutine, _REF_EXTRA=extra

end

Can you spot the problem?

The _REF_EXTRA keyword is only intended to appear in the declaration of a function or procedure.Within the body of the code, you should always use the _EXTRA keyword when making calls to other routines.

In the form shown above, the code is basically ignored at execution time.  It serves no useful purpose.

Up until IDL 8.6, illegal use of the _REF_EXTRA syntax would simply be skipped by the compiler and interpreter. In the example above, MySubroutine would be called without any keywords, regardless of what was passed to MyRoutine.

In IDL 8.6, the compiler has been beefed up to complain about the invalid syntax. If you have code that fits this pattern your code will not compile. You may want to simply remove the flagged code because it has never been operational. Or you may want to change the syntax.

function MyRoutine, _REF_EXTRA=extra

    MySubroutine, _EXTRA=extra

end

Carefully consider the implications of changing the keyword, however. Modifying the syntax will also alter the behavior. You may end up modifying keywords on output that you hadn't intended to change!


Comments (0) Number of views (677) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

1

Dec

2016

Optimizing Max Kernel Operation in IDL

Author: Atle Borsholm

I found this optimization question on the comp.lang.idl-pvwave, and decided to give it a try. The question was to implement an algorithm that replaces every element in a 2-D array with its neighborhood maximum value. In this case the neighborhood size was 101x101 (i.e. -50 to +50), and the array size was 3200x3248. The nested FOR loop approach looks like the following code snippet.

  data = randomu(seed,3200,3248)
  dim = size(data,/dimension)
  nx = dim[0]
  ny = dim[1]
 
  t0 = tic('Nested FOR')
  result2 = data
  for i=0,nx-1 do begin
    for j=0,ny-1 do begin
      result2[i,j] = max(data[(i-50)>0:(i+50)<(nx-1),(j-50)>0:(j+50)<(ny-1)])
    endfor
  endfor
  toc,t0
 

My first thought was to use the > operator which returns the maximum of 2 arguments. It operates on arrays, and in conjunction with the SHIFT function it serves to return the larger of 2 neighbors. The other trick here is that since we are looking for a 101x101 neighborhood maximum, we can use a combination of smaller neighborhood maxima as input in a structured way in order to achieve the exact 101x101 neighborhood size. The code that I ended up with after some trial and error was the following.

  t0 = tic('Iterative >')
  ; Using SHIFT and > in an iterative way
  padded = replicate(min(data),size(data,/dimension)+100)
  padded[50,50] = data
  tmp3 = shift(padded,1,0) > padded > shift(padded,-1,0)
  tmp9 = shift(tmp3,3,0) > tmp3 > shift(tmp3,-3,0)
  tmp27 = shift(tmp9,9,0) > tmp9 > shift(tmp9,-9,0)
  tmp81 = shift(tmp27,27,0) > tmp27 > shift(temporary(tmp27),-27,0)
  tmp99 = shift(tmp9,44,0) > temporary(tmp81) > shift(temporary(tmp9),-44,0)
  tmp101 = shift(tmp3,49,0) > temporary(tmp99) > shift(temporary(tmp3),-49,0)
 
  ; Same for Y-dim
  tmp3 = shift(tmp101,0,1) > tmp101 > shift(temporary(tmp101),0,-1)
  tmp9 = shift(tmp3,0,3) > tmp3 > shift(tmp3,0,-3)
  tmp27 = shift(tmp9,0,9) > tmp9 > shift(tmp9,0,-9)
  tmp81 = shift(tmp27,0,27) > tmp27 > shift(temporary(tmp27),0,-27)
  tmp99 = shift(tmp9,0,44) > temporary(tmp81) > shift(temporary(tmp9),0,-44)
  tmp101 = shift(tmp3,0,49) > temporary(tmp99) > shift(temporary(tmp3),0,-49)
 
  result1 = (temporary(tmp101))[50:50+nx-1,50:50+ny-1]
  toc,t0
 

I didn’t say that optimized code always looks pretty, but the goal here is to run fast. Adding in some result comparison checking to make sure the results are equivalent.

 
  print, array_equal(result1,result2) ? 'Results are matching' : 'SOMETHING went wrong'
 

Finally, here are the results, which yielded an impressive amount of speed-up, the execution time went from 189.4 seconds to 1.4 seconds, and the results are identical:

  % Time elapsed Nested FOR: 189.39470 seconds.
  % Time elapsed Iterative >: 1.4241931 seconds.
  Results are matching
 
 
 

Comments (0) Number of views (616) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

12345678910 Last

MONTHLY ARCHIVE

MOST POPULAR POSTS

AUTHORS

Authors

Authors

Authors

Authors

Authors

Authors

Authors

Authors

Authors

Authors

GUEST AUTHORS

Authors

Authors

Authors

Authors

Authors

Authors

Authors



© 2017 Exelis Visual Information Solutions, Inc., a subsidiary of Harris Corporation