Coming Soon in ENVI 5.4: ENVITask Style Sheets

Author: Brian Griglak

Since the early days of ENVITasks in ENVI 5.2, there has been what we call the Dynamic UI.  Some of the toolbox items that wrap ENVITasks use it, but you can use it at any time to set some or all of the parameters of your task using the ENVIUI::SelectTaskParameters() method.  When you call this method, it will inspect the task to get the NAME, TYPE, and VALUE properties of each parameter, and construct a dialog with widgets for each.  If the VALUE is !NULL, then the widget will be empty, but if it has a valid value the widget will display that.  We have a set of heuristics that try to select the best widget to use for each TYPE, but it may not choose the widget you desire for a particular parameter.

To address this, we are introducing task style sheets in the upcoming release of ENVI 5.4.  The idea is borrowed from HTML and CSS, which separate content from presentation.  In the ENVITask context, the ENVITask template defines the content, and the style sheet defines the presentation logic.  These style sheets are nowhere as sophisticated as CSS can be, starting with the lack of cascading, but they don’t need to be for our purposes.  The style sheet is a JSON file or in-memory Hash that allows you to hide a couple of the widgets at the bottom of the dialog, and specify which widget to use for each parameter.  You don’t need to specify every parameter, those that are not in the style sheet will fall back to the heuristics.

Let’s make this concrete using the example from the help page:

  e = ENVI()
  Task = ENVITask('NNDiffusePanSharpening')
  ret = e.UI.SelectTaskParameters(Task)

The resulting pop-up dialog looks like this:

The “Pixel Size Ratio” parameter has to be an integer, so we used the edit widget with the increment/decrement buttons next to it.  But maybe you want to use a wheel instead; this is where a style sheet comes into play.

I can change the “Pixel Size Ratio” parameter to use a wheel widget, like  this:

  style = Hash("schema", "envitask_3.0", $
               "parameters", List(Hash("name", "pixel_size_ratio", $
                                       "type", "IDLWheel_UI")))
  ret = e.UI.SelectTaskParameters(Task, STYLE_SHEET=style)

One key point is that the “parameters” value must be a List of Hashes, even for a single parameter.  The equivalent JSON file is this:

    "schema": "envitask_3.0",
    "parameters": [
            "name": "pixel_size_ratio",
            "type": "IDLWheel_UI"

The dialog now looks like this:

All of the possible values for “type” are well documented, under the heading “User Interface Elements”.  Some of the widgets allow you to specify extra properties to customize the behavior.  The IDLWheel_UI class, for example, supports specification of MIN and MAX properties.  If the parameter has these properties set, they will be used, but as you can see in the case of NNDifusePanSharpening it doesn’t.  But we can specify them in the style sheet if we want to add them.  We do this by putting all these properties into a Hash under the key “keywords”:

  style = Hash("schema", "envitask_3.0", $
               "parameters", List(Hash("name", "pixel_size_ratio", $
                                       "type", "IDLWheel_UI", $
                                       "keywords", Hash("min", 1, $
                                                        "max", 9))))
  ret = e.UI.SelectTaskParameters(Task, STYLE_SHEET=style)

The equivalent JSON file is this:

    "schema": "envitask_3.0",
    "parameters": [
            "name": "pixel_size_ratio",
            "type": "IDLWheel_UI",
            "keywords": {
                "max": 9,
                "min": 1

The dialog now looks like this:

In all the examples, I have manually specified the STYLE_SHEET keyword in the call to SelectTaskParameters().  If you want the style sheet to be automatically used any time you call this method with your task, then create the JSON file named <task name>.style and put it in the same folder as the .task file.

Comments (0) Number of views (131) Article rating: 5.0

Categories: IDL Blog | IDL Data Point





App Update Check - Coming soon in IDL 8.6 and ENVI 5.4

Author: Jim Pendleton

First, a dose of nostalgia.

In the early days of commercial IDL, new drops of the application occurred when new features were added and as bugs were fixed for important customers. It was not unusual for there to be a couple different releases per month, a release cycle concept not dissimilar to that of most of today's smart phone apps.

The new releases were signaled through an update to a README-type of text file that was available on a node on a computer network called DECNET, shared by many universities and national labs in the days before the internet as we know it today.

At the organization where I worked, I had a nightly batch job that would alert me to the availability of a new version of IDL. It was a highlight of my week when new features appeared in IDL. Yes, I had that sort of a life back then. Frequent updates, from a customer point of view, kept the buzz alive with the products.  

Here we are about 30 years later and this functionality to check for updates is now built into the upcoming releases of both IDL 8.6 and ENVI 5.4. By default, IDL will check once a week automatically for updates when a new session is started.

Additionally, with the click of a single "Help/Check for Updates..." menu button in the IDL Workbench or from the ENVI UI, you can ensure that you, too, are enjoying the freshest and best we have to offer.

If your version is current, you will receive a dialog assuring you all is well.

To check that your ENVI version is current, find the menu item in the main application user interface.

You will need an internet connection to the commercial world for the status check to complete successfully.

Extra "80s-throwback" credit if you use a packet sniffer to find the syntax of the request and write a script that will automatically launch a WGET query each time you start a new IDL session.

Does this new feature signal a return to more frequent application updates? Have we gone back to the future? Only time will tell.

Comments (0) Number of views (512) Article rating: No rating

Categories: IDL Blog | IDL Data Point





Upsampling images using a Lagrange polynomial method

Author: Atle Borsholm

A few weeks ago I posted about using the Lanczos kernel for resampling images to a higher resolution. This week I am continuing with the same example, but adding in the Lagrange resampling method. Both Lagrange and Lanczos have some similar characteristics in that they show better detail than a purely linear interpolation. Both methods can also be adapted to an irregularly gridded dataset instead of the raster images used in my examples here. The code produces 4 upsampled images using different methods, and the results are shown below.
function lanczos, data
  xval = [-3:3:.25]
  lanc3 = 3*sin(!pi*xval)*(sin(!pi*xval/3d)/!pi/!pi/xval/xval)
  lanc3[where(xval eq 0)] = 1
  l2d = lanc3 # lanc3
  ; high resolution version
  msk = fltarr(data.dim*4)
  msk[0:*:4,0:*:4] = data
  hi = convol(msk, l2d, /edge_trunc)
  hi = byte(round(hi>0<255))
  return, hi
function lagrange, a, x, y
  compile_opt idl2, logical_predicate
  xf = floor(x)
  yf = floor(y)
  x1 = x - xf
  y1 = y - yf
  off = [-1,0,1,2]
  retval = replicate(0., size(x, /DIM))
  weightx = replicate(1., [size(x1, /DIM),4])
  weighty = replicate(1., [size(x1, /DIM),4])
  for i=0,3 do begin
    for j=0,3 do begin
      if i ne j then begin
        weightx[*,*,i] *= (x1 - off[j]) / (off[i] - off[j])
        weighty[*,*,i] *= (y1 - off[j]) / (off[i] - off[j])
  for i=0,3 do begin
    for j=0,3 do begin
      retval += weightx[*,*,j] * weighty[*,*,i] * a[xf+off[j], yf+off[i]]
  return, retval
pro upsample_example
  compile_opt idl2,logical_predicate
  ; Read the original image data
  f = filepath('moon_landing.png', subdir=['examples','data'])
  data = read_png(f)
  dim = data.dim
  window, xsize=dim[0], ysize=dim[1], 0, title='Original full size'
  tv, data
  ; Define a zoomed in are on the flag.
  xs = 120
  ys = 105
  dx = 60
  dy = 100
  ; display upsampled 4x with pixel replication
  window, xsize=4*dx, ysize=4*dy, 1, title='CONGRID pixel-replication'
  tv, congrid(data[xs:xs+dx-1,ys:ys+dy-1],4*dx,4*dy)
  ; display upsampled 4x with bilinear interpretation
  window, xsize=4*dx, ysize=4*dy, 2, title='CONGRID linear'
  tv, congrid(data[xs:xs+dx-1,ys:ys+dy-1],4*dx,4*dy,/interp)
  ; display upsampled 4x with Lanczos convolution
  window, xsize=4*dx, ysize=4*dy, 3, title='Lanczos'
  tv, (lanczos(data))[xs*4:xs*4+dx*4-1,ys*4:ys*4+dy*4-1]
  ; Lagrange
  window, xsize=4*dx, ysize=4*dy, 4, title='Lagrange'
  xcoord = [float(xs):xs+dx:0.25]
  ycoord = [float(ys):ys+dy:0.25]
  tv, byte(lagrange(float(data), $
    xcoord # replicate(1,1,ycoord.length), $
    replicate(1,xcoord.length) # ycoord)>0<255)
 Pixel replication


Bi-linear interpolation

Lanczos resampling

Lagrange resampling

Comments (0) Number of views (319) Article rating: 4.0

Categories: IDL Blog | IDL Data Point





Minimum Area Bounding Box

Author: Austin Coates

I find myself drawing bounding boxes around things a lot. I don’t know why I do it so much, but for whatever reason I do, and as of late I wanted to up my bounding box game. In the past, I have simply used the global min and max in both the x and y directions to get the coordinates to form the bounding box; however, this is not always the most elegant solution. For example, when my data follows somewhat of a linear trend, I am left with ample white space not filled by any valuable information

Figure 1: Simple Bounding Box

Figure 2: Minimum Area Bounding Box

This got me thinking, why am I not simply drawing a bounding box around only the data? Sounds great, right? The only problem was I had no idea how to do this. Luckily, there is this thing called the internet and it has vast stores of information and ideas to pull from. I found a very elegant solution by Jesse Buesking on stackoverflow.com which was posted on November 9, 2015. The solution was written in Python which I converted to IDL. My goal in posting this is to show an awesome way to draw a bounding box and also an example of translating between IDL and Python.

function bounding_box, pts = pts, plot_results = plot_results

 compile_opt IDL2

;Get the x and y coordinates

xs = pts[0,*]

ys = pts[1,*]


;Find the bounding points

Triangulate, xs, ys, triangles, hull, CONNECTIVITY=CONNECTIVITY

;order hull points in a [2,n] array   

 hull_points = [[xs[hull]]##1,[ys[hull]]##1]

;calculate edge angles

edges = hull_points[*,1:-1] - hull_points[*,0:-2]

angles = atan(edges[1,*], edges[0,*])

pi2 = !DPI/2.


angles = abs(angles - floor(angles / pi2) * pi2)

angles = angles[sort(angles)]

angles = angles[UNIQ(angles)]

;find rotation matrices 

rotations = transpose([[cos(angles)],[cos(angles-pi2)],[cos(angles+pi2)],[cos(angles)]])

rotations = REFORM(rotations, [2,2,n_elements(angles)])


;apply rotations to the hull 

rot_points = fltarr( n_elements(hull_points)/2, 2, n_elements(angles))

size_rot = size(rotations)

for group = 0 , size_rot[3]-1 do begin   

for row = 0 , size_rot[2]-1 do begin

rot_points[*,row,group] = TRANSPOSE(rotations[*,row,group]) # hull_points



;find the bounding points

min_x min(rot_points[*,0,*],DIMENSION=1, /NAN)

max_x max(rot_points[*,0,*],DIMENSION=1, /NAN)

min_y min(rot_points[*,1,*],DIMENSION=1, /NAN)

max_y max(rot_points[*,1,*],DIMENSION=1, /NAN)

;find the box with the best area

areas = (max_x - min_x) * (max_y - min_y)

min_val = min(areas, best_idx)

;return the best box

x1 = max_x[best_idx]

x2 = min_x[best_idx]

y1 = max_y[best_idx]

y2 = min_y[best_idx]

r = rotations[*,*,best_idx]

rval = fltarr(2,4)

rval[*,0] = TRANSPOSE(TRANSPOSE([x1, y2]) # transpose(r))

rval[*,1] = TRANSPOSE(TRANSPOSE([x2, y2]) # transpose(r))

rval[*,2] = TRANSPOSE(TRANSPOSE([x2, y1]) # transpose(r))

rval[*,3] = TRANSPOSE(TRANSPOSE([x1, y1]) # transpose(r))


;display results 

if KEYWORD_SET(plot_results) then begin


XRANGE=[min(rval[0,*])-1,max(rval[0,*])+1], YRANGE=[min(rval[1,*])-1,max(rval[1,*])+1])


FILL_COLOR="light steel blue", PATTERN_ORIENTATION=45, $



return, rval


Source of original Python code : http://stackoverflow.com/questions/13542855/python-help-to-implement-an-algorithm-to-find-the-minimum-area-rectangle-for-gi/33619018#33619018 

Comments (0) Number of views (391) Article rating: No rating

Categories: IDL Blog | IDL Data Point





Compressing hyperspectral data using Motion JPEG2000

Author: Brian Griglak

As I’ve blogged about before, you have to pay attention to a raster’s interleave so that you can iterate over the data efficiently.  That post was ENVI specific, this time I’m here to present a way to store a data cube for efficient BIL mode access that is pure IDL.

As more hyperspectral (HSI) sensors are produced, data cubes with hundreds or even thousands of bands will become more common.  These sensors are usually pushbroom scanners, with a fixed number of samples and bands and a highly variable number of lines collected.  The collection modality lends itself to the BIP and BIL storage interleaves, so each scanline of pixels can be written to disk without having to use large amounts of memory to hold many lines for BSQ or tiled output.  If you are using HSI data, you are probably also interested in most if not all of the bands for your analysis, so BIP and BIL interleaves are more efficient for those data access patterns.

The challenge of HSI data is that it can be large, due to the high number of bands.  Some form of data compression would be useful, as long as it is lossless – why spend the money collecting all those bands if you are going to lose the details with compression or interpolation artifacts?

There a few options for compressing a datacube in IDL:

The zip option is not attractive in that you have to unzip the entire file to use it, so there isn’t any space savings, in fact it’s worse than using the original ENVI file.  The IDL support for the TIFF format allows for 4 different compression modes:
  • None
  • LZW
  • PackBits
  • JPEG

The None option is pointless, since it doesn’t perform any compression.  The JPEG option is lossy and would introduce artifacts, but it can’t be used for this data in the first place.  Firstly, it only supports 8-bit values, so any other datatypes will fail.  Secondly, if you send more than 3 bands of data into the method, you get back the curious error message “Bogus input colorspace”.  This message actually comes from the libTIFF library, trying to tell you it can’t JPEG compress more than 3 band data.

The IDLffMJPEG2000 object is IDL’s interface to the Motion JPEG2000 compression format.  The Motion JPEG2000 standard is an extension to JPEG2000, the wavelet based successor to the venerable JPEG format.  The JPEG2000 format supports both lossy and lossless compression, and we’ll use the latter here to avoid introducing any artifacts to the data.  Unlike make video formats, Motion JPEG2000 only performs intraframe compression, no interframe compression.  While this likely results in lower compression ratios than those other formats, it also means that decompressing a single frame of the animation is faster because it doesn’t need to load other frames for the interframe interpolation.  There are a few degrees of freedom available when creating Motion JPEG2000 files – BIT_RATE, N_LAYERS, N_LEVELS, PROGRESSION, and TILE_DIMENSIONS.  I did not experiment with how changing these values effects the compression ratio or time needed to decompress, so playing with these one your own may be worthwhile.

First I’ll present the code I used to perform the creation of the different compressed formats , and then show the compression ratios and time to compress for a couple different datacubes.

pro compress_to_zip, raster, file   compile_opt idl2
  FILE_GZIP, raster.URI, file
pro compress_to_tiff, raster, file
  compile_opt idl2
  dat = raster.GetData()
pro compress_to_mj2k, raster, file
  compile_opt idl2
  dat = raster.GetData()
  signed = !True
  if (raster.Data_Type eq 'byte') then begin
    depth = 8
  endif else if (raster.Data_Type eq 'int') then begin
    depth = 16
  endif else if (raster.Data_Type eq 'long') then begin
    depth = 24
  endif else if (raster.Data_Type eq 'uint') then begin
    depth = 16
    signed = !False
  endif else if (raster.Data_Type eq 'ulong') then begin
    depth = 24
    signed = !False
  endif else begin
    Message, 'Unsupported pixel datatype : ' + raster.Data_Type
  mj2k = IDLffMJPEG2000(file, /WRITE, /REVERSIBLE, $
                        BIT_DEPTH=depth, SIGNED=signed, $
                        DIMENSIONS=[raster.nColumns, raster.nBands])
  for row = 0, raster.nRows-1 do begin
    !null = mj2k.SetData(Reform(dat[*, *, row]))
  print, mj2k.Commit(30000)


The IDLffMJPEG2000 class requires you to add data one frame at a time (or subset of a frame), hence the for loop over the number of rows in the raster.  I did have to stick a Reform() call in there, to collapse the 3D slice down to 2D, since the last dimension is always 1.  This object is actually threaded, so the SetData() calls don’t block, they queue up the data to compressed into the output file, so the Commit() function has an optional timeout argument, which is how many milliseconds to wait before returning.  In my testing it doesn’t take much time at all to do the commit, the processor can keep up with the compression requests.

Now the interesting part, compression performance:

Sensor Rows Size GZIP ZIP TIFF LZW TIFF PackBits MJP2
254 cols
242 bands
7.2 sec
10.6 sec
73.7 %
4.8 sec
82.7 %
2.8 sec
39.9 %
13.2 sec
614 cols
224 bands
75.7 %
15.2 sec
75.7 %
24.0 sec
96.5 %
7.8 sec
98.5 %
3.8 %
60.9 %
952 cols
224 bands
48.0 %
60.5 sec
48.0 %
92.3 sec
62.1 %
44.8 sec
65.9 %
25.4 sec
29.8 %
109.1 sec


It isn’t surprising that the GZIP and ZIP results are almost identical, though GZIP is way faster.  It also isn’t surprising that the LZW mode for TIFF is more efficient than the PackBits mode, though slower.  It is somewhat surprising that the LZW compression was so much work than GZIP and ZIP, and that Motion JPEG2000 is a bit better than GZIP and ZIP.

Now let’s say you wanted to read one of the Motion JPEG2000 compressed files back, but only need a spatial and/or spectral subset of it.  Here is a function that will do just that:

function read_mj2k, file, SUB_RECT=subRect, BANDS=bands
  compile_opt idl2
  mj2k = IDLffMJPEG2000(file)
  mj2k.GetProperty, DIMENSIONS=dims, N_FRAMES=nFrames, $
                    BIT_DEPTH=depth, SIGNED=signed
  if (~ISA(subRect)) then begin
    subRect = [0, 0, dims[0]-1, nFrames-1]
  if (~ISA(bands)) then begin
    bands = INDGEN(dims[1])
  startFrame = subRect[1]
  endFrame = subRect[3]
  leftCol = subRect[0]
  rightCol = subRect[2]
  topRow = MIN(bands)
  bottomRow = MAX(bands)
  if (depth le 8) then begin
    type = 1 ; Byte
  endif else if (depth le 16) then begin
    type = signed ? 2 : 12 ; Int or UInt
  endif else begin
    type = signed ? 3 : 13 ; Long or ULong
  outData = MAKE_ARRAY(rightCol-leftCol+1, N_ELEMENTS(bands), $
                       endFrame-startFrame+1, TYPE=type)
  region = [leftCol, 0, rightCol-leftCol+1, dims[1]]
  for frame = 0, endFrame-startFrame do begin
    dat = mj2k.GetData(frame+startFrame, REGION=region)
    outData[*, *, frame] = dat[*, bands]
  return, outData


I used the same keyword interpretation as ENVIRaster::GetData():

  • SUB_RECT = [x1, y1, x2, y2]
  • BANDS – any array of nonrepeating band indices

Comments (0) Number of views (329) Article rating: No rating

Categories: IDL Blog | IDL Data Point


12345678910 Last




© 2016 Exelis Visual Information Solutions, Inc., a subsidiary of Harris Corporation