2

Jun

2016

The Perks of Being a Beta Tester

Author: Barrett Sather

As part of the Commercial Services Group here at Harris, I have a unique opportunity of using our latest and greatest tech, often before it even goes public. (Making me a bit of a beta tester sometimes.) Over the past couple months, I’ve had the pleasure of getting to work with some of our newest tech, and it’s very exciting. The two pieces of technology I’ve had my hands on are our Geospatial Framework (GSF), and our machine learning tool known as MEGA.

The Geospatial Framework has given me a lot of power in setting up large amounts of processing either locally, or on a server. It allows for clustering, many different processing engines, and even lets you hook it into custom websites. From what I’ve seen, it’s very flexible and powerful tech.

MEGA, which I’m also extremely excited about, is a new way to tackle feature extraction problems. Though I don’t understand the underlying development of artificial intelligence, I have had hands on experience training this deep learning system. While the code is complex, the idea is not – first, feed the system examples of what you are looking for in imagery. Once the system is trained, the software will tell you locations and confidences for where those objects of interest are.

Image: http://stats.stackexchange.com/questions/tagged/deep-learning

Using these two technologies together, I’ve been classifying large amounts of image data quickly through a deep learning system, and the sky really is the limit. With GSF, the ability to scale up just depends on how much hardware you can allocate to a task.

The more I use and learn about these technologies, the more excited I become about them. The best part is that they are very new, so they have room to become more and more powerful and robust. Can’t wait!!

Comments (0) Number of views (1687) Article rating: 4.0

Categories: ENVI Blog | Imagery Speaks

Tags:

2

Jun

2016

Now you can write your own static variable methods

Author: Benjamin Foreback

 Static methods and properties have been in IDL for a while, and they have certainly become useful at times. They serve as a convenient, modern alternative to traditional routine-based IDL calls. You may have created your own class with static methods, but did you know you can add your own methods to IDL's classes? This is most useful with IDL's static methods on variables.

Is a variable greater than 10?

Let's try a basic example. I have an IDL variable, and I'd like to know whether that variable is greater than 10. I'll start with writing a new method on IDL_Variable, which I will call ::IsGreaterThanTen

function IDL_Variable::IsGreaterThanTen, var

Although this method will take no arguments when called statically from a variable, the method will contain one argument, which is the input variable, in this case var.

result = variable.IsGreaterThanTen()

As the name suggests, this method will need to be defined as static. This is done in the compile_opt statement. As always, it is advisable to use "idl2" in the compile_opt statement as well, and if you want the method to appear built-in, then also use "hidden."

The rest of this method is straightforward. First check that the variable is indeed a number, and then perform the inequality, returning the result.

function IDL_Variable::IsGreaterThanTen, var
  compile_opt idl2, hidden, static
  
  ; If it is not a number, then it is not greater than ten.
  if ~Isa(var, /NUMBER) then begin
    return, 0b
  endif
  
  return, var gt 10
  
end

State or province?

Here is a more useful example. A while back, I needed a way to determine if a given string was a valid abbreviation for a U.S. state or Canadian Province. This is a time where writing a new static method on IDL_String would be convenient.

function IDL_String::IsaStateOrProvince, str
  compile_opt idl2, hidden, static
  
  abbrevList = ['AL', 'AK', 'AZ', 'AR', 'CA', 'CO', 'CT', 'DE', 'DC', 'FL', $
    'GA', 'HI', 'ID', 'IL', 'IN', 'IA', 'KS', 'KY', 'LA', 'ME', 'MD', 'MA', $
    'MI', 'MN', 'MS', 'MO', 'MT', 'NE', 'NV', 'NH', 'NJ', 'NM', 'NY', 'NC', $
    'ND', 'OH', 'OK', 'OR', 'PA', 'RI', 'SC', 'SD', 'TN', 'TX', 'UT', 'VT', $
    'VA', 'WA', 'WV', 'WI', 'WY', 'AB', 'BC', 'MB', 'NB', 'NL', 'NT', 'NS', $
    'NU', 'ON', 'PE', 'QC', 'SK', 'YT']

  !null = Where(abbrevList.Matches(str), count)
  return, count gt 0
  
end

(Note that technically some of these are territories and not provinces. DC is in this list, too, but this is the functionality I desire.)

Benefits

Now I can add this to my personal code library. I prefer this to a library of many different, possibly unrelated routines because it is a sensible way to bundle static method code into the same file (i.e. keep code related to IDL_String in the same file). This makes it easier for me to find and keep track of the code in the future. Additionally, I find it more intuitive to call

result = state.IsaStateOrProvince()

instead of a traditional routine call of

result = IsaStateOrProvince(state)

I think it might be because the static way of calling reads left-to-right.

This concept is the topic of this blog I wrote a couple of years ago.

Also, moving the routine into a specific class frees up the main namespace. For example I could write a different IsaStateOrProvince routine that does something different, and similarly, I could write an ::IsaStateOrProvince method on a different class. Neither of these would not interfere with the example above.

Drawbacks

One minor drawback with writing a routine as a static method is that you can’t call the method directly on a value that isn’t a variable, meaning you can't call

result = 'NY'.IsaStateOrProvince()

because you would get a syntax error. However, if you simply add "IDL_String" plus the "dot syntax" in front, then it would work:

result = IDL_String.IsaStateOrProvince('NY')

This is different than overloading a class. Overloading a class involves writing a new subclass that inherits the class and then overloads the inherited class's methods and/or adds new methods. Some of IDL's classes, such as IDL_Object, serve the purpose of being used as a superclass for user-written classes. Although IDL variable classes aren't specifically intended to be overloaded, you can overload any class you'd like.

A separate blog post is dedicated to overloading IDL's built-in classes, which provides an example of overloading IDL's LIST.

Warning! Do not save this into a file called IDL_String__Define or IDL_Variable__Define or any other static classname. Instead, you will need to place this into a differently named file that serves as a library file. Therefore, you may need to compile the file using RESOLVE_ROUTINE or restore it from a savefile at the start of your application. In other words, treat it as a library file that contains multiple routines inside of it.

As an example, I have saved my IDL_String::IsaStateOrProvince method into a file called "bens_strings.pro" which will probably never conflict with other filenames in IDL.

Additional disclaimer: It is important to keep in mind that you are still sharing namespace with IDL, and IDL’s static methods should never be clobbered by user-written ones. You need to be very careful when naming new methods to make sure that their names do not conflict with existing IDL methods. IDL may introduce new static variable methods in future releases, which may then conflict with something you’ve written. Therefore, it is important to check the “What's New” help topic when installing a new version of IDL.

This is, of course, true for any user-written code because you are always sharing namespace with IDL, but at a low-level like this, it is extra important.

I would probably never recommend trying to overload the::GetProperty method of any of these static classes in order to try to add your own static properties. Just stick with methods.


Edit: I have recently learned that it is also possible to put a method into a file called [classname]__[method_name].pro. For instance in the example above, I could write a file called idl_string__isastateorprovince.pro (two underscores between the classname and method name), and IDL will automatically compile it. This saves having to manually compile/restore a helper file.

Comments (1) Number of views (976) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

26

May

2016

Imagery of the Month – Fort McMurray Fire Roars On

Author: Joey Griebel

The Fort McMurray Fire is still making headlines and shows no signs of slowing down more than three weeks after it began. The devastating path of the fire has burned over 2018 square miles and destroyed some 2,400 homes and buildings. Wildfire Manager Chad Morrison is expecting weeks, if not months, of fighting the fire as it continues its path North crossing over from Alberta into Saskatchewan. Hot, dry weather conditions combined with only millimeters of moisture on the southern end of the fire haven’t helped containment efforts.

NASA continues to aid the firefighting effort by capturing a vast amount of Imagery over the fire from their Modis, Landsat 8, and Suomi NPP satellites (shown below). The sensors on the satellites allow penetration of the clouds and smoke and make it possible to see hotspots using thermal and IR bands. This helps those managing the firefighting efforts to deploy assets in areas where there are hotspots, regardless of the smoke of cloud coverage. The imagery captured also aides in the recovery efforts and allows the government to widely asses the areas burned and see what has survived the blaze.

Clear weather aligned with Suomi NPP’s flight over Fort McMurray on May 24th giving a spectacular view of the fire. The image shows the sheer size of the burn and the intensity at which it continues to burn.

(Credit Nasa/NOAA)

The Image above has been processed, making it easy to identify the burn scar, as well as see where the fire continues to roar near the North side of the scar. Having both thermal and visible imagery makes it possible to classify cloud, smoke, land, and then the active blaze itself.

Though the Suomi Image is my image of the month, there was one additional image from Joshua Stevens at NASA Earth Observatory from May 12th that shows another close perspective from Landsat 8 of the burn scar itself surrounding Fort McMurray and the fire burning to the South/Southwest.

(Credits: NASA EarthObservatory image by Joshua Stevens, using Landsat data from the U.S. Geological Survey)

The contrast of the burn scar and surrounding healthy vegetation shows the extent the fire has scorched this area. Hopefully, additional wildfire crews that have been approved, and cooperating weather, will help get the upper hand on the blaze in the coming weeks. Containing the fire is just the first step in getting the residents of Fort McMurray on the road to rebuilding their town. 

Comments (0) Number of views (2215) Article rating: 5.0

Categories: ENVI Blog | Imagery Speaks

Tags:

26

May

2016

How to read XML metadata from a TIFF/JPEG/DNG file with IDL

Author: Atle Borsholm

This is an example showing how to extract various image metadata contained in an XML packet inside of a DNG file. The use of XML packets to store metadata is especially popular with UAV captured image data. The XML packets are not specific to the raw DNG image format, but can also occur in JPEG and TIFF image files. The code example below will work for JPEG, TIFF as well as DNG that contain an XML packet. In this case, I implemented the parsing code using an object class and static methods. This approach is nice if you have multiple functions that you want contained in the same .pro file, but that you can still easily call from the command line. Other methods could be added to this parser. Here is the content of my “dng__define.pro” source file:

; Extracts XML metadata from DNG
;
function dng::XmlMetadata, filename
  compile_opt idl2,logical_predicate,static
 
  info = OrderedHash()
 
  exif = read_exif(filename)
 
  xml = IDLffXmlDomDocument()
  xml->Load, STRING=string(exif['Exif_Image_XMLPacket'])
 
  iter = xml->CreateNodeIterator(obj_new(), what_to_show=5)
  node = iter->NextNode()
  while obj_valid(node) do begin
    if isa(node,'IDLffXmlDomText') then begin
      key = (node->GetParentNode())->GetTagName()
      info[key] = node->GetData()
    endif
    node = iter->NextNode()
  endwhile
  return, info
end
 
; Class definition
pro dng__define
  s = { dng, inherits IDL_Object }
end

I tested this code on a random sample DNG file that I found here:
http://www.phantompilots.com/threads/hands-on-experience-phantom-3-4k-plus-inside-the-hood-pictures.38694/ But, the code example is not specific to this file. It shows how to parse any XML packet embedded in a TIFF, JPEG, or DNG file.
The listing is rather lengthy, so, I am only including the first few lines to show how the code can be called:

IDL> dng.XmlMetadata(filename)
{
    "x:xmpmeta": "\n",
    "rdf:RDF": "\n\n",
    "rdf:Description": "\n ",
    "crs:Version": "3.7",
    "crs:RawFileName": "L1004220.DNG",
    "crs:WhiteBalance": "As Shot",
    "crs:Temperature": "6700",
    "crs:Tint": "+27",
    "crs:Exposure": "+0.20",
    "crs:Shadows": "0",
    "crs:Brightness": "+50",
    "crs:Contrast": "+32",
...
 
  

Comments (0) Number of views (1291) Article rating: No rating

Categories: IDL Blog | IDL Data Point

Tags:

24

May

2016

Community Data Enrichment

Author: Michael Ehrlich

I love IMDB. I love being able to find movies that I am interested in, read a plot synopsis and then review the millions of reviews that the site houses. To me, those millions of reviews are where the value is, that is what keeps bringing me back to it. That value isn’t in its ability to aggregate widely available data, it is that they are taking this massive amount of data and then providing a platform for a community of highly interested users to enrich the experience for everyone. This enrichment comes through the reviewers sharing of a unique perspective, presenting ideas and interpretations of the movies through different optics than your own or simply telling you that Bruce Willis is a ghost in Sixth Sense and not marking **SPOILER ALERT** in the description.

*Image showing annotations in the Jagwire

I am writing about my love of IMDB not just because I love IMDB, but because the concept of data enrichment has been something that I have been a big advocate of within the Harris Geospatial Team for a while. With our Jagwire product, we had a great opportunity to lean forward and help drive the type of environment where we could enable this experience for our customers. 

Most Unmanned Platforms (or “drones” for our non-Defense customers) have metadata that is available out of the box. This metadata (which can also include data from the sensor) typically is focused on providing spatial and temporal context(“where” was it and “when” was it there). This information is critical but, like IMDB, the ability to add community enriched data, (those user reviews in IMDB) to the geospatial products in Jagwire provides an unbelievable amount of value.  Machine learning, automated image analytics and other technologies that help interpret objects, actions and even intent in data will continue to improve exponentially, but today nothing beats an analyst identifying an object as a “water tower” in an image or an engineer identifying damage to a utility pole after a storm. This enriched data is powerful. This type of data helps rapidly close the gap between “looking” for data and “taking action”.

The understanding of the value of this data is why Jagwire has so many entry points for our communities of users to share their understanding, expertise and perspective. Annotations, chat rooms and visual mark ups are a few examples of how Jagwire enables this type of interaction with the data. Over the next year we are going to continue to collaborate with our customers and partners in the Defense, Utility and Public Safety communities to expand in these areas within Jagwire. Our customers are continually challenging us to find ways to extract more value from their data and we are up to the challenge. 

Comments (0) Number of views (1885) Article rating: 5.0
First567891011121314Last

MONTHLY ARCHIVE

«March 2017»
SunMonTueWedThuFriSat
2627281234
567891011
12131415161718
19202122232425
2627282930311
2345678

MOST POPULAR POSTS

AUTHORS

Authors

Authors

Authors

Authors

Authors

Authors

Authors

Authors

Authors

Authors

GUEST AUTHORS

Authors

Authors

Authors

Authors

Authors

Authors

Authors



© 2017 Exelis Visual Information Solutions, Inc., a subsidiary of Harris Corporation