Welcome to the Harris Geospatial product documentation center. Here you will find reference guides, help documents, and product libraries.


Harris Geospatial / Docs Center / Geospatial Services Framework / GSF Tutorial: Custom Parameter Mapper

Geospatial Services Framework

GSF Tutorial: Custom Parameter Mapper

GSF - Tutorial - Custom Parameter Mapper

Parameter mappers are designed to convert input values before an engine sees them and output values before a client sees them. This allows a lot of flexibility in normalizing data for the right usage. Often this means providing a download URL to a client and a file path to an engine, but parameter mappers are fairly open ended.

Defining a custom parameter mapper can open up a lot of flexibility in data access for the server. For instance, if a data catalog had its own URL format, a parameter mapper would be a good way to add support for that data source. Information on using parameter mappers within clustered configurations is provided here.

This tutorial demonstrates how to map a folder to a URL, but that is just a starting point.

The ENVI Data Parameter Mapper


The envi-data-parameter-mapper allows clients to use files from the ENVI data folder. It maps them from the ese/data folder to the ENVI installation folder on the local disk. An example file in the data folder is qb_boulder_msi. This is an ENVI format file and is used in most of these examples as input.

Specifying this as input:

INPUT_RASTER: {
  url    : 'http://localhost:9191/ese/data/qb_boulder_msi',
  factory: 'URLRaster'
}

is then translated to the respective values below:

// Windows host
INPUT_RASTER: {
  url    : 'c:\Program Files\Harris\ENVI54\data\qb_boulder_msi',
  factory: 'URLRaster'
}
// Linux host
INPUT_RASTER: {
  url    : '/usr/local/harris/envi54/data',
  factory: 'URLRaster'
}

The translated paths are what an engine receives when it starts processing. This way, the conversion logic is handled by the server, and the engine does not need to know how to find the data via URL.

The envi-data-parameter-mapper is also configurable. See its help pages for the configuration options, which include changes to the destination folder or input URL.

Creating A Custom Parameter Mapper

Create a new folder under the server directory named custom-parameter-mapper. This folder will contain all the files created during this tutorial.

Create the following files:

  • package.json - Contains various metadata relevant to the parameter mapper.
  • config.json - Contains configuration options for the parameter mapper.
  • parameterMapper.js - Contains the javascript to handle the parameter mapper.

Note: As in other tutorials, the full example code blocks are provided here at the bottom of the page.

package.json

Create a file called package.json in the custom-parameter-mapper folder. Add the following contents to the file:

{
  "name": "custom-parameter-mapper",
  "version": "1.0.0",
  "description": "Custom parameter mapper",
  "main": "parameterMapper.js"
}

This file provides information about the parameter mapper and its dependencies, which in this case, are quite simple.

config.json

Create a file called config.json in the custom-parameter-mapper folder. Add the following contents to the file:

{
  "URLBase": "/customdata/",
  "folderPath": "C:\\customData\\"
}

The URLBase is the URL path to host the data. An example URL would look like http://localhost:9191/customdata/myFile.txt.

When the parameter mapper sees this, it maps the URL to the folder path.

parameterMapper.js

Create a file called parameterMapper.js in the custom-parameter-mapper folder.

Add the following javascript code to the parameterMapper.js file. This is all of the necessary code for the parameter mapper.

First, define

/**
 * Custom ENVI Parameter Mapper module.
 * This checks urls for the customdata path
 * and maps them to the specified folder
 */
var defaultConfig = require('./config.json');
var url = require('url');
var extend = require('util')._extend;
var path = require('path');

Next, create the CustomParameterMapper function template and define the exports for the module.

function CustomParameterMapper(rootConfig) {
    <!-- TODO fill me in -->
}
module.exports = CustomParameterMapper;

Within the function, first override the default configuration with server level options.

  var config = extend({}, defaultConfig);
  config = extend(config, rootConfig);

Then, stub functions to comply with the interface expectations. In this case, the init and reverseTranslate functions are trivial.

  this.init = function() {
    // Empty init function
  };
  // This parameter mapper doesn't support reverseTranslate
  this.reverseTranslate = function() {
    return false;
  };

The translate function, however, is non-trivial. Still within the CustomParameterMapper function, place the following code, which returns a boolean indicating if the parameter is translatable or not and callsback with results if the boolean is true.

  this.translate = function(name, value, callback) {
    var translated = false;
    var newValue;
    // Check for url
    try {
      var URLValue = url.parse(value);
      if (URLValue.pathname.startsWith(config.URLBase)) {
        newValue = config.folderPath + URLValue.pathname.substring(config.URLBase.length);
        var relative = path.relative(config.folderPath, newValue);
        var error;
        // Prevent navigating outside of the specified folder
        if(relative.startsWith('.')) {
          error = 'ERROR: value can not be outside of the mapped folder';
          newValue = null;
        }
        // Translate should be asynchronous.
        process.nextTick(function() {       
          callback(error, newValue);
        });
        translated = true;
      }
    } catch (e) {
      // Do nothing on error, we couldn't translate this one.
    }
    return translated;
  };

Configure The Server

The last step is to add the new custom-parameter-mapper to the list of parameter mappers.

Open the top-level server config.json file, and add the new parameter mapper to the parameterMappers array.

To do this from a command line, start a command prompt in the GSFxx directory and execute the following command:

node updateConfig.js config.json --arrayadd parameterMappers={\"type\":\"./custom-parameter-mapper\"}

*Note that calling this command multiple times will add multiple parameter mappers to the config file.

The updateConfig.js script will automatically back up the original config.json file for you.

You may also manually enable the custom-parameter-mapper by editing the config.json as shown below. It is highly recommended that you save a backup copy of the server-level config.json file before making any changes.

  "parameterMappers": [
    ...,
    {
      "type": "./custom-parameter-mapper"
    }
  ],

Restart the server any time this file is changed so it reflects the new configuration.

Using The New Parameter Mapper

With the new parameter mapper now added to the server, the data in the new mapped folder can be accessed and used.

In order to test it out, copy qb_boulder_msi from the ENVI data folder to C:\customData\myFile, and qb_boulder_msi.hdr to C:\customData\myFile.hdr.

Submit a job to the ese-request-handler by opening this URL in a browser:

http://localhost:9191/ese/services/ENVI/SpectralIndex/submitJob?INPUT_RASTER={"url":"http://localhost:9191/customdata/myFile","factory":"URLRaster"}&INDEX="Normalized Difference Vegetation Index"

Full Example Code Blocks

Place all of the following files in the custom-parameter-mapper folder and add the provided contents. Remember to Configure the Server before attempting to execute these processes.

package.json

{
  "name": "custom-parameter-mapper",
  "version": "1.0.0",
  "description": "Custom parameter mapper",
  "main": "parameterMapper.js"
}

config.json

{
  "URLBase": "/customdata/",
  "folderPath": "C:\\customData\\"
}

parameterMapper.js

/**
 * Custom ENVI Parameter Mapper module.
 * This checks urls for the customdata path
 * and maps them to the specified folder
 */
var defaultConfig = require('./config.json');
var url = require('url');
var extend = require('util')._extend;
var path = require('path');
function CustomParameterMapper(rootConfig) {
  // Override the default configuration with server level options.
  var config = extend({}, defaultConfig);
  config = extend(config, rootConfig);
  // Need stub functions to comply with interface
  this.init = function() {
    // Empty init function
  };
  // This parameter mapper doesn't support reverseTranslate
  this.reverseTranslate = function() {
    return false;
  };
  // Returns a boolean indicating if the parameter is translatable or not.
  // Callback with results if true.
  this.translate = function(name, value, callback) {
    var translated = false;
    var newValue;
    // Check for url
    try {
      var URLValue = url.parse(value);
      if (URLValue.pathname.startsWith(config.URLBase)) {
        newValue = config.folderPath + URLValue.pathname.substring(config.URLBase.length);
        var relative = path.relative(config.folderPath, newValue);
        var error;
        // Prevent navigating outside of the specified folder
        if(relative.startsWith('.')) {
          error = 'ERROR: value can not be outside of the mapped folder';
          newValue = null;
        }
        // Translate should be asynchronous.
        process.nextTick(function() {       
          callback(error, newValue);
        });
        translated = true;
      }
    } catch (e) {
      // Do nothing on error, we couldn't translate this one.
    }
    return translated;
  };
}
module.exports = CustomParameterMapper;

Parameter Mappers in Clustered Configurations

A well-designed parameter mapper requires no additional logic for a clustered configuration. The main concern is that file paths must be handled properly when clustering. The translate and reverseTranslate functions should do that in a way that is usable throughout the cluster. Ideally, the URLs can also be used to access the data directly.

When a parameter mapper is asked to reverse translate output values, it is expected to turn local file paths into URLs. These URLs should be created using the external address configuration, and they should contain enough information to reverse the process in a translate call. This is true regardless of whether that server is part of a cluster.

To translate a file that is in a job workspace, the parameter mapper should be able to extract the job ID and a file path that is relative to the job's workspace folder. This allows the workspace to be in control of the files and potentially move files after the job is complete. The parameter mapper can then use the ParameterMapperIOC object to get the local file path from the workspaceManager. This provides an opportunity for the workspaceManager to move data to the current node for processing when needed.



© 2017 Exelis Visual Information Solutions, Inc. |  Legal
My Account    |    Buy    |    Contact Us