3353 Rate this article:
No rating

Using IDL_IDLBridge to run multiple processes in parallel

Atle Borsholm

I noticed a question on the newsgroup about running multiple time consuming IDL commands in parallel processes. This is a topic that we in the Custom Solutions group come accross fairly frequently. Below is a simple example of how to manage a number of processes running at the same time.

Notice in the code, that it is often important to control the working directory as well as the search path for all of the bridge processes so that the bridge processes can find the necessary IDL code as well as any data needed.

pro multi_process
  compile_opt idl2, logical_predicate
 
  cd, current=dir
 
  n_proc = 10
 
  tlb = widget_base(title='Initializing',/column, xsize=500, ysize=500)
  t = widget_text(tlb, value='Initializing...')
  widget_control, tlb, /realize
 
 
  br = objarr(n_proc)
  for i=0, n_proc-1 do begin
    br[i] = IDL_IDLbridge()
    br[i]->SetVar,'dir',dir
    br[i]->Execute,'cd,dir'
    br[i]->SetVar,'!path',!path
    br[i]->Execute,'wait,50*randomu(seed)',/nowait
  endfor
 
  widget_control, tlb, /destroy
  tlb = widget_base(title='Processing progress',/column)
  t = widget_table(tlb, xsize=2, ysize=n_proc, value=strarr(2,n_proc), $
    column_labels=['Process #','Status'], column_width=[250,250])
  widget_control, tlb, /realize
 
  ; Monitor progress
  status = replicate(1, n_proc)
  table = strarr(2,n_proc)
  table[0,*] = strtrim(indgen(n_proc),2)
  codes = ['Idle','Executing','Completed','Error','Aborted']
  while max(status) gt 0 do begin
    wait, 1
    for i=0, n_proc-1 do begin
      status[i] = br[i]->Status()
    endfor
    table[1,*] = codes[status]
    widget_control, t, set_value=table
  endwhile
  widget_control, tlb, /destroy
end
This code example will display a simple GUI showing the status of the IDL processes.

This is a great approach if you want to speed up some time consuming IDL processes that can run individually on the same system. In more complex situations, you want to instead look into ENVI Services Engine as it provides much more functionality for configuring how "jobs" are being run, and also supports distributing jobs over multiple systems (nodes). Despite the word "ENVI" here, it also supports running pure IDL code.

Please login or register to post comments.