Welcome to the Harris Geospatial product documentation center. Here you will find reference guides, help documents, and product libraries.


  >  Docs Center  >  Libraries  >  ASTROLIB  >  WEBGET

WEBGET

WEBGET

Name


    WEBGET()

Purpose


    Use the IDL SOCKET procedure to get data from http servers

Explanation


    WEBGET() can access http servers - even from behind a firewall -
    and perform simple downloads. Currently, text and FITS files can be
    accessed.

Calling Sequence


      a=webget(URL)

Inputs


      URL - scalar string giving a fully qualified url of the form
          'http://server.eso.org/path/file.html'. WEBGET() can
          also use other valid URLs that contain 'GET' or 'POST' codes.

Optional Input Keyword Parameters


      COPYFILE - if set to a valid filename (file must have write permission),
            the data contents of the web server's answer is copied to that
            file.
      HTTP10 - If set, then use the HTTP 1.0
      POST - if set to a structure, the structure tags and values
              will be used as post variables and POST'ed to the URL.
              If POST is not set, the normal HTTP GET is used to
              retrieve the URL.
      /SILENT - If set, the information error messages are suppressed
      TIMEOUT - Integer scalar giving number of seconds to wait to connect
                or for data to arrive before giving up and issuing an error.
                Default=15 seconds
  OUTPUTS: A structure with the following fields:
            .Header - the HTTP header sent by the server
            .Text - The text part of the downloaded file. If the
                    content type of the file was not of class
                    'text', this will be an empty string.
            .ImageHeader - Header file of a FITS-image. FITS images
                          are read when the content type is
                          'image/fits' or 'application/octet-stream'
                          (for dss-access). If the file is not a FITS
                          image, this will be an empty string.
            .Image - The FITS image read from the server. If the file
                    did not contain a FITS image, this will be zero.

Restrictions


    The mime-type recognition is extremely limited. Only the content-type is
    determined. Any text-file will be stored in out.Text. The only other
    category which can be fetched is FITS files, which will be stored in
    out.Image and out.ImageHeader.
    PROXY: If you are behind a firewall and have to access the net through a
        Web proxy, set the environment variable 'http_proxy' to point to
        your proxy server and port, e.g.
        'setenv http_proxy=http://web-proxy.mpia-hd.mpg.de:3128'
              The URL *MUST* begin with "http://".

Procedure


    Open a socket to the webserver and download the header. After deciding
    whether it is text or binary, either store the text or try to read a
    FITS file.

Example


      IDL> a=webget('http://www.mpia.de/index.html')
      IDL> print,a.Text
      or
          > PointingRA=0.0
          > PointingDE=30.0
          > QueryURL = strcompress("http://archive.eso.org/dss/dss/image?ra="+$
          > string(PointingRA)+$
          > "&dec="+$
          > string(PointingDE)+$
          > "&x=10&y=10&Sky-Survey=DSS1&mime-type=download-fits", $
          > /remove)
          > a=webget(QueryURL)
          > tvscl,a.Image
          > print,a.ImageHead

Modification History


    Written by M. Feldt, Heidelberg, Oct 2001 <mfeldt@mpia.de>
    Use /swap_if_little_endian keyword to SOCKET W. Landsman August 2002
    Less restrictive search on Content-Type W. Landsman April 2003
    Modified to work with FIRST image server- A. Barth, Nov 2006
    Better recovery from errors W. Landsman April 2007
    Add support for POST access J.D. Smith June 2007
    Recognize "fits" image type used by SKYVIEW W. Landsman June 2007
    Upgraded, partially, to HTTP 1.1 M. Perrin, July 2007
      The HTTP 1.1 support is presently INCOMPLETE: virtual servers are
      supported, but chunked transfer encoding is not yet supported, so
      technically this is not fully HTTP 1.1 compliant.
    Added http10 keyword W. Landsman August 2007
    Assume since V5.6, sockets always available W. Landsman Nov 2007
    Fix problem when using proxy server W. Landsman July 2008
    Fix problem with /SILENT keyword W. Landsman Jan 2009
    Added check for missing Mime TYPE in CLASSANDTYPE, Zarro, December 2011
    Timeout applies to connecting as well as reading, default is now 15
              seconds W Landsman January 2012
    Allow http_proxy to be upper or lower case W.L./D. Palmer Feb 2013



© 2018 Harris Geospatial Solutions, Inc. |  Legal
My Account    |    Store    |    Contact Us