Cloudy SolarSoftware

From RHESSI Wiki

Nugget
Number:	TBDwarning.png"TBD" is not a number.
1st Author:	Laszlo I. Etesi
2nd Author:	TBD
Published:	TBDwarning.pngThe date "TBD" was not understood.
Next Nugget:	TBD
Previous Nugget:	TBD
List all

Contents 1 NOTE 2 Introduction 3 Requirements 4 Step 1: Finding 5 Step 2: Pre-processing 6 Step 3: Visualizing 7 Documentation 8 Contacts

NOTE

This nugget is unfinished and still undergoing changes.

Introduction

In our project "Extending the Virtual Solar Observatory (VSO)" we have combined features available in Solar Software (SSW) to produce an integrated environment, supporting data location, retrieval, preparation, and analysis. This workflow and examples are shown below. Our goal is an integrated analysis experience in IDL, easy-to-use but flexible enough to allow more sophisticated procedures such as multi-instrument analysis. To that end, we have made the transition from a locally oriented setting where all the analysis is done on the user’s computer, to an extended analysis environment where IDL has access to services available on the Internet. We have implemented a form of Cloud Computing that uses the VSO search and a new data retrieval and pre-processing server (PrepServer) that provides remote execution of instrument-specific data preparation. The raw and pre-processed data can be displayed with our plotting suite, PLOTMAN, which can handle different data types and perform basic data operations. Figure 1 gives an overview of the entire framework with numbers corresponding to the steps 1-2-3 explained below.

Our environment supports a growing number of solar instruments that currently includes RHESSI, SOHO/EIT, TRACE, SECCHI/EUVI, HINODE/XRT, and HINODE/EIS.

Requirements

• IDL 6.4 or higher
• Java 1.5 or higher
• SolarSoftware (SSW) with GEN package (standard)

Step 1: Finding

SHOW_SYNOP is a graphical user interface (GUI) to access and search the VSO and other locations for instrument data within a specified time interval. Search results can be directly downloaded into the active SSW IDL environment. To start searching with the widget, type SHOW_SYNOP in your SSW IDL environment. Figure 2 shows the SHOW_SYNOP widget; the red box highlights the VSO search form with "Start Time" and "End Time" specifying the search interval and "remote sites ->" defining the instrument. A click on "Search" will query the VSO for data files, which will be displayed in the list indicated by the green box in Figure 2.

VSO_FILES allows for data search from an SSW IDL command line:

Command
urls = vso_files(tstart, tend, instrument=instrument)

Input
tstart: Start time ('Day-Month-Year Hour:Minute')
tend: End time ('Day-Month-Year Hour:Minute')
instrument: Instrument to search for (e.g. 'eit', 'euvi', etc.)

Returns
urls: An array of URLs pointing to remote data file repositories

Examples
; This will return one URL to the EIT image closest to January 25, 2007 06:55 and downloads the data file with SOCK_COPY
urls = vso_files('25-Jan-2007 06:55', instr='eit')
print, urls
http://sohodata.nascom.nasa.gov//archive/soho/private/data/processed/eit/lz/2007/01/efz20070125.070013
sock_copy, urls

; This will return URLs to XRT images between January 25, 2007 06:00 and January 25, 2007 07:00
urls = vso_files('25-Jan-2007 06:00', '25-Jan-2007 07:00', instrument='xrt')
print, urls
http://kurasuta.cfa.harvard.edu/DATA/2007/01/25/H0600/XRT20070125_060117.2.fits
http://kurasuta.cfa.harvard.edu/DATA/2007/01/25/H0600/XRT20070125_060120.2.fits
...


Step 2: Pre-processing

Typically, instrument data found with the VSO are unprocessed level-0 data. The PrepServer offers remote pre-processing of those data from within SHOW_SYNOP or from the IDL command line before or after downloading. In SHOW_SYNOP, search results are displayed in the list beneath the VSO search form (see Figure 2, green box). There are two different pre-processing behaviors configurable:

• By default a click on "Download" retrieves the level-0 data file, stores it on the local hard drive and adds an entry to the list in Figure 2 (blue box). The level-0 data file is pre-processed by selecting its entry in the blue box and by clicking on "Display". This uploads the level-0 data file to the PrepServer where it is pre-processed, and downloads the pre-processed product to the local hard drive.
• In auto-preprocessing mode, which can be activated in the configuration menu (Figure 2, yellow box), SHOW_SYNOP sends the URL of the level-0 data file to the PrepServer and only downloads the pre-processed data.

Please note: SHOW_SYNOP and the PrepServer are dependent of data providers for downloading speed. Be aware that downloading data may take a few minutes, during which SHOW_SYNOP or your SSW IDL session will be unresponsive.

VSO_PREP allows for data pre-processing from an SSW IDL command line:

Command
vso_prep, infile, outfile=outfile, instrument=instrument, oprep=oprep, _extra=extra

Input
infile: A local or remote file to be pre-processed
instrument: (optional) The instrument name. Default: Determined from infile
_extra: (optional) This parameter will collect all additional input parameters

Output
outfile: (optional) A file name for the pre-processed file. Default: prep_infile
oprep: (optional) An object that will be created after the pre-processing. It will allow for direct access to the data (i.e. for display)

; This searches the VSO for an EIT file and sends the returned link to the PrepServer for pre-processing
urls = vso_files('25-Jan-2007 06:55', inst='eit')
vso_prep, urls, out='eit.fits'
% VSO_PREP_MAIN: Prepping completed successfully.
% VSO_PREP_MAIN: Wrote prepped data to - /home/user/eit.fits

; This command will reconstruct a RHESSI clean 6-12keV image on the PrepServer
vso_prep, instrument='rhessi', im_time_interval=['25-Jan-2007 06:53:44', '25-Jan-2007 06:57:40'], image_alg='clean', im_energy_binning=[6,12]
% VSO_PREP_MAIN: Prepping completed successfully.
% VSO_PREP_MAIN: Wrote prepped data to - /home/user/prepped_temp.fits

Step 3: Visualizing

The data are visualized with PLOTMAN, which handles different data types such as light curves, images, spectra, and spectrograms. It provides basic display operations such as zooming, image overlays, solar rotation, etc. To display data with PLOTMAN from within SHOW_SYNOP, simply select data files (Figure 2, blue box) and click on "Display". If the data file has already been pre-processed, it will be displayed immediately in a PLOTMAN window, otherwise it is first sent to the PrepServer for processing. If multiple file have been selected, all are displayed in the same PLOTMAN window.

PLOTMAN can also be called from the command line:

; This command pre-processes an XRT image, writes it into an object and displays the XRT image in PLOTMAN (see Figure 3)
vso_prep, 'http://kurasuta.cfa.harvard.edu/DATA/2007/01/25/H0600/XRT20070125_065656.3.fits', oprep=xrt
xrt->plotman, /color

; The first command searches the VSO for the TRACE data file (contains multiple images) closest to January 25, 06:30. The second command detects the instrument type TRACE and opens a window for image selection (see Figure 4), then pre-processes the specified images. The last command displays all pre-processed images in PLOTMAN (see Figure 5)
urls = vso_files('25-Jan-2007 06:30', instr='trace')
vso_prep, urls, oprep=trace
trace->plotman, /color

; This last example first reconstructs a RHESSI clean 6-12keV image on the PrepServer; then searches the VSO for the EIT data file closest to January 25 2007 6:55 and sends the URL to the PrepServer for pre-processing; and finally opens both images in the same PLOTMAN window (parameter plotman=p) where the RHESSI contours are manually overlaid with the EIT image to create the image in Figure 6.
vso_prep, instrument='rhessi', im_time_interval=['25-Jan-2007 06:53:44', '25-Jan-2007 06:57:40'], image_alg='clean', im_energy_binning=[6,12], oprep=rhessi
urls = vso_files('25-Jan-2007 06:55', inst='eit')
vso_prep, urls, oprep=eit
rhessi->plotman, plotman=p
eit->plotman, plotman=p


Documentation

• SolarSoftware
• Virtual Solar Observatory
• PrepServer
• PLOTMAN
• VSO_FILES
• VSO_PREP
• SOCK_COPY
• Complete Code Example

Contacts

• SHOW_SYNOP: Dominic Zarro (dominic dot zarro at nasa dot gov)
• PrepServer: Laszlo I. Etesi (laszlo dot etesi at nasa dot gov)
• PLOTMAN: Kim Tolbert (kim dot tolbert at nasa dto gov)