HSI_CONTACT2FITS is the name of the IDL routine used for level 0 processing and Quicklook processing. It is run every 5 minutes as part of a cronjob on one of the servers (presently sunny.ssl.berkeley.edu for level 0 processing and sunstruck.ssl.berkeley.edu for quicklook processing and images.) Multiple processes are not permitted, when a process is started, a lock file is created, and the process will not start if the lock is present.
Here is a description of the level 0 processing.
- Contact files contain all of the vc1 and vc3 packets for a contact
- Bad packets are checked for and flagged, but retained in the
array, (HSI_BAD_PAK) Flags:
- Reed-Solomon bit set to 1 for uncorrected bit error (flag=100)
- bad App_id (flag=1+RS_flag)
- Packet time way out of range, or before jul-2000 (flag=2+RS_flag)
- Note that the packet times themselves are never changed, but the array used to sort them internally is changed, and all flags are retained and put in the lookup table.
- The next step is to find the orbit data, this is input from data files provided by the GDS. File times are truncated to the obs summary ephemeris time interval 20.0 seconds. Files start and end in eclipse periods.
- Loop over orbits:
- For each orbit, data from previous contacts is checked, if there is a data file for this orbit already, those packets are input and combined with the current array.
- Packet Headers are checked to find duplicate packets, if found, duplicates are deleted, preferentially those with bad_pak_flag=0
- Packets are then written to FITS files, using HSI_PACKET2FITS For orbits with previous data, the version number in the filename is incremented, to avoid overwriting of old files
- The HSI_FILEDB structure is updated, this tells what time range of data exists for what file. The filedb structure is written into monthly FITS and text files, (hsi_filedb_yyyymm.fits or .txt) stored in /disks/sunny/home/dbase for use by the SOC processes, and in /disks/hessidata/metadata/dbase for mirroring.
At the same time, there is another process that calls HSI_CONTACT2FITS with the /archive option set. Each time that this program runs, it processes the first file in the database that has filedb.status_flag set to 0, and all other files in that orbit time range. For each file:
- First create Observing Summary Objects and Instrument log Objects:
- Hsi_obs_summ_rate -- contains spin-averaged count rates
- hsi_ephemeris -- Spacecraft position, 20 sec or 5 spin periods
- hsi_obs_summ_flags -- Various data flags,
- hsi_qlook_monitor_rate -- monitor rates, incl. particle rates
- hsi_qlook_packet_rate -- packet rates, all app_ids
- hsi_qlook_rate_per_det -- Count rates for each detector segment, all energies
- hsi_qlook_pointing -- Spacecraft pointing, and quality
- hsi_qlook_roll_period -- Roll (spin) period
- hsi_qlook_roll_angle -- Roll angle
- hsi_flare_list -- Flare list
The observing summary and instrument log are stored in /disks/hessidata/metadata/catalog (files hsi_obssumm_yyyymmdd_nnn.fits, hsi_inslog_yyyymmdd_nnn.fits). There are also filedb files for the obssumm and inslog files, which are stored with the level 0 filedb files.
The flare list is written as monthly fits files (hessi_flare_list_yyyymm.fits ) in /disks/sunny/home/dbase and in /disks/hessidata/dbase. Monthly text files files and a text file for the whole mission are also written in /disks/hessidata/dbase.
see also: Observing Summary and Quicklook
16-Oct-2004, jmm