HESSI DATA FLOW OPERATIONS, QUICK-LOOK GENERATION

Start with a detected photon:

Photon arrival time, Energy Channel, and detector element (A2d_index 0 to 27) is stored

Livetime is calculated every 512 binary mu-sec.

Strategies for Large Flares: If flares are too large, (too many SXR photons)

  • Attenuators: In front of grids, Cut down SXR photon flux. Start, 1 open, 1 closed
  • Decimation: Only a fraction of front-segment photons are recorded, less low energy flux.
  • Fast Rate Mode: Record Counts in energy Bands, lose the individual time tags.
  • Thresholds are programmable from the Ground

Events are packed into source packets (App. ID #100, for a list of all of the packet types, see the telemetry document:ftp://apollo.ssl.berkeley.edu/pub/hessi/icd/HSI_SYS_007D.pdf )

Master Frame and Virtual Channel headers are added when the packets are transmitted down. Vc1 and vc3 are stored, vc2 is real-time Science Data.

Once on the ground, the contact files for vc1 and vc3 are backed up on tape.

Next the files are shipped to the MOC, where ITOS strips the Packets from frames and adds a 12 byte header.

In the SOC, hsi_contact2fits combines vc1 and vc3 packets, splits files into files per each orbit and appends catalog data.

Hsi_contact2fits:

  • Reads packet files
  • Reads orbit data for packet time range
  • For each orbit:
    • Finds packets in time range, concatenates vc1 & vc3 packets, and sorts by time
    • Checks the database file (hsi_filedb.fits) to see if there exists a file from the previous contact for this orbit
    • If there are previous files, and they are available, read in those packets, and concatenate to the current array, and sort
    • Remove duplicate packets (Not yet implemented)
    • Delete old file, increment version number, set status flag for deleted file to -1 (deleted)
    • Write packets to a new file, add an entry to the file database structure.
    • Next Orbit
  • After all of the contact data is read, loop through files and add Catalog extensions:
    • Observing Summary
    • Flare list
    • Instrument Log
    • QuickLook Images
    • QuickLook Spectra

Distribution

The catalog data are appended to each file. Also the Observing summary and flare lists are appended to separate files. Orbit files are held until the next contact in a holding area, so that imcomplete orbital files are not written to the CDs. After the next contact the data can be written to CDs, and distributed. The flare list and observing summary files are ftp'd to SDAC and ETH, and put online.

Details

Contents of Observing Summary:

The observing Summary is obtained from the event packets. The events are time-binned and the count rate is compressed to bytes (0.03 accuracy up to 1.0e6 counts/sec). Modulation variance is calculated for the 2 coarsest collimators by calculating the standard deviation of counts for the given 4 sec interval and dividing by the square root of the total counts in that interval.

Info Structure:

  • .file_id = a string identifying the input file, or output file, if the summary was generated from a score, or concatenated from summaries from multiple files max 80 characters FOR FITS output
  • .summary_start_time = the summary start time, in a string, ecs format e.g., 1998/08/16 17:05:00.000
  • .summary_end_time = the summary end time, in a string, ecs format e.g., 1998/08/16 17:05:00.000
  • .a2d_index_mask = a bytarr(27) set to 1 for each detector element that is used
  • .n_time_intv = the number of time intervals, for this summary
  • .av_spin_period = the nominal duration, in seconds, of a single time interval
  • .n_ephemeris = the number of ephemeris data points, for this summary
  • .dt_ephemeris = the time interval between ephemeris recordings
  • .n_energy_bands= the number of energy bands for the front detector count rates, hardwired to be 8
  • .energy_edges = the energy bin edges, in KeV [3.0, 10.0, 20.0, 60.0, 100.0, 300.0, 1000.0, 10000.0, 100000.0]
  • .nflags = the number of data flags, to a max of 8
  • .flag_ids = string identifiers for each data flag, e.g., ['Attenuator State', 'SAA passage', 'Fast Rate Mode', 'Day/night', 'Flare']
  • .variance_nbin = the number of time bins, per spin period, used to obtain the variance for detection of modulation. The default is 40.

Data Structure, one for each 4 sec interval:

  • .countrate = count rate of front detectors, summed over the detectors, in n_energy_bands energy bands, compressed cps,
  • .monitor_rate = count rate for particle detector, compressed cps,
  • .mod_variance = a bytarr(2), for the two coarsest collimators the variance in the count rate for front detector elements, scaled by the poisson variance, then*10
  • .flags = a bytarr(8) of data flags

Ephemeris Structure, one every 20 sec.

  • .xyz = fltarr(3) of the spacecraft position in KM

Contents of Flare List

The flare list is obtained by comparing the observing summary count rate and modulation variance to thresholds. The count rate threshold can be set, or determined from the background level. Defaults can be changed for threshold levels, energy range, minimum flare duration, minimum time between flares. A check using position information may be added. The flare position is found using a 20 arcsec resolution 64X64 MEM image, and four coarsest collimators. (Tests work well down to 500 photons per detector, down to 100 depending on background level.)

  • .List_ID = an identifying string
  • .n_flares = the number of flares
  • .time_range = the time range covered
  • .energy_range_found = the energy range used to find the flares, Default of [3.0, 20.0] keV
  • .flare_list_entry = an array of flare list entries, for each:
    • id_number = An ID number,
    • start_time = flare start time
    • end_time = flare end time
    • peak_time = flare peak time
    • energy_range = energy range of observed flare
    • peak_countrate = peak count rate
    • total_counts = total # of counts
    • position = flare position in arcsec from sun center
    • filename = file(s) which have the data for this flare, separated by commas
    • eclipse_flag = set to 1 if the flare is cut off be eclipse
    • saa_flag = set to 1 if flare is cut off by SAA

Instrument Log (TBD)

QuickLook Images (Mostly TBD)

Images will be generated for each energy band in the Obs Summary if data exists. The flare interval will be spilt up according to accumulated counts in each channel. This depends heavily on how fast the imaging algorithms can be made to run. Faster code = more images. Slow code = 1 image per energy band per flare. Speed considerations may limit these images to 5-7 arcsec resolution

Quicklook Spectra (Mostly TBD)

Spectra will be generated for each flare, 10(?) second cadence, along with full interval spectra. Traditional thermal plus double power law spectra.

File Database: Hsi_filedb

Each file that is written is recorded as an entry in the filedb structure. This is written into a FITS file, which is opened and read and rewritten for every contact.

Hessi Filedb structure, for each file

  • file_id = file name
  • orbit_number = Orbit number (from GDS)
  • data_start_time = data start time
  • data_end_time = data end time
  • orbit_start_time = orbit start time
  • orbit_end_time = orbit end time
  • status_flag = set to -1 if the file has been deleted, and the packets are contained in another file , set to 1 if the file has been written to CD.
  • npackets = Number of packets in file

Now the data is ready for analysis...

16-Oct-1999, jmm