SPD 2000 TITLE
THE HESSI CATALOG
J.McTiernan and A.Csillaghy
Space Sciences Lab
University of California, Berkeley
The HESSI mission is scheduled to be launched in late 2000-early
2001. HESSI will provide images and spectra of solar flares in the
energy range from 3 keV to 10 MeV.
HESSI data are stored as telemetry packets, as received from the
spacecraft, in FITS binary tables. The HESSI data catalog for each
file is appended to that data file as FITS binary tables. The catalog
includes: the Observing Summary, which contains count rates in 9
energy bands, and other information, the Flare list, a summary of the
solar flares observed for the time range covered by the file, the
Instrument Log, which summarizes the instrument state-of-health data
for the given time range, quick-look images for different energy bands
for each flare, and quicklook spectra for each flare
Here we present a sample of HESSI catalog data derived from
simulations for a week of data. We will also demonstrate how to access
the HESSI catalog from the data files.
Simulated HESSI data files are available at
http://hessi.ssl.berkeley.edu/data/test_data .
17-Jun-2000, jmm
HESSI SCIENCE OPERATIONS
Telemetry to CD production:
Start with telemetry files:
An IDL routine, HSI_CONTACT2FITS, writes the original telemetry files into FITS files 1 per each orbit, named hsi_yyyymmdd_hhmm_nnn.fits. The date and time, yyyymmdd_hhmm, corresponds to the middle of spacecraft night between two orbits.
The "nnn" is a version number, for instances in which a file is generated that has the same yyyymmdd_hhmm as a previous file.
The catalog data will be generated as part of HSI_CONTACT2FITS, and appended to the hsi_yyyymmdd_hhmm_nnn.fits files.
These files are then written to CD.
Contents of the various files:
Inputs to hsi_contact2fits:
- Initial files contain telemetry frames:
- ephemeris files contain:
- position and velocity of spacecraft at 10 sec intervals
- times of spacecraft day and night, and SAA pasage
Outputs from hsi_contact2fits:
- FITS file contains (as binary table extensions):
- Telemetry packets
- Catalog for the given time interval
- Catalog contains (in binary table extensions):
- Observing Summary: Count rates in 9 energy bands, summed over detectors,
particle detector count rate, modulation variance, and S/C position.
- Instrument Log: Housekeeping, State of Health data for the Imager,
Spectrometer, Particle Detector
- Flare List: Id number, Start, peak and end times, in the energy
range [6.0, 25.0] keV, peak count rate, total counts, Highest energy
of observed photons, Flare position, arcsec from Sun center, Flags for
flares that start/end in Eclipse or SAA.
- Quicklook Spectra: Count and photon spectra as a function of time,
Thermal + double power law photon spectra, with dE = E/2 energy resolution.
- Quicklook Images: Images in the 8 energy bands given by the
Observing Summary, If possible, Cadence determined by Count Rate
Other database files needed by hsi_contact2fits:
- The Calibration database contains the data needed for spectral response matrices. From /ssw/hessi/dbase
- The Grid response database contains the data needed for grid response matrices. From /ssw/hessi/dbase
- The Stellar position database contains the position of stars used by the RAS/SAS SW to determine the aspect solution. From /ssw/hessi/dbase
- The Solar ephemeris database contains the position, size, and orientation of the Sun.
Other data to be generated after contact:
- Real-time Science data catalog: Generated by hsi_contact2fits from
telemetry packets for each contact. Posted on the HESSI ftp site for
quick access.
- Whole-Mission HESSI catalog: During catalog generation by
hsi_contact2fits, the catalog extensions are saved in separate files
and are appended to form daily files,
hessi_catalog_yyyymmdd.fits. These will be accessible for web-based
browsing.
- Whole-Mission Flare List: To be distributed as part of /ssw/hessi/dbase
- File database: A list of each FITS file generated including the filename the start and end times for the file, and the contact from which it came.
17-Jun-2000, jmm
HESSI Observing Summary:
The observing Summary is obtained directly from the telemetry
packets. The photons are time-binned and the count rate is compressed
to bytes (0.03 accuracy up to 1.0e6 counts/sec). Modulation variance
is calculated for the 2 coarsest collimators by calculating the
standard deviation of counts for the given 4 sec interval and dividing
by the square root of the total counts in that interval. The
spacecraft position and data from the particle detector are also
included.
Accessing the Observing Summary
The observing summary is an object that contains pointers to
objects that contain the data. When the software package is
complete, there will be GUI and traditional IDL command-line access to
the data. Here we use the object syntax. Given a filename or a time
range, you get the observing summary as follows:
IDL> obs_summ_obj = obj_new('hsi_obs_summary')
IDL> data = obs_summ_obj -> getdata(filename = filename)
Or
IDL> data = obs_summ_obj -> getdata(ut_time_range = ut_time_range)
The getdata method returns an array of structures containing time
and count rates in 9 energy channels. If the data is not found, the
object is returned with all of the pointers set to -1.
The data is assumed to be in a directory that is pointed to by
the environmental variable $HSI_DATA_ARCHIVE. This can be changed
by setting the data_dir keyword to the appropriate directory.
Contents of the Observing Summary:
Each type of data in the summary is contained in a separate object.
- version = version number
- summary_id = Summary identification, a string identifying the summary
- info_ptr = a pointer to the obs_summ_info structure
- obs_summ_rate = Countrate Object
- ephemeris = Ephemeris Data Object
- mod_variance = Modulation Variance Object
- particle_rate = Particle Rate Object
- obs_summ_flag = Data Flag object
In order to access the objects or structures in a given
obs_summary, use the Get method:
IDL> info = obs_summ_obj -> get(/info)
IDL> rate_obj = obs_summ_obj -> get(/obs_summ_rate)
Info Structure:
- version = A version number for the structure
- summary_start_time = the summary start time, in a string, ccsds format e.g., 1998-08-16T17:05:00.000Z
- summary_end_time = the summary end time, in a string, ccsds format e.g., 1998-08-16T19:05:00.000Z
- concat_flag = Flag set to 1 if the summary has been concatenated from
summaries appended to level 0 data files.
- simulated_data = Flag set to 1 if the data is simulated
Time Arrays
Data times are not saved with the observing summary structure, but
they can be obtained using Get, e.g., data_times = obs_summ_obj ->
get(/data_times) Returns the time array for the given observing
summary. Also the Getdata method will return a structure including times.
So if, for example, you want to plot the countrate in the 25 to
50 keV channel, you could do this:
IDL> countrate = obs_summ_obj -> get(/countrate)
IDL> data_times = obs_summ_obj -> get(/data_times)
IDL> utplot, anytim(data_times, /yohkoh), countrate[3, *]
A typical plot of simulated data is shown below.
Details for the individual Objects:
Each object in the observing summary contains its own info
structure, and data structure. The get method can be used on the
individual objects as follows, to obtain the info structure and the
data structure. Each object can be read individually from the
data file also.
IDL> rate_info = rate_obj -> get(/info)
IDL> rate_data = rate_obj -> get(/data)
Or
IDL> rate_obj = obj_new('hsi_obs_summ_rate')
IDL> data = rate_obj -> getdata(filename = filename)
Obs_summ_rate Info:
- version = A version number for the structure
- file_id = a string identifying the input file, if applicable
- ut_ref = Reference Time, ANYTIM format, i.e., seconds from 1-jan-1979 0:00 UT.
- a2d_index_mask = a bytarr(27) set to 1 for each detector element that is used
- n_time_intv = the number of time intervals, for this summary
- av_spin_period = the nominal duration, in seconds, of a single time interval. Set to 4 seconds.
- n_energy_bands= the number of energy bands for the front detector count rates, hardwired to be 9
- energy_edges = the energy bin edges, in KeV [3.0, 6.0, 12.0, 25.0, 50.0, 100.0, 300.0, 1000.0, 2500.0, 20000.0]
Obs_summ_rate Data Structure, one for each 4 sec interval:
- version = A version number for the structure
- countrate = count rate, summed over the detectors, in
n_energy_bands energy bands, compressed cps,
Mod_variance Info:
- version = A version number for the structure
- file_id = a string identifying the input file, if applicable
- ut_ref = Reference Time, ANYTIM format, i.e., seconds from 1-jan-1979 0:00 UT.
- n_time_intv = the number of time intervals, for this summary
- av_spin_period = the nominal duration, in seconds, of a single time interval. Set to 4 seconds.
- energy_edges = the energy bin edges, in KeV. [6.0, 25.0] is the default
- variance_nbin = the number of time bins, per spin period, used to obtain the variance for detection of modulation. The default is 40.
Mod_variance Data Structure, one for each 4 sec interval:
- version = A version number for the structure
- mod_variance = a bytarr(2), for the two coarsest collimators the
variance in the count rate for front detector elements, scaled by the
poisson variance, then*10
Ephemeris Info Structure:
- version = A version number for the structure
- file_id = a string identifying the input file, if applicable
- n_eph = the number of ephemeris data points, for this summary
- dt_eph = the time interval between ephemeris recordings. Default
is 20 seconds
- eph_ut_ref = Reference time for the ephemeris data.
Ephemeris Data Structure, one every 20 sec.
- version = A version number for the structure
- xyz = fltarr(3) of the spacecraft position in KM.
ECI (Earth Centered Inertial) Coordinates.
Obs_summ_flag Info:
- version = A version number for the structure
- file_id = a string identifying the input file, if applicable
- ut_ref = Reference Time, ANYTIM format, i.e., seconds from 1-jan-1979 0:00 UT.
- n_time_intv = the number of time intervals, for this summary
- av_spin_period = the nominal duration, in seconds, of a single time interval. Set to 4 seconds.
- nflags = the number of data flags, to a max of 16
- flag_ids = string identifiers for each data flag, e.g., ['Attenuator State', 'SAA passage', 'Fast Rate Mode', 'Eclipse', 'Flare']
Obs_summ_flag Data Structure, one for each 4 sec interval:
- version = A version number for the structure
- flags = a bytarr(16) of data flags
Particle_rate Info:
- version = A version number for the structure
- file_id = a string identifying the input file. If applicable
- ut_ref = Reference Time, ANYTIM format, i.e., seconds from 1-jan-1979 0:00 UT.
- n_time_intv = the number of time intervals, for this summary
- av_spin_period = the nominal duration, in seconds, of a single time interval. Set to 4 seconds.
Particle_rate Data Structure, one for each 4 sec interval:
- version = A version number for the structure
- countrate = count rate, summed over the detectors, in
2 energy bands, compressed cps,
17-Jun-2000, jmm
HESSI Countrate Plot
The countrate in the energy band of 6 to 100 keV is plotted for the
first flare on the list (#124). Spacecraft night is indicated at the
start of the plot. The simulated flare is small in order to save time
during simulations. (HESSI observes every photon, so every photon must
be simulated; this takes a long time!). The background interval used
to find the flare times is indicated by the red lines.
IDL> data = flare_list_obj -> getdata(filename = filename)
Or
IDL> data = flare_list_obj -> getdata(ut_time_range = ut_time_range)
The Getdata method returns an array of data structures, one for
each flare in the list. If the data is not found, the object is
returned with all of the pointers set to -1.
The data is assumed to be in a directory that is pointed to by
the environmental variable $HSI_DATA_ARCHIVE. This can be changed
by setting the data_dir keyword to the appropriate directory.
Contents of the Flare List:
- version = version number
- List_ID = an identifying string
- info = a pointer to the info structure
- data = Pointer to the array of flare list entries
In order to access the structures in a given flare list, use the
Get method:
IDL> info = flare_list_obj -> get(/info)
IDL> data = flare_list_obj -> get(/data)
The Flare List Structures
Flare List Info Structure:
- version = A version number for the structure
- file_id = a string identifying the input file, or output file, if
the Flare list was generated from a score, or concatenated from summaries
from multiple files. Max. of 80 characters FOR FITS output
- n_flares = The number of flares
- list_start_time = the list start time, in a string, ccsds format e.g., 1998-08-16T17:05:00.000Z
- list_end_time = the list end time, in a string, ccsds format e.g., 1998-08-16T19:05:00.000Z
- energy_range_found = the energy range used to find the flares, in keV,
the default is to use 6 to 25 keV.
- nflags = the number of data flags, to a max of 32
- flag_ids = string identifiers for each data flag, e.g., ['SAA At
Start', 'SAA at End', ' 'SAA', 'Eclipse at Start','Eclipse At End',
'Flare at SOF', 'Flare at EOF', 'Fast Rate Mode', 'Decimation', 'Att
State at peak']
- simulated_data = Flag set to 1 if the data is a simulation
- concat_flag = Flag set to 1 if the flare list has been concatenated from
flare lists appended to level 0 data files.
Flare List Data Structure
- version = A version number for the structure
- id_number = An ID number,
- start_time = flare start time, ANYTIM format, i.e., seconds from
1-Jan-1979 0:00 UT.
- end_time = flare end time, ANYTIM format.
- peak_time = flare peak time, ANYTIM format.
- bck_time = Background accumulation time range, ANYTIM format.
- energy_hi = Edges of highest energy band of observed data.
- peak_countrate = peak count rate
- bck_countrate = background count rate
- total_counts = total # of counts
- position = flare position in arcsec from sun center
- filename = file(s) which have the data for this flare, separated by commas
- flags = a bytarr(32) of flare flags
Writing the Flare List
To Write the flare list to a FITS file, use the write method, and
for an ASCII file, use the write_acsii method:
IDL> flare_list_obj -> write, filename = filename
IDL> flare_list_obj -> write_ascii, ascii_filename = ascii_filename
A sample of an Ascii flare list is shown below:
HESSI FLARE LIST FILE
Created: 2000-06-17T16:24:37.000Z
ID: flare list: 2000-08-31T00:16:20.000Z to 2000-08-31T00:42:08.000Z
Number of Flares: 2
Time Range: 2000-08-31T00:16:20.000Z
To: 2000-08-31T00:42:08.000Z
Energy Range Found (keV): 6.00000 25.0000
Simulated Data
DATA:
id_number | Start_date | Start_time | End_time | Peak_time | hi_eband0 | hi_eband1 | peak_rate | total_cnt | xcen | ycen | Flags |
124 | 2000-08-31 | 00:16:20 | 00:16:40 | 00:16:28 | 100.00 | 300.00 | 624.00 | 8048.00 | 402.50 | 228.50 | -1 |
125 | 2000-08-31 | 00:41:48 | 00:42:08 | 00:41:56 | 50.00 | 100.00 | 272.00 | 3696.00 | -612.50 | -191.50 | -1 |
FLAGS
-1 : NONE
0 : SAA At Start
1 : SAA at End
2 : SAA
3 : Eclipse at Start
4 : Eclipse At End
5 : Flare at SOF
6 : Flare at EOF
7 : Fast Rate Mode
8 : Decimation
9 : Att State at peak
HESSI Flare Position
Flare position is found using a 35 arcsec resolution
back-projection image of the sun, using collimators 6, 7, and 8, which
have spatial resolutions of 36, 63, and 109 arcsec. This image is a
sample from the first flare on the list (#124), synthesized using only
the emission at the peak 4 second interval; a total of 860
photons. The brightest point is the white pixel at position (400, 200)
and this is indeed the location of the simulated flare. The red pixel
at position (-600, -200) is a reflection of the true image and is an
artifact due to the form of the simulated aspect solution. The reflection
problem is reduced when using larger numbers of photons, but it may be
an issue for very small flares (e.g., #125).