WET Data Formats

There are many possible output formats from the various CCD photometry extraction routines, but each uses a different format for their output data. It would essentially be impossible for the WET to support all of these different formats; additionally, the variety of formats results in a lack of uniformity in the data submitted to HQ. For these reasons, the WET HQ will support data submitted in one of two formats:
  • Standard Quilt 9 format
  • WET CCD format described below
Quilt 9 format is used with WET standard PMT photometers, while the CCD format is discussed below. PMT photometers can use the CCD format, and in fact are encouraged to do so, since it passes a comparative wealth of information to HQ.

For any questions about formatting, or how to get your software compliant with the WET format, email Reed Riddle with any questions.

The WET CCD format

The data file is separated into two sections. The first section is a data header, and the second the times and counts from the photometry. The header is based on a keyword system, so items can be sorted into any order, with the exception of the first line which defines the format. Each header line starts with a hash mark (#), followed by a space, the keyword, a colon and the keyword parameters. Header lines must follow this convention, or the data reduction software will be unable to read the lines. Header lines must be no more than 80 characters long. The times and counts follow the header lines, with no blank lines separating the data from the header. Time formats are discussed below; be sure to follow the instructions closely when including times for the data. The counts are the sum of the pixels in each aperture; there should be no scaling, peak fitting or other methods used to determine the counts. There should also not be any conversions to magnitudes, as the WET data reduction does not use magnitudes. It is possible to submit different aperture sizes for each channel (see the APSCALE keyword), but it is best to use uniform apertures for photometry extraction when at all possible.

Header keywords

The keywords for the data file header are included in the following table. Note that there is an explanation for each keyword; be sure that your software uses the keywords appropriately.

Keyword Values Description
General keywords
XQEDDATA none All files start with this keyword on the first line.
RUNNAME text Name of the data run. If not used, then the run name is presumed to be the name of the submitted file.
OBSERVER text The people who gathered the data. Can be entered on one line, or on multiple header lines.
NCHAN integer Number of data channels submitted.
NPTS integer Number of data points submitted.
FILTER keyword Filter used for observations.
COMMENT text Comment line. This is good for passing on any information about the run, especially useful stuff. Can be used on multiple lines.
HEADERLINE text CCD header line from the FITS file. As with a comment, used to pass information about the run. Can be used on multiple lines.
Observatory keywords
OBSKEY text Observatory/telescope keyword. If this is used, then none of the following items are necessary for the header.
OBSERVATORY text The observatory where the telescope is located.
TELESCOPE text Telescope used to acquire the data.
LONGITUDE HH:MM:SS.S Telescope longitude W of Greenwich, Eng. Use negative numbers for telescopes E. of Greenwich.
LATITUDE DD:MM:SS.S Telescope latitude, positive for North and negative for South.
ALTITUDE meters Telescope altitude above sea level, in meters.
Object keywords
OBJKEY text Object keyword. If this is used, then none of the following items are necessary for the header.
OBJECT text The star that was observed.
OBJCLASS text The class of the object observed.
RA HH:MM:SS.S Right ascension of star at EPOCH
DEC DD:MM:SS.S Declination of star at EPOCH
EPOCH YYYY.Y Epoch of object coordinates. This will usually be a standard date (like 2000.0), but can also be the observation epoch. The coordinates MUST match the epoch!
Photometer keywords
PHOTKEY text Photometer keyword. If this is used, then none of the following items are necessary for the header.
PHOTOMETER text Name of the photometer used to acquire data
PHOTTYPE keyword Type of photometer used. The only two keyword choices are "CCD" or "PMT".
ACQSOFT text Data acquisition software.
EXTRSOFT text CCD aperture photometry extraction software.
Time keywords
DATE-OBS YYYY-MM-DD Date at start of first integration
TIME-BEG HH:MM:SS.SSS Time at start of first integration
DATE-END YYYY-MM-DD Date at start of last integration
TIME-END HH:MM:SS.SSS Time at start of last integration
EXP-TIME seconds Exposure (integration) time
READ-TIME seconds CCD readout time (average)
CYC-TIME seconds Length of exposure and CCD readout time
DRIFT seconds Computer clock drift through the night (compared to a stable clock). Time is drift from start to end of night. The drift is positive if your clock is running fast and negative if your clock is running slow.
TIME-FMT keyword The format of the submitted times. Formats are discussed below.
Aperture keywords
APTYPE keywords The type of data in each of the submitted channels. These are marked with keywords and listed in the order of the data channels.
APSHAPE keyword The kind of aperture used in the observations.
APSCALE numerical list A list of numbers showing the ratios of each aperture if they are not all the same size. They should be related to the target aperture.
APSIZE pixels The size of the apertures, in pixels

The following header keywords are required to be included in any data file submitted to WET HQ:

  • XQEDDATA
  • OBSERVER
  • FILTER
  • DATA-OBS
  • TIME-BEG
  • EXP-TIME
  • TIME-FMT
  • APTYPE
  • PHOTTYPE
  • OBJKEY or OBJECT (plus other information if necessary)
  • PHOTKEY or PHOTOMETER
  • OBSKEY or TELESCOPE (plus other information if necessary)
Other header keywords may be necessary, depending on the configuration of your observing instrument or extraction software. It is best to include more information than less, as HQ can always use more information when a problem arises with data.

Filters

WET PMT photometers regularly work without filters, as the characteristics of the PMT sensitivity mesh well with the usual WET objects (white dwarfs and other hot subdwarf stars). CCDs gernerally are not as well matched, and the WET has observed other objects that required filters. It is necessary for photometry to know what filters were used, so keywords have been created for them. The keywords in the following table should be entered along with the FILTER header keyword: 
# FILTER: FILTER_CLEAR
In general, the combination of BG39 or BG40 filters (the red cutoff half of a B filter) and a thinned, back illuminated CCD is a good match for the standard WET PMT photometer tubes and sensitivity.

Keywords for the filter
Keyword Matching filter
FILTER_CLEAR No filter used, or one that allowed all visible wavelengths of light through.
FILTER_U Standard Johnson U filter.
FILTER_B Standard Johnson B filter.
FILTER_V Standard Johnson V filter.
FILTER_R Standard Johnson R filter.
FILTER_I Standard Johnson I filter.
FILTER_BG39 BG39 filter, the red cutoff of a Johnson B filter.
FILTER_BG40 BG40 filter, the red cutoff of a Johnson B filter.
FILTER_OTHER A different filter was used. A COMMENT line should be added to the header to indicate the filter used in the observations.
FILTER_NONE No filter was used.

Aperture information

The Quilt9 data format always presumes that the target is in channel 1, the comparison star in channel 2 and the sky in channel 3. This channel ordering is not necessarily going to be the case with a CCD extraction routine, which doesn't really care what the object in the aperture is. For that reason, it is necessary to tell HQ and the reduction software what type of data is in each channel of the data file. This is done with the APTYPE keyword; for the Quilt9 data, the header line would be: 
# APTYPE: DATA COMP SKY
or 
# APTYPE: D C S
This information must be passed to HQ in order for the data reduction to be successful.

The aperture shape isn't necessary for data reduction, but it can be helpful if data artifacts are found. An ANNULAR aperture is likely the ideal, as it gives a sky value for each star that is right next to the star on the CCD, but it isn't always possible to use. When using an ANNULAR aperture, the header should look like: 

# APSHAPE: ANNULAR
# APTYPE: DATA SKY COMP SKY COMP SKY    (etc)
where each SKY channel follows the corresponding DATA or COMP channel.

Detailed aperture information keywords
Keyword Description
APTYPE: aperture organization
DATA The channel with the target light curve. More than one can be in the data, but we usually only have a single target in a WET field.
D Same as the DATA keyword.
COMP A comparison star channel.
C Same as the COMP keyword.
SKY A sky channel.
S Same as the SKY keyword.
APSHAPE: the shape of the aperture
ANNULAR The data aperture is a circle centered on the object, with a sky channel extracted in an annulus around the data aperture.
"shape" Text string describing the shape of the aperture if an annular system is not used.

Time formatting

As we don't have a WET CCD photometry extraction routine, it is necessary for each observing site to use their own extraction software. The drawback of this is that each will use a different format for the time of observation. The reduction software takes this into account, with the caveat that all times that are submitted to HQ are to be Universal Time (UTC) only. HQ cannot accept JD, HJD, MJD, BJED, local observatory time or any other time format.

The times submitted can be formatted either in the FITS standard format or in a 24 hour clock format. Times can be at the beginning or middle of the exposure integration, or the beginning and end times of an integration can be submitted together. These will be used to find the center of the integration, which is the proper time to use (instead of the center of the integration plus readout cycle). It isn't necessary to have an even time between subsequent start time of images, but it is important that the integration time be the same between images.

The formats below must be followed as they are shown in the examples.

Time formatting keywords
Keyword Description Example
PMT_SYSTEM No times are entered, as it is presumed that the times are evenly spaced and the integrations fill the entire time between data points.
FITS_FORMAT Standard FITS format times, as seen in CCD image headers. Time at the beginning of the exposure. 2002-05-12T06:00:00.000
UT_ONLY Universal time only. Time at the beginning of the exposure. 06:00:00.000
MID_EXPOSE_FITS Standard FITS format times, as seen in CCD image headers. Time at the midpoint of the exposure (not the cycle!). 2002-05-12T06:00:05.000
MID_EXPOSE_UT Universal time only. Time at the midpoint of the exposure (not the cycle!). 06:05:00.000
START_END One column of times at the start of exposures, the second at the end. Times are in UT, not FITS format. 06:00:00.000 06:00:10.500

Examples

The following are examples that show how to use the new WET HQ data format. They only show the first three data points, just to give a sample of what the data should looke like.

The minimum data file
This is the format of a file that includes the absolute mimimum information that must be sent to HQ. Note that the APSHAPE keyword is included because this example uses an ANNULAR aperture; it can be left out otherwise. 

# XQEDDATA 
# DATE-OBS:  2004-02-29
# TIME-BEG:  12:32:43.000    
# EXP-TIME:  10.000
# OBSERVER:  Britney Spears 
# OBJKEY: pg1336
# OBSKEY: keck1
# PHOTKEY: SP1 
# APTYPE: DATA SKY COMP SKY COMP SKY 
# APSHAPE:  ANNULAR 
# FILTER: FILTER_BG40
# TIME-FMT:  FITS_FORMAT
2004-02-29T12:32:42.500  12147  8490  13181  8654  11881  9002  
2004-02-29T12:32:55.000  12199  8422  13177  8621  11923  8887 
2004-02-29T12:33:07.500  12307  8448  13123  8633  11874  9028
A more complete example
This example includes almost every keyword in the format. The OBSKEY, PHOTKEY and OBJKEY keywords have been left out in order to show the use of the full format, but they can be included as well. 
# XQEDDATA 
# DATE-OBS:  2004-02-29
# TIME-BEG:  12:32:43.000     
# DATE-END:  2004-02-29
# TIME-END:  21:52:12.000     
# OBSERVER:  Britney Spears 
# RUNNAME: brit-0001.dat
# OBJECT:  PG 1336-018 
# OBJCLASS: sdB 
# RA: 13:38:47.60
# DEC: -02:01:48.7
# EPOCH: 2000.0 
# OBSERVATORY: Keck Observatory, Mauna Kea, HI
# TELESCOPE:  Keck I, 10 meters
# LATITUDE: 19:49:42
# LONGITUDE: 155:28:42
# ALTITUDE: 4160
# PHOTOMETER: CCD prototype 
# PHOTTYPE:  CCD
# ACQSOFT:  Spears 1.0b12 
# EXTRSOFT:  IRAF DAOPHOT 
# OBSERVER:  Bob Dylan 
# NCHAN: 6 
# NPTS: 3562
# FILTER: FILTER_BG40
# APTYPE: ALLSKY 
# APSIZE: 10 
# APTYPE: DATA SKY COMP SKY COMP SKY 
# APSHAPE:  ANNULAR 
# EXP-TIME:  10.000
# READ-TIME:  2.500
# CYC-TIME:  12.500
# DRIFT: 12 
# TIME-FMT:  FITS_FORMAT
# COMMENT:  Oops, I did it again..... 
2004-02-29T12:32:42.500  12147  8490  13181  8654  11881  9002  
2004-02-29T12:32:55.000  12199  8422  13177  8621  11923  8887 
2004-02-29T12:33:07.500  12307  8448  13123  8633  11874  9028

Contact the WETmaster.
Last updated: 03/08/2021 20:21:54