The text below describes
the some of the details and problems with the raw data. It is
mainly informational.
CCD
characteristics and the basic method for reduction of Hectospec
fiber spectra.
Before describing the steps to extract sky-subtracted, wavelength calibrated spectra, here are the data features that will need to be addressed or incorporated by the process.
The spectra are imaged onto the two CCDs such that wavelength runs along columns. There are no fibers that fall in the gap between the two detectors. Because the fibers are arranged in two columns at the entrance of the spectrograph offset in the wavelength direction, alternate spectra at the CCDs are offset in wavelength by about 30 pixels (~40A).
Both CCDs are read out with 2 amplifiers, and recorded as multiextension files, with 4 extensions for the data. Amplifiers 2 and 4 are flipped in X with respect to the pixels in amps 1 & 3, which is taken care of in the reduction. Each of these amplifiers has its own characteristics which must be accounted for, but in the mean, the gains are 1.0 e-/ADU and the readout noise is 2.8e-. The amplifiers have a time-constant problem which results in small, but long tails following large charges. This can cause some problems with flat fields, but is a potentially serious issue for observations where bright objects have faint neighbors. We correct for this in software.
The cosmic ray rate is about 1 event/second over the surface of the CCDs used for the spectra (e.g., for a 300 sec exposure, you would on average see one cosmic ray per spectrum). These can be detected and removed by a number of ways.
There is a noticeable charge diffusion effect on these CCDs whereby electrons created by blue photons diffuse more than those due to red photons. The net result is that the PSF in the blue is 1.2 pixels rms larger than in the red. There is also an overall difference between the two CCDs, at about the 0.05 pixel level. The former effect will not affect sky subtraction (though it might affect some science projects), while the latter will affect sky-subtraction in that sky fibers cannot be used for spectra on the other CCD.
These E2V CCDs have fringing that begins longward of 6500A. The spatial scale of the fringes is about 20 pixels from peak to valley, and the amplitude is about 8%. Removal via dome flats seems to be good to 1-2%.
The amount of light incident on the CCDs is a function of several things along the path from the telescope focal plane to the detectors. The fiber throughput at a fixed wavelength varies among fibers at a level of about +/- 5%, though differences up to 20% occur for some fibers. There is also a dependence of throughput on wavelength among fibers, part of which is fixed, being due to obstructions in the spectrograph beam, but another part appears to vary among exposures and is thus probably due to changes in fiber throughput as the fibers are repositioned or the telescope moved. The combined dependence of throughput on wavelength varies by up to 20% among fibers. We currently cannot remove all of this variable response, thus the quality of sky subtraction is limited to about 4% in the continuum, particularly in the blue.
The sky area accepted by the fibers is dependent on the field angle in the telescope focal plane . The change is from 1.76 sq arcs in the center to 1.62 sq arcs at the edge of the usable focal plane. A correction can be made based on the positions of the fibers on the focal plane. Note that the dome, twilight and arc lamp exposures are taken with the fibers all at the same radius, so the correction is not necessary for those exposures.
The location of the fiber images on the CCDs can wander by about a pixel during the night. This is possibly due to temperature changes in the spectrograph room. Thus, calibrations taking many hours earlier may not be accurate to better than that value. In particular, for better wavelength calibrations, night sky lines should be used to adjust the zero points.
The shutter at the spectrograph end of the fibers causes some vignetting for exposures shorter than 3seconds, in the sense that half of the fibers are exposed longer than the other half. Thus, flats should be exposed for longer than 3 sec.
The red end of about 1/3 of the fibers is contaminated by a light source in the fiber positioner. The pollution begins at about 8500A, and will depend on where the fiber is placed in the focal plane. Currently, I assume that data longward of 8500A is unusable.
Finally, the spectrograph optics produce a faint, ghost pupil image, in the form of a doughnut shaped object centered on the two CCDs. The peak level of this extra light, which is achromatic, is about 1% of the peak level reached in the fiber images. Tests have shown that removing this pupil does not noticeably improve the extracted spectra, and so it is not recommended to bother with removing the ghost at this time.
Spectral
Extraction Recommendations
Here are the detailed steps, in words, of the reduction process. Scripts are used to accomplish these tasks, and will soon be available from the CfA Telescope Data Center (TDC).
1) The raw data are corrected for the amplifier time-constant problem.
2) The files are debiased, trimmed, and corrected for one bad column in the first channel data (IM1). The header parameters for these steps are ok, but not final. Currently, neither the zero frames nor the dark frames seem to be necessary.
3) The different channels are multiplied by small factors to put them on the same gain.
4) The channels are combined (flipping channels 2 & 4 in X), so that the result is a single, 2d image.
5) Dome and twilight flats are combined into one image, using median to remove cosmic rays. (note that scaling must use an image section that contains only a fiber, and no background)
6) A flat field image is made from the dome flat frame via the IRAF task, apflatten. This image has a mean of 1.0 (the spectrum of the dome lights has been removed), and shows the pixel-pixel variations of the CCDs, particularly the fringing (see Fig 1). This frame will be divided into all the other frames. This method appears to work better than extracting the flatfield spectra, because of the wanderings of the images on the CCDs.
7) This flat frame is divided into the combined twilight sky frame, which will be used for the first attempt at a throughput correction. The IRAF program apall is run which extracts the 1d spectra. Wavelength calibration is supplied via the arc lamp image which has been reduced the same way (not described here). The average spectrum of a few central fibers is formed, and divided into all of the 1d spectra. This spectral ratio frame is then fit with a low order spline function. (see Fig 3a) Thus, in theory, division by this frame will correct other frames for variations as a function of fiber and wavelength.
8) Back to the program data now. Cosmic rays are detected via subtraction of multiple images, flagging the high or low pixels. The flagged pixels are interpolated over, and the resultant images are combined via an average (see Fig 2). This method avoids problems with clipping that other programs suffer from, but does require that the data be fairly similar (i.e., clouds will be a problem).
9) The combined frame is divided by the dome flat.
10) The spectra are extracted via apall, using the twilight sky flat as an aperture reference. The apertures are allowed to be recentered en masse using the brightest apertures, to compensate for possible wandering on the CCDs. The spectra are rebinned to the wavelength solution.
11) The spectra are corrected for the different sky areas covered, and divided by the twilight sky correction frame.
12) Since the PSFs are slightly different between the two CCDs, we now separate the spectra into two parts for sky subtraction (1-150 form one group, 151-300 form the other).
13) Early efforts revealed that the twilight sky correction frame did not do everything we had hoped. Thus the next step was developed. The fluxes of 4 night sky emission lines are measured for each fiber. An average value is formed, and the ratio of the fluxes for each spectrum to this average is calculated. This ratio will be another correction applied to each spectrum, in another attempt to place all the spectra on the same flux scale (see Fig 3b). Obvious problems can occur here when very bright objects are observed, or strong source lines appear at the same wavelength as the chosen night sky lines. Some attempt has been made to deal with these, but undoubtedly some spectra will have problems here and must be dealt with manually. At this point, spectra that were supposed have only sky in them are also examined to see if some objects have snuck in. Such spectra are reclassified as objects in preparation for the next step.
14) The IRAF program skysub is run, which computes an average sky from the spectra designated as sky, and subtracts that from all the spectra (see Fig 3c,d).
15) The two separate images are recombined into one.
16) Unassigned fibers have their pixel values set to zero (Fig 5).
IRAF Details
IRAF's
MSCRED package can be used to do the CCD reductions prior to the
spectral
analysis. To support this you must get
from SAO and install the "sao" subdirectory
of "mscdb"
and then configure your parameter
files:
epar
mscred
(instrument = "mscdb$sao/mmto/mmt/hectospec.dat")
CCD instrument file
setinstrument
(directory = "mscdb$sao/") Instrument directory
This
will allow mscred to find the necessary badpixel masks and translate the image types,
etc.
The aperture maps for each exposure
can be found in a file <name>_map. This is a combined map that
will match
the image file created by "imjoining" all 4 fits
extensions. Alternatively, the files
<name>_map1 and <name>_map2 are aperture maps organized by
chip. This information is also stored in FITS / BINTABLE format in extensions 5 and 6
of the original FITS file. The
*_map*
files are derived from that and formatted into IRAF format for use with
APEXTRACT. The columns of the aperture
map are:
aperture beam object ra
dec target fiber platex
platey
where
aperture -
number that is monotonically increasing from 1 to 300
beam
- IRAF convention code 1 object, 0 sky, -1
broken/not-used
object
- sky or object - matches an object
in user catalog
unused - not assigned to an object -
probably parked
ra
- actual fiber
position ra (~
cat ra - but
not exactly)
dec
- actual fiber
position dec (~ cat dec
-
but not exactly)
target
- line number from user catalog
fiber
- fiber number (1-300)
platex
- fiber position in mm
from focal plane center
platey
- fiber position in mm
from focal plane center
If
displaying the image data with DS9 you can overlay the fiber/aperture
labels using
the region file in: mscdb$sao/mmto/mmt/spec/specaps.reg. (You may want to copy this file to your home
directory or other more easily accessible location.)
And you may want to invoke ds9 with the
switches:
"-orient
xy
-rotate 270" which will display the
spectra horizontally (rather than
vertically) and with blue on the left.
The linelist
we use for the henear lamps in our lamp
boxes can be
found in: mscdb$sao/mmto/mmt/spec/spec.linelist
and a sample
wavelength solution can be found in mscdb$sao/mmto/mmt/spec/idcomp.ms
(and comp.ms.fits)
Nice
plots and the linelist can be found at this
web site.
Figure 1. Pixel--Pixel Flat Field, formed from dome flats
Figure 2. 3 exposures of a deep field co-added after cosmic ray removal. Red is at the bottom
Figure 3. The 300 extracted spectra all compressed together and shown in 2d format. Red is at the bottom. From left to right: Throughput correction file derived from twilight sky exposures (note the wavelength dependence), throughput correction derived from night sky emission lines measured on one object frame (no wavelength dependence), object frame before sky subtraction, object frame after sky subtraction.
Figure 4.
Sample extracted spectra (unfluxed)