SMA

SMA Home

Eric Keto's MIR Website

SMA Data Reduction

  • Introduction
  • Getting Started
  • Calibrating SMA Data
  • Getting Help

Introduction

Welcome to Eric Keto's MIR Website.

History of MIR

Millimeter Interferometer Reduction (MIR) is an IDL based software package, originally developed by Nick Scoville at Caltech for use with the OVRO millimeter array. While Nick was working at OVRO on MIR (late 1990's), at the SMA, Eric Keto and Ken Young were developing the SMA's offline and online software that handles the SMA data. Eric Keto designed the SMA raw data format and calibration procedure following MIR and adapted many of the MIR programs for the SMA. In the end, MIR was never used at OVRO, principally because of the merger of OVRO and BIMA, and the SMA inherited responsibility for the MIR package. At the SMA, Chunhua (Charlie) Qi has been responsible for maintaining MIR. Over the years, he has contributed many useful modifications.

The examples and the programs

The MIR cookbook, https://www.cfa.harvard.edu/~cqi/mircook.html, describes the full capabilities of MIR.

The web pages here provide examples illustrating basic calibration with an emphasis on before-and-after graphics showing the effect of each of the calibration steps. The description in the first example includes all the equations used in the SMA flux calibration. This is useful for understanding both the basic procedure as well as the finer points.

The flux calibration shown in these examples uses flux calibration programs that I modified and wrote to provide graphics illustrating the flux calibration procedure. While the mathematics of flux calibration is the same in these programs as in those described in the MIR cookbook, the programs and the procedure are slightly different.

Also my flux calibration programs use new models for the submillimeter fluxes of the asteroids, planets, and their satellites. These new models are those from ALMA's CASA calibration package. While the differences between the old and new models are small in the 230 - 350 GHz range where the SMA operates, the new models provide consistency with ALMA and consistency going forward as calibration of the SMA data with the CASA package becomes more common.

One can access my flux calibration programs by including the directory containing these programs first in order in the IDL path or by copying the programs into the current working directory. Details are provided in the web page accessed from the tab, Calibrating SMA Data, Section, 1. Preliminaries.


Getting Started

Since MIR only calibrates the SMA data, once you are finished the calibration you will need to convert the data from its native SMA/MIR format into a format readable by one of the packages used for interferometric imaging such as AIPS, MIRIAD, or CASA.

There are two conversion programs in MIR.

The program idl2miriad writes a MIRIAD style UV data set. This program requires a shared object library (libidlsmamiriad.so) built specifically for MIR from compiled MIRIAD programs. The location of this shared object library is indicated by the IDL variable e.idl_sav. If you are using our CF or RGLINUX computers, all this is already managed.

Alternatively one can image in AIPS by writing an AIPS style UV-FITS file using the IDL program write_fits or its interactive equivalent mir_fits. In this case no shared object library is needed.

CASA can read AIPS UV-FITS file. We have not yet tested a procedure within CASA for imaging UV data that has been calibrated in MIR.

The sample calibration on this website uses data from June 3, 2010 (100603_02:56:50), which is non-proprietary and therefore directly downloadable.

Calibrating SMA Data

      Overview

A visual calibration procedure developed by Eric Keto for the MIR software.
"Calibrating SMA Data" (pdf version)


First draft: 2013
Updated: November 2015

I developed the data analysis flow described in this example so that the astronomer can see each calibration procedure in the graphics of the MIR package.

The sample calibration on this website uses data from June 3, 2010 (100603_02:56:50), which is non-proprietary and therefore directly downloadable.

eketo@cfa.harvard.edu

 1.  Preliminaries

1.1      Bandpass, Gain, and Flux Calibration

 

There are 3 steps to calibrate SMA data. First, the bandpass calibration corrects for variations as a function of frequency across the bandpass. Second, the gain calibration corrects for variations in time over the night's observation. Third, the flux calibration sets the absolute flux level.

 

All the calibrations, bandpass, gain, and flux, compare observations of a calibrator against a model. Whatever corrections are needed to bring the observations of the calibrator in line with the model are then applied to the observations of the science target. The assumption is that the same corrections needed for the calibrator are the same needed to improve the observations of the science target.

 

1.2      The calibration steps in order

 

What is the correct order for the 3 steps in the calibration process, bandpass, gain, and flux?

 

The gain and flux calibration use the continuum flux of the respective calibrators. The SMA does not measure a broad-band continuum of the correlated power but constructs one by averaging all the spectral line channels together. Therefore, the bandpass calibration which removes variations in phase and amplitude with frequency should be done first to improve the signal-to-noise of the averaging.

 

The gain calibration corrects for variations over time in the response of the instrument including effects of changing sky conditions that affect the measured amplitudes and phases but does not set an absolute value for the flux. The second step of the flux calibration, comparison with planets, sets this absolute value by multiplying all the observed amplitudes by the ratio of the modeled and observed amplitudes of planets or their satellites. Mathematically it does not matter whether we multiply the data by this ratio before or after the gain calibration. I will describe a procedure with the gain calibration done before the flux calibration. This allows us to work with observations of the gain calibrator that are phase-calibrated for better signal-to-noise and for consistency in their estimated fluxes. The motivation for this order should become clearer as the procedure is described.

 

1.3      Improved version of the flux calibration

 

The procedure described here uses versions of the calibration programs that I modified to use models for the brightness of the planets taken from ALMA's CASA software. These models include corrections for spectral line contamination in Neptune and Titan. The modified flux calibration program also plots the observed and modeled visibilities and prints out a detailed list for each baseline so that the astronomer can assess the reliability of the calibration. In the latest version, I used programs taken from the standard MIR distribution in October 2015. The modified programs are compatible to the MIR distribution at least of that date.

 

The modified programs are:

 

flux_cal.pro

flux_primary.pro

gain.pro

flux_measure.pro

 

In addition rad.pro is required to read the planet models.

 

Of course, the planet models themselves are required.

 

Here is a modified setup file that will include the modified programs and the planet models and run the other programs in the standard distribution of MIR. This setup file works on our CF computers.

 

#XDXDIR is the directory with the improved flux calibration programs

setenv XDXDIR /sma/SMAusers/keto/mir/idl/new

#

# Specify the directory with the models of the planets.

setenv PLANET_MODELS_DIR /sma/SMAusers/keto/mir/idl/new/planetmodels

#

#RDXDIR is the directory with all the rest of the MIR programs

setenv RDXDIR /sma/mir 

#

setenv IDL_STARTUP $RDXDIR/idl/idl_startup

#

#These lines are normal idl setup lines except that $RDXDIR/idl has

#been added to the front of the path

#

#Include the XDXDIR directory in the IDL_PATH ahead of the RDXDIR.

#

setenv IDL_PATH \+$XDXDIR/:\+$RDXDIR/idl/pro/:\+$IDL_DIR/lib

setenv IDL_DEVICE X

setenv CAMPUS_LOGIN sma

 

Source the setup file and start IDL as usual.

 

On our CF computers:

source /sma/SMAusers/keto/mir/setup

idl

 

On our RGLINUX computers

source /czdata/keto/mir/setup

idl

 

 2.  Loading the data into memory, assessing the observations, initial flagging

 

 

Read the data from the current working directory

readdata

 

With no arguments, select shows the sources, etc., in the data set

select

 

Shows the data has these sources

saturn

mars

1058+015

PG1119+120

1310+323

      PG1404+226

 

 

You can usually figure out the observing strategy from this plot. (This is plo_var not plot_var.) Select the continuum band for plotting.

 

select, /p, /re, band='c1'

result = plo_var('dhrs','el',frames_per_page=1)

 

Figure 21 Elevation vs time. The purpose of this plot is to understand how the observations were made or the observing strategy. The plot shows two science targets (green) PG1119 and PG1404, each with its own gain calibrator, 1058+0 and 1310+3, and mars which is used for both flux and bandpass calibration.

 

Mars is observed first and is both the bandpass and flux calibrator.

1058+015 is the gain calibrator for PG1119+120.

1310+323 is the gain calibrator for PG1404+226.

1058+015 is usually about 0.5 Jy

1310+323 about the same

Use mars for passband

There is no data for Saturn.

 

 

Plot the system temperature vs time. This uses plot_var instead of plo_var.

plot_var, x='int', frames=9

 

Figure 22 This plot shows that antenna 5 has problems because of the sudden drop in system temperature around integration 400.  Needs some flagging.

 

 

Flag using the integration number to select the data.

 

select, /p, /re,ant=5

select,int=[355,537]

flag,/flag

 

select, /p, /re,ant=5

select,int=[649,697]

flag,/flag

 

select, /p, /re,ant=5

select,int=[775,810],source='PG1119+120'

flag,/flag

 

 

Plot the system temperature again to check on the flagging.

 

select, /p, /re, band='c1'

plot_var, x='int', frames=16

 

Figure 23 Plot the data again to be sure the flagging was done correctly.

 

 

Looks OK now.

 

We have finished the initial assessment and flagging of the data.

 

The next step is to set the flux levels of the data with the chopper wheel calibration.

 

 

 3.  Initial flux calibration using the chopper-wheel calibration method

 

3.1      Chopper-wheel calibration

 

The initial flux calibration is done with the "chopper-wheel" calibration method invented by Bob Wilson and Arno Penzias (Penzias and Burrus, 1973, ARAA, 11, 51). This calibration is determined by the SMA online system before the data is recorded. This method measures the difference in voltage for a single dish telescope or in correlator counts for an interferometer between observations of a room temperature (hot) load inserted in the beam and the cold blank sky. The SMA uses a combination of the two as will be explained in the mathematics of the chopper-wheel calibration. In any case, the working assumption is that the emission of the hot load is that of a black body at the air temperature while the emission of the sky is also that of a black body at the same temperature but multiplied by the hopefully small atmospheric opacity. The ratio of the difference in measured voltages or counts and the difference in the known or modeled emission of the hot load and the sky is the gain or response of the telescope which is used to scale the measured voltage or counts of the science target to the true brightness temperature.

 

The method corrects for the variation in instrumental gain with time and for changes in the atmospheric opacity which may occur as a result of changes in the elevation of the observation or weather conditions. This is a first step to make the response of the interferometer uniform over the course of a night.

 

The scaling determined by the online system is applied in the offline calibration with the command apply_tsys which is always run at the beginning of the data analysis. There are no inputs or options. The calibration is done entirely using data provided by the system. The apply_tsys command also weights the data by the square of the measured system temperature, Tsys2, to improve the signal-to-noise of the averaging of the individual integrations observed over the night.

 

The apply_tsys command also converts the system temperature measured by the online system from a double side band (DSB) definition to a single side band (SSB) definition. These differ by a factor of 2. Receiver temperatures in other SMA documentation generally refer to the lower temperatures in the DSB definition so one has to keep this in mind when comparing the system temperatures before or after apply_tsys to the expected receiver temperatures.

 

3.2      A simple derivation of the method

 

The details of the chopper-wheel calibration are handled by the online system. This derivation of the mathematics is intended to explain how the method accounts for variations in gain and atmospheric opacity to better understand the accuracy of the method.

 

The power measured from the blank sky, Psky, which could be measured in any units can be described as,

where G is a time dependent gain that converts brightness temperature to whatever units, voltage or correlator counts, the system is measuring. This represents the response of the entire telescope. The factor B, between 0 and 1, is the fraction of the beam that is looking at the sky while the factor (1-B) is the fraction of the beam not looking at the sky, for example, looking at the ground (spill over) or the support legs of the secondary mirror. The optical depth and brightness temperature of the sky are τ and Tatm, and Tearth is the temperature of the ground and the antennas. The contribution to the measured power due to receiver noise and other sources of instrumental noise is PRX which we usually refer to in units of temperature, TRX = PRX/G. All the factors vary on the timescale of the observation, so I have not written in an explicit dependence, as for example G(t).

 

If most of the water vapor, the major emitter in the submillimeter, is in the lower atmosphere near the ground, then the atmosphere and the earth may be assumed to be at the same temperature, T, then,

At a single dish telescope, the power received when observing an astronomical source is,


However, at the SMA, the power measured on a source is the correlated power between two antennas. In this case, there is no contribution from the sky or the receiver because the noise power is uncorrelated. At the SMA, the gain G2 is different by a factor of 2. The gain is different because the sky power is measured by a broadband continuum detector which measures the power before the signal is sent to the correlator. At this point the bandwidth includes both sidebands of the receiver. The correlator separates the signals from the two sidebands with each having half the width. Since the power is integrated over the bandwidth, the change in power from a source measured through the sideband separating correlator is half the change in power of the same source measured through the the broad band continuum detector.

with G2 = ½ G.

Define,

for a single dish telescope, and

 

for the SMA. The single dish telescope requires a separate observation of the blank sky usually done by chopping and nodding the antenna.

 

With an additional measurement of the power received from a source of known temperature we can get the scaling necessary to determine the true brightness temperature of our astronomical source. At a single dish telescope this is usually done by observing through a disk divided into quadrants, two open and two blocked by an absorber. Every so often the disk is rotated and the power from the sky and the absorber are measured synchronously. The SMA uses a vane that rotates into the beam. Measurements of the vane are typically made when the antennas are slewing.

 

where the factor of 2 applies to the SMA with G2 = 1/2G but not to a single dish telescope.

 

The SMA online system actually records the system temperature which is defined as,

 

If you like, you can think of Tsys as the sky temperature corrected for atmospheric absorption although this isn't relevant. The definition is useful because Tsys is easily measured with the chopper wheel or vane as in the equation above, and because the brightness temperature of our target source can be found by scaling the observed power by Tsys,

 

where the factor of 2 applies again to the SMA and ΔPsrc is our normalized correlator count. The command apply_tsys absorbs the factor of 2 into the system temperature. So before apply_tsys, Tsys recorded in the data is the double sideband system temperature measured by the SMA broadband continuum detectors. After apply_tsys, Tsys is exactly twice as large. This corresponds to the single sideband system temperature since a source must be twice as hot to produce the same power in half the bandwidth.

 

There is one more step to convert the source brightness temperature to flux in Jy/beam.

The factor Ω/Δ2 is the area of the antenna, 9π m2, so S = 98 Tsrc Jy/beam. However the effective area of the SMA dish is not exactly that of 6m diameter disk. There are inefficiencies in the blockage from the secondary mirror and its support legs, beam spillover, and imperfect reflecting surfaces. The apply_tsys program assumes an efficiency of 75% so that S = 130 Tsrc Jy/beam. The following table from the SMA Engineering Test Program web page shows that this is a pretty good assumption for observations around 230 GHz, but the true efficiency is lower by 8.7% in the 345GHz band. These efficiencies were measured by comparison with the fluxes expected in observations of planets.




Figure 31 Measured aperture efficiencies. The dates of the last measurement refer to the surface accuracy and not the efficiency.
An efficiency of 75% is a good estimate for observations at 230 GHz and 66% at 345 GHz.

 

 

This chopper wheel calibration is a reasonably good calibration. It removes variations in gain and atmospheric opacity over the night and provides an estimate of the brightness temperature. We don't have an estimate of the accuracy of this calibration that is measured with the SMA, but from experience with other telescopes, probably 20% or better.

 

With this explanation, you can assess for yourself the accuracy of the chopper-wheel calibration or at least have a better appreciation of the sources of uncertainty or error. The Tsys is measured at least as often as the SMA slews between the gain calibrator and the science target. So the method corrects for changes in gain and opacity at least this often. In fact, the sky temperature is measured continuously since the power of the science target is negligible compared to the power from the sky. As for the dish efficiency, one would suppose that it is fairly stable in the 230 GHz although perhaps less so at 345 GHz. Still, the maximum variation in the efficiencies of the 6 dishes measured at 345 GHz is only 12%.

 

So maybe the accuracy is 10% or 20%, but certainly not as bad as a factor of 2.

 

The goal of the flux calibration with planets and their satellites is to improve on this.

 

3.3      Applying the chopper-wheel calibration

 

The function apply_tsys will calibrate the data by the chopper wheel method, weight by Tsys2, and double the system temperatures to the single sideband definition.

 

select, /p, /re

apply_tsys

 

Average the data to recalculate the continuum.

 

uti_avgband

mir_save,'mir-save-tsys-applied'

 

That's it.




 4.  Passband calibration

 

The passband calibration is done in two steps, phase and then amplitude.

4.1      Phase only pass band calibration.

 

View the spectrum before calibration.

 

plot_spectra, /norm, source='mars', frames_per_page='16'




Figure 41 Phase and amplitude vs channel for the observation of Mars before the passband calibration. The panels show different baselines. The phase in the upper and lower sidebands (orange and blue) drift in opposite directions vs channel. The amplitudes show the shape of the 104 MHz filters on the individual chunks. The first step is to correct the phase.

 

Usually the MIR plotting programs produce several pages of panels or frames showing the various baselines or spectra. In this tutorial, I am showing just the first page in most cases.








Passband calibration. Phase and amplitude separately. Phase first.

 

select, /p, /re

pass_cal, cal_type='pha', tel_bsl='telescope', refant=5, smooth=8,frames_per_page='16'

 

In the revolving menu type "mars yes" to select Mars as the passband calibrator. The calibration solution derived from the Mars data will ultimately be applied to all the other sources in the menu not selected for use, those marked "NO".

 

Figure 42 Plot produced by pass_cal program showing the variation in phase across the individual 104 MHz chunks. The panels are labeled by "1" for antenna 1, "345" for the receiver, and "sNN" where NN is the number of the individual 104 MHz chunk of the spectrum. The blue asterisks are the data and the red line is the fit to the data. The passband calibration consists of applying the correction determined by the fit to the data. The goal is have the fit follow the significant variations in the phase across each chunk but not follow the noise. Change the length of the smoothing over channel to achieve this.

 

Since the smoothed function follows the data reasonably well.

 

Answer "Yes" to apply the calibration to Mars itself and the other sources in the menu.

 

 

 

 

 

 

 

 

 

 

 

 

 

You can check the pass band calibration by running the calibration function pass_cal again with the same parameters. The plot shows the calibrated data and estimates a new smoothed correction. Don't worry about the edges. These overlap the neighboring filters and won't be used.

 

 

Figure 43 Plot of the phase vs channel after the phase calibration.

 

The plot shows the calibration was successful because the phases are reasonably flat across the filters. Don't worry about edges of the chunks. The chunks are spaced to overlap each other and the edges are never used in final data.

 

After the plotting, be sure to answer "No" when asked whether to apply the pass band correction otherwise the second correction will just add noise to the data. For example, the noise in the second correction is easily seen in jagged red solution for the S13 filter (lower left). Next recalculate the continuum with the corrected phases.

 

select, /p, /re

uti_avgband

 

 

 

 

 

 

 

 

 

 

 

With the phases corrected, run the pass band calibration again to correct the amplitudes.

 

select, /p, /re

pass_cal, cal_type='amp', tel_bsl='telescope', refant=5, smooth=8,frames_per_page='16'

 

Figure 44 Plot of the variation in amplitude vs channel number for each of the individual chunks. This plot shows the variation of the amplitude before the amplitude calibration. The blue histogram shows the data and the orange line shows the fit or essentially the correction. The panels are labeled as explained in figure Figure 42 Plot produced by pass_cal program showing the variation in phase across the individual 104 MHz chunks. The panels are labeled by "1" for antenna 1, "345" for the receiver, and "sNN" where NN is the number of the individual 104 MHz chunk of the spectrum. The blue asterisks are the data and the red line is the fit to the data. The passband calibration consists of applying the correction determined by the fit to the data. The goal is have the fit follow the significant variations in the phase across each chunk but not follow the noise. Change the length of the smoothing over channel to achieve this.

 

The plot shows the typical 104 MHz filter shape. The smoothing follows the data reasonably well so apply the calibration by answering "Yes". Then recalculate the continuum. This applies the solution to Mars and also the other sources in the menu.

 

select,/re,/p

uti_avgband

 

 

 

 

 

Run the pass band calibration with the same parameters to check the correction. Don't worry about messages that the fitting failed. Here we are just using the pass_cal function to plot the data. We won't be applying the correction.

 

 

Figure 45 Plot of the amplitude vs channel after the amplitude calibration.

 

The correction looks OK. Again the drop off at the edge of the chunks can be ignored. Answer "no", do not apply because we already applied this correction. This time we are running pass_cal again just to produce this plot and check the result.

 

 

 

Check the combine phase and amplitude calibration by plotting the spectrum of Mars again. The phases and amplitudes are now flat except for the spikes from the edges of the filters. These can be ignored.

 

plot_spectra, /norm, source='mars', frames_per_page='16'

 

 

 

Figure 46 Plot of the both the phase and amplitude vs channel similar to plot Figure 41 except here after the phase and amplitude calibration. Both are flat except for the spikes on the amplitude at the edges of the chunks.

 

 

 

Check that the spikes really are from the filter edges by plotting the spectrum again with the option ntrim=8 that cuts out the 8 channels on either side of the 128 channel

104 MHz filter band.

 

plot_spectra, /norm, source='mars', frames_per_page='16',ntrim=8

 

Figure 47 Same as figure Figure 46 except using the ntrim=8 option to exclude plotting the channels at the edges of the chunks. Now the spikes in figure Figure 46 are gone.

 

 

This plot also shows different noise levels on different baselines. The longer baselines have lower signal-to-noise because Mars is partially resolved, and the signal strength is less on the longer baselines.

 

 

 

Here is a plot that shows the inverse correlation of flux with baseline length. The amplitude (y-axis) is plotted versus projected baseline length (x-axis).

 

result=plo_var('prbl','ampave',frames_per_page=16)

 

 

Figure 48 Plot of the average amplitude vs projected baseline length. This plot shows that the amplitude is strongest on the shortest baselines meaning that this source, Mars, is at least partially resolved by the SMA baselines. This plot is not really useful for the calibration procedure. Included just to illustrate this point.

 

Baselines 1-6 and 3-6 are the longest and have the lowest amplitudes. Baselines 2-6, 3-8, and 4-5 are the shortest and have the highest amplitudes. The noise in the spectrum is inversely correlated with the amplitude. This plot is not part of the calibration procedure, but is here for illustration.

 

 

We use antenna-based solutions selected by the option, tel_bsl='telescope' for the phase calibration so that phase closure is preserved. We use baseline-based solutions for the amplitude calibration, tel_bsl='baseline'.The observed amplitudes depend on the individual correlator boards as well as the receivers and IF electronics. The antenna-based calibration finds corrections for each antenna by averaging the baselines belonging to each antenna and finding the best corrections as a least-squares solution. This will not produce good results for amplitude calibration as the following plot shows.

 

 

 

 

Figure 49 Plot of the spectrum of a single baseline after antenna-based pass band calibration showing poor amplitude normalization. The variations are due to the different sensitivities of individual boards in the correlator. Always use baseline-based pass band calibration to avoid this. This data is from an observation of 3C454.3, a particularly bright quasar, and the calibration errors are obvious in this high signal-to-noise spectrum. This observation is a different data set than the one illustrated in this example.

With the pass band calibration finished, this is a good place to save the data.

 

mir_save, 'passband_calibrated'





 5.  Gain calibration

 

 

Next, run the gain calibration to remove the variation of phase and amplitude with time.

The observation was set up with two targets and a different gain calibrator for each. For each pair, run the phase and amplitude gain calibration separately.

 

5.1      Gain calibration phase only

 

This data set has two science targets and a different gain calibrator for each.

1058+015 is the gain calibrator for PG1119+120

1310+323 is the gain calibrator for PG1404+226

 

Run the gain_cal procedure on your gain calibrator (quasar) selecting just the phase and not the amplitude for calibration. Include the science targets in the selection before running gain_cal. Exclude the flux calibrator. When your gain calibrator comes up in the rotating (endless loop) menu, it will show a value for its flux. This was assigned by the procedure apply_tsys. If the fluxes are zero (this can happen if the calibration procedure was restarted after apply_tsys with mir_restore and mir_save) exit the loop by hitting return a couple of times and run flux_measure. The flux of the gain calibrator is the reasonable estimate from the chopper-wheel calibration, but we can ignore it for phase calibration.

 

Here is the command to get or update the fluxes if necessary.

 

flux_measure,/scalar

 

Scalar average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

            mars         72.8769    2.0640    35.3     24      11.9        3.02

        1058+015          1.3074    0.0056   232.9    268      82.3        6.10

      PG1119+120          0.1692    0.0007   254.3    480     237.4        5.93

        1310+323          0.9707    0.0046   213.3     71      35.1       10.09

      PG1404+226          0.1750    0.0010   175.2    240     118.7       10.11

 

 

 

 

Start the gain calibration for the first pair.

 

select,/re,/p,source=['1058+015', 'PG1119+120']

 

gain_cal,cal_type='pha',x_var='hours',tel_bsl='telescope',$

refant=4,frames_per_page='16',smooth=2

 

This starts the repeating loop for source selection. Enter the name of the gain calibrator followed by yes as shown in red below. Notice that the program has picked up the flux of the sources from the last execution of flux_measure although the flux is not relevant to the phase calibration. Then hit Enter to exit the loop and start the calibration.

 

% Compiled module: GAIN_CAL.

-------------------------------------------

      Single Band Calibration Mode        

          Main Band Calibration           

-------------------------------------------

% Compiled module: GAIN_INI.

Set which sources will be used as calibrators and set the calibrator fluxes

The flux should be in Jy at the frequency of the observation

These are the sources and their current gain codes

name:  1058+015                gain code: NO   flux (Jy):       1.3074342

name:  PG1119+120              gain code: NO   flux (Jy):       0.1694476

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

or hit Return if all the sources are correctly specified

: 1058+015 yes

These are the sources and their current gain codes

name:  1058+015                gain code: YES  flux (Jy):       1.3074342

name:  PG1119+120              gain code: NO   flux (Jy):       0.1694476

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

 

 

 

Figure 51  Plot of the phase and coherence vs time for observations of the gain calibrator before the phase calibration. The coherence can be ignored.

 

Massive phase jump. This looks like pointing data in the middle of the time range. When you are done plotting, you will have the option to apply the calibration or not. I want to flag the pointing data, so exit from gain_cal by entering NO as shown in red below to not apply the calibration

 

EXITING FROM THE INTERACTIVE MODE

% Compiled module: GAIN_STORE.

% Compiled module: CAL_STORE.

% Compiled module: DAT_COMB_SEP.

Stacking gain curves to the ca structure

% Compiled module: PLO_PLID_REL.

Apply gain solution? [NO <YES>]:  No

NO: nothing done

IDL>

 

 

 

Flag the data by integration time. Pointing data have less than 10 seconds time on each integration, so the following filter will select all the pointing data.  

 

result=dat_filter(s_f,' "integ" lt "10" ')

flag,/flag

select,/re,/p,source=['1058+015', 'PG1119+120']

 

 

 

Also the solution, the red line in the plot above, is not following the data very well, for example on antenna 5, lower left corner where the phase shows some variation between the hours of 3 and 6. So run the amplitude calibration again with a shorter smoothing time, one half hour to better follow the variations in phase with time.

 

gain_cal,cal_type='pha',x_var='hours',tel_bsl='telescope', $

        refant=4,frames_per_page='16',smooth=0.5

 

 

Figure 52  Phase vs time, same as figure Figure 51, but after flagging and shortening the smoothing time. The red line now looks like a good fit to the data.

 

Looks good. Now the red line of the fit follows the wiggles between the hours 3 and 6 in the phase of antenna 5. Exit the plotting mode and then answer Yes to apply this calibration.

 

EXITING FROM THE INTERACTIVE MODE

Stacking gain curves to the ca structure

Apply gain solution? [NO <YES>]:  yes

YES: apply gains

Applying Gains

% Compiled module: CAL_APPLY.

% Compiled module: UTI_INTERP.

IDL>

 

 

uti_avgband

 

Compare the vector and scalar averages of the fluxes.

 

flux_measure,/vector

Vector average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1058+015          1.5041    0.0051   294.7    145      71.7        5.95

      PG1119+120          0.0011    0.0010     1.1    480     237.4        5.93

 

flux_measure,/scalar

Scalar average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1058+015          1.5662    0.0051   310.1    145      71.7        5.95

      PG1119+120          0.1694    0.0007   251.6    480     237.4        5.93

 

Now the scalar and vector averages for 1045+015 are less than 6% different, and we are done with the phase calibration using the gain calibrator.

 

5.2      Setting a representative flux value for the gain calibrator

 

Now we need to set the flux of the gain calibrator. Run gain_cal again, this time for amplitude but do not apply the results. We just want to look at the plot of the measured amplitude with time over the course of the night. Hopefully, this is flat. If not, and this is unusual, the chopper-wheel calibration has failed to correct for the variation in gain with time. The gain_cal procedure will fix this, but it leaves us with the question of what flux we should use for the quasar in combination with the flux calibration. There are at least three choices.

 

The usual case would be to use the average conditions over the whole night. However, the observation of the flux calibrator was done at a particular time and elevation. Two possibilities are to restrict the comparison in the flux calibration step to either this time or this elevation. Which is the correct choice among the three? Again, the cases where the chopper-wheel calibration fails badly are not common, and I do not have an actual example. Therefore I have drawn three hypothetical cases by hand, shown in the figures below. All three figures plot the amplitude of the quasar (gain calibrator) versus time.

 

Figure 53 Case 1: The gain (response) of the system is stable over time or stable enough that the chopper-wheel calibration is able to correct any trends or drifts. In this case the amplitude of the quasar is constant. We can use the average flux over the night.

 

Figure 54 Case 2: The gain varies with elevation so that the flux of the quasar appears to become brighter and then fainter from rise to set. Hypothetically, the gain and the calibrator flux could increase with elevation as the atmospheric optical depth decreases as the path length through the atmosphere decreases. The elevation of course varies with time as the target rises and sets. In this case, we might select observations of the quasar at the same elevation as the observation of the flux calibrator. Here Tcal is the time when the gain calibrator is at the same elevation as the flux calibrator.

 

 

Figure 55 Case 3: The gain varies, but not obviously with elevation.  We might or might know what causes this, but we could select observations of the quasar around the time that the flux calibrator was observed.

 

At this point, we are not saying that the true flux of the quasar is the flux selected by one of the methods above. We are going to correct this flux to the true flux in the flux calibration steps coming up in a moment. In other words, if the correction in the flux calibration step is appropriate only at some elevation or some time, then if we correct the flux that the quasar had at that same elevation or time, then it will have its true flux after the correction. Because the same corrections are applied to the target, then the target will also have its true flux after the corrections.

 

 

In any case, whatever flux you decide to use in the amplitude gain calibration, here is how it is done.

 

Use the select or dat_filter commands to select a set of observations of the gain calibrator quasar either around the elevation of the flux calibrator, or the time it was observed, or all the observations of the gain calibrator over the whole night. Run the command flux_measure to get an estimate of the flux of the quasar with this selection. The numerical value in Jy/beam is relying for the moment on the chopper-wheel calibration done by apply_tsys. This command also applied the same calibration to the amplitudes of the target. So the calibrator and target fluxes are consistent

 

The command flux_measure has an option for either the /scalar or /vector average of the amplitudes.The gain_cal program scales the amplitudes so that the scalar average matches the input flux. Thus you must use the /scalar option. This is the default. We are not saying that the true flux of the quasar is given by the scalar average. This is just what the software uses in this calculation.

 

5.3      Gain calibration for amplitude

 

Let's see how this works on our example data set

 

The antenna based amplitude calibration does not work. This is simply a bug in MIR. So we will use baseline based calibration.

 

gain_cal,cal_type='amp',x_var='hours',tel_bsl='baseline',$

refant=4,frames_per_page='16',smooth=1

 

Figure 56 Amplitude vs time for different baselines and sidebands. Plot shows the amplitudes before the amplitude calibration. The red line is a fit to model the variation in amplitude over time.

 

These amplitudes vary a lot. They decline for several hours, then increase after the observations were interrupted to optimize the pointing. Perhaps the new pointing offsets successfully centered the primary beams on the gain calibrator, and the fluxes after the pointing are a better estimate of the true flux of the quasar. I can't think of a better hypothesis, so use the select command and flux_measure to average the amplitudes of the quasar from 6.5 hours to the end of the observations at 9 hours.

 

Also, I don't like fit of the smoothed function where it crosses the time when the pointing data was taken. I don't want to force the quasar amplitudes to follow the red line here because this would force the data on the science target to follow the same correction. I think the transition should be a jump from lower to higher fluxes rather than ramp. I will flag all the data in this transition period, calibrator and target source.

 

First, get out of the gain_cal program. At the prompt, answer "NO", do not apply the calibration.

 

Flag the data from from 6.1 to 6.4. The MIR header variable is dhrs.

 

select,/re,/p

result = dat_filter(s_f,' "dhrs" gt "6.1" and "dhrs" lt "6.4" ')

flag,/flag

 

 

Run the selection again to get the average flux of the quasar after 6.5 hours. Remember to use the scalar average which is default.

 

This select command clears the previous selection, and the dat_filter command restricts the time.

 

select,/re,/p

result = dat_filter(s_f,' "dhrs" gt "6.5" ')

 

IDL> flux_measure,/vector

 

Vector average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1058+015          1.7780    0.0062     289.0     56      27.7        7.73

      PG1119+120          0.1757    0.0011     163.9    204     100.9        7.60

        1310+323          0.9707    0.0046     213.3     71      35.1       10.09

      PG1404+226          0.1750    0.0010     175.2    240     118.7       10.11

IDL>

 

 

 

Run the gain calibration program after first selecting the gain calibrator and the science target. This time, in the rotating endless-loop menu, type in the value for the amplitude of the gain calibrator that was just obtained with flux_measure. This value may already be there on the line for the gain calibrator. If not, type it in like this. Also shown in red below.

 

1058+015 YES 1.7780

 

 

IDL> select,/re,/p,source=['1058+015', 'PG1119+120']

Here are the data selection criteria

 ("source" like "1058+015" or  "source" like "PG1119+120")

     1196286 passed in filter

IDL>

IDL> gain_cal,cal_type='amp',x_var='hours',tel_bsl='baseline',$

IDL> refant=4,frames_per_page='16',smooth=0.5

-------------------------------------------

      Single Band Calibration Mode        

          Main Band Calibration           

-------------------------------------------

Set which sources will be used as calibrators and set the calibrator fluxes

The flux should be in Jy at the frequency of the observation

These are the sources and their current gain codes

name:  1058+015                gain code: YES  flux (Jy):       1.3074342

name:  PG1119+120              gain code: NO   flux (Jy):       0.1757000

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

or hit Return if all the sources are correctly specified

: 1058+015 YES 1.7780

These are the sources and their current gain codes

name:  1058+015                gain code: YES  flux (Jy):       1.7780000

name:  PG1119+120              gain code: NO   flux (Jy):       0.1757000

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

or hit Return if all the sources are correctly specified

 

 

This will force all the amplitudes of the gain calibrator to this value and apply the same correction to the data of the science target. Again, as explained in the section above, we are not saying, yet, that this is the final true value for the flux of the quasar. It would be if we stay with the calibration set by the chopper-wheel method. However, we will still run the flux calibration later for a final correction based on the observation of Mars.

 

 

Figure 57 Amplitude vs time same as Figure 56 but after flagging the data between 6.1 and 6.4 hours.

The fit of the red line looks OK. No bad data. So, answer Yes, at the prompt to apply the solution.

 

Then run gain_cal again to check the correction. This time we use type='amp,pha' to check both the phase and amplitude calibration. I use tel_bsl='telescope' (antenna based) to reduce the number of plots.

 

 

Figure 58 Amplitude and phase of the gain calibrator after calibration

That finishes the gain calibration for the first calibrator-target pair. Run the same procedure on the next pair.

 

5.4      Phase and amplitude on the next calibrator-target pair

 

select,/re,/p,source=['PG1404+226','1310+323' ]

 

Need a flux for the gain cal.

 

Just for interest or to make sure the sources have fluxes, you can run flux_measure to compare the scalar and vector averages.

 

IDL> flux_measure,/vector

 

Vector average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1310+323          0.1185    0.0137   8.7     71      35.1       10.09

      PG1404+226          0.0021    0.0015   1.4    240     118.7       10.11

IDL> flux_measure,/scalar

 

Scalar average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1310+323          0.9707    0.0046     213.3     71      35.1       10.09

      PG1404+226          0.1750    0.0010     175.2    240     118.7       10.11

 

 

Now run the gain calibration for phase, and enter the calibrator name and yes in the menu as shown in red below.

 

gain_cal,cal_type='pha',x_var='hours',tel_bsl='telescope',$

refant=4,frames_per_page='16',smooth=1

 

-------------------------------------------

      Single Band Calibration Mode        

          Main Band Calibration           

-------------------------------------------

Set which sources will be used as calibrators and set the calibrator fluxes

The flux should be in Jy at the frequency of the observation

These are the sources and their current gain codes

name:  1310+323                gain code: NO   flux (Jy):       0.1184527

name:  PG1404+226              gain code: NO   flux (Jy):       0.0021214

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

or hit Return if all the sources are correctly specified

: 1310+323 yes

These are the sources and their current gain codes

name:  1310+323                gain code: YES  flux (Jy):       0.1184527

name:  PG1404+226              gain code: NO   flux (Jy):       0.0021214

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

or hit Return if all the sources are correctly specified

 

 

Figure 59 Phase vs time same as Figure 51 but for the second gain calibrator in the data set.

 

Looks OK.

 

 

Yes, apply to the data. Check the flux measurements again.

 

 

IDL> flux_measure,/vector

 

Vector average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1310+323          0.8775    0.0054   163.0     71      35.1       10.09

      PG1404+226          0.0027    0.0015     1.8    240     118.7       10.11

IDL> flux_measure,/scalar

 

Scalar average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

        1310+323          0.9707    0.0046   213.3     71      35.1       10.09

      PG1404+226          0.1750    0.0010   175.2    240     118.7       10.11

IDL>

 

The vector flux on the gain calibrator is now much closer to the scalar flux. The science target has not improved, but this source turns out to be a non-detection. So there is no correlated flux. Since we just ran flux_measure,/scalar with the scalar average, this is the value stored as the flux of the gain calibrator, and this is what we need for amplitude gain calibration.

 

                                                                             

 

Now the baseline-based gain amplitude calibration for the second pair.

 

gain_cal,cal_type='amp',x_var='hours',tel_bsl='baseline',$

refant=4,frames_per_page='16',smooth=1

 

 

Figure 510 Amplitude vs time, same as Figure 56, but for the second gain calibrator in the data set.

 

The amplitudes for this quasar are constant with time in the sense of not drifting, so we can use the average amplitude that was just set by flux_measure. This is the value that was already shown in the menu when we ran gain_cal. The fit of the red line to the data looks fine. So we can apply these corrections.

 

Answer, Yes, to apply the corrections.

 

Now run the gain cal again with both amplitude and phase to check the results for the second pair of target and calibrator.

 

gain_cal,cal_type='amp,pha',x_var='hours',tel_bsl='telescope',

refant=4,frames_per_page='16',smooth=1

 

 

Figure 511 Phase and amplitude vs time for the second gain calibrator after the phase and amplitude calibration.

 

 

Looks, OK. Answer No, do not apply. This last run of gain_cal was just to produce this plot as a check.

 

 

 

Now we finished the gain calibration for both our science targets.

 

select,/p,/re

mir_save,'gain_calibrated'

 

 

 

 

 6.  Flux calibration

 

The flux calibration program is easier to run than it is to describe. The concept is perfectly simple. Find the correction as the ratio of modeled and observed fluxes, but there are details in the modeling.

 

6.1      Models of Planetary Brightness temperature

 

The models for the planets and their satellites include data on the brightness temperature of the body and the apparent size from Earth. I use the models for the brightness temperature from ALMA's CASA software. The apparent size of the planet is computed by the SMA's guidance software and included among the header variables for each observation.

 

The figure below shows several models for the brightness temperature of Uranus. The red or pink crosses show a model from the old OVRO MMA which I included when I set up the SMA's flux calibration programs in 2003. This was later replaced with the model shown by the blue diamonds, and later with a functional fit shown as the green dashed line. The ALMA-CASA model is the white line. At 227 GHz, the old MMA model and its later replacement differ by 12%. The more recent functional fit and the ALMA model agree quite well within the SMA's operating band of 200 – 400 GHz. This history provides some indication of the uncertainties in the model of brightness temperature.

     

 

Figure 61 Comparison of 4 models for the brightness temperature of Uranus plotted as temperature versus frequency. See text above.

 

 

 

 

 

 

 

 

The ALMA-CASA models for Neptune and Titan (shown in the two figures below) include significant contamination from spectral lines, particularly CO. Uranus as shown in the figure above is relatively unaffected.

 

Figure 62 The ALMA-CASA model of the brightness temperature versus frequency for Neptune

 

 

 

 

Figure 63 The ALMA-CASA model of the brightness temperature versus frequency for Titan.

 

 

The effect of the spectral lines, principally CO, are included in the models of Neptune and Titan as shown in the figures below.

 

Figure 64 Model of the brightness temperature of Neptune as a function of frequency. The ALMA-CASA model for Neptune is shown by the white curve with the red crosses indicating the individual frequency-temperature pairs. My flux calibration software interpolates a set of points inside the bandwidth (vertical yellow lines) of the observation and integrates to the average shown by the horizontal green line using the trapezoidal rule.

 

The next figure shows the model for Titan for the same observation. Very preliminary results show agreement deriving the fluxes of quasars using observations of Neptune and Titan and these two models.

 

Figure 65 Model of the brightness temperature of Titan as a function of frequency. The interpolation within the flux calibration program (shown on the plot above Figure 64 for Neptune) is not shown here although the flux calibration program corrects for this contamination in exactly the same way.

 

 

The brightness temperature is converted to flux following the blackbody formula.

 

flux= (1.088791135e7)*freq^3*radius^2/(exp(0.047992*freq/temp)-1)

 

Neptune versus Titan

 

Use Uranus for flux calibration if possible or one of the Galilean moons.

 

My original implementation of the flux calibration software (2001) did not have an accurate brightness model for Neptune. For many years Neptune was not considered a good calibrator for the SMA. This is no longer the case with the ALMA-CASA models, but the prejudice still exists. While the broad CO absorption line in Neptune is more significant relative to the continuum than is the emission line in Titan, Neptune is considerably stronger, 20 Jy/beam versus 1.5 Jy/beam at 230 GHz. Assuming that that the ALMA-CASA model of Neptune is adequate, it might be preferred over Titan because of its stronger flux and therefore better signal-to-noise.

 

 

6.2      Visibilities

 

The planets are at least partially resolved by the SMA even in its more compact configurations. The range of apparent sizes of the three planets useful as flux calibrators is:

 

Mars     3.5" – 25.0"

Uranus 3.3" – 4.0"

Neptune 2.2" – 2.4"

 

 

The comparison of the modeled and observed fluxes takes this into account. The planets are assumed to be circular disks with apparent sizes given by the SMA's internal ephemeris, and their Fourier transforms are computed as 2-dimensional sinc functions. The flux measured on each of the SMA's baselines is then the product of the flux given by the blackbody model for the planet and the response of the interferometer on the spatial (angular) scales of the baselines, often called the visibility function, which is in this case the sinc function.

 

For example, suppose we are observing Mars when the apparent size is 24" in diameter. The 1D Fourier transform of a rectangle extending ± 12" is

 

FT(Π(12")) = Sinc[ π (1/3600 π/180 12") (B/λ))

 

where B/λ is the baseline length measured by the wavelength of the observation. Of course, the system does not observe negative flux, so use the absolute value of the sinc function.

 

 

Figure 66 The visibility function for Mars with an apparent diameter of 24". The x-axis is baseline length in units of 1000 wavelengths, kilo-lambda. If the observing frequency is 1 mm, the units of the baseline length are meters. The dashed line shows the Sinc function, and the solid line shows the absolute value which is what the interferometer measures.

 

 

 

6.3      Running flux_cal for flux calibration

 

The first step of the flux calibration was the chopper-wheel calibration which is already done. The second step is the comparison of the observed and modeled fluxes of a planet or satellite. The first step in this second phase of the flux calibration is to correct the phases of the observation of the planet.

 

Use the progam gain_cal as before except that this time we set the keyword non_point. Use the select command to select only the flux calibrator 'mars' and run gain_cal like this. Normally the system will recognize Mars as a calibrator and the gain code will be set correctly. If not, set it by hand by entering "mars yes" at the prompt. Be sure to spell mars the same way it appears in the menu. Source names are case sensitive. The measured flux of the calibrator is not relevant as long as it is not zero. You can use flux_measure to set the flux if necessary.

 

Be sure not to include amplitude in the gain_cal because this would force the amplitudes of the flux calibrator to agree with the flux value that shows in the rotating menu. This flux value shown in the menu was derived earlier in the chopper-wheel calibration procedure. So in this case, the ratio of the modeled and observed amplitudes would be one and the flux calibration would result in no correction.

 

 

IDL> select,/p,/re,source='mars'

IDL> gain_cal, cal_type='pha', x_var='int', tel_bsl='telescope', $

refant=5, frames_per_page='8', /non_point

-------------------------------------------

      Single Band Calibration Mode        

          Main Band Calibration           

-------------------------------------------

Set which sources will be used as calibrators and set the calibrator fluxes

The flux should be in Jy at the frequency of the observation

These are the sources and their current gain codes

name:  mars                    gain code: YES  flux (Jy):      72.8769073

Enter source, cal code, and if cal, flux in Jy, eg: 3C273 YES 3.1

or hit Return if all the sources are correctly specified:

 

Figure 67 Phase vs integration in the observation of Mars. The different panels show averages for different antennas and sidebands. The flux_cal program will correct the phases to a model of the visibilities of the planetary disk.

 

Finish by examining the plots, and if reasonable, applying the calibration to correct the phases to the disk model. This will only affect Mars since this was the only source selected by the select command before running gain_cal.

 

We will not calibrate the flux calibrator for amplitude because this would force all the observed amplitudes to whatever value we set in the menu and the calibration would be meaningless.

 

Instead, go ahead and run the flux_cal program. First we have to include the other sources that should have the amplitude corrections applied to them. We are going to include the 2 targets and the 2 gain calibrators. We won't use the flux-calibrated gain calibrators, but it is good to know their fluxes for reference.

 

 

IDL> select,/p,/re,source=['mars','1058+015','PG1119+020','1310+323','PG1404+226']

IDL> flux_cal,minvis=0.4

Running Eric Keto's version of flux_cal with the ALMA-CASA planet models

Current Table

#   Source     Flux Cal.           Flux(Jy)

      mars       PRIMARY            73.

  1058+015            NO            1.7

  1310+323            NO           0.88

PG1119+020            NO         0.0021  

PG1404+226            NO         0.0027

Enter source, new flux cal code:

Use p, P, or 1 as code for primary flux cal

Use s, S, or 2 as code for secondary flux cal

Use n, N, NO, or 0 to deselect a calibrator

or hit Return if all the sources are correctly specified

:

Found primary cal mars

Running Eric Keto's version of flux_primary with the ALMA-CASA planet models

Opening file  /sma/SMAusers/keto/mir/idl/new/planetmodels/Mars_Tb.dat

Using ALMA model for Mars

number of data selected for flux cal           96

# USE    SOURCE    Vis*FLUX     SFact   SB   INT BSL    UVDIST   VIS   TSSB     COH    EL

    1      mars      281.86      1.43   u    24  2-5      20.5  0.62    252    0.29  80.2

    1      mars      281.88      1.30   u    23  2-5      20.5  0.62    253    0.29  80.1

    1      mars      281.90      1.32   u    22  2-5      20.5  0.62    254    0.29  80.0

    1      mars      281.92      1.45   u    21  2-5      20.5  0.62    257    0.29  79.9

    1      mars      281.94      1.30   u    20  2-5      20.5  0.62    257    0.29  79.8

    1      mars      281.96      1.32   u    19  2-5      20.5  0.62    262    0.29  79.8

    1      mars      281.98      1.33   u    18  2-5      20.5  0.62    254    0.29  79.7

    1      mars      282.00      1.33   u    17  2-5      20.5  0.62    253    0.29  79.6

    1      mars      282.02      1.31   u    16  2-5      20.5  0.62    258    0.29  79.5

 

                             --- / lots of similar lines /---

 

     1      mars      290.37      2.23   l    16  4-5      18.4  0.68    265    0.36  79.5

    1      mars      290.38      1.42   l    17  4-5      18.4  0.68    260    0.35  79.6

    1      mars      290.38      1.39   l    18  4-5      18.4  0.68    262    0.36  79.7

    1      mars      290.39      1.38   l    19  4-5      18.4  0.68    270    0.35  79.8

    1      mars      290.40      1.37   l    20  4-5      18.4  0.68    266    0.36  79.8

    1      mars      290.41      1.40   l    21  4-5      18.4  0.69    265    0.35  79.9

    1      mars      290.42      1.37   l    22  4-5      18.4  0.69    263    0.35  80.0

    1      mars      290.43      1.35   l    23  4-5      18.4  0.69    261    0.35  80.1

    1      mars      290.44      1.53   l    24  4-5      18.4  0.69    260    0.36  80.2

# USE    SOURCE    Vis*FLUX     SFact   SB   INT BSL    UVDIST   VIS   TSYS     COH    EL

Maximum measured coherence  for mars is      0.969351

Maximum measured visibility for mars is      0.685077

Minimum allowed coherence        0.00000

Minimum allowed visibility      0.400000

 

  Flux scaling factor for each sideband:

  l    1.33E+00   u      1.48E+00

 

Stacking gain curves to the ca structure

 

 

Ready to apply scaling factors. Default is NO.

Apply Flux Calibration? [NO <YES>]: 

 

Here Mars is a primary flux calibrator. Primary means that the program will model the flux of Mars by calculating its brightness temperature and its visibility function. If we had no primary flux calibrator, we could use a secondary flux calibrator. In this case, the program would assume that the flux is given by value set in the menu and that the flux is a point source. The latter means that the visibility function is uniquely 1. The SMA has no database of secondary flux calibrators so this option is generally not useful.

 

The flux_cal program produces a table, a plot, and calculates the flux correction to be applied to each sideband. The table is written to the current working directory as flux_cal_visibility_data.txt. Part of the table is written to the screen.

Explanation of the columns:


1

USE

This indicates whether this integration is uses in the calculation of the flux correction. Only the integrations-baselines selected are printed to the screen so all the numbers are 1. The file flux_cal_visibility_data.txt has all the integrations and baselines. Those not selected have a 0 in this column.

2

SOURCE

In principle, the program allows for multiple flux calibrators. A lot of SMA data sets have only one flux calibrator, so this option has not been well tested.

3

Vis*FLUX

This is the visibility function times the total flux and the ordinate axis on the plot.

4

SFact

This is the correction factor for this integration-baseline which is the ratio of the modeled to observed flux.

5

SB

Sideband

6

INT

The number or index of the integration. Might be useful for flagging.

7

BSL

The antenna pair.

8

VIS

The visibility function from 0 to 1. The default minimum for an integration-baseline pair to be included in the flux calibration is 0.15. This can be changed with the argument minvis=. In this example, the minimum visibility is set to 0.4.

9

TSSB

The system temperature.

10

COH

The coherence of the data. This function was defined at OVRO as the ratio of the vector averaged flux over the scalar averaged flux averaged over time. The SMA data generally does not have enough time samples for a meaningful statistic so the definition was changed to the ratio averaged over channel or frequency. This new statistic has been described as a "not entirely useless measure of data quality". Accordingly, an integration-baseline pair is included in the average if its coherence is "not entirely zero". In other words, the default value for the minimum value of the coherence is zero. This can be changed with the option mincoh=.

11

EL

The elevation of the antennas at the time of the integration.

 

So one can look at the table on the screen or in the text file flux_cal_visibility_data.txt to see if the results are acceptable.

 

All this data is summarized in a plot produced by the flux_cal program shown in the figure below.

 

The red diamonds are the model visibilities for the projected baseline lengths of the SMA observation. These are calculated as above. There are two sets of red diamonds, one set for each sideband. Because the average wavelength of each sideband is slightly different, so are the projected baselines measured in cycles (kilo-lambda). The model amplitudes are also slightly different because of the conversion from the model brightness temperature of the planet to flux density in Janskys. The blue and yellow asterisks show the amplitudes of all the visibilities measured by the SMA. The blue asterisks indicate the visibilities included in the calculation of the flux correction. These correspond to the 1 in column 1 of the table. The yellow asterisks are the visibilities not selected. These have a 0 in the first column of the data file written to disk. These are not written to the screen. The model (red diamonds) is not a fit to the data. It shows the expected flux. The flux calibration consists of finding the constant (one for each sideband) required to shift the observed visibilities up or down to best fit the model.

 

 

Figure 68 Modeled (red diamonds) and observed (blue and yellow asterisks) visibilities for the flux calibrator Mars. The blue visibilities are those included in the calculation of flux correction. The minimum value for inclusion is 0.4.

 

In this example, the minimum value of the visibilities included in the calculation is 0.4. This is based on the modeled visibility not the observed. This data is problematic because the highest visibility data with presumably the best signal-to-noise is inexplicably low. The suggested corrections for the two sidebands, 1.33 and 1.48 are at odds with the chopper-wheel calibration. The correction factor is unexpectedly large. A large correction could result from bad pointing on the flux calibrator which might or might not indicate bad pointing in the rest of the observations. However, here The data in yellow, those not selected to calculate the correction, suggest that the correction based on the short baselines alone (blue points – high visibility) is too large.

 

Let's try the default value for the minimum visibility of 0.15

 

IDL> flux_cal          

Running Eric Keto's version of flux_cal with the ALMA-CASA planet models

Current Table

#   Source     Flux Cal.           Flux(Jy)

      mars       PRIMARY            73.

Enter source, new flux cal code:

Use p, P, or 1 as code for primary flux cal

Use s, S, or 2 as code for secondary flux cal

Use n, N, NO, or 0 to deselect a calibrator

or hit Return if all the sources are correctly specified

:

Found primary cal mars

Running Eric Keto's version of flux_primary with the ALMA-CASA planet models

Opening file  /sma/SMAusers/keto/mir/idl/new/planetmodels/Mars_Tb.dat

Using ALMA model for Mars

number of data selected for flux cal          336

# USE    SOURCE    Vis*FLUX     SFact   SB   INT BSL    UVDIST   VIS   TSSB     COH    EL

    1      mars       83.02      0.69   u     1  1-3      34.7  0.18    359    0.97  78.0

    1      mars       83.06      0.70   u     2  1-3      34.7  0.18    358    0.97  78.1

    1      mars       83.10      0.69   u     3  1-3      34.7  0.18    358    0.97  78.2

    1      mars       83.13      0.69   u     4  1-3      34.7  0.18    350    0.97  78.3

    1      mars       83.17      0.65   u     5  1-3      34.7  0.18    352    0.97  78.4

    1      mars       83.20      0.65   u     6  1-3      34.6  0.18    350    0.97  78.5

    1      mars       83.24      0.70   u     7  1-3      34.6  0.18    348    0.97  78.6

    1      mars       83.28      0.76   u     8  1-3      34.6  0.18    347    0.97  78.7

 

                                    --- / ---

 

    1      mars      290.38      1.42   l    17  4-5      18.4  0.68    260    0.35  79.6

    1      mars      290.38      1.39   l    18  4-5      18.4  0.68    262    0.36  79.7

    1      mars      290.39      1.38   l    19  4-5      18.4  0.68    270    0.35  79.8

    1      mars      290.40      1.37   l    20  4-5      18.4  0.68    266    0.36  79.8

    1      mars      290.41      1.40   l    21  4-5      18.4  0.69    265    0.35  79.9

    1      mars      290.42      1.37   l    22  4-5      18.4  0.69    263    0.35  80.0

    1      mars      290.43      1.35   l    23  4-5      18.4  0.69    261    0.35  80.1

    1      mars      290.44      1.53   l    24  4-5      18.4  0.69    260    0.36  80.2

# USE    SOURCE    Vis*FLUX     SFact   SB   INT BSL    UVDIST   VIS   TSYS     COH    EL

Maximum measured coherence  for mars is      0.969351

Maximum measured visibility for mars is      0.685077

Minimum allowed coherence        0.00000

Minimum allowed visibility      0.150000

 

  Flux scaling factor for each sideband:

  l    1.01E+00   u      1.08E+00

 

Stacking gain curves to the ca structure

 

 

Ready to apply scaling factors. Default is NO.

Apply Flux Calibration? [NO <YES>]: 

 

And the plot is below.

 

Figure 69 Modeled and observed visibilities of the flux calibrator Mars. The minimum acceptable visibility is 0.15.

 

Now the suggested correction are 1% and 8% for the lower and upper sidebands. What should you do? Apply the correction or stick with the chopper-wheel calibration? I'm not sure. I would probably nudge them up as suggested. Answering yes to the prompt applies the correction and produces your final average fluxes.

 

Ready to apply scaling factors. Default is NO.

Apply Flux Calibration? [NO <YES>]:  y

YES: apply flux cal

Re-calculating continuum fluxes

Averaging all filtered spectral bands to regenerate continuum ...

Done!

Calibrated fluxes:

 

Scalar average for band c1

        Source          Flux        StdDev    SNR     Nscans  On source   Mean UT

                        (Jy)        (Jy)                      time (min)  (hours)

            mars         76.0275    2.1492    35.4     24      11.9        3.02

        1058+015          1.8011    0.0035   516.9    133      65.8        5.93

      PG1119+120          0.1988    0.0008   246.4    480     237.4        5.93

        1310+323          0.9157    0.0030   301.1     71      35.1       10.09

      PG1404+226          0.1688    0.0010   173.9    240     118.7       10.11

 

 

 

The default minimum for the value of the visibility function, 0.15, rejects all the points beyond the first null. There is nothing wrong in principle in including some of them. Of  course, you cannot apply flux calibration corrections a second time after already having done this once. (You could reload the data and the IDL environment from the last mir_save and start the flux calibration over.) As with all the calibration programs, you can run the program again to make a plot just for illustration.

 

IDL> flux_cal,minvis=0.10

Running Eric Keto's version of flux_cal with the ALMA-CASA planet models

Current Table

#   Source     Flux Cal.           Flux(Jy)

      mars       PRIMARY            73.

Enter source, new flux cal code:

Use p, P, or 1 as code for primary flux cal

Use s, S, or 2 as code for secondary flux cal

Use n, N, NO, or 0 to deselect a calibrator

or hit Return if all the sources are correctly specified

:

Found primary cal mars

Running Eric Keto's version of flux_primary with the ALMA-CASA planet models

Opening file  /sma/SMAusers/keto/mir/idl/new/planetmodels/Mars_Tb.dat

Using ALMA model for Mars

number of data selected for flux cal          624

# USE    SOURCE    Vis*FLUX     SFact   SB   INT BSL    UVDIST   VIS   TSSB     COH    EL

    1      mars       47.79      0.83   l     1  1-8      62.8 -0.11    352    0.93  78.0

    1      mars       47.80      0.83   l     2  1-8      62.8 -0.11    351    0.93  78.1

    1      mars       47.81      0.96   l     3  1-8      62.8 -0.11    350    0.93  78.2

    1      mars       47.81      0.92   l     4  1-8      62.8 -0.11    341    0.93  78.3

    1      mars       47.82      0.80   l     5  1-8      62.8 -0.11    342    0.93  78.4

    1      mars       47.83      0.79   l     6  1-8      62.8 -0.11    342    0.93  78.5

    1      mars       47.83      0.85   l     7  1-8      62.8 -0.11    340    0.93  78.6

    1      mars       47.84      0.91   l     8  1-8      62.8 -0.11    341    0.93  78.7

    1      mars       47.85      0.83   l     9  1-8      62.8 -0.11    344    0.93  78.8

 

                                    --- / ----

 

    1      mars      290.37      2.23   l    16  4-5      18.4  0.68    265    0.36  79.5

    1      mars      290.38      1.42   l    17  4-5      18.4  0.68    260    0.35  79.6

    1      mars      290.38      1.39   l    18  4-5      18.4  0.68    262    0.36  79.7

    1      mars      290.39      1.38   l    19  4-5      18.4  0.68    270    0.35  79.8

    1      mars      290.40      1.37   l    20  4-5      18.4  0.68    266    0.36  79.8

    1      mars      290.41      1.40   l    21  4-5      18.4  0.69    265    0.35  79.9

    1      mars      290.42      1.37   l    22  4-5      18.4  0.69    263    0.35  80.0

    1      mars      290.43      1.35   l    23  4-5      18.4  0.69    261    0.35  80.1

    1      mars      290.44      1.53   l    24  4-5      18.4  0.69    260    0.36  80.2

# USE    SOURCE    Vis*FLUX     SFact   SB   INT BSL    UVDIST   VIS   TSSB     COH    EL

Maximum measured coherence  for mars is      0.969351

Maximum measured visibility for mars is      0.685077

Minimum allowed coherence        0.00000

Minimum allowed visibility      0.100000

 

  Flux scaling factor for each sideband:

  l    1.09E+00   u      1.15E+00

 

Stacking gain curves to the ca structure

 

 

Ready to apply scaling factors. Default is NO.

Apply Flux Calibration? [NO <YES>]: 

 

 

The suggested corrections are now 9% and 15%. I would not use these because some of the visibilities included in the ratio are not much above the noise. The noise level can be estimated from the plot. Each vertical set of visibilities represents all the integrations on one projected baseline length. So the vertical spread shows the noise. Taking ratios of two numbers whose amplitude is within the noise will not produce a reliable result.

 

Figure 610 Observed and modeled visibilities. Points selected to calculate the flux correction (blue) are those with modeled visibilities above 0.10. Some of these look to be at the noise level or below.

 

Finished the flux cal.

 

 

 

 

 7.  Accuracy of the flux calibration

 

The accuracy of the flux calibration obviously depends on the accuracy of the observations of the flux calibrator. If the chopper-wheel calibration is as good as 20%, then the correction based on the flux calibrator needs a higher accuracy than this to make any improvement.

The flux calibration in this data is particularly problematic, one reason it was chosen as an example. Here are some plots of observations and models of Mars and Neptune from a different data set observed on 2016-06-02.

 

The example here of Mars shows that there can be uncertainties or problems beyond the signal-to-noise of the observations of the flux calibrator. Mars is a very bright source. In this data set, some of the antennas may have different effective efficiencies leading to individual baselines with higher and lower fluxes than expected. This might be due to pointing since the observation of Mars was made before the pointing was corrected in the middle of the night. It may not be possible to identify the cause just from the data set with only one observation of one flux calibrator.

 

The flux calibration in this data is particularly problematic, one reason it was chosen as an example. Here are some plots of observations and models of Mars and Neptune from a different data set observed on 2016-06-02.

 

 

Figure 71 Observation and model of Mars from a different data set, 2014-06-02 where the flux calibration is working better. The X and Y axes are projected baseline length and visibility. The next figure shows Neptune from this same data set.

 

Figure 72 Observation and model of Neptune from the data of 2014-06-02.

 

With two flux calibrators in the same data set, we can compare to assess the accuracy. The observations of Mars and Neptune were made at the beginning and end of the night of 2014-06-02. The scaling factors for the upper and lower sidebands suggested by Mars are 0.73 and 0.79 and by Neptune 0.84 and 0.88. You can enter more than one flux calibrator in the source selection, for example,

select,/p,/re,source=['Neptune', '1733-130', 'G0.489+0.010', '1744-312', 'Mars']

and the flux_cal program will average the corrections and apply them to all the sources in the selection. In this case the suggested corrections are 0.82 and 0.86. The plot produced by flux_cal auto-scales for the brightest visibility as shown below. Mars is 86 Jy, and Neptune at 11 Jy is barely visible above the X-axis.

Figure 73 Mars and Neptune, 2014-06-02, on the same plot when both flux calibrators are selected prior to running flux_cal. Neptune is just above the X-axis on this plot because the scale is set by much brighter Mars.

 



Of course, you can run the flux_cal program again after applying the corrections to see the results. The model plotted before the flux calibration is not a fit to the data. It is what the data should look like. After the flux calibration, the data are scaled to best match the model.

Figure 74 Mars, 2014-06-02, after the flux calibration. The data are now scaled to match the model and the same correction is applied to all the other sources selected.

 

Figure 75 Neptune, 2014-06-02, after the flux calibration.

 

 8.  Doppler corrections and save

 

PG1119+120 was tracked.  Doppler correct PG1404+226 according to:

 

select, /p, /re

uti_doppler_fix, reference='PG1119+120', source='PG1404+226'

 

Final save of MIR calibration

mir_save,'fully_calibrated'

 

 

 

 9.  Checks

 

 

select,/p,/re,source='1058+015'

plot_var, x='prbl', y='ampave', frame_var='sb', color='blcd'

 

 

 

 

Looks like a point source, equal ampl. with baseline. Not surprising since it was self-calibrated in gain_cal . This test would be more meaningful if the observations had been set up with a second (check) quasar.



Plot the spectrum of the target.

 

select,/p,/re,source='PG1119+120'

plot_spectra, x_var='fsky', frame_var='sb', color='band', ntrim=8

 

 

 

This would be more meaningful if there were strong lines in the source. If there were lines, the sidebands could be selected individually and plotted for more detail.

 

 

 

 

 

10. Write the MIRIAD data sets

 

 

.compile idl2miriad

.compile idl2miriad

 

select,/re,/p

 

idl2miriad, dir='100603_025520_PG1119+120_usb', source='PG1119+120', sideband='u'

idl2miriad, dir='100603_025520_PG1119+120_lsb', source='PG1119+120', sideband='l'

idl2miriad, dir='100603_025520_PG1404+226_usb', source='PG1404+226', sideband='u'

idl2miriad, dir='100603_025520_PG1404+226_lsb', source='PG1404+226', sideband='l'

 

 


Getting Help

Eric Keto's MIR Distribution

E-mail us at eketo@cfa.harvard.edu and put the following in the subject line:

  • "data" if related to computer access, MIR, etc.
  • "website" if related to website viewing.



Official MIR Distribution

To view the MIR cookbook, visit https://www.cfa.harvard.edu/~cqi/mircook.html

Updates


SectionSub-sectionTitleAuthorDate
Calibrating SMA DataN/ANEW RELEASEEric Keto11/15/2015
Calibrating SMA DataGain Calibration Phase Wrapping
(pdf file)
Mark Thomas Graham09/26/2014