$ idl

This data is single receiver only and just 400MB so it can all be read into MIR.

IDL> readdata,dir='/sma/data/science/mir_data.2008/081102_17:02:05/'

View a summary of the data.

IDL> select

All Sources:   3c273 NGC3175 0750+125 NGC2339 1037-295 0854+201 titan target
All Baselines:  1-3 1-4 1-5 1-6 1-7 1-8 3-4 3-5 3-6 3-7 3-8 4-5 4-6 4-7 4-8 5-6 5-7 5-8 6-7 6-8 7-8
All Recs:  230
All Bands:  c1 s01 s02 s03 s04 s05 s06 s07 s08 s09 s10 s11 s12 s13 s14 s15 s16 s17 s18 s19 s20 s21 s22 s23 s24
All Sidebands :  l u
All Polarization states:  hh vv hv vh
All Integrations: 0-754

Pointing data will have an integration time < 10seconds. Use dat_filter to select just this data, then flag the selection.

IDL> result=dat_filter(s_f,'"integ" lt "9"',/reset)
IDL> if result gt 0 then flag,/flag

Use the if loop to prevent all data being flagged in the event there is no pointing data.

Afterwards, re-select only the positive weighted (unflagged data).

IDL> select,/pos,/res

Plot the pseudo-continuum channel [1]. This is an average of the spectral channels for each point for both ampltude and phase.

IDL> plot_continuum [1]

The amplitude scale is being stretched by outliers which can be flagged. Use interactive plotting to get the scan number.

Scroll through the plots by right-clicking. Use the middle-click to bring up a menu in the terminal. One option will be f for flag. Select f then left-click on an point to flag it [2]. When you flag a point interactively the scan number is reported in the terminal. Once you have finished flagging exit with a right-click.

You will see outliers appearing at the same position across baselines and sidebands [2]. To avoid having to flag each one separately, use the interactive plotting to get the scan number, then select and flag it on all baselines.

IDL> select,/pos,/re,int=[167,167]
IDL> flag,/flag

Select just positive weighted data and re-plot.

IDL> select,/pos,/res
IDL> plot_continuum

As the scaling changes another outlier appears - scan number 7. This step may require a few iterations. Finally the data will look like [3].

IDL> select,/pos,/re,int=[7,7]
IDL> flag,/flag
IDL> select,/pos,/res
IDL> plot_continuum [3]

View the system temperature. Check the points and profile for outliers.

IDL> plot_var [4]

Here the data all looks reasonable. The correction can be applied without further issue. See the SWARM page for examples of fixing problems with system temperature data

IDL> apply_tsys

Then regenerate the continuum channel.

IDL> uti_avgband

Inspect the frequency data for spikes. by default plot_spectra has x='channel' [5]. You can plot using x='fsky' for frequency, but should do so separately for each sideband as there is a large separation between them.

IDL> plot_spectra,color='band' [5]

Right-click to scroll through the plots for each source. If you see a spike you will need to plot only the spectral window it lies in, in order to determine it's correct channel number for flagging. Although it's not very big we will flag the spike in spectral window s08.

IDL> select,/pos,/res,band='s08'
IDL> plot_spectra [6]

Zooming in you can identify the channel number as 35. Then apply the fix.

IDL> uti_chanfix, chan=35

Reselect all data.

IDL> select,/pos,/res

By default, uti_chanfix replaces the value of the given channel with an average of the adjacent 3rd channel on each side. Set which channels will be used to produce the average with the option 'sample'. If a spike is a few channels wide for example, using sample=10 will use the 10th channel on either side for an average.

The SWARM example uses the uti_checkspike task to automate this step.

Apply the phase based bandpass first. Use the /preavg option which averages the selected number of channels into a single point.

IDL> pass_cal,cal_type='pha',preavg=4, smoothing=1, ntrim=12, tel_bsl='telescope', refant=5 [7]

 Check: Found a total of            6  different sources in the data set.
 Set which sources will be used as calibrators and set the calibrator fluxes
 The flux should be in Jy at the frequency of the observation
 These are the sources and their current passband codes 
 name:  3c273              passband cal: NO 
 name:  NGC3175            passband cal: NO 
 name:  0750+125           passband cal: NO 
 name:  NGC2339            passband cal: NO 
 name:  1037-295           passband cal: NO 
 name:  titan              passband cal: NO 
 Enter source and new cal code. eg: 3C273 YES
 or hit Return if all the sources are correctly specified
  : 3c273 yes

Once it has generated a solution it will create a plot of the fits [7]. Right-click through the solutions for each baseline. After exiting the plot window it will prompt to accept or reject the solutions.

If pass_cal fails reporting insufficient flux, try adding more sources to the list (if suitable ones are available), or increasing the 'preave' value to increase the signal-to-noise.

NOTE: ASIC data may have a different resolution for each chunk. Be sure to perform bandpass calibration separately for each resolution.

Now regenerate the continuum. Fixing the phases will slightly boost the average amplitudes for the next step.

IDL> uti_avgband

Next repeat the process in the same way for amplitude [8].

IDL> pass_cal,cal_type='amp',preavg=4, smoothing=1, ntrim=12, tel_bsl='telescope', refant=5 [8]

Check the success of the bandpass calibration by viewing the spectra for the bandpass calibrator. The phases and amplitudes look nice and flat [9].

IDL> plot_spectra,color='band',ntrim='12',source='3c273' [9]

Select all unflagged data, regenerate the continuum and save in MIR format. It is important to save the dataset before starting flux measurement as the phases will be irreversibly disrupted.

IDL> select,/p,/re
IDL> uti_avgband
IDL> mir_save,'081102_170205.mir',/new

It does not matter which order gain calibration and flux calibration occur.

Select all the flux and gain calibrators.

IDL> select,/p,/re, band='c1',source=['-3c273','-NGC']

Apply the gain_cal in phase first with the /connect option to force the phases to zero for every scan [10]. The /non-point option takes into account the size of planets, on some baselines resolved planets may have a phase of ±180°.

IDL> gain_cal,cal_type='pha',/connect, tel_bsl='telescope', refant=5, /non_point [10]

    : all yes 

Next run amplitude based gain_cal [11]. Here, only select the flux calibrator (usually a planet or moon). Using the /non-point option it will retrieve its flux.

IDL> gain_cal, cal_type='amp',x='hours',poly=0,tel_bsl='telescope',refant=4, /preavg,/non_point[11]

    : all no
    : titan yes

Note: the task sma_flux_cal performs the same amplitude based gain calibration as above, but baseline rather than antenna based.

Ideally, select quasar scans near the flux calibrator in time and/or elevation. The conditions (pointing, opacity, system) should be as similar as possible when scaling between the flux calibrator and gain calibrators. In this data, the quasar scans near the flux calibrator are below 35° elevation which isn't ideal.

Now measure the calibrated flux with flux_measure. It will return either vector or scalar values for each source. Scalar values ignore phase noise, so the amplitudes are typically slightly higher.

IDL> flux_measure

  % Compiled module: FLUX_MEASURE.
  % Compiled module: SMA_FLUX_MEASURE.
       21726 passed in list
  Scalar Average or Vector Average ? [S ]: v
  Vector average: 
  #   Source   Flags   Nscans  Flux(Jy)   SNR    meantime    REAL       IMAG
       3c273             80   16.2778     452     19.53     16.2778      0.0138
     NGC3175            181    0.0017       0     19.53      0.0008      0.0015
    0750+125             42    2.3014     400     19.14      2.3014      0.0010
     NGC2339            101    0.0034       1     18.63     -0.0019      0.0029
    1037-295             72    0.8437     248     19.86      0.8437     -0.0006
       titan      g      50    1.0562     139     21.43      1.0562      0.0054

With known fluxes for the gain calibrators, restore the previously saved file and perform gain calibration.

IDL> mir_restore,'mir2018_rx240.mir'
IDL> select,/p,/re

Select just the science and the gain calibrator sources.

IDL> select,/pos,/res,source=['NGC', '0750','1037']

Apply the phase based gain_cal using smoothing and /preavg [12].

IDL> gain_cal,cal_type='pha',x='hours',smoothing=0.3,/preavg, tel_bsl='telescope', refant=4 [12]

   : 0750+125 yes 2.3014
   : 1037-295 yes  0.8437

Next apply the amplitude based gain calibration - this time supply the real fluxes calculated by flux_measure [13].

IDL> gain_cal,cal_type='amp',x='hours',smoothing=0.6,/preavg, tel_bsl='telescope',refant=4 [13]

Have a final check of the calibrated dataset [14]. There remains some scatter in the calibrator amplitudes, but the phases are flat and overall the calibration looks good. This data was taken in very extended configuration which is generally worse than data taken in compact configuration.

IDL> plot_continuum [14]

Then save the data in MIR format.

IDL> mir_save,"081102_170205_cal.mir",/new

Export continuum and spectral data to MIRIAD.

IDL> select,/pos,/re
IDL> idl2miriad,dir='NGC3175_cont_u.uv',source='NGC3175',sideband='u',/cont
IDL> idl2miriad,dir='NGC3175_line_u.uv',source='NGC3175',sideband='u'
IDL> idl2miriad,dir='NGC3175_cont_l.uv',source='NGC3175',sideband='l',/cont
IDL> idl2miriad,dir='NGC3175_line_l.uv',source='NGC3175',sideband='l'

Save the data in uvfits format to be read by CASA.

IDL> select,/p,/re
IDL> autofits,source='NGC3175'