Date: Fri, 18 Nov 2011 15:52:31 -0500 (EST)
From: Jun-Hui Zhao 
To: Ken Young 
Cc: jgallagher@cfa.harvard.edu, eketo@cfa.harvard.edu, danlizhao1@msn.com,
     rblundell@cfa.harvard.edu, gpetitpas@cfa.harvard.edu, mwright@astro.berkeley.edu,
     miriad-dev@astro.berkeley.edu
Subject: Problem 1 in SMA datasets produced from an ad hoc correlator mode


Dear Dr. Ken Young,

     The problematic file (111108_09:56:47, item 1 below) has
been investigated. There is an inconsistence between MIR header
informations for this "new or ad hoc correlator configuration mode"
(item 2 below). The header file "codeRead" (item 3 below) does
describe the data set with one continuum chunk (c1, at count=49)
and forty-eight spectral chunks (s01 to s48, at count=87 to 134).
However the spectral header "spRead" (item 4 below) skips
iband= 14, 15, 16 and 34, 35, 36 giving no informations for the
"SMA spectral chunks" s14, s15, s16 and s34, s35, s36, which
results a frequency configuration as described in Miriad in item 2
below. Since no header informations and no visibility data for
the spectral chunks s14, s15, s16 and s34, s35, s36, and Miriad
format does not provide a room for spectral id (the sky frequencies
are adequate for users to identify particular spectral chunks.),
the corresponding output data set only gives 49-6 = 43 spectral
windows (chunks) in Miriad format. Somehow, Glen (a user) treated
this data taken with an ad hoc SMA mode as a redular file (48+1
spectral chunks) and resulted in an error "Invalid window selection"
while plotting the data using a task in Miriad (see his report,
item 1 below). This explains the completely failure in plotting
Upper 2 GHz described in Glen's report.

     Dr. Mel Wright, a Miriad expert and senior scientist, from
UC Berkely has described us as well as SMA Miriad users a way to
handle such an ad hoc SMA dataset (item 5 below). Ignoring such a
suggestion, you persist in forcing us (Miriad developers) to match
the IDL output. We could develop a patch to match what you want.
Please answer our questions below in order to get a Miriad output
file produced from dataset taken with an ad hoc SMA correlator mode
that exactly match your desire.

1. Are you going to keep the inconsistence MIR format for the ad hoc
SMA correlator mode, i.e. keep skipping the frequency configuration
informations for those chunks with no visibility data? If so, please
provide a formula to fetch frequency header data for the missing chunks.

2. How do you calculate the continuum data (c1) from the spectral chunk
data? Would you please give us the formula including the weighting?

3. Please check the frequency configuration interpreted from the ad hoc
SMA correlator mode to Miriad (item 2 below). Would you please let us
know if anything wrong there?

    Jun-Hui Zhao


1. File: 111108_09:56:47
   Observed on 2011 November 08. Glen reported errors as described below

If you wanted to see the plots, my summary output is here.
http://sma1.sma.hawaii.edu/projects/ql/index.html?dir=111108_09:56:47&obsid=3243

The "Lower 2 GHz" plots show the zeroing of the phase Taco mentioned.

The "Upper 2 GHz" plots fail completely because I manually select the
upper 24 chunks with the 'select' option, and since there are less than 24
(because the zeroed chunks are not counted) smauvplt fails with a "Invalid
window selection" error.


2. Correlator configuration
Lower side band
Frequency Configuration 1
Spectral ID   Channels  Freq(chan=1)  Increment
        c 1        1     218.60721    -0.416000 GHz
        s48       64     220.59187    -0.001625 GHz
        s47       64     220.50987    -0.001625 GHz
        s46      512     220.43550    -0.000203 GHz
        s45       64     220.35350    -0.001625 GHz
        s44       64     220.26387    -0.001625 GHz
        s43       64     220.18187    -0.001625 GHz
        s42       64     220.10750    -0.001625 GHz
        s41       64     220.02550    -0.001625 GHz
        s40       64     219.93587    -0.001625 GHz
        s39       64     219.85387    -0.001625 GHz
        s38       64     219.77950    -0.001625 GHz
        s37       64     219.69750    -0.001625 GHz
        s36      512     219.60929    -0.000203 GHz
        s35        no information
        s34        no information
        s33        no information
        s32       64     219.27987    -0.001625 GHz
        s31       64     219.19787    -0.001625 GHz
        s30       64     219.12350    -0.001625 GHz
        s29       64     219.04150    -0.001625 GHz
        s28       64     218.95187    -0.001625 GHz
        s27       64     218.86987    -0.001625 GHz
        s26       64     218.79550    -0.001625 GHz
        s25      512     218.71350    -0.000203 GHz
        s24      128     218.59268    -0.000812 GHz
        s23       64     218.50987    -0.001625 GHz
        s22       64     218.43550    -0.001625 GHz
        s21       64     218.35350    -0.001625 GHz
        s20       64     218.26387    -0.001625 GHz
        s19       64     218.18187    -0.001625 GHz
        s18       64     218.10750    -0.001625 GHz
        s17      512     218.02550    -0.000203 GHz
        s16      512     217.93729    -0.000203 GHz
        s15        no information
        s14        no information
        s13        no information
        s12       64     217.60787    -0.001625 GHz
        s11       64     217.52587    -0.001625 GHz
        s10       64     217.45150    -0.001625 GHz
        s 9      512     217.36950    -0.000203 GHz
        s 8       64     217.27987    -0.001625 GHz
        s 7       64     217.19787    -0.001625 GHz
        s 6      512     217.12350    -0.000203 GHz
        s 5      128     217.04150    -0.000812 GHz
        s 4       64     216.95187    -0.001625 GHz
        s 3       64     216.86987    -0.001625 GHz
        s 2       64     216.79550    -0.001625 GHz
        s 1       64     216.71350    -0.001625 GHz

Upper side band
Frequency Configuration 1
Spectral ID   Channels  Freq(chan=1)  Increment
        c 1        1     230.38778     0.416000 GHz
        s 1       64     228.61112     0.001625 GHz
        s 2       64     228.69312     0.001625 GHz
        s 3      512     228.76750     0.000203 GHz
        s 4       64     228.84950     0.001625 GHz
        s 5       64     228.93912     0.001625 GHz
        s 6       64     229.02112     0.001625 GHz
        s 7       64     229.09550     0.001625 GHz
        s 8       64     229.17750     0.001625 GHz
        s 9       64     229.26712     0.001625 GHz
        s10       64     229.34912     0.001625 GHz
        s11       64     229.42350     0.001625 GHz
        s12       64     229.50550     0.001625 GHz
        s13      512     229.59370     0.000203 GHz
        s14        no information
        s15        no information
        s16        no information
        s17       64     229.92312     0.001625 GHz
        s18       64     230.00512     0.001625 GHz
        s19       64     230.07950     0.001625 GHz
        s20       64     230.16150     0.001625 GHz
        s21       64     230.25112     0.001625 GHz
        s22       64     230.33312     0.001625 GHz
        s23       64     230.40750     0.001625 GHz
        s24      512     230.48950     0.000203 GHz
        s25      128     230.61031     0.000812 GHz
        s26       64     230.69312     0.001625 GHz
        s27       64     230.76750     0.001625 GHz
        s28       64     230.84950     0.001625 GHz
        s29       64     230.93912     0.001625 GHz
        s30       64     231.02112     0.001625 GHz
        s31       64     231.09550     0.001625 GHz
        s32      512     231.17750     0.000203 GHz
        s33      512     231.26570     0.000203 GHz
        s34        no information
        s35        no information
        s36        no information
        s37       64     231.59512     0.001625 GHz
        s38       64     231.67712     0.001625 GHz
        s39       64     231.75150     0.001625 GHz
        s40      512     231.83350     0.000203 GHz
        s41       64     231.92312     0.001625 GHz
        s42       64     232.00512     0.001625 GHz
        s43      512     232.07950     0.000203 GHz
        s44      128     232.16150     0.000812 GHz
        s45       64     232.25112     0.001625 GHz
        s46       64     232.33312     0.001625 GHz
        s47       64     232.40750     0.001625 GHz
        s48       64     232.48950     0.001625 GHz

3. header file "codeRead"
count=49 v_name=band icode=0 code=c1 ncode=0
count=87 v_name=band icode=1 code=s01 ncode=1
count=88 v_name=band icode=2 code=s02 ncode=2
count=89 v_name=band icode=3 code=s03 ncode=3
count=90 v_name=band icode=4 code=s04 ncode=4
count=91 v_name=band icode=5 code=s05 ncode=5
count=92 v_name=band icode=6 code=s06 ncode=6
count=93 v_name=band icode=7 code=s07 ncode=7
count=94 v_name=band icode=8 code=s08 ncode=8
count=95 v_name=band icode=9 code=s09 ncode=9
count=96 v_name=band icode=10 code=s10 ncode=10
count=97 v_name=band icode=11 code=s11 ncode=11
count=98 v_name=band icode=12 code=s12 ncode=12
count=99 v_name=band icode=13 code=s13 ncode=13
count=100 v_name=band icode=14 code=s14 ncode=14
count=101 v_name=band icode=15 code=s15 ncode=15
count=102 v_name=band icode=16 code=s16 ncode=16
count=103 v_name=band icode=17 code=s17 ncode=17
count=104 v_name=band icode=18 code=s18 ncode=18
count=105 v_name=band icode=19 code=s19 ncode=19
count=106 v_name=band icode=20 code=s20 ncode=20
count=107 v_name=band icode=21 code=s21 ncode=21
count=108 v_name=band icode=22 code=s22 ncode=22
count=109 v_name=band icode=23 code=s23 ncode=23
count=110 v_name=band icode=24 code=s24 ncode=24
count=111 v_name=band icode=25 code=s25 ncode=25
count=112 v_name=band icode=26 code=s26 ncode=26
count=113 v_name=band icode=27 code=s27 ncode=27
count=114 v_name=band icode=28 code=s28 ncode=28
count=115 v_name=band icode=29 code=s29 ncode=29
count=116 v_name=band icode=30 code=s30 ncode=30
count=117 v_name=band icode=31 code=s31 ncode=31
count=118 v_name=band icode=32 code=s32 ncode=32
count=119 v_name=band icode=33 code=s33 ncode=33
count=120 v_name=band icode=34 code=s34 ncode=34
count=121 v_name=band icode=35 code=s35 ncode=35
count=122 v_name=band icode=36 code=s36 ncode=36
count=123 v_name=band icode=37 code=s37 ncode=37
count=124 v_name=band icode=38 code=s38 ncode=38
count=125 v_name=band icode=39 code=s39 ncode=39
count=126 v_name=band icode=40 code=s40 ncode=40
count=127 v_name=band icode=41 code=s41 ncode=41
count=128 v_name=band icode=42 code=s42 ncode=42
count=129 v_name=band icode=43 code=s43 ncode=43
count=130 v_name=band icode=44 code=s44 ncode=44
count=131 v_name=band icode=45 code=s45 ncode=45
count=132 v_name=band icode=46 code=s46 ncode=46
count=133 v_name=band icode=47 code=s47 ncode=47
count=134 v_name=band icode=48 code=s48 ncode=48
....


4. header "spRead" for spectral information

Skip spectral chunks (iband) 14, 15, 16 and 34, 35, 36.
No data for those spectral chunks (windows).

count=0 sphid=1 blhid=1 inhid=0 iband=0 fsky=218.815210 nch=1
count=1 sphid=2 blhid=1 inhid=0 iband=1 fsky=220.540683 nch=64
count=2 sphid=3 blhid=1 inhid=0 iband=2 fsky=220.458683 nch=64
count=3 sphid=4 blhid=1 inhid=0 iband=3 fsky=220.383597 nch=512
count=4 sphid=5 blhid=1 inhid=0 iband=4 fsky=220.302308 nch=64
count=5 sphid=6 blhid=1 inhid=0 iband=5 fsky=220.212683 nch=64
count=6 sphid=7 blhid=1 inhid=0 iband=6 fsky=220.130683 nch=64
count=7 sphid=8 blhid=1 inhid=0 iband=7 fsky=220.056308 nch=64
count=8 sphid=9 blhid=1 inhid=0 iband=8 fsky=219.974308 nch=64
count=9 sphid=10 blhid=1 inhid=0 iband=9 fsky=219.884683 nch=64
count=10 sphid=11 blhid=1 inhid=0 iband=10 fsky=219.802683 nch=64
count=11 sphid=12 blhid=1 inhid=0 iband=11 fsky=219.728308 nch=64
count=12 sphid=13 blhid=1 inhid=0 iband=12 fsky=219.646308 nch=64
count=13 sphid=14 blhid=1 inhid=0 iband=13 fsky=219.557394 nch=512->skip iband 14,15,16
count=14 sphid=15 blhid=1 inhid=0 iband=17 fsky=219.228683 nch=64
count=15 sphid=16 blhid=1 inhid=0 iband=18 fsky=219.146683 nch=64
count=16 sphid=17 blhid=1 inhid=0 iband=19 fsky=219.072308 nch=64
count=17 sphid=18 blhid=1 inhid=0 iband=20 fsky=218.990308 nch=64
count=18 sphid=19 blhid=1 inhid=0 iband=21 fsky=218.900683 nch=64
count=19 sphid=20 blhid=1 inhid=0 iband=22 fsky=218.818683 nch=64
count=20 sphid=21 blhid=1 inhid=0 iband=23 fsky=218.744308 nch=64
count=21 sphid=22 blhid=1 inhid=0 iband=24 fsky=218.661597 nch=512
count=22 sphid=23 blhid=1 inhid=0 iband=25 fsky=218.541089 nch=128
count=23 sphid=24 blhid=1 inhid=0 iband=26 fsky=218.458683 nch=64
count=24 sphid=25 blhid=1 inhid=0 iband=27 fsky=218.384308 nch=64
count=25 sphid=26 blhid=1 inhid=0 iband=28 fsky=218.302308 nch=64
count=26 sphid=27 blhid=1 inhid=0 iband=29 fsky=218.212683 nch=64
count=27 sphid=28 blhid=1 inhid=0 iband=30 fsky=218.130683 nch=64
count=28 sphid=29 blhid=1 inhid=0 iband=31 fsky=218.056308 nch=64
count=29 sphid=30 blhid=1 inhid=0 iband=32 fsky=217.973597 nch=512
count=30 sphid=31 blhid=1 inhid=0 iband=33 fsky=217.885394 nch=512->skip iband 34,35,36
count=31 sphid=32 blhid=1 inhid=0 iband=37 fsky=217.556683 nch=64
count=32 sphid=33 blhid=1 inhid=0 iband=38 fsky=217.474683 nch=64
count=33 sphid=34 blhid=1 inhid=0 iband=39 fsky=217.400308 nch=64
count=34 sphid=35 blhid=1 inhid=0 iband=40 fsky=217.317597 nch=512
count=35 sphid=36 blhid=1 inhid=0 iband=41 fsky=217.228683 nch=64
count=36 sphid=37 blhid=1 inhid=0 iband=42 fsky=217.146683 nch=64
count=37 sphid=38 blhid=1 inhid=0 iband=43 fsky=217.071597 nch=512
count=38 sphid=39 blhid=1 inhid=0 iband=44 fsky=216.989902 nch=128
count=39 sphid=40 blhid=1 inhid=0 iband=45 fsky=216.900683 nch=64
count=40 sphid=41 blhid=1 inhid=0 iband=46 fsky=216.818683 nch=64
count=41 sphid=42 blhid=1 inhid=0 iband=47 fsky=216.744308 nch=64
count=42 sphid=43 blhid=1 inhid=0 iband=48 fsky=216.662308 nch=64
count=43 sphid=45 blhid=2 inhid=0 iband=0 fsky=218.815210 nch=1
count=44 sphid=46 blhid=2 inhid=0 iband=1 fsky=220.540683 nch=64
count=45 sphid=47 blhid=2 inhid=0 iband=2 fsky=220.458683 nch=64
count=46 sphid=48 blhid=2 inhid=0 iband=3 fsky=220.383597 nch=512
count=47 sphid=49 blhid=2 inhid=0 iband=4 fsky=220.302308 nch=64
count=48 sphid=50 blhid=2 inhid=0 iband=5 fsky=220.212683 nch=64
count=49 sphid=51 blhid=2 inhid=0 iband=6 fsky=220.130683 nch=64
count=50 sphid=52 blhid=2 inhid=0 iband=7 fsky=220.056308 nch=64
count=51 sphid=53 blhid=2 inhid=0 iband=8 fsky=219.974308 nch=64
....

5. A way of handling ad hoc SMA correlator configuration modes

Dr. Mel Wright, a senior scientist and Miriad expert from UC Berkeley,
has described Dr. Ken Young a way to handle such specific SMA data
sets in Miriad (see the email communication below):

Date: Tue, 8 Nov 2011 10:07:24 -0800
From: melvyn wright 
To: Ken Young 
Cc: Jun-Hui Zhao , smamiriad@cfa.harvard.edu
Subject: Re: Data set Glen could not process

Hi Ken,





On 11/8/11, Ken Young  wrote:
> On Tue, 8 Nov 2011, melvyn wright wrote:
>
>> Is there any reason not to discard spectra with 0 channels ?
>
> Not that I can think of.   It may be that smalod loses track of which
> spectrum is which if one or more of them has 0 channels.   Anyway, I
think
> you're right that they should simply be discarded, but we do want the
> spectral window numbers to match up with the chunk numbers, so that
chunk
> s25 does not become window 24 if chunk s12 has 0 channels.

if you want the MIRIAD spectra window number to match the SMA chunk
number,
this precludes discarding spectral windows, or using UVCAT to write out
subsets of spectral windows which is often expedient to reduce the size
and convenience of data handlling.

One could add an item to the MIRIAD data to expressly identify
spectral windows with specific SMA hardware.  Each spectral
window is generically identified by sfreq, sdf, restfreq etc.

Mel.

>
> Taco
>
>