from:	Tomasz (Tomek) Kaminski 
to:	"Thomas, Holly Sarah" ,
"Zhang, Qizhou" ,
Luca Matra ,
Garrett Keating ,
"Qi, Chunhua" ,
Attila Kovacs 
cc:	Jun-Hui Zhao 
date:	Jul 31, 2018, 3:18 PM
subject:	Re: SMA to CASA working group

Dear all,

it appears there is a SMA->CASA pipeline being developed by Jun-Hui, independently to what Taco had, and needs some testing. Shall we briefly meet tomorrow at 3 pm to talk about it? Let me know if you cannot make it so we find a better time.

Tomek


Tomasz (Tomek) Kaminski
SubMillimeter Array fellow
             *  *  *
 Harvard-Smithsonian
Center for Astrophysics
60 Garden Street, MS 78
Cambridge, MA  02138

On Mon, Jul 30, 2018 at 2:57 PM, Thomas, Holly Sarah  wrote:

    Recent comments from Magnus on his tweaks for anyone interested.....


    Well, I have the "fix" that I have implemented, but I am not sure it is correct.
    I am attaching the diff output for my fix vs the original sma2casa.py file with
    some short comments. When I use this the beginning and end of the spectral
    coverage seems ok. However when I flag the first and last 250 channels for each
    SWARM chunk (in data rechunked by x4), the spectral windows do not create a
    continous coverage in frequency.

    One thing I haven't checked yet is how the data is stored in the FITS-IDI
    structure. The raw data has some chunks with width<0, i.e. descending frequency
    with increasing channel (?). I don't know if this is correctly handled when
    writing the FITS-IDI files.

    On Fri, Jul 13, 2018 at 8:52 PM, Luca Matra  wrote:

        Hi all,


        Today we had a discussion about forming a working group to deal with importing SMA raw data into CASA, so we are going to have our first discussion on a date TBD - I will send a doodle once Attila gets back.
        In brief, this is so that in the future we can provide users with the possibility of carrying out the calibration and imaging in CASA, and to have a dedicated guide accessible for people to be able to calibrate the data in CASA if they wish to do so. 

        Now, this mainly requires us to translate the SMA raw data into CASA MS format. Today I explored the previous attempt made by Taco with a recent dataset of mine - now I have a fair idea at least of what it does on on the CASA side (smaImportFix.py task). What we also need to make sure we understand and are able to manage/keep up to date is the reading the SMA data, applying Tsys and calculating the weights, plus writing to FITS-IDI format (sma2casa.py task). 

        Taco’s script seems pretty robust so I now feel it’s the best starting point. The attempt on my SWARM 230 data seems to have worked ok (with some shallow checks) using an older version of CASA (4.5.3). Indeed it didn’t work in later versions of CASA I tested (5.1.0 and 5.3.0), as some of you mentioned.
        I suspect the issue there was that CASA may have updated the importfitsidi task, which now actually seems to be able to read the information to go into the SYSCAL table of the MS from the FITS-IDI file, whereas before Taco was importing this manually. This manual import Taco was doing is what now fails, so once those few lines are commented out in his smaImportFix.py code everything is back to functioning apparently fine in the newer versions of CASA. I can easily patch his code so we can make it available again.

        Following that, we should probably talk about providing at least some basic CASA script that people can use for calibration and imaging.


        Best,

        Luca

        ------------------------------------------------------------
        Dr. Luca Matrà
        Submillimeter Array (SMA) Fellow
        Harvard-Smithsonian Center for Astrophysics
        60 Garden Street
        Cambridge, MA 02138, USA
        E-mail: luca.matra@cfa.harvard.edu
        Web: https://www.cfa.harvard.edu/~lmatra
        Phone: +1 (617)-495-7085
        ------------------------------------------------------------