from:	Mark Gurwell 
to:	"Young, Ken" 
cc:	Qizhou Zhang ,
Glen Petitpas ,
Chunhua Qi ,
David Wilner ,
Jun-Hui Zhao ,
Nimesh Patel ,
Eric Keto ,
Jonathan Weintroub 
date:	Mon, Aug 3, 2015 at 9:30 AM
subject:	Re: Next Quadrant of SWARM

Hi Taco,

First, I'm excited that we are close to having a new quadrant. However, I am frankly somewhat dismayed at the prospect of not running at full speed.  In such a case, there will be potentially significant gaps between the SWARM chunks (the size of which i have yet to receive an answer on...another source of annoyance).

1) Unless we are running at full speed (11/11), and until we know we are operating SWARM *perfectly* (or at least better than the ASIC correlator),  I see no reason to retire ASIC. I find this particularly true given that Edward has stated to me that he hopes to have new 1.1 mm band receivers (LO 210 to 270) installed in September(!)...as many as four of them.  This means that dual receiver operation for line work will open up a LOT, since we can do either dual pol (or full pol) with two receivers over a major fraction of the SMA bandwidth.  Throwing away the ability to cover as much of that bandwidth as possible strikes me as premature. 

Now, I can't claim to even a smidgen of understanding on how SWARM interacts with the current dual receiver action...as far as I'm aware, we cannot do that right now (is this correct?) but at some point that can't always be the case.  So the question could be couched as 'how often will including ASIC help?' and it may be more significant than we realize (or it may be a lot less). I just don't understand the hardware *and* software limitations involved right now.  For example, you say this quadrant does '4-6'.  But do you mean 4-8? And in a dual pol mode then 4-6 (or perhaps 6-8? I hear John's new bdc is much more flexible then our current system).  Would it be possible to run ASIC 4-6 GHz dual pol, and SWARM quad 2 6-8 dual pol (and SWARM quad 1 does...what 8-10? )

So, without understanding limitations, I can't answer these questions accurately. But I believe throwing away bandwidth should not be a high priority unless (a) it will be very shortlived, e.g. SWARM quad 3 is months away, or (b) it is entirely too complicated to keep ASIC running. 

Please note though that by offering ASIC this semester, we are obligating ourselves to 7 GHz of continuum, and that is what people are expecting in terms of noise performance.  Furthermore, it may be impossible for us to tell from the proposals if there are important lines that might be missed when using SWARM quads 1 & 2 only and there is a gap between the chunks of each.

2) My preference would be to make the lowest IF frequency chunks the lowest numbered chunks (so, s49 and s50)...this would suggest SWARM quad 2 covers s49 and s50, then SWARM quad 1 (current SWARM) become s51 and s52.  Clearly, MIR should have no issues with this, since all the information should be passed in the headers on which chunk has which frequency range.

3) I can say that from use, MIR can handle the increased number of chunks - Charlie wrote the readdata routine to rebin SWARM bands into further new chunks, and as far as i'm aware (Charlie, please let us know) there are no restrictions on that. However, the raw data size may be an issue, since an 80 GB raw file would require upwards of 200 to 250 GB of RAM, which I don't think any machine has.  Or are you assuming the 'expanded' data file is 80 GB? If so, that might be ok. Charlie?

However, if your rechunker program (which seems to be a misnomer, btw...you aren't creating new chunks, you are resampling/binning the old chunks...) will work on mulitple SWARM quadrants, then perhaps that isn't such a drawback...


Mark


On Mon, Aug 3, 2015 at 8:40 AM, Young, Ken  wrote:

    Dear SMA Data Enthusiast,

       We will probably have the next quadrant of SWARM available before the end, and perhaps before the beginning of the next SMA semester.   As I understand it, that quadrant of SWARM will process the 4->6 GHz portion of the IF - the portion of the IF now processed by the ASIC correlator.   Several questions come to mind:

    1) When the next quadrant of SWARM is running, at at least 10/11 speed, do we want to keep the ASIC correlator running?   We are offering the ASIC correlator in the Call for Proposals, but it is possible that nobody will request a higher spectral resolution than SWARM provides.   If that is the case, should we just retire the ASIC?   I know that there are special situations where the ASIC could add additional bandwidth even with two SWARM quadrants (dual receiver observations).   Is that reason enough to keep the ASIC running?

    2) Assuming that we keep the ASIC running, should I just make the two new SWARM chunks s51 and s52?

    3) Are our data reduction packages up to the task of processing ~80 GByte data files?   If not, what needs to be done to make them ready?   Should I produce some simulated two quadrant SWARM + ASIC data files, to see if we can handle them?

    Thanks,

    Taco