from:	Jun-Hui Zhao 
to:	Thomas Cooper ,
Holly Sarah 
bcc:	"Moran, Jim" 
date:	Jun 22, 2018, 12:47 PM
subject:	Re: smamiriad account home directory in Hilo exceeds 10G limit

Hi Mac,
Oops, I have been using this account to test the new pipeline software package swarm2casa
for debugging and preparing for release to general SMA users. I will remove the test data from
the home directory.

The first version of this pipeline is being developed with serial code (using one CPU unit)
and is being tested on SMA Hilo system as well as RTDC/RG system to understand the performance
of this software on the computing systems at CfA.

For the Hilo system, the test has been taken on the three data-reduction severs (hilodr1, hilodr2 and hilodr3)
while on the RTDC system, the server rtdc9 has been used as suggested by Holly.

I have questions for both you and Holly in order to understand the performance and distribution of this pipeline software:
1) according to the report from "top" monitoring, the fraction of CPU used on the Hilo servers is a factor
of 5 to 10 less than the RTDC server. Such poor performance occurs on the Hilo servers with both smamiriad
and jzhao accounts. Both of you may help me to understand the issue why the CPU unit used for
swarm2casa on sleep state for most of time on the Hilo servers. In addition to the CPU issue,
the tolerance time for network idle is short for the communication link between my desktop
(bigfoot.cfa.harvard.edu, IP-address 131.142.9.68) and the servers on Hilo system. The combination
of both the CPU issue and short tolerance time for network idle prevent me from successfully
accomplishing a test for the new pipeline software swarm2casa.
Mac: Would you help me to reset tolerance time for network idle. Please let me know
if you are difficult to do so or you think that the problem occurs from the end of Cambridge(?).

2) I also noticed that the data transaction is slow between my desktop and Hilo. Fetching over
a SMA data to display on my desktop with CASA GUI program plotms appears to take
forever. I have been not able to plot successfully SMA visibility data on my desktop with
CASA from a sever of Hilo system. I have no trouble to plot JVLA visibility data with a
similar size as the SMA from NRAO computers of the nmpost . The slow data transaction
appeared/appears to be also an issue for timely debugging graphic problems that sometime happens
in the SMA Miriad distribution on the Hilo system.
Mac: Would you please to increase the bandwidth of data transaction between my desktop and the Hilo system?
Or the small bandwidth for data transaction may occur from the end at CfA in Cambridge.
Please help me sort out this issue too.

3) where to install the software? I would suggest to put this pipeline on /usr/local area; it would be also good
to put it under /sma/local like we did for the distribution of other data reduction software
on the Hilo system. /sma/local seems to have not been existing on the RTDC/RG system yet.
rtdc9.cfa.harvard.edu cd /sma/local
/sma/local: No such file or directory.

Please let me know which one is the best for both of you concerning the distribution/updating and users' support.

4) the CASA on the RTDC/RG has been updated to the latest version 5.3.0 but has issues to convert SMA coordinates
to CASA (according to Holly's email notification last week). I also noticed similar issues and am preparing to send
NRAO helpdesk a ticket concerning CASA. The CASA versions on the Hilo servers appear way too old with more issues.

A guide for pre-user's test of this pipeline on RTDC/RG system has been posted:
https://www.cfa.harvard.edu/~jzhao/SMA-CASA/
while more detailed documents for guidance are being prepared.

Regards,

Jun-Hui