.. Copyright 2009-2012 Peter Williams

   This file is part of miriad-python.

   Miriad-python is free software: you can redistribute it and/or
   modify it under the terms of the GNU General Public License as
   published by the Free Software Foundation, either version 3 of the
   License, or (at your option) any later version.

   Miriad-python is distributed in the hope that it will be useful, but
   WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
   General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with miriad-python.  If not, see <http://www.gnu.org/licenses/>.

.. _executing:
.. sectionauthor:: Peter Williams <peter@newton.cx>

Executing MIRIAD Tasks: :mod:`mirexec`
===============================================

.. module:: mirexec
   :synopsis: Execute MIRIAD tasks from Python.
.. moduleauthor:: Peter Williams <peter@newton.cx>

The :mod:`mirexec` module makes it convenient to launch tasks
from within Python. The simplest invocation looks a lot like
what one would run on the command line::

  from mirexec import TaskUVFlag
  TaskUVFlag (vis='fx64a-3c286-2700', select='ant(26)', flagval='f',
              noquery=True).run ()

If you need them, however, the :mod:`mirexec` module provides
more sophisticated facilities allowing you to retrieve the output
of a task, run many tasks in parallel, and so on.

Running a MIRIAD task in :mod:`mirexec` requires three steps which,
as shown above, can often be condensed into a single line of Python:

#. Create an instance of a "task" class corresponding to the task
   you wish to run.
#. Set the task keywords and options.
#. Call a method that actually launches the task.

The following code is equivalent to the first example, but breaks
down the steps more explicitly::

  from miriad import VisData
  from mirexec import TaskUVFlag
  v = VisData ('fx64a-3c286-2700')
  # Create instance:
  t = TaskUVFlag ()
  # Set some keywords:
  t.vis = v
  t.select = 'ant(26)'
  t.flagval = 'f'
  # Set some options:
  t.noquery = True
  t.hms = False # i.e., do *not* specify the "hms" option
  # Launch task.
  # Executes: uvflag vis=fx64a-3c286-2700 select=ant(26) flagval=f options=noquery
  t.run ()


Creating Task Instances
-----------------------

The :class:`TaskUVFlag` class shown in the above examples is a
subclass of the :class:`TaskBase` class, which provides a generic
structure for invoking MIRIAD tasks. The :mod:`mirexec` module defines
such subclasses for many, but far from all, of the tasks provided with
MIRIAD. It's easy to create your own :class:`TaskBase` subclass for
anything you need that's not provided with *miriad-python*,
however. See :ref:`below <customtasks>` for more information.

The :class:`TaskBase` class provides functions for setting keyword
arguments and actually invoking the task. For the full details, see
the detailed API documentation. Subclasses specify the name of the
particular task that is run and the keywords and options it accepts.

Task instances can be reused: you can create an object, set arguments,
and run it, then change some or all of the arguments and run it
again. Among other uses, this makes it easy to apply a task to several
datasets::

  t = TaskUVAver ()
  t.interval = 5
  t.line = 'chan,800,101'
  t.nocal = True
  for v in listManyDatasets ():
     # The set() method returns 'self' for easy chaining of
     # method invocations.
     t.set (vis=v, out=v.vvis ('av')).run ()


Setting Task Parameters
-----------------------

You can set the task parameters in several ways: as a property on the
object, as in the example above, as a keyword argument to the object's
constructor, or as a keyword argument to the object's
:meth:`~TaskBase.set` method. The latter two forms are shown in
the example below::

  from miriad import VisData
  from mirexec import TaskUVFlag
  v = VisData ('fx64a-3c286-2700')
  # This is equivalent to the previous example.
  t = TaskUVFlag (vis=v, flagval='f', noquery=True)
  t.select = 'ant(26)'
  t.run ()
  # As is this.
  t.set (vis=v, select='ant(26)', flagval='f', noquery=True)
  t.run ()

Thus, the most succinct way to execute a task is to write something
like::

  TaskUVFlag (vis=v, flagval='f', select='pol(yy)').run ()

The names and values of keywords in Python are mapped to command-line
arguments with the following rules:

* Keyword arguments have the same name in Python as they do on the
  command-line if possible. If the MIRIAD keyword is a Python
  keyword (*e.g.*, "in"), the keyword is accessible in Python by
  suffixing it with an underscore ("in\_").
* In most cases, the textual value of each MIRIAD keyword is the
  stringification of the Python variable assigned to it. If the Python
  value is :const:`None`, the keyword is not supplied on the
  command-line.
* However, if the Python variable assigned to the keyword is a
  non-string iterable, the textual value of the keyword is the
  stringification of each item in the iterable, joined together with
  commas. For instance, if you run::

    from mirexec import TaskMfCal
    TaskMfCal (vis=foo, line=['chan', 60, 15]).run ()

  the *line* keyword of ``mfcal`` will be ``chan,60,15``.
* The keyword "options" isn't used directly. Instead, each possible
  option to a task is a separate field on the task object that should
  be set to a :class:`bool`. The option is supplied if the field is
  :const:`True`. There are rare tasks that have an option with the same
  name as a keyword; in those cases, the keyword is the one controlled
  by the property on the task object.

There are several functions that will actually execute the task. Each
has different uses:

* :meth:`~TaskBase.run` executes the task and waits for it to
  finish. The task output is sent to the stdout of the Python program
  and the task input is set to ``/dev/null``.
* :meth:`~TaskBase.snarf` executes a task and waits for it to
  finish. The task's output to its standard output and standard error
  streams are returned to the caller.
* :meth:`~TaskBase.runsilent` executes the task and waits for it to
  finish. The task output is sent to ``/dev/null``.
* :meth:`~TaskBase.launch` starts the task but doesn't wait for it to
  finish; instead, it returns a :class:`MiriadSubprocess` instance
  that allows interaction with the launched subprocess.
* :meth:`~TaskBase.launchpipe` starts the task but doesn't wait for it to
  finish. The output of the task is redirected to pipes that can be
  read using the :class:`MiriadSubprocess` instance.
* :meth:`~TaskBase.launchsilent` starts the task but doesn't wait for it to
  finish. The output of the task is redirected to ``/dev/null``.


.. _customtasks:

Defining Your Own Task Classes
-------------------------------

In most cases, it's straightforward to define your own task class. To
wrap the task "newtask", you should write something like::

  from mirexec import TaskBase

  class TaskNewTask (TaskBase):
      _keywords = ['vis', 'line', 'flux', 'refant']
      _options =  ['nocal', 'nopass', 'mfs']

  def demo (vis):
      t = TaskNewTask (vis=vis)
      t.flux = 1.0
      t.nocal = True
      t.run ()

The name of the task executable is inferred from the class name by
stripping off the prefix "Task" and lower-casing the rest of the
letters. If this heuristic won't work, you can specify the task name
explicitly by setting *_name* on the class::

  from mirexec import TaskBase

  class DifferentNames (TaskBase):
      _name = 'newtask'
      _keywords = ['vis', 'line', 'flux', 'refant']
      _options =  ['nocal', 'nopass', 'mfs']


If you're feeling fancy, here's a less typing-intensive way of
generating arrays of short strings::

  from mirexec import TaskBase

  class TaskNewTask (TaskBase):
      _keywords = 'vis line flux refant'.split ()
      _options =  'nocal nopass mfs'.split ()


.. _mirexecapiref:

:mod:`mirexec` API Reference
-----------------------------

This section presents a detailed API reference for the :mod:`mirexec`
module.

Generic Task Class
^^^^^^^^^^^^^^^^^^

.. autoclass:: TaskBase
   :members:

.. autoclass:: MiriadSubprocess
   :members:

.. autoexception:: TaskLaunchError

.. exception:: TaskFailError(returncode, cmd)

   Signals that a task exited indicating failure, though it was
   able to be launched.

   :exc:`TaskFailError` may be a subclass of
   :exc:`subprocess.CalledProcessError`, if such a class exists. (It
   was introduced in Python 2.5.) Otherwise, it is a functional
   equivalent to that class.

   Instances have an attribute **returncode** indicating the exit code
   of the task. This will be nonzero, since zero indicates success. As
   far as I know, all MIRIAD tasks exit with a code of 1 unless they
   die due to a POSIX signal (in which case, the exit code is
   conventionally the negative of the signal number).

   Instances also have an attribute **cmd** which is a string version
   of the command line that was executed. The arguments are joined
   together with spaces, so there's potential for ambiguity if some
   of the argument values contain spaces.


Specific Task Classes
^^^^^^^^^^^^^^^^^^^^^

We try to keep this list up-to-date, but it may not be complete. If
you discover a wrapped task that isn't documented here, please notify
the author. As mentioned above, it's straightforward to wrap a new
task yourself: see :ref:`customtasks`.

.. autoclass:: TaskCgDisp
.. autoclass:: TaskUVList
.. autoclass:: TaskUVPlot
.. autoclass:: TaskInvert
.. autoclass:: TaskClean
.. autoclass:: TaskRestore
.. autoclass:: TaskImStat
.. autoclass:: TaskImHead
.. autoclass:: TaskIMom
.. autoclass:: TaskImFit
.. autoclass:: TaskUVAver
.. autoclass:: TaskGPCopy
.. autoclass:: TaskMSelfCal
.. autoclass:: TaskSelfCal
.. autoclass:: TaskPutHead
.. autoclass:: TaskGPPlot
.. autoclass:: TaskPrintHead
.. autoclass:: TaskClosure
.. autoclass:: TaskUVFlag
.. autoclass:: TaskUVSpec
.. autoclass:: TaskUVSort
.. autoclass:: TaskMfCal
.. autoclass:: TaskUVIndex
.. autoclass:: TaskUVCat
.. autoclass:: SmaUVPlot
.. autoclass:: SmaUVSpec
.. autoclass:: TaskUVGen
.. autoclass:: TaskUVGen2
.. autoclass:: TaskUVCal
.. autoclass:: TaskUVFlux
.. autoclass:: TaskUVFit
.. autoclass:: SmaMfCal
.. autoclass:: TaskGPCal
.. autoclass:: TaskMaths
.. autoclass:: TaskImGen
.. autoclass:: TaskLinMos
.. autoclass:: TaskImSub
.. autoclass:: TaskImMedian
.. autoclass:: TaskRegrid
.. autoclass:: TaskSFind
.. autoclass:: TaskFFT

Setting up Subprocess Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: addEnvironmentClassic

.. autofunction:: addEnvironmentAutotools

Utility Classes
^^^^^^^^^^^^^^^

.. autoclass:: DefaultedTaskType
   :members: