\chapter{Gretl and Python}
\label{chap:gretlPython}

\section{Introduction}
\label{Python-intro}

According to \url{www.python.org}, \app{Python} is ``an easy to learn,
powerful programming language. It has efficient high-level data
structures and a simple but effective approach to object-oriented
programming. Python's elegant syntax and dynamic typing, together with
its interpreted nature, make it an ideal language for scripting and
rapid application development in many areas on most platforms.''

Indeed, \app{Python} is widely used in a great variety of
contexts. Numerous add-on modules are available; the ones likely to be
of greatest interest to econometricians include \app{NumPy} (``the
fundamental package for scientific computing with Python''---see
\url{www.numpy.org}); \app{SciPy} (which builds on \app{NumPy}---see
\url{www.scipy.org}); and \app{Statsmodels}
(\url{http://statsmodels.sourceforge.net/}).

\section{Python support in gretl}
\label{sec:Python-support}

The support offered for \app{Python} in gretl is similar to that
offered for \app{Octave} (chapter~\ref{chap:gretlOctave}). You can
open and edit \app{Python} scripts in the gretl GUI.  Clicking
the ``execute'' icon in the editor window will send your code to
\app{Python} for execution. In addition you can embed \app{Python}
code within a gretl script using a \texttt{foreign} block, as
described in connection with \app{R}.

When you launch \app{Python} from within gretl one variable and
two convenience functions are pre-defined, as follows.
\begin{code}
gretl_dotdir
gretl_loadmat(filename, autodot=1)
gretl_export(M, filename, autodot=1)
\end{code}
The variable \verb|gretl_dotdir| holds the path to the user's ``dot
directory.''  The first function loads a matrix of the given
\texttt{filename} as written by gretl's \texttt{mwrite}
function, and the second writes matrix \texttt{M}, under the given
\texttt{filename}, in the format wanted by gretl.

By default the traffic in matrices goes via the dot directory on the
\app{Python} side; that is, the name of this directory is prepended to
\texttt{filename} for both reading and writing. (This is complementary
to use of the \textsl{export} and \textsl{import} parameters with
gretl's \texttt{mwrite} and \texttt{mread} functions, respectively.)
However, if you wish to take control over the reading and writing
locations you can supply a zero value for \texttt{autodot} (or give an
absolute path) when calling \verb|gretl_loadmat| and
\verb|gretl_export|: in that case the \texttt{filename} argument is
used as is.

Note that \verb|gretl_loadmat| and \verb|gretl_export| depend on
\app{NumPy}; they make use of the functions \texttt{loadtxt} and
\texttt{savetxt} respectively. Nonetheless, the presence of
\app{NumPy} is not an absolute requirement if you don't need
to use these two functions.

\section{Illustration: linear regression with multicollinearity}
\label{sec:Python-longley}

Listing~\ref{Python-longley} compares the numerical accuracy of
gretl's \texttt{ols} command with that of the function
\texttt{linalg.lstsq} in \app{NumPy}, using the notorious Longley test
data which exhibit extreme multicollinearity.  Unlike some
econometrics packages, \app{NumPy} does a good job on these data. The
script computes and prints the log-relative error in estimation of the
regression coefficients, using the NIST-certified values as a
benchmark;\footnote{See
  \url{http://www.itl.nist.gov/div898/strd/lls/data/Longley.shtml}.}
the error values correspond to the number of correct digits (with a
maximum of 15). The results will likely differ somewhat by computer
architecture and compiler.

\begin{script}[htbp]
  \caption{Comparing regression results with \app{Python}}
\begin{scode}
set verbose off

function matrix logrel_err (const matrix est, const matrix true)
  return -log10(abs(est - true) ./ abs(true))
end function

open longley.gdt -q
list LX = prdefl .. year
ols employ 0 LX -q
matrix b_gretl = $coeff

mwrite({employ} ~ {const} ~ {LX}, "alldata.mat", 1)

foreign language=python
   import numpy as np
   X = gretl_loadmat('alldata.mat', 1)
   # NumPy's OLS   
   b = np.linalg.lstsq(X[:,1:], X[:,0])[0]
   gretl_export(np.transpose(np.matrix(b)), 'py_b.mat', 1)
end foreign

# NIST's certified coefficient values
matrix b_nist = {-3482258.63459582, 15.0618722713733,
    -0.358191792925910E-01, -2.02022980381683,
    -1.03322686717359, -0.511041056535807E-01,
     1829.15146461355}'

matrix b_numpy = mread("py_b.mat", 1)
matrix E = logrel_err(b_gretl, b_nist) ~ logrel_err(b_numpy, b_nist)
colnames(E, "gretl python")
printf "Log-relative errors, Longley coefficients:\n\n%#12.5g\n", E
printf "Column means\n%#12.5g\n", meanc(E)
\end{scode}
\begin{code}
Log-relative errors, Longley coefficients:

       gretl      python
      12.844      12.850
      11.528      11.414
      12.393      12.401
      13.135      13.121
      13.738      13.318
      12.587      12.363
      12.848      12.852

Column means
      12.725      12.617
\end{code}
\label{Python-longley}
\end{script}

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "gretl-guide"
%%% End: 

