MUMPS should be readily packaged by most Linux distributions. Don't forget to install a develpment version of the library.
Below is a detailed instruction on how to compile and install MUMPS and some of its dependencies by hand (for the case you wish to do so).
In the following, we describe the compilation process of MUMPS and its dependencies starting with BLACS and following with SCALAPACK. The compilation of MUMPS and its dependencies is not as straightforward is we would like, however, it is not so difficult that we can not talk about it. That said, we cannot discuss every possible choice of parameters, because this is too involved (and we would probably much rather be doing something productive with the interface and deal.II). Thus we give only a short walk-through mentioning the most important parameters needed for successful compliation and linking.
The compilation scheme involves repeating the same three steps for each piece of software we want installed: (i) Selecting an appropriate makefile; (ii) Modifying symbols in this makefile to suit the environment; and (iii) running make to actually compile the software.
Note: If you are using a system, which provides Intel's MKL, you should only have to compile MUMPS, since BLACS and SCALPACK are already included in the MKL. Therefore you can jump directly to section 3, where the compilation of MUMPS is considered. Otherwise start right below with the installation of BLACS.
The BLACS library provides basic linear algebra routines together with a communication model such that they can be used in parallel. The compilation of this library is highly dependent on the system and software used and we can only give a general guide through the configuration process.
The library can be found on the
official BLACS
webpage. We require the three
files mpiblacs.tgz
,
mpiblacs-patch03.tgz
, and
blacstester.tgz
. After extraction of these
three files in the same directory, we find a subfolder
named BMAKES
in that directory. Inside this
subfolder there are lots of different Bmake-files for
different architectures. The naming of the files are
conventional: Bmake.XXX-YYY
,
where XXX
stands for the message passing software
of your system
(e.g. MPI, PVM)
and
YYY
for the system architecture (e.g. LINUX,
SUN4, SUN4SOL). Pick the corresponding file for your combination
of message passing software and system architecture, copy it to
the root directory of BLACS, and rename it
Bmake.inc
. Now that we have the correct makefile,
we have to adapt it to the system we are using.
We start by opening Bamke.inc in our favourite text editor and taking a look at the following three sections:
SECTION 1: PATHS AND LIBRARIES
set the
MPI environment correctly by changing the
variable MPIDIR
to the directory, where your
MPI installation is located
in. Furthermore you will probably have to adjust the
variables MPILIBdir
,
MPIINCdir
and MPILIB
.
SECTION 2: BLACS INTERNALS
the most important variables are INTFACE
and
TRANSCOMM
. If you are using OpenMPI, then set
these as follows: INTFACE = -DAdd_
and TRANSCOMM = -DUseMpi2
. Otherwise, consult
the
BLACS
webpage for details on how to set these values
correctly.
SECTION 3:
Compilers
. Here we set the
variables F77
and CC
to
correspond to the MPI Fortran and CC
compiler respectively, e.g.
mpif77
and mpicc
. An important
issue here are the variables F77FLAGS
and CCFLAGS
. It is absolutely vital to add the
flag -fPIC
to both of them, otherwise it will
not be possible to link deal.II with the
BLACS library correctly when shared
libraries are enabled.
Assuming we have done all of that correctly, we go back to
the BLACS root directory and compile the
library with the command make mpi
.
The SCALAPACK
library provides high-performance linear algebra routines for
distributed-memory message-passing MIMD
computers. The library can be found on the
official SCALAPACK
webpage. We require the
file scalapack.tgz
version <= 1.8.0 only.
After extracting the file and changing to the directory which
was created, we find a file
called SLmake.inc.example
and rename it
to SLmake.inc
. In this file we have to set a few
parameters as we did before with BLACS'
own Bmake.inc
.
home
. Here you have to take care, that
this points to the SCALAPACK directory
(i.e. where you are right now). Next, the variable
BLACSdir
points to the directory, where your
BLACS libraries are located. Then set the
variable SMPLIB
to the MPI
library.
BLACSFINIT
,
BLACSCINIT
, and
BLACSLIB
.
BLACSFINIT
has to point to your
F77init-library of BLACS. Check the
name of the BLACS library file
containing F77init
in its name and put in
this name here;
BLACSCINIT
has to point to
the BLACS library file
containing Cinit
in its name;
BLACSLIB
points to the remaining
BLACS library file.
F77
and CC
should point to
your MPI Fortran and CC compiler
respectively. Also at this place we have to add the
flag -fPIC
to the F77FLAGS
and to
the CCFLAGS
. To the variable CDEFS
you have to add the value of the
variable INTFACE
in the configuration
file Bmake.inc
in BLACS.
BLAS
and
LAPACK
point to your BLAS
and LAPACK libraries compiled on your
system.
If all variables are set correctly we can go ahead and compile
the SCALAPACK library with the
command make
.
Now we come to the main program for which we are doing all this. We want to install the parallel sparse direct solver MUMPS.
After downloading the solver from its
official webpage,
extracting it, and changing to the directory which was created, we
find a subfolder named
Make.inc
where there are bunch of different
Makefiles for different machine architecture. Here it is
important to realize that for each architecture there are two
different files — one with the ending .SEQ
and one without. The makefiles with the extension
.SEQ
are the ones required for the sequential
version of MUMPS. Since we want to use MUMPS
in parallel, we take the file
without this extension which looks suitable for the
architecture we are building on and copy this into
the MUMPS root directory and rename it
as Makefile.inc
.
As before, we open the makefile Makefile.inc
in
our favourite text editor and start modifying variables:
CC
should point to
your MPI CC
compiler, e.g. mpicc
. The
variables FC
and FL
should both
point to your MPI Fortran90 compiler,
e.g. mpif90
.
SCALAP
. This one should point to your
SCALAPACK0 and BLACS
installations as follows:
-L/XXX -lscalapack -L/YYY -lblacs -lblacsCinit
-lblacsF77init -lblacs -lblacsCinit
;
where XXX
stands for the directory containing
the SCALAPACK library
and YYY
for the directory containing the
BLACS library. The
variable INCPAR
should point to
the MPI include directory.
LIBPAR
which
should contain the same value as the
variable SCALAP
plus a pointer to
the MPI library. Therefore we propose to
set it to the following value: $(SCALAP) -L/XXX
-lmpi
, where XXX
stands for the
directory containing your MPI
libraries. Please check, if the
variable LIBBLAS
points to your BLAS
installation. The variable
CDEFS
has to contain the same value as the variable
INTFACE
of
BLACS' own Bmake.inc
. If you
are using OpenMPI this should be the
value -DAdd_
.
-fPIC
to the
three compiler flag variables OPTF
,
OPTC
, and OPTL
which follow
immediately after the above. Without adding this last flag it
will not be possible to link deal.II with
MUMPS as a shared library.
Now we are done with configuration and can
compile MUMPS with the
command make
.
Support for MUMPS will be enabled automatically if a system wide installation of MUMPS can be found. To use a self compiled version, specify
-DMUMPS_DIR=/path/to/mumpson the command line when invoking
cmake
.
For SCALAPACK and BLACS a hint - if necessary - may be given by
-DSCALAPACK_DIR=/path/to/scalapack -DBLACS_DIR=/path/to/blacs
You can override the autodetection by manually specifying
-DDEAL_II_WITH_MUMPS=OFF|ON.
Note: Throughout this description of the compilation process
of MUMPS we have emphasised adding the compiler
flag -fPIC
. This is a definite requirement if we
are compiling deal.II with shared libraries
(which is the default). If we had preferred to be compiling
deal.II without shared libraries, that's ok too;
in that case we would do exactly the same thing as described above,
but this time omitting the -fPIC
flag from the scheme.