Interfacing deal.II to MUMPS

MUMPS should be readily packaged by most Linux distributions. Don't forget to install a develpment version of the library.

Below is a detailed instruction on how to compile and install MUMPS and some of its dependencies by hand (for the case you wish to do so).

How to compile and install MUMPS by hand

In the following, we describe the compilation process of MUMPS and its dependencies starting with BLACS and following with SCALAPACK. The compilation of MUMPS and its dependencies is not as straightforward is we would like, however, it is not so difficult that we can not talk about it. That said, we cannot discuss every possible choice of parameters, because this is too involved (and we would probably much rather be doing something productive with the interface and deal.II). Thus we give only a short walk-through mentioning the most important parameters needed for successful compliation and linking.

The compilation scheme involves repeating the same three steps for each piece of software we want installed: (i) Selecting an appropriate makefile; (ii) Modifying symbols in this makefile to suit the environment; and (iii) running make to actually compile the software.

Note: If you are using a system, which provides Intel's MKL, you should only have to compile MUMPS, since BLACS and SCALPACK are already included in the MKL. Therefore you can jump directly to section 3, where the compilation of MUMPS is considered. Otherwise start right below with the installation of BLACS.

1. BLACS

The BLACS library provides basic linear algebra routines together with a communication model such that they can be used in parallel. The compilation of this library is highly dependent on the system and software used and we can only give a general guide through the configuration process.

The library can be found on the official BLACS webpage. We require the three files mpiblacs.tgz, mpiblacs-patch03.tgz, and blacstester.tgz. After extraction of these three files in the same directory, we find a subfolder named BMAKES in that directory. Inside this subfolder there are lots of different Bmake-files for different architectures. The naming of the files are conventional: Bmake.XXX-YYY, where XXX stands for the message passing software of your system (e.g. MPI, PVM) and YYY for the system architecture (e.g. LINUX, SUN4, SUN4SOL). Pick the corresponding file for your combination of message passing software and system architecture, copy it to the root directory of BLACS, and rename it Bmake.inc. Now that we have the correct makefile, we have to adapt it to the system we are using.

We start by opening Bamke.inc in our favourite text editor and taking a look at the following three sections:

Assuming we have done all of that correctly, we go back to the BLACS root directory and compile the library with the command make mpi.

2. SCALAPACK

The SCALAPACK library provides high-performance linear algebra routines for distributed-memory message-passing MIMD computers. The library can be found on the official SCALAPACK webpage. We require the file scalapack.tgz version <= 1.8.0 only.

After extracting the file and changing to the directory which was created, we find a file called SLmake.inc.example and rename it to SLmake.inc. In this file we have to set a few parameters as we did before with BLACS' own Bmake.inc.

If all variables are set correctly we can go ahead and compile the SCALAPACK library with the command make.

3. MUMPS

Now we come to the main program for which we are doing all this. We want to install the parallel sparse direct solver MUMPS.

After downloading the solver from its official webpage, extracting it, and changing to the directory which was created, we find a subfolder named Make.inc where there are bunch of different Makefiles for different machine architecture. Here it is important to realize that for each architecture there are two different files — one with the ending .SEQ and one without. The makefiles with the extension .SEQ are the ones required for the sequential version of MUMPS. Since we want to use MUMPS in parallel, we take the file without this extension which looks suitable for the architecture we are building on and copy this into the MUMPS root directory and rename it as Makefile.inc.

As before, we open the makefile Makefile.inc in our favourite text editor and start modifying variables:

Now we are done with configuration and can compile MUMPS with the command make.

Linking with deal.II

Support for MUMPS will be enabled automatically if a system wide installation of MUMPS can be found. To use a self compiled version, specify


    -DMUMPS_DIR=/path/to/mumps
      
on the command line when invoking cmake.

For SCALAPACK and BLACS a hint - if necessary - may be given by


    -DSCALAPACK_DIR=/path/to/scalapack
    -DBLACS_DIR=/path/to/blacs
      

You can override the autodetection by manually specifying


    -DDEAL_II_WITH_MUMPS=OFF|ON
      
.

Note: Throughout this description of the compilation process of MUMPS we have emphasised adding the compiler flag -fPIC. This is a definite requirement if we are compiling deal.II with shared libraries (which is the default). If we had preferred to be compiling deal.II without shared libraries, that's ok too; in that case we would do exactly the same thing as described above, but this time omitting the -fPIC flag from the scheme.


The deal.II Authors $Date: 2013-11-03 05:56:42 -0500 (Sun, 03 Nov 2013) $