Scientific and High Performance libraries

There are several mathematical and/or high performance libraries installed on the various cluster. The two major advantages that libraries offer are functionality and performce. Most of the libraries installed exist primarily to offer functionality, but some exist also to provide performance.
Consequently, when a certain functionality is provided by more than one library (with comparable quality), the user should pick the one that has the highest performance. A list of the most common libraries follows:

• FFTW

FFTW is a portable, free, high performance DFT library which implements arbitrary radix, serial and multiple dimensional transforms in serial or parallel (MPI and POSIX threads models). One unique aspect of FFTW is its optional use of self-optimizing strategies, whereby subsequent calls become faster by building on previous timings. Wrapper functions provide a Fortran API to this native C library.



Package	Platform	Version	Where	Docs
fftw	Briareo	2.1.3(*)(**)	/usr/local/lib	Vendor
fftw	Opteron	2.1.5 (64 bit)	/opt/fftw-2.1.5(***)	Vendor
fftw	Helium	2.1.5 (64-bit)	/opt/fftw-2.1.5_64	Vendor
fftw	Helium	2.1.5 (32-bit)	/opt/fftw-2.1.5_32	Vendor

(*) compiled with MPI support
(**) compiled with pgi fortran interface
(***) for users of pw-scf and other espresso package codes the version in /opt/fftw-2.1.5_test on Opteron is to be used. See /opt/espresso/make.sys for details and example linking directives.

How to use the FFTW library

Here below the value $FFTW environment variable should be set accordingly to the value in third column in the above table.

      % gcc   $FFTW -lfftw -lm ...  or 
      % g77/g95   $FFTW -lfftw  ...  or 
      % ifort   $FFTW -lfftw ...  or
      % mpi_gcc   $FFTW -lfftw_mpi -lfftw -lm ... 
    

Example Programs

• Look in $FFTW_ROOT/examples for the following and more.
• Makefile
• Serial - xfftw_1d.c, xfftw_1d_fort.f
• Parallel (MPI) - xfftw_3d_mpi.c

Documentation

for FFTW on-line documentation in HTML format, see the FFTW website .




---




• LAPACK

Linear Algebra PACKage (LAPACK) provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. It supports both single and double precision.



Package	Platform	Version	Where	Docs
Lapack	Briareo	3.0	/usr/local/lib	Vendor
Lapack	Opteron	NOT INSTALLED (*)		Vendor
Lapack	Helium	3.03 64 bit	/usr/lib64/	Vendor
Lapack	Helium	3.03 32 bit	/usr/lib/	Vendor

(*) On opteron a version with the PGI compiler is available in /opt/pgi/linux86-64/5.2/lib/liblapack.a however it is not recommended for a, other compilers b, performance. The library /opt/atlas/ibatlas_amd64.a and libatlas_x86_64.a contains LAPACK combined with BLAS and atlas for 64 bit codes. This library is recommended.


How to use them




More Information

See LAPACK Users' Guide for subroutine calls and description of calling sequences.



---




• ATLAS 3.4.1

Automatically Tuned Linear Algebra Software

The ATLAS (Automatically Tuned Linear Algebra Software) project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance. At present, it provides C and Fortran77 interfaces to a portably efficient BLAS implementation, as well as a *few routines* from LAPACK.

ATLAS does not provide a full LAPACK library. However, we include in a full LAPACK library the ATLAS faster LAPACK routines.

To use them:


pgf77 foo.f -L/usr/local/lib -lpgiblas -lpgiatlas (PGI Fortran compiler)
fcc foo.f -L/usr/local/lib -lifcblas -lifcatlas Intel Fortran compiler)
g77 foo.f -L/usr/local/lib -lgnublas -lgnuatlas (GNU Fortran compiler)
mpi_pgf77 -O3 foo.f -L/usr/local/lib -lpgiblas -lpgiatlas

On opteron the library /opt/atlas/libatlas_amd64.a and libatlas_x86_64.a contains LAPACK combined with BLAS and ATLAS for 64 bit codes. This library is recommended. Linking directives

ifort -L/opt/atlas -latlas_x86_64

See for example make.sys in /opt/espresso

or Makefile in /opt/cpmd





---




• INTEL MKL library

The "Math Kernel Library" consists of functions in the following computational areas:


• BLAS (vector-vector, matrix-vector, matrix-matrix operations) and extended BLAS for sparse computations
• LAPACK for linear algebraic equations solutions and eigensystem analysis
• Fast Fourier transforms
• Interface for use by both C and Fortran compilers, for all of the above functionality in addition to availability for all precisions and/or types. Some of functionality have improved performance by multi-threaded implementation.
In addition, MKL also offers a set of functions collectively known as VML -- the "Vector Math Library". VML is a set of vectorized transcendental functions which offer both high performance and excellent accuracy compared to the /libm/ functions for most of the processors.
For the Pentium 3 processor, the accuracy of these functions approaches that of libm, but these vectorized functions are considerably faster for vectors longer than a few elements.

For running MKL and/or VML, the library directory, name and the include directory needs to be included in your makefile or command line. Below is an example of how to compile and link a program which calls a BLAS function in the MKL. Note that the library is for use in a single node, hence can be used by both serial compilers or by MPI wrapper scripts.

mpicc -O3 -I/usr/local/intel/mkl/include foo.c -L/usr/local/intel/mkl/lib/32 -lmkl_p3





---




• ACML 2.5.0

AMD Core Math Library (ACML). Contains optimized routines from Level 2 and 3 BLAS, LAPACK, and 2D and 3D FFT optimized for AMD arcitecture. See
http://www.developwithamd.com/appPartnerProg/acml/home/index.cfm?action=home
As a seperate library there is also specialized math functions for vectorized sin, cos, log etc. On opteron, see: /opt/acml2.5.0/
libacml-ACML, BLAS, LAPACK FFT
libacml_mv-Vectorized math functions
Libraries are compiled with both gnu
/opt/acml2.5.0/gnu64 (useful for all compilers)
and specificly for pgi
/opt/acml2.5.0/pgi64

Linking directives:

ifort fool.f -L/opt/acml2.5.0/gnu64/lib/. -lacml -lg2c
mpi_ifort ... -L/opt/acml2.5.0/gnu64/lib -lacml -lg2c -lacml_mv

(1) the vectorized math functions may also be used seperately with ATLAS etc. Example:

mpi_ifort -L/opt/atlas -latlas_x86_64 \
-L/opt/acml2.5.0/gnu64/lib -lacml_mv \
-Xlinker -Bdynamic -lc -lm -Xlinker -Bstatic

(2) Use of ACML (as compiled) requires the g2c library.
(3) See /opt/espresso/make.sys and /opt/cpmd/Makefile for examples.

Tips and Tricks

• ATLAS 3.6.0

To use them:

ATLAS+LAPACK
g77 foo.f -L/usr/local/lib -lgnulapack -lgnublas -lgnuatlas (GNU Fortran compiler)
ifc foo.f -L/usr/local/lib -lifclapack -lifcblas -lifcatlas (Intel Fortran compiler 7.1)
ifort foo.f -L/usr/local/lib -lifortlapack -lifortblas -lifortatlas (Intel Fortran compiler 8.0)
pgf77 foo.f -L/usr/local/lib -lpgilapack -lpgiblas -lpgiatlas (PGI Fortran compiler)


ATLAS+BLAS
g77 foo.f -L/usr/local/lib -lgnublas -lgnuatlas (GNU Fortran compiler)
ifc foo.f -L/usr/local/lib -lifcblas -lifcatlas (Intel Fortran compiler 7.1)
ifort foo.f -L/usr/local/lib -lifortblas -lifortatlas (Intel Fortran compiler 8.0)
pgf77 foo.f -L/usr/local/lib -lpgiblas -lpgiatlas (PGI Fortran compiler)


To use them with gnu and ifort

ifort -O3 foo.f -L/opt/acml2.5.0/gnu64/lib/. -lacml -lg2c -O3 foo.f -L/opt/acml2.5.0/gnu64/lib/. -lacml -lg2c -lacmlmv \
-Xlinker -Bdynamic -lc -lm -Xlinker -Bstati
mpi_pgf77 -O3 foo.f -L/opt/acml2.5.0/pgi64/lib





---




• ATLAS

provided by Axel Kohlmeyer
Libraries contain BLAS LAPACK AND ATLAS.
Libraries available for Intal and AMD architectures)

To use them:

pgf77 foo.f -L/opt/atlas/ -latlas_x86_64 (PGI Fortran compiler)
ifcc foo.f -L/opt/atlas/ -latlas_x86_64 (Intel Fortran compiler)
g77 foo.f -L/opt/atlas/ -latlas_x86_64 (GNU Fortran compiler)
mpi_pgf77 -O3 foo.f -L/opt/atlas/ -latlas_x86_64





---




• INTEL MKL library

(/opt/interl/mkl72)
The "Math Kernel Library" consists of functions in the following computational areas:

• BLAS (vector-vector, matrix-vector, matrix-matrix operations) and extended BLAS for sparse computations
• LAPACK for linear algebraic equations solutions and eigensystem analysis
• Fast Fourier transforms
• Interface for use by both C and Fortran compilers, for all of the above functionality in addition to availability for all precisions and/or types. Some of functionality have improved performance by multi-threaded implementation
In addition, MKL also offers a set of functions collectively known as VML -- the "Vector Math Library". VML is a set of vectorized transcendental functions which offer both high performance and excellent accuracy compared to the /libm/ functions for most of the processors. For the Pentium 3 processor, the accuracy of these functions approaches that of libm, but these vectorized functions are considerably faster for vectors longer than a few elements.

For running MKL and/or VML, the library directory, name and the include directory needs to be included in your makefile or command line. Below is an example of how to compile and link a program which calls a BLAS function in the MKL. Note that the library is for use in a single node, hence can be used by both serial compilers or by MPI wrapper scripts.

mpicc -O3 -I/opt/intel/mkl72/include foo.c -L/opt/intel/mkl72//lib/64 -lmkl_lapack -lmkl_ipf -lguide -pthreads

For additional documentation and reference on MKL, both pdf and html-based, please look in the directory /usr/local/intel/mkl/doc.