Accounts High Performance Computing Systems Visualization Facilities Software
Systems System Utilities File Transfer Remote Connectivity Remote Display
Domain Expertise Software Development / Programming Support
Seminars Workshops Tutorials User Groups Case Studies White Papers

PETSc

Overview

The following overview is taken from the PETSc web page:

PETSc, pronounced PET-see (the S is silent), is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations. It employs the MPI standard for parallelism.

PETSc is intended for use in large-scale application projects. PETSc is easy to use for beginners. Moreover, its careful design allows advanced users to have detailed control over the solution process. PETSc includes a large suite of parallel linear, nonlinear equation solvers and ODE integrators that are easily used in application codes written in C, C++, Fortran and now Python. PETSc provides many of the mechanisms needed within parallel application codes, such as simple parallel matrix and vector assembly routines that allow the overlap of communication and computation. In addition, PETSc includes support for parallel distributed arrays useful for finite difference methods. 

For a full list of features please refer to the PETSc features page available at URL http://www.mcs.anl.gov/petsc/petsc-as/features/index.html.

Setup

Use of PETSc does not require any special setup.

Availability

The version of PETSc you use is selected at link time by which set of libraries you choose to link into your code. The current installed and recommended version to use is 3.1-p8.

The PETSc library is also available in production and debugging modes.

PETSc is supported for the Intel, PGI, and GNU compilers on RCC systems. You select which combinaation of version, mode, and compiler by selecting settings from the following table.

Versions installed are:

Available PETSc Architectures
Compiler Architecture
Intellinux-intel11-opt
linux-intel11-debug
PGIlinux-pgi-opt
linux-pgi-debug
GNUlinux-gnu-opt
linux-gnu-debug

Usage

To use PETSc you must first set two environment variables based on the release of PETSc (PETSC_DIR) and the particular architecture (PETSC_ARCH).

The recommended setting for PETSC_DIR is:

PETSC_DIR=/usr/global/petsc/3.1-p8

The setting of PETSC_ARCH is determined by using the table above for compiler and architecture versions.

Examples:

For production code using the Intel compiler the settings would be:

export PETSC_DIR=/usr/global/petsc/3.1-p8
export PETSC_ARCH=linux-intel11-opt

Codes with debugging enabled using the Intel compiler would likewise use:

export PETSC_DIR=/usr/global/petsc/3.1-p8
export PETSC_ARCH=linux-intel11-debug

Codes with debugging enabled using the GNU compiler would likewise use:

export PETSC_DIR=/usr/global/petsc/3.1-p8
export PETSC_ARCH=linux-gnu-debug

Linking strings when using the Intel compiler

When compiling and linking applications that use PETSc you must use the appropriate MPI compiler. Be sure to load the correct MPI module file for the compiler you wish to use.

You then use the PETSC_DIR and PETSC_ARCH setting to link in the appropriate PETSc library. You also must link in the BLAS and LaPACK libraries required by PETSc. Additionally you must also link in the X11 library.

Examples:

Using the Intel compiler with the production optimized PETSc library you would use the following instructions to link your code:

First set the appropriate environment variable, make sure correct openmpi module is loaded.

export PETSC_DIR=/usr/global/petsc/3.1-p8
export PETSC_ARCH=linux-intel11-opt

module load openmpi/intel

Use the MPI compiler for your chose language and be sure to use the following include and library link settings.

-I$PETSC_DIR/include -I$PETSC_DIR/$PETSC_ARCH/include 
-L$PETSC_DIR/$PETSC_ARCH/lib -lpetsc 
-L/usr/global/intel/mkl/10.3.1.107/mkl/lib/intel64
-Wl,-R/usr/global/intel/mkl/10.3.1.107/mkl/lib/intel64
-lmkl_solver_lp64_sequential 
-Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group
-lX11

Examples

Further Examples

Please refer to URL http://www.mcs.anl.gov/petsc/petsc-as/documentation/tutorials/index.html for a list of available on-line PETSc tutorials.

Documentation and Support

Please refer to the PETSc documentation pages available at URL http://www.mcs.anl.gov/petsc/petsc-as/documentation/index.html.