From NWChem

Compiling NWChem from source

On this page, a step-by-step description of the build process and necessary and optional environment variables is outlined. In addition, based on the experiences of developers and users how-to's for various platforms have been created. These how-to's will be updated with additional platforms and better environment variables over time.

Setting up the proper environment variables

NWCHEM_TOP defines the top directory of the NWChem source tree, e.g.

  % setenv NWCHEM_TOP <your path>/nwchem

NWCHEM_TARGET defines your target platform, e.g.

  % setenv NWCHEM_TARGET LINUX64

The platforms that available are:
NWCHEM_TARGET	Platform	OS/Version	Compilers
LINUX	x86 ppc	RedHat, MDK, SLES YD2.1, SLES	GNU, Intel, PGI GNU, xlf
LINUX64	ia64 x86_64 ppc64	RedHat SLES, RedHat SLES, RedHat	Intel GNU, PGI, PathScale, Intel xlf
BGL BGP	Bluegene/L Bluegene/P	SLES	blrts_xlf bgxlf
LAPI LAPI64	IBM SP	AIX/LAPI
IBM IBM64	IBM RS/6000	AIX
MACX MACX64	Apple MacOSX	OSX, Lion	GNU, xlf, Intel
CYGWIN WIN32	Intel x86	Windows with Cygwin Windows	GNU Compaq
SGI_N32 SGITFP	SGI 64 bit OS using 32 bit integers SGI 64 bit OS	IRIX
HPUX HPUX64	PA-RISC PA-RISC, ia64	HPUX

ARMCI_NETWORK must be defined in order to achieve best performance on high performance networks, .e.g.

  % setenv ARMCI_NETWORK OPENIB

For a single processor system, this environment variable does not have to be defined.

Supported combination of ARMCI_NETWORK and NWCHEM_TARGET variables:
ARMCI_NETWORK	NWCHEM_TARGET	Network	Protocol
OPENIB	LINUX, LINUX64	InfiniBand	OpenIB
MPI-MT MPI-SPAWN	LINUX64	Myrinet MX, Infiniband	MPI2
MELLANOX	LINUX, LINUX64	InfiniBand	VAPI
GM	LINUX, LINUX64	Myrinet	GM
GEMINI	LINUX64	Cray Gemini	Gemini
PORTALS	LINUX64	Cray SeaStar/HyperTransport	PORTALS
BGMLMPI	BGL	IBM BlueGene Torus	BGLMPI
DCMFMPI	BGP	Global Tree/Interrupt	DCMF, MPI
VIA	LINUX	Giganet/CLAN	VIA

Some more information can be found on the global arrays/armci web page.

MPI variables needed to compile. For a single processor system, these environment variables do not have to be defined.

USE_MPI	Set to "y" to indicate that NWChem should be compiled with MPI
USE_MPIF	Set to "y" for the NWPW module to use fortran-bindings of MPI (Generally set when USE_MPI is set)
USE_MPIF4	Set to "y" for the NWPW module to use Integer*4 fortran-bindings of MPI. (Generally set when USE_MPI is set on most platforms)
LIBMPI	Name of the MPI library that should be linked with -l (eg. -lmpich)
MPI_LIB	Directory where the MPI library resides
MPI_INCLUDE	Directory where the MPI include files reside

E.g. for MPICH2, this looks like:

  % setenv USE_MPI y
  % setenv USE_MPIF y
  % setenv USE_MPIF4 y
  % setenv LIBMPI "-lmpich -lopa -lmpl -lrt -lpthread"
  % setenv MPI_LIB <Your path to MPICH2 libraries>/lib 
  % setenv MPI_INCLUDE <Your path to MPICH2 libraries>/include

For some specific implementations the settings for MPI_LIB, MPI_INCLUDE, and LIBMPI look like:
MPI Implementation	Environment variables
MPICH	setenv MPI_LOC /usr/local #location of mpich installation setenv MPI_LIB $MPI_LOC/lib setenv MPI_INCLUDE $MPI_LOC/include setenv LIBMPI "-lfmpich -lmpich -lpmpich"
MPICH2	setenv MPI_LOC /usr/local #location of mpich2 installation setenv MPI_LIB $MPI_LOC/lib setenv MPI_INCLUDE $MPI_LOC/include setenv LIBMPI "-lmpich -lopa -lmpl -lrt -lpthread"
OPENMPI	setenv MPI_LOC /usr/local #location of openmpi installation setenv MPI_LIB $MPI_LOC/lib setenv MPI_INCLUDE $MPI_LOC/include setenv LIBMPI "-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil"

Note:

When MPI is used, the appropriate MPI run command should be used to start an NWChem calculation, e.g.

  % mpirun -np 8 $NWCHEM_TOP/bin/${NWCHEM_TARGET}}/nwchem h2o.nw

When all nodes are connected via shared memory and the ch_shmem version of MPICH is installed and used, NWChem can be called directly, e.g.

  % $NWCHEM_TOP/bin/${NWCHEM_TARGET}/nwchem -np 8 h2o.nw

NWCHEM_MODULES defines the modules to be compiled, e.g.

  % setenv NWCHEM_MODULES "all python"

The following modules are available:
Module	Description
all	Everything useful
all python	Everything useful plus python
qm	All quantum mechanics modules
md	MD only build

Note that additional environment variables need to be defined to specify the location of the Python libraries, when the python module is compiled. See the optional environmental variables section for specifics.

Adding optional environmental variables

LARGE_FILES can be set circumvent the 2 GB limit where possible (note that your system administrator must also enable large files in the file system), e.g.

  % setenv LARGE_FILES TRUE

USE_NOFSCHECK can be set to avoid NWChem creating files for each process when testing the size of the scratch directory (a.k.a. creation of junk files), e.g.

  % setenv USE_NOFSCHECK TRUE

LIB_DEFINES can be set to pass additional defines to the C preprocessor (for both Fortran and C), e.g.

  % setenv LIB_DEFINES -DDFLT_TOT_MEM=16777216

Note: -DDFLT_TOT_MEM sets the default dynamic memory available for NWChem to run, where the units are in doubles. Instead of manually defining these one can optionally use the "getmem.nwchem" script in the $NWCHEM_TOP/contrib directory. This script should be run after an initial build of the binary has been completed. The script will assess memory availability and make an educated guess, recompile the appropriate files and relink.

MRCC_THEORY can be set to request the multireference coupled cluster capability to be included in the code, e.g.

  % setenv MRCC_THEORY TRUE

Setting Python environment variables

Python programs may be embedded into the NWChem input and used to control the execution of NWChem. To build with Python, Python needs to be available on your machine. The software can be download from http://www.python.org . Follow the Python instructions for installation and testing. NWChem has been tested with Python versions 1.5.1, 1.5.2, 1.6 and 2.0.

The following environment variables need to be set when compiling with Python:

  % setenv PYTHONHOME /usr/local/Python-1.5.1
  % setenv PYTHONVERSION 1.5
  % setenv USE_PYTHON64  y

Note that the third number in the version should not be kept: 2.2.3 should be set as 2.2

To run with Python, make sure that PYTHONHOME is set as mentioned above. You will also need to set PYTHONPATH to include any modules that you are using in your input. Examples of Python within NWChem are in the $NWCHEM_TOP/QA/tests/pyqa and $NWCHEM_TOP/contrib/python directories.

Using optimized math libraries

By default NWChem uses its own basic linear algebra subroutines (BLAS). To include faster BLAS routines, the environment variable BLASOPT needs to be set before building the code. For example, with ATLAS

  % setenv BLASOPT "-L/usr/local/ATLAS -lf77blas -latlas"

Good choices of optimized BLAS libraries on x86 (e.g. LINUX and LINUX64) hardware include:
GotoBLAS	http://www.tacc.utexas.edu/tacc-projects/gotoblas2
Intel MKL	http://www.intel.com/software/products/mkl/
AMD ACML	http://developer.amd.com/libraries/acml/pages/default.aspx
ATLAS	http://math-atlas.sf.net
Cray LibSci	Available only on Cray x86_64 hardware, it is automatically linked when compiling on Cray XT and XE computers.

NWChem can also take advantage of the ScaLAPACK library if it is installed on your system. The following environment variables need to be set:

  % setenv USE_SCALAPACK y

  % setenv SCALAPACK "location of Scalapack and BLACS library"

WARNING: In the case of 64-bit platforms, most vendors optimized BLAS libraries cannot be used. This is due to the fact that while NWChem uses 64-bit integers (i.e. integer*8) on 64-bit platforms, most of the vendors optimized BLAS libraries used 32-bit integers. BLAS libraries not supporting 64-bit integers (at least in their default options/installations) include CXML (DECOSF), ESSL (LAPI64), MKL (LINUX64/ia64 and x86_64), ACML(LINUX64/x86_64), and GotoBLAS2(LINUX64). The same holds for the ScaLAPACK libraries, which internally use 32-bit integers.

A method is now available to link against the libraries mentioned above, using the following procedure:

  % cd $NWCHEM_TOP/src
  % make clean
  % make 64_to_32
  % make USE_64TO32=y HAS_BLAS=yes BLASOPT=" optimized BLAS"

E.g., for IBM64 this looks like

  % make  USE_64TO32=y HAS_BLAS=yes  BLASOPT="-lessl -lmass"

Notes:

GotoBLAS2 can be installed with 64bit integers. This is accomplished by compiling the GotoBLAS2 library after having by edited the GotoBLAS2 Makefile.rule file and un-commenting the line containing the INTERFACE64 definition. In other words, the line

          #INTERFACE64 = 1

                needs to be changed to

          INTERFACE64 = 1

ACML and MKL can support 64-bit integers if the appropriate library is chosen. For MKL, one can choose the ILP64 Version of Intel® MKL, while for ACML the int64 libraries should be chosen, e.g. in the case of ACML 4.4.0 using a PGI compiler /opt/acml/4.4.0/pgi64_int64/lib/libacml.a

Linking in NBO

The current versions of NBO provide a utility to generate source code that can be linked into computational chemistry packages such as NWChem. To utilize this functionality, follow the instructions in the NBO package to generate an nwnbo.f file. Linking NBO into NWChem can be done using the following procedure:

  % cd $NWCHEM_TOP/src
  % cp nwnbo.f $NWCHEM_TOP/src/nbo/.
  % make nwchem_config NWCHEM_MODULES="all nbo"
  % make

One can now use "task nbo" and incorporate NBO input into the NWChem input file directly:

 nbo
   $NBO NRT $END
   ...
 end

 task nbo

Building the NWChem binary

Once all required and optional environment variables have been set, NWChem can be compiled:

  % cd $NWCHEM_TOP/src

  % make nwchem_config

  % make >& make.log

The make above will use the standard compilers available on your system. To use compilers different from the default one can either set environment variables:

  % setenv FC <fortran compiler>
  % setenv CC <c compiler>

Or one can supply the compiler options to the make command, e.g:

  % make FC=ifort CC=icc

For example, on Linux FC could be set either equal to ifort, gfortran or pgf90

Note 1: If in a Linux environment, FC is set equal to anything other than the tested compilers, there is no guarantee of a successful installation, since the makefile structure has not been tested to process other settings. In other words, please avoid make FC="ifort -O3 -xhost" and stick to make FC="ifort", instead

Note 2: It's better to avoid redefining CC, since a) NWChem does not have C source that is a computational bottleneck and b) we typically test just the default C compiler. In other words, the recommendation is to compile with make FC=ifort

How-to: Linux workstation platforms

Common environmental variables for building in serial or in parallel with MPI

 % setenv NWCHEM_TOP <your path>/nwchem
 % setenv NWCHEM_TARGET LINUX64
 % setenv NWCHEM_MODULES  all

Common environmental variables for building with MPI

The following environment variables need to be set when NWChem is compiled with MPI:

 % setenv USE_MPI y
 % setenv USE_MPIF y
 % setenv USE_MPIF4 y
 % setenv MPI_LOC <your path>/openmpi-1.4.3  (for example, if you are using OpenMPI)
 % setenv MPI_LIB <your path>/openmpi-1.4.3/lib
 % setenv MPI_INCLUDE <your path>/openmpi-1.4.3/include
 % setenv LIBMPI "-lmpi_f90 -lmpi_f77 -lmpi -lpthread"

Compiling the code once all variables are set

  % cd $NWCHEM_TOP/src

  % make nwchem_config

  % make FC=gfortran >& make.log

How to: Mac platforms

Compilation of NWChem 6.1.1 release on Mac OS X Lion 10.7

Download and unpack latest NWChem tarball to the directory of your choosing, say /Users/johndoe/nwchem
Download XCode from the App store
From within Xcode install command line tools
Download gfortran 4.6.2 from http://gcc.gnu.org/wiki/GFortranBinaries#MacOS
To compile serial version of NWChem 6.1.1 set (unset) the following environmental variables

unsetenv USE_MPI
setenv NWCHEM_MODULES all
setenv OLD_GA  yes
setenv NWCHEM_TARGET MACX
setenv NWCHEM_TOP /Users/johndoe/nwchem

Go to your source directory, configure, and compile

 cd /Users/johndoe/nwchem/src
 make nwchem_config
 make

That is it !

How-to: Cray platforms

Common environmental variables for building and running on the Cray XT and XE:

  % setenv NWCHEM_TOP <your path>/nwchem
  % setenv NWCHEM_TARGET LINUX64
  % setenv NWCHEM_MODULES all
  % setenv USE_MPI y
  % setenv USE_MPIF y
  % setenv USE_MPIF4 y
  % setenv USE_SCALAPACK y
  % setenv USE_64TO32 y
  % setenv LIBMPI " "

Portals, e.g. XT3, XT4, XT5

Set the following environmental variable for compilation:

  % setenv ARMCI_NETWORK PORTALS

Gemini, e.g. XE6

Load the onsided module by executing the command

  % module load onesided

Note: Preferred version of onesided is 1.5.0 or later ones.

Set the environmental variable for compilation:

  % setenv ARMCI_NETWORK GEMINI
  % setenv ONESIDED_USE_UDREG 1

Compiling the code on Cray once all variables are set

  % cd $NWCHEM_TOP/src

  % make nwchem_config

  % make 64_to_32

  % make FC=ftn >& make.log

How-to: IBM platforms

Compiling NWChem on BLUEGENE/L

The following environment variables need to be set

  % setenv NWCHEM_TOP <your path>/nwchem
  % setenv NWCHEM_TARGET BGL
  % setenv ARMCI_NETWORK BGMLMPI
  % setenv BGLSYS_DRIVER /bgl/BlueLight/ppcfloor
  % setenv BGLSYS_ROOT ${BGLSYS_DRIVER}/bglsys
  % setenv BLRTS_GNU_ROOT ${BGLSYS_DRIVER}/blrts-gnu
  % setenv BGDRIVER ${BGLSYS_DRIVER}
  % setenv BGCOMPILERS ${BLRTS_GNU_ROOT}/bin
  % setenv USE_MPI y
  % setenv LARGE_FILES TRUE
  % setenv MPI_LIB ${BGLSYS_ROOT}/lib
  % setenv MPI_INCLUDE ${BGLSYS_ROOT}/include
  % setenv LIBMPI "-lfmpich_.rts -lmpich.rts -lmsglayer.rts -lrts.rts -ldevices.rts"
  % setenv BGMLMPI_INCLUDE /bgl/BlueLight/ppcfloor/bglsys/include
  % setenv BGMLLIBS /bgl/BlueLight/ppcfloor/bglsys/lib

To compile, the following commands should be used:

 % cd $NWCHEM_TOP/src

 % make nwchem_config

 % make FC=blrts_xlf >& make.log

Compiling NWChem on BLUEGENE/P

The following environment variables need to be set

  % setenv NWCHEM_TARGET BGP
  % setenv ARMCI_NETWORK DCMFMPI
  % setenv MSG_COMMS DCMFMPI
  % setenv USE_MPI y
  % setenv LARGE_FILES TRUE
  % setenv BGP_INSTALLDIR /bgsys/drivers/ppcfloor
  % setenv BGCOMPILERS /bgsys/drivers/ppcfloor/gnu-linux/bin
  % setenv BGP_RUNTIMEPATH  /bgsys/drivers/ppcfloor/runtime
  % setenv ARMCIDRV ${BGP_INSTALLDIR}
  % setenv BGDRIVER ${ARMCIDRV}
  % setenv MPI_LIB ${BGDRIVER}/comm/lib
  % setenv MPI_INCLUDE ${BGDRIVER}/comm/include
  % setenv LIBMPI "-L${MPI_LIB} -lfmpich_.cnk -lmpich.cnk -ldcmfcoll.cnk -ldcmf.cnk -lpthread -lrt -L${BGP_RUNTIMEPATH}/SPI -lSPI.cna"
  % setenv BGMLMPI_INCLUDE ${MPI_INCLUDE}

To compile, the following commands should be used:

 % cd $NWCHEM_TOP/src

 % make nwchem_config

 % make FC=bgxlf >& make.log

Compiling NWChem on IBM PowerPC architectures

The following environment variables should be set:

  % setenv NWCHEM_TOP <your path>/nwchem
  % setenv NWCHEM_TARGET IBM64
  % setenv ARMCI_NETWORK MPI-MT
  % setenv OBJECT_MODE 64
  % setenv USE_MPI y
  % setenv LARGE_FILES TRUE
  % setenv MPI_LIB /usr/lpp/ppe.poe/lib
  % setenv MPI_INCLUDE /usr/lpp/ppe.poe/include
  % setenv LIBMPI "-lmpi -lpthreads"

To compile, the following commands should be used:

 % cd $NWCHEM_TOP/src

 % make nwchem_config

 % make FC=xlf >& make.log

How-to: Commodity clusters with Infiniband

Common environmental variables for building and running on most Infiniband clusters are:

  % setenv NWCHEM_TOP <your path>/nwchem
  % setenv NWCHEM_TARGET LINUX64
  % setenv NWCHEM_MODULES "all
  % setenv USE_MPI y
  % setenv USE_MPIF y
  % setenv USE_MPIF4 y
  % setenv MPI_LIB <Location of MPI library>/lib
  % setenv MPI_INCLUDE <Location of MPI library>/include
  % setenv LIBMPI <MPI library, e.g -lmtmpi or -lmpich>

On Infiniband clusters with the OpenIB software stack, the following environment variables should be defined

  % setenv ARMCI_NETWORK OPENIB
  % setenv IB_INCLUDE <Location of Infiniband libraries>/include
  % setenv MSG_COMMS MPI

On Infiniband clusters that do not support OpenIB, such as Myrinet MX, the MPI2 protocol can be used

  % setenv ARMCI_NETWORK MPI-MT

Compiling the code on an Infiniband cluster once all variables are set

  % cd $NWCHEM_TOP/src

  % make nwchem_config

  % make >& make.log

How-to: Windows Platforms

The current recommended approach for building a NWChem binary for a Windows platform is to build within Cygwin with make, perl, and gcc/gfortran version 4 installed.

  % setenv NWCHEM_TOP <your path>/nwchem
  % setenv NWCHEM_TARGET CYGWIN
  % setenv NWCHEM_MODULES all

  % cd $NWCHEM_TOP/src

  % make nwchem_config

  % make >& make.log

It used to be possible to build a version for native Windows using the Compaq Visual Fortran compiler. This has not been tested for the last couple of releases as the NWChem.

  % setenv NWCHEM_TOP <your path>/nwchem
  % setenv NWCHEM_TARGET WIN32
  % setenv NWCHEM_MODULES all

To start the compilation, start the Microsoft makefile utility from the top level source directory by typing "nmake". Reminder: For Compaq visual fortran don't forget to execute the "dfvars" script.

General site installation

The build procedures outlined above will allow use of NWChem within the NWChem directory structure. The code will look for the basis set library file in a default place within that directory structure. To install the code in a general, public place (e.g., /usr/local/NWChem) the following procedure can be applied:

Determine the local storage path for the install files. (e.g., /usr/local/NWChem).
Make directories

  mkdir /usr/local/NWChem
  mkdir /usr/local/NWChem/bin
  mkdir /usr/local/NWChem/data

Copy binary

  cp $NWCHEM_TOP/bin/${NWCHEM_TARGET}/nwchem /usr/local/NWChem/bin
  cd /usr/local/NWChem/bin 
  chmod 755 nwchem

Set links to data files (basis sets, force fields, etc.)

  cd $NWCHEM_TOP/src/basis
  cp -r libraries /usr/local/NWChem/data

  cd $NWCHEM_TOP/src/
  cp -r data /usr/local/NWChem

  cd $NWCHEM_TOP/src/nwpw
  cp -r libraryps /usr/local/NWChem/data

Each user will need a .nwchemrc file to point to these default data files. A global one could be put in /usr/local/NWChem/data and a symbolic link made in each users $HOME directory is probably the best plan for new installs. Users would have to issue the following command prior to using NWChem: ln -s /usr/local/NWChem/data/default.nwchemrc $HOME/.nwchemrc

Contents of the default.nwchemrc file based on the above information should be:

  nwchem_basis_library /usr/local/NWChem/data/libraries/
  nwchem_nwpw_library /usr/local/NWChem/data/libraryps/
  ffield amber
  amber_1 /usr/local/NWChem/data/amber_s/
  amber_2 /usr/local/NWChem/data/amber_q/
  amber_3 /usr/local/NWChem/data/amber_x/
  amber_4 /usr/local/NWChem/data/amber_u/
  spce    /usr/local/NWChem/data/solvents/spce.rst
  charmm_s /usr/local/NWChem/data/charmm_s/
  charmm_x /usr/local/NWChem/data/charmm_x/

Of course users can copy this file instead of making the symbolic link described above and change these defaults at their discretion.

It is can also be useful to use the NWCHEM_BASIS_LIBRARY environment variable when testing a new installation when an old one exists. This will allow you to overwrite the value of nwchem_basis_library in your .nwchemrc file and point to the new basis library. For example:

    % setenv NWCHEM_BASIS_LIBRARY "$NWCHEM/data-5.0/libraries/"

Do not forget the trailing "/".

Search

Navigation

SEARCH

TOOLBOX

LANGUAGES

Forum Menu

Compiling NWChem

From NWChem

Contents

Compiling NWChem from source

Setting up the proper environment variables

Adding optional environmental variables

Building the NWChem binary

How-to: Linux workstation platforms

How to: Mac platforms

Compilation of NWChem 6.1.1 release on Mac OS X Lion 10.7

How-to: Cray platforms

How-to: IBM platforms

How-to: Commodity clusters with Infiniband

How-to: Windows Platforms

General site installation

Toolbox