compile nwchem-6-3 with openmpi1.6.5 & ifort

From NWChem

Viewed 3134 times, With a total of 9 Posts
Jump to: navigation, search
Forum >> NWChem's corner >> Compiling NWChem

Clicked A Few Times
Threads 2
Posts 6
8:38:44 AM PDT - Mon, Jun 2nd 2014
Dear all,

I have been fighting like an animal to compile nwchem on my workstation (intel xeon, centos 6.5)
I have installed intel fortran compiler, intel cc, mkl.
I have compiled openmpi 1.6.5 with CC=icc CXX=icpc F77=ifort FC=ifort

But, when I try to compile nwchem - I see the following error:

checking for mpi.h... yes
checking for library containing MPI_Init... no
configure: error: test for COMEX_NETWORK=MPI_TS failed
configure: error: ../../ga-5-2/comex/configure failed for comex
make[1]: *** [build/config.status] Error 1
make: *** [libraries] Error 1

My script is following:

  1. !/bin/tcsh

cd nwchem-6.3
setenv NWCHEM_TOP `pwd`
setenv NWCHEM_TARGET LINUX64

setenv USE_MPI y
setenv USE_MPIF y
  1. openmpi version 1.6.5
setenv MPI_LOC /home/minenky/programs/openmpi_intel

setenv MPI_LIB $MPI_LOC/lib
setenv MPI_INCLUDE $MPI_LOC/include

  1. setenv LIBMPI "-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil"
  2. "-pthread -L/opt/hwloc/lib -L/opt/ompi-1.6.3/lib -lmpi_f90 -lmpi_f77 -lmpi -ldl -lhwloc -Wl,--export-dynamic -lrt -lnsl -lutil -lm -ldl -Wl,-rpath -Wl,/opt/hwloc/lib -Wl,-rpath -Wl,/opt/hwloc/lib"
setenv LIBMPI "-lmpi -lmpigf -lmpigi -lrt -lpthread"
setenv LARGE_FILES TRUE
  1. setenv LIB_DEFINES -DDFLT_TOT_MEM=16777216
setenv TCGRSH /usr/bin/ssh

  1. blas
setenv MKLROOT /opt/intel/mkl
setenv BLASOPT "-L$MKLROOT/lib/intel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -lm"
setenv BLAS_LIB "$BLASOPT"
setenv BLAS_SIZE 8

setenv NWCHEM_MODULES "all"
cd $NWCHEM_TOP/src

make nwchem_config
make FC=ifort CC=icc >& make.log

Any kind of help is appreciated!

With kind regards,
Yury

Forum Vet
Threads 7
Posts 1165
3:22:53 PM PDT - Mon, Jun 2nd 2014
Yury
Could you post the output of the following command

/home/minenky/programs/openmpi_intel/bin/mpif90 -show

Clicked A Few Times
Threads 2
Posts 6
10:17:06 PM PDT - Mon, Jun 2nd 2014
Edoapra,
thank you for reply. The output:

ifort -I/home/minenky/programs/openmpi_intel/include -I/home/minenky/programs/openmpi_intel/lib -Wl,-rpath -Wl,/home/minenky/programs/openmpi_intel/lib -Wl,--enable-new-dtags -L/home/minenky/programs/openmpi_intel/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

Quote:Edoapra Jun 2nd 3:22 pm
Yury
Could you post the output of the following command

/home/minenky/programs/openmpi_intel/bin/mpif90 -show

Forum Vet
Threads 7
Posts 1165
1:20:37 PM PDT - Tue, Jun 3rd 2014
Yury
Your MPI settings are not compatible with your MPI installation.
You should use the following (MPI_INCLUDE and MPI_LIB are the same ones use set, the only difference should be in
LIBMPI)

MPI_INCLUDE=/home/minenky/programs/openmpi_intel/include

MPI_LIB=/home/minenky/programs/openmpi_intel/lib

LIBMPI=" -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi"

If you still get a failure, please give us access to the following files

$NWCHEM_TOP/src/tools/build/config.log

$NWCHEM_TOP/src/tools/build/armci/config.log

Clicked A Few Times
Threads 2
Posts 6
1:41:16 PM PDT - Thu, Jun 5th 2014
Thanks for helping! Just got another error - gn -w -g -vec-report1 -i8 -align -w -g -vec-report1 -O3 -prefetch -unroll -tpp7 -ip -xP -no-prec-div -xHost -no-prec-div -I. -I/home/minenky/programs/nwchem/nwchem-6.3/src/include -I/home/minenky/programs/nwchem/nwchem-6.3/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DIFCV8 -DIFCLINUX -DPARALLEL_DIAG -DMPI Parallel.F
ifort -c -i8 -i8 -align -w -g -vec-report1 -i8 -align -w -g -vec-report1 -i8 -align -w -g -vec-report1 -i8 -align -w -g -vec-report1 -i8 -align -w -g -vec-report1 -O3 -prefetch -unroll -tpp7 -ip -xP -no-prec-div -xHost -no-prec-div -I. -I/home/minenky/programs/nwchem/nwchem-6.3/src/include -I/home/minenky/programs/nwchem/nwchem-6.3/src/tools/install/include -DEXT_INT -DLINUX -DLINUX64 -DIFCV8 -DIFCLINUX -DPARALLEL_DIAG -DMPI Parallel-mpi.F
ifort: command line remark #10382: option '-xHOST' setting '-xSSE4.2'
ifort: command line remark #10382: option '-xHOST' setting '-xSSE4.2'
Parallel-mpi.F(29): #error: can't find include file: mpif.h
Parallel-mpi.F(158): #error: can't find include file: mpif.h
Parallel-mpi.F(264): #error: can't find include file: mpif.h
Parallel-mpi.F(314): #error: can't find include file: mpif.h
Parallel-mpi.F(456): #error: can't find include file: mpif.h
Parallel-mpi.F(500): #error: can't find include file: mpif.h
Parallel-mpi.F(538): #error: can't find include file: mpif.h
Parallel-mpi.F(581): #error: can't find include file: mpif.h
Parallel-mpi.F(624): #error: can't find include file: mpif.h
Parallel-mpi.F(672): #error: can't find include file: mpif.h
Parallel-mpi.F(729): #error: can't find include file: mpif.h
Parallel-mpi.F(795): #error: can't find include file: mpif.h
Parallel-mpi.F(839): #error: can't find include file: mpif.h
Parallel-mpi.F(879): #error: can't find include file: mpif.h
Parallel-mpi.F(927): #error: can't find include file: mpif.h
Parallel-mpi.F(1004): #error: can't find include file: mpif.h
Parallel-mpi.F(1093): #error: can't find include file: mpif.h
Parallel-mpi.F(1126): #error: can't find include file: mpif.h
Parallel-mpi.F(1175): #error: can't find include file: mpif.h
Parallel-mpi.F(1223): #error: can't find include file: mpif.h
Parallel-mpi.F(1274): #error: can't find include file: mpif.h
Parallel-mpi.F(1339): #error: can't find include file: mpif.h
Parallel-mpi.F(1412): #error: can't find include file: mpif.h
Parallel-mpi.F(1459): #error: can't find include file: mpif.h
Parallel-mpi.F(1505): #error: can't find include file: mpif.h
Parallel-mpi.F(1558): #error: can't find include file: mpif.h
Parallel-mpi.F(1625): #error: can't find include file: mpif.h
Parallel-mpi.F(1754): #error: can't find include file: mpif.h
make[4]: *** [/home/minenky/programs/nwchem/nwchem-6.3/lib/LINUX64/libnwpwlib.a(Parallel-mpi.o)] Error 28
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [optimized] Error 2
make[2]: *** [subdirs] Error 1
make[1]: *** [subdirs] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [libraries] Error 1


The problem is that ifort cannot compile the above mentioned file: arallel-mpi.F
If I specify FC=mpifort in the compiling script, it does the job - but the resulting version of nwchem can not run in parallel normally.

  • Huub Forum:Admin, Forum:Mod, NWChemDeveloper, bureaucrat, sysop
    Profile
    Send PM
Forum Regular
Threads 1
Posts 185
3:29:09 PM PDT - Thu, Jun 5th 2014
Hi Yuri,

The problem you are seeing is caused by the fact that the location of the MPI include files is not specified on the compile line for Parallel-mpi.F. I.e. there should have been a flag "-I/home/minenky/programs/openmpi_intel/include" but it is not there. This flag should have been added to LIB_DEFINES in the makefile src/nwpw/nwpwlib/Parallel/GNUmakefile. I wonder why that didn't happen. Anyway as a work around you could add it manually (even though that still feels unsatisfactory).

Huub

Clicked A Few Times
Threads 2
Posts 6
2:00:15 AM PDT - Fri, Jun 6th 2014
Hello Huub,
Thanks for helping! I have added the flags the script to include the info about openmpi_intel/include:

and got it compiled.. BUT -
I can not run the code. When I try to run it mpirun nwchem Carbene.nw it says:
 Unable to open nwchem.nw --- appending .nw

------------------------------------------------------------------------

nwchem: failed to open the input file        0

------------------------------------------------------------------------

There is an error in the input file

------------------------------------------------------------------------

For more information see the NWChem manual at 

http://www.nwchem-sw.org/index.php/NWChem_Documentation




For further details see manual section:


But when I create an emty file nwchem.nw - it runs, I see it via top i, but nothing happens - no output from the code. Did anyone have such experience?

Clicked A Few Times
Threads 2
Posts 6
3:03:28 AM PDT - Fri, Jun 6th 2014
OK, finally found the mistake and got it compiled.

I think I am getting closer to the final normally working version. The main problem now is following. When running it with openmpi (mpirun) it is almost 8 times slower on 8 cores comparing to running it on just one core which I find very strange. Any suggestions?

Forum Vet
Threads 7
Posts 1165
10:41:33 AM PDT - Fri, Jun 6th 2014
What input file are you trying to run?
It might be too small of a task to see it scaling

Gets Around
Threads 18
Posts 133
5:20:26 PM PDT - Fri, Jun 6th 2014
Yury, I am using GNU compilers and the default openmpi system package that comes with Ubuntu 14.04 rather than Intel tools, but I too had poor performance with MPI-TS. I normally build with ARMCI_NETWORK=SOCKETS but I wanted to try MPI-TS in an attempt to fix problems similar to those reported here: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id1298/armci_malloc%3Amalloc_1_fail...

I am using the most recent code snapshot in downloads: http://www.nwchem-sw.org/download.php?f=Nwchem-6.3.revision25564-src.2014-05-03.tar.gz

Running serially, the QA test case tce_polar_ccsd_small took 314 seconds wall clock time.

With ARMCI_NETWORK=SOCKETS the same case ran in 139 seconds using 4 cores on a 4 core i7 system. That's OK scaling, not great, but my scaling is limited on most non-direct jobs because of slow disk I/O. I also tried an "mpi serial" job with the same build using mpirun -np 1 and that took 317 seconds wall clock time, basically unchanged from simple serial.

With ARMCI_NETWORK=MPI-TS on the same hardware and OS the wall clock time went to 518 seconds on 4 cores for tce_polar_ccsd_small. That's slower than serial, as you experienced.


Forum >> NWChem's corner >> Compiling NWChem



Who's here now Members 0 Guests 1 Bots/Crawler 0

AWC's: 2.5.10 MediaWiki - Stand Alone Forum Extension
Forum theme style by: AWC
Retrieved from "http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id1303/compile_nwchem-6-3_with_openmpi1....html"