Segmentation fault with Scalapack from MKL

From NWChem

Viewed 1324 times, With a total of 5 Posts
Jump to: navigation, search

Clicked A Few Times
Threads 2
Posts 6
I compiled NWChem version 6.5 on a local cluster with openmpi 1.6.2, intel compilers version 13.1.1 and MKL

My configuration file is:

export NWCHEM_TOP=/home/attaccalite/SOFTWARE/Nwchem-6.5 
export NWCHEM_TARGET=LINUX64
export ARMCI_NETWORK=MPI-TS
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/opt/openmpi/
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include
export LIBMPI='-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil'
export NWCHEM_MODULES="all"
export USE_NOFSCHECK=TRUE
export LARGE_FILES=TRUE
export MKLROOT=/opt/intel/composer_xe_2013.3.163/mkl/lib/intel64
export BLASOPT="-L$MKLROOT -lmkl_core -lmkl_sequential -lmkl_intel_ilp64"
export CC=icc
export FC=ifort
export MPICC=/opt/openmpi/bin/mpicc
export BLAS_SIZE=8
export LAPACK_SIZE=8
export FOPTIMIZE="-O3 -axSS2,SSE3,SSSE3 -i8"
export COPTIMIZE="-O3 -axSS2,SSE3,SSSE3"
#export SCALAPACK_SIZE=8
#export USE_SCALAPACK=y
#export SCALAPACK="-L$MKLROOT -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"
#export SCALAPACK_LIB="$SCALAPACK

when I compile nwchem works fine also in parallel, but if I try to use Scalpack from MKL I get "Segementation fault"

..............
 ao basis        = "ao basis"
functions = 25
atoms = 3
closed shells = 5
open shells = 0
charge = 0.00
wavefunction = RHF
input vectors = atomic
output vectors = ./h2o.movecs
use symmetry = T
symmetry adapt = T
Forming initial guess at 0.1s
Segmentation fault


any hints? any ideas ?

regards
Claudio
Edited On 9:10:00 AM PDT - Tue, Apr 14th 2015 by Attacc

Forum Vet
Threads 10
Posts 1606
The following settings are a recipe for disaster

export FOPTIMIZE="-O3 -axSS2,SSE3,SSSE3 -i8"

export COPTIMIZE="-O3 -axSS2,SSE3,SSSE3"

My suggestion is to recompile everything from scratch without redefining FOPTIMIZE and COPTIMIZE

Clicked A Few Times
Threads 2
Posts 6
Same problem
I tried to remove FOPTIMIZE and COPTIMIZE but it gives the same error.

Anyway I'm obligated to use FOPTIMIZE and COPTIMIZE for production runs, because the nodes of the cluster are different from the computer were the code is compiled. I will try to recompile SCALPACK from sources and see it works.

Claudio
Edited On 2:13:35 AM PDT - Wed, Apr 15th 2015 by Attacc

Clicked A Few Times
Threads 2
Posts 6
Test with Scalpack from Netlib failed
Ok I tried to compile my own scalapack and then link them to nwchem

export NWCHEM_TOP=/home/attaccalite/SOFTWARE/Nwchem-6.5
export NWCHEM_TARGET=LINUX64
export ARMCI_NETWORK=MPI-TS
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/opt/openmpi/
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include
export LIBMPI='-lmpi_f90 -lmpi_f77 -lmpi -ldl -Wl,--export-dynamic -lnsl -lutil'
export NWCHEM_MODULES="all"
export USE_NOFSCHECK=TRUE
export LARGE_FILES=TRUE
export MKLROOT=/opt/intel/composer_xe_2013.3.163/mkl/lib/intel64
export BLASOPT="-L$MKLROOT -lmkl_core -lmkl_sequential -lmkl_intel_lp64"
export USE_64TO32=y
export CC=icc
export FC=ifort
export MPICC=/opt/openmpi/bin/mpicc
export BLAS_SIZE=4
export LAPACK_SIZE=4
export SCALAPACK_SIZE=4
export USE_SCALAPACK=y
export SCALAPACK="/home/attaccalite/SOFTWARE/scalapack-2.0.2/libscalapack.a"
export SCALAPACK_LIB="$SCALAPACK"

but I get a new error

Starting SCF solution at       0.7s
------------------------------------------------------------------------
ga_iter_lsolve: dgesv failed -4294967296
------------------------------------------------------------------------
------------------------------------------------------------------------
current input line :
26: task scf optimize
------------------------------------------------------------------------

ok, I can live without scalapack :-)
but if you have suggestion I can try differently to compile scalapack+nwchem

notice that I compiled them with 32bit integer because it is the unix standard and other code use them with this integer dimension

cla

Forum Vet
Threads 10
Posts 1606
Did you execute the following command before compiling?

make 64_to_32

More details at
http://www.nwchem-sw.org/index.php/Compiling_NWChem#Optimized_math_libraries

Clicked A Few Times
Threads 2
Posts 6
It works
Thank you

I completely forgot that point, now it works with Scalapack

sincerely
Claudio


Forum >> NWChem's corner >> Compiling NWChem



Who's here now Members 0 Guests 1 Bots/Crawler 0


AWC's: 2.5.10 MediaWiki - Stand Alone Forum Extension
Forum theme style by: AWC