undefined references compiling in Archer EPW4

[AlexA]

I am wondering if anyone can help with the error I am obtaining when compiling in archer.

I received many undefined references like

Thanks
Alex

dgerb2m_gen.f:(.text+0x15d): undefined reference to `mkl_lapack_dgeqrf_pf'
dgerb2m_gen.f:(.text+0x188): undefined reference to `mkl_lapack_dlacpy'
dgerb2m_gen.f:(.text+0x1b3): undefined reference to `mkl_lapack_dlacpy'
dgerb2m_gen.f:(.text+0x34a): undefined reference to `mkl_blas_dtrmm'
dgerb2m_gen.f:(.text+0x440): undefined reference to `mkl_lapack_dgelqf_pf'
dgerb2m_gen.f:(.text+0x46b): undefined reference to `mkl_lapack_dlacpy'
dgerb2m_gen.f:(.text+0x4a0): undefined reference to `mkl_lapack_dlacpy'
dgerb2m_gen.f:(.text+0x66b): undefined reference to `mkl_blas_dtrmm'
dgerb2m_gen.f:(.text+0x72d): undefined reference to `mkl_lapack_dlacpy'
dgerb2m_gen.f:(.text+0x899): undefined reference to `mkl_blas_dtrmm'
dgerb2m_gen.f:(.text+0x8c4): undefined reference to `mkl_lapack_dlacpy'

The make.sys file I am using is below (I adapted it as per the archer website for QE compilation)

# make.sys. Generated from make.sys.in by configure.

# compilation rules

.SUFFIXES :
.SUFFIXES : .o .c .f .f90

# most fortran compilers can directly preprocess c-like directives: use
# $(MPIF90) $(F90FLAGS) -c $<
# if explicit preprocessing by the C preprocessor is needed, use:
# $(CPP) $(CPPFLAGS) $< -o $*.F90
# $(MPIF90) $(F90FLAGS) -c $*.F90 -o $*.o
# remember the tabulator in the first column !!!

.f90.o:
$(MPIF90) $(F90FLAGS) -c $<

# .f.o and .c.o: do not modify

.f.o:
$(F77) $(FFLAGS) -c $<

.c.o:
$(CC) $(CFLAGS) -c $<



# Top QE directory, not used in QE but useful for linking QE libs with plugins
# The following syntax should always point to TOPDIR:
# $(dir $(abspath $(filter %make.sys,$(MAKEFILE_LIST))))

TOPDIR = /home/e05/e05/alexa/bin/QE/espresso-5.4.0

# DFLAGS = precompilation options (possible arguments to -D and -U)
# used by the C compiler and preprocessor
# FDFLAGS = as DFLAGS, for the f90 compiler
# See include/defs.h.README for a list of options and their meaning
# With the exception of IBM xlf, FDFLAGS = $(DFLAGS)
# For IBM xlf, FDFLAGS is the same as DFLAGS with separating commas

# MANUAL_DFLAGS = additional precompilation option(s), if desired
# BEWARE: it does not work for IBM xlf! Manually edit FDFLAGS
MANUAL_DFLAGS =
DFLAGS = -D__INTEL -D__DFTI -D__MPI -D__PARA -D__SCALAPACK
FDFLAGS = $(DFLAGS) $(MANUAL_DFLAGS)

# IFLAGS = how to locate directories with *.h or *.f90 file to be included
# typically -I../include -I/some/other/directory/
# the latter contains .e.g. files needed by FFT libraries

IFLAGS = -I../include -I/opt/intel/composer_xe_2015.2.164/mkl/include

# MOD_FLAGS = flag used by f90 compiler to locate modules
# Each Makefile defines the list of needed modules in MODFLAGS

MOD_FLAG = -I

# Compilers: fortran-90, fortran-77, C
# If a parallel compilation is desired, MPIF90 should be a fortran-90
# compiler that produces executables for parallel execution using MPI
# (such as for instance mpif90, mpf90, mpxlf90,...);
# otherwise, an ordinary fortran-90 compiler (f90, g95, xlf90, ifort,...)
# If you have a parallel machine but no suitable candidate for MPIF90,
# try to specify the directory containing "mpif.h" in IFLAGS
# and to specify the location of MPI libraries in MPI_LIBS

MPIF90 = ftn
#F90 = ifort
CC = cc
F77 = ftn

# C preprocessor and preprocessing flags - for explicit preprocessing,
# if needed (see the compilation rules above)
# preprocessing flags must include DFLAGS and IFLAGS

CPP = cpp
CPPFLAGS = -P -C -traditional $(DFLAGS) $(IFLAGS)

# compiler flags: C, F90, F77
# C flags must include DFLAGS and IFLAGS
# F90 flags must include MODFLAGS, IFLAGS, and FDFLAGS with appropriate syntax

CFLAGS = -O3 $(DFLAGS) $(IFLAGS)
F90FLAGS = $(FFLAGS) -nomodule -fpp $(FDFLAGS) $(IFLAGS) $(MODFLAGS)
FFLAGS = -O2 -assume byterecl -g -traceback

# compiler flags without optimization for fortran-77
# the latter is NEEDED to properly compile dlamch.f, used by lapack

FFLAGS_NOOPT = -O0 -assume byterecl -g -traceback

# compiler flag needed by some compilers when the main program is not fortran
# Currently used for Yambo

FFLAGS_NOMAIN = -nofor_main

# Linker, linker-specific flags (if any)
# Typically LD coincides with F90 or MPIF90, LD_LIBS is empty

LD = ftn
LDFLAGS = -static -I/opt/intel/composer_xe_2015.2.164/mkl/include/ -I/opt/intel/composer_xe_2015.2.164/mkl/include/intel64/lp64/
LD_LIBS =

# External Libraries (if any) : blas, lapack, fft, MPI

# If you have nothing better, use the local copy :
# BLAS_LIBS = /your/path/to/espresso/BLAS/blas.a
# BLAS_LIBS_SWITCH = internal

BLAS_LIBS = /opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/libmkl_sequential.a /opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group
BLAS_LIBS_SWITCH = external

# If you have nothing better, use the local copy :
# LAPACK_LIBS = /your/path/to/espresso/lapack-3.2/lapack.a
# LAPACK_LIBS_SWITCH = internal
# For IBM machines with essl (-D__ESSL): load essl BEFORE lapack !
# remember that LAPACK_LIBS precedes BLAS_LIBS in loading order

LAPACK_LIBS = /opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/libmkl_core.a
LAPACK_LIBS_SWITCH = external

ELPA_LIBS_SWITCH = disabled
SCALAPACK_LIBS = /opt/intel/composer_xe_2015.2.164/mkl/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group

# nothing needed here if the the internal copy of FFTW is compiled
# (needs -D__FFTW in DFLAGS)

FFT_LIBS =

# For parallel execution, the correct path to MPI libraries must
# be specified in MPI_LIBS (except for IBM if you use mpxlf)

MPI_LIBS =

# IBM-specific: MASS libraries, if available and if -D__MASS is defined in FDFLAGS

MASS_LIBS =

# ar command and flags - for most architectures: AR = ar, ARFLAGS = ruv

AR = ar
ARFLAGS = ruv

# ranlib command. If ranlib is not needed (it isn't in most cases) use
# RANLIB = echo

RANLIB = ranlib

# all internal and external libraries - do not modify

FLIB_TARGETS = all

LIBOBJS = ../clib/clib.a ../iotk/src/libiotk.a
LIBS = $(SCALAPACK_LIBS) $(LAPACK_LIBS) $(FFT_LIBS) $(BLAS_LIBS) $(MPI_LIBS) $(MASS_LIBS) $(LD_LIBS)

# wget or curl - useful to download from network
WGET = wget -O

# Install directory - not currently used
PREFIX = /usr/local

[sponce]

Dear Alex,

Is the error specific to EPW ?

What I mean is: can you correctly compile and run Quantum Espresso 5.4.0 on Archer and you only have a problem at the level of EPW?

I was able to compile and run QE and EPW on Archer.

For some reason I cannot access Archer now so I cannot tell you exactly what I did but basically you just have to load the intel module and then set some environment variable like F90 etc. The configure file of QE is then able to correctly generate the make.sys

Actually, you can directly ask them to install you QE 5.4.0 with EPW as a module (some older version of QE already exist). They should be able to optimize it very well.

Best,

Samuel

[AlexA]

Hi Samuel thanks for the reply.

Yes QE 5.4.0 compiles correctly, the only problem is with EPW. I have also managed to obtain the EPW executable so I didn't think anything of it. When I then tried to run the EPW calculation on Archer it did run but couldn't link to the Wannier file. For some reason it was printing a .win file and not a case.win file.

I will try the compilation again. In anycase I also emailed Archer and they will hopefully load a central module which I think will be a great solution.

Regards
Alex

[sponce]

Oh if its a problem of liking with Wannier, it might be because two compilations of EPW are required to correctly link with Wannier.

Please, try the following:

Code: Select all

cd espresso-5.4.0 && ./configure
make pwall
make ph 
cd EPW/src
make 
make

[AlexA]

Unfortunately my skills at compilation are somewhat very poor. I am trying to compile the new version of EPW that came with the q.e v6 release in archer. Following the instructions on the archer website EPW is not successfully compiling although there is no issue in compiling q.e v6. Could anyone be so kind in letting me use there executable or giving me the instructions for the make.inc file?

Thanks
Alex

[sponce]

Hello Alex,

On archer, do the following:
1) Inside QE6 do "make distclean"
2) Copy paste the following in your terminal:

Code: Select all

module unload PrgEnv-cray
module load   PrgEnv-intel
export CC=cc
export FC=ftn
export F77=ftn
export F90=ftn

MKLHOME=$MKLROOT
export LDFLAGS="-static  -I$MKLHOME/include/ -I$MKLHOME/include/intel64/lp64/"
export BLAS_LIBS="-L$MKLHOME/lib/intel64/libmkl_sequential.a -L$MKLHOME/lib/intel64/libmkl_blacs_intelmpi_lp64.a -lpthread "
export LAPACK_LIBS="-L$MKLHOME/lib/intel64/libmkl_intel_lp64.a -L$MKLHOME/lib/intel64/libmkl_core.a"
export SCALAPACK_LIBS="-L$MKLHOME/lib/intel64/libmkl_scalapack_lp64.a "
export FFT_LIBS="-L$MKLHOME/lib/intel64/libmkl_cdft_core.a"
./configure

3) Once the configure is finished, do "make pwall"
4) make ph
5) cd EPW/src
6) make
7) make

Then it should work.

Do not forget to export the same as 2) in your submission scripts.

Cheers,

Samuel

[AlexA]

Thanks Samuel,

That is so much help, it is working well now.

Regards
Alex

[sponce]

Hello Alex,

Glad to hear it.

Actually it would be helpful (also for other people) if you could ask the people of Archer to install QE 6+EPW4.1 on archer as a module.
Indeed, they might optimize it more than what I gave you (i.e. the code will run faster). For example, although I'm exporting the fft, I do not think
the configure actually use it and fall back on the internal.

I would personally be interested if they do that so that I can see what they did best.

Cheers,

Samuel