Phonon interpolation problem with EPW

Hello,

I’m new to EPW, and want to compute the e-ph coupling for a solid hydrogen phase. I’m following the steps from the “Electron-phonon coupling with EPW” tutorial (Wed.4 at https://docs.epw-code.org/doc/School2018.html - PDF attached). I’m able to run the code up to page 6 of the PDF, and produce the second epw.x output. It looks like the code runs properly (input and output files attached). The electronic bands are computed correctly (band.eig, band.ps). However, the phonon frequencies have a problem. Many of the interpolated frequencies are correct, but parts of the path are completely off by orders of magnitude (phband.frq). To compare, I plotted the phonon dispersions from epw.x (phbands.freq.png) and matdyn.x (scf.freq.png). As you can see, there is a reasonable match for the points where epw.x was able to give sensible results, but parts of the plot are trash. I also attach the decay plots. I don’t know how much decay I should expect in order to be confident in the calculation.

I’d appreciate any comments. Since I’m very new to epw, I might be missing something obvious so don’t hesitate to give me any suggestions that might be relevant. Thank you!

Attachments: https://www.dropbox.com/sh/skp3wglmlzhdx7w/AADtdupVBVMI0z7Q2HqsBdE8a?dl=0

Best,
Mehmet Dogan

Dear Mehmet Dogan:

First, I would suggest you to use the recent official version of EPW v5.3 (https://gitlab.com/QEF/q-e/-/releases) or the recent development version of EPW (https://gitlab.com/QEF/q-e); I have no experience with EPW v5.1 you are using.

There are several changes in EPW v5.3 (See the Releases page at https://docs.epw-code.org/doc/Releases.html); in your case, you should (1) not use “nosym=.true.” in nscf.in, (2) remove “nbndskip = 0”, and (3) remove the q-point list in epw.in.

If you encounter the same issues when using EPW v5.3 or the development version, please provide scf_band.kpt, scf.in, ph.in, and nscf.in.

Sincerely,

H. Lee

Dear Mehmet Dogan:

You can also try lifc=.true. with the IFC file. Please check the following post at https://forum.epw-code.org/t/epws-phonon-dispersion-is-inconsistent-with-qes-result/608/4 .

Sincerely,

H. Lee

Dear H. Lee,

Thank you very much for your suggestions! I will first try again after setting “lifc=.true.” when the machine I’m running these calculations on comes back online tonight. I’m hesitant to recompile the new version of QE (6.6) because I’ve already computed the phonons of these systems, which takes several weeks, and there are several systems. In case I need to upgrade to EPW 5.3, do you think it would work with the QE version that I’m using (6.4.1)? I would simply replace the EPW folder in the QE-6.4.1 package with the one from QE-6.6, and recompile. I can then run the nscf and epw as you described. Do you think that might work?

Best,
Mehmet

Dear Mehmet Dogan:

I would simply replace the EPW folder in the QE-6.4.1 package with the one from QE-6.6, and recompile.

I think that you would encounter the error in compiling since several changes of QE v6.6 were reflected in the EPW v5.3.

Instead, you would try to just use the results of phonon calculations with QE v6.4.1 stored in the save folder which EPW imports and rerun scf, nscf, and EPW using QE v6.6 and EPW v5.3.
However, (I am sorry) I can’t be 100% sure whether or not it has no issue.

Sincerely,

H. Lee

Dear H. Lee,

Thank you for your helpful explanations. I will try these suggestions as soon as I can, and will let you know what happens.

Best,
Mehmet

Dear H. Lee,

I tried the quick fix first, using your instructions on [https://forum.epw-code.org/t/epws-phonon-dispersion-is-inconsistent-with-qes-result/608/4], but I had the exact same results for the phonon frequencies. So I compiled QE-6.6 as you suggested, and ran the scf and nscf calculations. When I run the first epw calculation, however, I get the “BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES” error. I tried to fix the epw input based on the changes to the 5.3 version, but I may have done it incorrectly. Could you take a look at the new “epw1.in” and “epw1.out” files at [https://www.dropbox.com/sh/skp3wglmlzhdx7w/AADtdupVBVMI0z7Q2HqsBdE8a?dl=0] and let me know what the issue might be? Thank you so much!

Best,
Mehmet

Dear Mehmet:

First, I found one suspicious thing in your old output of epw2.out as you can see below:

Using q-mesh file: scf_band.kpt
WARNING: q-point weigths do not add up to 1 [loadqmesh_serial]
Size of q point mesh for interpolation: 297
Using k-mesh file: scf_band.kpt
WARNING: k-point weigths do not add up to 1 [loadkmesh_para]
Size of k point mesh for interpolation: 594
Max number of k points per pool: 20

Fermi energy coarse grid = 15.506147 eV

===================================================================

Fermi energy corresponds to the coarse k-mesh

===================================================================

ibndmin = 1 ebndmin = 1.075
ibndmax = 3 ebndmax = 1.212


Number of ep-matrix elements per pool : 1080 ~= 8.44 Kb (@ 8 bytes/ DP)

A selecq.fmt file was found but re-created because selecqread == .false.
Number selected, total 100 100
Number selected, total 200 269
We only need to compute 224 q-points

Initially, you have 297 q points in the k(q)-point list of scf_band.kpt.
However, “fsthick = 1” in your epw2.in removes some of them, finally leading to 224 q points.
So I would suggest you the following things (I hope that you didn’t remove the old folder from EPW v5.1):
(1) Check the default value of fsthick in EPW/src/epw_readin.f90 in EPW v5.1 .
(2) if the default value of fsthick is very large, for instance, 1.d10, please comment or remove the line of “fsthick = 1” in your epw2.in
(3) Rerun epw2.in with EPW v5.1

Second, regarding the error from EPW v5.3:
It seems that you encounter the error when epmatq is written. In your case, this file is very large, but I guess that you have no problem with EPW v5.1.
So I would suggest you the followings:
(1) Please check your job log files (generated when your job was stopped) to see whether or not there are some further hints for your error.
(2) Please run test-suite in q-e/test-suite by issuing “make run-tests-epw-parallel” in order to confirm that there are no issues with your build.

Sincerely,

H. Lee

Dear H. Lee,

Thank you very much for your thoughtful suggestions!

  1. EPW 5.1 calculation seems to have worked after getting rid of “fsthick = 1”. I added “phband.freqs.2.png” to the folder [https://www.dropbox.com/sh/skp3wglmlzhdx7w/AADtdupVBVMI0z7Q2HqsBdE8a?lst=]. I also added the input and output files (“epw1-5.1.in”, “epw1-5.1.out”, “epw2-5.1.in”, “epw2-5.1.out”). Thank you so much!

  2. I ran the test that you suggested on QE-6.6 and it seems to have passed (see “qe-test-results” in the folder). The error message in the log file (“epw1-5.3.err”) basically reads “output statement overflows record” for the .epb files. I don’t know what this means but probably it has something to do with the size of these files. Let me know if you have any suggestions about how I might fix this issue. Thank you!

Best,
Mehmet

Dear Mehmet:

  1. Regarding the overflow error:
    You passed all parallel EPW tests; since they include writing prefix.epb* files, this fact means that (in general) you have no issue with writing prefix.epb* files.

I don’t know what this means but probably it has something to do with the size of these files

I guess that you had no issue when using the same inputs with EPW v5.1 on the same cluster and using the same compiler to build EPW .

Frankly, I don’t see anything suspicious in your inputs and outputs regarding this error.

Writing epb files is done in the part below of EPW/src/elphon_shuffle_wrap.f90.

      IF (epbwrite) THEN
        OPEN(iuepb, FILE = tempfile, FORM = 'unformatted')
        WRITE(stdout, '(/5x, "Writing epmatq on .epb files"/)')
        WRITE(iuepb) nqc, xqc, et_loc, dynq, epmatq, zstar, epsi
        CLOSE(iuepb)
        WRITE(stdout, '(/5x, "The .epb files have been correctly written"/)')
      ENDIF

As you can see, you encounter the overflow error when writing one (or more) arrays among nqc, xqc, et_loc, dynq, epmatq, zstar, epsi .
I think that in order to resolve this issue, we need to first know when the overflow error arises.

  1. Regarding the phonon dispersion:
    Compared with scf.freqs.png or scf.freqs.1.png, your plots still show some (small) degeneracy breaking.
    If you don’t like this (though there might be negligible difference in your final results), you can try the followings:

You already tried lifc=.true.; if even with lifc=.true. you have still degeneracy breaking, you can try " use_ws = .true. " (I guess use_ws is implemented in EPW v5.1)

Sincerely,

H. Lee

Dear Mehmet:

In addition to my previous message posted today:

You can also skip writing epb files by adding both “epbwrite=.false.” and “epbread=.false.”.
In most cases, epb files are not necessary since usually we restart with the electron-phonon vertex on coarse grids in Wannier representation.

However, it is still good to eventually solve your overflow error.

Sincerely,

H. Lee

Dear H. Lee,

Thank you once again for your suggestions. I recompiled QE-6.6 after making sure the make.inc file for the compilation was as similar to that of QE-6.4.1 as possible. I added both to the folder if you’d like to take a look. I then repeated the test, which was successfully completed. However, when I run scf, nscf and then epw1, I get the same exact error. I did a bit more investigation on the issue. There are 32 epb files that should be created (1 for each processor), each of which is 3.4 GB in size. Actually, they are all exactly the same size (3626047868 bytes). These files can be created without a problem with EPW-5.1. With EPW-5.3, however, each time, 20 of them seem to fail, and not the same 20. I tried this calculation 3 times now, and I can see that each time a different set of 20 failed to write. However, the remaining 12 seem to write to disk properly, with the correct size. So it’s not that the operation reliably fails for each file of this size, but somehow a certain number of them fail and a certain number of them don’t. Do you have any ideas about what might be going on?

Thank you for your suggestion about skipping writing the epb files altogether. I’ll try that as well and see if I can complete all the calculations I need that way.

Best,
Mehmet

Dear Mehmet:

Could you make the following parts of your make.inc in QE 6.6 the same as those in QE 6.4.1?
I just would like to know whether or not this issue originates from the building environments.

53c56
< IFLAGS         = -I$(TOPDIR)/include -I$(TOPDIR)/FoX/finclude -I$(TOPDIR)/S3DE/iotk/include/ -I/opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/include
---
> IFLAGS         = -I$(TOPDIR)/include -I$(TOPDIR)/FoX/finclude -I$(TOPDIR)/S3DE/iotk/include/ -I/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/include
79c83
< MPIF90         = mpif90
---
> MPIF90         = mpiifort
98c102
< CPPFLAGS       = -P -traditional $(DFLAGS) $(IFLAGS)
---
> CPPFLAGS       = -P -traditional -Uvector $(DFLAGS) $(IFLAGS)
122c127
< LD             = mpif90
---
> LD             = mpiifort

Sincerely,

H. Lee

Dear H. Lee,

Actually I had added the wrong make.inc corresponding to QE-6.6. That was the first one I compiled, but then I compiled a second version which should be built in essentially the saw way as my QE-6.4.1. Now I uploaded the correct make.inc for QE-6.6. As I said, with both versions of QE-6.6, I get the same error with epw. Sorry for the inconvenience!

Best,
Mehmet

Dear H. Lee,

In the meantime, in a separate folder, I’m going through the steps without writing the epb files to the disk. In the second epw calculation, I get the error:

     Using q-mesh file: scf_band.kpt

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     Error in routine loadqmesh_serial (1):
     ERROR: Specify either crystal or cartesian coordinates in the filqf file
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

     stopping ...

It looks like I need to define the coordinate type for this file, starting at EPW 5.2. However, I couldn’t find where to make that definition in my input file. I added the “filkf” file to the folder for your reference. Thank you!

Best,
Mehmet

Dear Mehmet:

You can find the format of the q-point list file by looking at the source code as below:

From grid.f90 in EPW/src directory:

    IF (mpime == ionode_id) THEN
      IF (filqf /= '') THEN ! load from file
        !
        ! Each pool gets its own copy from the action=read statement
        !
        WRITE(stdout, *) '    Using q-mesh file: ', TRIM(filqf)
        IF (lscreen) WRITE(stdout, *) '     WARNING: if lscreen=.TRUE., q-mesh needs to be [-0.5:0.5] (crystal)'
        OPEN(UNIT = iunqf, FILE = filqf, STATUS = 'old', FORM = 'formatted', IOSTAT = ios)
        IF (ios /= 0) CALL errore('loadqmesh_serial', 'opening file ' // filqf, ABS(ios))
        READ(iunqf, *) nqtotf, coordinate_type
        IF (TRIM(coordinate_type) .EQ. ' ') coordinate_type = 'crystal'
        IF (.NOT. imatches("crystal", coordinate_type) .AND. .NOT. imatches("cartesian", coordinate_type)) &
          CALL errore('loadqmesh_serial', 'ERROR: Specify either crystal or cartesian coordinates in the filqf file', 1)
        !
        ALLOCATE(xqf(3, nqtotf), STAT = ierr)
        IF (ierr /= 0) CALL errore('loadqmesh_serial', 'Error allocating xqf', 1)
        ALLOCATE(wqf(nqtotf), STAT = ierr)
        IF (ierr /= 0) CALL errore('loadqmesh_serial', 'Error allocating wqf', 1)
        !
        DO iq = 1, nqtotf
          !
          READ (iunqf, *) xqf(:, iq), wqf(iq)
          !
        ENDDO
        CLOSE(iunqf)
        IF (imatches("cartesian", coordinate_type)) THEN
          CALL cryst_to_cart(nqtotf, xqf, at, -1)
        ENDIF
...

It seems that the part below might lead to the problem in some cases.

       READ(iunqf, *) nqtotf, coordinate_type
        IF (TRIM(coordinate_type) .EQ. ' ') coordinate_type = 'crystal'
        IF (.NOT. imatches("crystal", coordinate_type) .AND. .NOT. imatches("cartesian", coordinate_type)) &
          CALL errore('loadqmesh_serial', 'ERROR: Specify either crystal or cartesian coordinates in the filqf file', 1)

To be safe, please specify explicitly the type of coordinates as below:
297 crystal
or
297 cartesian

Regarding your overflow error:
To be honest, I still have no idea. It is not easy to identify the origin of error unless I reproduce your calculations from scratch.

Sincerely,

H. Lee

Dear H. Lee,

Thank you! Adding “crystal” next to the number of k-points fixed the error.

For the overflow error, I’m retrying the calculation with 64 processors instead of 32 to see if that makes a difference. I will post an update when I get the results.

Best,
Mehmet

Dear H. Lee,

When I increased the number of processors from 32 to 64, the epb files ended up being written correctly. So somehow the file sizes were too big for EPW 5.3 even though EPW 5.1 had no problem writing them to the disk. Thanks for all your help!

Best,
Mehmet