SCDM-k bug with different pseudopotentials?

[adenchfi]

Hello!

I was very interested in getting SCDM-k working while using EPW. However, there seems to be an issue that pops up with using different kinds of pseudopotentials that I wasn't able to find documented anywhere.

If I run the test-suite epw_scdm case (on Silicon) without changing anything, it all works, epw1-epw3. Good so far.

However, performing a minimal change of the pseudopotential (for the SCF and NSCF calculations)
from the 'q-e/pseudo/Si_r.upf' to either Si.pbe-nl-kjpaw_psl.1.0.0.UPF or Si.pbe-n-rrkjus_psl.1.0.0.UPF pseudopotentials

seems to produce a crash when running epw2.in and epw3.in (both of the SCDM examples) with the following output:
from allocate_bec_type : error # 5014
cannot allocate bec%k

Now, the test-suite example only has a 3x3x3 k-point mesh with 12 bands, so I don't expect that bec%k is too large to be allocated. To be clear, when I used a different pseudopotential, I made sure to redo the scf, nscf, and phonon calculations with the new pseudopotential.

So does the SCDM-k method work only with certain pseudopotentials? The above other pseudopotentials I tried are from the PSLibrary, both a PAW and USPP failed. If so, what pseudopotentials are usable with EPW? I ask because I'm working with lanthanide compounds, and so have a limited set of pseudopotentials that are both accurate and converge well.

[hlee]

Dear adenchfi:

Personally, I would suggest you to use norm-conserving pseudopotentials (PPs) for the SCDM method even though officially the paper (https://arxiv.org/abs/1909.00433) mentions no restriction in choice of PPs (See Appendix A in this paper).

Well, could you confirm that you are using the most-recent version of EPW downloaded at https://gitlab.com/QEF/q-e so that I can reproduce your problem and check the code?

Meanwhile, I would suggest you to try to get Wannier functions using Wannier90 (W90) >=v3.0. Now you need to check whether the SCDM method works in your case by changing a few parameters and rerunning the Wannierization, and for this purpose, it is more convenient to start with W90.
If it works, you can just use the same relevant inputs in the input of EPW.

PS) As I already mentioned it my previous post, you had better get Wannier functions using the SCDM method for the smallest "isolated" band manifold including your reduced one. Otherwise, you might encounter problems with the Fermi energy since it is possible that nbndskip is not constant.

Sincerely,

H. Lee

[hlee]

Dear adenchfi:

I reproduced your problem and found a minor bug in the subroutine of compute_mmn_para() in EPW/src/pw2wan2epw.f90 ; this error is due to the double allocation of bec%k.

The temporary fix is as follows:

(1) Find the part below in the subroutine of compute_mmn_para()

Code: Select all

    !                                                                                                                                                                                                       
    !   USPP                                                                                                                                                                                                
    !                                                                                                                                                                                                       
    IF (any_uspp) THEN
      CALL allocate_bec_type(nkb, nbnd, becp)
      IF (noncolin) THEN
        ALLOCATE(becp2_nc(nkb, 2, nbnd), STAT = ierr)
        IF (ierr /= 0) CALL errore('compute_mmn_para', 'Error allocating becp2_nc', 1)
      ELSE
        ALLOCATE(becp2(nkb, nbnd), STAT = ierr)
        IF (ierr /= 0) CALL errore('compute_mmn_para', 'Error allocating becp2', 1)
      ENDIF

(2) Before the line including

Code: Select all

CALL allocate_bec_type(nkb, nbnd, becp)

, insert the following line:

Code: Select all

CALL deallocate_bec_type(becp)

PS) You should consider some cautions in use of the SCDM method I mentioned in my post above.

Sincerely,

H. Lee

[adenchfi]

Hello H. Lee,

Thank you for your reply! I would have replied sooner, but the "Notify me when a reply is posted" feature seemed to not send me an email. I'll have to double-check my information.

I have been working on finding or generating norm-conserving (NC) pseudopotentials the past couple days using atomic/ld1.x exactly because of your recommendation, and let me tell you, for some rare-earth elements (in my case La, Ce, through Dy) it is very difficult and doesn't seem to be very transferable! Regardless, I will be trying to do that.

I had just cloned q-e from the GitHub (not GitLab, but I assumed they were synced), so I assumed I was using the latest. However, I see the GitLab version is more up-to-date; I will compile that version now!

And yes, I will try it with the smallest isolated band subspace for my particular problem. Thanks for your help!

[hlee]

Dear adenchfi:

I have been working on finding or generating norm-conserving (NC) pseudopotentials the past couple days using atomic/ld1.x exactly because of your recommendation, and let me tell you, for some rare-earth elements (in my case La, Ce, through Dy) it is very difficult and doesn't seem to be very transferable! Regardless, I will be trying to do that.

You are right. La is ok, but regarding Ce through Dy it is VERY difficult to construct a transferrable norm-conserving pseudopotentials (NCPPs). If you succeed in it, you should provide in the supplementary section of your future paper your inputs to ld1.x so that you can get credit for your rare-earth NCPPs.
To the best of my knowledge, there is no PPs for Ce through Dy in pseudo-dojo (http://www.pseudo-dojo.org/), pslibrary (https://dalcorso.github.io/pslibrary/), and gbrv (https://www.physics.rutgers.edu/gbrv/).

Even if you succeed in constructing PPs, you might need the Hubbard U correction. In this case, there is no problem with scf, phonon, and Wannier90 (W90), but EPW is currently incompatible with DFT+U (you need some modification).

In any case, you need good reference PPs; although I didn't check, you can try "Rare Earth PAW datasets" (http://www.mineralscloud.com/resources/ ... ndex.shtml I don't know the reason, but this page is currently inaccessible)

I had just cloned q-e from the GitHub (not GitLab, but I assumed they were synced), so I assumed I was using the latest. However, I see the GitLab version is more up-to-date; I will compile that version now!

The most-recent development version of EPW is available in GitLab (https://gitlab.com/QEF/q-e).
However, before using EPW I would suggest you to check whether phonon calculation and Wannierization with the SCDM method work with PAWs mentioned above.
EPW heavily relies on the results from phonon (ph.x) and W90. Therefore, if you can't obtain the reasonable results from ph.x and W90, it makes no sense to use EPW.

Sincerely,

H. Lee

[adenchfi]

Hello H. Lee,

In particular I'm studying the RTe_3 compounds, where R is a lanthanide. Chemical intuition tells me Te likes to fill its valence, and so the R will contribute some electrons, and act as if it's ionized. Therefore, I'm trying the R_3+ pseudopotentials on pseudo_dojo (designed for when the lanthanide is highly ionized) right now. My inputs with ld1.x so far have produced pseudopotentials that aren't quite right, but I may keep experimenting!

The PAW pseudopotentials for my system correctly work for the relax/scf/phonon/EPW calculations (this is what I've been doing the past few months), and after adding your bug fix, I tested the SCDM-k Silicon example with PAW pseudopotentials, which works now!

So I expect either way that I'll be good (either with the above NC_pseudos or using the bugfix which now allows US/PAWs with the SCDM-k).

Thanks again!

[hlee]

Dear adenchfi:

Good to know that your calculations work. Well, currently the QR factorization with column pivoting, an essential part of the SCDM method, is not parallelized both in Wannier90 and EPW; thus, for large configurations, this step could be slow.

Lastly, as you probably know, the bug in the SCDM method with PAW or USPP was fixed in the most-recent development version of EPW:
https://gitlab.com/QEF/q-e/-/merge_requests/928

I fixed the bug slightly different from the temporal fix I mentioned in this thread.

Sincerely,

H. Lee

[adenchfi]

Hi H. Lee,

Is the slow part you mentioned in this part of pw2wan2epw.f90?

! vv: Compute unk's on a real grid (the fft grid)
!
psic(:) = czero
psic(dffts%nl(igk_k(1:npw, ik))) = evc(1:npw, ibnd)
CALL invfft('Wave', psic, dffts)
!
! vv: Build Psi_k = Unk * focc at G-point only
pnorm = REAL(SUM(psic(1:nrtot) * CONJG(psic(1:nrtot))), KIND = DP)
norm_psi = DSQRT(pnorm)
psi_gamma(1:nrtot, ibnd1) = (psic(1:nrtot) / norm_psi) * f_gamma

because that's where my system seems to be taking awhile (in fact it hasn't gotten past it yet). I've tried excluding bands by telling Wannier to exclude bands (since specifying the energy window doesn't seem to modify the excluded_bands boolean which SCDM-k checks), but I'm not sure it's working. I have 71 bands and am excluding the first 50, so I'd figure it could handle 21 bands.

Yes, I saw you had merged your changes in there, so I seem to have added your temporary fix in addition to your merged changes. I'll have to undo that!

[hlee]

Dear adenchfi:

Is the slow part you mentioned in this part of pw2wan2epw.f90?

The part you mentioned could be slow, but the main bottleneck is below:

Code: Select all

      IF (noncolin) THEN
        CALL ZGEQP3(numbands, 2 * nrtot, TRANSPOSE(CONJG(psi_gamma)), numbands, &
                    piv, qr_tau, cwork, lcwork, rwork, info)
      ELSE
        CALL ZGEQP3(numbands, nrtot, TRANSPOSE(CONJG(psi_gamma)), numbands, &
                    piv, qr_tau, cwork, lcwork, rwork, info)
      ENDIF

psi_gamma is a matrix with the dimension of numbands by 2 * nrtot, where nrtot is the number of grids in the soft FFT grid.
This matrix operation is done only on the meta_ionode (=0).

If I remember correctly, numbands is the number of Wannier functions when scdm_entanglement = 'isolated'; otherwise, it is the total number of bands minus the excluded bands.

I have 71 bands and am excluding the first 50, so I'd figure it could handle 21 bands.

If you want to exclude the bands from 1 to 50 in the development version of EPW, you should add to the input of EPW the following:

Code: Select all

bands_skipped = 'exclude_bands = 1-50'

You had better check the *.wout file in your working directory so that all parameters related to Wannierization is fine.

PS) I would like to mention again the following:

Meanwhile, I would suggest you to try to get Wannier functions using Wannier90 (W90) >=v3.0. Now you need to check whether the SCDM method works in your case by changing a few parameters and rerunning the Wannierization, and for this purpose, it is more convenient to start with W90.
If it works, you can just use the same relevant inputs in the input of EPW.

Sincerely,

H. Lee

[adenchfi]

Hi H. Lee,

If you want to exclude the bands from 1 to 50 in the development version of EPW, you should add to the input of EPW the following:

Yes, I figured that out by looking in the code. It's good to get confirmation!

PS) I would like to mention again the following:

I assume this is because the SCDM-k is working and parallelized in Wannier90 already. I'll likely take this suggestion next, despite the extra steps in getting WFs via W90 first via manually.

The part you mentioned could be slow, but the main bottleneck is below:

They both involve nrtot, so I suppose with the default FFT grid (160x160x54, according to my epw.out) it must really be bottlenecking at those spots. What must be happening is that the one node's worth of memory isn't handling it, because it's been a couple hours and it hasn't even looped through the first band when performing the

Code: Select all

         
         psic(:) = czero
          psic(dffts%nl(igk_k(1:npw, ik))) = evc(1:npw, ibnd)
          CALL invfft('Wave', psic, dffts)

lines, e.g. the above hasn't completed once, since my print statement before the above line executes, but not the one after.

Anyway, it sounds like the SCDM-k implementation is more complete on the purely W90 side, so I'll work on generating the WFs via W90 before using them in EPW. Is there documentation for having EPW use WFs generated by W90 first, as you are suggesting?