Why EPW consume so much memory?

[Ruichun]

Why EPW consume so much memory? In my calculation (1T-TiTe2, 3 atoms in unit cell, 44 valence electrons, 24 q-points in ph.x calculation) I used 48 CPUs, 180 GB memory and 90 GB swap, however the calculation was crashed due to the exhaustion of memory!
I attach the k and q meshes I used here.

Code: Select all

  nk1         = 12
  nk2         = 12
  nk3         = 8
  nq1         = 6
  nq2         = 6
  nq3         = 4
  nkf1 = 36
  nkf2 = 36
  nkf3 = 24
  nqf1 = 18
  nqf2 = 18
  nqf3 = 12

How much memory do you think it requires? How to reduce the memory except for using etf_mem = .true.?
Looking forward to your guidance!
Best regards,

Ruichun Xiao

[sponce]

Hello,

It depends at which states it crashes.

If its before the writing of the *.epmatwp file then you can try to reduce the memory need by
- increasing the number of CPUs
- Reducing the initial coarse k-grid. Are you sure you really need 12x12x8 ? Maybe you can try 6x6x4 for the initial k-grid. Usually the Wannier interpolation works well and you do not need such large k-grid.
- With 6x6x4 k-grid you can use a maximum of 144 CPUs. I recommend to use that number if you have enough resource.
- using etf_mem = .false. (the .true. means you put everything in memory).

Best,

Samuel

[Ruichun]

Dear Samuel,
Thanks for your reply.
I will reduce the initial coarse k-grid and try again. But some problems also happen.
-When etf_mem = .false. used, the memory indeed decreases, but the communication is as large as 200 MB/s. So the CPUs are dead due to waiting for the data read form the disk.
-In some cases I found that the memory is also not insufficient after the *.epmatwp file generated. How to reduce? Does it mean I should increase the memory of my sever if I do not increase the number of CPU?
-By the way, whats the relationship between the number of CPU and the meshes of nk and nkf?

Best regards,
Ruichun Xiao

[sponce]

Dear Ruichun Xiao,

EPW is parallelized on coarse and fine k-mesh.

During the first part of EPW, the parallelization is done on nk and on nkf during the interpolation part.

Therefore you can only use nk1*nk2*nk3 CPU max during the first part and nkf1*nkf2*nkf3 CPU max during the second part.

The second part (if you want to use more CPU than nk1*nk2*nk3 in the second part) is new and will be release with QE 6.0 (next week or so).

Best,

Samuel