Page 1 of 1
Why EPW consume so much memory?
Posted: Sat Sep 24, 2016 3:32 pm
by Ruichun
Why EPW consume so much memory? In my calculation (1T-TiTe2, 3 atoms in unit cell, 44 valence electrons, 24 q-points in ph.x calculation) I used 48 CPUs, 180 GB memory and 90 GB swap, however the calculation was crashed due to the exhaustion of memory!
I attach the k and q meshes I used here.
Code: Select all
nk1 = 12
nk2 = 12
nk3 = 8
nq1 = 6
nq2 = 6
nq3 = 4
nkf1 = 36
nkf2 = 36
nkf3 = 24
nqf1 = 18
nqf2 = 18
nqf3 = 12
How much memory do you think it requires? How to reduce the memory except for using etf_mem = .true.?
Looking forward to your guidance!
Best regards,
Ruichun Xiao
Re: Why EPW consume so much memory?
Posted: Sat Sep 24, 2016 5:11 pm
by sponce
Hello,
It depends at which states it crashes.
If its before the writing of the *.epmatwp file then you can try to reduce the memory need by
- increasing the number of CPUs
- Reducing the initial coarse k-grid. Are you sure you really need 12x12x8 ? Maybe you can try 6x6x4 for the initial k-grid. Usually the Wannier interpolation works well and you do not need such large k-grid.
- With 6x6x4 k-grid you can use a maximum of 144 CPUs. I recommend to use that number if you have enough resource.
- using etf_mem = .false. (the .true. means you put everything in memory).
Best,
Samuel
Re: Why EPW consume so much memory?
Posted: Sun Sep 25, 2016 3:22 am
by Ruichun
Dear Samuel,
Thanks for your reply.
I will reduce the initial coarse k-grid and try again. But some problems also happen.
-When etf_mem = .false. used, the memory indeed decreases, but the communication is as large as 200 MB/s. So the CPUs are dead due to waiting for the data read form the disk.
-In some cases I found that the memory is also not insufficient after the *.epmatwp file generated. How to reduce? Does it mean I should increase the memory of my sever if I do not increase the number of CPU?
-By the way, whats the relationship between the number of CPU and the meshes of nk and nkf?
Best regards,
Ruichun Xiao
Re: Why EPW consume so much memory?
Posted: Mon Sep 26, 2016 3:45 pm
by sponce
Dear Ruichun Xiao,
EPW is parallelized on coarse and fine k-mesh.
During the first part of EPW, the parallelization is done on nk and on nkf during the interpolation part.
Therefore you can only use nk1*nk2*nk3 CPU max during the first part and nkf1*nkf2*nkf3 CPU max during the second part.
The second part (if you want to use more CPU than nk1*nk2*nk3 in the second part) is new and will be release with QE 6.0 (next week or so).
Best,
Samuel