Dear Shubham,
You may want to check the behavior at a finite q-point instead of Gamma, since q = 0 can sometimes be special (e.g., degeneracies or phase/gauge issues) that may affect the apparent Hermiticity.
Also, does your system preserve time-reversal symmetry? This can influence the symmetry properties of the electron–phonon matrix elements.
In addition, please check the quality of your Wannierization (e.g., spreads, band interpolation), as poor Wannierization can also lead to such inconsistencies.
It would be useful to verify these points and see if the issue persists away from gamma.
Best regards,
Shashi