Large Scale DFT Export with Multiple Processes

[mamiyaakira]

Running the MEEP program using mpi.
MPI runs node-parallel until sim.run and succeeds.
In subsequent output _ dft, the program terminates abnormally due to insufficient memory.
I want to flush the results of dft _ fields to a file, but I don't have enough memory.
Please let me know if there is any way to avoid it.

[smartalecH]

Currently, the codebase first broadcasts the dft fields to all processes, effectively creating a duplicate copy for each process.

You can either...

• modify the codebase to export the hdf5 without performing the broadcast (edit: see output_dft should use parallel I/O under MPI #1707)
• use fewer processes for your computation. This means your time stepping will slow down, but the number of duplicate DFT fields will also decrease (this is the easiest option to try first)
• buy more RAM

[mamiyaakira]

Thank you for your reply.
Currently, any solution is difficult.
Are there any plans to revise the following?

•modify the codebase to export the hdf5 without performing the broadcast (this is pretty hard, and I believe has even been discussed in other issues...)

[smartalecH]

Not that I know of. But we welcome contributions!

[mamiyaakira]

Thank you for your reply.
I would like to consider reducing the process for now.