[Pw_forum] scalability of the code

[Gabriele Sclauzero]

I've been using QE on BG/P for several months and I'm pretty happy with that, thanks to the effort of Carlo Cavazzoni, Paolo, and others.
As Paolo mentioned here below, it is worth running on BG only if you have a big system in hands (~1000 atoms), because the sysadmins
will ask you to run on at least 2048 cores to show that the machine was really worth the lots of money it costed  :D
... no, actually also because that's the smallest piece of machine that can be connected in the torus topology (peculiarity of the BG network).

I agree with Paolo about I/O, it becomes very expensive because usually there are only little I/O nodes compared to number of computing nodes.
I use io_level='low', which should be safe enough because BG/P nodes crash very rarely (at least on the machine I use) and they have a very good 
diagnostic system that helps admins to foresee possible node failures.

Since you will compute systems with many electrons you will definitely need to compile with scalapack support and use the -ndiag option to pw.x.

The OpenMP parallelization is useful only if the code stops complaining that there are not enough plane waves. In my experience, 
before that number of processors is reached, the MPI parallelization gives much better performances than the OpenMP one, probably because
there are still several parts of the code that have not been "threaded" so far.


HTH


GS


Il giorno 07/feb/2012, alle ore 10.57, Paolo Giannozzi ha scritto:
>> We are indeed interested in installing the code in a massively parallel 
>> machine, but before doing it we would like to have more information.
> 
> QE has been installed (with little effort) on several BlueGene machines.
> It works (i.e. it has ben used for real scientific purposes and has
> produced published results in at least one case I am aware of). You 
> have to be careful with I/O, which can become easily a limiting factor.
> You may need to use all available parallelization levels, in particular
> OpenMP, to satisfy the machine owners who, for sure, wants you to run 
> on a number of cores vastly exceeding what you actually need
> 
> P. 
> -- 
> Paolo Giannozzi, IOM-Democritos and University of Udine, Italy
> 
> 
> _______________________________________________
> Pw_forum mailing list
> Pw_forum at pwscf.org
> http://www.democritos.it/mailman/listinfo/pw_forum


§ Gabriele Sclauzero, EPFL SB ITP CSEA
   PH H2 462, Station 3, CH-1015 Lausanne







-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.democritos.it/pipermail/pw_forum/attachments/20120207/49899410/attachment.htm