<br>Dear Riccardo,<br><br>First of all let me clarify that "-in" is not an option for mpirun but for espresso itself, i.e. you should type something like <br><br>mpirun -np 4 directory/where/espresso/is/pw.x -in <a href="http://file.in">file.in</a> > file.out<br>
<br>instead of<br><br>mpirun -np 4 directory/where/espresso/is/pw.x < <a href="http://file.in">file.in</a> > file.out<br><br>That's all.<br><br>Second, specifying options like "mpirun -np 4 --host localhost" or "mpirun h C ..." are not likely to solve your problem.<br>
<br><br>Now, I don't know how often people explains step by step the configuration for a particular machine but since I got a 4-core Xeon just one week ago I will try to post here all the steps that I have made so far (please: execute the first four steps as root, the symbols ">" that appear starting some of the lines are just there for signaling the terminal prompt, you don't have to write them):<br>
<br>1- I downloaded the latest versions of icc, ifort, and mkl (10.1.015, 10.1.015, and <a href="http://10.0.2.018">10.0.2.018</a> respectively) from Intel's web site as a non-supported version and installed all the packages including idbg (idbg comes twice, with icc and ifort but it is the same package)<br>
<br>2- I copied the files: ifortvars.sh, iccvars.sh, idbvars.sh, and mklvarsem64t.sh (all of this if you are using bash as your shell) that come with the compilers and the mkl libraries, to the directory /etc/profile.d/ (which should already exist assuming that you don't use ubuntu in your Xeon machine).<br>
<br>3- now, you should download the latest version of LAM/MPI (I found this one easier to configure than mpich2 in runlevel2, since it does not require a certain mpd daemon in order to run properly)<br><br>Let's say that you unpack your lam/mpi inside a directory named /opt/MPI/ then you will have a directory that looks like : /opt/MPI/lam-7.1.4 . Now you can create inside that directory a configure script that looks like:<br>
<br><br>==================================================================<br><br>>vi myscript.sh<br>#!/bin/sh<br><br>cd /opt/MPI/lam-7.1.4<br>CC=icc<br>CXX=icpc<br>FC=ifort<br><br>export CC CXX FC<br><br># Lam without gm<br>
./configure --prefix=/opt/MPI/lam-7.1.4/lam --with-rpi=usysv --without-threads --disable-tv-queue<br><br>make<br>make install<br><br>exit<br><br>=================================================================<br><br><br>
And enable it for execution :<br><br>> chmod 755 myscript.sh<br><br>This version of the script is the most basic that you can have in order to compile your lam/mpi, but I do not know what other options could be used in order to enhance our lam/mpi performance for a Xeon machine and the specific compilers that I am using. If somebody who knows how to improve on this posts this information I would be very grateful as well.<br>
<br>execute this by typing:<br><br>> ./myscript.sh<br><br><br>IF YOU'VE MANAGED TO MAKE ALL OF THIS TO WORK WITH A DIFFERENT COMPILER YOU DON"T NEED TO DO IT AGAIN UNLESS YOU THINK THAT INTEL COMPILERS CAN DO A BETTER JOB.<br>
<br>when the process ends successfully you need to create also a hostfile inside /opt/MPI//lam-7.1.4/lam/etc/ , go inside there and do:<br><br><br>===================================================================<br><br>
>vi hostfile<br>localhost<br>localhost<br>localhost<br>localhost<br><br>===================================================================<br><br><br>4- Then, all what you have to do now is to create yet another file inside /etc/profile.d/ :<br>
<br><br>===================================================================<br><br>>vi lam.sh<br>#!/bin/sh<br><br>export LAM=/opt/MPI/lam-7.1.4/lam<br><br>export OMP_NUM_THREADS=1<br><br>PATH=/opt/MPI/lam-7.1.4/lam/include:/opt/MPI/lam-7.1.4/lam/bin:$PATH<br>
<br>export PATH<br><br>LD_LIBRARY_PATH=/opt/MPI/lam-7.1.4/lam/lib:$LD_LIBRARY_PATH<br><br>export LD_LIBRARY_PATH<br><br>MANPATH=/opt/MPI/lam-7.1.4/lam/man:$MANPATH<br><br>export MANPATH<br><br>export LAMRSH=ssh<br><br>===================================================================<br>
<br><br>Note the following:<br> -export OMP_NUM_THREADS=1 ,as recently suggested by Axel Kohlmeyer is there in order to avoid bad behavior from mkl libraries.<br> -PATH=/opt/MPI/lam-7.1.4/lam/include without this piece the espresso's "./configure" command will not properly compile in parallel since the file "mpif.h" with the definitions is inthere.<br>
<br> What I observed by running "mpirun -np 4" with a non parallel pw.x in my machine was, that with the "<" option the code collapsed, while with the "-in" option the code seems to run but it does the very same calculation in each core, and therefore, it writes the same output four times inside a single output file, of course, still using the same time that one single processor would use.<br>
<br>5- Now, as normal user (logout from root) go and test that your parallel environment is working (maybe you will want to reboot your machine, or "source" each one of the "xxx.sh" scripts recently created inside /etc/profile.d, or open a new terminal)<br>
<br><br>> laminfo<br> LAM/MPI: 7.1.4<br> Prefix: /opt/MPI/lam-7.1.4/lam<br> Architecture: x86_64-unknown-linux-gnu<br> Configured by: root<br> Configured on: Sun Apr 27 16:38:22 EDT 2008<br>
Configure host: GiNUx3<br> Memory manager: ptmalloc2<br> C bindings: yes<br> C++ bindings: yes<br> Fortran bindings: yes<br> C compiler: icc<br> C++ compiler: icpc<br> Fortran compiler: ifort<br>
Fortran symbols: underscore<br> C profiling: yes<br> C++ profiling: yes<br> Fortran profiling: yes<br> C++ exceptions: no<br> Thread support: no<br> ROMIO support: yes<br> IMPI support: no<br>
Debug support: no<br> Purify clean: no<br> SSI boot: globus (API v1.1, Module v0.6)<br> SSI boot: rsh (API v1.1, Module v1.1)<br> SSI boot: slurm (API v1.1, Module v1.0)<br> SSI coll: lam_basic (API v1.1, Module v7.1)<br>
SSI coll: shmem (API v1.1, Module v1.0)<br> SSI coll: smp (API v1.1, Module v1.2)<br> SSI rpi: crtcp (API v1.1, Module v1.1)<br> SSI rpi: lamd (API v1.0, Module v7.1)<br> SSI rpi: sysv (API v1.0, Module v7.1)<br>
SSI rpi: tcp (API v1.0, Module v7.1)<br> SSI rpi: usysv (API v1.0, Module v7.1)<br><br>>lamboot<br><br>LAM 7.1.4/MPI 2 C++/ROMIO - Indiana University<br><br><br>>lamnodes<br>n0 localhost.localdomain:1:origin,this_node<br>
<br>>lamhalt<br><br>LAM 7.1.4/MPI 2 C++/ROMIO - Indiana University<br><br><br><br>NOW ESPRESSO,<br><br>6- After having done the previous steps on my machine the configuration of espresso just worked by doing ./configure , but I had to do the following in order to make espresso understand that the system is a parallel machine,<br>
<br>as root:<br><br>>cp /opt/MPI/lam-7.1.4/lam/bin/mpif77 /opt/MPI/lam-7.1.4/lam/bin/mpif90<br><br>In order to make sure that the parallel compiler is named mpif90, mpif77 is just the default name that lam/mpi uses and it is capable of compiling any fortran code. The last part of the output of ./configure then looks like this after execution:<br>
<br>--------------------------------------------------------------------<br>ESPRESSO can take advantage of several optimized numerical libraries<br>(essl, fftw, mkl...). This configure script attempts to find them,<br>but may fail if they have been installed in non-standard locations.<br>
If a required library is not found, the local copy will be compiled.<br><br>The following libraries have been found:<br> BLAS_LIBS= -lmkl_em64t<br> LAPACK_LIBS= -lmkl_em64t<br> FFT_LIBS=<br>Please check if this is what you expect.<br>
<br>If any libraries are missing, you may specify a list of directories<br>to search and retry, as follows:<br> ./configure LIBDIRS="list of directories, separated by spaces"<br><br>Parallel environment detected successfully.<br>
Configured for compilation of parallel executables.<br><br>For more info, read the ESPRESSO User's Guide (Doc/users-guide.tex).<br>--------------------------------------------------------------------<br>configure: success<br>
<br><br><br>7- The make.sys file should NOW contain:<br><br>=======================================================<br><br>DFLAGS = -D__INTEL -D__FFTW -D__USE_INTERNAL_FFTW -D__MPI -D__PARA<br><br>MPIF90 = mpif90<br>
<br>LD = mpif90<br><br>=======================================================<br><br>Interestingly enough, with this process I can direct input with "<" instead of using "-in" without getting any error, and then I can run all the examples in parallel.<br>
<br><br><br>CHEERS,<br><br>J. A. MONTOYA<br><br><br><br><br><br><div class="gmail_quote">On Fri, Apr 25, 2008 at 10:46 AM, Riccardo Sabatini <<a href="mailto:sabatini@sissa.it">sabatini@sissa.it</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi again,<br>
<br>
i wrote "3 times" meaning that even if i launch mpirun with 4<br>
processors the error comes out just three times, maybe mpirun launches<br>
only three instances of the program,sorry for the misunderstanding. I<br>
tried to launch with the direct location to the file instead the std<br>
input redirection but nothing still works (i don't usually launch pw<br>
interactively, was just the pasted snippet to let the mailing list<br>
understand me).<br>
<br>
I haven't tried anyway the -in flag, seems the openmpi i have<br>
doesn't understand that flag but i'll look in the help manual to see<br>
if there's something similar, maybe the problem is there. I'll let you<br>
know as soon as i try.<br>
<br>
Thanks for the help, regards<br>
<font color="#888888"><br>
Riccardo<br>
</font><div><div></div><div class="Wj3C7c"><br>
Quoting Axel Kohlmeyer <<a href="mailto:akohlmey@cmm.chem.upenn.edu">akohlmey@cmm.chem.upenn.edu</a>>:<br>
<br>
> On Fri, 25 Apr 2008, Riccardo Sabatini wrote:<br>
><br>
> RS> Hello everyone,<br>
><br>
> hi riccardo,<br>
><br>
> RS><br>
> RS> i finally compiled espresso with MPI (thanks for the suggestion,<br>
> RS> with gfortran worked perfectly). I had no problem in the compilation<br>
> RS> but i can't make it run. I'm trying a super easy run: from the<br>
> RS> exercise01 the <a href="http://si.scf.cg.in" target="_blank">si.scf.cg.in</a>.<br>
> RS><br>
> RS> Now, if i run the file with the espresso 3.2 compiled without mpi<br>
> RS> obviously runs perfectly but if i try the same file with the mpi<br>
> RS> version it gives me this error (3 times)<br>
><br>
> doing the same thing 3 times doesn't make it more likely to work...<br>
><br>
> RS> stopping ...<br>
> RS><br>
> ---------------------------------------------------------------------------------<br>
> RS><br>
> RS> My launch command is (i'm running on a four cores processor now)<br>
> RS><br>
> RS> mpirun -np 4 ../QE-MPI/bin/pw.x < <a href="http://prova.in" target="_blank">prova.in</a><br>
><br>
> have you tried the -in flag? not all MPI implementations<br>
> replicate the input across all nodes and i personally<br>
> think it is in general a bad idea to read an input from<br>
> stdin. we don't run anything interactively these days<br>
> anyways and being able to check file status etc. is<br>
> a bit advantage.<br>
><br>
> cheers,<br>
> axel.<br>
><br>
> RS> Is there something i'm missing ? Maybe a line do add for parallel<br>
> RS> compilation in the input file ? I've tried the only option in the<br>
> RS> INPUT_PW about parallel compilation, wf_collect, but nothing changes.<br>
> RS> Since the compilation gave me 0 errors maybe the problem is the<br>
> RS> combination openMPI+gfotran+espresso-4.0.<br>
> RS><br>
> RS> Thanks for the help,<br>
> RS><br>
> RS> Riccardo<br>
> RS><br>
> RS> ----------------------------------------------------------------<br>
> RS> SISSA Webmail <a href="https://webmail.sissa.it/" target="_blank">https://webmail.sissa.it/</a><br>
> RS> Powered by Horde <a href="http://www.horde.org/" target="_blank">http://www.horde.org/</a><br>
> RS><br>
> RS><br>
> RS> _______________________________________________<br>
> RS> Pw_forum mailing list<br>
> RS> <a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
> RS> <a href="http://www.democritos.it/mailman/listinfo/pw_forum" target="_blank">http://www.democritos.it/mailman/listinfo/pw_forum</a><br>
> RS><br>
><br>
> --<br>
> =======================================================================<br>
> Axel Kohlmeyer <a href="mailto:akohlmey@cmm.chem.upenn.edu">akohlmey@cmm.chem.upenn.edu</a> <a href="http://www.cmm.upenn.edu" target="_blank">http://www.cmm.upenn.edu</a><br>
> Center for Molecular Modeling -- University of Pennsylvania<br>
> Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323<br>
> tel: 1-215-898-1582, fax: 1-215-573-6233, office-tel: 1-215-898-5425<br>
> =======================================================================<br>
> If you make something idiot-proof, the universe creates a better idiot.<br>
><br>
<br>
</div></div>----------------------------------------------------------------<br>
<div class="Ih2E3d"> SISSA Webmail <a href="https://webmail.sissa.it/" target="_blank">https://webmail.sissa.it/</a><br>
</div><div class="Ih2E3d"> Powered by Horde <a href="http://www.horde.org/" target="_blank">http://www.horde.org/</a><br>
<br>
<br>
</div><div><div></div><div class="Wj3C7c">_______________________________________________<br>
Pw_forum mailing list<br>
<a href="mailto:Pw_forum@pwscf.org">Pw_forum@pwscf.org</a><br>
<a href="http://www.democritos.it/mailman/listinfo/pw_forum" target="_blank">http://www.democritos.it/mailman/listinfo/pw_forum</a><br>
</div></div></blockquote></div><br>