[mesa-users] OpenMP vs. MPI
Brian Mulligan
bwmulligan at astro.as.utexas.edu
Wed Sep 24 20:43:02 EDT 2014
This is going to be heavily dependant upon the scheduling done on your
particular cluster. Here at TACC you are guaranteed all 16 cores on any
node you are allocated, so you don't need to worry about fighting for
memory or processor time. That also means that you can't submit truly
single core jobs, only single node. Other centers may have different plans.
I'd be very surprised if one job per core (i.e. OMP_NUM_THREADS =1) is
the most efficient way to submit unless you are really trying to save
every last CPU hour that you have. In my experience there is good
scaling up to at least 8 cores. I haven't tested anything on a larger
machine yet. I seem to recall Bill saying he tested it on a larger
machine than that (12 cores?).
I guess it depends as well on what you mean by most efficient. If you
have 16 single node jobs it will take slightly longer for all of them to
receive a node, but their running time will be
shorter than 1 single node job with 16 serial processes running.
-Brian
On 09/24/14 17:03, Robert FARMER wrote:
> Also single jobs per core also helps with the job scheduling. Single
> cores jobs can be fitted in to the gaps in the scheduler as it will be
> easier to find machines with single cores free than many cores free.
> Thus your jobs start sooner. I've had clusters where you have to wait
> a day or more to run your code as its so over subscribed, thus being
> able to start asap (even if it will take longer to run as you have
> less cores) gets you a result sooner.
>
> Rob
>
> On Wed, Sep 24, 2014 at 2:54 PM, Ehsan Moravveji
> <e.moravveji at gmail.com <mailto:e.moravveji at gmail.com>> wrote:
>
> Hi all,
> I practically calculate grids most of the time, and take up what
> Rob and Dave already suggested. One job per each core, and that's
> the most efficient way to go.
>
>> But if you have a separate job for each core, then there is a
>> chance to do better if the swapping between jobs is very efficient.
>
> With SLURM and/or PBS/Torque, the job queueing is pretty
> efficient, and the overhead is minimal, specifically if one makes
> use their "job array" feature.
> If you use either SLURM or PBS, I can already help setting you up.
>
> Best
> Ehsan.
>>
>>
>>
>>
>> On Sep 24, 2014, at 2:16 PM, Dave Spiegel wrote:
>>
>>> I did some experimenting on this a few months ago. Just to
>>> reiterate what others are saying:
>>> The most efficient way to run N MESA jobs on N cores is to run
>>> each on its own core. If you have fewer than N jobs and N
>>> cores, then you can take advantage of OpenMP by running your
>>> jobs in parallel, but probably the most efficient thing for you
>>> to do is just run each on its own core.
>>>
>>> On Wed, Sep 24, 2014 at 5:02 PM, Brian Jackson
>>> <decaelus at gmail.com <mailto:decaelus at gmail.com>> wrote:
>>>
>>> Many thanks for the responses.
>>>
>>> Brian
>>>
>>> On Wed, Sep 24, 2014 at 2:58 PM, Robert FARMER
>>> <rjfarmer at asu.edu <mailto:rjfarmer at asu.edu>> wrote:
>>> > Hi,
>>> > There right that an individual job will be no faster on
>>> the cluster than on
>>> > your local machine, in fact maybe slower depending on how
>>> many cores you can
>>> > request for each run, if you can get all 16 cores on the
>>> same machine it
>>> > should be faster than your desktop. Although i don't know
>>> how well mesa
>>> > scales past 10 cores so being limited to at most 16 may
>>> not be an issue.
>>> >
>>> > But what you could do is run many mesa jobs
>>> simultaneously, each taking 1
>>> > core each, so each job takes longer overall but a grid of
>>> mesa models will
>>> > be quicker (depending on how busy your cluster is)
>>> >
>>> > There are guides here
>>> http://mesastar.org/tools-utilities/mesa-on-clusters
>>> > from people running mesa on clusters
>>> >
>>> > Rob
>>> >
>>> > On Wed, Sep 24, 2014 at 1:55 PM, Bill Paxton
>>> <paxton at kitp.ucsb.edu <mailto:paxton at kitp.ucsb.edu>> wrote:
>>> >>
>>> >> Hi Brian,
>>> >>
>>> >> Your IT folks got it right. MESA is designed to make use
>>> of OpenMP
>>> >> shared-memory parallelism. The only value of the cluster
>>> will come if you
>>> >> want to run lots of MESA's doing separate jobs, such as
>>> in a population
>>> >> study or a study of the effects of different parameter
>>> values. The
>>> >> cluster's many nodes aren't going to help on individual
>>> MESA runs. For
>>> >> that, multicores help, at least for up to 8 cores or so
>>> (I have one
>>> >> workstation in my office with 12 cores and another with
>>> 20 -- the 1st an
>>> >> aged mac, the 2nd a new workstation from Microway). So
>>> you might consider
>>> >> trading in your iMac for something with at least 4 cores
>>> and at least 8 MB
>>> >> of ram. To anticipate your next question, GPU's aren't
>>> helpful. ;D They
>>> >> are great for some things, but not for full stellar
>>> evolution, at least not
>>> >> in the way it is done in MESA.
>>> >>
>>> >> Hope that helps,
>>> >> Bill
>>> >>
>>> >>
>>> >>
>>> >> On Sep 24, 2014, at 1:43 PM, Brian Jackson wrote:
>>> >>
>>> >> > Dear MESA users,
>>> >> > I'm planning to run a grid of MESA calculations, and I
>>> was hoping to
>>> >> > use the cluster at my home institution to speed things
>>> up. However,
>>> >> > after getting MESA installed on the cluster, the IT
>>> folks here sent me
>>> >> > the following message (copy-pasted since I don't really
>>> understand the
>>> >> > issues involved):
>>> >> >
>>> >> > "Spent some more time looking at how MESA functioned
>>> and it seems that
>>> >> > it utilizes the OpenMP library, rather than an MPI
>>> library. The issue
>>> >> > with this is programs that utilize OpenMP run on a
>>> shared memory
>>> >> > model, which means your MESA jobs would be limited to
>>> one node (16
>>> >> > cores). Because our cluster is focused on quantity of
>>> processors, the
>>> >> > speed of each individual processor is most likely not
>>> as high as those
>>> >> > on your workstation over there (2.0 GHz). This means
>>> you would likely
>>> >> > experience better results running MESA locally (and
>>> potentially
>>> >> > investing in a more powerful workstation if need be)."
>>> >> >
>>> >> > So it sounds like running MESA on my iMac with a 3.4
>>> GHz processor
>>> >> > might be faster than on the cluster. Any
>>> suggestions/work-arounds?
>>> >> >
>>> >> > Thanks,
>>> >> > Brian
>>> >> >
>>> >> > ---------------------------------------------------
>>> >> > Brian Jackson, Assistant Professor
>>> >> > Dept. of Physics
>>> >> > Boise State University
>>> >> > 1910 University Drive
>>> >> > Boise ID 83725-1570
>>> >> >
>>> >> > E-mail: bjackson at boisestate.edu
>>> <mailto:bjackson at boisestate.edu>
>>> >> > URL: http://www.astrojack.com <http://www.astrojack.com/>
>>> >> > Phone: (208) 426-3723 <tel:%28208%29%20426-3723>
>>> >> > ---------------------------------------------------
>>> >> >
>>> >> >
>>> >> >
>>> ------------------------------------------------------------------------------
>>> >> > Meet PCI DSS 3.0 Compliance Requirements with EventLog
>>> Analyzer
>>> >> > Achieve PCI DSS 3.0 Compliant Status with
>>> Out-of-the-box PCI DSS Reports
>>> >> > Are you Audit-Ready for PCI DSS 3.0 Compliance?
>>> Download White paper
>>> >> > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with
>>> EventLog Analyzer
>>> >> >
>>> >> >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>>> >> > _______________________________________________
>>> >> > mesa-users mailing list
>>> >> > mesa-users at lists.sourceforge.net
>>> <mailto:mesa-users at lists.sourceforge.net>
>>> >> > https://lists.sourceforge.net/lists/listinfo/mesa-users
>>> >>
>>> >>
>>> >>
>>> >>
>>> ------------------------------------------------------------------------------
>>> >> Meet PCI DSS 3.0 Compliance Requirements with EventLog
>>> Analyzer
>>> >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box
>>> PCI DSS Reports
>>> >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download
>>> White paper
>>> >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with
>>> EventLog Analyzer
>>> >>
>>> >>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>>> >> _______________________________________________
>>> >> mesa-users mailing list
>>> >> mesa-users at lists.sourceforge.net
>>> <mailto:mesa-users at lists.sourceforge.net>
>>> >> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>> >
>>> >
>>>
>>> ------------------------------------------------------------------------------
>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI
>>> DSS Reports
>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download
>>> White paper
>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>> Analyzer
>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> mesa-users mailing list
>>> mesa-users at lists.sourceforge.net
>>> <mailto:mesa-users at lists.sourceforge.net>
>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>
>>>
>>>
>>>
>>> --
>>> Dave Spiegel, Ph.D.
>>> ------------------------------------------------------------------------------
>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>> Reports
>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk_______________________________________________
>>> mesa-users mailing list
>>> mesa-users at lists.sourceforge.net
>>> <mailto:mesa-users at lists.sourceforge.net>
>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>
>> ------------------------------------------------------------------------------
>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>> Reports
>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk_______________________________________________
>> mesa-users mailing list
>> mesa-users at lists.sourceforge.net
>> <mailto:mesa-users at lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>
> ------------------------------------------------------------------------------
> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
> Reports
> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> <mailto:mesa-users at lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>
>
>
> ------------------------------------------------------------------------------
> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
>
>
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.mesastar.org/pipermail/mesa-users/attachments/20140924/f6ad60f9/attachment.html>
More information about the Mesa-users
mailing list