[mesa-users] MESA on intel xeon phi

[Robert FARMER]

Hi
So as Bill mentioned running on a phi, with an unmodified MESA, is much
slower than a running on a laptop/desktop machine. With some tweaking i can
get it down to only a few times slower, but thats reaching a limit with
small changes. The problem is that some of MESA scales well to as many
threads as you have and some don't and those that don't are also the most
time consuming to start with! And the cores the phi have are slow, so
unless we get MESA to parallelise perfectly and also be well vectorized the
phi will be slower than a normal machine.

If you want to give it a go on a phi machine then:
cp utils/makefile_header_test_phi_native utils/makefile_header
cp crlibm/build_and_test_mic_native crlibm/build_and_test

 So then you'll want to build mesa for the phi, which you have to do on the
host, but you cant run the executable on the host so you need to disable
the compile time checks:
create an empty file $MESA_DIR/skip_tests
./mk
ssh mic0
mv $MESA_DIR/skip_tests $MESA_DIR/skip_build
then run ./mk to run the tests on the phi machine.
log out and then compile the work directory for your problem

You'll also want to copy the mesa_cache folders onto the /tmp space on the
phi so you dont have to keep reading them over the network. You'll find
them in $MESA_DIR/data/{eosDT,eosPT,kap,ionization,rates}/cache and then
you need to set the $MESA_CACHES_DIR on the phi to point to the top level
folder ( for this path its $MESA_DIR/data but substitute $MEASA_DIR to a
folder on /tmp)

And then run and hope it works, let me know if you try and need more help.

Something you'll want to setup for the phi is to network mount your
filesystems on the phi so the phi can "see" all your normal folders,
otherwise things get tricky with moving data around.

As for running multiple mesa's on a phi, i don't know i haven't tried it
but i'd expect you would get better performance just using a normal cluster
or even just running on the host machine. The advantage of a phi is running
very many threads which are all well vectorized over a big lump of
contiguous memory. I'd expect running 2 mesas on a phi would destroy all
performance gains you get from giving both jobs ~100 threads each.

Rob


On Sun, Nov 23, 2014 at 11:37 AM, Bill Paxton <paxton at kitp.ucsb.edu> wrote:

> Hi Tassos,
>
> Good to hear from you.  Hope all is well in Geneva.  ;D
>
> Rob Farmer is working on MESA + Phi, so he may want to chime in with more
> details.
>
> My focus is to use the Phi to prepare for workstation architectures of 5+
> years in the future.
> I'm betting that the standard workstation in 5+ years will have a
> processor that has 100's of cores.
> It will be similar to the Phi, but it won't be a coprocessor -- it will
> the the main processor itself.
>
> So my plan is to work with the Phi now to learn how to use 100's of cores
> effectively.  I don't want to get
> distracted by questions of how to use a coprocessor efficiently, since I
> think that is a short term issue.
> I want to concentrate on using the many core architecture, and the Phi is
> just a way to get started now.
>
> That means the Phi itself may not be a short term practical tool for
> improved performance;
> the individual cores on the Phi are about 3x slower than on the host and
> the overhead of running
> as a coprocessor may slow it down too much to compete with the host.   But
> it can still be very useful
> as a development tool to see how/if we can restructure MESA to use over
> 100 hardware threads.
>
> Corollary 1: I don't care at this time about speed on Phi vs speed on
> host.  That will come later.
> For now, I only care about speed up on the Phi as we go from 8 or 10
> threads to 80 or 100.
> That is the measure of how well we are doing going from multi-core to
> many-core.
> And that's where the long term payoff will come.  And, btw, we have a lot
> of work to do!
> I don't want to go into precise numbers yet since we are still working on
> it (Rob in particular).
>
> You can of course run MESA in native mode on the Phi -- that's what we're
> doing for testing.
> (Rob's the expert.)   It will currently be several times slower than the
> host, but that's still useful
> if the alternative is doing nothing with the Phi at all!   One caveat --
> crlibm doesn't work on the Phi
> in the sense that we cannot guarantee bit-for-bit identical results.
>  There is an effort
> in progress (by the original crlibm group) to create a successor that will
> support the Phi,
> but it isn't ready yet.   So if you run things on the Phi, you can't
> expect bit-for-bit reproducibility
> on a workstation.
>
> Corollary 2: I don't want to get distracted by issues of supporting MESA
> on the Phi now.
> We aren't to that stage yet.   So if you decide to try MESA with the Phi,
> please keep that in mind.
>
> Cheers,
> Bill
>
>
>
> On Nov 23, 2014, at 3:29 AM, Anastasios Fragkos wrote:
>
> > Dear Bill and the rest of the MESA community,
> >
> > I remember from a past email in this list that Bill, and maybe some
> other users, have been running MESA on a Xeon Phi co-processor. I will soon
> buy a new workstation and I am considering adding a Xeon Phi, so I was
> wondering if anyone can share their experience on running MESA on Xeon Phi.
> For example, what kind of performance do you see compared to the host cpu?
> Can you run more than one MESA simulations on the same Xeon Phi
> simultaneously? Are there any limitations that we should be aware of? Are
> there any MESA updates planned in the near/mid future from the developers
> side  that would be relevant for Xeon Phi?
> >
> > Thank you!
> > Tassos
> >
> >
> ------------------------------------------------------------------------------
> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> > with Interactivity, Sharing, Native Excel Exports, App Integration & more
> > Get technology previously reserved for billion-dollar corporations, FREE
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> > _______________________________________________
> > mesa-users mailing list
> > mesa-users at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.mesastar.org/pipermail/mesa-users/attachments/20141124/6318b1f5/attachment.html>