[mesa-users] problem with different results on different platforms -- mac vs. linux
Tomasz Plewa
tomekplewa at gmail.com
Mon Jan 14 12:47:16 EST 2013
Related to Mike's comments on the role of floating point accuracy, which hold true in general.
More specifically, MESA solves a system of nonlinear (algebraic and differential) equations iteratively. Then the expectation is that once numerical convergence is achieved at every time step, the following numerical solution should be unique. If the numerical solution at later times is not unique, then either the numerical scheme is unstable or the physical solution is unstable at the level of perturbations allowed by numerical convergence criteria.
And so even if numerical convergence is obtained (code runs smoothly), then (1) it does not mean there is no bug and (2) obtaining a unique solution may require using stringent convergence criteria. In either case, there is no guarantee the solution will be physically correct, and this is where common sense and expert insights would be most helpful.
By the way, one way of verifying codes with iterative convergence criteria is to request convergence to the machine precision. It is a formal test, of course, but helps to identify convergence bottlenecks.
Tomek
--
On 01/12/2013 03:39 PM, Michael Zingale wrote:
> A few comments on this, as this is always an interesting problem. I am
> not running MESA and don't know the magnitude of the differences, but
> seeing differences on different machines is itself not unusual. As you
> know, floating point math is an approximation (if anyone hasn't read
> the paper "What Every Computer Scientist Should Know About
> Floating-Point Arithmetic", they should), and the system of equations
> that MESA solves is highly nonlinear, so any small differences in the
> roundoff behavior of some operations (for instance, maybe one chip is
> doing a multiply-add as a single instruction while the other does it
> as two) can get amplified greatly over the life of the simulation.
>
> Gfortran (and gcc) have a huge number of options that control exactly
> what sort of transformations are allowed when optimizing and what
> hardware features to use. In order to understand what is going on
> better, you want to make sure that you control for any optimizations
> that the compilers are doing on the different machines.
>
> Are they the same chip family? I am assuming that they are both
> 64-bit, if not, that's a big difference in instruction sets. Within
> the same 'bitness', there are still big differences between chip
> versions. Perhaps you can try forcing a specific instruction set by
> using -march (see "man gcc" to see the list of different architectures
> just within the Intel family).
>
> Are there defaults for the compilers hidden somewhere that are
> different on the two machines? The gcc FAQ shows how to see exactly
> what options are used at the different optimization levels
>
> http://gcc.gnu.org/wiki/FAQ#What_specific_flags_are_enabled_by_-O1_.28-O2.2C_-O3_or_-Os.29.3F
>
> Do this on each machine and 'diff' the output and account for any
> differences there.
>
> It may also be a good idea to scale back on the optimization level in
> general and see if the solutions agree then.
>
> Finally, just to reiterate, these are highly nonlinear problems, so if
> the excutable does some rounding a little different on one machine vs.
> the other, this can propagate and cause a large divergence in the
> results. Knowing this, and knowing that if you ran with a different
> set of compilers you'd get different #s, when doing simulations, the
> thing you should focus on are the trends and the spectrum of possible
> outcomes and variability allowed in the outcomes. The tracks should
> qualitatively look similar, but obsessing on specific numbers is often
> futile when solving nonlinear systems.
>
> Just my2 cents.
>
>
>
>
>
> On Sat, Jan 12, 2013 at 2:50 PM, Jakub Ostrowski
> <ostrowski at astro.uni.wroc.pl <mailto:ostrowski at astro.uni.wroc.pl>> wrote:
>
> Bill, the differences are huge. You might be right, but it's hard
> to believe that "within the accuracy tolerances" one model at the
> same spot on the HR diagram has Y ~ 0.70 while the other one has Y
> ~ 0.35.
>
> I don't require perfect agreement between the platforms but I
> definitely need to know which one is closer to reality. It's
> obvious that at least one of these two solutions is totally wrong.
> And I have observed that changing parameters have dramatic effect
> on the shape of the blue loop. Well, I guess I'll have a hard time
> trying to explaining my supervisor why I obtained two results with
> one code :)
>
> I think we have to take some effort to achieve better
> compatibility, I mean only one solution. I'm going to work with
> stars with range of masses, let's say, 12-25 M_S so I can help I
> guess.
>
> Thank you for your help. I'll keep you updated about any other
> tests I'm going to make.
>
> Cheers,
> Jakub
>
>
> On 12 sty 2013, at 20:32, Bill Paxton <paxton at kitp.ucsb.edu
> <mailto:paxton at kitp.ucsb.edu>> wrote:
>
> >
> > On Jan 12, 2013, at 10:55 AM, Jakub Ostrowski wrote:
> >
> >> I use SDK in version 20121106 (the latest one) both on Mac and
> Linux. And yes, I'm sure that I use gfortran from SDK on the
> supercomputer. So we have a situation where I use the same version
> of compiler on both machines. The only difference is an operating
> system.
> >
> > Excellent! Now perhaps I understand the problem.
> >
> > Aaron Dotter has recently done comparisons of mesa/star results
> on Mac vs Linux;
> > he found "small" differences even when using the same gfortran
> and SDK,
> > where small means within the accuracy tolerances of the test case.
> > It is of course distressing that there are any differences at
> all -- ideally,
> > the results would be bit-for-bit identical. But that goal has
> not been reached.
> >
> > So if you require bit-for-bit identical on Mac and Linux, then
> mesa is currently out.
> >
> > If you can tolerate some differences, then it becomes a question
> of the sensitivity
> > of your problem to small perturbations caused by the different
> platforms.
> > For example, I believe the details of blue-loops are very
> sensitive so they might
> > show large cumulative effects from the many small perturbations
> that happen
> > when moving to a different platform. You might confirm that by
> checking
> > their sensitivity to small changes in parameters when running on
> the same platform;
> > changes in numerical parameters (such as mesh resolution) as well as
> > physical ones (such as mixing length alpha) might result in big
> changes
> > in the shape of the blue loop.
> >
> > If you are trying to study a problem that is very sensitive,
> then you have a challenge ahead.
> >
> > Perhaps other mesa-users will have suggestions.
> >
> > Good luck,
> > Bill
> >
> >
> >
> >
> >
> >
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET <http://ASP.NET>,
> C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills
> current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122912
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> <mailto:mesa-users at lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>
>
>
> --
> Michael Zingale
> Associate Professor
>
> Dept. of Physics & Astronomy \95 Stony Brook University \95 Stony Brook,
> NY 11794-3800
> /phone/: 631-632-8225
> /e-mail/: Michael.Zingale at stonybrook.edu
> <mailto:Michael.Zingale at stonybrook.edu>
> /web/: http://www.astro.sunysb.edu/mzingale
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122912
>
>
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users
More information about the Mesa-users
mailing list