[mesa-users] Installing MESA on a supercomputer (Richard Townsend)

Kevin Moore kmoore at physics.ucsb.edu
Tue Jan 10 14:44:19 EST 2012


Success!

I just got MESA compiled and ran through inlist_first_thing_to_try and everything looks to be working fine! It took 59s of cpu time and 13s of wall time to run.

Here's what I had to do to get it working on Triton if anyone else is interested:

Add the following lines to my .bashrc file:
module purge
module load intel
module load openmpi_mx
export MX_RCACHE=0
export LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/072/lib/intel64:$LD_LIBRARY_PATH

Comment out these lines in mesa/adipls/test/ck:
scan-amdl.d amdl.mesa >> .list
check_okay

For some reason, the adipls test was working fine, but there was a weird error when the script called scan-amd.d and this was dumped to the diff.txt file:
forrtl: severe (47): write to READONLY file, unit 5, file stdin
Image              PC                Routine            Line        Source       
      
scan-amdl.d.x      000000000047B8ED  Unknown               Unknown  Unknown
scan-amdl.d.x      000000000047A3F5  Unknown               Unknown  Unknown
scan-amdl.d.x      00000000004348C0  Unknown               Unknown  Unknown
scan-amdl.d.x      00000000004063BF  Unknown               Unknown  Unknown
scan-amdl.d.x      0000000000405BF2  Unknown               Unknown  Unknown
scan-amdl.d.x      000000000042E755  Unknown               Unknown  Unknown
scan-amdl.d.x      0000000000402B6F  Unknown               Unknown  Unknown
scan-amdl.d.x      0000000000402ACC  Unknown               Unknown  Unknown
libc.so.6          00000038F5C1D994  Unknown               Unknown  Unknown
scan-amdl.d.x      00000000004029D9  Unknown               Unknown  Unknown

I couldn't track down why this was happening, but commenting out those lines in the test script fixed things.

Then just ./install in the mesa directory as usual (Triton thankfully has ifort 11.1). I haven't tried linking with Triton's MKL libraries yet so I just used the default MESA blas & lapack libraries.

-Kevin

On Jan 7, 2012, at 6:52 PM, Kevin Moore wrote:

> Yes, the purpose of doing this was to do parameter surveys as well as have a centralized location where students in our stars class could run MESA. We get free time on a variety of computing centers so this is a simple solution for us.
> 
> Anyway, just an update - I actually got the people at Triton to compile a newer glibc (2.14) for me and now the installation gets all the way to the mtx module (yay?), giving the error:
> /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib/liblapack.a(xerbla.o): In function `xerbla_':
> xerbla.f:(.text+0x66): undefined reference to `_gfortran_transfer_character_write'
> xerbla.f:(.text+0x76): undefined reference to `_gfortran_transfer_integer_write'
> collect2: ld returned 1 exit status
> make: *** [tester] Error 1
> 
> This is enough to convince me I should probably be compiling it from source directly so will try that and post again if it succeeds/fails.
> 
> Thanks for all the advice!
> 
> -Kevin
> 
> On Jan 5, 2012, at 2:51 PM, Richard Townsend wrote:
> 
>> Agreed - and in fact I've done these sorts of calculations myself, using Bill's earlier EZ code.
>> 
>> But in this context (and based on Ben's remarks about the need for specialized compilers) I was assuming 'supercomputer' means something with specialized hardware, such as a cluster linked together with Infiniband interconnects. While such systems can certainly be used for parameter surveys, a simple Ethernet-networked computer cluster can offer the same performance at a fraction of the price.
>> 
>> Cheers,
>> 
>> Rich
>> 
>> On Jan 5, 2012, at 7:55 AM, Francis Timmes <fxt44 at mac.com> wrote:
>> 
>>> hi rich,
>>> 
>>> there is lots to be gained by running multiple jobs simultaneously
>>> on a cluster; mesa (or any other non-mpi code) in embarrassingly parallel mode. 
>>> for example, a parameter survey involving a 100 stars.
>>> 
>>> fxt
>>> 
>>> 
>>> 
>>> 
>>> On Jan 5, 2012, at 2:41 AM, Richard Townsend wrote:
>>> 
>>>> I'm not even sure what is to be gained from running MESA on a supercomputer. Parallelism in MESA is only via OpenMP - there is no MPIA usage. As such, MESA can take advantage of shared memory architectures, but not clustered architectures.
>>>> 
>>>> Cheers,
>>>> 
>>>> Rich
>>>> 
>>>> On Jan 5, 2012, at 7:31 AM, Francis Timmes <fxt44 at mac.com> wrote:
>>>> 
>>>>> hi ben, rich, kevin (and the mesa community),
>>>>> 
>>>>> i had no luck compiling mesa under pgi on the nersc
>>>>> machines hopper and franklin this past summer. the issue 
>>>>> boiled down to certain pointer structures used in mesa
>>>>> not being supported in that version of the pgi compiler.
>>>>> 
>>>>> i would be interested in hearing from anyone about 
>>>>> successes or failures on running mesa on any supercomputer
>>>>> or largish parallel cluster.
>>>>> 
>>>>> fxt
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Jan 4, 2012, at 3:56 PM, Ben Brown wrote:
>>>>> 
>>>>>> Kevin,
>>>>>> To follow up on Rich's note: you are almost certainly not going to be 
>>>>>> able to use the MESA SDK on a supercomputer.  From my experiences with 
>>>>>> those systems (using a very different code), they tend to require highly 
>>>>>> specialized libraries and compilers.  A general purpose tool like the SDK 
>>>>>> won't work, because often the libraries being referenced need to be 
>>>>>> compiled against very specific hardware.
>>>>>> 
>>>>>> Put differently, the login nodes and the compute nodes tend to run very 
>>>>>> different operating systems and codes targeting the compute nodes need to 
>>>>>> be compiled using the specific compilers supported by UCSD for Triton. 
>>>>>> Which means you'll need to come up with a specialized solution for this 
>>>>>> system.  I don't have access to Triton myself, so I can't be much more 
>>>>>> specific than that.  Looking over the Triton webpages it looks like both 
>>>>>> PGI and Intel compilers are available on the system:
>>>>>> 
>>>>>> http://tritonresource.sdsc.edu/compile_jobs.php
>>>>>> 
>>>>>> If you run into problems, you might try contacting Theodore Sande 
>>>>>> (formerly at MIT) for help in getting a version of MESA to compile, as he 
>>>>>> has had the code running on large-scale systems in the past.
>>>>>> 
>>>>>> Otherwise I could take a look at Ranger or Kraken (supercomputers at TACC 
>>>>>> and NICS; both have Intel and PGI compilers) later this week or next week.
>>>>>> They won't be a perfect match, but they may get you started.
>>>>>> --Ben
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> Hi Kevin --
>>>>>>> 
>>>>>>> This is a common problem with systems using 'older' (pre-2.11) versions of the GNU C library (glibc), which is a standard (and significant) part of all Linux distributions. At the moment, I have no plans to support pre-2.11 glibc versions, since 2.11 has been out for over two years. So, the only workaround would be to upgrade to 2.11 or later.
>>>>>>> 
>>>>>>> cheers,
>>>>>>> 
>>>>>>> Rich
>>>>>>> 
>>>>>>> On Jan 4, 2012, at 7:23 PM, Kevin Moore wrote:
>>>>>>> 
>>>>>>>> Hi all,
>>>>>>>> 
>>>>>>>> I'm trying to install MESA on the Triton computing cluster at UCSD and am getting some weird errors. I know next to nothing about this so am just following the instructions on Rich's website for using the MESA SDK (which has worked fine for my macbook many times). uname -m returns "x86_64" so I'm using the 64-bit linux version of the SDK. All the setup steps seem to work fine (no errors anyway), but I run into problems during the ./install step.
>>>>>>>> 
>>>>>>>> The first error I encounter is:
>>>>>>>> gfortran: /lib64/libc.so.6: version `GLIBC_2.11' not found (required by gfortran)
>>>>>>>> make: *** [const_def.o] Error 1
>>>>>>>> 
>>>>>>>> Perhaps tellingly, there is not a file named libc.so.6 in my mesa/utils/mesasdk/lib64.
>>>>>>>> 
>>>>>>>> In addition, now whenever I try and use the man command I get the error:
>>>>>>>> /usr/bin/gtbl: error while loading shared libraries: /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib64/libstdc++.so.6: ELF file OS ABI invalid
>>>>>>>> 
>>>>>>>> Any ideas what's going on from someone who may have done this before?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Kevin
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>> _______________________________________________
>>>>>>>> mesa-users mailing list
>>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------
>>>>>>> 
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>> 
>>>>>>> ------------------------------
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> mesa-users mailing list
>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>> 
>>>>>>> 
>>>>>>> End of mesa-users Digest, Vol 29, Issue 2
>>>>>>> *****************************************
>>>>>>> 
>>>>>> 
>>>>>> ------------------------------------------------------------------------------
>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>> _______________________________________________
>>>>>> mesa-users mailing list
>>>>>> mesa-users at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>> 
>>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>> _______________________________________________
>>>>> mesa-users mailing list
>>>>> mesa-users at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>> 
>> ------------------------------------------------------------------------------
>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>> infrastructure or vast IT resources to deliver seamless, secure access to
>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>> _______________________________________________
>> mesa-users mailing list
>> mesa-users at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mesa-users
> 
> 
> ------------------------------------------------------------------------------
> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
> infrastructure or vast IT resources to deliver seamless, secure access to
> virtual desktops. With this all-in-one solution, easily deploy virtual 
> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users





More information about the Mesa-users mailing list