[mesa-users] Installing MESA on a supercomputer (Richard Townsend)

Francis Timmes fxt44 at mac.com
Wed Jan 11 03:06:20 EST 2012


nice! thanks for sharing this info kevin.

fxt



On Jan 10, 2012, at 12:44 PM, Kevin Moore wrote:

> Success!
> 
> I just got MESA compiled and ran through inlist_first_thing_to_try and everything looks to be working fine! It took 59s of cpu time and 13s of wall time to run.
> 
> Here's what I had to do to get it working on Triton if anyone else is interested:
> 
> Add the following lines to my .bashrc file:
> module purge
> module load intel
> module load openmpi_mx
> export MX_RCACHE=0
> export LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/072/lib/intel64:$LD_LIBRARY_PATH
> 
> Comment out these lines in mesa/adipls/test/ck:
> scan-amdl.d amdl.mesa >> .list
> check_okay
> 
> For some reason, the adipls test was working fine, but there was a weird error when the script called scan-amd.d and this was dumped to the diff.txt file:
> forrtl: severe (47): write to READONLY file, unit 5, file stdin
> Image              PC                Routine            Line        Source       
> 
> scan-amdl.d.x      000000000047B8ED  Unknown               Unknown  Unknown
> scan-amdl.d.x      000000000047A3F5  Unknown               Unknown  Unknown
> scan-amdl.d.x      00000000004348C0  Unknown               Unknown  Unknown
> scan-amdl.d.x      00000000004063BF  Unknown               Unknown  Unknown
> scan-amdl.d.x      0000000000405BF2  Unknown               Unknown  Unknown
> scan-amdl.d.x      000000000042E755  Unknown               Unknown  Unknown
> scan-amdl.d.x      0000000000402B6F  Unknown               Unknown  Unknown
> scan-amdl.d.x      0000000000402ACC  Unknown               Unknown  Unknown
> libc.so.6          00000038F5C1D994  Unknown               Unknown  Unknown
> scan-amdl.d.x      00000000004029D9  Unknown               Unknown  Unknown
> 
> I couldn't track down why this was happening, but commenting out those lines in the test script fixed things.
> 
> Then just ./install in the mesa directory as usual (Triton thankfully has ifort 11.1). I haven't tried linking with Triton's MKL libraries yet so I just used the default MESA blas & lapack libraries.
> 
> -Kevin
> 
> On Jan 7, 2012, at 6:52 PM, Kevin Moore wrote:
> 
>> Yes, the purpose of doing this was to do parameter surveys as well as have a centralized location where students in our stars class could run MESA. We get free time on a variety of computing centers so this is a simple solution for us.
>> 
>> Anyway, just an update - I actually got the people at Triton to compile a newer glibc (2.14) for me and now the installation gets all the way to the mtx module (yay?), giving the error:
>> /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib/liblapack.a(xerbla.o): In function `xerbla_':
>> xerbla.f:(.text+0x66): undefined reference to `_gfortran_transfer_character_write'
>> xerbla.f:(.text+0x76): undefined reference to `_gfortran_transfer_integer_write'
>> collect2: ld returned 1 exit status
>> make: *** [tester] Error 1
>> 
>> This is enough to convince me I should probably be compiling it from source directly so will try that and post again if it succeeds/fails.
>> 
>> Thanks for all the advice!
>> 
>> -Kevin
>> 
>> On Jan 5, 2012, at 2:51 PM, Richard Townsend wrote:
>> 
>>> Agreed - and in fact I've done these sorts of calculations myself, using Bill's earlier EZ code.
>>> 
>>> But in this context (and based on Ben's remarks about the need for specialized compilers) I was assuming 'supercomputer' means something with specialized hardware, such as a cluster linked together with Infiniband interconnects. While such systems can certainly be used for parameter surveys, a simple Ethernet-networked computer cluster can offer the same performance at a fraction of the price.
>>> 
>>> Cheers,
>>> 
>>> Rich
>>> 
>>> On Jan 5, 2012, at 7:55 AM, Francis Timmes <fxt44 at mac.com> wrote:
>>> 
>>>> hi rich,
>>>> 
>>>> there is lots to be gained by running multiple jobs simultaneously
>>>> on a cluster; mesa (or any other non-mpi code) in embarrassingly parallel mode. 
>>>> for example, a parameter survey involving a 100 stars.
>>>> 
>>>> fxt
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Jan 5, 2012, at 2:41 AM, Richard Townsend wrote:
>>>> 
>>>>> I'm not even sure what is to be gained from running MESA on a supercomputer. Parallelism in MESA is only via OpenMP - there is no MPIA usage. As such, MESA can take advantage of shared memory architectures, but not clustered architectures.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Rich
>>>>> 
>>>>> On Jan 5, 2012, at 7:31 AM, Francis Timmes <fxt44 at mac.com> wrote:
>>>>> 
>>>>>> hi ben, rich, kevin (and the mesa community),
>>>>>> 
>>>>>> i had no luck compiling mesa under pgi on the nersc
>>>>>> machines hopper and franklin this past summer. the issue 
>>>>>> boiled down to certain pointer structures used in mesa
>>>>>> not being supported in that version of the pgi compiler.
>>>>>> 
>>>>>> i would be interested in hearing from anyone about 
>>>>>> successes or failures on running mesa on any supercomputer
>>>>>> or largish parallel cluster.
>>>>>> 
>>>>>> fxt
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Jan 4, 2012, at 3:56 PM, Ben Brown wrote:
>>>>>> 
>>>>>>> Kevin,
>>>>>>> To follow up on Rich's note: you are almost certainly not going to be 
>>>>>>> able to use the MESA SDK on a supercomputer.  From my experiences with 
>>>>>>> those systems (using a very different code), they tend to require highly 
>>>>>>> specialized libraries and compilers.  A general purpose tool like the SDK 
>>>>>>> won't work, because often the libraries being referenced need to be 
>>>>>>> compiled against very specific hardware.
>>>>>>> 
>>>>>>> Put differently, the login nodes and the compute nodes tend to run very 
>>>>>>> different operating systems and codes targeting the compute nodes need to 
>>>>>>> be compiled using the specific compilers supported by UCSD for Triton. 
>>>>>>> Which means you'll need to come up with a specialized solution for this 
>>>>>>> system.  I don't have access to Triton myself, so I can't be much more 
>>>>>>> specific than that.  Looking over the Triton webpages it looks like both 
>>>>>>> PGI and Intel compilers are available on the system:
>>>>>>> 
>>>>>>> http://tritonresource.sdsc.edu/compile_jobs.php
>>>>>>> 
>>>>>>> If you run into problems, you might try contacting Theodore Sande 
>>>>>>> (formerly at MIT) for help in getting a version of MESA to compile, as he 
>>>>>>> has had the code running on large-scale systems in the past.
>>>>>>> 
>>>>>>> Otherwise I could take a look at Ranger or Kraken (supercomputers at TACC 
>>>>>>> and NICS; both have Intel and PGI compilers) later this week or next week.
>>>>>>> They won't be a perfect match, but they may get you started.
>>>>>>> --Ben
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> Hi Kevin --
>>>>>>>> 
>>>>>>>> This is a common problem with systems using 'older' (pre-2.11) versions of the GNU C library (glibc), which is a standard (and significant) part of all Linux distributions. At the moment, I have no plans to support pre-2.11 glibc versions, since 2.11 has been out for over two years. So, the only workaround would be to upgrade to 2.11 or later.
>>>>>>>> 
>>>>>>>> cheers,
>>>>>>>> 
>>>>>>>> Rich
>>>>>>>> 
>>>>>>>> On Jan 4, 2012, at 7:23 PM, Kevin Moore wrote:
>>>>>>>> 
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> I'm trying to install MESA on the Triton computing cluster at UCSD and am getting some weird errors. I know next to nothing about this so am just following the instructions on Rich's website for using the MESA SDK (which has worked fine for my macbook many times). uname -m returns "x86_64" so I'm using the 64-bit linux version of the SDK. All the setup steps seem to work fine (no errors anyway), but I run into problems during the ./install step.
>>>>>>>>> 
>>>>>>>>> The first error I encounter is:
>>>>>>>>> gfortran: /lib64/libc.so.6: version `GLIBC_2.11' not found (required by gfortran)
>>>>>>>>> make: *** [const_def.o] Error 1
>>>>>>>>> 
>>>>>>>>> Perhaps tellingly, there is not a file named libc.so.6 in my mesa/utils/mesasdk/lib64.
>>>>>>>>> 
>>>>>>>>> In addition, now whenever I try and use the man command I get the error:
>>>>>>>>> /usr/bin/gtbl: error while loading shared libraries: /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib64/libstdc++.so.6: ELF file OS ABI invalid
>>>>>>>>> 
>>>>>>>>> Any ideas what's going on from someone who may have done this before?
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Kevin
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>>> _______________________________________________
>>>>>>>>> mesa-users mailing list
>>>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> ------------------------------
>>>>>>>> 
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>> 
>>>>>>>> ------------------------------
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> mesa-users mailing list
>>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>>> 
>>>>>>>> 
>>>>>>>> End of mesa-users Digest, Vol 29, Issue 2
>>>>>>>> *****************************************
>>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>> _______________________________________________
>>>>>>> mesa-users mailing list
>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>> 
>>>>>> 
>>>>>> ------------------------------------------------------------------------------
>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>> _______________________________________________
>>>>>> mesa-users mailing list
>>>>>> mesa-users at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>> 
>>> ------------------------------------------------------------------------------
>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>> _______________________________________________
>>> mesa-users mailing list
>>> mesa-users at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>> 
>> 
>> ------------------------------------------------------------------------------
>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>> infrastructure or vast IT resources to deliver seamless, secure access to
>> virtual desktops. With this all-in-one solution, easily deploy virtual 
>> desktops for less than the cost of PCs and save 60% on VDI infrastructure 
>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>> _______________________________________________
>> mesa-users mailing list
>> mesa-users at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mesa-users
> 
> 
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create 
> new or port existing apps to sell to consumers worldwide. Explore the 
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users





More information about the Mesa-users mailing list