[mesa-users] Installing MESA on a supercomputer (Richard Townsend)
Francis Timmes
fxt44 at mac.com
Wed Jan 11 03:06:20 EST 2012
nice! thanks for sharing this info kevin.
fxt
On Jan 10, 2012, at 12:44 PM, Kevin Moore wrote:
> Success!
>
> I just got MESA compiled and ran through inlist_first_thing_to_try and everything looks to be working fine! It took 59s of cpu time and 13s of wall time to run.
>
> Here's what I had to do to get it working on Triton if anyone else is interested:
>
> Add the following lines to my .bashrc file:
> module purge
> module load intel
> module load openmpi_mx
> export MX_RCACHE=0
> export LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/072/lib/intel64:$LD_LIBRARY_PATH
>
> Comment out these lines in mesa/adipls/test/ck:
> scan-amdl.d amdl.mesa >> .list
> check_okay
>
> For some reason, the adipls test was working fine, but there was a weird error when the script called scan-amd.d and this was dumped to the diff.txt file:
> forrtl: severe (47): write to READONLY file, unit 5, file stdin
> Image PC Routine Line Source
>
> scan-amdl.d.x 000000000047B8ED Unknown Unknown Unknown
> scan-amdl.d.x 000000000047A3F5 Unknown Unknown Unknown
> scan-amdl.d.x 00000000004348C0 Unknown Unknown Unknown
> scan-amdl.d.x 00000000004063BF Unknown Unknown Unknown
> scan-amdl.d.x 0000000000405BF2 Unknown Unknown Unknown
> scan-amdl.d.x 000000000042E755 Unknown Unknown Unknown
> scan-amdl.d.x 0000000000402B6F Unknown Unknown Unknown
> scan-amdl.d.x 0000000000402ACC Unknown Unknown Unknown
> libc.so.6 00000038F5C1D994 Unknown Unknown Unknown
> scan-amdl.d.x 00000000004029D9 Unknown Unknown Unknown
>
> I couldn't track down why this was happening, but commenting out those lines in the test script fixed things.
>
> Then just ./install in the mesa directory as usual (Triton thankfully has ifort 11.1). I haven't tried linking with Triton's MKL libraries yet so I just used the default MESA blas & lapack libraries.
>
> -Kevin
>
> On Jan 7, 2012, at 6:52 PM, Kevin Moore wrote:
>
>> Yes, the purpose of doing this was to do parameter surveys as well as have a centralized location where students in our stars class could run MESA. We get free time on a variety of computing centers so this is a simple solution for us.
>>
>> Anyway, just an update - I actually got the people at Triton to compile a newer glibc (2.14) for me and now the installation gets all the way to the mtx module (yay?), giving the error:
>> /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib/liblapack.a(xerbla.o): In function `xerbla_':
>> xerbla.f:(.text+0x66): undefined reference to `_gfortran_transfer_character_write'
>> xerbla.f:(.text+0x76): undefined reference to `_gfortran_transfer_integer_write'
>> collect2: ld returned 1 exit status
>> make: *** [tester] Error 1
>>
>> This is enough to convince me I should probably be compiling it from source directly so will try that and post again if it succeeds/fails.
>>
>> Thanks for all the advice!
>>
>> -Kevin
>>
>> On Jan 5, 2012, at 2:51 PM, Richard Townsend wrote:
>>
>>> Agreed - and in fact I've done these sorts of calculations myself, using Bill's earlier EZ code.
>>>
>>> But in this context (and based on Ben's remarks about the need for specialized compilers) I was assuming 'supercomputer' means something with specialized hardware, such as a cluster linked together with Infiniband interconnects. While such systems can certainly be used for parameter surveys, a simple Ethernet-networked computer cluster can offer the same performance at a fraction of the price.
>>>
>>> Cheers,
>>>
>>> Rich
>>>
>>> On Jan 5, 2012, at 7:55 AM, Francis Timmes <fxt44 at mac.com> wrote:
>>>
>>>> hi rich,
>>>>
>>>> there is lots to be gained by running multiple jobs simultaneously
>>>> on a cluster; mesa (or any other non-mpi code) in embarrassingly parallel mode.
>>>> for example, a parameter survey involving a 100 stars.
>>>>
>>>> fxt
>>>>
>>>>
>>>>
>>>>
>>>> On Jan 5, 2012, at 2:41 AM, Richard Townsend wrote:
>>>>
>>>>> I'm not even sure what is to be gained from running MESA on a supercomputer. Parallelism in MESA is only via OpenMP - there is no MPIA usage. As such, MESA can take advantage of shared memory architectures, but not clustered architectures.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Rich
>>>>>
>>>>> On Jan 5, 2012, at 7:31 AM, Francis Timmes <fxt44 at mac.com> wrote:
>>>>>
>>>>>> hi ben, rich, kevin (and the mesa community),
>>>>>>
>>>>>> i had no luck compiling mesa under pgi on the nersc
>>>>>> machines hopper and franklin this past summer. the issue
>>>>>> boiled down to certain pointer structures used in mesa
>>>>>> not being supported in that version of the pgi compiler.
>>>>>>
>>>>>> i would be interested in hearing from anyone about
>>>>>> successes or failures on running mesa on any supercomputer
>>>>>> or largish parallel cluster.
>>>>>>
>>>>>> fxt
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jan 4, 2012, at 3:56 PM, Ben Brown wrote:
>>>>>>
>>>>>>> Kevin,
>>>>>>> To follow up on Rich's note: you are almost certainly not going to be
>>>>>>> able to use the MESA SDK on a supercomputer. From my experiences with
>>>>>>> those systems (using a very different code), they tend to require highly
>>>>>>> specialized libraries and compilers. A general purpose tool like the SDK
>>>>>>> won't work, because often the libraries being referenced need to be
>>>>>>> compiled against very specific hardware.
>>>>>>>
>>>>>>> Put differently, the login nodes and the compute nodes tend to run very
>>>>>>> different operating systems and codes targeting the compute nodes need to
>>>>>>> be compiled using the specific compilers supported by UCSD for Triton.
>>>>>>> Which means you'll need to come up with a specialized solution for this
>>>>>>> system. I don't have access to Triton myself, so I can't be much more
>>>>>>> specific than that. Looking over the Triton webpages it looks like both
>>>>>>> PGI and Intel compilers are available on the system:
>>>>>>>
>>>>>>> http://tritonresource.sdsc.edu/compile_jobs.php
>>>>>>>
>>>>>>> If you run into problems, you might try contacting Theodore Sande
>>>>>>> (formerly at MIT) for help in getting a version of MESA to compile, as he
>>>>>>> has had the code running on large-scale systems in the past.
>>>>>>>
>>>>>>> Otherwise I could take a look at Ranger or Kraken (supercomputers at TACC
>>>>>>> and NICS; both have Intel and PGI compilers) later this week or next week.
>>>>>>> They won't be a perfect match, but they may get you started.
>>>>>>> --Ben
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Hi Kevin --
>>>>>>>>
>>>>>>>> This is a common problem with systems using 'older' (pre-2.11) versions of the GNU C library (glibc), which is a standard (and significant) part of all Linux distributions. At the moment, I have no plans to support pre-2.11 glibc versions, since 2.11 has been out for over two years. So, the only workaround would be to upgrade to 2.11 or later.
>>>>>>>>
>>>>>>>> cheers,
>>>>>>>>
>>>>>>>> Rich
>>>>>>>>
>>>>>>>> On Jan 4, 2012, at 7:23 PM, Kevin Moore wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I'm trying to install MESA on the Triton computing cluster at UCSD and am getting some weird errors. I know next to nothing about this so am just following the instructions on Rich's website for using the MESA SDK (which has worked fine for my macbook many times). uname -m returns "x86_64" so I'm using the 64-bit linux version of the SDK. All the setup steps seem to work fine (no errors anyway), but I run into problems during the ./install step.
>>>>>>>>>
>>>>>>>>> The first error I encounter is:
>>>>>>>>> gfortran: /lib64/libc.so.6: version `GLIBC_2.11' not found (required by gfortran)
>>>>>>>>> make: *** [const_def.o] Error 1
>>>>>>>>>
>>>>>>>>> Perhaps tellingly, there is not a file named libc.so.6 in my mesa/utils/mesasdk/lib64.
>>>>>>>>>
>>>>>>>>> In addition, now whenever I try and use the man command I get the error:
>>>>>>>>> /usr/bin/gtbl: error while loading shared libraries: /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib64/libstdc++.so.6: ELF file OS ABI invalid
>>>>>>>>>
>>>>>>>>> Any ideas what's going on from someone who may have done this before?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Kevin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>>> _______________________________________________
>>>>>>>>> mesa-users mailing list
>>>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> mesa-users mailing list
>>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>>>
>>>>>>>>
>>>>>>>> End of mesa-users Digest, Vol 29, Issue 2
>>>>>>>> *****************************************
>>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>> _______________________________________________
>>>>>>> mesa-users mailing list
>>>>>>> mesa-users at lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>> _______________________________________________
>>>>>> mesa-users mailing list
>>>>>> mesa-users at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>
>>> ------------------------------------------------------------------------------
>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>> _______________________________________________
>>> mesa-users mailing list
>>> mesa-users at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>
>>
>> ------------------------------------------------------------------------------
>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>> infrastructure or vast IT resources to deliver seamless, secure access to
>> virtual desktops. With this all-in-one solution, easily deploy virtual
>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>> _______________________________________________
>> mesa-users mailing list
>> mesa-users at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users
More information about the Mesa-users
mailing list