[Mesa-users] installing mesa as a module on a cluster
Warrick Ball
W.H.Ball at bham.ac.uk
Tue Jul 25 15:35:37 UTC 2023
Hi Rob,
Did you perhaps mean to try to install Mesa, the 3D graphics library?
https://www.mesa3d.org/
This is the user list for the Modules for Experiments in Stellar Astrophysics, which simulates how starts evolve over long times...
https://docs.mesastar.org/
Cheers,
Warrick
___________
Warrick Ball
Senior Research Software Engineer
University of Birmingham
W.H.Ball at bham.ac.uk
On Tue, 25 Jul 2023, Rob Kudyba wrote:
> CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.
> > Next, to turn this into an ad hoc module, what directories and files need to be moved?
>
> Again, if you mean Environment Modules [1], then you probably want to ask in a forum that supports that, since I don't think any users here will have done this. Still, I glanced around for some instructions and found this document useful
> for understanding how a module is defined:
>
> https://researchcomputing.princeton.edu/support/knowledge-base/custom-modules
>
>
> Indeed that is what I meant this entire time.
> I'd just move the whole of `$MESA_DIR` somewhere accessible to all the users that need in, then use the module to export `MESA_DIR`, `MESASDK_ROOT` and do the same things that
>
> $ source $MESASDK_ROOT/bin/mesasdk_init.sh
>
> does. I imagine you could probably get away with not moving some items but most of the space is taken up by things that (for default controls) need to be under `$MESA_DIR`, like equation of state tables.
>
>
> I took a shot at creating one based on some hints from this Git repo, which has a modulefile for the Mesa SDK (paths obfuscated):
>
> #%Module -*- tcl -*-
> ##
> ## modulefile
> ##
> proc ModulesHelp { } {
>
> puts stderr "\tAdds mesa 23.05.1 to your environment variables,"
> }
>
> module-whatis "adds mesa 23.05.1 to your environment variables"
>
> set root /path/to/mesa/mesa-r23.05.1
> prepend-path PATH $root/binary
> setenv MESASDK_ROOT /path/to/mesa/sdk23.7.3/mesasdk
> setenv OMP_NUM_THREADS 2
> setenv MESA_DIR /path/to/mesa/mesa-r23.05.1
> prepend-path PATH /path/to/mesa/sdk23.7.3/mesasdk/bin/
> prepend-path MANPATH /path/to/mesa/sdk23.7.3/mesasdk/share/man/
> setenv PGPLOT_DIR /path/to/mesa/sdk23.7.3/mesasdk/lib/pgplot
> if { [ module-info mode load ] } {
> system "source /path/to/mesa/sdk23.7.3/mesasdk/bin/mesasdk_init.sh"
> }
> It looks like how you make this module discoverable for all users will depend on the system you're deploying on.
>
>
> RHEL 8, Bright Computing provisioning
>
> I've never done this before so I might be completely wrong. Good luck! and let us know what works in the end.
>
>
> I realized I never mentioned what the use case is. We have a request for installing MRtrix3 or use it in a Singularity container. I'm getting this error from mrview, which is likely related in the Remote display issues FAQ:
>
> libGL error: No matching fbConfigs or visuals found
>
> libGL error: failed to load driver: swrast
>
> QOpenGLWidget: Failed to make context current
>
> QOpenGLShaderProgram: could not create shader program
>
> QOpenGLShader: could not create shader
>
> Could not link shader program:
>
> ""
>
>
> mrview: [SYSTEM FATAL CODE: SIGSEGV (11)] Segmentation fault: Invalid memory access
>
>
> I have a pending post in the MRtrix3 forum but wanted to ask here if there are perhaps other environment variables that are needed to get MRtrix3 to use the mesa module I've created.
>
> I'm focused on this comment:
> We expect this will only work using the open-source Mesa 3D graphics library drivers (they provide OpenGL 3.3 support as of version 10.x), most likely using their Gallium llvmpipe software rasteriser
>
>
> On Mon, 24 Jul 2023, Rob Kudyba wrote:
>
> > CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.
> >
> > These might sound like trivial things but there are various things in the `build.log` that suggest you haven't configured MESA_DIR correctly (or are perhaps trying to build MESA from outside MESA_DIR?). e.g.
> >
> > > MESA Version
> > > cat: /path/to/mesa/23.05.01/data/version_number: No such file or directory
> >
> > This should work and return `r23.05.1`. It seems at this point in the `help` script that MESA_DIR is `/path/to/mesa`, which might not exist.
> >
> >
> > I was just trying to obfuscate the paths, which I neglected to do in the log file.
> >
> > > $MESA_DIR
> > > /burg/opt/mesa/23.05.01
> >
> > The previous line uses $MESA_DIR, so I don't know why this line in the same script (called `$MESA_DIR/help`) gets a different value of MESA_DIR.
> >
> > > /path/to/me/mesa-r23.05.1/const
> >
> > Lines like this suggest that MESA_DIR should be set to `/path/to/mesa/mesa-r23.05.1`.
> >
> >
> > Doh, facepalm! I was thinking MESA_DIR should be where it gets installed into. Once I fixed that :
> >
> > MESA installation was successful
> >
> >
> > Couple follow ups. On https://docs.mesastar.org/en/latest/installation.html there is a broken link to https://www.astro.wisc.edu/~townsend/static.php?ref=mesasdk at MESA SDK installation instructions
> >
> > Next, to turn this into an ad hoc module, what directories and files need to be moved? I'm assuming anything in the buildlog that says "has been built, tested, and exported"? Or is there an install path-related variable?
> >
> > I see these:
> >
> > -rw-r--r-- 1 rk3199 user 172 Jul 24 10:58 testhub.yml
> >
> > drwxr-xr-x 2 rk3199 user 12288 Jul 24 10:55 include
> >
> > drwxr-xr-x 2 rk3199 user 4096 Jul 24 10:55 lib
> >
> > drwxr-xr-x 7 rk3199 user 4096 Jul 13 16:27 gyre
> >
> > drwxr-xr-x 5 rk3199 user 4096 Jul 13 16:27 sample
> >
> > drwxr-xr-x 13 rk3199 user 4096 Jul 13 16:26 data
> >
> > drwxr-xr-x 11 rk3199 user 4096 Jul 13 16:25 kap
> >
> > drwxr-xr-x 16 rk3199 user 4096 Jul 13 16:21 eos
> >
> > drwxr-xr-x 8 rk3199 user 4096 Jul 13 16:08 utils
> >
> > drwxr-xr-x 8 rk3199 user 4096 May 26 17:48 auto_diff
> >
> > drwxr-xr-x 11 rk3199 user 4096 May 26 17:48 binary
> >
> > drwxr-xr-x 8 rk3199 user 4096 May 26 17:48 chem
> >
> >
> > On Fri, 14 Jul 2023, Rob Kudyba wrote:
> >
> > > CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.
> > > Now this:
> > > build.log
> > >
> > > *** Copy and paste the following information into your email to mesa-users ***
> > >
> > > MESA Version
> > > cat: /path/to/mesa/23.05.01/data/version_number: No such file or directory
> > >
> > > uname -a
> > > Linux server 4.18.0-193.el8.x86_64 #1 SMP Fri Mar 27 14:35:58 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
> > >
> > > gfortran -v
> > > Using built-in specs.
> > > COLLECT_GCC=/path/to/me/mesasdk/bin/gfortran.exec
> > > COLLECT_LTO_WRAPPER=/path/to/me/mesasdk/bin/../libexec/gcc/x86_64-pc-linux-gnu/12.1.0/lto-wrapper
> > > Target: x86_64-pc-linux-gnu
> > > Configured with: /home/user/sdk2-tmp/build/gcc/configure CC= CXX= --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --prefix=/home/user/sdk2-tmp/mesasdk --with-gmp=/home/user/sdk2-tmp/mesasdk
> > > --with-mpfr=/home/user/sdk2-tmp/mesasdk --with-mpc=/home/user/sdk2-tmp/mesasdk --enable-languages=c,c++,fortran --disable-multilib --disable-nls --disable-libsanitizer --enable-clocale=generic
> > > Thread model: posix
> > > Supported LTO compression algorithms: zlib
> > > gcc version 12.1.0 (GCC)
> > >
> > > $MESASDK_ROOT
> > > /path/to/me/mesasdk
> > >
> > > MESASDK version
> > > x86_64-linux-22.6.1
> > >
> > > $PATH
> > >/path/to/me/mesasdk/bin:/path/to/me/mesasdk/bin:/path/to/me/mesasdk/bin:/cm/shared/apps/slurm/current/sbin:/cm/shared/apps/slurm/current/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/path/to/me/abin:/path/to/me/bin:
> /p
> >
> > > ath/to/me/abin
> > >
> > > $MESA_DIR
> > > /burg/opt/mesa/23.05.01
> > >
> > >
> > > /path/to/me/mesa-r23.05.1/const
> > > building const package.
> > >
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/const has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/utils
> > > building utils package.
> > >
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/utils has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/math
> > > building math package.
> > >
> > > make: 'libmath.a' is up to date.
> > > make: 'libmath.a' is up to date.
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/math has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/mtx
> > > building mtx package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/mtx has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/auto_diffbuilding auto_diff package.
> > >
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/auto_diff has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/hdf5io
> > > building hdf5io package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/hdf5io has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/num
> > > building num package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/num has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/interp_1dbuilding interp_1d package.
> > >
> > > mBreakfastake: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/interp_1d has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/interp_2dbuilding interp_2d package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/interp_2d has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/chem
> > > building chem package.
> > >
> > > make: 'libchem.a' is up to date.
> > > make: 'libchem.a' is up to date.
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/chem has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/colors
> > > building colors package.
> > >
> > > make: 'libcolors.a' is up to date.
> > > make: 'libcolors.a' is up to date.
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/colors has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/eos
> > > building eos package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > > /path/to/me/mesa-r23.05.1/eos/eosDT_builder
> > > make: '../ceos' is up to date.
> > >
> > > mesa/eos has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/kap
> > > building kap package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > > make: '../ckap' is up to date.
> > >
> > > mesa/kap has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/rates
> > > building rates package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/rates has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/neu
> > > building neu package.
> > >
> > > make: 'libneu.a' is up to date.
> > > make: 'libneu.a' is up to date.
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/neu has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/net
> > > building net package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > >
> > > mesa/net has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/star_databuilding star_data package.
> > >
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/star_data has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/turb
> > > building turb package.
> > >
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/turb has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/atm
> > > building atm package.
> > >
> > > make: Nothing to be done for 'all'.
> > > export
> > > done
> > > make: Nothing to be done for 'all'.
> > >
> > > mesa/atm has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/sample
> > > building sample package.
> > >
> > > make: Nothing to be done for 'all'.
> > > done
> > >
> > > mesa/sample has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/gyre
> > > building gyre package.
> > >
> > > make: 'libgyre.a' is up to date.
> > > make: 'libgyre.a' is up to date.
> > > make: '../tester' is up to date.
> > > export
> > > done
> > >
> > > mesa/gyre has been built, tested, and exported.
> > >
> > > ************************************************
> > >
> > > /path/to/me/mesa-r23.05.1/adipls
> > > building adipls package.
> > >
> > > makefile:6: /burg/opt/mesa/23.05.01/utils/makefile_header: No such file or directory
> > > make: *** No rule to make target '/burg/opt/mesa/23.05.01/utils/makefile_header'. Stop.
> > >
> > > /path/to/me/mesa-r23.05.1/adipls/make
> > > FAILED
> > >
> > >
> > > /path/to/me/mesa-r23.05.1/adipls
> > > ./build_and_test FAILED
> > >
> > >
> > > On Fri, Jul 14, 2023, 10:11 AM RICHARD H D TOWNSEND <townsend at astro.wisc.edu> wrote:
> > > Looking through the build log, I see the system compiler is being used. This indicates that the SDK is not properly initialized.
> > >
> > > > On Jul 14, 2023, at 7:17 AM, Rob Kudyba via Mesa-users <mesa-users at lists.mesastar.org> wrote:
> > > >
> > > > <<<I'm not sure how many MESA users have any experience with *installing* MESA as a module but we can try our best to help!<<<
> > > >
> > > > Thanks, I saw a few folks attempt it in the archives, e.g., https://lists.mesastar.org/pipermail/mesa-users/2023-May/014517.html, where getting it to work with an older source package.
> > > >
> > > > <<First, what is required to install MESA as a module? Do you want to do it with the SDK, so that loading the module also sets those environment variables? Or do you want to do it without the SDK and have modules load
> the
> > > dependencies?<<<<
> > > >
> > > > Ideally the latter but open to the former.
> > > >
> > > > <<<Can you explain precisely what you've done to try to build MESA, and attach the full `build.log` file that the `./install` scripts creates?<<<
> > > >
> > > > Just ran ./instal, with the SDK downloaded and 2 variables set:
> > > > cat build.log
> > > > *** Copy and paste the following information into your email to mesa-users ***
> > > >
> > > > MESA Version
> > > > cat: /data/version_number: No such file or directory
> > > >
> > > > uname -a
> > > > Linux ourserver 4.18.0-193.el8.x86_64 #1 SMP Fri Mar 27 14:35:58 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > >
> > > > gfortran -v
> > > > Using built-in specs.
> > > > COLLECT_GCC=gfortran
> > > > COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
> > > > OFFLOAD_TARGET_NAMES=nvptx-none
> > > > OFFLOAD_TARGET_DEFAULT=1
> > > > Target: x86_64-redhat-linux
> > > > Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared
> > > --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only
> > > --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic
> > > --with-arch_32=x86-64 --build=x86_64-redhat-linux
> > > > Thread model: posix
> > > > gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
> > > > $MESASDK_ROOT
> > > >
> > > >
> > > > MESASDK version
> > > > MESASDK not setup
> > > >
> > > > $PATH
> > > > /sbin:/bin:/usr/sbin:/usr/bin
> > > >
> > > > $MESA_DIR
> > > >
> > > >
> > > >
> > > > ****************************************************************
> > > > * *
> > > > * MESA should not be installed with root privaleges *
> > > > * Using sudo is a bad idea and unlikely to fix the *
> > > > * problem you think you have. *
> > > > * *
> > > > * If you have permission issues, *
> > > > * contact the mesa-users mailing list *
> > > > * *
> > > > * If you wish to install mesa in a root location *
> > > > * then we assume you can edit the *
> > > > * ./install file to remove this check *
> > > > * *
> > > > ****************************************************************
> > > >
> > > >
> > > > Note the typo in 'privaleges'...
> > > >
> > > >
> > > > On Fri, Jul 14, 2023 at 3:34 AM Warrick Ball <W.H.Ball at bham.ac.uk> wrote:
> > > > Hi Rob,
> > > >
> > > > For context, I presume what you mean is that you want to install MESA as an Environment Module [1], which is what many HPC systems use to allow users to load different versions of installed software with commands like
> > > >
> > > > $ module load MESA/r23.05.1
> > > >
> > > > I'm not sure how many MESA users have any experience with *installing* MESA as a module but we can try our best to help!
> > > >
> > > > First, what is required to install MESA as a module? Do you want to do it with the SDK, so that loading the module also sets those environment variables? Or do you want to do it without the SDK and have modules load
> the
> > dependencies?
> > > >
> > > > Second, it's not clear what error you've actually encountered. The segment of the build log you show implies GYRE built fine but the ADIPLS failed to find the makefile, which seems odd. (Why would ADIPLS try to "build"
> the
> > makefile
> > > header?) It also isn't clear that this has anything to do with `sudo`. Can you explain precisely what you've done to try to build MESA, and attach the full `build.log` file that the `./install` scripts creates?
> > > >
> > > > Cheers,
> > > > Warrick
> > > >
> > > > [1] https://modules.readthedocs.io/
> > > >
> > > > ___________
> > > >
> > > > Warrick Ball
> > > > Senior Research Software Engineer
> > > > University of Birmingham
> > > > W.H.Ball at bham.ac.uk
> > > >
> > > > On Fri, 14 Jul 2023, Rob Kudyba via Mesa-users wrote:
> > > >
> > > > > CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.
> > > > > I'd like to install Mesa as a module,.so users can load ad hoc, so multiple users could have access to it at will.
> > > > >
> > > > > On Thu, Jul 13, 2023, 9:20 PM Francis Timmes <fxt44 at mac.com> wrote:
> > > > > hi rob,
> > > > >
> > > > > i do not understand the relationship between root and making "this" (presumably all of mesa) a module. can you explain in more detail what you are trying to accomplish?
> > > > >
> > > > > fxt
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > On Jul 13, 2023, at 1:35 PM, Rob Kudyba via Mesa-users <mesa-users at lists.mesastar.org> wrote:
> > > > > >
> > > > > > I see the warnings to not use root.But I want to make this a module.
> > > > > >
> > > > > > FC ../tester
> > > > > > export
> > > > > > done
> > > > > >
> > > > > > mesa/gyre has been built, tested, and exported.
> > > > > >
> > > > > > ************************************************
> > > > > >
> > > > > > /path/to/me/mesa-r23.05.1/adipls
> > > > > > building adipls package.
> > > > > >
> > > > > > makefile:6: /path/to/mesa/23.05.01/utils/makefile_header: No such file or directory
> > > > > > make: *** No rule to make target '/path/to/mesa/23.05.01/utils/makefile_header'. Stop.
> > > > > >
> > > > > > /path/to/me/mesa-r23.05.1/adipls/make
> > > > > > FAILED
> > > > > >
> > > > > >
> > > > > > /path/to/me/mesa-r23.05.1/adipls
> > > > > > ./build_and_test FAILED
> > > > > >
> > > > > > Do I just remove the root check in the install script?
> > > > > > _______________________________________________
> > > > > > mesa-users at lists.mesastar.org
> > > > > > https://lists.mesastar.org/mailman/listinfo/mesa-users
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > _______________________________________________
> > > > mesa-users at lists.mesastar.org
> > > > https://lists.mesastar.org/mailman/listinfo/mesa-users
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
More information about the Mesa-users
mailing list