[mesa-users] HTCondor and MESA

Earl Bellinger bellinger at mps.mpg.de
Thu Oct 13 11:20:20 EDT 2016

Hi Aaron,

Active condor user here. I wrote a script called maybe_sub.sh that
submits jobs to condor to run in parallel if condor is available
(hence 'maybe'). You can see it here:


You can do something like

maybe_sub.sh -p 8 echo hi

which will execute "echo hi" using 8 processors on the cluster. In
addition to running the program on the cluster as normal, it will
create a directory called maybe_sub_logs which inside details the
outputs of the calculation ("hi"). You can run maybe_sub.sh -h to see
all the options.

[I personally drop this file in a ~/scripts directory, make it
executable (chmod +x maybe_sub.sh), and add ~/scripts to my $PATH
variable in my .bashrc file so that I can call it from anywhere.]

My workflow is to write a simple bash script for setting up my MESA
job and then use maybe_sub.sh to execute that bash script optionally
in parallel using the -p argument to specify the number of processors
- the trick with MESA+condor is to send along the $OMP_NUM_THREADS
variable to condor.

For example, in our recent paper [1] I made a quasirandom grid of
main-sequence solar-like oscillators. For each combination of mass,
metallicity, etc, I called a bash script called dispatch.sh which can
be seen here:


which, on the compute nodes, copies over my "mesa template", changes
the inlist settings to the desired properties, and runs the track in
parallel. This would be called for example with

maybe_sub.sh -p 4 dispatch.sh -M 1.2 -Z 0.001

to run a track with a mass of 1.2 and a metallicity of 0.001 using 4 processors.

To make a grid, I have a python program called sobol_dispatcher.py:


which calls maybe_sub.sh on dispatch.sh with varied arguments, with
something like

bash_cmd = "maybe_sub.sh -p 2 ./dispatch.sh -M 0.9"
subprocess.Popen(bash_cmd.split(), shell=False)

You can write your own script (which I personally find good to do for
each individual project at hand) or link up maybe_sub.sh to one of the
existing MESA scripts.

Hope this is helpful.

[1] Earl P. Bellinger & George C. Angelou et al., "Fundamental
Parameters of Main-Sequence Stars in an Instant with Machine
Learning," 2016 ApJ 830 31,

Best regards,
Earl Bellinger

Earl Bellinger, Ph.D. Candidate
Department of Astronomy, Yale University
Stellar Ages & Galactic Evolution Group, Max Planck Institute

On Thu, Oct 13, 2016 at 10:28 AM, Aaron Dotter <aaron.dotter at gmail.com> wrote:
> Hi folks,
> I'm writing to ask if anyone out there has used the HTCondor software to
> submit jobs on a cluster.  It's a different queue system than I've used in
> the past.
> I've found plenty of tutorials but if you have a MESA submission script that
> you use on this system, I'd be grateful if you'd share it with me.
> Thanks!
> Aaron
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users

More information about the Mesa-users mailing list