[Mesa-users] Docker Container
Evan Bauer
ebauer at physics.ucsb.edu
Mon Mar 18 17:01:35 EDT 2019
Hi Ian,
Increasing the frequency of garbage collection sounds like a good idea to me, especially if your star is evolving through new EOS regions quickly. There really isn’t much downside to this other than a small speed hit.
If you’re very memory constrained and want to go back to the old way of doing things, you also have the option of turning off the new EOS tables with
use_eosDT2 = .false.
use_eosELM = .false.
Cheers,
Evan
> On Mar 18, 2019, at 10:46 AM, Ian Foley <ifoley2008 at gmail.com> wrote:
>
> Thanks Rob for the detailed explanation. I will follow your suggestion and check for memory leaks. btw I'm using Windows 10 Professional.
>
> I may also have to increase the frequency of garbage collection to avoid a crash. 2GB is a lot more memory to need in 400 models when we are a long way into the evolution and I have a limit of 8GB of real memory.
>
> Kind regards
> ian
>
>
>
> On Mon, 18 Mar 2019 at 20:49, Rob Farmer <r.j.farmer at uva.nl <mailto:r.j.farmer at uva.nl>> wrote:
> Hi,
> >
> Num EOS files loaded 13000 7 0 17 12 17
> Num EOS files loaded 13001 0 0 10 4 17
>
> The ordering of the numbers is in line 410 in star/job/run_star_support.f90,
>
> write(*,*) "Num EOS files loaded", s%model_number, num_DT, num_PT, &
> num_DT2, num_PTEH, num_ELM
>
> So its telling you how many of each type of eos is currently loaded into memory. Then by comparing the before and after the garbage collection we can see whether we removed any eos files.
>
> So in you case we removed 7 eosDT files, 7 eosDT2 files, 8 PTEH files and no ELM or PT files. This is only meant as a diagnostic but does show that in this case you removed ~40% of the loaded eos files which should be a good memory saving.
>
> >What amazed me was that between model 12460 and 12810 MESA has needed nearly 2 MB of memory! which it has had to grab from the swap space leaving less than 1MB available. That seems a huge amount over a short evolution period. (1475904 to 3373664)
>
> I assume you meant GB here? What is likely happening is your model is entering a new region of parameter space so we need to load in more eos data files.
>
> But to check that its not a memory leak, run the model once up to some model number and record the ~memory used at the end. Then do a restart from say a 1000 steps before the end and record its memory usage at the end. If there ~same then that is just normal mesa memory usage for this problem. If the first run uses alot more memory then we have leaked memory somewhere.
>
> Also are you using the windows home (or pro?) docker container? If home, you can configure the memory it uses, if you look in the win_home_dockerMESA.sh file at the docker-machine create line you can configure the memory it has with --virtualbox-memory=2048 (in mb). You may need to delete the old virtual machine first with the utils/uninstall_win_home.sh script if you change the memory value.
>
> Rob
>
>
> On Mon, 18 Mar 2019 at 04:31, Ian Foley via Mesa-users <mesa-users at lists.mesastar.org <mailto:mesa-users at lists.mesastar.org>> wrote:
> Hi Evan,
>
> Thanks for setting up r11554 in the MESA-Docker container. I have deleted older versions as you suggested. Everything seems to be working well except in an inlist for a 1M model evolution it crashed in a way like running out of memory near model 13000. I've attached files I think sufficient for you to reproduce the effect.
>
> Memory is cleaned up at model 12,000 and 13,000 because of the following setting. I might have been able to prevent the crash by decreasing this setting.
> num_steps_for_garbage_collection = 1000
> report_garbage_collection = .true.
> After the crash, I restarted the run at model 12,000 and since I modified "re" and "rn" to run star in the background, I can monitor the memory with "free". I entered the model number in the terminal so I can record when I executed "free".
>
> What amazed me was that between model 12460 and 12810 MESA has needed nearly 2 MB of memory! which it has had to grab from the swap space leaving less than 1MB available. That seems a huge amount over a short evolution period. (1475904 to 3373664)
>
> This is the report garbage collection output at model 13000. I haven't yet gone to the source code to find out what the number mean.
>
> Num EOS files loaded 13000 7 0 17 12 17
> Num EOS files loaded 13001 0 0 10 4 17
>
> Terminal output for run from model 12000 to 13010.
>
> docker at a9e770e1dc66:~/docker_work/1M$ 12450
> -bash: 450: command not found
> docker at a9e770e1dc66:~/docker_work/1M$ free
> total used free shared buff/cache available
> Mem: 3056888 2885884 84456 0 86548 30556
> Swap: 4194300 1475904 2718396
> docker at a9e770e1dc66:~/docker_work/1M$ 12460
> -bash: 12460: command not found
> docker at a9e770e1dc66:~/docker_work/1M$ 12810
> -bash: 12810: command not found
> docker at a9e770e1dc66:~/docker_work/1M$ free
> total used free shared buff/cache available
> Mem: 3056888 2895980 76212 0 84696 21444
> Swap: 4194300 3373664 820636
> docker at a9e770e1dc66:~/docker_work/1M$ 12900
> -bash: 12900: command not found
> docker at a9e770e1dc66:~/docker_work/1M$ free
> total used free shared buff/cache available
> Mem: 3056888 2893880 69184 0 93824 18968
> Swap: 4194300 3348584 845716
> docker at a9e770e1dc66:~/docker_work/1M$ 12990
> -bash: 12990: command not found
> docker at a9e770e1dc66:~/docker_work/1M$ free
> total used free shared buff/cache available
> docker at a9e770e1dc66:~/docker_work/1M$ free
> total used free shared buff/cache available
> Mem: 3056888 2883048 79752 0 94088 29472
> Swap: 4194300 3935380 258920
> docker at a9e770e1dc66:~/docker_work/1M$ 13010
> -bash: 13010: command not found
> docker at a9e770e1dc66:~/docker_work/1M$ free
> total used free shared buff/cache available
> Mem: 3056888 2905660 75560 0 75668 16104
> Swap: 4194300 2024256 2170044
>
> The use of such a large memory chunk in such a short number of models is what is concerning me. Should I expect this with r11554 or is there some bug?
>
> Attached files re2.txt is the redirected terminal output
> The photo is for model 12,000 used for the restart on my Windows 10 Professional software environment.
> I hope that is all you need.
>
> kind regards
> Ian
>
>
>
> On Sun, 17 Mar 2019 at 06:34, Evan Bauer <ebauer at physics.ucsb.edu <mailto:ebauer at physics.ucsb.edu>> wrote:
> Hi Ian,
>
> 11554 should be ready to go if you just “git pull” in the MESA-docker repository to update. Let me know if that isn’t working for you. I definitely recommend the upgrade.
>
> While you’re at it, I’ll also remind you that it’s probably a good idea to clean up your older docker images to save hard drive space. You can remove the image of 11532 with this command:
> docker rmi evbauer/mesa_lean:11532.01
>
> You can also check what other older images might be sitting around (and how much space they’re using) with this command:
> docker images
>
> If you’re not regularly using the older MESA versions in those images, you should probably get rid of them too with the “docker rmi” command.
>
> Cheers,
> Evan
>
>
> _______________________________________________
> mesa-users at lists.mesastar.org <mailto:mesa-users at lists.mesastar.org>
> https://lists.mesastar.org/mailman/listinfo/mesa-users <https://lists.mesastar.org/mailman/listinfo/mesa-users>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.mesastar.org/pipermail/mesa-users/attachments/20190318/7814d53b/attachment.html>
More information about the Mesa-users
mailing list