[Mesa-users] Docker Container
Ian Foley
ifoley2008 at gmail.com
Mon Mar 18 18:21:23 EDT 2019
Thanks for your advice.
Ian
On Tue, 19 Mar 2019 at 8:01 am, Evan Bauer <ebauer at physics.ucsb.edu> wrote:
> Hi Ian,
>
> Increasing the frequency of garbage collection sounds like a good idea to
> me, especially if your star is evolving through new EOS regions quickly.
> There really isn’t much downside to this other than a small speed hit.
>
> If you’re very memory constrained and want to go back to the old way of
> doing things, you also have the option of turning off the new EOS tables
> with
> use_eosDT2 = .false.
> use_eosELM = .false.
>
> Cheers,
> Evan
>
>
> On Mar 18, 2019, at 10:46 AM, Ian Foley <ifoley2008 at gmail.com> wrote:
>
> Thanks Rob for the detailed explanation. I will follow your suggestion and
> check for memory leaks. btw I'm using Windows 10 Professional.
>
> I may also have to increase the frequency of garbage collection to avoid a
> crash. 2GB is a lot more memory to need in 400 models when we are a long
> way into the evolution and I have a limit of 8GB of real memory.
>
> Kind regards
> ian
>
>
> On Mon, 18 Mar 2019 at 20:49, Rob Farmer <r.j.farmer at uva.nl> wrote:
>
>> Hi,
>> >
>> Num EOS files loaded 13000 7 0 17
>> 12 17
>> Num EOS files loaded 13001 0 0 10
>> 4 17
>>
>> The ordering of the numbers is in line 410 in
>> star/job/run_star_support.f90,
>>
>> write(*,*) "Num EOS files loaded", s%model_number, num_DT, num_PT, &
>> num_DT2, num_PTEH, num_ELM
>>
>> So its telling you how many of each type of eos is currently loaded into
>> memory. Then by comparing the before and after the garbage collection we
>> can see whether we removed any eos files.
>>
>> So in you case we removed 7 eosDT files, 7 eosDT2 files, 8 PTEH files and
>> no ELM or PT files. This is only meant as a diagnostic but does show that
>> in this case you removed ~40% of the loaded eos files which should be a
>> good memory saving.
>>
>> >What amazed me was that between model 12460 and 12810 MESA has needed
>> nearly 2 MB of memory! which it has had to grab from the swap space leaving
>> less than 1MB available. That seems a huge amount over a short evolution
>> period. (1475904 to 3373664)
>>
>> I assume you meant GB here? What is likely happening is your model is
>> entering a new region of parameter space so we need to load in more eos
>> data files.
>>
>> But to check that its not a memory leak, run the model once up to some
>> model number and record the ~memory used at the end. Then do a restart from
>> say a 1000 steps before the end and record its memory usage at the end. If
>> there ~same then that is just normal mesa memory usage for this problem. If
>> the first run uses alot more memory then we have leaked memory somewhere.
>>
>> Also are you using the windows home (or pro?) docker container? If home,
>> you can configure the memory it uses, if you look in the
>> win_home_dockerMESA.sh file at the docker-machine create line you can
>> configure the memory it has with --virtualbox-memory=2048 (in mb). You may
>> need to delete the old virtual machine first with the
>> utils/uninstall_win_home.sh script if you change the memory value.
>>
>> Rob
>>
>>
>> On Mon, 18 Mar 2019 at 04:31, Ian Foley via Mesa-users <
>> mesa-users at lists.mesastar.org> wrote:
>>
>>> Hi Evan,
>>>
>>> Thanks for setting up r11554 in the MESA-Docker container. I have
>>> deleted older versions as you suggested. Everything seems to be working
>>> well except in an inlist for a 1M model evolution it crashed in a way like
>>> running out of memory near model 13000. I've attached files I think
>>> sufficient for you to reproduce the effect.
>>>
>>> Memory is cleaned up at model 12,000 and 13,000 because of the following
>>> setting. I might have been able to prevent the crash by decreasing this
>>> setting.
>>> num_steps_for_garbage_collection = 1000
>>> report_garbage_collection = .true.
>>> After the crash, I restarted the run at model 12,000 and since I
>>> modified "re" and "rn" to run star in the background, I can monitor the
>>> memory with "free". I entered the model number in the terminal so I can
>>> record when I executed "free".
>>>
>>> What amazed me was that between model 12460 and 12810 MESA has needed
>>> nearly 2 MB of memory! which it has had to grab from the swap space leaving
>>> less than 1MB available. That seems a huge amount over a short evolution
>>> period. (1475904 to 3373664)
>>>
>>> This is the report garbage collection output at model 13000. I haven't
>>> yet gone to the source code to find out what the number mean.
>>>
>>> Num EOS files loaded 13000 7 0 17
>>> 12 17
>>> Num EOS files loaded 13001 0 0 10
>>> 4 17
>>>
>>> Terminal output for run from model 12000 to 13010.
>>>
>>> docker at a9e770e1dc66:~/docker_work/1M$ 12450
>>> -bash: 450: command not found
>>> docker at a9e770e1dc66:~/docker_work/1M$ free
>>> total used free shared buff/cache
>>> available
>>> Mem: 3056888 2885884 84456 0 86548
>>> 30556
>>> Swap: 4194300 1475904 2718396
>>> docker at a9e770e1dc66:~/docker_work/1M$ 12460
>>> -bash: 12460: command not found
>>> docker at a9e770e1dc66:~/docker_work/1M$ 12810
>>> -bash: 12810: command not found
>>> docker at a9e770e1dc66:~/docker_work/1M$ free
>>> total used free shared buff/cache
>>> available
>>> Mem: 3056888 2895980 76212 0 84696
>>> 21444
>>> Swap: 4194300 3373664 820636
>>> docker at a9e770e1dc66:~/docker_work/1M$ 12900
>>> -bash: 12900: command not found
>>> docker at a9e770e1dc66:~/docker_work/1M$ free
>>> total used free shared buff/cache
>>> available
>>> Mem: 3056888 2893880 69184 0 93824
>>> 18968
>>> Swap: 4194300 3348584 845716
>>> docker at a9e770e1dc66:~/docker_work/1M$ 12990
>>> -bash: 12990: command not found
>>> docker at a9e770e1dc66:~/docker_work/1M$ free
>>> total used free shared buff/cache
>>> available
>>> docker at a9e770e1dc66:~/docker_work/1M$ free
>>> total used free shared buff/cache
>>> available
>>> Mem: 3056888 2883048 79752 0 94088
>>> 29472
>>> Swap: 4194300 3935380 258920
>>> docker at a9e770e1dc66:~/docker_work/1M$ 13010
>>> -bash: 13010: command not found
>>> docker at a9e770e1dc66:~/docker_work/1M$ free
>>> total used free shared buff/cache
>>> available
>>> Mem: 3056888 2905660 75560 0 75668
>>> 16104
>>> Swap: 4194300 2024256 2170044
>>>
>>> The use of such a large memory chunk in such a short number of models is
>>> what is concerning me. Should I expect this with r11554 or is there some
>>> bug?
>>>
>>> Attached files re2.txt is the redirected terminal output
>>> The photo is for model 12,000 used for the restart on my Windows 10
>>> Professional software environment.
>>> I hope that is all you need.
>>>
>>> kind regards
>>> Ian
>>>
>>>
>>> On Sun, 17 Mar 2019 at 06:34, Evan Bauer <ebauer at physics.ucsb.edu>
>>> wrote:
>>>
>>>> Hi Ian,
>>>>
>>>> 11554 should be ready to go if you just “git pull” in the MESA-docker
>>>> repository to update. Let me know if that isn’t working for you. I
>>>> definitely recommend the upgrade.
>>>>
>>>> While you’re at it, I’ll also remind you that it’s probably a good idea
>>>> to clean up your older docker images to save hard drive space. You can
>>>> remove the image of 11532 with this command:
>>>> docker rmi evbauer/mesa_lean:11532.01
>>>>
>>>> You can also check what other older images might be sitting around (and
>>>> how much space they’re using) with this command:
>>>> docker images
>>>>
>>>> If you’re not regularly using the older MESA versions in those images,
>>>> you should probably get rid of them too with the “docker rmi” command.
>>>>
>>>> Cheers,
>>>> Evan
>>>>
>>>>
>>>> _______________________________________________
>>> mesa-users at lists.mesastar.org
>>> https://lists.mesastar.org/mailman/listinfo/mesa-users
>>>
>>>
> --
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.mesastar.org/pipermail/mesa-users/attachments/20190319/75a1d196/attachment.html>
More information about the Mesa-users
mailing list