[mesa-users] how to extract columns from existing log file

[Michael Zingale]

I'd go the python route too.  Here's a stripped down script that doesn't
know about MESA or do any sort of error checking:

#!/bin/env python

import numpy as np
import sys

def doit(file, col):

    data = np.genfromtxt(file, names=True)

    od = data[col]

    for n in range(len(od)):
        print od[n]


if __name__ == "__main__":
    doit(sys.argv[1], sys.argv[2])



put it in an executable like select_col.py, then

./select_col.py filename colname

will print out the column whose header (assumed to be the first line) is
colname



On Mon, Oct 20, 2014 at 2:31 PM, Robert FARMER <rjfarmer at asu.edu> wrote:

> Here's a python script based on a python library i made:
>
> You'll need python and numpy
> Assuming your in a mesa work dir (so that history.data is in
> LOGS/history.data)
> Run:
> python /path/to/extractMesa.py col_names.txt
>
> and assuming in col_names.txt each column name is on separate line (also
> the file name for the columns isn't fixed just needs to be first parameter
> to the script)
>
> Output goes to stdout
> You'll get the header plus the columns numbers as well. So it should look
> like a normal history file except now smaller
>
> It should also remove the backups and retries
>
> Never tried this though on 100mb file before.
>
> Rob
>
>
>
>
> On Mon, Oct 20, 2014 at 10:49 AM, Bill Paxton <paxton at kitp.ucsb.edu>
> wrote:
>
>> Dear Santa's Helpers,
>>
>> Thanks for the quick replies.
>>
>> ideally, i'd give this marvelous tool a file with a list of the column
>>>> names that i want and it would do the rest.
>>>>
>>>
>> Now I just need a wrapper script that reads a file with column names,
>> uses the header of the history.data file to map names to column numbers,
>> and then calls gawk to do the work.   Please Santa, can I have that too?  ;D
>>
>> -Bill
>>
>>
>>
>>
>>
>> On Oct 20, 2014, at 10:42 AM, Aaron Dotter wrote:
>>
>> Hi Bill,
>>
>> I was just about to recommend gawk but Rob sorta beat me to it.  It's
>> like awk but traditional awk might choke on 100s of MBs of data.  gawk
>> should not.
>>
>> Otherwise it works just like awk in Rob's email.
>>
>> Aaron
>>
>> On Mon, Oct 20, 2014 at 1:39 PM, Robert FARMER <rjfarmer at asu.edu> wrote:
>>
>>> If you know the column numbers you could do:
>>>
>>> awk '{print $NUM1,$NUM2...}' file
>>>
>>> where $NUM1 is column numbers,
>>> $0 is the whole line
>>> $1 first column etc
>>>
>>> If you want to skip the header then you can do this:
>>> awk '{if(NR>2) print $1,$2...}' file
>>>
>>> Where NR tracks the number of rows
>>>
>>> Rob
>>>
>>>
>>>
>>> On Mon, Oct 20, 2014 at 10:35 AM, Bill Paxton <paxton at kitp.ucsb.edu>
>>> wrote:
>>>
>>>> Some unix wizard out there will have a solution for this one I'm sure.
>>>>
>>>> I have a monster history.data file (100s of MB) with lots and lots of
>>>> columns, most of which are unneeded for my current task --
>>>> it would be lovely to have a simple way to extract a small subset of
>>>> the columns to create a much smaller history.data file.
>>>>
>>>> and of course i don't want to have to do any work to create this tool
>>>> for myself.  i want Santa Claus to give it to me.  now.  ;D
>>>>
>>>> Thanks,
>>>> Bill
>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Comprehensive Server Monitoring with Site24x7.
>>>> Monitor 10 servers for $9/Month.
>>>> Get alerted through email, SMS, voice calls or mobile push
>>>> notifications.
>>>> Take corrective actions from your mobile device.
>>>> http://p.sf.net/sfu/Zoho
>>>> _______________________________________________
>>>> mesa-users mailing list
>>>> mesa-users at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Comprehensive Server Monitoring with Site24x7.
>>> Monitor 10 servers for $9/Month.
>>> Get alerted through email, SMS, voice calls or mobile push notifications.
>>> Take corrective actions from your mobile device.
>>> http://p.sf.net/sfu/Zoho
>>> _______________________________________________
>>> mesa-users mailing list
>>> mesa-users at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>
>>>
>>
>>
>
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> mesa-users mailing list
> mesa-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>


-- 
Michael Zingale
Associate Professor

Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY
11794-3800
*phone*:  631-632-8225
*e-mail*: Michael.Zingale at stonybrook.edu
*web*: http://www.astro.sunysb.edu/mzingale
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.mesastar.org/pipermail/mesa-users/attachments/20141020/2cce6d28/attachment.html>