how to assign log as a variable in bash

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I want to assign log output to "line", I tried the following
line=tail -1000 /var/log/syslog



but it doesn't work.



myscript is like below



#!/bin/bash

line=`tail -1000 /var/log/syslog`
d1=$(date --date="-10 min" "+%b %_d %H:%M")
d2=$(date "+%b %_d %H:%M")
while read line; do
[[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $line
done






share|improve this question


















  • 1




    You are doing wrong: you are thinking like a procedural programmer for shell scripting. Use pipes and xargs (to execute at every line your command).
    – Giacomo Catenazzi
    Dec 22 '17 at 10:03






  • 2




    xargs and uncontrolled output from a log sounds like a bad idea. — It seems that what you are actually trying to do is to get the log from the last 10 minutes, right? This should be easily done with journalctl. — Also you needs quotes around your variables, see mywiki.wooledge.org/BashPitfalls . If you still want to do this as a shell script, go ahead. Otherwise just use Python.
    – Martin Ueding
    Dec 22 '17 at 10:27














up vote
1
down vote

favorite












I want to assign log output to "line", I tried the following
line=tail -1000 /var/log/syslog



but it doesn't work.



myscript is like below



#!/bin/bash

line=`tail -1000 /var/log/syslog`
d1=$(date --date="-10 min" "+%b %_d %H:%M")
d2=$(date "+%b %_d %H:%M")
while read line; do
[[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $line
done






share|improve this question


















  • 1




    You are doing wrong: you are thinking like a procedural programmer for shell scripting. Use pipes and xargs (to execute at every line your command).
    – Giacomo Catenazzi
    Dec 22 '17 at 10:03






  • 2




    xargs and uncontrolled output from a log sounds like a bad idea. — It seems that what you are actually trying to do is to get the log from the last 10 minutes, right? This should be easily done with journalctl. — Also you needs quotes around your variables, see mywiki.wooledge.org/BashPitfalls . If you still want to do this as a shell script, go ahead. Otherwise just use Python.
    – Martin Ueding
    Dec 22 '17 at 10:27












up vote
1
down vote

favorite









up vote
1
down vote

favorite











I want to assign log output to "line", I tried the following
line=tail -1000 /var/log/syslog



but it doesn't work.



myscript is like below



#!/bin/bash

line=`tail -1000 /var/log/syslog`
d1=$(date --date="-10 min" "+%b %_d %H:%M")
d2=$(date "+%b %_d %H:%M")
while read line; do
[[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $line
done






share|improve this question














I want to assign log output to "line", I tried the following
line=tail -1000 /var/log/syslog



but it doesn't work.



myscript is like below



#!/bin/bash

line=`tail -1000 /var/log/syslog`
d1=$(date --date="-10 min" "+%b %_d %H:%M")
d2=$(date "+%b %_d %H:%M")
while read line; do
[[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $line
done








share|improve this question













share|improve this question




share|improve this question








edited Dec 22 '17 at 10:18









Jeff Schaller

31.8k848109




31.8k848109










asked Dec 22 '17 at 9:53









Prabash

226




226







  • 1




    You are doing wrong: you are thinking like a procedural programmer for shell scripting. Use pipes and xargs (to execute at every line your command).
    – Giacomo Catenazzi
    Dec 22 '17 at 10:03






  • 2




    xargs and uncontrolled output from a log sounds like a bad idea. — It seems that what you are actually trying to do is to get the log from the last 10 minutes, right? This should be easily done with journalctl. — Also you needs quotes around your variables, see mywiki.wooledge.org/BashPitfalls . If you still want to do this as a shell script, go ahead. Otherwise just use Python.
    – Martin Ueding
    Dec 22 '17 at 10:27












  • 1




    You are doing wrong: you are thinking like a procedural programmer for shell scripting. Use pipes and xargs (to execute at every line your command).
    – Giacomo Catenazzi
    Dec 22 '17 at 10:03






  • 2




    xargs and uncontrolled output from a log sounds like a bad idea. — It seems that what you are actually trying to do is to get the log from the last 10 minutes, right? This should be easily done with journalctl. — Also you needs quotes around your variables, see mywiki.wooledge.org/BashPitfalls . If you still want to do this as a shell script, go ahead. Otherwise just use Python.
    – Martin Ueding
    Dec 22 '17 at 10:27







1




1




You are doing wrong: you are thinking like a procedural programmer for shell scripting. Use pipes and xargs (to execute at every line your command).
– Giacomo Catenazzi
Dec 22 '17 at 10:03




You are doing wrong: you are thinking like a procedural programmer for shell scripting. Use pipes and xargs (to execute at every line your command).
– Giacomo Catenazzi
Dec 22 '17 at 10:03




2




2




xargs and uncontrolled output from a log sounds like a bad idea. — It seems that what you are actually trying to do is to get the log from the last 10 minutes, right? This should be easily done with journalctl. — Also you needs quotes around your variables, see mywiki.wooledge.org/BashPitfalls . If you still want to do this as a shell script, go ahead. Otherwise just use Python.
– Martin Ueding
Dec 22 '17 at 10:27




xargs and uncontrolled output from a log sounds like a bad idea. — It seems that what you are actually trying to do is to get the log from the last 10 minutes, right? This should be easily done with journalctl. — Also you needs quotes around your variables, see mywiki.wooledge.org/BashPitfalls . If you still want to do this as a shell script, go ahead. Otherwise just use Python.
– Martin Ueding
Dec 22 '17 at 10:27










2 Answers
2






active

oldest

votes

















up vote
0
down vote













Try this change



fd=""
tmplog="/tmp/temp_$$$_$RANDOM"
tail -1000 /var/log/syslog > “$tmplog”

exec fd<“$tmplog"

while IFS='' read -r -u $fd line || [[ -n $line ]]; do
#your commands here
done

exec fd>&-
rm "$tmplog"


How it works:
Assign fd as a variable.
Create a temp file using the script PID and the built in “RANDOM” number (you could also use the mktemp command in place of this).
Open the file descriptor fd for reading of the temp file.
Use while loop with IFS set to nothing and run read command with -u option to read from fd.
Put your commands inside the loop using the same line variable.
Close the file descriptor.
Remove the temp file.






share|improve this answer






















  • @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
    – EnterUserNameHere
    Dec 30 '17 at 7:06










  • Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
    – Prabash
    Dec 31 '17 at 6:48


















up vote
0
down vote













Your assignment works fine -- although $( ... ) is more robust and better style than backticks. But read line doesn't read from line, it reads from stdin (the terminal, since you didn't redirect it) into line, destroying the previous contents. Try one of:



data=$(tail -1000 /file) 
while read line; do echo "$line"; done <<<"$data"
# bash ksh zsh but maybe not older shells
# or
while read line; do echo "$line"; done < <(tail -1000 file)
# ditto

data=$(tail -1000 /file)
echo "$data" | while read line; do echo "$line"; done
# or
tail -1000 file | while read line; do echo "$line"; done
# most shells, but any var set (or other shell change made) within the loop
# will disappear on _some_ shells because pipelines are run in subshells


with appropriate commands added in the loop.



In general to keep data unmangled by read you need -r as shown by EnterUserNameHere, but syslog entries are pretty structured and shouldn't need it for any case I can think of. OTOH adding it won't hurt and is good practice.



Note comparing datestrings won't work early on the first day of some months; for example if now-10min is "Nov 31 23:55" and now is "Dec 01 00:05" then all of the datestrings that are actually between those two times will fail the test $line > $d1 && $line < $d2






share|improve this answer




















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );








     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f412459%2fhow-to-assign-log-as-a-variable-in-bash%23new-answer', 'question_page');

    );

    Post as a guest






























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    0
    down vote













    Try this change



    fd=""
    tmplog="/tmp/temp_$$$_$RANDOM"
    tail -1000 /var/log/syslog > “$tmplog”

    exec fd<“$tmplog"

    while IFS='' read -r -u $fd line || [[ -n $line ]]; do
    #your commands here
    done

    exec fd>&-
    rm "$tmplog"


    How it works:
    Assign fd as a variable.
    Create a temp file using the script PID and the built in “RANDOM” number (you could also use the mktemp command in place of this).
    Open the file descriptor fd for reading of the temp file.
    Use while loop with IFS set to nothing and run read command with -u option to read from fd.
    Put your commands inside the loop using the same line variable.
    Close the file descriptor.
    Remove the temp file.






    share|improve this answer






















    • @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
      – EnterUserNameHere
      Dec 30 '17 at 7:06










    • Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
      – Prabash
      Dec 31 '17 at 6:48















    up vote
    0
    down vote













    Try this change



    fd=""
    tmplog="/tmp/temp_$$$_$RANDOM"
    tail -1000 /var/log/syslog > “$tmplog”

    exec fd<“$tmplog"

    while IFS='' read -r -u $fd line || [[ -n $line ]]; do
    #your commands here
    done

    exec fd>&-
    rm "$tmplog"


    How it works:
    Assign fd as a variable.
    Create a temp file using the script PID and the built in “RANDOM” number (you could also use the mktemp command in place of this).
    Open the file descriptor fd for reading of the temp file.
    Use while loop with IFS set to nothing and run read command with -u option to read from fd.
    Put your commands inside the loop using the same line variable.
    Close the file descriptor.
    Remove the temp file.






    share|improve this answer






















    • @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
      – EnterUserNameHere
      Dec 30 '17 at 7:06










    • Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
      – Prabash
      Dec 31 '17 at 6:48













    up vote
    0
    down vote










    up vote
    0
    down vote









    Try this change



    fd=""
    tmplog="/tmp/temp_$$$_$RANDOM"
    tail -1000 /var/log/syslog > “$tmplog”

    exec fd<“$tmplog"

    while IFS='' read -r -u $fd line || [[ -n $line ]]; do
    #your commands here
    done

    exec fd>&-
    rm "$tmplog"


    How it works:
    Assign fd as a variable.
    Create a temp file using the script PID and the built in “RANDOM” number (you could also use the mktemp command in place of this).
    Open the file descriptor fd for reading of the temp file.
    Use while loop with IFS set to nothing and run read command with -u option to read from fd.
    Put your commands inside the loop using the same line variable.
    Close the file descriptor.
    Remove the temp file.






    share|improve this answer














    Try this change



    fd=""
    tmplog="/tmp/temp_$$$_$RANDOM"
    tail -1000 /var/log/syslog > “$tmplog”

    exec fd<“$tmplog"

    while IFS='' read -r -u $fd line || [[ -n $line ]]; do
    #your commands here
    done

    exec fd>&-
    rm "$tmplog"


    How it works:
    Assign fd as a variable.
    Create a temp file using the script PID and the built in “RANDOM” number (you could also use the mktemp command in place of this).
    Open the file descriptor fd for reading of the temp file.
    Use while loop with IFS set to nothing and run read command with -u option to read from fd.
    Put your commands inside the loop using the same line variable.
    Close the file descriptor.
    Remove the temp file.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Dec 22 '17 at 23:46

























    answered Dec 22 '17 at 10:27









    EnterUserNameHere

    4318




    4318











    • @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
      – EnterUserNameHere
      Dec 30 '17 at 7:06










    • Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
      – Prabash
      Dec 31 '17 at 6:48

















    • @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
      – EnterUserNameHere
      Dec 30 '17 at 7:06










    • Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
      – Prabash
      Dec 31 '17 at 6:48
















    @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
    – EnterUserNameHere
    Dec 30 '17 at 7:06




    @Prabash - did this solution work for you? If so please mark it as correct. If not what problem did you encounter?
    – EnterUserNameHere
    Dec 30 '17 at 7:06












    Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
    – Prabash
    Dec 31 '17 at 6:48





    Actually ,I want to get count of number of records in last 1 hour log. below logic and code one work fines for me. Thnks for ur support , nowdate=$(date +'%H:%M:') pastdate=$(date +'%H:%M:' -d "1 hour ago") echo $nowdate echo $pastdate sed -rne "/$pastdate/,/$nowdate/p" /data/recvdmsg.log | wc -l
    – Prabash
    Dec 31 '17 at 6:48













    up vote
    0
    down vote













    Your assignment works fine -- although $( ... ) is more robust and better style than backticks. But read line doesn't read from line, it reads from stdin (the terminal, since you didn't redirect it) into line, destroying the previous contents. Try one of:



    data=$(tail -1000 /file) 
    while read line; do echo "$line"; done <<<"$data"
    # bash ksh zsh but maybe not older shells
    # or
    while read line; do echo "$line"; done < <(tail -1000 file)
    # ditto

    data=$(tail -1000 /file)
    echo "$data" | while read line; do echo "$line"; done
    # or
    tail -1000 file | while read line; do echo "$line"; done
    # most shells, but any var set (or other shell change made) within the loop
    # will disappear on _some_ shells because pipelines are run in subshells


    with appropriate commands added in the loop.



    In general to keep data unmangled by read you need -r as shown by EnterUserNameHere, but syslog entries are pretty structured and shouldn't need it for any case I can think of. OTOH adding it won't hurt and is good practice.



    Note comparing datestrings won't work early on the first day of some months; for example if now-10min is "Nov 31 23:55" and now is "Dec 01 00:05" then all of the datestrings that are actually between those two times will fail the test $line > $d1 && $line < $d2






    share|improve this answer
























      up vote
      0
      down vote













      Your assignment works fine -- although $( ... ) is more robust and better style than backticks. But read line doesn't read from line, it reads from stdin (the terminal, since you didn't redirect it) into line, destroying the previous contents. Try one of:



      data=$(tail -1000 /file) 
      while read line; do echo "$line"; done <<<"$data"
      # bash ksh zsh but maybe not older shells
      # or
      while read line; do echo "$line"; done < <(tail -1000 file)
      # ditto

      data=$(tail -1000 /file)
      echo "$data" | while read line; do echo "$line"; done
      # or
      tail -1000 file | while read line; do echo "$line"; done
      # most shells, but any var set (or other shell change made) within the loop
      # will disappear on _some_ shells because pipelines are run in subshells


      with appropriate commands added in the loop.



      In general to keep data unmangled by read you need -r as shown by EnterUserNameHere, but syslog entries are pretty structured and shouldn't need it for any case I can think of. OTOH adding it won't hurt and is good practice.



      Note comparing datestrings won't work early on the first day of some months; for example if now-10min is "Nov 31 23:55" and now is "Dec 01 00:05" then all of the datestrings that are actually between those two times will fail the test $line > $d1 && $line < $d2






      share|improve this answer






















        up vote
        0
        down vote










        up vote
        0
        down vote









        Your assignment works fine -- although $( ... ) is more robust and better style than backticks. But read line doesn't read from line, it reads from stdin (the terminal, since you didn't redirect it) into line, destroying the previous contents. Try one of:



        data=$(tail -1000 /file) 
        while read line; do echo "$line"; done <<<"$data"
        # bash ksh zsh but maybe not older shells
        # or
        while read line; do echo "$line"; done < <(tail -1000 file)
        # ditto

        data=$(tail -1000 /file)
        echo "$data" | while read line; do echo "$line"; done
        # or
        tail -1000 file | while read line; do echo "$line"; done
        # most shells, but any var set (or other shell change made) within the loop
        # will disappear on _some_ shells because pipelines are run in subshells


        with appropriate commands added in the loop.



        In general to keep data unmangled by read you need -r as shown by EnterUserNameHere, but syslog entries are pretty structured and shouldn't need it for any case I can think of. OTOH adding it won't hurt and is good practice.



        Note comparing datestrings won't work early on the first day of some months; for example if now-10min is "Nov 31 23:55" and now is "Dec 01 00:05" then all of the datestrings that are actually between those two times will fail the test $line > $d1 && $line < $d2






        share|improve this answer












        Your assignment works fine -- although $( ... ) is more robust and better style than backticks. But read line doesn't read from line, it reads from stdin (the terminal, since you didn't redirect it) into line, destroying the previous contents. Try one of:



        data=$(tail -1000 /file) 
        while read line; do echo "$line"; done <<<"$data"
        # bash ksh zsh but maybe not older shells
        # or
        while read line; do echo "$line"; done < <(tail -1000 file)
        # ditto

        data=$(tail -1000 /file)
        echo "$data" | while read line; do echo "$line"; done
        # or
        tail -1000 file | while read line; do echo "$line"; done
        # most shells, but any var set (or other shell change made) within the loop
        # will disappear on _some_ shells because pipelines are run in subshells


        with appropriate commands added in the loop.



        In general to keep data unmangled by read you need -r as shown by EnterUserNameHere, but syslog entries are pretty structured and shouldn't need it for any case I can think of. OTOH adding it won't hurt and is good practice.



        Note comparing datestrings won't work early on the first day of some months; for example if now-10min is "Nov 31 23:55" and now is "Dec 01 00:05" then all of the datestrings that are actually between those two times will fail the test $line > $d1 && $line < $d2







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Dec 23 '17 at 5:15









        dave_thompson_085

        1,9451810




        1,9451810






















             

            draft saved


            draft discarded


























             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f412459%2fhow-to-assign-log-as-a-variable-in-bash%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            How to check contact read email or not when send email to Individual?

            Bahrain

            Postfix configuration issue with fips on centos 7; mailgun relay