Converting unix epoch timestamp column in every row using sed or awk

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
1
down vote

favorite












I have a file with the following input sample data:



1137921146.499 180900 61.153.158.197 1409 
1137921158.698 181622 61.153.158.197 1409
1137921758.163 180026 221.226.124.114 1374
1137921802.016 179485 121.13.128.132 1409


the 1st column is unix epoch timestamp which i need to convert to human readable format plus i want the data to be delimited as follows



Sun Jan 22 01:12:26 PST 2006|180900|61.153.158.197|1409 
Sun Jan 22 01:12:38 PST 2006|181622|61.153.158.19|1409


i tried to add delimeters using sed 's/ 1,/|/g' and converting the date using date -d @1137921146.499. But i am unable to club these two together in one command..










share|improve this question























  • Downvoted because the problem consists of two separate problems and each can be found simply via a quick search.
    – ceremcem
    May 23 '17 at 7:11







  • 1




    After the edit it's okay, I think. He knows how to solve each problem, but not how to do combine it
    – Philippos
    May 23 '17 at 7:27














up vote
1
down vote

favorite












I have a file with the following input sample data:



1137921146.499 180900 61.153.158.197 1409 
1137921158.698 181622 61.153.158.197 1409
1137921758.163 180026 221.226.124.114 1374
1137921802.016 179485 121.13.128.132 1409


the 1st column is unix epoch timestamp which i need to convert to human readable format plus i want the data to be delimited as follows



Sun Jan 22 01:12:26 PST 2006|180900|61.153.158.197|1409 
Sun Jan 22 01:12:38 PST 2006|181622|61.153.158.19|1409


i tried to add delimeters using sed 's/ 1,/|/g' and converting the date using date -d @1137921146.499. But i am unable to club these two together in one command..










share|improve this question























  • Downvoted because the problem consists of two separate problems and each can be found simply via a quick search.
    – ceremcem
    May 23 '17 at 7:11







  • 1




    After the edit it's okay, I think. He knows how to solve each problem, but not how to do combine it
    – Philippos
    May 23 '17 at 7:27












up vote
1
down vote

favorite









up vote
1
down vote

favorite











I have a file with the following input sample data:



1137921146.499 180900 61.153.158.197 1409 
1137921158.698 181622 61.153.158.197 1409
1137921758.163 180026 221.226.124.114 1374
1137921802.016 179485 121.13.128.132 1409


the 1st column is unix epoch timestamp which i need to convert to human readable format plus i want the data to be delimited as follows



Sun Jan 22 01:12:26 PST 2006|180900|61.153.158.197|1409 
Sun Jan 22 01:12:38 PST 2006|181622|61.153.158.19|1409


i tried to add delimeters using sed 's/ 1,/|/g' and converting the date using date -d @1137921146.499. But i am unable to club these two together in one command..










share|improve this question















I have a file with the following input sample data:



1137921146.499 180900 61.153.158.197 1409 
1137921158.698 181622 61.153.158.197 1409
1137921758.163 180026 221.226.124.114 1374
1137921802.016 179485 121.13.128.132 1409


the 1st column is unix epoch timestamp which i need to convert to human readable format plus i want the data to be delimited as follows



Sun Jan 22 01:12:26 PST 2006|180900|61.153.158.197|1409 
Sun Jan 22 01:12:38 PST 2006|181622|61.153.158.19|1409


i tried to add delimeters using sed 's/ 1,/|/g' and converting the date using date -d @1137921146.499. But i am unable to club these two together in one command..







awk sed timestamps






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 17 at 20:41









Rui F Ribeiro

38.2k1475123




38.2k1475123










asked May 23 '17 at 6:57









Prat

113




113











  • Downvoted because the problem consists of two separate problems and each can be found simply via a quick search.
    – ceremcem
    May 23 '17 at 7:11







  • 1




    After the edit it's okay, I think. He knows how to solve each problem, but not how to do combine it
    – Philippos
    May 23 '17 at 7:27
















  • Downvoted because the problem consists of two separate problems and each can be found simply via a quick search.
    – ceremcem
    May 23 '17 at 7:11







  • 1




    After the edit it's okay, I think. He knows how to solve each problem, but not how to do combine it
    – Philippos
    May 23 '17 at 7:27















Downvoted because the problem consists of two separate problems and each can be found simply via a quick search.
– ceremcem
May 23 '17 at 7:11





Downvoted because the problem consists of two separate problems and each can be found simply via a quick search.
– ceremcem
May 23 '17 at 7:11





1




1




After the edit it's okay, I think. He knows how to solve each problem, but not how to do combine it
– Philippos
May 23 '17 at 7:27




After the edit it's okay, I think. He knows how to solve each problem, but not how to do combine it
– Philippos
May 23 '17 at 7:27










3 Answers
3






active

oldest

votes

















up vote
4
down vote













You can use awk program like this:



awk '"$3"' file


the core is to use strftime function to convert epoch to date format



Here is the output:



#awk '"$3"' file
Sun Jan 22 10:12:26 2006|180900|61.153.158.197|1409
Sun Jan 22 10:12:38 2006|181622|61.153.158.197|1409
Sun Jan 22 10:22:38 2006|180026|221.226.124.114|1374
Sun Jan 22 10:23:22 2006|179485|121.13.128.132|1409


P.S. Or you can use implicit output delimiter:



awk 'BEGIN " $1= strftime("%c",$1) 1' file





share|improve this answer





























    up vote
    1
    down vote













    Or with your shell:



    while read timestamp pid ip port; do
    echo "$(date -d @$timestamp)|$pid|$ip|$port"
    done <yourfile





    share|improve this answer





























      up vote
      1
      down vote













      Using what you already know:



      • GNU date can convert a timestamp to a formatted date by giving it @timestamp.

      • Replacing spaces by | will give you the output you want.

      To that, we add



      • GNU date may operate on a file, converting dates in one batch.

      To batch convert dates with GNU date we have to extract the timestamps and prefix them with @:



      $ sed 's/^([^ ]*).*$/@1/' data.in
      @1137921146.499
      @1137921158.698
      @1137921758.163
      @1137921802.016


      The sed expression substitutes each line with the first space-delimited field prefixed with @.



      With bash (and ksh93, or any shell that understands process substitutions):



      $ date -f <( sed 's/^([^ ]*).*$/@1/' data.in )
      Sun Jan 22 10:12:26 CET 2006
      Sun Jan 22 10:12:38 CET 2006
      Sun Jan 22 10:22:38 CET 2006
      Sun Jan 22 10:23:22 CET 2006


      Then we need to take the other fields of the input data and replace the delimiters:



      $ cut -d ' ' -f 2- data.in | tr ' ' '|'
      180900|61.153.158.197|1409
      181622|61.153.158.197|1409
      180026|221.226.124.114|1374
      179485|121.13.128.132|1409


      Then we paste these two things together with a | as delimiter:



      $ paste -d '|' <( date -f <( sed 's/^([^ ]*).*$/@1/' data.in ) ) <( cut -d ' ' -f 2- data.in | tr ' ' '|' )
      Sun Jan 22 10:12:26 CET 2006|180900|61.153.158.197|1409
      Sun Jan 22 10:12:38 CET 2006|181622|61.153.158.197|1409
      Sun Jan 22 10:22:38 CET 2006|180026|221.226.124.114|1374
      Sun Jan 22 10:23:22 CET 2006|179485|121.13.128.132|1409





      share|improve this answer




















        Your Answer








        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "106"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        convertImagesToLinks: false,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: null,
        bindNavPrevention: true,
        postfix: "",
        imageUploader:
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        ,
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );













         

        draft saved


        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f366706%2fconverting-unix-epoch-timestamp-column-in-every-row-using-sed-or-awk%23new-answer', 'question_page');

        );

        Post as a guest















        Required, but never shown

























        3 Answers
        3






        active

        oldest

        votes








        3 Answers
        3






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes








        up vote
        4
        down vote













        You can use awk program like this:



        awk '"$3"' file


        the core is to use strftime function to convert epoch to date format



        Here is the output:



        #awk '"$3"' file
        Sun Jan 22 10:12:26 2006|180900|61.153.158.197|1409
        Sun Jan 22 10:12:38 2006|181622|61.153.158.197|1409
        Sun Jan 22 10:22:38 2006|180026|221.226.124.114|1374
        Sun Jan 22 10:23:22 2006|179485|121.13.128.132|1409


        P.S. Or you can use implicit output delimiter:



        awk 'BEGIN " $1= strftime("%c",$1) 1' file





        share|improve this answer


























          up vote
          4
          down vote













          You can use awk program like this:



          awk '"$3"' file


          the core is to use strftime function to convert epoch to date format



          Here is the output:



          #awk '"$3"' file
          Sun Jan 22 10:12:26 2006|180900|61.153.158.197|1409
          Sun Jan 22 10:12:38 2006|181622|61.153.158.197|1409
          Sun Jan 22 10:22:38 2006|180026|221.226.124.114|1374
          Sun Jan 22 10:23:22 2006|179485|121.13.128.132|1409


          P.S. Or you can use implicit output delimiter:



          awk 'BEGIN " $1= strftime("%c",$1) 1' file





          share|improve this answer
























            up vote
            4
            down vote










            up vote
            4
            down vote









            You can use awk program like this:



            awk '"$3"' file


            the core is to use strftime function to convert epoch to date format



            Here is the output:



            #awk '"$3"' file
            Sun Jan 22 10:12:26 2006|180900|61.153.158.197|1409
            Sun Jan 22 10:12:38 2006|181622|61.153.158.197|1409
            Sun Jan 22 10:22:38 2006|180026|221.226.124.114|1374
            Sun Jan 22 10:23:22 2006|179485|121.13.128.132|1409


            P.S. Or you can use implicit output delimiter:



            awk 'BEGIN " $1= strftime("%c",$1) 1' file





            share|improve this answer














            You can use awk program like this:



            awk '"$3"' file


            the core is to use strftime function to convert epoch to date format



            Here is the output:



            #awk '"$3"' file
            Sun Jan 22 10:12:26 2006|180900|61.153.158.197|1409
            Sun Jan 22 10:12:38 2006|181622|61.153.158.197|1409
            Sun Jan 22 10:22:38 2006|180026|221.226.124.114|1374
            Sun Jan 22 10:23:22 2006|179485|121.13.128.132|1409


            P.S. Or you can use implicit output delimiter:



            awk 'BEGIN " $1= strftime("%c",$1) 1' file






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited May 22 at 3:01

























            answered May 23 '17 at 7:35









            Romeo Ninov

            4,76431626




            4,76431626






















                up vote
                1
                down vote













                Or with your shell:



                while read timestamp pid ip port; do
                echo "$(date -d @$timestamp)|$pid|$ip|$port"
                done <yourfile





                share|improve this answer


























                  up vote
                  1
                  down vote













                  Or with your shell:



                  while read timestamp pid ip port; do
                  echo "$(date -d @$timestamp)|$pid|$ip|$port"
                  done <yourfile





                  share|improve this answer
























                    up vote
                    1
                    down vote










                    up vote
                    1
                    down vote









                    Or with your shell:



                    while read timestamp pid ip port; do
                    echo "$(date -d @$timestamp)|$pid|$ip|$port"
                    done <yourfile





                    share|improve this answer














                    Or with your shell:



                    while read timestamp pid ip port; do
                    echo "$(date -d @$timestamp)|$pid|$ip|$port"
                    done <yourfile






                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited May 23 '17 at 7:43

























                    answered May 23 '17 at 7:38









                    Philippos

                    5,98211547




                    5,98211547




















                        up vote
                        1
                        down vote













                        Using what you already know:



                        • GNU date can convert a timestamp to a formatted date by giving it @timestamp.

                        • Replacing spaces by | will give you the output you want.

                        To that, we add



                        • GNU date may operate on a file, converting dates in one batch.

                        To batch convert dates with GNU date we have to extract the timestamps and prefix them with @:



                        $ sed 's/^([^ ]*).*$/@1/' data.in
                        @1137921146.499
                        @1137921158.698
                        @1137921758.163
                        @1137921802.016


                        The sed expression substitutes each line with the first space-delimited field prefixed with @.



                        With bash (and ksh93, or any shell that understands process substitutions):



                        $ date -f <( sed 's/^([^ ]*).*$/@1/' data.in )
                        Sun Jan 22 10:12:26 CET 2006
                        Sun Jan 22 10:12:38 CET 2006
                        Sun Jan 22 10:22:38 CET 2006
                        Sun Jan 22 10:23:22 CET 2006


                        Then we need to take the other fields of the input data and replace the delimiters:



                        $ cut -d ' ' -f 2- data.in | tr ' ' '|'
                        180900|61.153.158.197|1409
                        181622|61.153.158.197|1409
                        180026|221.226.124.114|1374
                        179485|121.13.128.132|1409


                        Then we paste these two things together with a | as delimiter:



                        $ paste -d '|' <( date -f <( sed 's/^([^ ]*).*$/@1/' data.in ) ) <( cut -d ' ' -f 2- data.in | tr ' ' '|' )
                        Sun Jan 22 10:12:26 CET 2006|180900|61.153.158.197|1409
                        Sun Jan 22 10:12:38 CET 2006|181622|61.153.158.197|1409
                        Sun Jan 22 10:22:38 CET 2006|180026|221.226.124.114|1374
                        Sun Jan 22 10:23:22 CET 2006|179485|121.13.128.132|1409





                        share|improve this answer
























                          up vote
                          1
                          down vote













                          Using what you already know:



                          • GNU date can convert a timestamp to a formatted date by giving it @timestamp.

                          • Replacing spaces by | will give you the output you want.

                          To that, we add



                          • GNU date may operate on a file, converting dates in one batch.

                          To batch convert dates with GNU date we have to extract the timestamps and prefix them with @:



                          $ sed 's/^([^ ]*).*$/@1/' data.in
                          @1137921146.499
                          @1137921158.698
                          @1137921758.163
                          @1137921802.016


                          The sed expression substitutes each line with the first space-delimited field prefixed with @.



                          With bash (and ksh93, or any shell that understands process substitutions):



                          $ date -f <( sed 's/^([^ ]*).*$/@1/' data.in )
                          Sun Jan 22 10:12:26 CET 2006
                          Sun Jan 22 10:12:38 CET 2006
                          Sun Jan 22 10:22:38 CET 2006
                          Sun Jan 22 10:23:22 CET 2006


                          Then we need to take the other fields of the input data and replace the delimiters:



                          $ cut -d ' ' -f 2- data.in | tr ' ' '|'
                          180900|61.153.158.197|1409
                          181622|61.153.158.197|1409
                          180026|221.226.124.114|1374
                          179485|121.13.128.132|1409


                          Then we paste these two things together with a | as delimiter:



                          $ paste -d '|' <( date -f <( sed 's/^([^ ]*).*$/@1/' data.in ) ) <( cut -d ' ' -f 2- data.in | tr ' ' '|' )
                          Sun Jan 22 10:12:26 CET 2006|180900|61.153.158.197|1409
                          Sun Jan 22 10:12:38 CET 2006|181622|61.153.158.197|1409
                          Sun Jan 22 10:22:38 CET 2006|180026|221.226.124.114|1374
                          Sun Jan 22 10:23:22 CET 2006|179485|121.13.128.132|1409





                          share|improve this answer






















                            up vote
                            1
                            down vote










                            up vote
                            1
                            down vote









                            Using what you already know:



                            • GNU date can convert a timestamp to a formatted date by giving it @timestamp.

                            • Replacing spaces by | will give you the output you want.

                            To that, we add



                            • GNU date may operate on a file, converting dates in one batch.

                            To batch convert dates with GNU date we have to extract the timestamps and prefix them with @:



                            $ sed 's/^([^ ]*).*$/@1/' data.in
                            @1137921146.499
                            @1137921158.698
                            @1137921758.163
                            @1137921802.016


                            The sed expression substitutes each line with the first space-delimited field prefixed with @.



                            With bash (and ksh93, or any shell that understands process substitutions):



                            $ date -f <( sed 's/^([^ ]*).*$/@1/' data.in )
                            Sun Jan 22 10:12:26 CET 2006
                            Sun Jan 22 10:12:38 CET 2006
                            Sun Jan 22 10:22:38 CET 2006
                            Sun Jan 22 10:23:22 CET 2006


                            Then we need to take the other fields of the input data and replace the delimiters:



                            $ cut -d ' ' -f 2- data.in | tr ' ' '|'
                            180900|61.153.158.197|1409
                            181622|61.153.158.197|1409
                            180026|221.226.124.114|1374
                            179485|121.13.128.132|1409


                            Then we paste these two things together with a | as delimiter:



                            $ paste -d '|' <( date -f <( sed 's/^([^ ]*).*$/@1/' data.in ) ) <( cut -d ' ' -f 2- data.in | tr ' ' '|' )
                            Sun Jan 22 10:12:26 CET 2006|180900|61.153.158.197|1409
                            Sun Jan 22 10:12:38 CET 2006|181622|61.153.158.197|1409
                            Sun Jan 22 10:22:38 CET 2006|180026|221.226.124.114|1374
                            Sun Jan 22 10:23:22 CET 2006|179485|121.13.128.132|1409





                            share|improve this answer












                            Using what you already know:



                            • GNU date can convert a timestamp to a formatted date by giving it @timestamp.

                            • Replacing spaces by | will give you the output you want.

                            To that, we add



                            • GNU date may operate on a file, converting dates in one batch.

                            To batch convert dates with GNU date we have to extract the timestamps and prefix them with @:



                            $ sed 's/^([^ ]*).*$/@1/' data.in
                            @1137921146.499
                            @1137921158.698
                            @1137921758.163
                            @1137921802.016


                            The sed expression substitutes each line with the first space-delimited field prefixed with @.



                            With bash (and ksh93, or any shell that understands process substitutions):



                            $ date -f <( sed 's/^([^ ]*).*$/@1/' data.in )
                            Sun Jan 22 10:12:26 CET 2006
                            Sun Jan 22 10:12:38 CET 2006
                            Sun Jan 22 10:22:38 CET 2006
                            Sun Jan 22 10:23:22 CET 2006


                            Then we need to take the other fields of the input data and replace the delimiters:



                            $ cut -d ' ' -f 2- data.in | tr ' ' '|'
                            180900|61.153.158.197|1409
                            181622|61.153.158.197|1409
                            180026|221.226.124.114|1374
                            179485|121.13.128.132|1409


                            Then we paste these two things together with a | as delimiter:



                            $ paste -d '|' <( date -f <( sed 's/^([^ ]*).*$/@1/' data.in ) ) <( cut -d ' ' -f 2- data.in | tr ' ' '|' )
                            Sun Jan 22 10:12:26 CET 2006|180900|61.153.158.197|1409
                            Sun Jan 22 10:12:38 CET 2006|181622|61.153.158.197|1409
                            Sun Jan 22 10:22:38 CET 2006|180026|221.226.124.114|1374
                            Sun Jan 22 10:23:22 CET 2006|179485|121.13.128.132|1409






                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered May 23 '17 at 7:55









                            Kusalananda

                            116k15218352




                            116k15218352



























                                 

                                draft saved


                                draft discarded















































                                 


                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f366706%2fconverting-unix-epoch-timestamp-column-in-every-row-using-sed-or-awk%23new-answer', 'question_page');

                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown






                                Popular posts from this blog

                                How to check contact read email or not when send email to Individual?

                                Bahrain

                                Postfix configuration issue with fips on centos 7; mailgun relay