Bash convert xC3x89 to É?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
3
down vote

favorite












I'm running iwlist wlo1 scan | grep ESSID inside a script.



It displays French characters in the following format



xC3x89 for É and xC3xA9 for é.



I'm not sure what this format is called. I tried using an answer for converting unicode echo -ne 'xC3xA9' | iconv -f utf-16be but it converted to 쎩.



What is the official name for this format and how can I convert it in bash?







share|improve this question




















  • echo -ne "xC3x89" gives É
    – RomanPerekhrest
    Oct 28 '17 at 14:10










  • @RomanPerekhrest thank you, that fixes my issue.
    – Philip Kirkbride
    Oct 28 '17 at 14:12










  • @RomanPerekhrest do you want to post as an answer?
    – Philip Kirkbride
    Oct 28 '17 at 14:14










  • Philip Kirkbride, ok, posted ...
    – RomanPerekhrest
    Oct 28 '17 at 14:39















up vote
3
down vote

favorite












I'm running iwlist wlo1 scan | grep ESSID inside a script.



It displays French characters in the following format



xC3x89 for É and xC3xA9 for é.



I'm not sure what this format is called. I tried using an answer for converting unicode echo -ne 'xC3xA9' | iconv -f utf-16be but it converted to 쎩.



What is the official name for this format and how can I convert it in bash?







share|improve this question




















  • echo -ne "xC3x89" gives É
    – RomanPerekhrest
    Oct 28 '17 at 14:10










  • @RomanPerekhrest thank you, that fixes my issue.
    – Philip Kirkbride
    Oct 28 '17 at 14:12










  • @RomanPerekhrest do you want to post as an answer?
    – Philip Kirkbride
    Oct 28 '17 at 14:14










  • Philip Kirkbride, ok, posted ...
    – RomanPerekhrest
    Oct 28 '17 at 14:39













up vote
3
down vote

favorite









up vote
3
down vote

favorite











I'm running iwlist wlo1 scan | grep ESSID inside a script.



It displays French characters in the following format



xC3x89 for É and xC3xA9 for é.



I'm not sure what this format is called. I tried using an answer for converting unicode echo -ne 'xC3xA9' | iconv -f utf-16be but it converted to 쎩.



What is the official name for this format and how can I convert it in bash?







share|improve this question












I'm running iwlist wlo1 scan | grep ESSID inside a script.



It displays French characters in the following format



xC3x89 for É and xC3xA9 for é.



I'm not sure what this format is called. I tried using an answer for converting unicode echo -ne 'xC3xA9' | iconv -f utf-16be but it converted to 쎩.



What is the official name for this format and how can I convert it in bash?









share|improve this question











share|improve this question




share|improve this question










asked Oct 28 '17 at 13:36









Philip Kirkbride

2,2922470




2,2922470











  • echo -ne "xC3x89" gives É
    – RomanPerekhrest
    Oct 28 '17 at 14:10










  • @RomanPerekhrest thank you, that fixes my issue.
    – Philip Kirkbride
    Oct 28 '17 at 14:12










  • @RomanPerekhrest do you want to post as an answer?
    – Philip Kirkbride
    Oct 28 '17 at 14:14










  • Philip Kirkbride, ok, posted ...
    – RomanPerekhrest
    Oct 28 '17 at 14:39

















  • echo -ne "xC3x89" gives É
    – RomanPerekhrest
    Oct 28 '17 at 14:10










  • @RomanPerekhrest thank you, that fixes my issue.
    – Philip Kirkbride
    Oct 28 '17 at 14:12










  • @RomanPerekhrest do you want to post as an answer?
    – Philip Kirkbride
    Oct 28 '17 at 14:14










  • Philip Kirkbride, ok, posted ...
    – RomanPerekhrest
    Oct 28 '17 at 14:39
















echo -ne "xC3x89" gives É
– RomanPerekhrest
Oct 28 '17 at 14:10




echo -ne "xC3x89" gives É
– RomanPerekhrest
Oct 28 '17 at 14:10












@RomanPerekhrest thank you, that fixes my issue.
– Philip Kirkbride
Oct 28 '17 at 14:12




@RomanPerekhrest thank you, that fixes my issue.
– Philip Kirkbride
Oct 28 '17 at 14:12












@RomanPerekhrest do you want to post as an answer?
– Philip Kirkbride
Oct 28 '17 at 14:14




@RomanPerekhrest do you want to post as an answer?
– Philip Kirkbride
Oct 28 '17 at 14:14












Philip Kirkbride, ok, posted ...
– RomanPerekhrest
Oct 28 '17 at 14:39





Philip Kirkbride, ok, posted ...
– RomanPerekhrest
Oct 28 '17 at 14:39











3 Answers
3






active

oldest

votes

















up vote
5
down vote



accepted










Hexdecimal numeric constants are usually represented with 0x prefix.
Character and string constants may express character codes in hexadecimal with the prefix x followed by two hex digits.



echo -ne 'xC3x89' should give you É.




  • -e - enable interpretation of backslash escapes(including xHH - byte with hexadecimal value HH (1 to 2 digits))


To deal with better portability use printf function:



printf "%b" 'xC3x89'
É





share|improve this answer






















  • echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
    – Charles Duffy
    Oct 28 '17 at 17:29











  • ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
    – Charles Duffy
    Oct 28 '17 at 17:38










  • @CharlesDuffy, you have my update
    – RomanPerekhrest
    Oct 28 '17 at 20:05

















up vote
3
down vote













Try



 % echo -n éÉ | hd
00000000 c3 a9 c3 89 |....|
00000004


where hd is the hexdump utility.



So your thing is just UTF-8 encoding (and your observed output xC3x89 is the hexadecimal representation of UTF-8). Today we have UTF-8 everywhere.



My environment has both LANG=en_US.UTF-8 and LC_ALL=en_US.UTF-8 (even if I am French).



Also,



echo 'xC3x89xC3xA9'


produces



Éé


as expected.



And so does



echo "\xC3\x89\xC3\xA9" | iconv -t utf-8


Read also locale(7).



Notice that some character encodings (e.g. old plain ASCII) don't have any é or É characters, and others (e.g. ISO-8859-1) encode them differently.






share|improve this answer






















  • When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
    – Philip Kirkbride
    Oct 28 '17 at 14:10






  • 1




    You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
    – Basile Starynkevitch
    Oct 28 '17 at 14:12


















up vote
1
down vote













iwlist displays all the bytes that don't correspond to ASCII non-control characters as the xHH where HH is the byte number in hexadecimal.



To undo that encoding, you can pipe the output of iwlist to



perl -pe 's/\x([da-f]2)/chr(hex($1))/gie'


Byte 0xc3 followed by 0xa9 make up the UTF-8 encoding of the é character.



If your locale also uses the UTF-8 encoding, then the output of perl will show that é character. If not (unlikely as UTF-8 is the norm), you can always pipe the output of perl to iconv -f utf-8 to convert it from UTF-8 to the character encoding used in your locale.






share|improve this answer




















    Your Answer







    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "106"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f401064%2fbash-convert-xc3-x89-to-%25c3%2589%23new-answer', 'question_page');

    );

    Post as a guest






























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    5
    down vote



    accepted










    Hexdecimal numeric constants are usually represented with 0x prefix.
    Character and string constants may express character codes in hexadecimal with the prefix x followed by two hex digits.



    echo -ne 'xC3x89' should give you É.




    • -e - enable interpretation of backslash escapes(including xHH - byte with hexadecimal value HH (1 to 2 digits))


    To deal with better portability use printf function:



    printf "%b" 'xC3x89'
    É





    share|improve this answer






















    • echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
      – Charles Duffy
      Oct 28 '17 at 17:29











    • ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
      – Charles Duffy
      Oct 28 '17 at 17:38










    • @CharlesDuffy, you have my update
      – RomanPerekhrest
      Oct 28 '17 at 20:05














    up vote
    5
    down vote



    accepted










    Hexdecimal numeric constants are usually represented with 0x prefix.
    Character and string constants may express character codes in hexadecimal with the prefix x followed by two hex digits.



    echo -ne 'xC3x89' should give you É.




    • -e - enable interpretation of backslash escapes(including xHH - byte with hexadecimal value HH (1 to 2 digits))


    To deal with better portability use printf function:



    printf "%b" 'xC3x89'
    É





    share|improve this answer






















    • echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
      – Charles Duffy
      Oct 28 '17 at 17:29











    • ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
      – Charles Duffy
      Oct 28 '17 at 17:38










    • @CharlesDuffy, you have my update
      – RomanPerekhrest
      Oct 28 '17 at 20:05












    up vote
    5
    down vote



    accepted







    up vote
    5
    down vote



    accepted






    Hexdecimal numeric constants are usually represented with 0x prefix.
    Character and string constants may express character codes in hexadecimal with the prefix x followed by two hex digits.



    echo -ne 'xC3x89' should give you É.




    • -e - enable interpretation of backslash escapes(including xHH - byte with hexadecimal value HH (1 to 2 digits))


    To deal with better portability use printf function:



    printf "%b" 'xC3x89'
    É





    share|improve this answer














    Hexdecimal numeric constants are usually represented with 0x prefix.
    Character and string constants may express character codes in hexadecimal with the prefix x followed by two hex digits.



    echo -ne 'xC3x89' should give you É.




    • -e - enable interpretation of backslash escapes(including xHH - byte with hexadecimal value HH (1 to 2 digits))


    To deal with better portability use printf function:



    printf "%b" 'xC3x89'
    É






    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Oct 28 '17 at 20:05

























    answered Oct 28 '17 at 14:38









    RomanPerekhrest

    22.5k12145




    22.5k12145











    • echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
      – Charles Duffy
      Oct 28 '17 at 17:29











    • ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
      – Charles Duffy
      Oct 28 '17 at 17:38










    • @CharlesDuffy, you have my update
      – RomanPerekhrest
      Oct 28 '17 at 20:05
















    • echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
      – Charles Duffy
      Oct 28 '17 at 17:29











    • ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
      – Charles Duffy
      Oct 28 '17 at 17:38










    • @CharlesDuffy, you have my update
      – RomanPerekhrest
      Oct 28 '17 at 20:05















    echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
    – Charles Duffy
    Oct 28 '17 at 17:29





    echo -e is nonportable -- even bash doesn't always offer it, since when both posix and xpg_echo options are enabled echo -e just prints -e on output, as standard compliance requires. Better to use printf '%bn' 'xC3x89'
    – Charles Duffy
    Oct 28 '17 at 17:29













    ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
    – Charles Duffy
    Oct 28 '17 at 17:38




    ...to demonstrate this, consider bash -c 'set -o posix; shopt -s xpg_echo; printf "%bn" "$1"; echo -e "$1"' _ 'xC3x89' (and note that xpg_echo can be set on-by-default at compile time).
    – Charles Duffy
    Oct 28 '17 at 17:38












    @CharlesDuffy, you have my update
    – RomanPerekhrest
    Oct 28 '17 at 20:05




    @CharlesDuffy, you have my update
    – RomanPerekhrest
    Oct 28 '17 at 20:05












    up vote
    3
    down vote













    Try



     % echo -n éÉ | hd
    00000000 c3 a9 c3 89 |....|
    00000004


    where hd is the hexdump utility.



    So your thing is just UTF-8 encoding (and your observed output xC3x89 is the hexadecimal representation of UTF-8). Today we have UTF-8 everywhere.



    My environment has both LANG=en_US.UTF-8 and LC_ALL=en_US.UTF-8 (even if I am French).



    Also,



    echo 'xC3x89xC3xA9'


    produces



    Éé


    as expected.



    And so does



    echo "\xC3\x89\xC3\xA9" | iconv -t utf-8


    Read also locale(7).



    Notice that some character encodings (e.g. old plain ASCII) don't have any é or É characters, and others (e.g. ISO-8859-1) encode them differently.






    share|improve this answer






















    • When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
      – Philip Kirkbride
      Oct 28 '17 at 14:10






    • 1




      You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
      – Basile Starynkevitch
      Oct 28 '17 at 14:12















    up vote
    3
    down vote













    Try



     % echo -n éÉ | hd
    00000000 c3 a9 c3 89 |....|
    00000004


    where hd is the hexdump utility.



    So your thing is just UTF-8 encoding (and your observed output xC3x89 is the hexadecimal representation of UTF-8). Today we have UTF-8 everywhere.



    My environment has both LANG=en_US.UTF-8 and LC_ALL=en_US.UTF-8 (even if I am French).



    Also,



    echo 'xC3x89xC3xA9'


    produces



    Éé


    as expected.



    And so does



    echo "\xC3\x89\xC3\xA9" | iconv -t utf-8


    Read also locale(7).



    Notice that some character encodings (e.g. old plain ASCII) don't have any é or É characters, and others (e.g. ISO-8859-1) encode them differently.






    share|improve this answer






















    • When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
      – Philip Kirkbride
      Oct 28 '17 at 14:10






    • 1




      You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
      – Basile Starynkevitch
      Oct 28 '17 at 14:12













    up vote
    3
    down vote










    up vote
    3
    down vote









    Try



     % echo -n éÉ | hd
    00000000 c3 a9 c3 89 |....|
    00000004


    where hd is the hexdump utility.



    So your thing is just UTF-8 encoding (and your observed output xC3x89 is the hexadecimal representation of UTF-8). Today we have UTF-8 everywhere.



    My environment has both LANG=en_US.UTF-8 and LC_ALL=en_US.UTF-8 (even if I am French).



    Also,



    echo 'xC3x89xC3xA9'


    produces



    Éé


    as expected.



    And so does



    echo "\xC3\x89\xC3\xA9" | iconv -t utf-8


    Read also locale(7).



    Notice that some character encodings (e.g. old plain ASCII) don't have any é or É characters, and others (e.g. ISO-8859-1) encode them differently.






    share|improve this answer














    Try



     % echo -n éÉ | hd
    00000000 c3 a9 c3 89 |....|
    00000004


    where hd is the hexdump utility.



    So your thing is just UTF-8 encoding (and your observed output xC3x89 is the hexadecimal representation of UTF-8). Today we have UTF-8 everywhere.



    My environment has both LANG=en_US.UTF-8 and LC_ALL=en_US.UTF-8 (even if I am French).



    Also,



    echo 'xC3x89xC3xA9'


    produces



    Éé


    as expected.



    And so does



    echo "\xC3\x89\xC3\xA9" | iconv -t utf-8


    Read also locale(7).



    Notice that some character encodings (e.g. old plain ASCII) don't have any é or É characters, and others (e.g. ISO-8859-1) encode them differently.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Oct 28 '17 at 14:43

























    answered Oct 28 '17 at 13:49









    Basile Starynkevitch

    7,9231940




    7,9231940











    • When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
      – Philip Kirkbride
      Oct 28 '17 at 14:10






    • 1




      You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
      – Basile Starynkevitch
      Oct 28 '17 at 14:12

















    • When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
      – Philip Kirkbride
      Oct 28 '17 at 14:10






    • 1




      You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
      – Basile Starynkevitch
      Oct 28 '17 at 14:12
















    When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
    – Philip Kirkbride
    Oct 28 '17 at 14:10




    When I run echo "\xC3\x89\xC3\xA9" | iconv -t utf-8 my output is the same as the input, same for normal echo. LANG=en_CA.UTF-8 and LC_ALL not set. I'm hoping to have something that will work on a variety of machines.
    – Philip Kirkbride
    Oct 28 '17 at 14:10




    1




    1




    You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
    – Basile Starynkevitch
    Oct 28 '17 at 14:12





    You can't display é on an ASCII system, because that letter does not exist in ASCII encoding.
    – Basile Starynkevitch
    Oct 28 '17 at 14:12











    up vote
    1
    down vote













    iwlist displays all the bytes that don't correspond to ASCII non-control characters as the xHH where HH is the byte number in hexadecimal.



    To undo that encoding, you can pipe the output of iwlist to



    perl -pe 's/\x([da-f]2)/chr(hex($1))/gie'


    Byte 0xc3 followed by 0xa9 make up the UTF-8 encoding of the é character.



    If your locale also uses the UTF-8 encoding, then the output of perl will show that é character. If not (unlikely as UTF-8 is the norm), you can always pipe the output of perl to iconv -f utf-8 to convert it from UTF-8 to the character encoding used in your locale.






    share|improve this answer
























      up vote
      1
      down vote













      iwlist displays all the bytes that don't correspond to ASCII non-control characters as the xHH where HH is the byte number in hexadecimal.



      To undo that encoding, you can pipe the output of iwlist to



      perl -pe 's/\x([da-f]2)/chr(hex($1))/gie'


      Byte 0xc3 followed by 0xa9 make up the UTF-8 encoding of the é character.



      If your locale also uses the UTF-8 encoding, then the output of perl will show that é character. If not (unlikely as UTF-8 is the norm), you can always pipe the output of perl to iconv -f utf-8 to convert it from UTF-8 to the character encoding used in your locale.






      share|improve this answer






















        up vote
        1
        down vote










        up vote
        1
        down vote









        iwlist displays all the bytes that don't correspond to ASCII non-control characters as the xHH where HH is the byte number in hexadecimal.



        To undo that encoding, you can pipe the output of iwlist to



        perl -pe 's/\x([da-f]2)/chr(hex($1))/gie'


        Byte 0xc3 followed by 0xa9 make up the UTF-8 encoding of the é character.



        If your locale also uses the UTF-8 encoding, then the output of perl will show that é character. If not (unlikely as UTF-8 is the norm), you can always pipe the output of perl to iconv -f utf-8 to convert it from UTF-8 to the character encoding used in your locale.






        share|improve this answer












        iwlist displays all the bytes that don't correspond to ASCII non-control characters as the xHH where HH is the byte number in hexadecimal.



        To undo that encoding, you can pipe the output of iwlist to



        perl -pe 's/\x([da-f]2)/chr(hex($1))/gie'


        Byte 0xc3 followed by 0xa9 make up the UTF-8 encoding of the é character.



        If your locale also uses the UTF-8 encoding, then the output of perl will show that é character. If not (unlikely as UTF-8 is the norm), you can always pipe the output of perl to iconv -f utf-8 to convert it from UTF-8 to the character encoding used in your locale.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Oct 28 '17 at 21:11









        Stéphane Chazelas

        283k53521855




        283k53521855



























             

            draft saved


            draft discarded















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f401064%2fbash-convert-xc3-x89-to-%25c3%2589%23new-answer', 'question_page');

            );

            Post as a guest













































































            Popular posts from this blog

            How to check contact read email or not when send email to Individual?

            Displaying single band from multi-band raster using QGIS

            How many registers does an x86_64 CPU actually have?