Can't see the contents of a data file in terminal
Clash Royale CLAN TAG#URR8PPP
I have downloaded a number of input files from the NASDAQ website (for example ftp://emi.nasdaq.com/ITCH/01302018.NASDAQ_ITCH50.gz) After decompressing (I double click on a Mac) I can't see the file contents using head or tail or cat. I only see gibberish characters. I have tried downloading each file three times or so. Are there any settings in the terminal that I should change (it's my first time doing this on this machine).
Also I tried checking their MD5 with what is provided on the website. It appears they don't match. What can be done in this case? Again, I downloaded them multiple times to account for the possibility of bits being dropped, etc.
I assume popular website like that will not upload corrupt data files, right?
Could you guide me through what I need to do to see the contents of the files? I have never faced this problem before; I was always able to inspect a file by doing head or cat.
The files are in this folder: ftp://emi.nasdaq.com/ITCH/
terminal osx cat character-encoding
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
add a comment |
I have downloaded a number of input files from the NASDAQ website (for example ftp://emi.nasdaq.com/ITCH/01302018.NASDAQ_ITCH50.gz) After decompressing (I double click on a Mac) I can't see the file contents using head or tail or cat. I only see gibberish characters. I have tried downloading each file three times or so. Are there any settings in the terminal that I should change (it's my first time doing this on this machine).
Also I tried checking their MD5 with what is provided on the website. It appears they don't match. What can be done in this case? Again, I downloaded them multiple times to account for the possibility of bits being dropped, etc.
I assume popular website like that will not upload corrupt data files, right?
Could you guide me through what I need to do to see the contents of the files? I have never faced this problem before; I was always able to inspect a file by doing head or cat.
The files are in this folder: ftp://emi.nasdaq.com/ITCH/
terminal osx cat character-encoding
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
1
just to be sure, you compared the md5sum of the initial, compressed download file?
– Jeff Schaller
Feb 23 at 1:51
No! I uncompressed the file first!
– Tapal Goosal
Feb 23 at 1:55
Then, assuming the MD5s match (they should or they would know immediately, right?), what can I do?
– Tapal Goosal
Feb 23 at 1:55
1
I noticed that the md5sum filename containedgz, so I assume they mean for you to compare to the .gz file.
– Jeff Schaller
Feb 23 at 1:56
add a comment |
I have downloaded a number of input files from the NASDAQ website (for example ftp://emi.nasdaq.com/ITCH/01302018.NASDAQ_ITCH50.gz) After decompressing (I double click on a Mac) I can't see the file contents using head or tail or cat. I only see gibberish characters. I have tried downloading each file three times or so. Are there any settings in the terminal that I should change (it's my first time doing this on this machine).
Also I tried checking their MD5 with what is provided on the website. It appears they don't match. What can be done in this case? Again, I downloaded them multiple times to account for the possibility of bits being dropped, etc.
I assume popular website like that will not upload corrupt data files, right?
Could you guide me through what I need to do to see the contents of the files? I have never faced this problem before; I was always able to inspect a file by doing head or cat.
The files are in this folder: ftp://emi.nasdaq.com/ITCH/
terminal osx cat character-encoding
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
I have downloaded a number of input files from the NASDAQ website (for example ftp://emi.nasdaq.com/ITCH/01302018.NASDAQ_ITCH50.gz) After decompressing (I double click on a Mac) I can't see the file contents using head or tail or cat. I only see gibberish characters. I have tried downloading each file three times or so. Are there any settings in the terminal that I should change (it's my first time doing this on this machine).
Also I tried checking their MD5 with what is provided on the website. It appears they don't match. What can be done in this case? Again, I downloaded them multiple times to account for the possibility of bits being dropped, etc.
I assume popular website like that will not upload corrupt data files, right?
Could you guide me through what I need to do to see the contents of the files? I have never faced this problem before; I was always able to inspect a file by doing head or cat.
The files are in this folder: ftp://emi.nasdaq.com/ITCH/
terminal osx cat character-encoding
terminal osx cat character-encoding
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
asked Feb 23 at 1:42
Tapal GoosalTapal Goosal
1
1
1
just to be sure, you compared the md5sum of the initial, compressed download file?
– Jeff Schaller
Feb 23 at 1:51
No! I uncompressed the file first!
– Tapal Goosal
Feb 23 at 1:55
Then, assuming the MD5s match (they should or they would know immediately, right?), what can I do?
– Tapal Goosal
Feb 23 at 1:55
1
I noticed that the md5sum filename containedgz, so I assume they mean for you to compare to the .gz file.
– Jeff Schaller
Feb 23 at 1:56
add a comment |
1
just to be sure, you compared the md5sum of the initial, compressed download file?
– Jeff Schaller
Feb 23 at 1:51
No! I uncompressed the file first!
– Tapal Goosal
Feb 23 at 1:55
Then, assuming the MD5s match (they should or they would know immediately, right?), what can I do?
– Tapal Goosal
Feb 23 at 1:55
1
I noticed that the md5sum filename containedgz, so I assume they mean for you to compare to the .gz file.
– Jeff Schaller
Feb 23 at 1:56
1
1
just to be sure, you compared the md5sum of the initial, compressed download file?
– Jeff Schaller
Feb 23 at 1:51
just to be sure, you compared the md5sum of the initial, compressed download file?
– Jeff Schaller
Feb 23 at 1:51
No! I uncompressed the file first!
– Tapal Goosal
Feb 23 at 1:55
No! I uncompressed the file first!
– Tapal Goosal
Feb 23 at 1:55
Then, assuming the MD5s match (they should or they would know immediately, right?), what can I do?
– Tapal Goosal
Feb 23 at 1:55
Then, assuming the MD5s match (they should or they would know immediately, right?), what can I do?
– Tapal Goosal
Feb 23 at 1:55
1
1
I noticed that the md5sum filename contained
gz, so I assume they mean for you to compare to the .gz file.– Jeff Schaller
Feb 23 at 1:56
I noticed that the md5sum filename contained
gz, so I assume they mean for you to compare to the .gz file.– Jeff Schaller
Feb 23 at 1:56
add a comment |
1 Answer
1
active
oldest
votes
Your file contains a NASDAQ TotalView-ITCH Data Feed, as described in Data News #2008 - 91.
The format used by the file is not a pure text format but a binary encoded format.
This binary format is described in detail in the PDF document NASDAQ TotalView-ITCH 4.0. The document NASDAQ TotalView-ITCH 3.1 describes a text-based encoding format.
This is just as a way of allowing you to get further with these files. There is probably software out there that will read these highly domain specific data files. The data provider (NASDAQ) may possibly reference this in their documentation.
The MD5 checksums should be checked against the original, compressed files. Not the uncompressed files (or even re-compressed files).
In general, when a data provider gives you a MD5 checksum for a data file, it will be the checksum of the file that they give to you, in the format that they give it to you.
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f502438%2fcant-see-the-contents-of-a-data-file-in-terminal%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your file contains a NASDAQ TotalView-ITCH Data Feed, as described in Data News #2008 - 91.
The format used by the file is not a pure text format but a binary encoded format.
This binary format is described in detail in the PDF document NASDAQ TotalView-ITCH 4.0. The document NASDAQ TotalView-ITCH 3.1 describes a text-based encoding format.
This is just as a way of allowing you to get further with these files. There is probably software out there that will read these highly domain specific data files. The data provider (NASDAQ) may possibly reference this in their documentation.
The MD5 checksums should be checked against the original, compressed files. Not the uncompressed files (or even re-compressed files).
In general, when a data provider gives you a MD5 checksum for a data file, it will be the checksum of the file that they give to you, in the format that they give it to you.
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
add a comment |
Your file contains a NASDAQ TotalView-ITCH Data Feed, as described in Data News #2008 - 91.
The format used by the file is not a pure text format but a binary encoded format.
This binary format is described in detail in the PDF document NASDAQ TotalView-ITCH 4.0. The document NASDAQ TotalView-ITCH 3.1 describes a text-based encoding format.
This is just as a way of allowing you to get further with these files. There is probably software out there that will read these highly domain specific data files. The data provider (NASDAQ) may possibly reference this in their documentation.
The MD5 checksums should be checked against the original, compressed files. Not the uncompressed files (or even re-compressed files).
In general, when a data provider gives you a MD5 checksum for a data file, it will be the checksum of the file that they give to you, in the format that they give it to you.
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
add a comment |
Your file contains a NASDAQ TotalView-ITCH Data Feed, as described in Data News #2008 - 91.
The format used by the file is not a pure text format but a binary encoded format.
This binary format is described in detail in the PDF document NASDAQ TotalView-ITCH 4.0. The document NASDAQ TotalView-ITCH 3.1 describes a text-based encoding format.
This is just as a way of allowing you to get further with these files. There is probably software out there that will read these highly domain specific data files. The data provider (NASDAQ) may possibly reference this in their documentation.
The MD5 checksums should be checked against the original, compressed files. Not the uncompressed files (or even re-compressed files).
In general, when a data provider gives you a MD5 checksum for a data file, it will be the checksum of the file that they give to you, in the format that they give it to you.
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
Your file contains a NASDAQ TotalView-ITCH Data Feed, as described in Data News #2008 - 91.
The format used by the file is not a pure text format but a binary encoded format.
This binary format is described in detail in the PDF document NASDAQ TotalView-ITCH 4.0. The document NASDAQ TotalView-ITCH 3.1 describes a text-based encoding format.
This is just as a way of allowing you to get further with these files. There is probably software out there that will read these highly domain specific data files. The data provider (NASDAQ) may possibly reference this in their documentation.
The MD5 checksums should be checked against the original, compressed files. Not the uncompressed files (or even re-compressed files).
In general, when a data provider gives you a MD5 checksum for a data file, it will be the checksum of the file that they give to you, in the format that they give it to you.
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
edited Feb 23 at 9:17
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
answered Feb 23 at 9:11
KusalanandaKusalananda
136k17257426
136k17257426
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f502438%2fcant-see-the-contents-of-a-data-file-in-terminal%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
just to be sure, you compared the md5sum of the initial, compressed download file?
– Jeff Schaller
Feb 23 at 1:51
No! I uncompressed the file first!
– Tapal Goosal
Feb 23 at 1:55
Then, assuming the MD5s match (they should or they would know immediately, right?), what can I do?
– Tapal Goosal
Feb 23 at 1:55
1
I noticed that the md5sum filename contained
gz, so I assume they mean for you to compare to the .gz file.– Jeff Schaller
Feb 23 at 1:56