Downloading files using wget

up vote
15
down vote

favorite

I am trying to download files from this website.

The URL is: http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

When I use this command:

wget http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

I get only index.html?acc=GSE48191 which is some kind of binary format.

How can I download the files from this HTTP site?

edited 3 mins ago

Aaron Franke

3611416

asked Jul 22 '14 at 16:31

user3138373

85541430

add a commentÂ |Â

up vote
15
down vote

favorite

I am trying to download files from this website.

The URL is: http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

When I use this command:

wget http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

I get only index.html?acc=GSE48191 which is some kind of binary format.

How can I download the files from this HTTP site?

edited 3 mins ago

Aaron Franke

3611416

asked Jul 22 '14 at 16:31

user3138373

85541430

add a commentÂ |Â

up vote
15
down vote

favorite

I am trying to download files from this website.

The URL is: http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

When I use this command:

wget http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

I get only index.html?acc=GSE48191 which is some kind of binary format.

How can I download the files from this HTTP site?

edited 3 mins ago

Aaron Franke

3611416

asked Jul 22 '14 at 16:31

user3138373

85541430

I am trying to download files from this website.

The URL is: http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

When I use this command:

wget http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

I get only index.html?acc=GSE48191 which is some kind of binary format.

How can I download the files from this HTTP site?

wget

edited 3 mins ago

Aaron Franke

3611416

asked Jul 22 '14 at 16:31

user3138373

85541430

edited 3 mins ago

Aaron Franke

3611416

asked Jul 22 '14 at 16:31

user3138373

85541430

edited 3 mins ago

Aaron Franke

3611416

edited 3 mins ago

Aaron Franke

3611416

edited 3 mins ago

Aaron Franke

3611416

asked Jul 22 '14 at 16:31

user3138373

85541430

asked Jul 22 '14 at 16:31

user3138373

85541430

asked Jul 22 '14 at 16:31

user3138373

85541430

add a commentÂ |Â

3 Answers
3

active

oldest

votes

up vote
21
down vote

I think your ? gets interpreted by shell (Correction by vinc17: more likely, it's the & which gets interpreted).

Just try with simple quotes around your URL:

wget 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

Note that the file you are requesting is a .tar file but the above command will save it as index.html?acc=GSE48191&format=file. To have it correctly named, you can either rename it to .tar:

mv 'index.html?acc=GSE48191&format=file' GSE4819.tar

Or you can give the name as an option to wget:

wget -O GSE48191.tar 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

The above command will save the downloaded file as GSE48191.tar directly.

edited Jul 22 '14 at 17:17

answered Jul 22 '14 at 16:46

Qeole

454510

It gets downloaded but it is not even a directory. If you look at the link ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE48191 , you can see there are multiple .gz files. I still can't access them??
â€“Â user3138373
Jul 22 '14 at 16:57

I suppose that the OP uses a shell that ignores ? as a wildcard since nothing matches. The main problem is &: this will run the part that precedes (thus with an incomplete URL) in the background. But the solution is the same: to quote the URL.
â€“Â vinc17
Jul 22 '14 at 17:07

Thanks to you terdon and vinc for edit/corrections. @user3138373: I can't find your .gz files on provided links, could you please tell again what URL you use to see/access them?
â€“Â Qeole
Jul 22 '14 at 17:10

1

@user3138373 the file you download is an archive (.tar file) that contains the .gz files. Once you have downloaded it, run tar xvf GSE4819.tar to expand the archive and access the files.
â€“Â terdonâ™¦
Jul 22 '14 at 17:25

add a commentÂ |Â

up vote
3
down vote

Another way that might possibly work is by using this command:

wget -O nameOfTar.tar "http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file"

The -O command will specify the name to download to.

Of course, your initial problem is because the "&" was being interpreted by the shell, surrounding the URL with double quotes fixes the issue.

edited Jul 22 '14 at 22:07

user67289

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

2

-O option is used to specify the name of the file in which dowloaded data is saved. It has no incidence on downloaded data (maybe that's what you meant, but I found it unclear).
â€“Â Qeole
Jul 22 '14 at 17:16

Yes sorry, I will make my correction
â€“Â ryekayo
Jul 22 '14 at 17:17

I'm not sure why this got downvoted.
â€“Â ryekayo
Jul 22 '14 at 17:51

3

I did not downvote, but that's probably because your solution does not fix problem: & is interpreted by shell, and download of .tar file will fail.
â€“Â Qeole
Jul 22 '14 at 17:54

add a commentÂ |Â

up vote
0
down vote

wget -O "name-you-want-to-save-as.format" http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

That should get you the file you want to download to the current directory you are in.

edited Jul 22 '15 at 4:43

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

wget: missing URL is what wget replies to that, because you are missing the argument to -O. Also, I think this probably doesn't solve the OP's problem anyway.
â€“Â Celada
Jul 19 '15 at 18:00

Because the URL contains &, this answer doesn't work unless you add "" or '' around the URL.
â€“Â Aaron Franke
Jan 8 at 2:33

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f145946%2fdownloading-files-using-wget%23new-answer', 'question_page');

);

Post as a guest

Name

3 Answers
3

active

oldest

votes

3 Answers
3

active

oldest

votes

up vote
21
down vote

I think your ? gets interpreted by shell (Correction by vinc17: more likely, it's the & which gets interpreted).

Just try with simple quotes around your URL:

wget 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

Note that the file you are requesting is a .tar file but the above command will save it as index.html?acc=GSE48191&format=file. To have it correctly named, you can either rename it to .tar:

mv 'index.html?acc=GSE48191&format=file' GSE4819.tar

Or you can give the name as an option to wget:

wget -O GSE48191.tar 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

The above command will save the downloaded file as GSE48191.tar directly.

edited Jul 22 '14 at 17:17

answered Jul 22 '14 at 16:46

Qeole

454510

It gets downloaded but it is not even a directory. If you look at the link ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE48191 , you can see there are multiple .gz files. I still can't access them??
â€“Â user3138373
Jul 22 '14 at 16:57

I suppose that the OP uses a shell that ignores ? as a wildcard since nothing matches. The main problem is &: this will run the part that precedes (thus with an incomplete URL) in the background. But the solution is the same: to quote the URL.
â€“Â vinc17
Jul 22 '14 at 17:07

Thanks to you terdon and vinc for edit/corrections. @user3138373: I can't find your .gz files on provided links, could you please tell again what URL you use to see/access them?
â€“Â Qeole
Jul 22 '14 at 17:10

1

@user3138373 the file you download is an archive (.tar file) that contains the .gz files. Once you have downloaded it, run tar xvf GSE4819.tar to expand the archive and access the files.
â€“Â terdonâ™¦
Jul 22 '14 at 17:25

add a commentÂ |Â

up vote
21
down vote

I think your ? gets interpreted by shell (Correction by vinc17: more likely, it's the & which gets interpreted).

Just try with simple quotes around your URL:

wget 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

Note that the file you are requesting is a .tar file but the above command will save it as index.html?acc=GSE48191&format=file. To have it correctly named, you can either rename it to .tar:

mv 'index.html?acc=GSE48191&format=file' GSE4819.tar

Or you can give the name as an option to wget:

wget -O GSE48191.tar 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

The above command will save the downloaded file as GSE48191.tar directly.

edited Jul 22 '14 at 17:17

answered Jul 22 '14 at 16:46

Qeole

454510

It gets downloaded but it is not even a directory. If you look at the link ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE48191 , you can see there are multiple .gz files. I still can't access them??
â€“Â user3138373
Jul 22 '14 at 16:57

I suppose that the OP uses a shell that ignores ? as a wildcard since nothing matches. The main problem is &: this will run the part that precedes (thus with an incomplete URL) in the background. But the solution is the same: to quote the URL.
â€“Â vinc17
Jul 22 '14 at 17:07

Thanks to you terdon and vinc for edit/corrections. @user3138373: I can't find your .gz files on provided links, could you please tell again what URL you use to see/access them?
â€“Â Qeole
Jul 22 '14 at 17:10

1

@user3138373 the file you download is an archive (.tar file) that contains the .gz files. Once you have downloaded it, run tar xvf GSE4819.tar to expand the archive and access the files.
â€“Â terdonâ™¦
Jul 22 '14 at 17:25

add a commentÂ |Â

up vote
21
down vote

I think your ? gets interpreted by shell (Correction by vinc17: more likely, it's the & which gets interpreted).

Just try with simple quotes around your URL:

wget 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

Note that the file you are requesting is a .tar file but the above command will save it as index.html?acc=GSE48191&format=file. To have it correctly named, you can either rename it to .tar:

mv 'index.html?acc=GSE48191&format=file' GSE4819.tar

Or you can give the name as an option to wget:

wget -O GSE48191.tar 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

The above command will save the downloaded file as GSE48191.tar directly.

edited Jul 22 '14 at 17:17

answered Jul 22 '14 at 16:46

Qeole

454510

I think your ? gets interpreted by shell (Correction by vinc17: more likely, it's the & which gets interpreted).

Just try with simple quotes around your URL:

wget 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

Note that the file you are requesting is a .tar file but the above command will save it as index.html?acc=GSE48191&format=file. To have it correctly named, you can either rename it to .tar:

mv 'index.html?acc=GSE48191&format=file' GSE4819.tar

Or you can give the name as an option to wget:

wget -O GSE48191.tar 'http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file'

The above command will save the downloaded file as GSE48191.tar directly.

edited Jul 22 '14 at 17:17

answered Jul 22 '14 at 16:46

Qeole

454510

edited Jul 22 '14 at 17:17

answered Jul 22 '14 at 16:46

Qeole

454510

answered Jul 22 '14 at 16:46

Qeole

454510

answered Jul 22 '14 at 16:46

Qeole

454510

It gets downloaded but it is not even a directory. If you look at the link ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE48191 , you can see there are multiple .gz files. I still can't access them??
â€“Â user3138373
Jul 22 '14 at 16:57

I suppose that the OP uses a shell that ignores ? as a wildcard since nothing matches. The main problem is &: this will run the part that precedes (thus with an incomplete URL) in the background. But the solution is the same: to quote the URL.
â€“Â vinc17
Jul 22 '14 at 17:07

Thanks to you terdon and vinc for edit/corrections. @user3138373: I can't find your .gz files on provided links, could you please tell again what URL you use to see/access them?
â€“Â Qeole
Jul 22 '14 at 17:10

1

@user3138373 the file you download is an archive (.tar file) that contains the .gz files. Once you have downloaded it, run tar xvf GSE4819.tar to expand the archive and access the files.
â€“Â terdonâ™¦
Jul 22 '14 at 17:25

add a commentÂ |Â

It gets downloaded but it is not even a directory. If you look at the link ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE48191 , you can see there are multiple .gz files. I still can't access them??
â€“Â user3138373
Jul 22 '14 at 16:57

I suppose that the OP uses a shell that ignores ? as a wildcard since nothing matches. The main problem is &: this will run the part that precedes (thus with an incomplete URL) in the background. But the solution is the same: to quote the URL.
â€“Â vinc17
Jul 22 '14 at 17:07

Thanks to you terdon and vinc for edit/corrections. @user3138373: I can't find your .gz files on provided links, could you please tell again what URL you use to see/access them?
â€“Â Qeole
Jul 22 '14 at 17:10

1

@user3138373 the file you download is an archive (.tar file) that contains the .gz files. Once you have downloaded it, run tar xvf GSE4819.tar to expand the archive and access the files.
â€“Â terdonâ™¦
Jul 22 '14 at 17:25

It gets downloaded but it is not even a directory. If you look at the link ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE48191 , you can see there are multiple .gz files. I still can't access them??
â€“Â user3138373
Jul 22 '14 at 16:57

I suppose that the OP uses a shell that ignores ? as a wildcard since nothing matches. The main problem is &: this will run the part that precedes (thus with an incomplete URL) in the background. But the solution is the same: to quote the URL.
â€“Â vinc17
Jul 22 '14 at 17:07

Thanks to you terdon and vinc for edit/corrections. @user3138373: I can't find your .gz files on provided links, could you please tell again what URL you use to see/access them?
â€“Â Qeole
Jul 22 '14 at 17:10

@user3138373 the file you download is an archive (.tar file) that contains the .gz files. Once you have downloaded it, run tar xvf GSE4819.tar to expand the archive and access the files.
â€“Â terdonâ™¦
Jul 22 '14 at 17:25

add a commentÂ |Â

up vote
3
down vote

Another way that might possibly work is by using this command:

wget -O nameOfTar.tar "http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file"

The -O command will specify the name to download to.

Of course, your initial problem is because the "&" was being interpreted by the shell, surrounding the URL with double quotes fixes the issue.

edited Jul 22 '14 at 22:07

user67289

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

2

-O option is used to specify the name of the file in which dowloaded data is saved. It has no incidence on downloaded data (maybe that's what you meant, but I found it unclear).
â€“Â Qeole
Jul 22 '14 at 17:16

Yes sorry, I will make my correction
â€“Â ryekayo
Jul 22 '14 at 17:17

I'm not sure why this got downvoted.
â€“Â ryekayo
Jul 22 '14 at 17:51

3

I did not downvote, but that's probably because your solution does not fix problem: & is interpreted by shell, and download of .tar file will fail.
â€“Â Qeole
Jul 22 '14 at 17:54

add a commentÂ |Â

up vote
3
down vote

Another way that might possibly work is by using this command:

wget -O nameOfTar.tar "http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file"

The -O command will specify the name to download to.

Of course, your initial problem is because the "&" was being interpreted by the shell, surrounding the URL with double quotes fixes the issue.

edited Jul 22 '14 at 22:07

user67289

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

2

-O option is used to specify the name of the file in which dowloaded data is saved. It has no incidence on downloaded data (maybe that's what you meant, but I found it unclear).
â€“Â Qeole
Jul 22 '14 at 17:16

Yes sorry, I will make my correction
â€“Â ryekayo
Jul 22 '14 at 17:17

I'm not sure why this got downvoted.
â€“Â ryekayo
Jul 22 '14 at 17:51

3

I did not downvote, but that's probably because your solution does not fix problem: & is interpreted by shell, and download of .tar file will fail.
â€“Â Qeole
Jul 22 '14 at 17:54

add a commentÂ |Â

up vote
3
down vote

Another way that might possibly work is by using this command:

wget -O nameOfTar.tar "http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file"

The -O command will specify the name to download to.

Of course, your initial problem is because the "&" was being interpreted by the shell, surrounding the URL with double quotes fixes the issue.

edited Jul 22 '14 at 22:07

user67289

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

Another way that might possibly work is by using this command:

wget -O nameOfTar.tar "http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file"

The -O command will specify the name to download to.

Of course, your initial problem is because the "&" was being interpreted by the shell, surrounding the URL with double quotes fixes the issue.

edited Jul 22 '14 at 22:07

user67289

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

edited Jul 22 '14 at 22:07

user67289

edited Jul 22 '14 at 22:07

user67289

edited Jul 22 '14 at 22:07

user67289

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

answered Jul 22 '14 at 17:02

ryekayo

2,81582448

2

-O option is used to specify the name of the file in which dowloaded data is saved. It has no incidence on downloaded data (maybe that's what you meant, but I found it unclear).
â€“Â Qeole
Jul 22 '14 at 17:16

Yes sorry, I will make my correction
â€“Â ryekayo
Jul 22 '14 at 17:17

I'm not sure why this got downvoted.
â€“Â ryekayo
Jul 22 '14 at 17:51

3

I did not downvote, but that's probably because your solution does not fix problem: & is interpreted by shell, and download of .tar file will fail.
â€“Â Qeole
Jul 22 '14 at 17:54

add a commentÂ |Â

2

-O option is used to specify the name of the file in which dowloaded data is saved. It has no incidence on downloaded data (maybe that's what you meant, but I found it unclear).
â€“Â Qeole
Jul 22 '14 at 17:16

Yes sorry, I will make my correction
â€“Â ryekayo
Jul 22 '14 at 17:17

I'm not sure why this got downvoted.
â€“Â ryekayo
Jul 22 '14 at 17:51

3

I did not downvote, but that's probably because your solution does not fix problem: & is interpreted by shell, and download of .tar file will fail.
â€“Â Qeole
Jul 22 '14 at 17:54

-O option is used to specify the name of the file in which dowloaded data is saved. It has no incidence on downloaded data (maybe that's what you meant, but I found it unclear).
â€“Â Qeole
Jul 22 '14 at 17:16

Yes sorry, I will make my correction
â€“Â ryekayo
Jul 22 '14 at 17:17

I'm not sure why this got downvoted.
â€“Â ryekayo
Jul 22 '14 at 17:51

I did not downvote, but that's probably because your solution does not fix problem: & is interpreted by shell, and download of .tar file will fail.
â€“Â Qeole
Jul 22 '14 at 17:54

add a commentÂ |Â

up vote
0
down vote

wget -O "name-you-want-to-save-as.format" http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

That should get you the file you want to download to the current directory you are in.

edited Jul 22 '15 at 4:43

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

wget: missing URL is what wget replies to that, because you are missing the argument to -O. Also, I think this probably doesn't solve the OP's problem anyway.
â€“Â Celada
Jul 19 '15 at 18:00

Because the URL contains &, this answer doesn't work unless you add "" or '' around the URL.
â€“Â Aaron Franke
Jan 8 at 2:33

add a commentÂ |Â

up vote
0
down vote

wget -O "name-you-want-to-save-as.format" http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

That should get you the file you want to download to the current directory you are in.

edited Jul 22 '15 at 4:43

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

wget: missing URL is what wget replies to that, because you are missing the argument to -O. Also, I think this probably doesn't solve the OP's problem anyway.
â€“Â Celada
Jul 19 '15 at 18:00

Because the URL contains &, this answer doesn't work unless you add "" or '' around the URL.
â€“Â Aaron Franke
Jan 8 at 2:33

add a commentÂ |Â

up vote
0
down vote

wget -O "name-you-want-to-save-as.format" http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

That should get you the file you want to download to the current directory you are in.

edited Jul 22 '15 at 4:43

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

wget -O "name-you-want-to-save-as.format" http://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE48191&format=file

That should get you the file you want to download to the current directory you are in.

edited Jul 22 '15 at 4:43

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

edited Jul 22 '15 at 4:43

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

answered Jul 19 '15 at 17:39

Samman Bikram Thapa

1113

wget: missing URL is what wget replies to that, because you are missing the argument to -O. Also, I think this probably doesn't solve the OP's problem anyway.
â€“Â Celada
Jul 19 '15 at 18:00

Because the URL contains &, this answer doesn't work unless you add "" or '' around the URL.
â€“Â Aaron Franke
Jan 8 at 2:33

add a commentÂ |Â

wget: missing URL is what wget replies to that, because you are missing the argument to -O. Also, I think this probably doesn't solve the OP's problem anyway.
â€“Â Celada
Jul 19 '15 at 18:00

Because the URL contains &, this answer doesn't work unless you add "" or '' around the URL.
â€“Â Aaron Franke
Jan 8 at 2:33

wget: missing URL is what wget replies to that, because you are missing the argument to -O. Also, I think this probably doesn't solve the OP's problem anyway.
â€“Â Celada
Jul 19 '15 at 18:00

Because the URL contains &, this answer doesn't work unless you add "" or '' around the URL.
â€“Â Aaron Franke
Jan 8 at 2:33

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Post as a guest

Name

搜尋此網誌

mjhjmtu