Infinite data for scp

up vote
1
down vote

favorite

I am trying to have an infinite scp between two hosts but of course there is no such file large enough for this.

I tried

scp -l 512 192.168.1.1:/dev/zero /dev/null

But scp says /dev/zero not a regular file.

I need a consistent traffic between two hosts so I can try something on my router/firewall and I really need it to run for a long time.

Any suggestions? It does not have to be scp but I need to be able to specify the speed.

Thanks

edited Jan 24 at 3:15

asked Jan 23 at 22:17

Ask and Learn

83731224

add a commentÂ |Â

up vote
1
down vote

favorite

I am trying to have an infinite scp between two hosts but of course there is no such file large enough for this.

I tried

scp -l 512 192.168.1.1:/dev/zero /dev/null

But scp says /dev/zero not a regular file.

I need a consistent traffic between two hosts so I can try something on my router/firewall and I really need it to run for a long time.

Any suggestions? It does not have to be scp but I need to be able to specify the speed.

Thanks

edited Jan 24 at 3:15

asked Jan 23 at 22:17

Ask and Learn

83731224

add a commentÂ |Â

up vote
1
down vote

favorite

I am trying to have an infinite scp between two hosts but of course there is no such file large enough for this.

I tried

scp -l 512 192.168.1.1:/dev/zero /dev/null

But scp says /dev/zero not a regular file.

I need a consistent traffic between two hosts so I can try something on my router/firewall and I really need it to run for a long time.

Any suggestions? It does not have to be scp but I need to be able to specify the speed.

Thanks

edited Jan 24 at 3:15

asked Jan 23 at 22:17

Ask and Learn

83731224

I am trying to have an infinite scp between two hosts but of course there is no such file large enough for this.

I tried

scp -l 512 192.168.1.1:/dev/zero /dev/null

But scp says /dev/zero not a regular file.

I need a consistent traffic between two hosts so I can try something on my router/firewall and I really need it to run for a long time.

Any suggestions? It does not have to be scp but I need to be able to specify the speed.

Thanks

edited Jan 24 at 3:15

asked Jan 23 at 22:17

Ask and Learn

83731224

edited Jan 24 at 3:15

asked Jan 23 at 22:17

Ask and Learn

83731224

asked Jan 23 at 22:17

Ask and Learn

83731224

asked Jan 23 at 22:17

Ask and Learn

83731224

add a commentÂ |Â

2 Answers
2

active

oldest

votes

up vote
4
down vote

accepted

The scp tool expects to copy a file. You can use ssh to transport an unending stream of bytes, and you can rate-limit with something like pv. The pertinent section of the man page for pv writes,

-L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A suffix of K, M, G, or T can be added to denote Kilobytes (*1024), Megabytes, and so on.

A suitable solution would be something like this, which rate-limits at approximately 10Mb/s (remember that 1MB/s is approximately 10Mb/s, after accounting for padding, network headers, etc.):

pv --rate-limit 1M </dev/zero | ssh user@example.net 'cat >/dev/null'

If you want bidirectional traffic flow, remove the quotes from the 'cat >/dev/null'.

answered Jan 23 at 22:39

roaima

39.7k545108

+1. you can also do this with scp itself if you use a modernish shell (e.g. bash, ksh, zsh. not ash or dash) that supports process substitution. scp <(pv ...) user@example.net:/dev/null
â€“Â cas
Jan 24 at 2:13

I tried this on CentOS 7 scp still compliants about not a regular file.
â€“Â Ask and Learn
Jan 24 at 3:11

so it does...I should have tested that before commenting. maybe there is some valid reason for scp to care whether a file is a regular file or not. or maybe it's just some over-zealous special-case handling. dunno.
â€“Â cas
Jan 24 at 6:17

@cas it introduces ambiguity, so it's blocked. Consider scp /dev/sdc3 remote:/dev/sdc3 where the destination does not exist. Should this copy the device node? What about its contents? Since sdc3 is missing on the target should we copy the contents to a file on the destination?
â€“Â roaima
Jan 24 at 8:04

1

rsync is different to ssh or scp. BTW, for live-migrating VMs, I zfs send the VM's ZVOL to my secondary VM host :), and that's usually very quick because it's only a small update since the most recent snapshot send. The DRBD idea in the strugglers.net article is nicely implemented in google's ganeti. I've got ganeti working with ZFS at home, but I don't really need ganeti here. libvirt plus my own scripts suffice. $workplaces that have paid me to do VM stuff tend to go for vmware, xen, openstack, or containers with docker. I quite like the latter 3.
â€“Â cas
Jan 24 at 10:14

Â |Â
show 3 more comments

up vote
0
down vote

from commandlinefu.com:
~~dd if=/dev/zero bs=4096 count=1048576 | ssh user@host.tld 'cat > /dev/null'~~

As a helpful user posted, this will stop at 4GiB, which is not infinite. Therefore:

</dev/zero ssh user@host.tld 'cat > /dev/null'

edited Jan 23 at 22:52

answered Jan 23 at 22:23

Xalorous

22118

4GB is not infinite. Why make this complicated? </dev/zero ssh user@host.tld 'cat > /dev/null'
â€“Â Gilles
Jan 23 at 22:49

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f419207%2finfinite-data-for-scp%23new-answer', 'question_page');

);

Post as a guest

Name

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

up vote
4
down vote

accepted

-L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A suffix of K, M, G, or T can be added to denote Kilobytes (*1024), Megabytes, and so on.

A suitable solution would be something like this, which rate-limits at approximately 10Mb/s (remember that 1MB/s is approximately 10Mb/s, after accounting for padding, network headers, etc.):

pv --rate-limit 1M </dev/zero | ssh user@example.net 'cat >/dev/null'

If you want bidirectional traffic flow, remove the quotes from the 'cat >/dev/null'.

answered Jan 23 at 22:39

roaima

39.7k545108

+1. you can also do this with scp itself if you use a modernish shell (e.g. bash, ksh, zsh. not ash or dash) that supports process substitution. scp <(pv ...) user@example.net:/dev/null
â€“Â cas
Jan 24 at 2:13

I tried this on CentOS 7 scp still compliants about not a regular file.
â€“Â Ask and Learn
Jan 24 at 3:11

so it does...I should have tested that before commenting. maybe there is some valid reason for scp to care whether a file is a regular file or not. or maybe it's just some over-zealous special-case handling. dunno.
â€“Â cas
Jan 24 at 6:17

@cas it introduces ambiguity, so it's blocked. Consider scp /dev/sdc3 remote:/dev/sdc3 where the destination does not exist. Should this copy the device node? What about its contents? Since sdc3 is missing on the target should we copy the contents to a file on the destination?
â€“Â roaima
Jan 24 at 8:04

1

rsync is different to ssh or scp. BTW, for live-migrating VMs, I zfs send the VM's ZVOL to my secondary VM host :), and that's usually very quick because it's only a small update since the most recent snapshot send. The DRBD idea in the strugglers.net article is nicely implemented in google's ganeti. I've got ganeti working with ZFS at home, but I don't really need ganeti here. libvirt plus my own scripts suffice. $workplaces that have paid me to do VM stuff tend to go for vmware, xen, openstack, or containers with docker. I quite like the latter 3.
â€“Â cas
Jan 24 at 10:14

Â |Â
show 3 more comments

up vote
4
down vote

accepted

-L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A suffix of K, M, G, or T can be added to denote Kilobytes (*1024), Megabytes, and so on.

A suitable solution would be something like this, which rate-limits at approximately 10Mb/s (remember that 1MB/s is approximately 10Mb/s, after accounting for padding, network headers, etc.):

pv --rate-limit 1M </dev/zero | ssh user@example.net 'cat >/dev/null'

If you want bidirectional traffic flow, remove the quotes from the 'cat >/dev/null'.

answered Jan 23 at 22:39

roaima

39.7k545108

+1. you can also do this with scp itself if you use a modernish shell (e.g. bash, ksh, zsh. not ash or dash) that supports process substitution. scp <(pv ...) user@example.net:/dev/null
â€“Â cas
Jan 24 at 2:13

I tried this on CentOS 7 scp still compliants about not a regular file.
â€“Â Ask and Learn
Jan 24 at 3:11

so it does...I should have tested that before commenting. maybe there is some valid reason for scp to care whether a file is a regular file or not. or maybe it's just some over-zealous special-case handling. dunno.
â€“Â cas
Jan 24 at 6:17

@cas it introduces ambiguity, so it's blocked. Consider scp /dev/sdc3 remote:/dev/sdc3 where the destination does not exist. Should this copy the device node? What about its contents? Since sdc3 is missing on the target should we copy the contents to a file on the destination?
â€“Â roaima
Jan 24 at 8:04

1

rsync is different to ssh or scp. BTW, for live-migrating VMs, I zfs send the VM's ZVOL to my secondary VM host :), and that's usually very quick because it's only a small update since the most recent snapshot send. The DRBD idea in the strugglers.net article is nicely implemented in google's ganeti. I've got ganeti working with ZFS at home, but I don't really need ganeti here. libvirt plus my own scripts suffice. $workplaces that have paid me to do VM stuff tend to go for vmware, xen, openstack, or containers with docker. I quite like the latter 3.
â€“Â cas
Jan 24 at 10:14

Â |Â
show 3 more comments

up vote
4
down vote

accepted

-L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A suffix of K, M, G, or T can be added to denote Kilobytes (*1024), Megabytes, and so on.

A suitable solution would be something like this, which rate-limits at approximately 10Mb/s (remember that 1MB/s is approximately 10Mb/s, after accounting for padding, network headers, etc.):

pv --rate-limit 1M </dev/zero | ssh user@example.net 'cat >/dev/null'

If you want bidirectional traffic flow, remove the quotes from the 'cat >/dev/null'.

answered Jan 23 at 22:39

roaima

39.7k545108

-L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A suffix of K, M, G, or T can be added to denote Kilobytes (*1024), Megabytes, and so on.

A suitable solution would be something like this, which rate-limits at approximately 10Mb/s (remember that 1MB/s is approximately 10Mb/s, after accounting for padding, network headers, etc.):

pv --rate-limit 1M </dev/zero | ssh user@example.net 'cat >/dev/null'

If you want bidirectional traffic flow, remove the quotes from the 'cat >/dev/null'.

answered Jan 23 at 22:39

roaima

39.7k545108

answered Jan 23 at 22:39

roaima

39.7k545108

answered Jan 23 at 22:39

roaima

39.7k545108

answered Jan 23 at 22:39

roaima

39.7k545108

+1. you can also do this with scp itself if you use a modernish shell (e.g. bash, ksh, zsh. not ash or dash) that supports process substitution. scp <(pv ...) user@example.net:/dev/null
â€“Â cas
Jan 24 at 2:13

I tried this on CentOS 7 scp still compliants about not a regular file.
â€“Â Ask and Learn
Jan 24 at 3:11

so it does...I should have tested that before commenting. maybe there is some valid reason for scp to care whether a file is a regular file or not. or maybe it's just some over-zealous special-case handling. dunno.
â€“Â cas
Jan 24 at 6:17

@cas it introduces ambiguity, so it's blocked. Consider scp /dev/sdc3 remote:/dev/sdc3 where the destination does not exist. Should this copy the device node? What about its contents? Since sdc3 is missing on the target should we copy the contents to a file on the destination?
â€“Â roaima
Jan 24 at 8:04

1

rsync is different to ssh or scp. BTW, for live-migrating VMs, I zfs send the VM's ZVOL to my secondary VM host :), and that's usually very quick because it's only a small update since the most recent snapshot send. The DRBD idea in the strugglers.net article is nicely implemented in google's ganeti. I've got ganeti working with ZFS at home, but I don't really need ganeti here. libvirt plus my own scripts suffice. $workplaces that have paid me to do VM stuff tend to go for vmware, xen, openstack, or containers with docker. I quite like the latter 3.
â€“Â cas
Jan 24 at 10:14

Â |Â
show 3 more comments

+1. you can also do this with scp itself if you use a modernish shell (e.g. bash, ksh, zsh. not ash or dash) that supports process substitution. scp <(pv ...) user@example.net:/dev/null
â€“Â cas
Jan 24 at 2:13

I tried this on CentOS 7 scp still compliants about not a regular file.
â€“Â Ask and Learn
Jan 24 at 3:11

so it does...I should have tested that before commenting. maybe there is some valid reason for scp to care whether a file is a regular file or not. or maybe it's just some over-zealous special-case handling. dunno.
â€“Â cas
Jan 24 at 6:17

@cas it introduces ambiguity, so it's blocked. Consider scp /dev/sdc3 remote:/dev/sdc3 where the destination does not exist. Should this copy the device node? What about its contents? Since sdc3 is missing on the target should we copy the contents to a file on the destination?
â€“Â roaima
Jan 24 at 8:04

1

rsync is different to ssh or scp. BTW, for live-migrating VMs, I zfs send the VM's ZVOL to my secondary VM host :), and that's usually very quick because it's only a small update since the most recent snapshot send. The DRBD idea in the strugglers.net article is nicely implemented in google's ganeti. I've got ganeti working with ZFS at home, but I don't really need ganeti here. libvirt plus my own scripts suffice. $workplaces that have paid me to do VM stuff tend to go for vmware, xen, openstack, or containers with docker. I quite like the latter 3.
â€“Â cas
Jan 24 at 10:14

+1. you can also do this with scp itself if you use a modernish shell (e.g. bash, ksh, zsh. not ash or dash) that supports process substitution. scp <(pv ...) user@example.net:/dev/null
â€“Â cas
Jan 24 at 2:13

I tried this on CentOS 7 scp still compliants about not a regular file.
â€“Â Ask and Learn
Jan 24 at 3:11

so it does...I should have tested that before commenting. maybe there is some valid reason for scp to care whether a file is a regular file or not. or maybe it's just some over-zealous special-case handling. dunno.
â€“Â cas
Jan 24 at 6:17

@cas it introduces ambiguity, so it's blocked. Consider scp /dev/sdc3 remote:/dev/sdc3 where the destination does not exist. Should this copy the device node? What about its contents? Since sdc3 is missing on the target should we copy the contents to a file on the destination?
â€“Â roaima
Jan 24 at 8:04

rsync is different to ssh or scp. BTW, for live-migrating VMs, I zfs send the VM's ZVOL to my secondary VM host :), and that's usually very quick because it's only a small update since the most recent snapshot send. The DRBD idea in the strugglers.net article is nicely implemented in google's ganeti. I've got ganeti working with ZFS at home, but I don't really need ganeti here. libvirt plus my own scripts suffice. $workplaces that have paid me to do VM stuff tend to go for vmware, xen, openstack, or containers with docker. I quite like the latter 3.
â€“Â cas
Jan 24 at 10:14

Â |Â
show 3 more comments

up vote
0
down vote

from commandlinefu.com:
~~dd if=/dev/zero bs=4096 count=1048576 | ssh user@host.tld 'cat > /dev/null'~~

As a helpful user posted, this will stop at 4GiB, which is not infinite. Therefore:

</dev/zero ssh user@host.tld 'cat > /dev/null'

edited Jan 23 at 22:52

answered Jan 23 at 22:23

Xalorous

22118

4GB is not infinite. Why make this complicated? </dev/zero ssh user@host.tld 'cat > /dev/null'
â€“Â Gilles
Jan 23 at 22:49

add a commentÂ |Â

up vote
0
down vote

from commandlinefu.com:
~~dd if=/dev/zero bs=4096 count=1048576 | ssh user@host.tld 'cat > /dev/null'~~

As a helpful user posted, this will stop at 4GiB, which is not infinite. Therefore:

</dev/zero ssh user@host.tld 'cat > /dev/null'

edited Jan 23 at 22:52

answered Jan 23 at 22:23

Xalorous

22118

4GB is not infinite. Why make this complicated? </dev/zero ssh user@host.tld 'cat > /dev/null'
â€“Â Gilles
Jan 23 at 22:49

add a commentÂ |Â

up vote
0
down vote

from commandlinefu.com:
~~dd if=/dev/zero bs=4096 count=1048576 | ssh user@host.tld 'cat > /dev/null'~~

As a helpful user posted, this will stop at 4GiB, which is not infinite. Therefore:

</dev/zero ssh user@host.tld 'cat > /dev/null'

edited Jan 23 at 22:52

answered Jan 23 at 22:23

Xalorous

22118

from commandlinefu.com:
~~dd if=/dev/zero bs=4096 count=1048576 | ssh user@host.tld 'cat > /dev/null'~~

As a helpful user posted, this will stop at 4GiB, which is not infinite. Therefore:

</dev/zero ssh user@host.tld 'cat > /dev/null'

edited Jan 23 at 22:52

answered Jan 23 at 22:23

Xalorous

22118

edited Jan 23 at 22:52

answered Jan 23 at 22:23

Xalorous

22118

answered Jan 23 at 22:23

Xalorous

22118

answered Jan 23 at 22:23

Xalorous

22118

4GB is not infinite. Why make this complicated? </dev/zero ssh user@host.tld 'cat > /dev/null'
â€“Â Gilles
Jan 23 at 22:49

add a commentÂ |Â

4GB is not infinite. Why make this complicated? </dev/zero ssh user@host.tld 'cat > /dev/null'
â€“Â Gilles
Jan 23 at 22:49

4GB is not infinite. Why make this complicated? </dev/zero ssh user@host.tld 'cat > /dev/null'
â€“Â Gilles
Jan 23 at 22:49

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Post as a guest

Name

搜尋此網誌

mjhjmtu