mjhjmtu

I have setup key authentication between two servers and I can ssh in without a key, for example:

ssh backup@hostname

this works fine and logs me in.
But when I try an SCP to pull a file, I get no files.

The target files have chmod 777 done to them for troubleshooting purposes but it's like it just doesn't find any files to copy even though they are there.

Here is the (ip's obfuscated) output of

scp -vvv backup@hostname:/var/stuff/backups/*.tgz /data/backups/

from the point it authenticates onward.

debug1: Authentication succeeded (publickey). 
Authenticated to xxx.xxx.xxx.xxx ([xxx.xxx.xxx.xxx]:22).
debug2: fd 4 setting O_NONBLOCK 
debug2: fd 5 setting O_NONBLOCK 
debug2: fd 6 setting O_NONBLOCK 
debug1: channel 0: new [client-session] 
debug3: ssh_session2_open: channel_new: 0 
debug2: channel 0: send open 
debug1: Entering interactive session. 
debug2: callback start 
debug2: fd 3 setting TCP_NODELAY 
debug3: packet_set_tos: set IP_TOS 0x08 
debug2: client_session2_setup: id 0 
debug1: Sending environment. 
debug3: Ignored env XDG_SESSION_ID 
debug3: Ignored env TERM 
debug3: Ignored env SHELL 
debug3: Ignored env SSH_CLIENT 
debug3: Ignored env SSH_TTY 
debug3: Ignored env http_proxy 
debug3: Ignored env USER 
debug3: Ignored env LS_COLORS 
debug3: Ignored env MAIL 
debug3: Ignored env PATH 
debug3: Ignored env QT_QPA_PLATFORMTHEME 
debug3: Ignored env PWD 
debug1: Sending env LANG = en_AU.UTF-8 
debug2: channel 0: request env confirm 0 
debug3: Ignored env SHLVL 
debug3: Ignored env HOME 
debug3: Ignored env LANGUAGE 
debug3: Ignored env LOGNAME 
debug3: Ignored env SSH_CONNECTION 
debug3: Ignored env LESSOPEN 
debug3: Ignored env XDG_RUNTIME_DIR 
debug3: Ignored env LESSCLOSE 
debug3: Ignored env _ 
debug3: Ignored env OLDPWD 
debug1: Sending command: scp -v -f /var/stuff/backups/*.tgz 
debug2: channel 0: request exec confirm 1 
debug2: callback done 
debug2: channel 0: open confirm rwindow 0 rmax 32768 
debug2: channel 0: rcvd adjust 131072 
debug2: channel_input_status_confirm: type 99 id 0 
debug2: exec request accepted on channel 0 
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 
debug2: channel 0: rcvd eof 
debug2: channel 0: output open -> drain 
debug2: channel 0: obuf empty 
debug2: channel 0: close_write 
debug2: channel 0: output drain -> closed 
debug2: channel 0: rcvd close 
debug2: channel 0: close_read 
debug2: channel 0: input open -> closed 
debug3: channel 0: will not send data after close 
debug2: channel 0: almost dead 
debug2: channel 0: gc: notify user 
debug2: channel 0: gc: user detached 
debug2: channel 0: send close 
debug2: channel 0: is dead 
debug2: channel 0: garbage collecting 
debug1: channel 0: free: client-session, nchannels 1 
debug3: channel 0: status: The following connections are open: 
 #0 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1)
debug1: fd 0 clearing O_NONBLOCK 
debug1: fd 1 clearing O_NONBLOCK 
debug1: fd 2 clearing O_NONBLOCK 
Transferred: sent 3736, received 2280 bytes, in 0.0 seconds 
Bytes per second: sent 764498.2, received 466556.7 
debug1: Exit status 1 
backup@ar-secubn03:/var/scripts$ 

Try this:

scp -vvv 'backup@hostname:/var/stuff/backups/*.tgz' /data/backups/

If that doesn't work try this:

scp -vvv 'backup@hostname:/var/stuff/backups/*.tgz' /data/backups/

debug1: Sending command: scp -v -f /var/stuff/backups/*.tgz 
debug2: channel 0: request exec confirm 1 
debug2: callback done 
debug2: channel 0: open confirm rwindow 0 rmax 32768 
debug2: channel 0: rcvd adjust 131072 
debug2: channel_input_status_confirm: type 99 id 0 
debug2: exec request accepted on channel 0 
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
...
debug1: Exit status 1

These lines indicate that the local scp instance requested the expected scp command to run on the remote system, and the remote server launched a process in response to the request. But the remote process exited with status code 1 almost immediately, and without emitting any output.

If the remote scp instance couldn't read the files to be copied, or if those files didn't exist, you would have gotten an error message to that effect. If the remote scp program couldn't be found or wasn't executable, then the exit code would have been 126 or 127 instead of 1.

I suspect one of these things is happening:

1. Something in the remote account's .bashrc or .bash_profile is causing the shell to exit early. It may be sensitive to the fact the session doesn't have a TTY, for example.


2. The scp program on the remote system is corrupt or malfunctioning.


3. The scp program on the remote system has been replaced with something else which isn't working like scp. Someone might have replaced it with a poorly written shell script, for example, or a copy of '/bin/false'.


4. The SSH software has been configured to block people from running scp. The key which you're using may be set in authorized_keys to run a specific command, or there may be a ForceCommand directive in the remote server's sshd_config file.

The simplest way to check these would be to ask the remote system administrator, or log into the remote system and inspect the scp executable. Running scp -h will print an scp-specific usage message, for example.

If you have to troubleshoot this remotely, the next thing I'd try is something like this:

$ ssh -T backup@hostname cat /etc/group

This should write the remote system's group file to your terminal. This would prove that you can run arbitrary commands on the remote system.

$ ssh -T backup@hostname scp -v -f /etc/group < /dev/zero

This simulates the remote part of an scp session. If the remote scp program is working, you should get output something like this:

C0644 674 group
root:x:0:
daemon:x:1:
[...]
devuser:x:1001:
Sending file modes: C0644 674 group
$ 

If all of that works, the question becomes why scp fails for "/var/stuff/backups/*.tgz" but works for "/etc/group". You could run this to confirm whether it's working or not:

ssh -T backup@hostname 'scp -v -f /var/stuff/backups/*.tgz' < /dev/zero

If that continues to fail, the nuclear option would be to arrange to run the remote session under strace and see exactly what each program is doing.

Your problem is clearly identified in these lines:

debug2: exec request accepted on channel 0 
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 
debug2: channel 0: rcvd eof 

which shows your channel is shut down as soon as it is opened, though the reason is not stated.

Normally,this occurs because of a faulty configuration in the /etc/passwd file, see for instance this Server Fault post. You may follow the suggestions in this post, or, better still, you may check the logs on the target machine,

 tail /var/log/auth.log

for further details.

I think you do 1st backups/* .tgz to change backups*.tgz
make sure don't use any space. might be 99% issue solved.

I had the same bad experience. I slight modification of the .bashrc on the remote system was the problem. I once added 'ulimit' and got the this problem during the next scp (ssh worked). After removing this line all was fine.

I was having a similar issue that I could exercise ssh commands, but not scp commands. I went through Kenster good debug techniques which still didn't get me anywhere, but very handy! I finally found out the issue was related to echo commands in my .bashrc script (after inserting additional ones). Once I commented out all the echo commands in the script I was able to successfully execute the scp command.