Digging SCP

SCP is one of the commands we frequently use, and I have always wondered how it works. 
 
SCP helps you copy file(s) remotely to a server securely. To understand the working of SCP, where else to look for it other than the code itself. So, I started looking at https://github.com/openssh/openssh-portable/blob/master/scp.c.
What did I find out?
SCP basically opens up a SSH session and copies content from the source to the destination system. We have a different lingo for that in the code. The mode where we open, read and send a file, is called the source mode, and the one where we receive the file and write it, is called the sink mode. SCP basically forks two processes and runs two agents on both the hosts. 
Source Mode
First, a message with the type, mode, length, filename are supplied, followed by a new line in plain text is sent, then the file content follows. We might have multiple messages like this, called protocol messages before the actual file contents are sent. The scp command actually waits for a reply before it sends another message. The response could be 0,1 or 2 depending on the action. 0 is for OK, 1 is for warning and 2 is for error, and the connection is closed after that.
A typical protocol message looks like this:
C0644 299 group
C here represents that it is a single file copy
0644 is the mode
299 is the length to be copied
group is the filename
Sending D in C0644 instead of C represents recursive directory copy.

After the messages, raw data is transferred. The consumer reads exactly that much data as specified in the length field.

void source(int argc, char **argv)
Sink Mode

The sink mode receives the file, and sends a 0,1 or 2 response once it receives the output. Every message should always be sent with a new line character. This is confirmed by a lot of places where this condition is put in the code. 

void sink(int argc, char **argv)
So, How does it work?
Basically SCP opens up a SSH connection between two systems, forks two SCP agents, opens and sends the file in source mode, retrieves and writes it in sink mode, with constantly sending responses in the sink mode, and shows progress meter by using a callback while writing the file with sending back the bytes already written.
Screen Shot 2017-03-02 at 7.22.09 PM.png
Observations
Progress Meter – In addition to this, we have a start_progress_meter in the source method, that is invoked by a callback by the atomicio library(https://github.com/openssh/openssh-portable/blob/master/atomicio.c) that writes to the file and is declared as follows:
start_progress_meter(curfile, stb.st_size, &statbytes);
 
File Size – The file size to transfer file is actually given in the following line:
 
(longlong) stb.st_size
 
This means we can technically transfer 2^40bytes which is is 1000GB, but obviously systems like FAT32 only supports 4GB per file.
 
Argument Flags – The arg flags are actually added based on what you receive, and there’s a huge switch statement which does the job. One of the examples is as follows:
case 'F':
  addargs(&remote_remote_args, "-%c", ch);
  addargs(&remote_remote_args, "%s", optarg);
  addargs(&args, "-%c", ch);
  addargs(&args, "%s", optarg);
  break;
Error Handling – The method to invoke errors is called SCREWUP 😀 and is declared as:
 
#define SCREWUP(str) { why = str;gotoscrewup; }
 
Shell Commands – We have the following commands to execute Shell Commands:
 
int do_cmd(char*host,char*remuser,char*cmd,int*fdin,int*fdout);
int do_cmd2(char*host,char*remuser,char*cmd,intfdin,intfdout);
So, it was really fun understanding how SCP works, and for a deeper understanding, please have a look at https://github.com/openssh/openssh-portable/blob/master/scp.c if you are up for looking at 1371 lines of a C file 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s