Sunday, April 4, 2021

Synchronizing files on two SSH hosts with one in the middle (accessing a SSH host via a SSH gateway?)

I believe that this is a rather common situation -- I want to copy files from one SSH host to the other; however, the SSH host I want to copy files from or copy files to is behind a SSH gateway. To describe the situation clearly, let's consider three SSH hosts, H, M, and O. I can log on to H via SSH, from H log on M via SSH, but from H not log on to O via SSH. To get to O, I have to log on to M first, and so M is a SSH gateway. In fact, this is a typical setup for many computational cluster, to access the cluster, you SSH to a SSH gateway. How do we copy files between H and O? Of course, we can always copy files from O to M, and M to H, which is somewhat inconvenient. Can we do better? Yes, via SSH's ProxyCommand option. Below we present two two scenarios.

1. SSH to O via M from H

Below is how we get to O "directly" from H assuming the username on M is alice_on_m and that on O alice_on_o.


ssh -o 'ProxyCommand ssh alice_on_m@M -W %h:%p' alice_on_o@O

Note that although we issue this command on H, H does not have to be able to resolve O; however, M must know O.

If you prefer the public key authentication to the password authentication, do the following instead,


ssh -o \
  'ProxyCommand ssh -i ~/.ssh/private_key_for_m alice_on_m@M -W %h:%p' \
  -i ~/.ssh/pri
  vate_key_for_o alice_on_o@O

where we assume that private_key_for_m and private_key_for_o are two private keys on host H, and their respective public keys are in ~alice_on_m/.ssh/authorized_keys at M and ~alice_on_o/.ssh/authorized_keys at O.

To me,  the most convenient method is to set up ~/.ssh/config file on H, with which, we will save some typing on the command line. Below is an example for the ~/.ssh/config file where we add two entries, one for M, and the other for O.

Host M
    HostName M
    ServerAliveInterval 240
    User alice_on_m
    AddKeysToAgent yes
    IdentityFile ~/.ssh/private_key_for_m

Host O
    HostName O
    ServerAliveInterval 240
    User alice_on_o
    AddKeysToAgent yes
    ProxyCommand ssh alice_on_m@M -W %h:%p
    IdentityFile ~/.ssh/private_key_for_o

Once we set up this, we can get on O from H (via M) by simply issing the following command on H,

ssh alice_on_o@O

2. Copying Files from O to H via M using rsync

Assuming we set up ~/.ssh/config correctly, to copy files from O to H via M using rsync, we simply issue a command like the following,


rsync -avzr alice_on_o@O:./foo/ ./foo/

which copies the directory recursively ~alice_on_o/foo at host O to ./foo at host H. Here we also rely on rsync's ability to use SSH as the transport protocol transparently.

Is this neat?

No comments:

Post a Comment