Showing posts with label rsync. Show all posts
Showing posts with label rsync. Show all posts

Sunday, April 4, 2021

Synchronizing files on two SSH hosts with one in the middle (accessing a SSH host via a SSH gateway?)

I believe that this is a rather common situation -- I want to copy files from one SSH host to the other; however, the SSH host I want to copy files from or copy files to is behind a SSH gateway. To describe the situation clearly, let's consider three SSH hosts, H, M, and O. I can log on to H via SSH, from H log on M via SSH, but from H not log on to O via SSH. To get to O, I have to log on to M first, and so M is a SSH gateway. In fact, this is a typical setup for many computational cluster, to access the cluster, you SSH to a SSH gateway. How do we copy files between H and O? Of course, we can always copy files from O to M, and M to H, which is somewhat inconvenient. Can we do better? Yes, via SSH's ProxyCommand option. Below we present two two scenarios.

1. SSH to O via M from H

Below is how we get to O "directly" from H assuming the username on M is alice_on_m and that on O alice_on_o.


ssh -o 'ProxyCommand ssh alice_on_m@M -W %h:%p' alice_on_o@O

Note that although we issue this command on H, H does not have to be able to resolve O; however, M must know O.

If you prefer the public key authentication to the password authentication, do the following instead,


ssh -o \
  'ProxyCommand ssh -i ~/.ssh/private_key_for_m alice_on_m@M -W %h:%p' \
  -i ~/.ssh/pri
  vate_key_for_o alice_on_o@O

where we assume that private_key_for_m and private_key_for_o are two private keys on host H, and their respective public keys are in ~alice_on_m/.ssh/authorized_keys at M and ~alice_on_o/.ssh/authorized_keys at O.

To me,  the most convenient method is to set up ~/.ssh/config file on H, with which, we will save some typing on the command line. Below is an example for the ~/.ssh/config file where we add two entries, one for M, and the other for O.

Host M
    HostName M
    ServerAliveInterval 240
    User alice_on_m
    AddKeysToAgent yes
    IdentityFile ~/.ssh/private_key_for_m

Host O
    HostName O
    ServerAliveInterval 240
    User alice_on_o
    AddKeysToAgent yes
    ProxyCommand ssh alice_on_m@M -W %h:%p
    IdentityFile ~/.ssh/private_key_for_o

Once we set up this, we can get on O from H (via M) by simply issing the following command on H,

ssh alice_on_o@O

2. Copying Files from O to H via M using rsync

Assuming we set up ~/.ssh/config correctly, to copy files from O to H via M using rsync, we simply issue a command like the following,


rsync -avzr alice_on_o@O:./foo/ ./foo/

which copies the directory recursively ~alice_on_o/foo at host O to ./foo at host H. Here we also rely on rsync's ability to use SSH as the transport protocol transparently.

Is this neat?