Secure Shell & Rsync Tips
Someone asked for this.
The secure shell, hereafter ssh, is a command line encrypted communication tool for Unix systems that was created the same year as Linux, back in 1991. This package provides encrypted replacements for telnet remote access, ftp file transfer, and a bunch of other fun features.
A couple of times this year I’ve talked to journalists with Macs who needed to do file transfer stuff and we did some slash and burn networking to get them their files. This post is going to be a few recipes I use all the time, a cheat sheet for those who don’t use ssh every day.
Keys:
You will need to know a bit about ssh key pairs. This is public key cryptography stuff, you have a public side of a key that you give out and a private side you keep to yourself.
This is a brand new account I made on my workstation just for this article, what you see here are the results of the ls -la command - list files, long listing, and show all files. Without that “a” you would not see the files and directories with a leading period in their names. This account has no ssh related stuff yet.
The first thing we’re going to do is create a key pair using the following command.
ssh-keygen -t ed25519
This will prompt you for a file name and then a passphrase. Just hit enter for all three. Some day you may need keys with a passphrase and an ssh agent and stuff, but for now this simple setup is fine.
The do this
cd ~/.ssh
ls -l
cat id_ed25519.pub
So we changed to the hidden .ssh directory under our home directory, and then we cat the public key. This is what you give to someone when they ask for your ssh key. The other file is the private side, you don’t share that with anyone, because if someone has both pieces they can pose as you.
Since you are hip and slick, you’re going to stick to ED25519 key pairs. There are several types, this one is a good algo and the short string is nice.
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIAL4kj2pVE/nCqhTYBPyFZKtB8mnj+gaPUK3gIP7uUH demo@desk
This mess is an RSA key. Not as secure as ED25519 and you get this big glob of junk instead of a single concise line. This is not like passwords where longer is better, it’s cryptographic hashes.
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTmOpiwEMuZ2zHqEidHh7Cx5b6XNs3+BHvbuh9y5Bq4Rl687jGRpiUOMv9j5SCaM9NY6HZVO3ZGJXEkyBGvVb4IHHJeZt1LeH79BkQ8glWs6ZTffWq/ozWP4+fz7n1lJ8Ec6JvXdk3yYZ5hkYRuzB6cjLYWdpMAHXIGLN9qk+kt9SXElFt8tX4xvK0N+OzRWWziqSyxVBSpX0ODjWV39NlvZ4rK3/UpW76pbTZBIEFtMCnu0yv9V8QDyvGyiKUsZKpigCf0wLEJ4Ke+aJqMEnfhiZtBesgOKaDFNr++/JjVtJ8oI8I6Yl8+aYzFH3TujszrHPqvQrwbED3wQXpZGjKqXW7+U1kSyZMw4bwqfCImedhbQGAT8qeyBN7xzZxlsOBuh4mmsMTM6SWZUjFBjD1EE0kX/nuhPYQDa//le67rYfKj9x6WEAISXccnuU8l7ObVr1eJStRLXogX0fV5pn+KKKHt2SWT21yRaXLPI/SDxQnLyz9QBQAghAUDCT+nVk= demo@desk
Installing Keys:
This is how you install a key on a remote system. I made an account called example, I logged in using the password, I created a .ssh directory, and then in that directory I created a file authorized_keys and put demo’s public key in it. The screen shot doesn’t show but you can end the cat command with a ctrl-d, that’s the unix end of file. Alternately you could
mkdir .ssh
nano ~/.ssh/authorized_keys
Paste the key in, and then save the file.
Using Keys:
Once the key for your shell is in the ~/.ssh/authorized_keys file, you can just ssh to the remote system and it’ll let you log in without prompting for a password. Seems scary, doesn’t it? As long as your desktop is secure this will not be a problem. If you are in a super secure environment you may have to put a passphrase on your keys, and then the keys get loaded into an ssh agent. When this is configured you enter the passphrase just once, and as long as you’re still logged in you can do the slick login w/o having to type stuff.
I started using ssh shortly after it came out. Over the last thirty three years I’ve inherited systems with problems, but I’ve never had a situation where my setup using keys has been compromised. But I am kinda paranoid, let me explain …
Paranoia:
The sshd program honors tcpwrappers - this is an old school per application firewall thing that we used to use back when computers were tiny, slow, and not everything came with a proper firewall. The configuration is kept in /etc/hosts.allow and this is a minimal setup.
Careful with this, if you goof you will lock yourself out. If it’s a VPS and you get remote console from the host then you can recover. If it’s a physical machine and it doesn’t have an IP accessible remote access method, you’re stuck until someone can manually reset it.
If you are an end user with a Mac to just trying send/receive files from a remote system, you don’t have to worry about this behind providing your origin IP to the person who runs the remote system. Finding your public IP is easily done with this website:
https://ifconfig.me
Further Paranoia:
The tpwrappers setup is pretty good. Because I deal with a higher level of threat on any of my servers I typically only have static routes to a few IP addresses I know and I also use ufw, the uncomplicated firewall. If the machine needs more general internet access it will get that via NAT on a virtual machine with a fail closed OpenVPN connection.
What can I say, I hate uninvited guests.
One thing I will do for back door access is make a Tor hidden service for ssh. The ssh invocation looks a little different and it typically feels like a dial up modem. Since it’s a hidden service you can just ssh directly, you have to arrange transport for it using torsocks.
torsocks ssh -l myshell kjtmglcwmvzpxdxg2nxfpavk7ikwuquckjfbt7hysmtyxrqlscveuead.onion
As a Mac or new Linux user, this stuff is easily accessible - the basic Tor install starts with a workable configuration and torsocks is typically bundled with it.
Copying Files:
Once you’ve got your key in place there are two ways to copy files.
The scp command is an encrypted system to system version of the plain ol' cp command. Here’s an example - this copies the public half of the key on your current machine to the authorized_keys file on a remote system. That’s a copy, not an add - you’ll step on the authorized_keys if it’s got other stuff in it. Do that and the other people who use the system will call you nasty names for weeks.
scp ~/.ssh/id_ed25519.pub myshell@remote:~/.ssh/authorized_keys
What scp is simple and quick for a few files. If I’m doing anything complex I will use the rsync command instead. This command is powerful and you’re going to have to do some reading, but here’s a simple example. I’m starting rsync, I want it to get all files, z means compress, v means be talkative about what it’s doing. It’s sending those files to the ~/mybackup directory belonging to myshell on myvps.
rsync -azv . myshell@myvps:~/mybackup
So one of the nice features is that you can optionally compress or not compress using the ‘z’ flag. If you’ve got a bunch of random files, using ‘z’ will reduce bandwidth. If you’re copying one ginormous zip file it doesn’t provide any benefit with pre-compressed data.
The other nice feature is that rsync is smart about what’s already been copied - you can start a transfer on a big directory tree, if the coffee shop kicks you out because they’re closing, you can just restart the command at home. It will log into the remote end, check things over a bit, and then pick up where it was.
The rsync command can do all sorts of other things, like keep a remote system perfectly matched to your local machine. I have a couple off site backups of my system, this is a simplified example of how that is done.
rsync -azv —delete . myshell@myvps:/backup/desktop
If I delete a file here, rsync is smart enough to figure that out and delete it on the far end, too. There are a LOT of options with this package, it’s a powerful took for system administrators.
Bandwidth Limits:
If you have a lot of files to transfer and other stuff to do you may want to limit the network usage of commands like scp or rsync. For this you need trickle. The documentation says that trickle is a userland bandwidth shaper for Unix-like systems.
What that means is the usually the things you adjust to control network usage are part of the operating system, but trickle is a command anyone can use. This command tracks the amount of time a trickle’d rsync command requires to do its work.
time trickle -u 1000 -d 1000 rsync -azv . myshell@myvps:~/ratelimitedupload
There is a component of trickle that can be added to an operating system so that every command using trickle has to get its network capacity from a system wide pool. This is something I did on machines twenty years ago, if you’re just doing one thing with a big batch of files a simple command line invocation like you see above is fine.
Remote Services:
If you’re just moving files you won’t have to do things like this, but sometimes there are services on machines that you want to see, but they’re not internet accessible. Here’s an example from today - I have a VPS I can ssh into and it’s got an ArangoDB restore running. The web interface for ArangoDB is at port 8529, but it’s firewalled. The following port forwarding command makes it accessible.
ssh -L8529:localhost:8529 myshell@myvps
ArangoDB is running on the remote system, and now I can access the web interface to it at this URL:
http://localhost:8529
The ssh session needs to remain in place as long as you access the remote system. There are ways to turn this into a systemd service on Linux - something that starts automatically and makes the remote service accessible as long as the systems are both online. We won’t do this here, just be aware that such things are possible.
Conclusion:
Apple and intentionally smooth distros, like the Ubuntu Budgie I use, have lowered the bar for using a proper operating system. ChatGPT is really good for coughing up one line examples of commands. But for true mastery SSH: The Secure Shell from 2005 is required reading.
