thallia
Filesharing with Rsync
For the past month or so, I’ve been attempting to find a way to file share between my main laptop and tower computer, with my server as the middle-man between the two that holds the main file repo. I wanted it to be the kind of file-tracking system where if one file is changed on one system, it’ll sense it and update the others.
At first I considered a private Git repository, but was unsure about the security and also the fact that I’d have to pay more money for private repositories. Ugh.
Then I considered Samba, but it seemed a little over the top for what I was thinking. Gector said I should consider making my own, but I don’t feel confident in my own experience to go full-on project with that right before I leave for college, when my own design could fail.
To predate this a little - I’ve been listening to a lot of cybersecurity and hacking podcasts lately during my long drives between work and home (around an hour, which is a great time chunk to download information into my head from stuff like podcasts). I recently finished an awesome series called Darknet Diaries, which is a dude who interviews people and tells stories about hacking and exploits. From the website:
Darknet Diaries is a podcast covering true stories from the dark side of the Internet. Stories about hackers, defenders, threats, malware, botnets, breaches, and privacy.
I highly recommend the podcast, it’s super entertaining and very educational in certain security aspects. There is language in some of the episodes, so be aware of that before taking a listen. But don’t let it stop you, the listen is worth it!
So how does this relate to my rsync server? I’ve been listening to another networking and security podcast called Section 9. I’ve only gone through a few of their episodes, but they are definitely interesting and fairly educational for me. They mentioned in their DHCP Failover #71 episode a service called rsync, which I’ve heard about, but never had a use for before. Looking it up again, it’s the exact thing I’ve been searching for for my file-tracking system.
Rsync is a file transferring and synchronization setup that you can use over SSH (or an unencrypted transfer thing, which isn’t a very good idea if you’re transferring information you don’t want meddled with). By itself it is not encrypted at all, but if it called going through an SSH connection, everything will be encrypted through SSH, which is exactly what I want.
First I set up a pair of SSH keys between my server and laptop (I’m away from home, so I can’t access my tower until I get back).
To set that up, I followed DigitalOcean’s great guide. They have some awesome stuff.
ssh-keygen
Should do the trick for ya. It’ll display something along the lines of:
Generating public/private rsa key pair.
Enter file in which to save the key (/<span class="highlight">thallia</span>/.ssh/id_rsa):
Which is typically the best directory to go with, unless you had another one in mind.
Enter passphrase (empty for no passphrase):
If you want an extra layer of security, you can enter a passphrase here. It’s just like a password, and it’ll prompt you to enter it every time you SSH into the server to verify the key. It’s a good setting to have, but not required.
Finally it’ll display a success message, and you can copy the public key to the remote server or other file-sharing device you want! To do that:
ssh-copy-id <span class="highlight">username</span>@<span class="highlight">remote_host</span>
And that should be all set up good.
Next, I had to organize all the files I wanted to sync into a folder. There’s a project/life/config/all things folder that I have set up, but not entirely organized, so I went through and figured that all out so configs wouldn’t mess up the systems I have going on (like fish, tmux, i3, etc).
Right now, I only want to push my laptop’s directory with all the goodies to my remote server. To do that, this command:
`rsync -a ~/dir1 <span class="highlight">username</span>@<span class="highlight">remote_host</span>:<span class="highlight">destination_directory</span>`
Should do the trick. The -a flag “is a combination flag. It stands for “archive” and syncs recursively and preserves symbolic links, device files, modification times, group, owner, and permissions. It is more commonly used than -r and is usually what you want to use.” (from digital ocean’s rsync tutorial).
I’ll replace ~/dir1 with the directory I want to copy, and my server’s login@ip-address:and/the/destination/directory/for/the/copied/files.
I had a lot of stuff in that copied directory, so it took a while, but it displayed every file there was during the syncing process.
Next step was to do it on my tower computer at home! This was an easy change, I generated the keys between the tower and my server, then ran _this _rsync command to sync from the server to the tower:
rsync -av username@remote_host:source_directory ~/destination/on/tower
and soon enough I had my tower synced up with my laptop :)
But to keep it this way, I don’t want to keep having to run these commands back and forth between computers. So I wrote a little bash script to fix that.
The script goes through and asks whether I’m at home, college, or somewhere else. Depending on which answer I give, different IP addresses will be used for rsync to be able to access my traveling laptop. Once that’s figured out, it runs the push and pull rsync commands and syncs all my computers up to my server.
I set the script up on my server with a cronjob, which is a timed-interval scheduling tool where you can run commands every time-interval that you want. Crontab syntax was weird looking and confusing, until I found this:
* * * * * command to be executed
- - - - -
| | | | |
| | | | +----- day of week (0 - 6) (Sunday=0)
| | | +------- month (1 - 12)
| | +--------- day of month (1 - 31)
| +----------- hour (0 - 23)
+------------- min (0 - 59)
I wanted my script to be run every 24 hours, so my the line in my crontab -e file looked like this:
* 23 * * * ./rsync.sh
To see whether the job ran, you can check out /var/log/syslog and scroll to the very bottom to see whether or not the cronjob has executed or not. :)
And now I have a solid filesharing system between my computers!
The only thing missing is a few public-IP features that I haven’t grabbed yet. Once I get those other IPs that my laptop connects to then the script will truly be finished. But the skeleton is certainly working, and I’m very happy about that, especially since it’s my first script.
happy scripting & syncing!
{thallia}