- GitHub LFS has a 2GB limit on file size. This never works with large projects like video productions.
- git-annex uses symlinks to manage annexed files. This doesn't work well on Windows.
So my conclusion at the moment is that Borg is the best backup software for mid/large-sized personal projects.
Weren't you using rsync.net? Why did you switch to Google Cloud?
rsync.net is quite handy for using with Borg, but it's slow from Japan. The server is located in Hong Kong, and the connection speed varies through the day. It might be a good solution for people in North America, Western Europe and China, but unfortunately it doesn't work well from my workplaces.
I should admit that Google Cloud Storage is super fast and stable. It also provides long-term/low-cost storage classes (nearline/coldline/archive) that are cost beneficial for backup use.
No. Although Borg is useful for taking project snapshots, it doesn't provide any version control features like git or hg. When I need to track file versions, I create a git repository in a subdirectory and make it contained in the Borg repository.
- Linux and Windows (WSL): Install pip
sudo apt install python3-pip
, then install gsutilsudo pip3 install gsutil
. - MacOS: Install pip
sudo easy_install -U pip
, then install gsutilpip install gsutil --user
. Theuser
flag is needed to avoid a package conflict problem withsix
.
After installing gsutil
, authenticate it with gsutil config
.
Go to Google Cloud Console and create a bucket.
Create Borg
folder in the bucket. The Borg repository files will be copied
into this folder.
- Linux:
sudo apt install borgbackup
- Windows (WSL): Install the
borgbackup
apt package from PPA. Don't use the Ubuntu package. Some issues on WSL are only fixed in the very recent versions. - MacOS:
brew cask install borgbackup
.
Then create a borg repository:
borg init --encryption=none Borg
Download borg.sh
from this gist or run the following command line.
curl -L https://git.io/Jv5k0 > borg.sh
Open borg.sh
with a text editor and modify BUCKET_NAME
to the Google Storage
bucket name. Then put it in the project root directory, and make it executable
using chmod
.
Run the following command line to initialize the Borg repository.
./borg.sh init
It's recommended to upload borg.sh
to the Google Storage bucket because it'll
be needed to restore the repository in another environment.
./borg.sh push