Skip to content

Instantly share code, notes, and snippets.

@mastervash

mastervash/upload_cloud

Created September 10, 2023 08:44
Show Gist options
  • Save mastervash/29beb1ca6c8b469b7c2febd85829881c to your computer and use it in GitHub Desktop.
Save mastervash/29beb1ca6c8b469b7c2febd85829881c to your computer and use it in GitHub Desktop.
Download ZIP
Rclone move script, based on @Animosity022 provided examples. (When there's over 100GB waiting to be uploaded - skip the Crontab)
Raw
upload_cloud
#!/bin/bash
# RClone Config file
RCLONE_CONFIG=/path/to/your/rclone.conf
export RCLONE_CONFIG
LOCKFILE="/var/lock/`basename $0`"
LOG_FILE="/path/to/your/logs/rclone-upload.log"
# Maximum log file size in bytes (10MB)
MAX_LOG_SIZE=$((10 * 1024 * 1024))
(
# Wait for lock for 5 seconds
flock -x -w 5 200 || exit 1
# Check if the log file needs rotation
if [ -e "$LOG_FILE" ] && [ $(stat -c %s "$LOG_FILE") -ge $MAX_LOG_SIZE ]; then
mv "$LOG_FILE" "$LOG_FILE.1" # Rename the current log file
fi
# Move older local files to the cloud using RClone and append to the log file
/usr/bin/rclone move /storage/media/upload dropcrypt:sort --checkers 6 --log-file "$LOG_FILE" -v --tpslimit 3 --transfers 6 --ignore-existing --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --drive-chunk-size 64M # --delete-empty-src-dirs
) 200> ${LOCKFILE}
@mastervash
Copy link
Author

Be sure you fill in the paths to your rclone.conf and the location where logs will be created.

Then modify the path to point at the dir. you want the script running on, and it's destination in one of your existing remotes. Customize the flags your rclone move command uses to fit your needs.

DON'T assume the flags that I'm using are the ones you should use.

running the script

chmod +x upload_cloud to make the script executable.

For testing I suggest nohup ./script.sh & followed by disown

For production I would urge you create a systemd service file to automate the process.

[Unit]
Description=Rclone continuous upload folder

[Service]
ExecStart=/path/to/your/script.sh
User=$USER
Restart=always

[Install]
WantedBy=multi-user.target

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment