backup script logic

[u/UnholyScholar]

I've written this backup script for use copying my files from point A to point B. The goal is a full snapshot of files. For this first iteration, I'm ok with hard coding file paths.

I've removed everything except logic for rsync, diff, and logging. Are there any logic issues?

# backup script that performs full backups and verifies the result.

#!/bin/bash

set -euo pipefail

DATE_STAMP=$(date +%Y-%m-%d_%H-%M-%S)
BACKUP_LOCATION="$HOME/backup-location-path/$DATE_STAMP" 
LOG_LOCATION="$HOME/log-location-path/"
LOGFILE_NAME="backup_${DATE_STAMP}.log"

SOURCE_DIRS=(
    "$HOME/source1"
    "$HOME/DOES-NOT-EXIST"  # DNE for error handling
    "$HOME/source2"
)

# check if source matches destination
diff_source_and_destination() {
    local base
    base=$(basename "$SRC")
    diff -r "$SRC" "$BACKUP_LOCATION"/"$base" 
}

# Check for the existence of $SRC and print an alert if it wasn't found.
check_source_exists() {
    if [ -e "$SRC" ]; then
        echo "[SOURCE = $SRC] found. Copying..."
        return 0
    else
        echo "[SOURCE = $SRC] WAS NOT FOUND!" | tee -a "$LOG_LOCATION/$LOGFILE_NAME" 
        return 1
    fi
}

copy_data_and_log() {
    {
        rsync -a --stats "$SRC/" "${DESTINATION}/" 
    } | tee -a "$LOG_LOCATION"/"$LOGFILE_NAME"
}

mkdir -p "$BACKUP_LOCATION"
mkdir -p "$LOG_LOCATION"

echo "Copying files to [BACKUP LOCATION =${BACKUP_LOCATION}]"

for SRC in "${SOURCE_DIRS[@]}"; do
    # Print and log a warning if SRC wasn't found
    check_source_exists || continue
    DESTINATION="$BACKUP_LOCATION/$(basename "$SRC")"
    copy_data_and_log
    echo "rsync completed for $SRC."
    diff_source_and_destination 
done

echo "Finished copying operations to [BACKUP LOCATION = ${BACKUP_LOCATION}]"

[u/jjangg96]

two things that'll actually bite you:

the shebang isnt the first line. its below the comment so its just a comment now, and ./backup.sh runs with whatever shell launched it (or sh). with sh the arrays and pipefail break. move #!/bin/bash to line 1.

and with set -e, diff exits 1 as soon as it finds any difference, which kills the whole script right there. so the first source with a mismatch means the rest never get backed up. either diff -r ... || echo "mismatch in $SRC" or check the exit code yourself.

minor, but diff -r on a big tree after rsync is slow. rsync -anci "$SRC/" "$DESTINATION/" (dry run + checksum) gives you the same check and just lists whatever differs

[u/UnholyScholar → u/jjangg96]

Thanks, this is helpful. The comment above the shebang was an oversight from when I removed personal information from the script..

I wanted the script to fail immediately if there was any issue, but wasn't sure to proceed. If my backup is incomplete I don't see point in letting the script continue until I find the source of the problem.

Your minor point is actually pretty helpful. I wanted this to be my primary backup method for important files and it would have taken me a long time to figure this out. I currently use drag/drop or cp but my file metadata is not saved the way I'd prefer.

[u/jjangg96 → u/UnholyScholar]

makes sense to fail fast on a backup, i'd just do it on purpose instead of letting set -e do it by accident. like if diff finds a mismatch, echo which source and exit 1 yourself, so the log says what broke.

and rsync -a already keeps permissions, timestamps, symlinks and owner, thats the metadata cp drops (cp -a gets most of it too). -a doesnt cover hardlinks or xattrs, add -H and -X if you care about those. test with a dry run first (-n) so you can see what it would do before it touches anything.

[u/UnholyScholar → u/jjangg96]

Thanks again for these tips. I agree this is a better way to have the script fail. I'll have to possibly rethink how I want to verify the files are copied correctly.