Track server configuration file changes using Git versioning system

By | 01st Jun 2012 | 4 min read

When a server is managed by more than one admin, it’s always a challenge to keep track of the changes made to the configuration. And when in a multiserver environment managed by more than one admin, this is going to be more complex. It would have been much saner if there was a utility to handle all this. The ones that we found were quite complicated and was made for handling huge numbers. All we wanted was a very simple utility to do just the job, without much bells and whistles. And so, we started out on our own. Here’s what we have now.

We were using the Git for our Drupal code base version controlling. We thought the same could be made use of for system configuration file versioning. We created a bash script that did just that and scheduled it to execute once every hour.

The script compares configuration files with the ones we have in the git repo, and if changes are found, it copies them to a directory called conf and adds the changes to git. It reads the details of files to be tracked from a configuration file kept for the same.

Cron was used for scheduling jobs.

The script can run both as root as well as any normal user.

If the script finds any difference it sends out an email to the recipient address stored in the script.

The script requires that you have a git server and that you have already created a repo for the same.

Here is the script (you might have to modify the email, hostname variables):. You can also follow this at https://github.com/anoopjohn/utils/tree/master/chkconfig

#!/bin/bash

DEBUG=0

LOG_FILE="log.txt"
email_to_address="[email protected]"
email_cc_address="[email protected]"

 # Debug function
function db {
  if [ $DEBUG -eq 1 ];
  then
    echo "$1"
  fi
}

# Log function
function log {
  # If there are parameters read from parameters
  if [ $# -gt 0 ]; then
    echo "[$(date +"%D %T")] [email protected]" >> $LOG_FILE
    db "[email protected]"
  else
    # If there are no parameters read from stdin
    while read data
    do
      echo "[$(date +"%D %T")] $data" >> $LOG_FILE
      db "$data"
    done
  fi
}

# Change to the dir this script resides in
script_path=`readlink -f $0`
script_dir=`dirname "$script_path"`
cd "$script_dir"
 # Run copy operations if run as root
if [ $(id -u) -eq 0 ]; then
  db "Running copy operations"
  # Copy files to the conf folder
  while read path
  do
    # Ignore comments and empty lines
    echo "$path" | egrep '(^\s*#)|(^\s*$)' >/dev/null 2>&1 && continue
    db "$path read from the file"
    source="$path"
    destination="./conf/`hostname`$path"
    if [ -f "$source" ]; then
      # If file then copy
      param='-f'
    elif [ -d $source ]; then
      # If folder then deep copy
      param='-fR'
      destination="`dirname \"$destination\"`"
      # Create the destination folder if it does not exist
      if [ ! -d "$destination" ]; then
        log "$destination does not exist. Creating dir"
        mkdir -p "$destination"
      fi
    else
      log "$path: Illegal path found."
      # Continue on to the next path
      continue
    fi
    db "Copying $source to $destination"
    (nice cp $param "$source" "$destination" 2>&1) | log
  done < ./`hostname`.conf
  db "Changing ownership of conf/`hostname` to metheuser"
  chown -R metheuser: metheuser "conf/`hostname`"
 # Run git operations if run as normaluser
else
  db "Running git operations"
  # Run git diff to find changes
  file_diff=`git diff --no-prefix`
  diff_lines=$(($(echo -n "$file_diff" | wc -l)))
  # Run git status to check if untracked files are present
  git status|grep untracked > /dev/null
  if [ $? -eq 0 ]; then
    has_untracked=1
    git_status=`git status`
  fi
  # If there is a difference
  if [[ $diff_lines -gt 0 || $has_untracked -eq 1 ]]; then
    log "$diff_lines line(s) of difference found"
    log "Has untracked = $has_untracked"
    # Get latest changes from other servers
    log "Pulling changes (if any) from server"
    (git pull 2>&1) | log
    # Commit the difference on the machine
    (git add -A 2>&1) | log
    (git commit -m "Adding changes from $(hostname)" 2>&1) | log
    (git push 2>&1) | log
    db $file_diff
    subject="[CONFIG-TRACK] `hostname` - Status Report - $(date)"
    git_differences="`echo -e "$file_diff\n$git_status"`"
    log "Sending differences via email"
    log "$git_differences"
    echo -e "$git_differences" | mail -s "$subject" -c $email_cc_address $email_to_address 2>&1 | log
  else
    db "No differences found"
  fi
fi