Hi Greg,
After much experimentation with various scripts, I am happy with this
latest rdiff-backup parser we have written. Script is written by
Christian Marie, who also wrote the rdiff-backup nagios plugin on
nagiosexchange.org.
The script is designed to be passed on stdin the output of an
rdiff-backup job or multiple jobs with -v5 or greater and
--print-statistics both turned on. It can deal with many jobs passed in
ie: backup.sh | parse-rdiff-backup
where backup.sh might run three jobs.
check it out - you get a nicely formatted email that says things like
Backup(s) OK 3/3 in the subject, and a summary of the job like so:
Status of backup(s) follow
Backup completed successfully
Direction: /home/sol1backup/cookie-data to /var/backupdisk/cookieman
Backup started: Tue Oct 17 23:00:06 2006
Source files: 61248
Source files size: 22290479056 (20.8 GB)
New files size: 6224482 (5.94 MB)
Deleted files size: 308229 (301 KB)
Destination size change: 18.4 MB
Errors: 79
Anway, Here is the script and config file. Read the source if you want
more info.... :)
Script:
--------------------------------
#!/usr/bin/env ruby
require 'yaml'
require 'tempfile'
class Parser
attr_reader :status
DEFAULT_CONFIG = File::SEPARATOR +
File.join('etc', 'parse-rdiff-backup.conf')
BACKUPOKCORRECT = /TotalDestinationSizeChange/
def initialize data, config = DEFAULT_CONFIG
begin
@log = data
@config = YAML::load_file(config)['parser']
@statistics_output = []
@statistics = []
@config['statistics'].each do |key,value|
@statistics << SessionStatistic.new(key, value)
end
parse!
rescue Exception => exception
@status = BackupStatus::ERROR
@error = "Error: #{exception.class} (#{exception.message})\n"
@error += "Backtrace:\n"
@error += exception.backtrace.join "\n"
end
end
def parse!
@status = BackupStatus::BROKEN
@log.each do |line|
@statistics.each do |statistic|
if line =~ /#{statistic.regex}/
if $1
@statistics_output << statistic.message + $1
end
end
end
if line =~ BACKUPOKCORRECT
@status = BackupStatus::OK
end
end
end
def statistics_output
return @statistics_output.join("\n").gsub(/^/, "\t")
end
def error
return @error.gsub(/^/, "\t")
end
private
class SessionStatistic
attr_accessor :message, :regex
def initialize message, regex
@regex = regex
@message = message + ': '
end
end
end
class BackupStatus
OK = 0
BROKEN = 1
ERROR = 2
end
class Message
attr_accessor :subject, :message, :to, :cc, :log
DEFAULT_CONFIG = File::SEPARATOR +
File.join('etc', 'parse-rdiff-backup.conf')
def initialize config=DEFAULT_CONFIG
@config = YAML::load_file(config)['message']
@subject = 'Unknown status'
@message = "Status of backup(s) follow\n\n"
@to = @config['main_contact']
@cc = @config['cc_contact']
end
def send
temp = Tempfile.new "rdiff_backup_log"
temp.puts @log
temp.close
zipped_log = "#{temp.path}.zip"
`zip #{zipped_log} #{temp.path}`
IO.popen(
'nail ' +
"-a '#{zipped_log}' " +
"-c 'address@hidden' " +
"-s 'address@hidden' 'address@hidden'",
'w'
) do |nail|
nail.puts @message
end
File.unlink zipped_log
end
end
def split_log log
marks = []
logs = []
log.each_with_index do |line, index|
if line =~ /Starting mirror|increment\ operation (.*) to (.*)/
marks << index
end
end
marks.size.times do |index|
first = marks[index]
last = marks[index+1] || log.size
logs << log[first..last].to_s
end
return logs
end
if __FILE__ == $0
parsers = []
input = ARGF.readlines
split_log(input).each do |log|
parsers << Parser.new(log)
end
ok_backups = 0
message = Message.new
parsers.each do |parser|
case parser.status
when BackupStatus::OK
ok_backups += 1
message.message += "Backup completed successfully\n"
message.message += parser.statistics_output
message.message += "\n\n"
when BackupStatus::BROKEN
message.message +="Backup did not complete\n"
message.message += parser.statistics_output
message.message += "\n\n"
when BackupStatus::ERROR
message.message += "Error running parsing script\n"
message.message += parser.error
message.message += "\n\n"
end
end
if parsers.size == 0
message.subject = 'Backup(s) unknown'
message.message = 'Status unknown, did not get a valid log.'
elsif ok_backups == parsers.size
message.subject = 'Backup(s) OK'
else
message.subject = 'Backup(s) failed'
end
message.subject += " (#{ok_backups}/#{parsers.size})"
message.log = input
message.send
end
--------------------------
Config file:
-------------------------
cat /etc/parse-rdiff-backup.conf
message:
main_contact: address@hidden
cc_contact: address@hidden
parser:
statistics:
Direction: Starting mirror|increment\ operation (.*)
Backup started: ^StartTime [\d\.]+ \((.*)\)$
#EndTime: ^EndTime (.*)$
#ElapsedTime: ^ElapsedTime (.*)$
Source files: ^SourceFiles (.*)$
Source files size: ^SourceFileSize (.*)$
#MirrorFiles: ^MirrorFiles (.*)$
#MirrorFileSize: ^MirrorFileSize (.*)$
#NewFiles: ^NewFiles (.*)$
New files size: ^NewFileSize (.*)$
#DeletedFiles: ^DeletedFiles (.*)$
Deleted files size: ^DeletedFileSize (.*)$
#ChangedFiles: ^ChangedFiles (.*)$
#Changed files size: ^ChangedFileSize (.*)$
#ChangedMirrorSize: ^ChangedMirrorSize (.*)$
#IncrementFiles: ^IncrementFiles (.*)$
#IncrementFileSize: ^IncrementFileSize (.*)$
Destination size change: ^TotalDestinationSizeChange \d+ \((.*)\)$
Errors: ^Errors (.*)$
Script is licensed under the GNU GPLv2 and Author is Christian Marie
while working for Solutions First.
thanks
dave
Greg Freemyer wrote:
> All,
>
> I've looked at the examples at
> http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem
> to address automated nightly scripts and error handling.
>
> Are there some more complex examples available?
>
> === Details
> I haven't used rdiff-backup for a couple of years and I never did have
> it integrated into a nightly cron script for production use.
>
> My needs have changed and I want to give it another shot.
>
> I'm backing up to a local directory on a dedicated backup disk so it is
> easy enough to add "rdiff-backup /src /backup" to my backup script.
>
> But what about catching errors?
>
> Seems like I should be sending output to a log file, grepping thru it
> and e-mailing it to myself if anything goes wrong.
>
> I've looked at the examples at
> http://www.nongnu.org/rdiff-backup/examples.html, but none of them seem
> to address this common need.
>
> Thanks
> Greg
> --
> Greg Freemyer
> The Norcross Group
> Forensics for the 21st Century
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> rdiff-backup-users mailing list at address@hidden
> http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki