Re: [rdiff-backup-users] Simple nightly script?

[David Kempe]

Hi

nail is a drop in replacement for mail.

http://heirloom.sourceforge.net/mailx.html



thats the correct link to it.

on my systems, apt-get install nail works :)

might be different on yours.



dave



"Greg Freemyer" <address@hidden> wrote:



> David,

> 

> I've been using your script (below) for a few months with great results,

> thanks.

> 

> I just upgraded my machine from SUSE 10.1 to 10.2.

> 

> Now I'm getting an error that "nail" Command Not Found.

> 

> I'm not very good at reading ruby, but it looks like the script is

> invoking nail as a seperate executable to do some kind of work.  Is it

> a standard tool I need to hunt down?

> 

> Any other idea what I need to do to get this working again?

> 

> Thanks

> Greg

> 

> On 10/19/06, David Kempe <address@hidden> wrote:

> > Hi Greg,

> > After much experimentation with various scripts, I am happy with this

> > latest rdiff-backup parser we have written. Script is written by

> > Christian Marie, who also wrote the rdiff-backup nagios plugin on

> > nagiosexchange.org.

> >

> > The script is designed to be passed on stdin the output of an

> > rdiff-backup job or multiple jobs with -v5 or greater and

> > --print-statistics both turned on. It can deal with many jobs passed in

> > ie: backup.sh | parse-rdiff-backup

> > where backup.sh might run three jobs.

> > check it out - you get a nicely formatted email that says things like

> > Backup(s) OK 3/3 in the subject, and a summary of the job like so:

> >

> > Status of backup(s) follow

> >

> > Backup completed successfully

> >         Direction: /home/sol1backup/cookie-data to
/var/backupdisk/cookieman

> >         Backup started: Tue Oct 17 23:00:06 2006

> >         Source files: 61248

> >         Source files size: 22290479056 (20.8 GB)

> >         New files size: 6224482 (5.94 MB)

> >         Deleted files size: 308229 (301 KB)

> >         Destination size change: 18.4 MB

> >         Errors: 79

> >

> >

> > Anway, Here is the script and config file. Read the source if you want

> > more info.... :)

> >

> > Script:

> > --------------------------------

> >

> > #!/usr/bin/env ruby

> > require 'yaml'

> > require 'tempfile'

> >

> > class Parser

> >    attr_reader :status

> >    DEFAULT_CONFIG = File::SEPARATOR +

> >                     File.join('etc', 'parse-rdiff-backup.conf')

> >    BACKUPOKCORRECT = /TotalDestinationSizeChange/

> >

> >    def initialize data, config = DEFAULT_CONFIG

> >      begin

> >        @log = data

> >        @config = YAML::load_file(config)['parser']

> >        @statistics_output = []

> >        @statistics = []

> >        @config['statistics'].each do |key,value|

> >          @statistics << SessionStatistic.new(key, value)

> >        end

> >        parse!

> >      rescue Exception => exception

> >        @status = BackupStatus::ERROR

> >        @error = "Error: #{exception.class} (#{exception.message})\n"

> >        @error += "Backtrace:\n"

> >        @error += exception.backtrace.join "\n"

> >      end

> >    end

> >

> >    def parse!

> >      @status = BackupStatus::BROKEN

> >      @log.each do |line|

> >        @statistics.each do |statistic|

> >          if line =~ /#{statistic.regex}/

> >            if $1

> >              @statistics_output << statistic.message + $1

> >            end

> >          end

> >        end

> >        if line =~ BACKUPOKCORRECT

> >          @status = BackupStatus::OK

> >        end

> >      end

> >    end

> >

> >    def statistics_output

> >        return @statistics_output.join("\n").gsub(/^/, "\t")

> >    end

> >

> >    def error

> >      return @error.gsub(/^/, "\t")

> >    end

> >

> >    private

> >    class SessionStatistic

> >      attr_accessor :message, :regex

> >      def initialize message, regex

> >        @regex = regex

> >        @message = message + ': '

> >      end

> >    end

> > end

> >

> > class BackupStatus

> >    OK = 0

> >    BROKEN = 1

> >    ERROR = 2

> > end

> >

> > class Message

> >    attr_accessor :subject, :message, :to, :cc, :log

> >    DEFAULT_CONFIG = File::SEPARATOR +

> >      File.join('etc', 'parse-rdiff-backup.conf')

> >

> >    def initialize config=DEFAULT_CONFIG

> >      @config = YAML::load_file(config)['message']

> >      @subject = 'Unknown status'

> >      @message = "Status of backup(s) follow\n\n"

> >      @to = @config['main_contact']

> >      @cc = @config['cc_contact']

> >    end

> >

> >    def send

> >      temp = Tempfile.new "rdiff_backup_log"

> >      temp.puts @log

> >      temp.close

> >

> >      zipped_log = "#{temp.path}.zip"

> >

> >      `zip #{zipped_log} #{temp.path}`

> >

> >      IO.popen(

> >        'nail ' +

> >        "-a '#{zipped_log}' " +

> >        "-c 'address@hidden' " +

> >        "-s 'address@hidden' 'address@hidden'",

> >        'w'

> >      ) do |nail|

> >        nail.puts @message

> >      end

> >

> >      File.unlink zipped_log

> >    end

> > end

> >

> > def split_log log

> >    marks = []

> >    logs = []

> >

> >    log.each_with_index do |line, index|

> >      if line =~ /Starting mirror|increment\ operation (.*) to (.*)/

> >        marks << index

> >      end

> >    end

> >

> >    marks.size.times do |index|

> >      first =  marks[index]

> >      last = marks[index+1] || log.size

> >

> >      logs << log[first..last].to_s

> >    end

> >    return logs

> > end

> >

> > if __FILE__ == $0

> >    parsers = []

> >

> >    input = ARGF.readlines

> >

> >    split_log(input).each do |log|

> >      parsers << Parser.new(log)

> >    end

> >

> >    ok_backups = 0

> >    message = Message.new

> >

> >    parsers.each do |parser|

> >      case parser.status

> >      when BackupStatus::OK

> >        ok_backups += 1

> >        message.message += "Backup completed successfully\n"

> >        message.message += parser.statistics_output

> >        message.message += "\n\n"

> >      when BackupStatus::BROKEN

> >        message.message +="Backup did not complete\n"

> >        message.message += parser.statistics_output

> >        message.message += "\n\n"

> >      when BackupStatus::ERROR

> >        message.message += "Error running parsing script\n"

> >        message.message += parser.error

> >        message.message += "\n\n"

> >      end

> >    end

> >

> >    if parsers.size == 0

> >      message.subject = 'Backup(s) unknown'

> >      message.message = 'Status unknown, did not get a valid log.'

> >    elsif ok_backups == parsers.size

> >      message.subject = 'Backup(s) OK'

> >    else

> >      message.subject = 'Backup(s) failed'

> >    end

> >    message.subject += " (#{ok_backups}/#{parsers.size})"

> >    message.log = input

> >

> >    message.send

> > end

> >

> > --------------------------

> > Config file:

> > -------------------------

> > cat /etc/parse-rdiff-backup.conf

> > message:

> >    main_contact: address@hidden

> >    cc_contact: address@hidden

> >

> > parser:

> >    statistics:

> >      Direction: Starting mirror|increment\ operation (.*)

> >      Backup started: ^StartTime [\d\.]+ \((.*)\)$

> >      #EndTime: ^EndTime (.*)$

> >      #ElapsedTime: ^ElapsedTime (.*)$

> >      Source files: ^SourceFiles (.*)$

> >      Source files size: ^SourceFileSize (.*)$

> >      #MirrorFiles: ^MirrorFiles (.*)$

> >      #MirrorFileSize: ^MirrorFileSize (.*)$

> >      #NewFiles: ^NewFiles (.*)$

> >      New files size: ^NewFileSize (.*)$

> >      #DeletedFiles: ^DeletedFiles (.*)$

> >      Deleted files size: ^DeletedFileSize (.*)$

> >      #ChangedFiles: ^ChangedFiles (.*)$

> >      #Changed files size: ^ChangedFileSize (.*)$

> >      #ChangedMirrorSize: ^ChangedMirrorSize (.*)$

> >      #IncrementFiles: ^IncrementFiles (.*)$

> >      #IncrementFileSize: ^IncrementFileSize (.*)$

> >      Destination size change: ^TotalDestinationSizeChange \d+ \((.*)\)$

> >      Errors: ^Errors (.*)$

> >

> >

> > Script is licensed under the GNU GPLv2 and Author is Christian Marie

> > while working for Solutions First.

> >

> > thanks

> >

> > dave

> >

> >

> >

> >

> >

> > Greg Freemyer wrote:

> > > All,

> > >

> > > I've looked at the examples at

> > > http://www.nongnu.org/rdiff-backup/examples.html, but none of them
seem

> > > to address automated nightly scripts and error handling.

> > >

> > > Are there some more complex examples available?

> > >

> > > === Details

> > > I haven't used rdiff-backup for a couple of years and I never did have

> > > it integrated into a nightly cron script for production use.

> > >

> > > My needs have changed and I want to give it another shot.

> > >

> > > I'm backing up to a local directory on a dedicated backup disk so it
is

> > > easy enough to add "rdiff-backup /src /backup" to my backup script.

> > >

> > > But what about catching errors?

> > >

> > > Seems like I should be sending output to a log file, grepping thru it

> > > and e-mailing it to myself if anything goes wrong.

> > >

> > > I've looked at the examples at

> > > http://www.nongnu.org/rdiff-backup/examples.html, but none of them
seem

> > > to address this common need.

> > >

> > > Thanks

> > > Greg

> > > --

> > > Greg Freemyer

> > > The Norcross Group

> > > Forensics for the 21st Century

> > >

> > >

> > >
------------------------------------------------------------------------

> > >

> > > _______________________________________________

> > > rdiff-backup-users mailing list at address@hidden

> > > http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users

> > > Wiki URL:

> http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

> >

> >

>