Re: [Xen-devel] [PATCH] sqlite_backup.pl: Create backup of DB using locks to avoid problems with queuerunner.

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2014-05-19 at 11:21 +0200, Birintxo SÃnchez wrote:
> Hi Ian,
> 
> I forgot to add the runes for the cronjob:
> 
> # SQLite backup for xen-devel-bugs DB. Runs everyday at 2:07 to get DB backup
> # ready for offsite backup by Citrix (run at 3:30 am)
> 7 2 * * *       /usr/bin/env
> PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/
> /srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl

Thanks, I have applied your patch, modifying it to add the new script to
the top-level Makefile and with the following additional changeset on
top.

I also ran the backups a couple of times by hand and verified they
worked, the crontab has been updated.

Thanks!

Ian.

commit 0ade7d795d21ab4729bda6c07fe9f4f169bd7c5f
Author: Ian Campbell <ian.campbell@xxxxxxxxxx>
Date:   Mon May 19 11:45:42 2014 +0100

    xen-bugs: Configure database backups

diff --git a/config/examples/xen-bugs.xenproject.org/README 
b/config/examples/xen-bugs.xenproject.org/README
index f49d934..cf989d0 100644
--- a/config/examples/xen-bugs.xenproject.org/README
+++ b/config/examples/xen-bugs.xenproject.org/README
@@ -15,10 +15,12 @@ $ make
 # chmod 1777 /srv/xen-devel-bugs/var/lock
 
 # mkdir /srv/xen-devel-bugs/var/raw /srv/xen-devel-bugs/var/run \
-       /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming
+       /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming \
+       /srv/xen-devel-bugs/var/backups
 # chown xen-devel-bugs:xen-devel-bugs \
        /srv/xen-devel-bugs/var/raw /srv/xen-devel-bugs/var/run \
-       /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming
+       /srv/xen-devel-bugs/var/spool/ /srv/xen-devel-bugs/var/spool/incoming \
+       /srv/xen-devel-bugs/var/backups
 
 # cp config/examples/xen-bugs.xenproject.org/emesinae.conf 
/srv/xen-devel-bugs/etc/
 
@@ -91,8 +93,8 @@ cron Configuration
 ==================
 
 # mkdir /srv/xen-devel-bugs/var/log
-# touch /srv/xen-devel-bugs/var/log/queuerunner.log
-# chown xen-devel-bugs:xen-devel-bugs 
/srv/xen-devel-bugs/var/log/queuerunner.log
+# touch /srv/xen-devel-bugs/var/log/{backups,queuerunner}.log
+# chown xen-devel-bugs:xen-devel-bugs 
/srv/xen-devel-bugs/var/log/{backups,queuerunner}.log
 
 # crontab -e -u xen-devel-bugs
 8<-----------------------------
@@ -100,5 +102,9 @@ SHELL=/bin/sh
 
 # Process queue every 15 minutes
 */15 * *   *   *     /usr/bin/env 
PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ 
/srv/xen-devel-bugs/lib/emesinae/queuerunner.pl 
>>/srv/xen-devel-bugs/var/log/queuerunner.log
+
+# Backup everyday at 2:07 ready for offsite backup by Citrix (run at 3:30 am)
+7    2 *   *   *     /usr/bin/env 
PERLLIB=/srv/xen-devel-bugs/share/perl/5.14.2/ 
/srv/xen-devel-bugs/lib/emesinae/sqlite_backup.pl 
>>/srv/xen-devel-bugs/var/log/backups.log
+
 8<-----------------------------
 
diff --git a/config/examples/xen-bugs.xenproject.org/emesinae.conf 
b/config/examples/xen-bugs.xenproject.org/emesinae.conf
index bf6fcf8..beefa3e 100644
--- a/config/examples/xen-bugs.xenproject.org/emesinae.conf
+++ b/config/examples/xen-bugs.xenproject.org/emesinae.conf
@@ -72,3 +72,6 @@ $c{ControlBlacklistPath} = 
"/srv/xen-devel-bugs/etc/control.blacklist";
 # Severity levels, in decending order of criticality
 @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
 $c{DefaultSeverity} = "normal";
+
+# Backup configuration
+$c{BackupDir}   = "/srv/xen-devel-bugs/var/backups/";

> 
> Cheers,
> 
> Birin
> 
> 
> On 19 May 2014 10:46, Birin Sanchez <birin.sanchez@xxxxxxxxxx> wrote:
> >
> > ---
> >  config/emesinae.conf               |  3 +++
> >  config/examples/test/emesinae.conf |  3 +++
> >  scripts/sqlite_backup.pl           | 39 
> > ++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 45 insertions(+)
> >  create mode 100755 scripts/sqlite_backup.pl
> >
> > diff --git a/config/emesinae.conf b/config/emesinae.conf
> > index 0aa1b3f..dd58c94 100644
> > --- a/config/emesinae.conf
> > +++ b/config/emesinae.conf
> > @@ -76,3 +76,6 @@ $c{ControlBlacklistPath} = 
> > "/etc/emesinae/control.blacklist";
> >  # Severity levels, in decending order of criticality
> >  @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
> >  $c{DefaultSeverity} = "normal";
> > +
> > +# Backup configuration
> > +$c{BackupDir}   = "/var/backups/";
> > diff --git a/config/examples/test/emesinae.conf 
> > b/config/examples/test/emesinae.conf
> > index 528dcb6..11e4fad 100644
> > --- a/config/examples/test/emesinae.conf
> > +++ b/config/examples/test/emesinae.conf
> > @@ -70,3 +70,6 @@ $c{ControlBlacklistPath} = 
> > "/srv/test/etc/control.blacklist";
> >  # Severity levels, in decending order of criticality
> >  @{ $c{SeverityLevels} } = qw/blocker critical normal wishlist/;
> >  $c{DefaultSeverity} = "normal";
> > +
> > +# Backup configuration
> > +$c{BackupDir}   = "/srv/test/var/backups/";
> > diff --git a/scripts/sqlite_backup.pl b/scripts/sqlite_backup.pl
> > new file mode 100755
> > index 0000000..557bc59
> > --- /dev/null
> > +++ b/scripts/sqlite_backup.pl
> > @@ -0,0 +1,39 @@
> > +#!/usr/bin/perl
> > +#
> > +# Creates a backup of the current DB in $c{BackupDir}
> > +# using locks to avoid race conditions with queuerunner
> > +
> > +use strict;
> > +use warnings;
> > +
> > +use Emesinae::Common;
> > +use Emesinae::Paths;
> > +use File::Copy;
> > +use File::Basename;
> > +
> > +readconfig();
> > +
> > +chdir( $c{BackupDir} ) || die("chdir to dir $c{BackupDir}: $!");
> > +
> > +
> > +my $lock = subsyslock('queuerunner');
> > +
> > +my $BackupDB = $c{BackupDir} . basename($c{DB});
> > +my $BackupDBxz = $c{BackupDir} . basename($c{DB}) . ".xz";
> > +my $OldBackupDBxz = $c{BackupDir} . basename($c{DB}) . ".old.xz";
> > +
> > +if ( -f $BackupDBxz ) {
> > +    move($BackupDBxz, $OldBackupDBxz) or die ("move failed: $!");
> > +}
> > +
> > +my $dbh = DBI->connect("dbi:SQLite:dbname=$c{DB}","","");
> > +$dbh->sqlite_backup_to_file( $BackupDB );
> > +undef $dbh;
> > +
> > +my @args = ("xz", "$BackupDB");
> > +system(@args) == 0
> > +    or die ("xz compress failed: $?");
> > +
> > +subsysunlock($lock);
> > +
> > +exit(0)
> > --
> > 1.8.3.2
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxx
> > http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel