ISPbackup

From ISPWiki

Jump to: navigation, search

ISPbackup is a program that can be used for backing up files. It was supposed to run on a separate machine that would perform main operations not to overload other servers.

Contents

General notes

ISPbackup functionality can be divided into three parts:

  1. Client part collects lists of files and included data, if necessary.
  2. Master machine (Server) part backs up the files based on the data from Client part.
  3. The third part extracts files from backups.

Client and Server interact through one pipe.

Client sends a list of files into the pipe. If there is a non-zero file, it waits for input replay. If there is "data\n", the file information will be followed by its content:

[<N = block size (4 bytes )> <N data bytes>] ...

The last block should have zero size. If the data size differs from the specified file size, its size will be modified.

The server receives a non-zero file and locates the previous file of the same size and modification day. If it is found, the link to this file from the previous backup will be included into the listing or this file will be taken from this backup and repacked. The server uses gzip to pack files and saves them in the storage file. A list of files and their attributes are saved separately. Files offset in the storage file cannot exceed MAXOFFSET bytes. If there are more data, a new file is created. Files that exceedMINSIZE are packed separately. If the last backup uses the entire storage file, a hard link to this file is created in the newly created backup. If a part of the file is used, the listing will include a link to this file from the previous backup (if it is allowed) or repacked into a new backup from the old one.

General parameters

-g <file>      Path to the configuration file. This file may include the following parameters:  exclude, archives,
               minsize, maxoffset and timeout. The file should not contain blank fields, extra spaces and comments.
               (By default /usr/local/ispmgr/etc/ispbackup.conf) 
               the exclude parameter is a regular expression
               the max_dirsize parameter excludes large directories (with a large number if files)
               Example:
               exclude=^\./tmp[abc]$
               archives=.tar.gz
               archives=.rar
               minsize=1048576
               maxoffset=100000000
               max_dirsize=2000000
               timeout=60


-C <directory>   changes the program's directory. The directory will be changed immediately, that is why the part of the parameters' list in which it was used is very important. 
-D <file>     Path to the file, where debugging information will be stored. (stderr is used as default.)
-d <level>   Debug level. 0 - minimum (fatal errors only), 9 - maximum. (1 as default.)
-o <file>     Use this file instead of stdout
-i <file>     Use this file instead of stdin
-u <seconds>    timeout for reading operation. 0 - disables timeout (60 as default.)

Client parameters

-c             Run the command in Client mode (to receive file and data listing)
-f             Do not wait for reply from the server, even if the file size is more than 0. (to receive file and database listing).  
-x <path>      Exclude the files which names begin with this line from the listing. (exclude in the configuration file)
               Parameter can be used several times

Server parameters

-s <file>      Run the command in Server mode. File is a path to the command pipe (should be used as input pipe for Client)
           
-t <ext>       Listing files' extension (.lst is used as default.)
-l <path>      Path  listing  from the previous backup. If it is a directory, all its .lst  files will be checked. The latest file will be used.  
-n <file>      Path to the first file in the storage. To receive a following file name, the figures in the name will increase by 1. E.g.:
               0000/99 -> 0001/00 . If "//" is used in the path,
              no links to previous storages will be made, if the beginning of their path do not coincide with the beginning of this path.
-S <size>      maxoffset. Maximum file offset in the storage file. 
-m <size>      minsize. Minimum file size, that can be saved as a separate file. 
-A <ext>       Extensions of files that should not to be packed (archive in the configuration files).
-R <file>      Delete an old backup. File is a path to listing. No links will be made to storages file of this backup. The parameter can be used separately to free space securely.  
-r <count>     count backups only (including a new one). All the old backups will be deleted (see. -R)
-T             Check storage mode. No data from previous backups will be used, but all of the files, which names, modification date and size coincide, will be equal.

Extract from backup

If neither - c, nor - s is used, the program will run in the recovery mode. The first parameter is path to listing.

-F             Do not stop in case of errors. 
-B <dir>      Add the specified line into storage file names. It can be either path or URL. 
-X <file>      Data extracted from backup will be packed into the tar archive named file. 
-z             Tar archive (see. -X) will be packed with gzip

Examples

mkfifo cmd
cat cmd | ssh server '/usr/local/ispmgr/sbin/ispbackup -c -C /vs/private/1.1.1.1 .' |
      /usr/local/ispmgr/sbin/ispbackup -s cmd > 2009-01-01.lst

Make a backup of the directory /vs/private/1.1.1.1 on the server server. Listing will be stored into the file 2009-01-01.lst. Storage files will be stored into the current directory named 00000001, 00000002 , etc.

/usr/local/ispmgr/sbin/ispbackup -X backup.tgz -z 2009-01-01.lst ./home

Extract a home directory from the backup with 2009-01-01.lst listing and pack data into tar archive backup.tgz

Settings

VDSmanager distribution has the vdsbackup-remote-new.sh cript. To create a backup, it should run on the backup server. This script reads settings from the ~/etc/vdsbackup-remote.conf file:

SERVERS="ip1 ip2 ip3"     # list of the server addresses to  backup 
BACKUP_DIR="/backup"      # directory where backups will be stored 
BACKUP_KEY="~/etc/backup" # ssh key for root access to the SERVERS server 

Backing up will be executed from all the servers simultaneously. You need to copy the ispbackup binary file to /root/bin/backup on the backup server.

Was this helpful? Yes | No
Personal tools