Configuration File Format

Configuration File Format
Prev	Chapter 5. Configuration	Next

Sample Configuration File

Both the Python source distribution and the Debian package come with a sample configuration file. The Debian package includes its sample in /usr/share/doc/cedar-backup2/examples/cback.conf.sample.

This is a sample configuration file similar to the one provided in the source package. Documentation below provides more information about each of the individual configuration sections.

<?xml version="1.0"?>
<cb_config>
   <reference>
      <author>Kenneth J. Pronovici</author>
      <revision>1.3</revision>
      <description>Sample</description>
   </reference>
   <options>
      <starting_day>tuesday</starting_day>
      <working_dir>/opt/backup/tmp</working_dir>
      <backup_user>backup</backup_user>
      <backup_group>group</backup_group>
      <rcp_command>/usr/bin/scp -B</rcp_command>
   </options>
   <peers>
      <peer>
         <name>debian</name>
         <type>local</type>
         <collect_dir>/opt/backup/collect</collect_dir>
      </peer>
   </peers>
   <collect>
      <collect_dir>/opt/backup/collect</collect_dir>
      <collect_mode>daily</collect_mode>
      <archive_mode>targz</archive_mode>
      <ignore_file>.cbignore</ignore_file>
      <dir>
         <abs_path>/etc</abs_path>
         <collect_mode>incr</collect_mode>
      </dir>
      <file>
         <abs_path>/home/root/.profile</abs_path>
         <collect_mode>weekly</collect_mode>
      </file>
   </collect>
   <stage>
      <staging_dir>/opt/backup/staging</staging_dir>
   </stage>
   <store>
      <source_dir>/opt/backup/staging</source_dir>
      <media_type>cdrw-74</media_type>
      <device_type>cdwriter</device_type>
      <target_device>/dev/cdrw</target_device>
      <target_scsi_id>0,0,0</target_scsi_id>
      <drive_speed>4</drive_speed>
      <check_data>Y</check_data>
      <check_media>Y</check_media>
      <warn_midnite>Y</warn_midnite>
   </store>
   <purge>
      <dir>
         <abs_path>/opt/backup/stage</abs_path>
         <retain_days>7</retain_days>
      </dir>
      <dir>
         <abs_path>/opt/backup/collect</abs_path>
         <retain_days>0</retain_days>
      </dir>
   </purge>
</cb_config>

Reference Configuration

The reference configuration section contains free-text elements that exist only for reference.. The section itself is required, but the individual elements may be left blank if desired.

This is an example reference configuration section:

<reference>
   <author>Kenneth J. Pronovici</author>
   <revision>Revision 1.3</revision>
   <description>Sample</description>
   <generator>Yet to be Written Config Tool (tm)</description>
</reference>

The following elements are part of the reference configuration section:

author

Author of the configuration file.

Restrictions: None

revision

Revision of the configuration file.

Restrictions: None

description

Description of the configuration file.

Restrictions: None

generator

Tool that generated the configuration file, if any.

Restrictions: None

Options Configuration

The options configuration section contains configuration options that are not specific to any one action.

This is an example options configuration section:

<options>
   <starting_day>tuesday</starting_day>
   <working_dir>/opt/backup/tmp</working_dir>
   <backup_user>backup</backup_user>
   <backup_group>backup</backup_group>
   <rcp_command>/usr/bin/scp -B</rcp_command>
   <rsh_command>/usr/bin/ssh</rsh_command>
   <cback_command>/usr/bin/cback</cback_command>
   <managed_actions>collect, purge</managed_actions>
   <override>
      <command>cdrecord</command>
      <abs_path>/opt/local/bin/cdrecord</abs_path>
   </override>
   <override>
      <command>mkisofs</command>
      <abs_path>/opt/local/bin/mkisofs</abs_path>
   </override>
   <pre_action_hook>
      <action>collect</action>
      <command>echo "I AM A PRE-ACTION HOOK RELATED TO COLLECT"</command>
   </pre_action_hook>
   <post_action_hook>
      <action>collect</action>
      <command>echo "I AM A POST-ACTION HOOK RELATED TO COLLECT"</command>
   </post_action_hook>
</options>

The following elements are part of the options configuration section:

starting_day

Day that starts the week.

Cedar Backup is built around the idea of weekly backups. The starting day of week is the day that media will be rebuilt from scratch and that incremental backup information will be cleared.

Restrictions: Must be a day of the week in English, i.e. monday, tuesday, etc. The validation is case-sensitive.

working_dir

Working (temporary) directory to use for backups.

This directory is used for writing temporary files, such as tar file or ISO filesystem images as they are being built. It is also used to store day-to-day information about incremental backups.

The working directory should contain enough free space to hold temporary tar files (on a client) or to build an ISO filesystem image (on a master).

Restrictions: Must be an absolute path

backup_user

Effective user that backups should run as.

This user must exist on the machine which is being configured and should not be root (although that restriction is not enforced).

This value is also used as the default remote backup user for remote peers.

Restrictions: Must be non-empty

backup_group

Effective group that backups should run as.

This group must exist on the machine which is being configured, and should not be root or some other “powerful” group (although that restriction is not enforced).

Restrictions: Must be non-empty

rcp_command

Default rcp-compatible copy command for staging.

The rcp command should be the exact command used for remote copies, including any required options. If you are using scp, you should pass it the -B option, so scp will not ask for any user input (which could hang the backup). A common example is something like /usr/bin/scp -B.

This value is used as the default value for all remote peers. Technically, this value is not needed by clients, but we require it for all config files anyway.

Restrictions: Must be non-empty

rsh_command

Default rsh-compatible command to use for remote shells.

The rsh command should be the exact command used for remote shells, including any required options.

This value is used as the default value for all managed clients. It is optional, because it is only used when executing actions on managed clients. However, each managed client must either be able to read the value from options configuration or must set the value explicitly.

Restrictions: Must be non-empty

cback_command

Default cback-compatible command to use on managed remote clients.

The cback command should be the exact command used for for executing cback on a remote managed client, including any required command-line options. Do not list any actions in the command line, and do not include the --full command-line option.

This value is used as the default value for all managed clients. It is optional, because it is only used when executing actions on managed clients. However, each managed client must either be able to read the value from options configuration or must set the value explicitly.

Note: if this command-line is complicated, it is often better to create a simple shell script on the remote host to encapsulate all of the options. Then, just reference the shell script in configuration.

Restrictions: Must be non-empty

managed_actions

Default set of actions that are managed on remote clients.

This is a comma-separated list of actions that the master will manage on behalf of remote clients. Typically, it would include only collect-like actions and purge.

This value is used as the default value for all managed clients. It is optional, because it is only used when executing actions on managed clients. However, each managed client must either be able to read the value from options configuration or must set the value explicitly.

Restrictions: Must be non-empty.

override

Command to override with a customized path.

This is a subsection which contains a command to override with a customized path. This functionality would be used if root's $PATH does not include a particular required command, or if there is a need to use a version of a command that is different than the one listed on the $PATH. Most users will only use this section when directed to, in order to fix a problem.

This section is optional, and can be repeated as many times as necessary.

This subsection must contain the following two fields:

command

Name of the command to be overridden, i.e. “cdrecord”.

Restrictions: Must be a non-empty string.

abs_path

The absolute path where the overridden command can be found.

Restrictions: Must be an absolute path.

pre_action_hook

Hook configuring a command to be executed before an action.

This is a subsection which configures a command to be executed immediately before a named action. It provides a way for administrators to associate their own custom functionality with standard Cedar Backup actions or with arbitrary extensions.

This section is optional, and can be repeated as many times as necessary.

This subsection must contain the following two fields:

action

Name of the Cedar Backup action that the hook is associated with. The action can be a standard backup action (collect, stage, etc.) or can be an extension action. No validation is done to ensure that the configured action actually exists.

Restrictions: Must be a non-empty string.

command

Name of the command to be executed. This item can either specify the path to a shell script of some sort (the recommended approach) or can include a complete shell command.

Note: if you choose to provide a complete shell command rather than the path to a script, you need to be aware of some limitations of Cedar Backup's command-line parser. You cannot use a subshell (via the `command` or $(command) syntaxes) or any shell variable in your command line. Additionally, the command-line parser only recognizes the double-quote character (") to delimit groupings or strings on the command-line. The bottom line is, you are probably best off writing a shell script of some sort for anything more sophisticated than very simple shell commands.

Restrictions: Must be a non-empty string.

post_action_hook

Hook configuring a command to be executed after an action.

This is a subsection which configures a command to be executed immediately after a named action. It provides a way for administrators to associate their own custom functionality with standard Cedar Backup actions or with arbitrary extensions.

This section is optional, and can be repeated as many times as necessary.

This subsection must contain the following two fields:

action

Name of the Cedar Backup action that the hook is associated with. The action can be a standard backup action (collect, stage, etc.) or can be an extension action. No validation is done to ensure that the configured action actually exists.

Restrictions: Must be a non-empty string.

command

Name of the command to be executed. This item can either specify the path to a shell script of some sort (the recommended approach) or can include a complete shell command.

Note: if you choose to provide a complete shell command rather than the path to a script, you need to be aware of some limitations of Cedar Backup's command-line parser. You cannot use a subshell (via the `command` or $(command) syntaxes) or any shell variable in your command line. Additionally, the command-line parser only recognizes the double-quote character (") to delimit groupings or strings on the command-line. The bottom line is, you are probably best off writing a shell script of some sort for anything more sophisticated than very simple shell commands.

Restrictions: Must be a non-empty string.

Peers Configuration

The peers configuration section contains a list of the peers managed by a master. This section is only required on a master.

This is an example peers configuration section:

<peers>
   <peer>
      <name>machine1</name>
      <type>local</type>
      <collect_dir>/opt/backup/collect</collect_dir>
   </peer>
   <peer>
      <name>machine2</name>
      <type>remote</type>
      <backup_user>backup</backup_user>
      <collect_dir>/opt/backup/collect</collect_dir>
      <ignore_failures>all</ignore_failures>
   </peer>
   <peer>
      <name>machine3</name>
      <type>remote</type>
      <managed>Y</managed>
      <backup_user>backup</backup_user>
      <collect_dir>/opt/backup/collect</collect_dir>
      <rcp_command>/usr/bin/scp</rcp_command>
      <rsh_command>/usr/bin/ssh</rsh_command>
      <cback_command>/usr/bin/cback</cback_command>
      <managed_actions>collect, purge</managed_actions>
   </peer>
</peers>

The following elements are part of the peers configuration section:

peer (local version)

Local client peer in a backup pool.

This is a subsection which contains information about a specific local client peer managed by a master.

This section can be repeated as many times as is necessary. At least one remote or local peer must be configured.

The local peer subsection must contain the following fields:

name

Name of the peer, typically a valid hostname.

For local peers, this value is only used for reference. However, it is good practice to list the peer's hostname here, for consistency with remote peers.

Restrictions: Must be non-empty, and unique among all peers.

type

Type of this peer.

This value identifies the type of the peer. For a local peer, it must always be local.

Restrictions: Must be local.

collect_dir

Collect directory to stage from for this peer.

The master will copy all files in this directory into the appropriate staging directory. Since this is a local peer, the directory is assumed to be reachable via normal filesystem operations (i.e. cp).

Restrictions: Must be an absolute path.

ignore_failures

Ignore failure mode for this peer

The ignore failure mode indicates whether “not ready to be staged” errors should be ignored for this peer. This option is intended to be used for peers that are up only intermittently, to cut down on the number of error emails received by the Cedar Backup administrator.

The "none" mode means that all errors will be reported. This is the default behavior. The "all" mode means to ignore all failures. The "weekly" mode means to ignore failures for a start-of-week or full backup. The "daily" mode means to ignore failures for any backup that is not either a full backup or a start-of-week backup.

Restrictions: If set, must be one of "none", "all", "daily", or "weekly".

peer (remote version)

Remote client peer in a backup pool.

This is a subsection which contains information about a specific remote client peer managed by a master. A remote peer is one which can be reached via an rsh-based network call.

This section can be repeated as many times as is necessary. At least one remote or local peer must be configured.

The remote peer subsection must contain the following fields:

name

Hostname of the peer.

For remote peers, this must be a valid DNS hostname or IP address which can be resolved during an rsh-based network call.

Restrictions: Must be non-empty, and unique among all peers.

type

Type of this peer.

This value identifies the type of the peer. For a remote peer, it must always be remote.

Restrictions: Must be remote.

managed

Indicates whether this peer is managed.

A managed peer (or managed client) is a peer for which the master manages all of the backup activites via a remote shell.

This field is optional. If it doesn't exist, then N will be assumed.

Restrictions: Must be a boolean (Y or N).

collect_dir

Collect directory to stage from for this peer.

The master will copy all files in this directory into the appropriate staging directory. Since this is a remote peer, the directory is assumed to be reachable via rsh-based network operations (i.e. scp or the configured rcp command).

Restrictions: Must be an absolute path.

ignore_failures

Ignore failure mode for this peer

The ignore failure mode indicates whether “not ready to be staged” errors should be ignored for this peer. This option is intended to be used for peers that are up only intermittently, to cut down on the number of error emails received by the Cedar Backup administrator.

The "none" mode means that all errors will be reported. This is the default behavior. The "all" mode means to ignore all failures. The "weekly" mode means to ignore failures for a start-of-week or full backup. The "daily" mode means to ignore failures for any backup that is not either a full backup or a start-of-week backup.

Restrictions: If set, must be one of "none", "all", "daily", or "weekly".

backup_user

Name of backup user on the remote peer.

This username will be used when copying files from the remote peer via an rsh-based network connection.

This field is optional. if it doesn't exist, the backup will use the default backup user from the options section.

Restrictions: Must be non-empty.

rcp_command

The rcp-compatible copy command for this peer.

The rcp command should be the exact command used for remote copies, including any required options. If you are using scp, you should pass it the -B option, so scp will not ask for any user input (which could hang the backup). A common example is something like /usr/bin/scp -B.

This field is optional. if it doesn't exist, the backup will use the default rcp command from the options section.

Restrictions: Must be non-empty.

rsh_command

The rsh-compatible command for this peer.

The rsh command should be the exact command used for remote shells, including any required options.

This value only applies if the peer is managed.

This field is optional. if it doesn't exist, the backup will use the default rsh command from the options section.

Restrictions: Must be non-empty

cback_command

The cback-compatible command for this peer.

The cback command should be the exact command used for for executing cback on the peer as part of a managed backup. This value must include any required command-line options. Do not list any actions in the command line, and do not include the --full command-line option.

This value only applies if the peer is managed.

This field is optional. if it doesn't exist, the backup will use the default cback command from the options section.

Note: if this command-line is complicated, it is often better to create a simple shell script on the remote host to encapsulate all of the options. Then, just reference the shell script in configuration.

Restrictions: Must be non-empty

managed_actions

Set of actions that are managed for this peer.

This is a comma-separated list of actions that the master will manage on behalf this peer. Typically, it would include only collect-like actions and purge.

This value only applies if the peer is managed.

This field is optional. if it doesn't exist, the backup will use the default list of managed actions from the options section.

Restrictions: Must be non-empty.

Collect Configuration

The collect configuration section contains configuration options related the the collect action. This section contains a variable number of elements, including an optional exclusion section and a repeating subsection used to specify which directories and/or files to collect. You can also configure an ignore indicator file, which lets users mark their own directories as not backed up.

Using a Link Farm

Sometimes, it's not very convenient to list directories one by one in the Cedar Backup configuration file. For instance, when backing up your home directory, you often exclude as many directories as you include. The ignore file mechanism can be of some help, but it still isn't very convenient if there are a lot of directories to ignore (or if new directories pop up all of the time).

In this situation, one option is to use a link farm rather than listing all of the directories in configuration. A link farm is a directory that contains nothing but a set of soft links to other files and directories. Normally, Cedar Backup does not follow soft links, but you can override this behavior for individual directories using the link_depth and dereference options (see below).

When using a link farm, you still have to deal with each backed-up directory individually, but you don't have to modify configuration. Some users find that this works better for them.

In order to actually execute the collect action, you must have configured at least one collect directory or one collect file. However, if you are only including collect configuration for use by an extension, then it's OK to leave out these sections. The validation will take place only when the collect action is executed.

This is an example collect configuration section:

<collect>
   <collect_dir>/opt/backup/collect</collect_dir>
   <collect_mode>daily</collect_mode>
   <archive_mode>targz</archive_mode>
   <ignore_file>.cbignore</ignore_file>
   <exclude>
      <abs_path>/etc</abs_path>
      <pattern>.*\.conf</pattern>
   </exclude>
   <file>
      <abs_path>/home/root/.profile</abs_path>
   </file>
   <dir>
      <abs_path>/etc</abs_path>
   </dir>
   <dir>
      <abs_path>/var/log</abs_path>
      <collect_mode>incr</collect_mode>
   </dir>
   <dir>
      <abs_path>/opt</abs_path>
      <collect_mode>weekly</collect_mode>
      <exclude>
         <abs_path>/opt/large</abs_path>
         <rel_path>backup</rel_path>
         <pattern>.*tmp</pattern>
      </exclude>
   </dir>
</collect>

The following elements are part of the collect configuration section:

collect_dir

Directory to collect files into.

On a client, this is the directory which tarfiles for individual collect directories are written into. The master then stages files from this directory into its own staging directory.

This field is always required. It must contain enough free space to collect all of the backed-up files on the machine in a compressed form.

Restrictions: Must be an absolute path

collect_mode

Default collect mode.

The collect mode describes how frequently a directory is backed up. See the section called “The Collect Action” (in Chapter 2, Basic Concepts) for more information.

This value is the collect mode that will be used by default during the collect process. Individual collect directories (below) may override this value. If all individual directories provide their own value, then this default value may be omitted from configuration.

Note: if your backup device does not suppport multisession discs, then you should probably use the daily collect mode to avoid losing data.

Restrictions: Must be one of daily, weekly or incr.

archive_mode

Default archive mode for collect files.

The archive mode maps to the way that a backup file is stored. A value tar means just a tarfile (file.tar); a value targz means a gzipped tarfile (file.tar.gz); and a value tarbz2 means a bzipped tarfile (file.tar.bz2)

This value is the archive mode that will be used by default during the collect process. Individual collect directories (below) may override this value. If all individual directories provide their own value, then this default value may be omitted from configuration.

Restrictions: Must be one of tar, targz or tarbz2.

ignore_file

Default ignore file name.

The ignore file is an indicator file. If it exists in a given directory, then that directory will be recursively excluded from the backup as if it were explicitly excluded in configuration.

The ignore file provides a way for individual users (who might not have access to Cedar Backup configuration) to control which of their own directories get backed up. For instance, users with a ~/tmp directory might not want it backed up. If they create an ignore file in their directory (e.g. ~/tmp/.cbignore), then Cedar Backup will ignore it.

This value is the ignore file name that will be used by default during the collect process. Individual collect directories (below) may override this value. If all individual directories provide their own value, then this default value may be omitted from configuration.

Restrictions: Must be non-empty

recursion_level

Recursion level to use when collecting directories.

This is an integer value that Cedar Backup will consider when generating archive files for a configured collect directory.

Normally, Cedar Backup generates one archive file per collect directory. So, if you collect /etc you get etc.tar.gz. Most of the time, this is what you want. However, you may sometimes wish to generate multiple archive files for a single collect directory.

The most obvious example is for /home. By default, Cedar Backup will generate home.tar.gz. If instead, you want one archive file per home directory you can set a recursion level of 1. Cedar Backup will generate home-user1.tar.gz, home-user2.tar.gz, etc.

Higher recursion levels (2, 3, etc.) are legal, and it doesn't matter if the configured recursion level is deeper than the directory tree that is being collected. You can use a negative recursion level (like -1) to specify an infinite level of recursion. This will exhaust the tree in the same way as if the recursion level is set too high.

This field is optional. if it doesn't exist, the backup will use the default recursion level of zero.

Restrictions: Must be an integer.

exclude

List of paths or patterns to exclude from the backup.

This is a subsection which contains a set of absolute paths and patterns to be excluded across all configured directories. For a given directory, the set of absolute paths and patterns to exclude is built from this list and any list that exists on the directory itself. Directories cannot override or remove entries that are in this list, however.

This section is optional, and if it exists can also be empty.

The exclude subsection can contain one or more of each of the following fields:

abs_path

An absolute path to be recursively excluded from the backup.

If a directory is excluded, then all of its children are also recursively excluded. For instance, a value /var/log/apache would exclude any files within /var/log/apache as well as files within other directories under /var/log/apache.

This field can be repeated as many times as is necessary.

Restrictions: Must be an absolute path.

pattern

A pattern to be recursively excluded from the backup.

The pattern must be a Python regular expression. ^[19] It is assumed to be bounded at front and back by the beginning and end of the string (i.e. it is treated as if it begins with ^ and ends with $).

If the pattern causes a directory to be excluded, then all of the children of that directory are also recursively excluded. For instance, a value .*apache.* might match the /var/log/apache directory. This would exclude any files within /var/log/apache as well as files within other directories under /var/log/apache.

This field can be repeated as many times as is necessary.

Restrictions: Must be non-empty

file

A file to be collected.

This is a subsection which contains information about a specific file to be collected (backed up).

This section can be repeated as many times as is necessary. At least one collect directory or collect file must be configured when the collect action is executed.

The collect file subsection contains the following fields:

abs_path

Absolute path of the file to collect.

Restrictions: Must be an absolute path.

collect_mode

Collect mode for this file

The collect mode describes how frequently a file is backed up. See the section called “The Collect Action” (in Chapter 2, Basic Concepts) for more information.

This field is optional. If it doesn't exist, the backup will use the default collect mode.

Note: if your backup device does not suppport multisession discs, then you should probably confine yourself to the daily collect mode, to avoid losing data.

Restrictions: Must be one of daily, weekly or incr.

archive_mode

Archive mode for this file.

The archive mode maps to the way that a backup file is stored. A value tar means just a tarfile (file.tar); a value targz means a gzipped tarfile (file.tar.gz); and a value tarbz2 means a bzipped tarfile (file.tar.bz2)

This field is optional. if it doesn't exist, the backup will use the default archive mode.

Restrictions: Must be one of tar, targz or tarbz2.

dir

A directory to be collected.

This is a subsection which contains information about a specific directory to be collected (backed up).

This section can be repeated as many times as is necessary. At least one collect directory or collect file must be configured when the collect action is executed.

The collect directory subsection contains the following fields:

abs_path

Absolute path of the directory to collect.

The path may be either a directory, a soft link to a directory, or a hard link to a directory. All three are treated the same at this level.

The contents of the directory will be recursively collected. The backup will contain all of the files in the directory, as well as the contents of all of the subdirectories within the directory, etc.

Soft links within the directory are treated as files, i.e. they are copied verbatim (as a link) and their contents are not backed up.

Restrictions: Must be an absolute path.

collect_mode

Collect mode for this directory

The collect mode describes how frequently a directory is backed up. See the section called “The Collect Action” (in Chapter 2, Basic Concepts) for more information.

This field is optional. If it doesn't exist, the backup will use the default collect mode.

Note: if your backup device does not suppport multisession discs, then you should probably confine yourself to the daily collect mode, to avoid losing data.

Restrictions: Must be one of daily, weekly or incr.

archive_mode

Archive mode for this directory.

The archive mode maps to the way that a backup file is stored. A value tar means just a tarfile (file.tar); a value targz means a gzipped tarfile (file.tar.gz); and a value tarbz2 means a bzipped tarfile (file.tar.bz2)

This field is optional. if it doesn't exist, the backup will use the default archive mode.

Restrictions: Must be one of tar, targz or tarbz2.

ignore_file

Ignore file name for this directory.

The ignore file is an indicator file. If it exists in a given directory, then that directory will be recursively excluded from the backup as if it were explicitly excluded in configuration.

The ignore file provides a way for individual users (who might not have access to Cedar Backup configuration) to control which of their own directories get backed up. For instance, users with a ~/tmp directory might not want it backed up. If they create an ignore file in their directory (e.g. ~/tmp/.cbignore), then Cedar Backup will ignore it.

This field is optional. If it doesn't exist, the backup will use the default ignore file name.

Restrictions: Must be non-empty

link_depth

Link depth value to use for this directory.

The link depth is maximum depth of the tree at which soft links should be followed. So, a depth of 0 does not follow any soft links within the collect directory, a depth of 1 follows only links immediately within the collect directory, a depth of 2 follows the links at the next level down, etc.

This field is optional. If it doesn't exist, the backup will assume a value of zero, meaning that soft links within the collect directory will never be followed.

Restrictions: If set, must be an integer ≥ 0.

dereference

Whether to dereference soft links.

If this flag is set, links that are being followed will be dereferenced before being added to the backup. The link will be added (as a link), and then the directory or file that the link points at will be added as well.

This value only applies to a directory where soft links are being followed (per the link_depth configuration option). It never applies to a configured collect directory itself, only to other directories within the collect directory.

This field is optional. If it doesn't exist, the backup will assume that links should never be dereferenced.

Restrictions: Must be a boolean (Y or N).

exclude

List of paths or patterns to exclude from the backup.

This is a subsection which contains a set of paths and patterns to be excluded within this collect directory. This list is combined with the program-wide list to build a complete list for the directory.

This section is entirely optional, and if it exists can also be empty.

The exclude subsection can contain one or more of each of the following fields:

abs_path

An absolute path to be recursively excluded from the backup.

If a directory is excluded, then all of its children are also recursively excluded. For instance, a value /var/log/apache would exclude any files within /var/log/apache as well as files within other directories under /var/log/apache.

This field can be repeated as many times as is necessary.

Restrictions: Must be an absolute path.

rel_path

A relative path to be recursively excluded from the backup.

The path is assumed to be relative to the collect directory itself. For instance, if the configured directory is /opt/web a configured relative path of something/else would exclude the path /opt/web/something/else.

If a directory is excluded, then all of its children are also recursively excluded. For instance, a value something/else would exclude any files within something/else as well as files within other directories under something/else.

This field can be repeated as many times as is necessary.

Restrictions: Must be non-empty.

pattern

A pattern to be excluded from the backup.

The pattern must be a Python regular expression. ^[19] It is assumed to be bounded at front and back by the beginning and end of the string (i.e. it is treated as if it begins with ^ and ends with $).

If the pattern causes a directory to be excluded, then all of the children of that directory are also recursively excluded. For instance, a value .*apache.* might match the /var/log/apache directory. This would exclude any files within /var/log/apache as well as files within other directories under /var/log/apache.

This field can be repeated as many times as is necessary.

Restrictions: Must be non-empty

Stage Configuration

The stage configuration section contains configuration options related the the stage action. The section indicates where date from peers can be staged to.

This section can also (optionally) override the list of peers so that not all peers are staged. If you provide any peers in this section, then the list of peers here completely replaces the list of peers in the peers configuration section for the purposes of staging.

This is an example stage configuration section for the simple case where the list of peers is taken from peers configuration:

<stage>
   <staging_dir>/opt/backup/stage</staging_dir>
</stage>

This is an example stage configuration section that overrides the default list of peers:

<stage>
   <staging_dir>/opt/backup/stage</staging_dir>
   <peer>
      <name>machine1</name>
      <type>local</type>
      <collect_dir>/opt/backup/collect</collect_dir>
   </peer>
   <peer>
      <name>machine2</name>
      <type>remote</type>
      <backup_user>backup</backup_user>
      <collect_dir>/opt/backup/collect</collect_dir>
   </peer>
</stage>

The following elements are part of the stage configuration section:

staging_dir

Directory to stage files into.

This is the directory into which the master stages collected data from each of the clients. Within the staging directory, data is staged into date-based directories by peer name. For instance, peer “daystrom” backed up on 19 Feb 2005 would be staged into something like 2005/02/19/daystrom relative to the staging directory itself.

This field is always required. The directory must contain enough free space to stage all of the files collected from all of the various machines in a backup pool. Many administrators set up purging to keep staging directories around for a week or more, which requires even more space.

Restrictions: Must be an absolute path

peer (local version)

Local client peer in a backup pool.

This is a subsection which contains information about a specific local client peer to be staged (backed up). A local peer is one whose collect directory can be reached without requiring any rsh-based network calls. It is possible that a remote peer might be staged as a local peer if its collect directory is mounted to the master via NFS, AFS or some other method.

This section can be repeated as many times as is necessary. At least one remote or local peer must be configured.

Remember, if you provide any local or remote peer in staging configuration, the global peer configuration is completely replaced by the staging peer configuration.

The local peer subsection must contain the following fields:

name

Name of the peer, typically a valid hostname.

For local peers, this value is only used for reference. However, it is good practice to list the peer's hostname here, for consistency with remote peers.

Restrictions: Must be non-empty, and unique among all peers.

type

Type of this peer.

This value identifies the type of the peer. For a local peer, it must always be local.

Restrictions: Must be local.

collect_dir

Collect directory to stage from for this peer.

The master will copy all files in this directory into the appropriate staging directory. Since this is a local peer, the directory is assumed to be reachable via normal filesystem operations (i.e. cp).

Restrictions: Must be an absolute path.

peer (remote version)

Remote client peer in a backup pool.

This is a subsection which contains information about a specific remote client peer to be staged (backed up). A remote peer is one whose collect directory can only be reached via an rsh-based network call.

This section can be repeated as many times as is necessary. At least one remote or local peer must be configured.

Remember, if you provide any local or remote peer in staging configuration, the global peer configuration is completely replaced by the staging peer configuration.

The remote peer subsection must contain the following fields:

name

Hostname of the peer.

For remote peers, this must be a valid DNS hostname or IP address which can be resolved during an rsh-based network call.

Restrictions: Must be non-empty, and unique among all peers.

type

Type of this peer.

This value identifies the type of the peer. For a remote peer, it must always be remote.

Restrictions: Must be remote.

collect_dir

Collect directory to stage from for this peer.

The master will copy all files in this directory into the appropriate staging directory. Since this is a remote peer, the directory is assumed to be reachable via rsh-based network operations (i.e. scp or the configured rcp command).

Restrictions: Must be an absolute path.

backup_user

Name of backup user on the remote peer.

This username will be used when copying files from the remote peer via an rsh-based network connection.

This field is optional. if it doesn't exist, the backup will use the default backup user from the options section.

Restrictions: Must be non-empty.

rcp_command

The rcp-compatible copy command for this peer.

The rcp command should be the exact command used for remote copies, including any required options. If you are using scp, you should pass it the -B option, so scp will not ask for any user input (which could hang the backup). A common example is something like /usr/bin/scp -B.

This field is optional. if it doesn't exist, the backup will use the default rcp command from the options section.

Restrictions: Must be non-empty.

Store Configuration

The store configuration section contains configuration options related the the store action. This section contains several optional fields. Most fields control the way media is written using the writer device.

This is an example store configuration section:

<store>
   <source_dir>/opt/backup/stage</source_dir>
   <media_type>cdrw-74</media_type>
   <device_type>cdwriter</device_type>
   <target_device>/dev/cdrw</target_device>
   <target_scsi_id>0,0,0</target_scsi_id>
   <drive_speed>4</drive_speed>
   <check_data>Y</check_data>
   <check_media>Y</check_media>
   <warn_midnite>Y</warn_midnite>
   <no_eject>N</no_eject>
   <refresh_media_delay>15</refresh_media_delay>
   <eject_delay>2</eject_delay>
   <blank_behavior>
      <mode>weekly</mode>
      <factor>1.3</factor>
   </blank_behavior>
</store>

The following elements are part of the store configuration section:

source_dir

Directory whose contents should be written to media.

This directory must be a Cedar Backup staging directory, as configured in the staging configuration section. Only certain data from that directory (typically, data from the current day) will be written to disc.

Restrictions: Must be an absolute path

device_type

Type of the device used to write the media.

This field controls which type of writer device will be used by Cedar Backup. Currently, Cedar Backup supports CD writers (cdwriter) and DVD writers (dvdwriter).

This field is optional. If it doesn't exist, the cdwriter device type is assumed.

Restrictions: If set, must be either cdwriter or dvdwriter.

media_type

Type of the media in the device.

Unless you want to throw away a backup disc every week, you are probably best off using rewritable media.

You must choose a media type that is appropriate for the device type you chose above. For more information on media types, see the section called “Media and Device Types” (in Chapter 2, Basic Concepts).

Restrictions: Must be one of cdr-74, cdrw-74, cdr-80 or cdrw-80 if device type is cdwriter; or one of dvd+r or dvd+rw if device type is dvdwriter.

target_device

Filesystem device name for writer device.

This value is required for both CD writers and DVD writers.

This is the UNIX device name for the writer drive, for instance /dev/scd0 or a symlink like /dev/cdrw.

In some cases, this device name is used to directly write to media. This is true all of the time for DVD writers, and is true for CD writers when a SCSI id (see below) has not been specified.

Besides this, the device name is also needed in order to do several pre-write checks (such as whether the device might already be mounted) as well as the post-write consistency check, if enabled.

Note: some users have reported intermittent problems when using a symlink as the target device on Linux, especially with DVD media. If you experience problems, try using the real device name rather than the symlink.

Restrictions: Must be an absolute path.

target_scsi_id

SCSI id for the writer device.

This value is optional for CD writers and is ignored for DVD writers.

If you have configured your CD writer hardware to work through the normal filesystem device path, then you can leave this parameter unset. Cedar Backup will just use the target device (above) when talking to cdrecord.

Otherwise, if you have SCSI CD writer hardware or you have configured your non-SCSI hardware to operate like a SCSI device, then you need to provide Cedar Backup with a SCSI id it can use when talking with cdrecord.

For the purposes of Cedar Backup, a valid SCSI identifier must either be in the standard SCSI identifier form scsibus,target,lun or in the specialized-method form <method>:scsibus,target,lun.

An example of a standard SCSI identifier is 1,6,2. Today, the two most common examples of the specialized-method form are ATA:scsibus,target,lun and ATAPI:scsibus,target,lun, but you may occassionally see other values (like OLDATAPI in some forks of cdrecord).

See the section called “Configuring your Writer Device” for more information on writer devices and how they are configured.

Restrictions: If set, must be a valid SCSI identifier.

drive_speed

Speed of the drive, i.e. 2 for a 2x device.

This field is optional. If it doesn't exist, the underlying device-related functionality will use the default drive speed.

For DVD writers, it is best to leave this value unset, so growisofs can pick an appropriate speed. For CD writers, since media can be speed-sensitive, it is probably best to set a sensible value based on your specific writer and media.

Restrictions: If set, must be an integer ≥ 1.

check_data

Whether the media should be validated.

This field indicates whether a resulting image on the media should be validated after the write completes, by running a consistency check against it. If this check is enabled, the contents of the staging directory are directly compared to the media, and an error is reported if there is a mismatch.

Practice shows that some drives can encounter an error when writing a multisession disc, but not report any problems. This consistency check allows us to catch the problem. By default, the consistency check is disabled, but most users should choose to enable it unless they have a good reason not to.

This field is optional. If it doesn't exist, then N will be assumed.

Restrictions: Must be a boolean (Y or N).

check_media

Whether the media should be checked before writing to it.

By default, Cedar Backup does not check its media before writing to it. It will write to any media in the backup device. If you set this flag to Y, Cedar Backup will make sure that the media has been initialized before writing to it. (Rewritable media is initialized using the initialize action.)

If the configured media is not rewritable (like CD-R), then this behavior is modified slightly. For this kind of media, the check passes either if the media has been initialized or if the media appears unused.

This field is optional. If it doesn't exist, then N will be assumed.

Restrictions: Must be a boolean (Y or N).

warn_midnite

Whether to generate warnings for crossing midnite.

This field indicates whether warnings should be generated if the store operation has to cross a midnite boundary in order to find data to write to disc. For instance, a warning would be generated if valid store data was only found in the day before or day after the current day.

Configuration for some users is such that the store operation will always cross a midnite boundary, so they will not care about this warning. Other users will expect to never cross a boundary, and want to be notified that something “strange” might have happened.

This field is optional. If it doesn't exist, then N will be assumed.

Restrictions: Must be a boolean (Y or N).

no_eject

Indicates that the writer device should not be ejected.

Under some circumstances, Cedar Backup ejects (opens and closes) the writer device. This is done because some writer devices need to re-load the media before noticing a media state change (like a new session).

For most writer devices this is safe, because they have a tray that can be opened and closed. If your writer device does not have a tray and Cedar Backup does not properly detect this, then set this flag. Cedar Backup will not ever issue an eject command to your writer.

Note: this could cause problems with your backup. For instance, with many writers, the check data step may fail if the media is not reloaded first. If this happens to you, you may need to get a different writer device.

This field is optional. If it doesn't exist, then N will be assumed.

Restrictions: Must be a boolean (Y or N).

refresh_media_delay

Number of seconds to delay after refreshing media

This field is optional. If it doesn't exist, no delay will occur.

Some devices seem to take a little while to stablize after refreshing the media (i.e. closing and opening the tray). During this period, operations on the media may fail. If your device behaves like this, you can try setting a delay of 10-15 seconds.

Restrictions: If set, must be an integer ≥ 1.

eject_delay

Number of seconds to delay after ejecting the tray

This field is optional. If it doesn't exist, no delay will occur.

If your system seems to have problems opening and closing the tray, one possibility is that the open/close sequence is happening too quickly — either the tray isn't fully open when Cedar Backup tries to close it, or it doesn't report being open. To work around that problem, set an eject delay of a few seconds.

Restrictions: If set, must be an integer ≥ 1.

blank_behavior

Optimized blanking strategy.

For more information about Cedar Backup's optimized blanking strategy, see the section called “Optimized Blanking Stategy”.

This entire configuration section is optional. However, if you choose to provide it, you must configure both a blanking mode and a blanking factor.

blank_mode

Blanking mode.

Restrictions:Must be one of "daily" or "weekly".

blank_factor

Blanking factor.

Restrictions:Must be a floating point number ≥ 0.

Purge Configuration

The purge configuration section contains configuration options related the the purge action. This section contains a set of directories to be purged, along with information about the schedule at which they should be purged.

Typically, Cedar Backup should be configured to purge collect directories daily (retain days of 0).

If you are tight on space, staging directories can also be purged daily. However, if you have space to spare, you should consider purging about once per week. That way, if your backup media is damaged, you will be able to recreate the week's backup using the rebuild action.

You should also purge the working directory periodically, once every few weeks or once per month. This way, if any unneeded files are left around, perhaps because a backup was interrupted or because configuration changed, they will eventually be removed. The working directory should not be purged any more frequently than once per week, otherwise you will risk destroying data used for incremental backups.

This is an example purge configuration section:

<purge>
   <dir>
      <abs_path>/opt/backup/stage</abs_path>
      <retain_days>7</retain_days>
   </dir>
   <dir>
      <abs_path>/opt/backup/collect</abs_path>
      <retain_days>0</retain_days>
   </dir>
</purge>

The following elements are part of the purge configuration section:

dir

A directory to purge within.

This is a subsection which contains information about a specific directory to purge within.

This section can be repeated as many times as is necessary. At least one purge directory must be configured.

The purge directory subsection contains the following fields:

abs_path

Absolute path of the directory to purge within.

The contents of the directory will be purged based on age. The purge will remove any files that were last modified more than “retain days” days ago. Empty directories will also eventually be removed. The purge directory itself will never be removed.

The path may be either a directory, a soft link to a directory, or a hard link to a directory. Soft links within the directory (if any) are treated as files.

Restrictions: Must be an absolute path.

retain_days

Number of days to retain old files.

Once it has been more than this many days since a file was last modified, it is a candidate for removal.

Restrictions: Must be an integer ≥ 0.

Extensions Configuration

The extensions configuration section is used to configure third-party extensions to Cedar Backup. If you don't intend to use any extensions, or don't know what extensions are, then you can safely leave this section out of your configuration file. It is optional.

Extensions configuration is used to specify “extended actions” implemented by code external to Cedar Backup. An administrator can use this section to map command-line Cedar Backup actions to third-party extension functions.

Each extended action has a name, which is mapped to a Python function within a particular module. Each action also has an index associated with it. This index is used to properly order execution when more than one action is specified on the command line. The standard actions have predefined indexes, and extended actions are interleaved into the normal order of execution using those indexes. The collect action has index 100, the stage index has action 200, the store action has index 300 and the purge action has index 400.

Warning

Extended actions should always be configured to run before the standard action they are associated with. This is because of the way indicator files are used in Cedar Backup. For instance, the staging process considers the collect action to be complete for a peer if the file cback.collect can be found in that peer's collect directory.

If you were to run the standard collect action before your other collect-like actions, the indicator file would be written after the collect action completes but before all of the other actions even run. Because of this, there's a chance the stage process might back up the collect directory before the entire set of collect-like actions have completed — and you would get no warning about this in your email!

So, imagine that a third-party developer provided a Cedar Backup extension to back up a certain kind of database repository, and you wanted to map that extension to the “database” command-line action. You have been told that this function is called “foo.bar()”. You think of this backup as a “collect” kind of action, so you want it to be performed immediately before the collect action.

To configure this extension, you would list an action with a name “database”, a module “foo”, a function name “bar” and an index of “99”.

This is how the hypothetical action would be configured:

<extensions>
   <action>
      <name>database</name>
      <module>foo</module>
      <function>bar</function>
      <index>99</index>
   </action>
</extensions>

The following elements are part of the extensions configuration section:

action

This is a subsection that contains configuration related to a single extended action.

This section can be repeated as many times as is necessary.

The action subsection contains the following fields:

name

Name of the extended action.

Restrictions: Must be a non-empty string consisting of only lower-case letters and digits.

module

Name of the Python module associated with the extension function.

Restrictions: Must be a non-empty string and a valid Python identifier.

function

Name of the Python extension function within the module.

Restrictions: Must be a non-empty string and a valid Python identifier.

index

Index of action, for execution ordering.

Restrictions: Must be an integer ≥ 0.

Prev	Up	Next
Chapter 5. Configuration	Home	Setting up a Pool of One