CedarBackup3.extend.amazons3

Store-type extension that writes data to Amazon S3.

This extension requires a new configuration section <amazons3> and is intended to be run immediately after the standard stage action, replacing the standard store action. Aside from its own configuration, it requires the options and staging configuration sections in the standard Cedar Backup configuration file. Since it is intended to replace the store action, it does not rely on any store configuration.

The underlying functionality relies on the U{AWS CLI interface <http://aws.amazon.com/documentation/cli/>}. Before you use this extension, you need to set up your Amazon S3 account and configure the AWS CLI connection per Amazon’s documentation. The extension assumes that the backup is being executed as root, and switches over to the configured backup user to communicate with AWS. So, make sure you configure AWS CLI as the backup user and not root.

You can optionally configure Cedar Backup to encrypt data before sending it to S3. To do that, provide a complete command line using the ${input} and ${output} variables to represent the original input file and the encrypted output file. This command will be executed as the backup user.

For instance, you can use something like this with GPG:

/usr/bin/gpg -c --no-use-agent --batch --yes --passphrase-file /home/backup/.passphrase -o ${output} ${input}

The GPG mechanism depends on a strong passphrase for security. One way to generate a strong passphrase is using your system random number generator, i.e.:

dd if=/dev/urandom count=20 bs=1 | xxd -ps

(See U{StackExchange <http://security.stackexchange.com/questions/14867/gpg-encryption-security>} for more details about that advice.) If you decide to use encryption, make sure you save off the passphrase in a safe place, so you can get at your backup data later if you need to. And obviously, make sure to set permissions on the passphrase file so it can only be read by the backup user.

This extension was written for and tested on Linux. It will throw an exception if run on Windows.

author

Kenneth J. Pronovici <pronovic@ieee.org>

Module Contents

CedarBackup3.extend.amazons3.logger
CedarBackup3.extend.amazons3.SU_COMMAND = ['su']
CedarBackup3.extend.amazons3.AWS_COMMAND = ['aws']
CedarBackup3.extend.amazons3.STORE_INDICATOR = cback.amazons3
class CedarBackup3.extend.amazons3.AmazonS3Config(warnMidnite=None, s3Bucket=None, encryptCommand=None, fullBackupSizeLimit=None, incrementalBackupSizeLimit=None)

Bases: object

Class representing Amazon S3 configuration.

Amazon S3 configuration is used for storing backup data in Amazon’s S3 cloud storage using the s3cmd tool.

The following restrictions exist on data in this class:

  • The s3Bucket value must be a non-empty string

  • The encryptCommand value, if set, must be a non-empty string

  • The full backup size limit, if set, must be a ByteQuantity >= 0

  • The incremental backup size limit, if set, must be a ByteQuantity >= 0

warnMidnite
s3Bucket
encryptCommand
fullBackupSizeLimit
incrementalBackupSizeLimit
__repr__()

Official string representation for class instance.

__str__()

Informal string representation for class instance.

__eq__(other)

Equals operator, iplemented in terms of original Python 2 compare operator.

__lt__(other)

Less-than operator, iplemented in terms of original Python 2 compare operator.

__gt__(other)

Greater-than operator, iplemented in terms of original Python 2 compare operator.

__cmp__(other)

Original Python 2 comparison operator. :param other: Other object to compare to

Returns

-1/0/1 depending on whether self is <, = or > other

class CedarBackup3.extend.amazons3.LocalConfig(xmlData=None, xmlPath=None, validate=True)

Bases: object

Class representing this extension’s configuration document.

This is not a general-purpose configuration object like the main Cedar Backup configuration object. Instead, it just knows how to parse and emit amazons3-specific configuration values. Third parties who need to read and write configuration related to this extension should access it through the constructor, validate and addConfig methods.

Note: Lists within this class are “unordered” for equality comparisons.

amazons3
__repr__()

Official string representation for class instance.

__str__()

Informal string representation for class instance.

__eq__(other)

Equals operator, iplemented in terms of original Python 2 compare operator.

__lt__(other)

Less-than operator, iplemented in terms of original Python 2 compare operator.

__gt__(other)

Greater-than operator, iplemented in terms of original Python 2 compare operator.

__cmp__(other)

Original Python 2 comparison operator. Lists within this class are “unordered” for equality comparisons. :param other: Other object to compare to

Returns

-1/0/1 depending on whether self is <, = or > other

validate()

Validates configuration represented by the object.

AmazonS3 configuration must be filled in. Within that, the s3Bucket target must be filled in

Raises

ValueError – If one of the validations fails

addConfig(xmlDom, parentNode)

Adds an <amazons3> configuration section as the next child of a parent.

Third parties should use this function to write configuration related to this extension.

We add the following fields to the document:

warnMidnite                 //cb_config/amazons3/warn_midnite
s3Bucket                    //cb_config/amazons3/s3_bucket
encryptCommand              //cb_config/amazons3/encrypt
fullBackupSizeLimit         //cb_config/amazons3/full_size_limit
incrementalBackupSizeLimit  //cb_config/amazons3/incr_size_limit
Parameters

  • xmlDom – DOM tree as from impl.createDocument()

  • parentNode – Parent that the section should be appended to

CedarBackup3.extend.amazons3.executeAction(configPath, options, config)

Executes the amazons3 backup action.

Parameters

  • configPath (String representing a path on disk) – Path to configuration file on disk

  • options (Options object) – Program command-line options

  • config (Config object) – Program configuration

Raises

  • ValueError – Under many generic error conditions

  • IOError – If there are I/O problems reading or writing files