Next: New Features in 9.2.0 Up: Main Reference Previous: New Features in 9.6.0 Contents Index
The picture shown above shows two Volumes (Volume0001 and Volume0002) with their parts in the cache. Below the cache, one can see that Volume0002 has been uploaded or synchronized with the Cloud.
Note: Regular Bacula disk Volumes are implemented as standard files that reside in the user defined Archive Directory. On the other hand, Bacula Cloud Volumes are directories that reside in the user defined Archive Directory. Each Cloud Volume's directory contains the cloud Volume parts which are implemented as numbered files (part.1, part.2, ...).
With most Cloud providers, uploads are usually free of charge, but downloads of data from the Cloud are billed. By using local cache and multiple small parts, you can configure Bacula to substantially reduce download costs.
The MaximumFileSize Device directive is still valid within the Storage Daemon and defines the granularity of a restore chunk. In order to limit volume parts to download during restore (specially when restoring single files), it might be useful to set the MaximumFileSize to a value smaller than or equal to the MaximumPartSize.
The Cache is treated much like a normal Disk based backup, so that in configuring Cloud the administrator should take care to set "Archive Device" in the Device resource to a directory where he/she would normally start data backed up to disk. Obviously, unless he/she uses the truncate/prune cache commands, the Archive Device will continue to fill.
The cache retention can be controlled per Volume with the “CacheRetention” attribute. The default value is 0, meaning that the pruning of the cache is disabled.
The “CacheRetention” value for a volume can be modified with the “update” command or via the Pool directive “CacheRetention” for newly created volumes.
Cloud choice:
1: List Cloud Volumes in the Cloud
2: Upload a Volume to the Cloud
3: Prune the Cloud Cache
4: Truncate a Volume Cache
5: Done
Select action to perform on Cloud (1-5):
The different choices should be rather obvious.
Device Type = Cloud
Cloud = S3Cloud
Device {
Name = CloudStorage
Device Type = Cloud
Cloud = S3Cloud
Archive Device = /opt/bacula/backups
Maximum Part Size = 10000000
Media Type = CloudType
LabelMedia = yes
Random Access = Yes;
AutomaticMount = yes
RemovableMedia = no
AlwaysOpen = no
}
As you can see from the above, the Cloud directive in the Device resource contains the name (S3Cloud) of the Cloud resource that is shown below.
Note also the Archive Device is specified in the same manner as one would use for a File device. However, in place of containing files with Volume names, the archive device for the Cloud drivers will contain the local cache, which consists of directories with the Volume name; and these directories contain the parts associated with the particular Volume. So with the above Device resource, and the two cache Volumes shown in figure fig:cloud0ay2 above would have the following layout on disk:
/opt/bacula/backups
/opt/bacula/backups/Volume0001
/opt/bacula/backups/Volume0001/part.1
/opt/bacula/backups/Volume0001/part.2
/opt/bacula/backups/Volume0001/part.3
/opt/bacula/backups/Volume0001/part.4
/opt/bacula/backups/Volume0002
/opt/bacula/backups/Volume0002/part.1
/opt/bacula/backups/Volume0002/part.2
/opt/bacula/backups/Volume0002/part.3
default east USA location:
Cloud {
Name = S3Cloud
Driver = "S3"
HostName = "s3.amazonaws.com"
BucketName = "BaculaVolumes"
AccessKey = "BZIXAIS39DP9YNER5DFZ"
SecretKey = "beesheeg7iTe0Gaexee7aedie4aWohfuewohGaa0"
Protocol = HTTPS
UriStyle = VirtualHost
Truncate Cache = No
Upload = EachPart
Region = "us-east-1"
MaximumUploadBandwidth = 5MB/s
}
central europe location:
Cloud {
Name = S3Cloud
Driver = "S3"
HostName = "s3-eu-central-1.amazonaws.com"
BucketName = "BaculaVolumes"
AccessKey = "BZIXAIS39DP9YNER5DFZ"
SecretKey = "beesheeg7iTe0Gaexee7aedie4aWohfuewohGaa0"
Protocol = HTTPS
UriStyle = VirtualHost
Truncate Cache = No
Upload = EachPart
Region = "eu-central-1"
MaximumUploadBandwidth = 4MB/s
}
For Amazon Cloud, refer to http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region to get a complete list of regions and corresponding endpoints and use them respectively as Region and HostName directive.
For CEPH S3 interface:
Cloud {
Name = CEPH_S3
Driver = "S3"
HostName = ceph.mydomain.lan
BucketName = "CEPHBucket"
AccessKey = "xxxXXXxxxx"
SecretKey = "xxheeg7iTe0Gaexee7aedie4aWohfuewohxx0"
Protocol = HTTPS
Upload = EachPart
UriStyle = Path # Must be set for CEPH
}
The directives of the above Cloud resource for the S3 driver are defined as follows:
The name of the Cloud resource. This is the logical Cloud name, and may be any string up to 127 characters in length. Shown as S3Cloud above. The description is used for display purposes as is the case with all resource. This defines which driver to use. It can be S3. There is also a File driver, which is used mostly for testing. This directive specifies the bucket name that you wish to use on the Cloud service. This name is normally a unique name name that identifies where you want to place your Cloud Volume parts. With Amazon S3, the bucket must be created previously on the Cloud service. The maximum bucket name size is 255 characters. The access key is your unique user identifier given to you by your cloud service provider. The secret key is the security key that was given to you by your cloud service provider. It is equivalent to a password. The protocol defines the communications protocol to use with the cloud service provider. The two protocols currently supported are: HTTPS and HTTP. The default is HTTPS. This directive specifies the URI style to use to communicate with the cloud service provider. The two Uri Styles currently supported are: VirtualHost and Path. The default is VirtualHost. This directive specifies when Bacula should automatically remove (truncate) the local cache parts. Local cache parts can only be removed if they have been uploaded to the cloud. The currently implemented values are:
This directive specifies when local cache parts will be uploaded to the Cloud. The options are:
The default is unlimited, but by using this directive, you may limit the upload bandwidth used globally by all devices referencing this Cloud resource.
The default is unlimited, but by using this directive, you may limit the download bandwidth used globally by all devices referencing this Cloud resource.
The Cloud resource can be configured to use a specific endpoint within a region. This directive is required for AWS-V4 regions. ex: Region="eu-central-1"
The following Cloud directives are ignored: Bucket Name, Access Key, Secret Key, Protocol, Uri Style. The directives Truncate Cache and Upload work on the local cache in the same manner as they do for the S3 driver.
The main difference to note is that the Host Name, specifies the destination directory for the Cloud Volume files, and this Host Name must be different from the Archive Device name, or there will be a conflict between the local cache (in the Archive Device directory) and the destination Cloud Volumes (in the Host Name directory).
As noted above, the File driver is mostly used for testing purposes, and we do not particularly recommend using it. However, if you have a particularly slow backup device you might want to stage your backup data into an SSD or disk using the local cache feature of the Cloud device, and have your Volumes transferred in the background to a slow File device.
Automatic WORM (Write Once Read Multiple) tapes detection has been added in 10.2.
When a WORM tape is detected, the catalog volume entry is changed automatically to set Recycle=no. It will prevent the volume from being automatically recycled by Bacula.
There is no change in how the Job and File records are pruned from the catalog as that is a separate issue that is currently adequately implemented in Bacula.
When a WORM tape is detected, the SD will show WORM on the device state output (must have debug greater or equal to 6) otherwise the status shows as !WORM
Device state:
OPENED !TAPE LABEL APPEND !READ !EOT !WEOT !EOF WORM !SHORT !MOUNTED ...
The output of the used volume status has been modified to include the worm state. It shows worm=1 for a worm cassette and worm=0 otherwise. Example:
Used Volume status:
Reserved volume: TestVolume001 on Tape device "nst0" (/dev/nst0)
Reader=0 writers=0 reserves=0 volinuse=0 worm=1
The following programs are needed for the WORM tape detection:
The new Storage Device directive Worm Command must be configured as well as the Control Device directive (used with the Tape Alert feature).
Device {
Name = "LTO-0"
Archive Device = "/dev/nst0"
Control Device = "/dev/sg0" # from lsscsi -g
Worm Command = "/opt/bacula/scripts/isworm %l"
...
}