// FIXME: Add the various scenarios, merge with configuration-examples.txt +
//        and the below content +
//  e.g. using different backup methods, +
//       using different output methods

= Scenarios

== Bootable ISO
If you simply want a bootable ISO on a central server, you would do:

[source,bash]
----
OUTPUT=ISO
OUTPUT_URL=http://server/path-to-push/
----

== Bootable ISO with an external (commercial) backup software
If you rely on your backup software to do the full restore of a system then you could define:

[source,bash]
----
OUTPUT=ISO
BACKUP=[TSM|NSR|DP|NBU|GALAXY10|SEP|DUPLICITY|BACULA|BAREOS|RBME|FDRUPSTREAM]
----

When using one of the above backup solution (commercial or open source) then there is no need to use +rear mkbackup+ as the backup workflow would be empty. Just use +rear mkrescue+

ReaR will incorporate the needed executables and libraries of your chosen backup solution into the rescue image of ReaR.

== Bootable ISO and storing archive on NFS/NAS
To create an ISO rescue image and using a central NFS/NAS server to store the archive, you could define:

[source,bash]
----
OUTPUT=ISO
BACKUP=NETFS
BACKUP_URL=nfs://remote-nfs-server/exports
#BACKUP_PROG_CRYPT_ENABLED=1
#BACKUP_PROG_CRYPT_KEY="my_Secret_pw"
----

The above example shows that it is even possible to protect the archives with a password.  Be aware, that the password entry +BACKUP_PROG_CRYPT_KEY="my_Secret_pw"+ will be deleted in the +/etc/rear/local.conf+ file within the rescue image. So, it is important to remember the password (if you use the rescue image months later could cause problems I guess).


== Bootable USB device with backup to USB
If you want a bootable USB device with a (tar) backup to USB as well, you
would use:

[source,bash]
----
BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
----


== Bootable tape drive (OBDR) with backup to tape
If you want an OBDR image and backup on tape, and use GNU tar for
backup/restore, you would use:

[source,bash]
----
BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
----


== Bootable tape drive (OBDR) and Bacula restore
If you want an OBDR image on tape, and the Bacula tools to recover your
backup, use:

[source,bash]
----
BACKUP=BACULA
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
----


== ReaR with Borg back end
 - Install Borg backup (https://borgbackup.readthedocs.io/en/stable/installation.html).

IMPORTANT:  We strongly recommend to use Borg standalone binary (https://github.com/borgbackup/borg/releases) as it includes all necessities for Borg operations.
			If you decide to go for different type of Borg installation types, make sure you include all needed files for Borg runtime into ReaR rescue/recovery system.
			E.g. by using `COPY_AS_IS_BORG=( '/usr/lib64/python3.4*' '/usr/bin/python3*' '/usr/bin/pyvenv*' '/usr/lib/python3.4*' '/usr/lib64/libpython3*' )`

=== Borg -> SSH
 - Setup ssh key infrastructure for user that will be running backup.
Issuing following command must work without any password prompts or remote host identity confirmation:

`ssh <BORGBACKUP_USERNAME>@<BORGBACKUP_HOST>`

 - Example _local.conf_:
[source,bash]
----
OUTPUT=ISO
OUTPUT_URL=nfs://foo.bar.xy/mnt/backup/iso

BACKUP=BORG
BORGBACKUP_HOST="foo.bar.xy"
BORGBACKUP_USERNAME="borg_user"
BORGBACKUP_REPO="/mnt/backup/client"
BORGBACKUP_REMOTE_PATH="/usr/local/bin/borg"

# Automatic archive pruning
# (https://borgbackup.readthedocs.io/en/stable/usage.html#borg-prune)
BORGBACKUP_PRUNE_WEEKLY=2

# Archive compression
# (https://borgbackup.readthedocs.io/en/stable/usage.html#borg-create)
BORGBACKUP_COMPRESSION="lzma,9"     # Slowest backup, best compression

# Repository encryption
# (https://borgbackup.readthedocs.io/en/stable/usage.html#borg-init)
BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE="S3cr37_P455w0rD"
COPY_AS_IS_BORG=( "/root/.config/borg/keys/" )

# Borg environment variables
# (https://borgbackup.readthedocs.io/en/stable/usage.html#environment-variables)
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

----
=== Borg -> USB

 - Example _local.conf_:
[source,bash]
----
OUTPUT=USB
BACKUP=BORG

USB_DEVICE=/dev/disk/by-label/REAR-000

BORGBACKUP_REPO="/my_borg_backup"
BORGBACKUP_UMASK="0002"

BORGBACKUP_PRUNE_WEEKLY=2

BORGBACKUP_ENC_TYPE="keyfile"
export BORG_PASSPHRASE="S3cr37_P455w0rD"

export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"

COPY_AS_IS_EXCLUDE=( "${COPY_AS_IS_EXCLUDE[@]}" )
COPY_AS_IS_BORG=( '/root/.config/borg/keys/' )

SSH_UNPROTECTED_PRIVATE_KEYS="yes"
SSH_FILES="yes"


----

IMPORTANT: If using BORGBACKUP_ENC_TYPE="keyfile", don't forget to make your
           encryption key available for case of restore!
           (using `COPY_AS_IS_BORG=( "/root/.config/borg/keys/" )` is a option to consider).
           Be sure to read https://borgbackup.readthedocs.io/en/stable/usage.html#borg-init,
           and make your self familiar how encryption in Borg works.

 - Executing `rear mkbackup` will create Relax-and-Recover rescue/recovery system and
 start Borg backup process. Once backup finishes, it will also prune old archives from repository,
 if at least one of `BORGBACKUP_PRUNE_*` variables is set.
 - To recover your system, boot Relax-and-Recover rescue/recovery system and trigger `rear recover`.
 You will be prompted which archive to recover from Borg repository, once ReaR finished with layout configuration.

```
...
Disk layout created.
Starting Borg restore

=== Borg archives list ===
Host:       foo.bar.xy
Repository: /mnt/backup/client

[1] rear_1 	Sun, 2016-10-16 14:08:16
[2] rear_2 	Sun, 2016-10-16 14:32:11

[3] Exit

Choose archive to recover from:


```


== Backup/restore alien file system using BLOCKCLONE and dd
=== Configuration

- First we need to set some global options to _local.conf_

```
# cat local.conf
OUTPUT=ISO
BACKUP=NETFS
BACKUP_OPTIONS="nfsvers=3,nolock"
BACKUP_URL=nfs://beta.virtual.sk/mnt/rear
```

- Now we can define variables that will apply only for targeted block device

```
# cat alien.conf
BACKUP=BLOCKCLONE                                        # Define BLOCKCLONE as backup method
BACKUP_PROG_ARCHIVE="alien"                              # Name of image file
BACKUP_PROG_SUFFIX=".dd.img"                             # Suffix of image file
BACKUP_PROG_COMPRESS_SUFFIX=""                           # Clear additional suffixes

BLOCKCLONE_PROG=dd                                       # Use dd for image creation
BLOCKCLONE_PROG_OPTS="bs=4k"                             # Additional options that will be passed to dd
BLOCKCLONE_SOURCE_DEV="/dev/sdc1"                        # Device that should be backed up

BLOCKCLONE_SAVE_MBR_DEV="/dev/sdc"                       # Device where partitioning information is stored (optional)
BLOCKCLONE_MBR_FILE="alien_boot_strap.img"               # Output filename for boot strap code
BLOCKCLONE_PARTITIONS_CONF_FILE="alien_partitions.conf"  # Output filename for partition configuration
BLOCKCLONE_ALLOW_MOUNTED="yes"                           # Device can be mounted during backup (default NO)
```

=== Running backup

 - Save partitions configuration, bootstrap code and create actual backup of /dev/sdc1

```
# rear -C alien mkbackuponly
```

 - Running restore from ReaR restore/recovery system

```
# rear -C alien restoreonly

Restore alien.dd.img to device: [/dev/sdc1]                 # User is always prompted for restore destination
Device /dev/sdc1 was not found.                             # If destination does not exist ReaR will try to create it (or fail if BLOCKCLONE_SAVE_MBR_DEV was not set during backup)
Restore partition layout to (^c to abort): [/dev/sdc]       # Prompt user for device where partition configuration should be restored
Checking that no-one is using this disk right now ... OK

Disk /dev/sdc: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new DOS disklabel with disk identifier 0x10efb7a9.
Created a new partition 1 of type 'HPFS/NTFS/exFAT' and of size 120 MiB.

/dev/sdc2:
New situation:

Device     Boot Start    End Sectors  Size Id Type
/dev/sdc1        4096 249855  245760  120M  7 HPFS/NTFS/exFAT

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
```


== Using Relax-and-Recover with USB storage devices
Using USB devices with Relax-and-Recover can be appealing for several reasons:

 - If you only need to have a bootable rescue environment, a USB device is
   a *cheap device* for storing only 25 to 60MB to boot from

 - You can leave the USB device inserted in the system and *opt-in booting*
   from it only when disaster hits (although we do recommend storing rescue
   environments off-site)

 - You can *store multiple systems and multiple snapshots* on a single device

 - In case you have plenty of space, it might be a simple solution to store
   complete Disaster Recovery images (rescue + backup) on a single device for
   a set of systems

 - For migrating a bunch of servers having a single device to boot from might
   be very appealing

 - We have implemented a specific workflow: inserting a REAR-000 labeled USB
   stick will invoke +rear udev+ and adds a rescue environment to the USB
   stick (updating the bootloader if needed)

However USB devices may be slow for backup purposes, especially on older
systems or with unreliable/cheap devices.

=== Configuring Relax-and-Recover for USB storage devices
The below configuration (_/etc/rear/local.conf_) gives a list of possible
options when you want to run Relax-and-Recover with USB storage.

[source,bash]
----
BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
----

IMPORTANT: On RHEL4 or older there are no _/dev/disk/by-label/_ udev aliases,
           which means we cannot use device by label. However it is possible
           to use +by-path+ references, however this makes it very specific
           to the USB port used.  We opted to use the complete device-name,
           which can be dangerous if you may have other _/dev/sdX_ devices
           (luckily we have CCISS block devices in _/dev/cciss/_).


=== Preparing your USB storage device
To prepare your USB device for use with Relax-and-Recover, do: +rear format /dev/sdX+

This will create a single partition, make it bootable, format it with ext3,
label it +REAR-000+ and disable warnings related filesystem check for the
device.


=== USB storage as rescue media

==== Configuring Relax-and-Recover to have Bacula tools
If the rescue environment needs additional tools and workflow, this can be
specified by using +BACKUP=BACULA+ in the configuration file
_/etc/rear/local.conf_:

[source,bash]
----
BACKUP=BACULA
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000
----

==== Making the rescue USB storage device
To create a rescue USB device, run +rear -v mkrescue+ as shown below after
you have inserted a *REAR-000* labeled USB device. Make sure the device name
for the USB device is what is configured for +USB_DEVICE+.

----
[root@system ~]# rear -v mkrescue
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Finished in 72 seconds.
----

WARNING: Doing the above may replace the existing MBR of the USB device.
         However any other content on the device is retained.


[[booting-from-usb]]
==== Booting from USB storage device
Before you can recover our DR backup, it is important to configure the BIOS to
boot from the USB device. In some cases it is required to go into the BIOS setup
(+F9+ during boot) to change the boot-order of devices. (In BIOS setup select
+Standard Boot Order (IPL)+)

Once booted from the USB device, select the system you like to recover from
the list. If you don't press a key within 30 seconds, the system will try to
boot from the local disk.

["aafigure",width="12cm",height="6cm",align="center",format="svg",options="textual",aspect="0.7",linewidth="1"]
----
+---------------------------------------------+
|        "Relax-and-Recover v1.12.0svn497"    |
+---------------------------------------------+
|  "Recovery images"                          |
|    "system.localdomain"                   > |
|    "other.localdomain"                    > |
|---------------------------------------------|
|  "Other actions"                            |
|    "Help for Relax-and-Recover"             |
|    "Boot Local disk (hd1)"                  |
|    "Boot BIOS disk (0x81)"                  |
|    "Boot Next BIOS device"                  |
|    "Hardware Detection tool"                |
|    "Memory test"                            |
|    "Reboot system"                          |
|    "Power off system"                       |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"

           "Automatic boot in 30 seconds..."
----

////
      .-------------------------------------------------------------.
      |               Relax-and-Recover v1.12.0svn497               |
      |-------------------------------------------------------------|
      |  Recovery images                                            |
      |   system.localdomain                                      > |
      |   other.localdomain                                       > |
      |-------------------------------------------------------------|
      |  Other actions                                              |
      |   Help for Relax-and-Recover                                |
      (>  Boot Local disk (hd1)                                    <)
      |   Boot BIOS disk (0x81)                                     |
      |   Boot Next BIOS device                                     |
      |   Hardware Detection tool                                   |
      |   Memory test                                               |
      |   Reboot system                                             |
      |   Power off system                                          |
      `-------------------------------------------------------------'

             Press [Tab] to edit options or [F1] for help

                        Automatic boot in 30 seconds...
////

WARNING: Booting from a local disk may fail when booting from a USB device.
         This is caused by the fact that the GRUB bootloader on the local
         disk is configured as if it is being the first drive +(hd0)+ but
         it is in fact the second disk +(hd1)+. If you do find menu entries
         not working from GRUB, please remove the +root (hd0,0)+ line from
         the entry.

Then select the image you would like to recover.

["aafigure",width="14cm",height="7cm",align="center",format="svg",options="textual",aspect="0.7",linewidth="1"]
----
+---------------------------------------------+
|           "system.localdomain"              |
+---------------------------------------------+
|  "2011-03-26 02:16 backup"                  |
|  "2011-03-25 18:39 backup"                  |
|  "2011-03-05 16:12 rescue image"            |
|---------------------------------------------|
|  "Back"                                     |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
|                                             |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"


"Backup using kernel 2.6.32-122.el6.x86_64"
"BACKUP=NETFS OUTPUT=USB OUTPUT_URL=usb:///dev/disk/by-label/REAR-000"
----

////
      .-------------------------------------------------------------.
      |                     system.localdomain                      |
      |-------------------------------------------------------------|
      |  2011-03-26 02:16 backup                                    |
      (> 2011-03-25 18:39 backup                                   <)
      |  2011-03-05 16:12 rescue image                              |
      |-------------------------------------------------------------|
      |  Back                                                     < |
      |                                                             |
      |                                                             |
      |                                                             |
      |                                                             |
      |                                                             |
      |                                                             |
      |                                                             |
      |                                                             |
      |                                                             |
      `-------------------------------------------------------------'

             Press [Tab] to edit options or [F1] for help



Backup using kernel 2.6.32-122.el6.x86_64
BACKUP=NETFS OUTPUT=USB OUTPUT_URL=usb:///dev/disk/by-label/REAR-000
////

TIP: When browsing through the images you get more information about the
     image at the bottom of the screen.

==== Restoring from USB rescue media
Then wait for the system to boot until you get the prompt.

On the shell prompt, type +rear recover+.

You may need to answer a few questions depending on your hardware
configuration and whether you are restoring to a (slightly)
different system.

----
RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Creating LVM VG vgrem
Creating LVM VG vgqry
Creating LVM VG vg00
Creating LVM volume vg00/lv00
Creating LVM volume vg00/lvdstpol
Creating LVM volume vg00/lvsys
Creating LVM volume vg00/lvusr
Creating LVM volume vg00/lvtmp
Creating LVM volume vg00/lvvar
Creating LVM volume vg00/lvopt
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Creating LVM VG vgbck
Creating LVM volume vgbck/lvetc
Creating LVM volume vgbck/lvvar
Creating LVM volume vgbck/lvmysql
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Creating LVM VG vgops
Creating LVM volume vgops/lvcachemgr
Creating LVM volume vgops/lvbackup
Creating LVM volume vgops/lvdata
Creating LVM volume vgops/lvdb
Creating LVM volume vgops/lvswl
Creating LVM volume vgops/lvcluster
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.

The system is now ready to restore from Bacula. You can use the 'bls' command
to get information from your Volume, and 'bextract' to restore jobs from your
Volume. It is assumed that you know what is necessary to restore - typically
it will be a full backup.

You can find useful Bacula commands in the shell history. When finished, type
'exit' in the shell to continue recovery.

WARNING: The new root is mounted under '/mnt/local'.

rear>
----


[[restoring-from-bacula-tape]]
==== Restoring from Bacula tape
Now you need to continue with restoring the actual Bacula backup, for this you
have multiple options of which +bextract+ is the most easy and
straightforward, but also the slowest and unsafest.


===== Using a bootstrap file
If you know the JobId of the latest successful full backup, and differential
backups the most efficient way to restore is by creating a bootstrap file with
this information and using it to restore from tape.

A bootstrap file looks like this:

----
Volume = VOL-1234
JobId = 914
Job = Bkp_Daily
----

or

----
Volume = VOL-1234
VolSessionId = 1
VolSessionTime = 108927638
----

Using a bootstrap file with bextract is easy, simply do:
+bextract -b bootstrap.txt Ultrium-1 /mnt/local+

TIP: It helps to know exactly how many files you need to restore, and using
     the +FileIndex+ and +Count+ keywords so +bextract+ does not require to
     read the whole tape. Use the commands in your shell history to access
     an example Bacula bootstrap file.


===== Using bextract
To use +bextract+ to restore *everything* from a single tape, you can do:
+bextract -V VOLUME-NAME Ultrium-1 /mnt/local+

----
rear> bextract -V VOL-1234 Ultrium-1 /mnt/local
bextract: match.c:249-0 add_fname_to_include prefix=0 gzip=0 fname=/
bextract: butil.c:282 Using device: "Ultrium-1" for reading.
30-Mar 16:00 bextract JobId 0: Ready to read from volume "VOL-1234" on device "Ultrium-1" (/dev/st0).
bextract JobId 0: -rw-r-----   1 252      bacula     3623795 2011-03-30 11:02:18  /mnt/local/var/lib/bacula/bacula.sql
bextract JobId 0: drwxr-xr-x   2 root     root          4096 2011-02-02 11:48:28  *none*
bextract JobId 0: drwxr-xr-x   4 root     root          1024 2011-02-23 13:09:53  *none*
bextract JobId 0: drwxr-xr-x  12 root     root          4096 2011-02-02 11:50:00  *none*
bextract JobId 0: -rwx------   1 root     root             0 2011-02-02 11:48:24  /mnt/local/.hpshm_keyfile
bextract JobId 0: -rw-r--r--   1 root     root             0 2011-02-22 12:38:03  /mnt/local/.autofsck
...
30-Mar 16:06 bextract JobId 0: End of Volume at file 7 on device "Ultrium-1" (/dev/st0), Volume "VOL-1234"
30-Mar 16:06 bextract JobId 0: End of all volumes.
30-Mar 16:07 bextract JobId 0: Alert: smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
30-Mar 16:07 bextract JobId 0: Alert: Home page is http://smartmontools.sourceforge.net/
30-Mar 16:07 bextract JobId 0: Alert:
30-Mar 16:07 bextract JobId 0: Alert: TapeAlert: OK
30-Mar 16:07 bextract JobId 0: Alert:
30-Mar 16:07 bextract JobId 0: Alert: Error counter log:
30-Mar 16:07 bextract JobId 0: Alert:            Errors Corrected by           Total   Correction     Gigabytes    Total
30-Mar 16:07 bextract JobId 0: Alert:                ECC          rereads/    errors   algorithm      processed    uncorrected
30-Mar 16:07 bextract JobId 0: Alert:            fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
30-Mar 16:07 bextract JobId 0: Alert: read:       1546        0         0         0       1546          0.000           0
30-Mar 16:07 bextract JobId 0: Alert: write:         0        0         0         0          0          0.000           0
165719 files restored.
----

WARNING: In this case +bextract+ will restore all the Bacula jobs on the
         provided tapes, start from the oldest, down to the latest. As a
         consequence, deleted files may re-appear and the process may take
         a very long time.


==== Finish recovery process
Once finished, continue Relax-and-Recover by typing +exit+.

----
rear> exit
Did you restore the backup to /mnt/local ? Ready to continue ? y
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 4424 seconds.
----

IMPORTANT: If you neglect to perform this last crucial step, your new system
           will not boot and you have to install a boot-loader yourself
           manually, or re-execute this procedure.


=== USB storage as backup media

==== Configuring Relax-and-Recover for backup to USB storage device
The below configuration (_/etc/rear/local.conf_) gives a list of possible
options when you want to run Relax-and-Recover with USB storage.

[source,bash]
----
BACKUP=NETFS
OUTPUT=USB
USB_DEVICE=/dev/disk/by-label/REAR-000

### Exclude certain items
ONLY_INCLUDE_VG=( vg00 )
EXCLUDE_MOUNTPOINTS=( /data )
----


==== Making the DR backup to USB storage device
Creating a combined rescue device that integrates the backup on USB, it is
sufficient to run +rear -v mkbackup+ as shown below after you have inserted
the USB device. Make sure the device name for the USB device is what is
configured.

----
[root@system ~]# rear -v mkbackup
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Creating archive 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz'
Total bytes written: 3644416000 (3.4GiB, 5.5MiB/s) in 637 seconds.
Writing MBR to /dev/sda
Modifying local GRUB configuration
Copying resulting files to usb location
Finished in 747 seconds.
----

IMPORTANT: It is advised to go into single user mode (+init 1+) before creating
           a backup to ensure all active data is consistent on disk (and no
           important processes are active in memory)


==== Booting from USB storage device
See the section <<booting-from-usb,Booting from USB storage device>> for more
information about how to enable your BIOS to boot from a USB storage device.


==== Restoring a backup from USB storage device
Then wait for the system to boot until you get the prompt.

On the shell prompt, type +rear recover+.

You may need to answer a few questions depending on your hardware
configuration and whether you are restoring to a (slightly)
different system.

----
RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Backup archive size is 1.2G (compressed)
To recreate HP SmartArray controller 1, type exactly YES: YES
To recreate HP SmartArray controller 7, type exactly YES: YES
Clearing HP SmartArray controller 1
Clearing HP SmartArray controller 7
Recreating HP SmartArray controller 1|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 7|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c1d0 (msdos)
Creating software RAID /dev/md126
Creating software RAID /dev/md127
Creating LVM PV /dev/md127
Restoring LVM VG vg00
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /boot on /dev/md126
Mounting filesystem /boot
Creating ext3-filesystem /data on /dev/mapper/vg00-lvdata
Mounting filesystem /data
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating swap on /dev/mapper/vg00-lvswap
Disk layout created.
Restoring from 'usb:///dev/sda1/system.localdomain/20110326.0216/backup.tar.gz'
Restored 3478 MiB in 134 seconds [avg 26584 KiB/sec]
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 278 seconds.
----

If all is well, you can now remove the USB device, restore the BIOS boot order
and reboot the system into the recovered OS.


== Using Relax-and-Recover with OBDR tapes
Using One-Button-Disaster-Recovery (OBDR) tapes has a few benefits.

 - Within large organisations tape media is already *part of a workflow*
   for offsite storage and is a *known and trusted technology*

 - Tapes can store large amounts of data reliably and restoring large
   amounts of data is *predictable* in time and effort

 - OBDR offers *booting from tapes*, which is very convenient

 - A single tape can hold both the rescue image as well as a *complete
   snapshot* of the system (up to 1.6TB with LTO4)

However, you need one tape per system as an OBDR tape can only store one
single rescue environment.


=== Configuring Relax-and-Recover for OBDR rescue tapes
The below configuration (_/etc/rear/local.conf_) gives a list of possible
options when you want to run Relax-and-Recover with a tape drive. This
example shows how to use the tape *only* for storing the rescue image,
the backup is expected to be handled by Bacula and so the Bacula tools
are included in the rescue environment to enable a Bacula restore.

[source,bash]
----
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
----


=== Preparing your OBDR rescue tape
To protect normal backup tapes (in case tape drives are also used by another
backup solution) Relax-and-Recover expects that the tape to use is labeled
*REAR-000*.  To achieve this is to insert a blank tape to use for
Relax-and-Recover and run the +rear format /dev/stX+ command.


=== OBDR tapes as rescue media

==== Configuring Relax-and-Recover to have Bacula tools
If the rescue environment needs additional tools and workflow, this can be
spcified by using +BACKUP=BACULA+ in the configuration file
_/etc/rear/local.conf_:

[source,bash]
----
BACKUP=BACULA
OUTPUT=OBDR
BEXTRACT_DEVICE=Ultrium-1
BEXTRACT_VOLUME=VOL-*
----

Using the +BEXTRACT_DEVICE+ allows you to use the tape device that is
referenced from the Bacula configuration. This helps in those cases where the
discovery of the various tape drives has already been done and configured in
Bacula.

The +BEXTRACT_VOLUME+ variable is optional and is only displayed in the
restore instructions on screen as an aid during recovery.


==== Making the OBDR rescue tape
To create a rescue environment that can boot from an OBDR tape, simply run
+rear -v mkrescue+ with a *REAR-000* -labeled tape inserted.

----
[root@system ~]# rear -v mkrescue
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Rewinding tape
Writing OBDR header to tape in drive '/dev/nst0'
Creating disk layout.
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-dag-ops.iso (48M)
Writing ISO image to tape
Modifying local GRUB configuration
Finished in 119 seconds.
----

WARNING: The message above about _/dev/cciss/c1d0_ not being used makes sense
as this is not a real disk but simply an entry for manipulating the controller.
This is specific to CCISS controllers with only a tape device attached.


[[booting-from-obdr]]
==== Booting from OBDR rescue tape
The One Button Disaster Recovery (OBDR) functionality in HP LTO Ultrium drives
enables them to emulate CD-ROM devices in specific circumstances (also known
as being in ''Disaster Recovery'' mode). The drive can then act as a boot
device for PCs that support booting off CD-ROM.

TIP: An OBDR capable drive can be switched into CD-ROM mode by *powering on
     with the eject button held down*. Make sure you keep it pressed when the
     tape drive regains power, and then release the button. If the drive is in
     OBDR mode, the light will blink regularly. This might be easier in some
     cases than the below procedure, hence the name One Button Disaster
     Recovery !


===== Using a HP Smart Array controller
To boot from OBDR, boot your system with the Relax-and-Recover tape inserted.
During the boot sequence, interrupt the HP Smart Array controller with the
tape attached by pressing *F8* (or *Escape-8* on serial console).

----
iLO 2 v1.78 Jun 10 2009 10.5.20.171

Slot 0 HP Smart Array P410i Controller       (512MB, v2.00)   1 Logical Drive
Slot 3 HP Smart Array P401 Controller        (512MB, v2.00)   1 Logical Drive
Slot 4 HP Smart Array P212 Controller          (0MB, v2.00)   0 Logical Drives
     Tape or CD-ROM Drive(s) Detected:
         Port 1I: Box 0: Bay 4
1785-Slot 4 Drive Array Not Configured
     No Drives Detected


  Press <F8> to run the Option ROM Configuration for Arrays Utility
  Press <ESC> to skip configuration and continue
----

Then select *Configure OBDR* in the menu and select the Tape drive by marking
it with *X* (default is on) and press *ENTER* and *F8* to activate this change
so it displays ''Configuration saved''.

Then press *ENTER* and *Escape* to leave the Smart Array controller BIOS.

----
**** System will boot from Tape/CD/OBDR device attached to Smart Array.
----


===== Using an LSI controller
To boot from OBDR when using an LSI controller, boot your system with the
Relax-and-Recover tape inserted. During the boot sequence, interrupt the
LSI controller BIOS that has the tape attached by pressing *F8* (or
*Escape-8* on serial console).

----
LSI Logic Corp. MPT BIOS
Copyright 1995-2006 LSI Logic Corp.
MPTBIOS-5.05.21.00
HP Build

<<<Press F8 for configuration options>>>
----

Then select the option +1. Tape-based One Button Disaster Recovery (OBDR).+:

----
Select a configuration option:
1. Tape-based One Button Disaster Recovery (OBDR).
2. Multi Initiator Configuration.                                 <F9 = Setup>
3. Exit.
----

And then select the correct tape drive to boot from:

----
   compatible tape drives found       ->
   NUM   HBA   SCSI ID   Drive information
    0     0       A       - HP       Ultrium 2-SCSI

   Please choose the NUM of the tape drive to place into OBDR mode.
----

If all goes well, the system will reboot with OBDR-mode enabled:

----
    The PC will now reboot to begin Tape Recovery....
----

During the next boot, OBDR-mode will be indicate by:

----
*** Bootable media located, Using non-Emulation mode ***
----


===== Booting the OBDR tape
Once booted from the OBDR tape, select the 'Relax-and-Recover' menu entry from
the menu. If you don't press a key within 30 seconds, the system will try to
boot from the local disk.

["aafigure",width="12cm",height="6cm",align="center",format="svg",options="textual",aspect="0.7",linewidth="1"]
----
+---------------------------------------------+
|        "Relax-and-Recover v1.12.0svn497"    |
+---------------------------------------------+
|  "Relax-and-Recover"                        |
|---------------------------------------------|
|  "Other actions"                            |
|    "Help for Relax-and-Recover"             |
|    "Boot Local disk (hd1)"                  |
|    "Boot BIOS disk (0x81)"                  |
|    "Boot Next BIOS device"                  |
|    "Hardware Detection tool"                |
|    "Memory test"                            |
|    "Reboot system"                          |
|    "Power off system"                       |
|                                             |
|                                             |
+---------------------------------------------+

      "Press [Tab] to edit options or [F1] for help"

           "Automatic boot in 30 seconds..."
----

////
      .-------------------------------------------------------------.
      |               Relax-and-Recover v1.12.0svn497               |
      |-------------------------------------------------------------|
      |  Relax-and-Recover                                          |
      |-------------------------------------------------------------|
      |  Other actions                                              |
      |   Help for Relax-and-Recover                                |
      (>  Boot Local disk (hd0)                                    <)
      |   Boot BIOS disk (0x80)                                     |
      |   Boot Next BIOS device                                     |
      |   Hardware Detection tool                                   |
      |   Memory test                                               |
      |   Reboot system                                             |
      |   Power off system                                          |
      `-------------------------------------------------------------'

             Press [Tab] to edit options or [F1] for help

                        Automatic boot in 30 seconds...
////


==== Restoring the OBDR rescue tape
Then wait for the system to boot until you get the prompt.

On the shell prompt, type +rear recover+.

You may need to answer a few questions depending on your hardware
configuration and whether you are restoring to a (slightly)
different system.

----
RESCUE SYSTEM:/ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
Rewinding tape
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Creating LVM VG vgrem
Creating LVM VG vgqry
Creating LVM VG vg00
Creating LVM volume vg00/lv00
Creating LVM volume vg00/lvdstpol
Creating LVM volume vg00/lvsys
Creating LVM volume vg00/lvusr
Creating LVM volume vg00/lvtmp
Creating LVM volume vg00/lvvar
Creating LVM volume vg00/lvopt
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Creating LVM VG vgbck
Creating LVM volume vgbck/lvetc
Creating LVM volume vgbck/lvvar
Creating LVM volume vgbck/lvmysql
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Creating LVM VG vgops
Creating LVM volume vgops/lvcachemgr
Creating LVM volume vgops/lvbackup
Creating LVM volume vgops/lvdata
Creating LVM volume vgops/lvdb
Creating LVM volume vgops/lvswl
Creating LVM volume vgops/lvcluster
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.

The system is now ready to restore from Bacula. You can use the 'bls' command
to get information from your Volume, and 'bextract' to restore jobs from your
Volume. It is assumed that you know what is necessary to restore - typically
it will be a full backup.

You can find useful Bacula commands in the shell history. When finished, type
'exit' in the shell to continue recovery.

WARNING: The new root is mounted under '/mnt/local'.

rear>
----


==== Restoring from Bacula tape
See the section <<restoring-from-bacula-tape,Restoring from Bacula tape>>
for more information about how to restore a Bacula tape.


=== OBDR tapes as backup media
An OBDR backup tape is similar to an OBDR rescue tape, but next to the rescue
environment, it also consists of a complete backup of the system. This is
very convenient in that a single tape can be use for disaster recovery, and
recovery is much more simple and completely automated.

CAUTION: Please make sure that the system fits onto a single tape uncompressed.
         For an LTO4 Ultrium that would mean no more than 1.6TB.


==== Configuring Relax-and-Recover for OBDR backup tapes
The below configuration (_/etc/rear/local.conf_) gives a list of possible
options when you want to run Relax-and-Recover with a tape drive. This example
shows how to use the tape for storing *both* the rescue image and the backup.

[source,bash]
----
BACKUP=NETFS
OUTPUT=OBDR
TAPE_DEVICE=/dev/nst0
----


==== Making the OBDR backup tape
To create a bootable backup tape that can boot from OBDR, simply run
+rear -v mkbackup+ with a *REAR-000* -labeled tape inserted.

----
[root@system ~]# rear -v mkbackup
Relax-and-Recover 1.12.0svn497 / 2011-07-11
Rewinding tape
Writing OBDR header to tape in drive '/dev/nst0'
Creating disk layout
Creating root filesystem layout
Copying files and directories
Copying program files and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-system.iso (45M)
Writing ISO image to tape
Creating archive '/dev/nst0'
Total bytes written: 7834132480 (7.3GiB, 24MiB/s) in 317 seconds.
Rewinding tape
Modifying local GRUB configuration
Finished in 389 seconds.
----

IMPORTANT: It is advised to go into single user mode (+init 1+) before creating
           a backup to ensure all active data is consistent on disk (and no
           important processes are active in memory)


==== Booting from OBDR backup tape
See the section <<booting-from-obdr,Booting from OBDR rescue tape>> for more
information about how to enable OBDR and boot from OBDR tapes.


==== Restoring from OBDR backup tape

----
RESCUE SYSTEM:~ # rear recover
Relax-and-Recover 1.12.0svn497 / 2011-07-11
NOTICE: Will do driver migration
Rewinding tape
To recreate HP SmartArray controller 3, type exactly YES: YES
To recreate HP SmartArray controller 0, type exactly YES: YES
Clearing HP SmartArray controller 3
Clearing HP SmartArray controller 0
Recreating HP SmartArray controller 3|A
Configuration restored successfully, reloading CCISS driver...  OK
Recreating HP SmartArray controller 0|A
Configuration restored successfully, reloading CCISS driver...  OK
Comparing disks.
Disk configuration is identical, proceeding with restore.
Type "Yes" if you want DRBD resource rBCK to become primary: Yes
Type "Yes" if you want DRBD resource rOPS to become primary: Yes
Start system layout restoration.
Creating partitions for disk /dev/cciss/c0d0 (msdos)
Creating partitions for disk /dev/cciss/c2d0 (msdos)
Creating software RAID /dev/md2
Creating software RAID /dev/md6
Creating software RAID /dev/md3
Creating software RAID /dev/md4
Creating software RAID /dev/md5
Creating software RAID /dev/md1
Creating software RAID /dev/md0
Creating LVM PV /dev/md6
Creating LVM PV /dev/md5
Creating LVM PV /dev/md2
Restoring LVM VG vgrem
Restoring LVM VG vgqry
Restoring LVM VG vg00
Creating ext3-filesystem / on /dev/mapper/vg00-lv00
Mounting filesystem /
Creating ext3-filesystem /dstpol on /dev/mapper/vg00-lvdstpol
Mounting filesystem /dstpol
Creating ext3-filesystem /dstpol/sys on /dev/mapper/vg00-lvsys
Mounting filesystem /dstpol/sys
Creating ext3-filesystem /usr on /dev/mapper/vg00-lvusr
Mounting filesystem /usr
Creating ext2-filesystem /tmp on /dev/mapper/vg00-lvtmp
Mounting filesystem /tmp
Creating ext3-filesystem /boot on /dev/md0
Mounting filesystem /boot
Creating ext3-filesystem /var on /dev/mapper/vg00-lvvar
Mounting filesystem /var
Creating ext3-filesystem /opt on /dev/mapper/vg00-lvopt
Mounting filesystem /opt
Creating swap on /dev/md1
Creating DRBD resource rBCK
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd2
Restoring LVM VG vgbck
Creating ext3-filesystem /etc/bacula/cluster on /dev/mapper/vgbck-lvetc
Mounting filesystem /etc/bacula/cluster
Creating ext3-filesystem /var/bacula on /dev/mapper/vgbck-lvvar
Mounting filesystem /var/bacula
Creating ext3-filesystem /var/lib/mysql/bacula on /dev/mapper/vgbck-lvmysql
Mounting filesystem /var/lib/mysql/bacula
Creating DRBD resource rOPS
Writing meta data...
initializing activity log
New drbd meta data block successfully created.
Creating LVM PV /dev/drbd1
Restoring LVM VG vgops
Creating ext3-filesystem /opt/cache on /dev/mapper/vgops-lvcachemgr
Mounting filesystem /opt/cache
Creating ext3-filesystem /dstpol/backup on /dev/mapper/vgops-lvbackup
Mounting filesystem /dstpol/backup
Creating ext3-filesystem /dstpol/data on /dev/mapper/vgops-lvdata
Mounting filesystem /dstpol/data
Creating ext3-filesystem /dstpol/databases on /dev/mapper/vgops-lvdb
Mounting filesystem /dstpol/databases
Creating ext3-filesystem /dstpol/swl on /dev/mapper/vgops-lvswl
Mounting filesystem /dstpol/swl
Creating ext3-filesystem /dstpol/sys/cluster on /dev/mapper/vgops-lvcluster
Mounting filesystem /dstpol/sys/cluster
Disk layout created.
Restoring from 'tape:///dev/nst0/system/backup.tar'
Restored 7460 MiB in 180 seconds [avg 42444 KiB/sec]
Installing GRUB boot loader

Finished recovering your system. You can explore it under '/mnt/local'.

Finished in 361 seconds.
----
