NDMP backup software solution from Bacula

Full, Incremental and Incremental Forever NDMP backup of filer data incl. ACLs, extended configuration and parameters. Single File Restore.

The Bacula Systems NDMP backup module makes use of the filer-to-server mode for easy integration into Bacula Enterprise.

This tool reduces demands on the network and enables localized NDMP backup and disaster recovery. In addition – and so important in the real world – it provides a fast, easy and full restore.

NDMP Protocol is an industry standard protocol created to ease the process of integrating backup and restore for network

attached storage (NAS) with data protection software such as Bacula Enterprise. Many storage hardware manufacturers have already incorporated NDMP support as a feature in their products, which makes their configuration and implementation much simpler.

The NDMP protocol is an open protocol, an industry standard that allows data transfers between storage and backup devices directly, eliminating the use of resources on servers, not only memory and network, but also also the use of the network bandwidth. NDMP implementation requires the use of hardware that supports its operation, and these include NAS storage devices and tape backup units.

ndmp backup architecture

NDMP backup architecture

Bacula’s NDMP Backup Module Benefits

#
Decrease in the execution times of NDMP backup and recovery operations
#
Optimized use of server resources, as well as storage devices and network bandwidth
#
Savings and optimization of the disk space
#
Possibility of having more tools and resources to configure and define backup policies in organizations, for heterogeneous environments and diverse scenarios

NDMP backup with Bacula’s module can significantly outperform the one running against mounted NFS/CIFS volumes, especially when dealing with millions of small files and ACLs.  With the NDMP backup software from Bacula Systems, the average backup speed is increasing significantly while decreasing the bandwidth load.

NDMP backup is designed ideally to protect NAS filers in major disaster scenarios. Bacula’s NDMP backup software enables full and incremental backups of the filer data, including ACLs, as well as its extended configuration and parameters.

Additionally, NDMP backup use in conjunction with specific Bacula Enterprise features, such as Progresive Virtual Full or Global Endpoint Deduplication, delivers excellent response times in daily backup and recovery operations.

Bacula Enterprise allows to backup NDMP with more flexibility in defining policies and schedule. You can consider more variables and criterias in your backup strategy, such as: criticalness and amount of data, schedules, applications, databases, operating systems, etc..

Restoring a single file with Bacula’s NDMP backup tool is possible either by using the built-in NAS snapshot capability or by restoring the NDMP datastream to a local machine.

Backup Target Variety

Bacula Systems NDMP backup plugin allows you to select between several possible backup targets to find the one (or several) that work best for you. The NDMP backup module supports backup targets like tape, disk and cloud so you’ll have a solution for any possible situation from short-term backup and recovery processes, to long-term data archiving and off-premises disaster recovery.

#
NDMP Backup to Tape

Tape backup remains an excellent target medium for reliable, low-error, long-term stable backup storage that offers storage stability measured in years, all at a low cost-per-gigabyte. This target choice still allows for partial file restoration as well, so you don’t have to restore the entire tape worth of data to get a single file from it. Tape also provides portability that allows it to be stored offsite and in secure locationsup to audited bunker facilities to meet your security requirements for those backups. Tape also remains one of the only ways to move petabytes of data over long distances to offsite storage, with low risk and minimal transport costs.

#
NDMP Backup to Disk

Backup to disk is one of the most common backup targets, and providesa near-line data copy and the ability to access it for your daily data recovery purposes (not including what you might already have with basic NAS snapshots). Disk backup is also often used in multi-site strategies that sync data between datacenters for disaster recovery and business continuity purposes. Disk backup is generally more expensive per gigabyte than tape, especially when considering ongoing operating costs and power budget. However, the advantages of already-online fast disk storage for backups are often enough to justify the costs and higher bit-rot rate.

#
NDMP Backup to Cloud

Backup using cloud storage providers canbe an economic tape alternative and  also suitable for long-term data storage if the cloud provider SLA supports it. For most users of cloud storage for backup purposes, the behavior is much like tape. It’s an offsite, reliable datastore that requires planning and time during the recovery process to return data to the primary site. Cloud offers many other distinct advantages over tape though, in that a datacenter could be created in a cloud provider essentially on-demand, so a cloud-based disaster recovery plan may include direct in-cloud pathways. It is worth mentioning that cloud target is quite effective, but if you’re working with a substantial amount of data to backup every day it is important to keep an eye on and be aware of the costs for your data storage, and especially costs for data access if they differ from those for storage. Cloud data storage costs can be quite substantial.

NDMP Backup Options

The NDMP standard allows for three basic backup architectures – filer-to-filer backup, local backup and remote backup or Filer-to-server backup.

Filer-to-Filer Backup Architecture

In the filer-to-filer model, filers (aka NAS or SAN storage devices) send and receive data to each other. In a simple setup, one filer is attached to the NAS only and the other one is attached to your tape library. Backups flow from the first filer to the second, and then on to the storage medium. This architecture allows for better usage of tape library resources since there can be multiple sources – filers – that are filling the same part of the tape library.

Local Backup Architecture

This backup option implies that your tape library is directly connected to your NAS, and the data moves from each NAS directly to the tape library with no third parties in this connection. The direct connection allows for faster data transfer overall, but might be the limiting factor for some types of backup. The usual protocols used in the connection between your NAS and your tape library are either Fibre Channel or SCSI. This type of backup sends backup data over the SAN which may create congestion or slowdowns for the clients accessing data.

Remote Backup Architecture/Filer-to-server backup

The remote backup method includes a new party in the connection between your NAS and your designated data storage – the backup server. Data travels from the filer to the backup server, and then on to the desired target storage.. This allows for multiple backup targets for NDMP backups, including NDMP-to-cloud strategies, and also allows the backup data stream to be taken onto a different network, usually ethernet, and off of the SAN. l. A single backup server can support multiple filers for ease of management and scalability. This is the preferred mode for Bacula Systems NDMP backups.

The Bacula Enterprise NDMP module, like most plugins, is installed with a Bacula File Daemon. This gives the File Daemon the necessary functionality to contact the NDMP-speaking network file device and request the data be sent back. Normally, this File Daemon is installed on the same server that hosts a Bacula Enterprise Storage Daemon which will write the data to disk or tape. The file daemon can be on a separate machine, but this will induce an additional network hop to send the data on to the storage daemon, so best practice has them on the same machine.

Assuming Bacula Enterprise is already installed, adding the NDMP backup module is as simple as:

  • Add the NDMP backup module repository to your yum/apt repository file.Yum example here, added to /etc/yum.repos.d/bacula.repo:

[ndmp]
name=Bacula Enterprise NDMP plugin
baseurl=https://my.bacula.repo/rpms/ndmp/10.2.2/rhel7-64
gpgcheck=0
  • Yum/apt install bacula-enterprise-ndmp (see sample below for RHEL 7)

[root@bacula yum.repos.d]# yum install bacula-enterprise-ndmp
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.unifiedlayer.com
* extras: mirrors.unifiedlayer.com
* updates: mirrors.unifiedlayer.com
Resolving Dependencies
–> Running transaction check
—> Package bacula-enterprise-ndmp.x86_64 0:10.2.2-19010920.el7 will be installed
–> Finished Dependency ResolutionDependencies Resolved
=====================
Package Arch Version Repository Size
=====================
Installing:
bacula-enterprise-ndmp x86_64 10.2.2-19010920.el7 ndmp 582 kTransaction Summary
=====================
Install 1 PackageTotal download size: 582 k
Installed size: 2.8 M
Is this ok [y/d/N]: y
Downloading packages:
bacula-enterprise-ndmp-10.2.2-19010920.el7.x86_64.rpm | 582 kB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : bacula-enterprise-ndmp-10.2.2-19010920.el7.x86_64 1/1
Verifying : bacula-enterprise-ndmp-10.2.2-19010920.el7.x86_64 1/1
Installed:
bacula-enterprise-ndmp.x86_64 0:10.2.2-19010920.el7
Complete!
  • Ensure that the plugin line exists in your bacula-fd.conf file. Note here the “PluginDirectory = “/opt/bacula/plugins” line

FileDaemon {
Name = “bacula-fd”
MaximumConcurrentJobs = 20
PidDirectory = “/opt/bacula/working”
PluginDirectory = “/opt/bacula/plugins”
WorkingDirectory = “/opt/bacula/working”
}
  • Restart the bacula-fd service, enter bconsole, and check the status of the client. The plugin should appear in the output here.

[root@bacula yum.repos.d]# service bacula-fd restart
Redirecting to /bin/systemctl restart bacula-fd.service[root@bacula yum.repos.d]# bconsole
Connecting to Director 192.168.56.101:9101
1000 OK: 10002 bacula-dir Version: 10.0.4 (22 August 2018)
Enter a period to cancel a command.
*status client=bacula-fd
Connecting to Client bacula-fd at bacula:9102bacula-fd Version: 10.2.2 (09 January 2019)
x86_64-redhat-linux-gnu-bacula-enterprise redhat Enterprise release
Daemon started 05-Feb-19 19:37. Jobs: run=0 running=0.
Heap: heap=135,168 smbytes=205,814 max_bytes=205,814 bufs=111 max_bufs=111
Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,2010 bwlimit=0kB/s
Plugin: bpipe-fd.so ndmp-fd.so
  • And with the plugin installed, the last step is to create a new fileset and related Bacula backup job for each NDMP device to back up. There are too many options to show in this guide, so please refer to Bacula Systems Support for assistance beyond what is shown here. A sample fileset would look like the following. A backup job configured to use this fileset would connect to the network storage device at the IP address listed, and back up the entire contents of vol1 on that device.:

FileSet {
Name = NDMP_home
Include {
Plugin=”ndmp:host=10.0.100.200 user=root pass=root volume=/vol/vol1”
}
}

Possible NDMP backup modes and their compatibility with different backup levels

The NDMP backup plugin supports the following NDMP formats:

  • DUMP – a vendor specific NDMP backup format
  • TAR – a standard (supported by many vendors) NDMP backup format
  • SMTAPE – NetApp specific raw binary format, it is favoured because it offers relatively fast backups

These are some prominent features of the NDMP backup plugin:

  • Incremental backup – allows for a possibility to backup only the objects that were modified after the previous backup run
  • Accurate support – keeping a track of renamed/deleted objects, it depends on the vendor platform
  • Exclusion/Inclusion – there is a possibility to include only certain directories from NDMP volumes
  • ACL – file Access Control Lists (ACLs) are backed up and can be fully restored using NDMP
  • File Level Restore – also known as Single Item Restore (SIR), allows for a restore of a single or a group of backed up objects, in contrast to a restore of a full backup data set

Not all of the NDMP formats can be used with all the features listed, the following table depicts their interoperability:

Feature \ Format DUMP/TAR SMTAPE
Incremental backup Yes Yes
Accurate support Yes Yes
Exclusion/Inclusion Yes No
ACL Yes Yes
File Level Restore Yes No

Further help on NDMP backup:

  • BWeb™ Management Suite is a comprehensive GUI management suite for Bacula Enterprise that provides the data reports and metrics during NDMP backups.
  • Training is available in different locations, depending on the Certified Bacula Systems Training Center you choose.