This Bacula feature backs up large numbers of files with an optimal speed and efficiency splitting the workload in a number of workers (concurrent backup processes).
Bacula’s Large Backup File Split will save significant time in certain demanding IT enviuronments, such as supercomputing and HPC environments when dealing with very large datasets. It can work with any kind of file system, and the backup is automatically managed by Bacula.
This capability is applicable to any file-server or NAS use-case. For example, a large government laboratory that needs to backup an extremely large set of files, but would be unhappy to lose significant resources and time in both manually splitting up the system to backup into smaller pieces to benefit from some concurrency or waiting for a long time to see analysis processes before protecting the information, while also managing the complete backup process.
Bacula resolves the issue by automating the process and doing it through the most efficient possible form, and likely combining it with other Bacula functions, such as Incremental Backup. The process starts immediately on receiving the first file (no pre-processing is necessary), and is very efficient, using a dynamic ‘Workers’ process, where any free worker will protect the next file as soon as it finishes with the previous one, and where the system administrator can control the number of them.
Another attraction of Bacula’s automation of this process is its ability to reduce the possibility of human error, as well as being fully dynamic, meaning no configuration update process is necessary regardless of any changes in the structure or size of the target dataset.
This tool can especially help where there are many users causing a complex folder structure as a result, and lots of different files. This kind of file structure may literally be being used by everybody (e.g. public/private, internal company users, shared data situations, etc.).
