Deduplication
Learn about deduplication and how it's used in Xopero ONE backup.
The deduplication feature is not supported (from ver. 2.0.5) on 32-bit operating systems. Previously created backups remain backward compatible and can be restored using agents on 32-bit operating systems.
Understanding deduplication
Deduplication is a data compression technique that eliminates duplicate copies of repeating data. It ensures that only unique data instances are stored, significantly reducing the required storage space.
Deduplication benefits
Deduplication provides significant advantages in data backup. By removing duplicate data blocks, it can reduce storage requirements by 50% to 90%, depending on data type and redundancy level. This results in substantial cost savings in physical storage, data center space, and energy consumption.
Optimization with encryption
While deduplication provides significant benefits, its effectiveness can be reduced when combined with encryption. If different encryption keys are used for backup plans targeting the same storage repository, deduplication becomes less efficient, as encryption alters data blocks to the extent that duplicates are no longer recognized.
To maintain optimal deduplication and maximize storage savings, it is recommended to use the same encryption key for all backup plans within a single storage repository.
Encryption modifies even identical data blocks to the point that duplicates are no longer recognized. Using the same encryption key ensures that the deduplication process can accurately identify and remove redundant data, resulting in improved data management.
Deduplication in Xopero ONE
Xopero ONE uses software-based global deduplication at the source, employing block-level deduplication to identify and remove duplicate blocks within and across files (when backup is performed, the process scans data for duplicate blocks). Rather than storing multiple copies of the same block, it saves a single instance and replaces subsequent duplicates with a reference to the original block.
How it works
In Xopero ONE, deduplication is performed at the source, meaning it begins on the client side (agent) before data is transmitted to the storage repository. This deduplication method ensures that redundant data is eliminated early in the process. The backup agent scans the data for duplicate blocks by retrieving a deduplication table from the storage repository, which contains information about previously stored blocks. By comparing new data against this table, the agent can identify duplicates and avoid transmitting them, significantly reducing the amount of data sent over the network.
Bandwidth and time efficiency
The primary advantage of source-side deduplication in Xopero ONE is the substantial reduction in network bandwidth usage and backup execution time. By identifying and eliminating duplicates before data transmission, only unique data blocks are sent to the storage repository, which optimizes bandwidth efficiency, making it especially beneficial in environments with limited network capacity or high data transfer costs.
Configuring deduplication settings
When configuring deduplication with encryption, use the same encryption key for all backup plans within the same storage repository.
If you create a backup plan without deduplication, it cannot be enabled later— you can’t change deduplication settings in an already configured plan. To use deduplication in a backup plan, you must create a new one. The existing plan can be easily cloned during editing.
To enable deduplication in Xopero ONE, login to your Xopero ONE Management Service (XMS) and go to Plans > Backup > ➕ Add plan. When configuring a new backup plan, scroll to Advanced settings, click Edit, and toggle the Deduplication option to enable it.
Useful links and items
Cleanup of deduplication dataLast updated