Rsync friendly compression

Hi AOMEI team! Thanks for your great product.

We wish that AOMEI have the ability to build rsync (and deduplication) friendly backup images. Currently, two images made from the same system leads to two complete unrelated files, so we can't gain any benefits with rsync or storage deduplication.

zlib allows the possibility to use Z_SYNC_FLUSH parameters to allow the compression output to be byte aligned. It does increase the size of the backup file, but the increase is minimal.

See https://github.com/madler/pigz/blob/master/pigz.c for an example.

Thanks
Tagged:

Comments

  • @Ldperron, Do you want to perform backup? or sync? The backup will create an image file. Sync will directly copy files to the destination.
    For Backup, our software actually does deduplication when performing backup.
  • It's using the backup functionality.
    - Since we store multiple system backups within a storage device that supports deduplication, having more byte alignment (for example, once every source input file + at some well chosen interval/pattern for bigger files) will increase the probability of having matching blocks between two different backups of the same system, and even backups of two different systems.
    - For all the remote workers, it will make the usage of rsync or remote differential compression way more efficient when an user needs to send his backup to the office through internet. As of 6.3.0, rsync effiency between two full system backups of the same machine (with little changes involved) is around 50%, which is pretty low.

    It may be impossible to improve... but it would help us a lot if it happens.

    Thanks
Sign In or Register to comment.