Home AOMEI Products Support

How does Backupper detect file changes in Image backup

I am trying to figure out a way to backup a Veracrypt data. When I backup a Veracrypt partition, I notice that backupper is unable to integrate with Veracrypt and do a sector by sector partition. This is not desirable size-wise, a 5Tb partition that is mostly empty will result in a backup that is 5TB in size.

Instead of using a encrypted partition, I leave the partition as exfat and then create a giant veracrypt file container. What we end up with is a 5 Tb veracrypt file. I then proceed to image backup the volume. What I have notice is that while the backup is really slow (it has to process 5 TB of data, so it's going to take like 24 hours), the resulting backup only contains actual used data. A 5 Tb container with 200 Mb of files will  result in a 200 mb backup. If you add another file, and run increment or differential backup, only what was changed will be backup (but still takes like 24 hours even if the file is only 1K). I have tried restoring and it appears to work so far.

My question is how does this work? How does backupper notice the change in the veracrypt file? What could cause the backup to break?

Thanks.


Comments

  • Paulsiu, When you do the disk/partition backup, AOMEI Backupper will read sectors to back up.
    Generally, for the encrypted drive, AOMEI Backupper will use a sector-by-sector backup way by default. That's to say, it will back up all sectors, whether used or not.
    When you create the veracrypt file container, the properties of the volume is not encrypted (only files in the volume are encrypted), AOMEI Backupper will only back up used sectors.
    AOMEI Backupper actually will detect the change of sectors, not file changes. 
  • Thanks for the explanation. I have two follow up questions.
    1. How does Backupper know what sectors are in use? Since it only works on particular file system, does it query the OS to find out which sectors are in use?
    2. How does Backupper know which sectors are changed on an incremental or differential? Does it parse through the full backup while also parsing the disk and compare them as it scan and only write sectors that are different?

    Why does the backup take so long. A 5TB that has an empty container may take 24 hours.

  • Paulsiu, AOMEI Backupper will detect the used sectors via the bitmap of the file system. Incremental/Differential backup is performed by comparing the changes of these used sectors.
    For encrypted drive, AOMEI Backupper cannot parse the file system, so it takes a long time to back up the entire 5TB space sector by sector.
  • Actually, the question is more about an encrypted file container sitting on top of a unencrypted file system. Here's what I did.
    1. Create a 4.5 Tb encrypted container.
    2. Mount container and copy in 80 Gb of files.
    3. Dismount container and then do a full backup. The backup took 24 hours and resulted in a 80 gb file.
    4. Mount container and copy in 1 Mb of file.
    5. Dismount container and then do an incremental backup. The backup took 24 hours and resulted in a 1 Mb file.
    6. Test by formatting drive and then ask backupper to restore.

    All seems to work as expected, except for the backup time, which seems to take roughly 24 hours. If it can detect change in bitmap wouldn't the backup be faster?


  • @Paulsiu, "Mount container"---Did you mount the container to a virtual drive? What file system is the virtual drive using?
    Did you create a partition backup for the virtual drive?

  • So the way a veracrypt container works is that you mount the file as a drive letter. However, when you back up the file, you need all activity to stop, so you dismount it. You don't want writes to the container while it's being backed up.

    The drive is a 5 Tb drive (shows up as 4.5 TB under windows). I format the drive as exfat and then create the veracrypt container on it. Essentially, it's a 4.5 TB drive with 1 exfat partition for the whole drive, and the partition has 1 file on it that's 4.5 Tb. I used drive backup but with 1 partition, it would be the same as a partition backup. Keep in mind that there is no system on the drive so I can't use system backup.

  • Paulsiu, For exfat partition, AOMEI Backupper will back up it with sector-by-sector way. So, it will back up all space.
    And, it seems that the veracrypt container file on the drive is incompressible. So, the backup file is approximately equal to the size of file.
  • Thank you for your reply.
    I think there's been a miscommunication, I thought I was being clear on my previous post. The current situation is:
    • The drive is a 4.5 TB Usb external drive. It is not a boot disk.
    • The drive is formatted with exfat.
    • There is a single 4.5 Tb veracrypt file container
    • When you fill in 80 Gb of data into the veracrypt file container and then backup the drive, it correctly generates a 80 Gb backup file. If it were doing a sector by sector, I would have gotten a 4.5 tb file. Restore works without issue.
    • When you add 1 Mb on top of the 80 Gb you added in the earlier step and then do a incremental backup, it correctly generates a 1 mb increment backup file. Restore also work without issue.
    My only issue is that it takes a long time to do a backup. A full backup seems to take 24 hours even thought it backed up only 80 Gb. If I do an increment, it also takes 24 hours or so. Why does it take so long to do a backup if it can detect the sector changed?
  • @Paulsiu, For exfat partition, AOMEI Backupper will use sector by sector backup way. It will read every sector to back up.
    Generally,  it will compress your data to the backup image. The backup size is affected by the data amount of compressable, and the total size of backup data. For the empty sectors, the compress level will be high. For the 80gb file, if it is incompressible or the compressible level is low, so the size of backup file will be close to the size of data.
    It seems that these files in veracrypt file container can't be compressed by our program.
  • What if the file system is NTFS. how would AOMEI backup that? Does it use the same sector by sector backup.

    Paul

  • Just to clarify, you were saying that the reason it took so long is because AOMEI will have to read every sector to backup the exfat volume. Do we need to do the same thing for NTFS backup? Is NTFS backup inherently faster (note I am referring to image backup).
  • OK, I did some experimentation. NTFS are significantly faster than exfat. is this because backupper is taking advantage of journaling?
  • @Paulsiu, For NTFS and FAT32 partitions (not encrypted), AOMEI Backupper will only back up used sectors. Yes, it will detect the used sectors via the bitmap of the file system and back up them.
Sign In or Register to comment.