Add block based backup (space/bandwidth efficient)
Currently Freezer checks only ctime and mtime inode information to verify if files are changed or not.
While this approach gives speed (time efficient), it is not bandwidth and storage efficient. Freezer needs to support both librsync and libarchive approach to execute incremental backups and restore.
- Freezerc has to be able to compute changes for blocks of files
- The blocks size needs to be configurable. By default will be 65K
- Signature for files blocks size needs to be generate for level 0 full backups
- The comparation needs to be done against:
- Available files (signature against local files)
- Available signatures (signature against signature)
- Available streams (signature against streams)
- The restore of the files can be done under the following conditions:
- Downloading valid incremental backups (i.e. level 0 to level n) and apply write
the differences on the file incrementally, block by block and in stream
- Implementation has to be memory efficient (operations are executed against blocks and not against whole files or whole streams)
- Implementation has to be Disk I/O/space efficient (operations are executed from and to stream of data and not anything like: download files/stream locally and then execute action against local files)
Implementation thoughts:
Methods/Functions:
1) gen_sig_
return a new full signature signature data struct (dict)
2) gen_sign_
return a new full signature signature data struct (dict)
3) gen_sign_
4) gen_sign_
5) patch_from_
6) patch_from_
Blueprint information
- Status:
- Started
- Approver:
- Fabrizio Fresco
- Priority:
- High
- Drafter:
- Fausto Marzi
- Direction:
- Approved
- Assignee:
- Ruslan Aliev
- Definition:
- Approved
- Series goal:
- Accepted for pike
- Implementation:
- Beta Available
- Milestone target:
- pike-2
- Started by
- Fausto Marzi
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
Code review:
- https:/
Gerrit topic: https:/
Addressed by: https:/
Support for rsync block based backups
Gerrit topic: https:/
Gerrit topic: https:/
Addressed by: https:/
[WIP] Block based incremental support - Rsync
Addressed by: https:/
[WIP] - Block based incremental support - rsync
Done by: https:/
another enhanced version exists here: https:/