blockdiff is a binary patch tool for block base file and disk formats (like ext2,3,4 and btrfs). It's similar to bsdiff but not as general, because blockdiff is built on a more stricter assumption about the internal file format. The diff algorithm only considers very long byte sequences, blocks of around ~4 KiB, not single bytes. That's why the runtime and memory usage of blockdiff is minimal compared to bsdiff. Of course at the cost of not being as general applicable as bsdiff. The main usage area of blockdiff should be filesystem based A/B Updates of embedded devices.
For the commandline interface and examples see blockdiff's manpage.
For more information about binary patch tools see bsdiff's homepage, the Wikipedia article Delta Update and Google's Chrome Updater Courgette.
NOTE: The commandline interface and the patch file format of blockdiff is not considered stable yet. Wait for the 1.0.0 release :-)
Features and Limitations
Features:
- The operation
blockdiff diff
runs in O(n + m) time and memory usage is limited by O(n) with n := 'blocks in the source file' and m := 'blocks in the target file'. - The operation
blockdiff patch
runs in O(m) time and the memory usage is constant with m := 'blocks in target file'. - Patch file format is streamable. You can pipe out the patch file in
blockdiff diff
and pipe in the patch file inblockdiff patch
. No need to download the whole patch file before processing it. - Every byte in the patch file format is protected by CRC32. Any data corruption in the patch file is detected by
blockdiff patch
orblockdiff info
. - The target file is checked against a cryptographic checksum while it's written in
blockdiff patch
. Whenblockdiff patch
has finished successfully you can be totally sure that the patch operation has produced the identical file as it has existed on while doingblockdiff diff
on the sender machine. - You can choose the cryptographic checksum that is stored in the patch file. Available options are (MD5, SHA1, SHA256 and SHA512). You can select the same checksum in the patch file as you used for your cryptographic signatures. Note: That's only for convenience, for not security, since if you cannot trust the patch file, you can also not trust the cryptographic checksum in the patch file. The final target file must be verified another external tool.
- Special cases, source or target file has zero length, are handled.
- Using the wrong source file in blockdiff patch is handled gracefully. Every source block is checked by CRC32. That catches simple human errors.
Limitations (Or Keep in mind):
- Does not work on arbitrary binary files (like executables, images, audio or video files). It only works on file/disk formats that are block based like ext2,3,4 and btrfs.
- Works best with disk images up to ~10GB. The limiting factor is time to read the source and target file. Using
blockdiff
for 3 TB disk images is not feasable, because the time to read a 3TB disk images is just too long. - 'blockdiff' does not use compression internally. You should gzip the patch file yourself to reduce the transferred bytes further.
- In-place patching is not supported.
Homepage and Code
The project's website is:
https://www.stefanchrist.eu/projects/blockdiff
The source code can be found in the git repositories:
$ git clone https://github.com/lengfeld/blockdiff
$ git clone https://git.stefanchrist.eu/blockdiff.git
Documentation and Install
To generate the manpage and documentation just execute:
$ make
Then to view the manpage execute:
$ man ./blockdiff.1
To install the program in $HOME/bin
execute:
$ make install
To install the programm and the manpage system-wide in /usr
execute:
$ make prefix=/usr install install-doc
For an overview of all Makefile targets execute
$ make help
License
The source code of blockdiff
is licensed under GPLv2 and GPLv3. For the license text see files LICENSE.GPLv2.txt
and LICENSE.GPLv3.txt
or visit GNU.org Licenses.
I chose not to license the program as GPLv3 only, because the normal company lawyer freaks out when they hear the license name GPLv3. So they can calm down and only have to acknowledge the GPLv2 license text.