logo (C) Bas Schuiling

BlackHole

BlackHole is an opensource data deduplicating network block device (a NBD server).
As BlackHole works on the block-level, it is filesystem agnostic. So you can create ext3, xfs, jfs, any filesystem you like on it. Tested with ext2, ext3, ntfs and reiserfs.
NBD is short for 'network block device'. It is comparable to iSCSI and FCoE. For allmost all platforms NBD-servers are available. NBD-clients are in the Linux, HURD and Solaris kernel.
BlackHole can use MD5, SHA-1 and SHA-256/384/512 to generate block-hashes.
Since version 0.3 it also includes synchronous and a-synchronous replication (other products with this functionality are called metro mirror and global mirror).
Since version 0.8 it can create snapshots in sparse files.
Since version 1.7 BlackHole can encrypt the datastore.
Since version 1.8 BlackHole can use Kyoto Cabinet (a NoSQL database).
Since version 2.0 BlackHole supports multiple LUNs, compression and encryption.

Warning

This is a beta version: don't use it with data you don't have a backup of.

Download

binarysourcecomment
BlackHole-2.3.jarbh-2.3.tgzFixed memory leak in MySQL backend
BlackHole-2.2.jarAdded zoning and IMDisk support - enabling microsoft windows clients
BlackHole-2.1.jarLDAP authentication in telnet interface, implemented the new NBD protocol functionality (barriers, trim, force flush)
BlackHole-2.0.jarMultiple LUNs support. Note: datastore is incompatible with the one from older versions

How to use it

See manual.pdf for a short manual.

Please note: the files in the datastore directory are sparse files so even they look like large files, they only use diskspace that is really used.


This program was developed with the help from .

License

In short: it is released under GPLv2.
BlackHole is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 2.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

Links

What is data de-duplication (wikipedia)
LessFS (open source) - LessFS is on filesystem level
Opendedup (open source)
ZFS is capable of automatic data de-duplication
DataDomain
NetApp
IBM


Menu with my UNIX/Linux softwareSupport this website e-mail me at: folkert@vanheusden.com
October 2, 2011