ZFS Deduplication

People who don't want to wait for Solaris Express Community Edition b129 or OpenSolaris dev repository to be updated to b129 (b128 is not going to be released) can easily try ZFS deduplication now.

Blindingly Fast Upgrade (BFU) archives for build 128 are available here now. These archives are used mainly by the developers but we can use them easily for testing as well. Please note that your OpenSolaris installation is not going to be consistent after BFU'ing from the pkg(1)'s point of view. It is likely that pkg image-update will not be able to upgrade such system in the future. Be prepared to re-install your system. I usually try new archives in unimportant VirtualBox/VMware VMs. Here are the commands I have used to upgrade my test VM running OpenSolaris b127 to b128 (as root):


wget http://dlc.sun.com/osol/on/downloads/b128/SUNWonbld.i386.tar.bz2
wget http://dlc.sun.com/osol/on/downloads/b128/on-bfu-nightly-osol-nd.i386.tar.bz2

gtar jxf SUNWonbld.i386.tar.bz2
pkgadd -d onbld SUNWonbld
gtar jxf on-bfu-nightly-osol-nd.i386.tar.bz2

export FASTFS=/opt/onbld/bin/i386/fastfs
export BFULD=/opt/onbld/bin/i386/bfuld
export GZIPBIN=/usr/bin/gzip
export PATH=/opt/onbld/bin:/opt/onbld/bin/i386:$PATH

bfu `pwd`/archives-nightly-osol-nd/i386

/opt/onbld/bin/acr


Reboot the machine. Newly booted system will have deduplication support. To try ZFS deduplication you can either create new pool (that's what I did) or upgrade 'rpool' zpool's format:

zpool upgrade rpool

Let's enable deduplication:

zfs set dedup=on tank

And run some tests:

# for i in 0 1 2 3 4 5 6 7 8 9; do zfs create tank/fs$i; done
# for i in 0 1 2 3 4 5 6 7 8 9; do gtar jxf /root/on-bfu-nightly-osol-nd.i386.tar.bz2 -C /tank/fs$i; done

# zfs list -r tank
NAME USED AVAIL REFER MOUNTPOINT
tank 3.90G 77.8G 33K /tank
tank/fs0 399M 77.8G 399M /tank/fs0
tank/fs1 399M 77.8G 399M /tank/fs1
tank/fs2 399M 77.8G 399M /tank/fs2
tank/fs3 399M 77.8G 399M /tank/fs3
tank/fs4 399M 77.8G 399M /tank/fs4
tank/fs5 399M 77.8G 399M /tank/fs5
tank/fs6 399M 77.8G 399M /tank/fs6
tank/fs7 399M 77.8G 399M /tank/fs7
tank/fs8 399M 77.8G 399M /tank/fs8
tank/fs9 399M 77.8G 399M /tank/fs9

# zpool list tank
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
tank 79.5G 404M 79.1G 0% 10.00x ONLINE -


Tarball extracted ten times and only 404MB of disk space allocated. 10x deduplication ratio. Nice. This gets even more interesting after enabling compression:


# zfs get dedup,compression tank
NAME PROPERTY VALUE SOURCE
tank dedup on local
tank compression on local

# zfs list -r tank
NAME USED AVAIL REFER MOUNTPOINT
tank 2.28G 78.0G 33K /tank
tank/fs0 234M 78.0G 234M /tank/fs0
tank/fs1 234M 78.0G 234M /tank/fs1
tank/fs2 234M 78.0G 234M /tank/fs2
tank/fs3 234M 78.0G 234M /tank/fs3
tank/fs4 234M 78.0G 234M /tank/fs4
tank/fs5 234M 78.0G 234M /tank/fs5
tank/fs6 234M 78.0G 234M /tank/fs6
tank/fs7 234M 78.0G 234M /tank/fs7
tank/fs8 234M 78.0G 234M /tank/fs8
tank/fs9 234M 78.0G 234M /tank/fs9

# zpool list tank
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
tank 79.5G 239M 79.3G 0% 10.00x ONLINE -

239MB allocated. Excellent!

pxegrub 'No Ethernet Card Found' error

Some BIOSes are able to load pxegrub but then report 'No Ethernet Card Found' error. I was getting this error on my test machine (ASUS M3N78-VM motherboard with AMIBIOS). I have found a great explanation and solution here:

http://opensolaris.org/jive/thread.jspa?messageID=377026

You can find pxegrub I have compiled from OpenSolaris sources some time ago here. I have commented out gateA20_set() call in netboot/undi.c file before compiling.

My take on Google server tray

For some time I wanted to have a dedicated test machine at home. I had only few requirements - I wanted a 64-bit CPU, 8GB RAM and network adapter that is using drivers that are present in OpenSolaris and VMware ESXi.

After looking at various dmesg outputs online I have selected ASUS M3N78-VM motherboard. The motherboard is using forcedeth driver on Linux for the network adapter. I knew that OpenSolaris had nge driver and VMware ESXi 3.5 had forcedeth driver for nForce Ethernet adapters so I thought that it was a safe choice. If not supported out of the box the drivers would require very little modification to get them working. The motherboard has an integrated video adapter which is a bonus.

For the CPU I have selected AMD Athlon X2 7850 Black Edition processor. It is obviously a 64-bit CPU and has virtualization extensions. It also comes with a heatsink in
the retail box.

I used Crucial to get 8GB of RAM and Amazon to get a power supply.

I really liked Google's approach to their servers and thought that I could do something similar. I went to a local Ryman stationery store and purchased two transparent document trays. Motherboard fits nicely into one of them. I am using the second one for the HDDs.

Here is the final result:



I had an old drive I was going to use with this machine and I was looking to use PXE boot/USB sticks for OS installations so optical drive was not required. All the parts came down to roughly to £230 with shipping:

AMD Athlon X2 7850 Black Edition: £55
ASUS M3N78-VM: £55
WinPower 600W Power Supply: £19
8GB RAM from Crucial: £80
Transparent document trays from Ryman: £6

Not a lot as for 8GB machine in my opinion :)

IPS repository mirror script

ips-mirror.py is a succesor to my IPS repository mirror script I have put together last year (indiana-discuss thread). I wrote the script in Python this time. It can mirror whole repositories or just parts of them now. It is possible to limit the architecture of the mirrored files now as well. Few interesting use cases:

You can mirror the whole OpenSolaris dev repository if you like. I don't recommend it as there are over 30GB of files to download :). It is doable though:

mkdir $HOME/dev; cd $HOME/dev
ips-mirror.py http://pkg.opensolaris.org/dev/

After the script has done its job, which can take few days in this case, you want to run pkg.depotd with the --rebuild parameter.

/usr/lib/pkg.depotd -d `pwd` -p 10000 --rebuild

pkg.depotd will generate a catalog from the manifests in the pkg directory and start serving the packages. It will also generate search indexes which can take a while.

Mirroring the OpenSolaris 2009.06 release packages for i386:

ips-mirror.py -r 5.11-0.111 -s -a i386 http://pkg.opensolaris.org/release/

Mirroring the latest OpenSolaris packages for i386 from the dev repository:

ips-mirror.py -l -a i386 http://pkg.opensolaris.org/dev/

It is also possible to mirror repositories over https with connection specific certificate and the key:

ips-mirror.py -c /var/pkg/ssl/OpenSolaris_extras.certificate.pem -k /var/pkg/ssl/OpenSolaris_extras.key.pem https://pkg.sun.com/opensolaris/extra/