Tuesday, November 03, 2009

Message Digest algorithms

I'm doing a home-made backup product that needs to compute checksum of tens of thousands of file. According to Wikipedia, there are quite a number of checksum algorithms.

This product will first base on Linux and some of the checksum utilities are already provided by Ubuntu 9.10. They are md5sum, sha1sum, sha224sum, sha256sum, sha384sum and sha512sum.

In case one day I will port this product to Windows, I also checked MessageDigest of Java Cryptography Architecture (JCA) in Java SE6. It supports MD2, MD5, SHA-1, SHA-256, SHA-384 and SHA-512.

Doing checksum is both CPU and IO intensive. I did a simple performance test today on how much CPU time is needed to compute different checksums (commonly supported by Linux and MessageDigest) of a 723,488,768-byte file.

Alg. Name Real Time User Time System Time
md5sum 0m14.655s 0m4.380s 0m1.300s
sha1sum 0m15.187s 0m12.641s 0m1.764s
sha256sum 0m25.859s 0m23.341s 0m1.656s
sha384sum 2m25.417s 2m23.841s 0m1.432s
sha512sum 2m25.733s 2m23.373s 0m2.116s

Now I have a clear idea of work load of these algorithms. I will do a Java implementation test later to see if I can get similar results.

No comments:

Post a Comment