ActiveBlog

Unix package checksum errors
by Kevin "kj" Woolley

Kevin "kj" Woolley, January 30, 2006

Frequently, people will write to support to let us know that one of our Unix software packages is broken and needs to be fixed. Inevitably, it's a checksum error from their Unix vendor's tar(1) program that brings about the problem....

Frequently, people will write to support to let us know that one of our Unix software packages is broken and needs to be fixed.  Inevitably, it's a checksum error from their Unix vendor's tar(1) program that brings about the problem.  Sun's Solaris tar is the worst offender, but it is also seen on AIX and HP/UX, among others.

As usual, the problem isn't what it seems.  The problem is really in how various versions of tar store long pathnames.

Historically (and I mean deep in history), Unix filenames were limited to fourteen characters or so.  These days, the limit is closer to 255 characters for most systems.  Unix has always been an operating system full of hard-coded limits on the size of data, and tarfiles are no exception -- indeed, they may well be the perfect example.

Many Unix tar implementations set limits of between 64 and 128 characters per pathname (filename plus directory names) for each stored file.  Back in the day of 14-character filenames (and therefore 14-character directory names), this probably did the job quite well.  I'm sure you can imagine, though, that these days it's not quite enough.  Even a single filename can overflow this limit.

Because there was no clear standard for how to store long pathnames in a tarfile, each vendor implemented it a little differently.  Their various implementations often had two properties in common with each other.  They were usually undocumented, and almost always closed-source, which meant that compatibility between versions was hard to come by.

The way we keep compatibility between versions is to use a different version entirely -- We use GNU tar, which can be built and run on all of our supported platforms, plus quite a few more.  As a bonus, it's the default tar for every major Linux distribution.  It's also free and open-source, so everyone can make it work on their platform if it doesn't already (which is highly unlikely).

If you don't have GNU tar and don't have access to a C compiler as required to build it, you can find pre-built binaries for your platform.  For Solaris, the place to look is sunfreeware.com.

In short -- use GNU tar, and all will be fine.

Subscribe to ActiveState Blogs by Email

Share this post:

About the Author: RSS

kj is ActiveState’s Systems Administrator. He was born in London, Ontario, grew up in North Vancouver, and spent several years in the Northwest Territories doing technical sales. As a technical infrastructure specialist, he brings over twenty years of systems administration experience to ActiveState.