Optimizing disks is a waste of time. There, I said it. The horse is out of the bag, the cat is out of the barn. So why do so many people believe that an optimizer is an essential part of any Mac user's tool kit? And what does it mean to optimize a disk, anyway?

Background Fragments

When you save a file to disk, the file system looks for an empty space to write the data. If there isn't a single space large enough, it divides the file among several smaller spaces. When a file is stored in more than one piece, we say it's fragmented. Each piece is called a fragment, or an extent.

A file may be broken into two fragments, or 20 fragments, or 200 fragments. The file system doesn't care; it can handle any number of fragments equally well. However, reading a fragmented file takes longer than reading an unfragmented one. The more fragments in the file, the longer it takes to read. That's because the hard disk's head must move to each fragment and read each one separately. Reading a single chunk of data sequentially is fast, even when the chunk is rather large. But moving the head from track to track for each fragment is comparatively slow. (And I mean "comparatively" - we're talking about additional milliseconds here.)

The solution to this slowdown? Defragmenting or optimizing. Some programs claim to defragment a disk, others claim to optimize it, and a few offer both functions. What's the difference?

Defragmenting combines files that are broken up across multiple fragments into a single fragment. But defragmenting files is only part of the problem, since the free space on a disk is often split into many pieces, a little here, and a little there. In effect, the free space is fragmented. You may have 5 GB of free space, but it could be in 5,000 chunks of 1 MB each. The next file saved may be fragmented, simply because there isn't enough unfragmented free space. That's where optimizing comes in - it defragments all the fragmented files and the free space.

Some optimizers also position similar files, such as all the operating system files, physically next to one another. The claim is that this speeds up the computer even more, because operating system files are likely to be accessed together, which prevents the disk head from needing to move long distances to read the next file. Although the concept sounds good at first blush, I'm dubious that this technique creates any perceptible speed increase. Beyond a few simple cases, it's very difficult to divine in advance which file the computer will want next.

So optimizing the disk should make your Mac run faster, right? Well, maybe. If a file you use all the time is fragmented, such as a key part of the operating system, then defragmenting that file could really help. But the operating system is usually written to the disk right after it has been freshly formatted. The disk is empty, so the operating system is rarely fragmented. If a file you rarely use is fragmented, such as that QuickTime movie from Aunt Ethel's birthday party, it doesn't matter as long as you can access the file - play the movie, in this case - normally. In short, avoiding fragmentation is helpful only on files that are accessed constantly.

So where did this cult of disk optimization come from? Back in the early days of Windows, and DOS before that, PCs used the FAT (File Allocation Table) file system. Legend has it that the FAT file system was pretty bad about fragmenting files, so disks quickly became badly fragmented. Back then, disks - and computers in general - were extremely slow, especially by today's standards. With those painfully slow disks and computers, optimizing a disk could provide noticeable performance improvements. Modern computers and disks are of course much faster, and they also have much larger and more sophisticated disk caches, all of which significantly reduces the impact of a fragmented disk.

When Apple designed the HFS (Hierarchical File System) file system for the Mac, and later when they replaced HFS with HFS+, they took special care to try to minimize fragmentation. All hard disks store data in 512 byte chunks called sectors. FAT, HFS, and HFS+ use larger chunks, called clusters on FAT and allocation blocks on HFS. One purpose of clusters and allocation blocks is to try to reduce fragmentation, by storing files in larger pieces. But HFS goes one step further. When saving a file to disk, the Mac file system allocates space in even larger chunks, called clumps, in a further effort to reduce fragmentation.

When the Mac file system saves a file, it looks for a free space large enough to hold the entire file. If there aren't any, it finds the largest free space available, then the next largest, and so on, in an effort to reduce fragmentation as much as possible. HFS will never fragment a file if it can be avoided.

Real World Fragmentation

There are two things that lead to disk fragmentation for most people: full disks and email.

Overall, the Mac's HFS+ file system does a good job of keeping fragmentation to a minimum, assuming a reasonable amount of free space remains on the disk to use when laying down files in contiguous chunks. How much free space should you maintain? There is no set answer, but leaving 20 to 25 percent of a disk free is a good rule of thumb.

If your 60 GB hard disk has only 5 GB free, that doesn't mean that you have a single empty space on the disk where the entire 5 GB is available. Rather, there are dozens, if not hundreds, of smaller free areas. The largest single chunk of free space may be only 500 MB. When a disk is very full, not only is there less total free space, but the size of the largest free area becomes much smaller. Thus the likelihood of fragmentation goes way up.

What kind of files tend to be fragmented? The most likely candidates are files that grow regularly, with little bits of data added to them over time. Each time the file system extends the file, it looks for another piece of free space, and the file fragments a little more. Various types of files fit this profile, but the prime candidate is email.

My email program's In box file has been fragmented into more than 100 pieces. Does this matter? No, it still functions perfectly. Doesn't it slow down my email program? Certainly, but not enough for me to notice. The main reason people optimize their disk is to make their Mac run faster. Doubtless it does make using the Mac somewhat faster, but I've rarely seen a perceptible speed increase in real world usage.

Pros and Cons

So increasing the speed of your Mac, even if the improvement is nearly imperceptible, is one reason to optimize your hard disk. There is a second reason to consider defragmenting files. If you suffer a disk crash, disk recovery software has a harder time recovering badly fragmented files than unfragmented files, simply because there are more pieces to track down. And which files are most likely to be fragmented? Email files, which are also the most likely to have changed recently, and thus the least likely to be in your last backup. (Obligatory reminder - if you don't have a recent backup, make one right after you finish reading this article. Really.)

There are also some good reasons not to optimize, and ironically, one of them is speed. Optimizers are slow. It takes many hours to optimize a disk. Does it make sense to tie up your Mac for hours just to make it respond a second faster when you're opening a mailbox?

More worrying is the fact that if the optimizer crashes, the disk could be, to use the technical term, "horked." That's because an optimizer must move nearly every piece of data on the disk. The best optimizers use algorithms that make it nearly impossible to lose data, even if the power goes out in the middle of a long optimization, but there's always a slim chance of something bad happening when you let a program move everything on your disk.

The problem is that no program is perfect. Earlier versions of some optimizers have had bugs that resulted in lost data or damaged disks. I don't know of any currently shipping optimizers with these types of catastrophic bugs. But that's not to say that some future version may not contain a bug, or that a current version won't have trouble when combined with a new version of the Mac OS. Be careful when you're using optimization software!

Optimization Advice

If you're going to optimize your disk, be sure to check the disk first with a program like Apple's Disk First Aid or Disk Utility. A damaged disk could cause even the best optimizer to crash when it runs across corrupted data or data in a completely unexpected place.

It's also a good idea to back up your entire disk (or at least your most important data) first. But once you have a backup, you could just erase the disk, and restore from your backup. Doing this optimizes the disk as effectively as running any optimizer. Plus, reformatting a hard disk ensures you have clean directory structures, and if you reformat it with the option of writing zeroes to every sector (which takes a long time and isn't worthwhile unless you've been experiencing odd disk problems), you'll also make the drive map out any bad blocks it may have developed. That's why I say my favorite optimizer is Retrospect - with it you can both protect your data and optimize your disk.

<http://www.dantz.com/ products/ mac_ express/>

Speed Disk, the optimizer in Symantec's Norton Utilities, has some useful features for analyzing a disk. It rates the overall disk fragmentation as light, moderate, or severe. It's almost certainly not worth optimizing a disk unless the fragmentation is severe, and often not even then. That's because Speed Disk considers a disk severely fragmented based on a combination of how many files are fragmented, how fragmented they are, and how fragmented the b-trees (disk directory structures) are. The last item is what can make it seem alarmist, because the b-trees act as triggers: if they're fragmented a certain amount, Speed Disk can automatically assign the whole disk a severe rating, even if the other files on the disk wouldn't otherwise generate that rating.

<http://www.symantec.com/ nu/ nu_ mac/>

Speed Disk shows a graph of the files and free space on the disk, letting you see how badly the free space is fragmented. It also lists the size of the largest free block, a useful piece of information to keep in mind because any file larger than that will, by necessity, be fragmented when it is saved. If you routinely work with files larger than your largest free block, optimizing the disk would be advisable.

Lastly, Speed Disk lists all fragmented files and the number of fragments per file, and it lets you defragment individual files. Why would you want to do this? HFS+ can track up to eight fragments in a file's catalog record. If a file has more than eight fragments, HFS+ creates additional records, called extent records, to track the extra fragments. Since files with more than eight fragments require accessing these additional records each time they are opened, a file with more than eight fragments is certainly a reasonable candidate to be defragmented, assuming of course that you access it frequently enough for defragmenting to make a real difference.

There's usually no need for Speed Disk's capability to defragment individual files. That's because you can usually defragment a file yourself, by simply duplicating it in the Finder. When the file system creates the duplicate file, it automatically uses only a single fragment for the file, assuming there is enough contiguous free space on the disk. Then you can delete the original and rename the copy with the original file's name.

Alsoft's DiskWarrior 3 offers the unique feature of showing a graph of a disk's directory, using a color gradient to show items that are out of order. DiskWarrior's "rebuild" function is usually used to repair damaged disks, but when used on healthy disks, it "optimizes" their directories. Although Alsoft calls this feature "optimization," it's quite different from what all other disk optimizers do. Other disk optimizers defragment the files on a disk. DiskWarrior puts the disk's catalog in order.

<http://www.alsoft.com/ DiskWarrior/>

The catalog is composed of nodes, which contain records that correspond to files. The nodes form a tree structure, with all the nodes linked together in a specific order. The file system tends to keep the nodes in order. But as files are added to and deleted from the disk, nodes are likewise created, deleted, and shuffled around, and they can end up out of order. This is not dangerous, or even wrong, just not optimal.

DiskWarrior reorders the nodes. In theory, this should make a disk faster for the same reason defragmenting a file makes it faster, namely that related information is stored together, so the disk's head doesn't have to seek to distant sectors when retrieving it. In the real world, I doubt the speed increase is noticeable, especially since the file system caches key pieces of the catalog in memory, making access much faster than when the information is stored only on the hard disk. Disk Warrior is excellent at recovering disks with damaged directories, but optimizing a properly functioning catalog is gratuitous.

Bottom Line

To sum up then, for most people, most of the time, there's simply not enough to gain by optimizing your disk to bother doing it. There's nothing wrong with optimizing a disk, and for a severely fragmented disk that is responding slowly when reading regularly accessed files, it may even be worthwhile. But in general, it's not necessary and carries a small risk. If you really want to optimize your disk, the best approach is to make a backup (with a second backup for safety's sake), reformat your hard disk, and restore from the backup.

[David Shayer was a senior engineer on Norton Utilities for Macintosh 3.0, 4.0, and 5.0. Before that he worked on Public Utilities, a disk repair program that won the MacUser Editor's Choice Award, and on Sedit, a low level disk editor.]