defragment

Visualisation of the fragmentation and then of the defragmentation

In the context of administering computer systems, defragmentation is a process that reduces the amount of fragmentation in file systems. It does this by physically organizing the contents of the disk to store the pieces of each file close together and contiguously. It also attempts to create larger regions of free space using compaction to impede the return of fragmentation. Some defragmenters also try to keep smaller files within a single directory together, as they are often accessed in sequence.

Causes of fragmentation

It has been suggested that this article or section be merged into File system fragmentation#Cause. (Discuss)

Fragmentation occurs when the operating system cannot or will not allocate enough contiguous space to store a complete file as a unit, but instead puts parts of it in gaps between other files (usually those gaps exist because they formerly held a file that the operating system has subsequently deleted or because the operating system allocated excess space for the file in the first place). Larger files and greater numbers of files also contribute to fragmentation and consequent performance loss. Defragmentation attempts to alleviate these problems.

Example

Consider the following scenario, as shown by the image on the right:

An otherwise blank disk has 5 files, A, B, C, D and E each using 10 blocks of space (for this section, a block is an allocation unit of that system, it could be 1K, 100K or 1 megabyte and is not any specific size). On a blank disk, all of these files will be allocated one after the other. (Example (1) on the image.) If file B is deleted, there are two options, leave the space for B empty and use it again later, or move all the files after B so that the empty space is at the end. This could be time consuming if there were hundreds or thousands of files which needed to be moved, so in general the empty space is simply left there, marked in a table as available for later use, then used again as needed.^[1] (Example (2) on the image.) Now, if a new file, F, is allocated 7 blocks of space, it can be placed into the first 7 blocks of the space formerly holding the file B and the 3 blocks following it will remain available. (Example (3) on the image.) If another new file, G is added, and needs only three blocks, it could then occupy the space after F and before C. (Example (4) on the image). Now, if subsequently F needs to be expanded, since the space immediately following it is no longer available, there are two options: (1) add a new block somewhere else and indicate that F has a second extent, or (2) move the file F to someplace else where it can be created as one contiguous file of the new, larger size. The latter operation may not be possible as the file may be larger than any one contiguous space available, or the file conceivably could be so large the operation would take an undesirably long period of time, thus the usual practice is simply to create an extent somewhere else and chain the new extent onto the old one. (Example (5) on the image.) Repeat this practice hundreds or thousands of times and eventually the file system has many free segments in many places and many files may be spread over many extents. If, as a result of free space fragmentation, a newly created file (or a file which has been extended) has to be placed in a large number of extents, access time for that file (or for all files) may become excessively long.

The process of creating new files, and of deleting and expanding existing files, may sometimes be colloquially referred to as churn, and can occur at both the level of the general root file system and in subdirectories. Fragmentation not only occurs at the level of individual files, but also when different files in a directory (and maybe its subdirectories), that are often read in a sequence, start to "drift apart" as a result of "churn".

A defragmentation program must move files around within the free space available to undo fragmentation. This is a memory intensive operation and cannot be performed on a file system with no free space. The reorganization involved in defragmentation does not change logical location of the files (defined as their location within the directory structure).

Common countermeasures

Partitioning

A common strategy to optimize defragmentation and to reduce the impact of fragmentation is to partition the hard disk(s) in a way that separates partitions of the file system that experience many more reads than writes from the more volatile zones where files are created and deleted frequently. The directories that contain the users' profiles are modified constantly (especially with the Temp directory and web browser cache creating thousands of files that are deleted in a few days). If files from user profiles are held on a dedicated partition (as is commonly done on UNIX systems), the defragmenter runs better since it does not need to deal with all the static files from other directories. For partitions with relatively little write activity, defragmentation performance greatly improves after the first defragmentation, since the defragmenter will need to defrag only a small number of new files in the future.

Problems

Immovable files

The presence of immovable system files, especially a swap file, can impede defragmentation. These files can be safely moved when the operating system is not in use. For example, ntfsresize moves these files to resize an NTFS partition, and the tool PageDefrag can defragment Windows system files such as the swap file and the files that store the Windows registry by running at boot time before the GUI is loaded. If the NTFS Master File Table must grow after the partition was formatted, fragmentation of the MFT can occur, and it cannot be safely defragmented while the partition is in use. Using the data from the Windows defragmenter's analyze report we can edit the registry to adjust the NtfsMftZoneReservation value under HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\FileSystem. But first you need to back up and reformat the volume in order to create a contiguous MFT using the higher zone reservation space, and then perform a full volume restore.^[2][3] Some defragmentation programs, such as Diskeeper or PerfectDisk (at boot time), are capable of defragmenting the MFT.

User and performance issues

In a wide range of modern multi-user operating systems, an ordinary user cannot defragment the system disks since superuser (or "Administrator") access is required to move system files. Additionally, file systems such as NTFS (and most Unix/Linux filesystems) are designed to decrease the likelihood of fragmentation.^[4][5] Improvements in modern hard drives such as RAM cache, faster platter rotation speed, and greater data density reduce the negative impact of fragmentation on system performance to some degree, though increases in commonly used data quantities offset those benefits. However, modern systems profit enormously from the huge disk capacities currently available, since partially filled disks fragment much less than full disks.^[6]

The use of disk defragmentation is not recommended on flash memory or solid-state drives (SSD), as it may reduce the lifespan of the drive.^[7] As flash memory does not rely on physical movement to read the drive, random file access is much faster than a mechanical hard drive, so defragmentation is not necessary.

Defragmenting and optimising

Besides defragmenting program files, the defragmenting tool can also reduce the time it takes to load programs and open files. For example, the Windows 9x defragmenter included the Intel Application Launch Accelerator which optimised programs on the disk.^[8] The outer tracks of a hard disk have a higher transfer rate than the inner tracks, therefore placing files on the outer tracks increases performance.^[9]

Approach and defragmenters by file system type

• FAT: MS-DOS 6.x and Windows 9x-systems come with a defragmentation utility called Defrag. The DOS version is a limited version of Norton SpeedDisk.^[10] The version that came with Windows 9x was licensed from Symantec Corporation, and the version that came with Windows 2000 and above (with the exceptions of Vista, Server 2008, and 7) is licensed from Diskeeper Corporation.
• NTFS: Windows 2000 and newer include a defragmentation tool based on Diskeeper. Windows NT 4.0 includes an application programming interface that third-party tools can use to perform defragmentation tasks, but no command-line or graphical tools are included that make use of it. Windows NT 3.51 and below do not include any defragmentation capabilities. A number of free and commercial third-party defragmentation products are available for Microsoft Windows.
• ext2 (Linux) uses an offline defragmenter called e2defrag, which does not work with its successor ext3. Instead, a filesystem-independent defragmenter like Shake^[11] may be used.
• ext4 (Linux) is rumored^{[citation needed]} to support online defragmenting, but as of the current kernel (v2.6.30), such a utility does not exist.
• vxfs has fsadm utility meant to perform also defrag operations.
• JFS has a defragfs^[12] utility on IBM operating systems.
• HFS Plus (Mac OS X) In 1998 it introduced a number of optimizations to the allocation algorithms in an attempt to defragment files while they are being accessed without a separate defragmenter. If the filesystem does become fragmented, the only way to defragment it is to purchase a utility such as Coriolis System's iDefrag; or to wipe the hard drive completely and install the system from scratch.
• WAFL in NetApp's ONTAP 7.2 operating system has a command called reallocate that is designed to defragment large files.
• XFS Provides an online defragmentation utility called xfs_fsr.
• SFS Processes the defragmentation feature in almost completely stateless way (apart from the location it is working on), so defragmentation can be stopped and started instantly.

See also

• Fragmentation
• File system fragmentation
• List of defragmentation software

References

Sources

• Norton, Peter (1994) Peter Norton's Complete Guide to DOS 6.22, page 521 - Sams (ISBN 067230614X)
• Woody Leonhard, Justin Leonhard (2005) Windows XP Timesaving Techniques For Dummies, Second Edition page 456 - For Dummies (ISBN 0-764578-839).
• Jensen, Craig (1994). Fragmentation: The Condition, the Cause, the Cure. Executive Software International (ISBN 0-9640049-0-9).
• Dave Kleiman, Laura Hunter, Mahesh Satyanarayana, Kimon Andreou, Nancy G Altholz, Lawrence Abrams, Darren Windham, Tony Bradley and Brian Barber (2006) Winternals: Defragmentation, Recovery, and Administration Field Guide - Syngress (ISBN 1-597490-792)
• Robb, Drew (2003) Server Disk Management in a Windows Environment Chapter 7 - AUERBACH (ISBN 0849324327)

External links

• Defragmentation on the Open Directory Project.
• Microsoft Windows XP defragmentation - How to schedule a weekly defragmentation
• Microsoft Windows 2000 Professional and Server defragmentation - How to schedule defragmentation

.