Defragmentation issue...

Hi folks...
I've just deleted about 20Gb of files from my HDD to clear some space to make a partition or two so I can start to dual-boot Linux. Once I'd done that I ran a defrag to get everything tidied up properly.

The process ended with a message saying that some files could not be defragmented. It then output a text file with the following in it...

The graphical view shows the fragmented files as scattered in small pieces right across the whole drive.

Is there something I can do to either defragment those files individually, or do before running another defrag to ensure they are defragmented along with all the others?


Any input on this would be appreciated.



Oh and while I remember to ask...
The graphical analysis result showed a two blocks of green "unmovable files" one is right in the middle of what (will eventually be) the "free space" at the end of the drive (I am assuming that the graphical "estimated disk usage" bars represent actual positions on the disk and not just a pretty picture for users to look at)

Is that anything to be at all concerned about (if looking for a very neat drive with fastest access times I can get from it) and if so is there something that can be done?

 

Im assuming you have rebooted and tried again. Also what defragmenter program are you using? The one that comes with windows or a differant one.

 

Yes I have. I've tried it 2 or 3 times now, in normal and in safe mode too.
I'm using the windows one.

 

Personaly I have had bad luck with the built in one. Not to mention its slow., i would look for a free one in the majorgeeks file archives. Can't remember the name of a free one but if you look you will find it.

 

Well, I will give it a shot and see how it works out.
I'll post again as soon as I have tried.

 

I wouldn't spend a lot of time on it -- I note that only three files are fragmented, and your swapfile is not one of them. But then, I'm not a performance hound.

An observation: some defraggers won't move hidden or system files. (Some ancient copy-protection schemes noted the location of a hidden file, and assumed that an illegal copy had been made if that file moved.) You might want to check the defragger's documentation to see what it does with system or hidden files. It can't defrag a file it won't move.

 

Rob - the main reason I'm defragging right now is so I can set up a Linux partition.

I don't have a great understanding of the way HDDs work, but as it stands those three files (between them taking up a bit over 26% of my drive space) are scattered right across it.
I assume that the best way to have the data spread is all on one part of the drive so the two partitions are actually two whole parts of the drive.

Even without that though, they are so spread that everything that goes onto the drive at this point is going to end up fragmented to fit around the current fragments too - I really want to avoid that if I can. I sometimes end up working with very big files so having them all in one place does improve performance a fair bit.


I just ran another defrag at boot time using DefragMentor, which has cut those three files down from thousands of fragments each, to just tens or hundreds, but they are still right across the disk.

 

I wouldn't worry to much though bigshot, your never gonna have a prefect defrraged machine unless its never in use.

 

Oh I'm not expecting perfection... Just a good start

I've been reading a bit about fragmentation on another comp while waiting for my machine to defrag and was quite surprised to see just how rapidly some files become scattered all over the drive.
I think I've defragmented about 8 times total in 18 years of computer use (though for much of that I was wondering why everything took so long - you don't get many answers when you're learning by yourself and have no network access) so was also surprised to see that some people schedule it for every week or so.

Basically I just want to get the drive into as good a condition as possible before I partition it - after that I'll just schedule the occasional defrag and be done with it.
I still can't get those three files sorted though.

 

Some linux distros (SuSE and freespire two that I have tried) ) do the partition thing for you.
Before setup it will ask you how to install (hard drive or run from cd) It will then look for enough space ,shrink that windoze partition and install your favorite flavor.
I just went to a dual boot (Freespire and win2000) and have been running FS for 4 days and have not needed to change to windows to do anything (hmmm...Might be something to this Linux stuff)

Anoobatlinux

 

Suse is an awesome distro, and i use it on all my server.

 

Ubuntu lets you set up partitions during install as well.

/Mimsy

 

In an ideal world,every file on your hard drive would be in contiguous allocation units, in the order the file would be read. A defragger attempts to do just that. Some will also attempt to put the most commonly used files close to the file allocation tables. The object of the exercise is to reduce the total distance the read heads must travel to read a file into memory. Every time the heads must seek to another sector to assemble a file and read it into memory costs another 3-8 ms. or more. A badly fragmented large file can require dozens of seeks to new sectors instead of just a few.

I take it that you will be moving and resizing the existing partition(s) to make room for the partitions that Linux will need - as distinct from repartitioning and reformatting the drive. Moving and resizing partitions will go faster (and be somewhat safer) if the partitions involved have been defragged before resizing the partition because the software managing the movement of the partition boundaries will have contiguous available empty space to take up the slack, instead of having to move data out of the way before moving a partition boundary.

Assuming that the OS can move those three files, you might try this:

• move the files to another drive or partition to get them out of the way of the defragger
• defrag the original partition
• copy the files back to the original partition

When the files are copied back to a freshly defragged partition, there would be few or no interstices between existing allocation units, with the result that the OS would place those files in empty allocation units at the end of the occupied space in the partition. If there are no occupied allocation units in that empty space, the files would be copied into place in contiguous allocation units in the correct order.

 

Petaluma and Mimsy - It isn't the forming of the partitions that is a problem, I'm pretty sure that most Distros (all those I've looked at) can make a partition for you when installing the OS.
Something to this Linux stuff? I'm convinced of it!


Colemanguy - thanks for the view on SuSe. It doesn't help all that much as I'm really torn between Ubuntu and SuSe, but knowing you've had a lot of success with one of them is encouraging.



Rob M.
At the moment I have one drive, one partition (C: ) and that's the lot. With the smaller two I considered burning them to DVD so I could delete the originals, defrag and then put the files back in one piece, but that's about as close to your method I can get (unless I can spare the time to transfer them via ethernet - which with files that size would take quite some time. I'll do it if it comes down to that, but would prefer not to need to resort to that if at all possible.

If it helps at all, this is a (probably not too clear) text representation of the "analysis" graphic of Disk Defragmentor for my C: Drive.
cccecccUUUcccccccFFcFcFFcccccccccccceeeeeeeeeeeeeFFeeFFeeeeFFeeFeFeFeeeeUUUeeeFFeFFFFFFFeeFeeFeeFFeeFFeFeFeeFFF

c=contiguous files
e=empty space
F=fragmented file
U=unmovable file

A pretty shoddy level of fragmentation, and one I think is probably less than ideal to be starting a partition and OS install with.



My intention is to move files and partition, not to reformat. To reformat I would need to spend a day setting up my system again and since I work on this machine, that isn't something I'm in any hurry to do.

Any more input before I resign myself to a long ethernet transfer and/or another couple of DVDs down the drain?

 

I note that you have something like 25GB of free space on that drive, which isn't a lot of space for the defragger to work in -- given that the three files you're concerned about total nearly 20GB. That could be why the defragger left those files alone. I'm not convinced, but it's possible.

If you have software like Partition Magic around (that will let you create, re-size, move, and delete partitions without having to knock down existing partitions), you might try shrinking the existing partition as much as you can, creating a temporary partition in the free space you've just created, then moving those three files to the temporary partition, defragging the original partition, then moving the three files back.

Once you've done that, you can delete the temporary partition or resize it to what you need for the Linux install, and expand the original partition to whatever you need.

There is a risk, but it's a low-probability thing. If something goes wrong during any one of those operations, you might lose those files or other data on the partition. I'd want backups of critical files at least before I started something like that -- but then, I'm one of the cautious ones when it comes to messing with partitions.

An observation: if you have to buy Partition Magic, you can probably buy a second hard drive for not a lot more. Presto -- there's the other partition you need!

Another observation: presumably, you'll create a separate partition (or two) for your Linux installation. The fragmentation in the original partition will have absolutely no impact on the performance of the Linux partition unless the Linux partition needs to access the original partition. If that's the case, hard drive performance will be impaired slightly because of the extra head movements necessary to read a fragmented file. But you knew that already -- you've got that problem now.

 

Ok, I've tried three different defraggers now. One of them did say something about there not being enough room to defrag them, so I think you're onto something with that point.

Unfortunately a second (internal) hard drive isn't an option for this machine as I'm working on a laptop. I can't say I like it (other than the bonus of being able to carry it around with ease and work "on the go") as the ergonomics are terrible - even with an external mouse and keyboard it isn't easy to get a comfortable working position, but for now it is all I have.
An external drive is on the cards, but I wouldn't expect a USB or FireWire drive to be anything like as fast as an internal one.

I think I'll try moving the two smaller fragmented files to another drive (It will take a while, but it should free up enough space) and then defrag again. Hopefully that will be enough to defrag the 10Gb file and then leave me with one lump of contiguous free space to move the two files back into.


One thing I could do with clearing up (I've been unable to find an answer so far) is that of exactly how partitions sit on the drive.
Is each partition entirely contiguous, or if I were to partition my drive in this state, would I end up with a new (Linux) partition that is itself fragmented?
I will be clearing up the rest of the fragmentation before I partition anyway, but it would be nice to know just how it works.

Also, (pretty much another way of asking a very similar question) are partitions physical (and contiguous) parts of a disk, or is the disk just one large storage area with some sort of change to the index to "separate" the partitions from eachother?

 

Both.

However, some partitioning software will show you what areas of the disk are allocated to which partition, i.e., the software will give you the cylinder numbers. The partitioning is done with reference to complete contiguous cylinders; it seems you can't have part of a cylinder in one partition, and part of the same cylinder in the adjoining partition.

When you create a partition, it will be nothing but free space that you then must format for whatever filesystem you want to put on it. In other words, a fresh/empty partition will be unfragmented.

When you copy a fragmented file to the fresh partition, the copy on the fresh partition will be unfragmented because the operating system will write the file to successive (and contiguous) allocation units. That was the basis for the suggestion in my previous post.

If you're not sure of what is actually referred to by a "cylinder" in terms of hard drive geometry, the pages at <http://www.rwc.uc.edu/koehler/comath/42.html> and <http://www.answers.com/topic/cylinder-head-sector> will help a little.

 

Cylinders I understand. It's like the 3D version of a Track, right?
At least, if I'm remembering my A-Level computing classes correctly they are.

Ok...
So making it very over simple, you have an index stating where the different partitions are. The index might say...
"Partition0 : Cylinder0 to Cylinder 10"
"Partition1 : Cylinder11 to Cylinder14"
and so on - but obviously they are all on the same disk.

So from that - could you get a fragmented partition?
ie...
"Partition3 : Cylinder16 to Cylinder17 and Cylinder 21 to Cylinder 25"
...and would that affect performance?

I know I'm probably massively over simplifying it - but that's something that helps me wrap my head around new things. Once I've got the very basic concepts down I can start picking up the more technical and "real world" aspects of it.

 

Yup. You could think of a cylinder as the area on all platters that can be read on a multi-head hard drive without having to move the heads. Of course, this doesn't make a lot of sense until you remember that the heads are ganged -- they all move together in a column because there is only one head actuator in the drive.

I believe that description is quite accurate -- and not oversimplified. That information is retained in the drive's partition tables.

I suppose that's theoretically possible, but I've never seen it. Partition Magic won't partition a drive that way. And yes, it would have a negative impact on data transfer rates because of the extra distance the heads would have to travel between the fragments of the partition.

That's a good way to start, in my book. Keep up the good work.