How not to upgrade a laptop hard disk

A while ago I managed to increase the size of my laptop's hard disk. As this was fairly error-strewn (2.5 days of worry), I thought I would describe what I did and the problems I had in case it helps anyone else, and as a reference for the future in case I ever need to do something similar. I am indepted to the many other web pages that offer advice on this process. Taken together, they allowed me to overcome all difficulties, but no one of them dealt with all the difficulties I had. I am reasonably technically competent, but do not upgrade disks every day and thus am not familiar with what is involved, and there are several aspects I don't understand even after the event. Thus I probably made just about every mistake in the book, so perhaps the following will be of some use.

My system

My laptop was a Lenovo Thinkpad T61p. It had a 160GB disk which was getting tight on space. I acquired a 500GB replacement. My aim was to clone the 160GB disk and grow my most important partition to use all of the new space. I have Windows XP and Linux (OpenSuse 11.4, 64-bit) on it. I mainly use Linux, and the Linux /home partition did not have much space left. The partitions were (in order) NTFS (Win XP), NTFS (rescue partition I think), Linux swap, an extended partition containing the Linux root ('/') (ext4) and Linux /home (ext3) partitions.

My aim

As I said above, my aim was to clone the old drive, and then grow the final linux /home parition. I wanted to do this to avoid the need to re-install both Windows XP and linux (and especially the various bits of extra software that one stacks up over time). The downside of this is that it makes the installation of a new drive more involved than if one simply inserts it and starts with a fresh install. Unfortunately I no longer have my XP system disks, and even if I had, did not want to through the various service packs. However, if you are just about to start the same process, think carefully about cloning versus installation from scratch. A fresh slate may be just what you want and is likely to relatively straightforward.

The details

Now follows a blow-by-blow account including all problems. Do not simply follow what I did, otherwise you will repeat my mistakes. At the end of each section, I summarise what I would do if I had to do it all over again. You should probably have some familiarity with linux to try to follow this. Although you might be lucky, my experience suggests that there will inevitably be unexpected problems and it is when these occur that it helps to have a little background knowledge.

Cloning the old drive

Requirements: USB enclosure for the 2.5-inch drives, a linux live CD (I used knoppix), the OpenSUse 11.4 installation DVD.

WARNING: This procedure and others in this page can easily destroy data if mistakes are made, so go no further if you do not have some sort of separate backup of your old drive.

I decided to use the linux utility 'dd' to clone my old drive. This is a trusted old friend for me. I mounted the new drive in a USB drive enclosure of a colleague (a quick search shows them to be pretty cheap). This avoided having to copy to a third disk (old --> third) followed by copying to the new disk (third --> new). The cloning is slow, so doing it once is better than twice.

I attached the new drive to my laptop via USB and booted from a Knoppix live CD which has 'dd' and many other useful utilities. It does not by default mount the hard disks, which is exactly what you want when cloning with 'dd'. Knoppix showed the hard disks as icons with names like sda1, sda2 etc, and sdb. I was pretty sure that 'sda#' referred to the old internal drive and 'sdb' to the new USB-mounted one, but "pretty sure" is not good enough here since if you copy in the wrong direction, its goodbye data. The numbers 'sda1', 'sda2' etc as opposed to the lone 'sdb' backed up my suspicion, but in addition, running 'fdisk -l' from a terminal to list all partitions is a good idea to make absolutely sure which drive is which. [Incidentally, you need to run 'fdisk' and several other commands as root, which you can get to using 'su -'.]

I then entered the command:

dd if=/dev/sda bs=4k conv=sync,noerror of=/dev/sdb

This performs a byte-by-byte copy of the entire disk called /dev/sda ('if' for input file) onto /dev/sdb ('of' for output file). It is absolutely critical that you get the disk names the right way round otherwise you could end up copying junk from your new disk and overwriting good data on your old disk. Take it slowly. The 'bs=4k' means that 'dd' works in blocks of 4kbytes; the best size of this parameter is unclear. Too small can make the transfer rate slow. Too large and .. well I don't know, but presumably there is a "too large". The 'conv=sync,noerror' ensures that if read errors occur then 'dd' carries on and writes zeroes to the output to keep things in sync on the new disk. I did not know it, but although the 'dd' command I typed was perfectly correct, I had already made two mistakes that I would pay for later.

There were in fact read errors about 9GB in; when they occurred 'dd' also told me the transfer rate -- 26 MB/sec. 160GB was going to take the best part of 2 hours, so I left it alone. 'dd' runs silently otherwise; if you want more information from it then you need to identify the process ID (pid) with e.g. 'ps aux | grep dd' and then send the process the USR1 signal using 'kill -USR1 pid'.

At the end of this, I swapped the new and old hard drives (the procedure for doing do depends upon your hardware of course). Booting up, horror of horrors, I got a message like "no operating system found" (I forgot the details, but when you see the message, you will know it). I had been expecting to see a grub listing of OS-es, but no such thing. Instead, just a list of devices (CD, HDD) saying things like "no valid operating system" against them. Not good. Rather stupidly, I then swapped the drives around again, and tried the same 'dd' command, but now with '32k'. Another 2 hours passed by, I swapped the disks yet again, but with the same result: no operating system was found. At this point I ran 'fdisk -l' again -- I should have done it earlier -- and the partition tables of the two disks did not seem to be the same. They were for the first three, but not the final two. After frantic googling, I came across a page that said you should always clone from the externally USB-mounted disk to the internally mounted disk because of issues to do with the BIOS. Lenovos were mentioned as one of the brands for which this was the case. I don't claim to understand this, but ours is not to reason why ...

Oh dear. I swapped the disks again (5 screws in all, undone and done up each time) for what was already seeming like the umpteenth time with no strong hope of success. I issued the 'dd' command for a third time, but now as:

dd if=/dev/sdb bs=1k conv=sync,noerror of=/dev/sda
Note especially here the reversal of sda and sdb because sda is always the internal drive and this was now the new drive while sdb, the USB drive contained the old drive I was trying to clone. I tried a smaller blocksize in the almost-certainly-mistaken belief that I had been using perhaps too large a blocksize before. The transfer rate slowed, but once done, the openSuse boot hung. At least the partition tables matched, so something had gone right; I think this must be down to the USB to internal copy direction. Hitting ESC during the linux boot showed messages of the form

waiting for device /dev/disk/by-id/
so it seemed that the linux partitions were labelled by long strings unique to the disk (basically the disk serial number), unfortunately these strings were correct for the old disk, not the new! This seems to be a relatively new feature in linux, and OpenSUse in particular, replacing the old /dev/sda4, /dev/sda5 convention which would have had no such problem.

I was getting slightly irked at this point, but fortunately my colleague was on hand and after booting using the OpenSUse installation disk, mounting the root partition, and editing ('vi') a few relevant files, we managed to get linux to boot! Even this was not without problems because in principle we should have been able to do the same with Knoppix, except that the disk I had was an old one and unable to mount 'ext4' partitions. While this sounds obvious, it was not when blundering around as I was, battling apparently enumerable problems. Once booted into linux then we used 'yast2' to switch to labelling the partitions by name rather than ID; we also changed the name of the custom location of the boot partition.

If I had to do it all again I would

  1. First use yast to switch to partition labelling by name (/dev/sda1 etc) rather than by ID
  2. Take the old drive out of the laptop, and mount it in the USB enclosure
  3. Put the new drive into the laptop
  4. Boot from a linux live CD e.g. Knoppix
  5. Run 'fdisk -l' to confirm which disk is which
  6. Clone the old disk (now in the USB) to the new (now internal) disk using 'dd':
    dd if=/dev/sdb bs=4k conv=sync,noerror of=/dev/sda
    
    I might experiment a bit with 'bs' to optimise the transfer rate.
In principle, this could take of order 20 minutes plus the time needed to clone, but this is just for my particular setup: expect problems if you don't have the same hardware and version of linux.

Re-partioning

Requirements: some form of partioning software. I used 'gparted'; I could probably have used the Knoppix CD but its problem with recognising the ext4 filesystem put me off. Also the 'super grub' live CD turned out to be extremely useful.

Anyway, once I managed to boot up, I went to the bootloader section of yast and set the boot partition to /dev/sda (I thought I had done so before, but never mind, it appeared to be set to /dev/sda#####). In summary this stage could have been accomplised quickly with:

  1. Use of a partition editor to extend the partition as described above.
  2. 'super grub' on hand to boot up linux in case of bootloader issues.

That's it; sorry is so long-winded, but I hope it shows that you can easily be hit by the unexpected if, like me, you are only partially competent with this sort of stuff. On several occasions I thought I would give it all up and make fresh install, but luckily in the end I managed to grope my way through to the end.