How to Configure External USB Hard Disks As Mirrored Storage Under Linux

This article discusses how to use external USB hard disks as mirrored storage. The steps outlined here should apply to any USB hard disk. I have had a positive experience with the 320GB and 500GB Western Digital MyBooks. They have a nice physical appearance, run quite, and auto power down when not in use. There have been no reliability issues in my experience.

A note regarding software RAID. Its use will always be slower than hardware RAID. But fast is in the eye of the beholder. If your application domain doesn't require extreme speed, than software RAID will probably be fast enough. In my day-to-day use, the speed of the file system has not perculated up to my conscious level. It's more of an academic subject with me.

I use software RAID for internal SATA drives and external USB drives (Western Digital MyBooks). The external drives are used for backups (add link here to my snapshot based backup system).

Why Mirror Backup Drives?
I use a snapshot (link here) based backup plan, where every night a backup of my internal mirrored drives is made. Fourteen of these daily backups are maintained at any given time, providing me access to the file system as it looked on any given day over the last two weeks.

Because of the snapshot nature of my backup system, it would be ashame if I lost all of this history because of a failure in my backup drive. RAID-1 comes to the rescue. By mirroring my backup drives, I can lose one of the drives without suffering any lose in my file system history.

With the ever decreasing cost of hard drives, using two hard drives for backup is no longer a concern. As I write this in early 2008, two 500 GB Western Digital MyBooks can be had for a bit shy of $250.

Toolkit
This is what you'll need to accomplish what I'm showing here:

• Linux Server (I run Slackware). You'll need a distribution that supports RAID in the Kernel. Most likely any Linux distribution released in the last few years will have everything you need.
  
• RAID Tools: mdadm (you can add a layer of complexity by adding a Volume Manager into the mix, but that's beyond the scope of this article).
  
• Some hardware discovery tools: lsusb, lsscsi, fdisk
  
• Formatting Tools: cfdisk (any disk formatter will do)
  

Just The Facts
In this example, I've connected two 320 GB USB Western Digital MyBooks to a Dell 400SC server, running Slackware 12 with the 2.6.21.5-smp kernel.

1. Connect the two drives to available USB ports and turn them on if they are not already on
2. Discover the device assignment. lsscsi will show the device assignment for each of the USB drives. Remember, USB devices are managed by Linux as SCSI devices.
3. Using your favorite format utility, create a single partition for each drive. I like cfdisk E.g., cfdisk /dev/sdc
   
   • The partition size for each USB drive should be the same size, or nearly so
   • Set the partition type to "Linux RAID Autodetect"
4. Create the mirror out of the two drives: mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
   
   • This command will create a new pseudo device (md0) that will consist of the two partitions. Note that the device name that you pick must begin with md followed by one or more digits. The device will operate as RAID-1 (mirror)
   • The mirror will be built over a time period of hours. For example, it took about 5 hours for two 320 GB drives to complete the sync process. Note that the /dev/md0 drive can be used during this period
   • You can check the status of the RAID build process with:
   • mdadm --detail -v /dev/md0
5. Create a file system on the new device. I like ext3, configured for journaling. But any will work.
   • mkfs.ext3 -j -LWD320 -v -m1 /dev/md0
   • Again, this can be done while the mirror sync is in progress
6. Once the file system has been created, go ahead and mount it:
• mount /dev/md0 /mnt/backups
7. You'll probably want to add the mount to your /etc/fstab file, but resist the temptation. It won't work when you reboot the system. The problem is that USB drives get assigned SCSI devices based on a first come first serve basis, i.e., whichever drive gets detected first will be assigned first. Also, which USB port the drive gets plugged into will affect the SCSI device assignment. Because of this, when Linux boots it will most likely fail to assemble the RAID device because it's looking for the original drives (/dev/sdc) that were used to create the device. So, you'll need to manually assemble the RAID device and than mount it. In one of your startup scripts (I use /etc/rc.d/rc.local) put these two lines:
   
   • mdadm --assemble --uuid=9ca2a418:d3eb6dfe:e14f8c51:39f1a438 /dev/md/0
   • Note that you'll use the UUID value reported by the mdadm --detail -v /dev/md0
     command
     
   • mount /dev/md/0 /mnt/backups

What's Going On Here