## 12.4 RAID ### 12.4.1 Software RAID #### 12.4.1.1 Concatenated Disk Driver (CCD) Configuration ***Original work by Christopher Shumway. ******Revised by Jim Brown. *** When choosing a mass storage solution the most important factors to consider are speed, reliability, and cost. It is rare to have all three in balance; normally a fast, reliable mass storage device is expensive, and to cut back on cost either speed or reliability must be sacrificed. In designing the system described below, cost was chosen as the most important factor, followed by speed, then reliability. Data transfer speed for this system is ultimately constrained by the network. And while reliability is very important, the CCD drive described below serves online data that is already fully backed up on CD-R's and can easily be replaced. Defining your own requirements is the first step in choosing a mass storage solution. If your requirements prefer speed or reliability over cost, your solution will differ from the system described in this section. ##### 12.4.1.1.1 Installing the Hardware In addition to the IDE system disk, three Western Digital 30GB, 5400 RPM IDE disks form the core of the CCD disk described below providing approximately 90GB of online storage. Ideally, each IDE disk would have its own IDE controller and cable, but to minimize cost, additional IDE controllers were not used. Instead the disks were configured with jumpers so that each IDE controller has one master, and one slave. Upon reboot, the system BIOS was configured to automatically detect the disks attached. More importantly, DragonFly detected them on reboot: ad0: 19574MB <WDC WD205BA> [39770/16/63] at ata0-master UDMA33 ad1: 29333MB <WDC WD307AA> [59598/16/63] at ata0-slave UDMA33 ad2: 29333MB <WDC WD307AA> [59598/16/63] at ata1-master UDMA33 ad3: 29333MB <WDC WD307AA> [59598/16/63] at ata1-slave UDMA33 **Note:** If DragonFly does not detect all the disks, ensure that you have jumpered them correctly. Most IDE drives also have a ***Cable Select*** jumper. This is ***not*** the jumper for the master/slave relationship. Consult the drive documentation for help in identifying the correct jumper. Next, consider how to attach them as part of the file system. You should research both [vinum(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#vinum§ion8) ([vinum-vinum.html Chapter 13]) and [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4). In this particular configuration, [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4) was chosen. ##### 12.4.1.1.2 Setting Up the CCD The driver [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4) allows you to take several identical disks and concatenate them into one logical file system. In order to use [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4), you need a kernel with [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4) support built in. Add this line to your kernel configuration file, rebuild, and reinstall the kernel: pseudo-device ccd 4 The [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4) support can also be loaded as a kernel loadable module. To set up [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4), you must first use [disklabel(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=disklabel§ion=8) to label the disks: disklabel -r -w ad1 auto disklabel -r -w ad2 auto disklabel -r -w ad3 auto This creates a disklabel for `ad1c`, `ad2c` and `ad3c` that spans the entire disk. The next step is to change the disk label type. You can use [disklabel(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#disklabel§ion8) to edit the disks: disklabel -e ad1 disklabel -e ad2 disklabel -e ad3 This opens up the current disk label on each disk with the editor specified by the `EDITOR` environment variable, typically [vi(1)](http://leaf.dragonflybsd.org/cgi/web-man?command#vi§ion1). An unmodified disk label will look something like this: 16 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60074784 0 unused 0 0 0 # (Cyl. 0 - 59597) Add a new `e` partition for [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4) to use. This can usually be copied from the `c` partition, but the `fstype` ***must*** be 4.2BSD. The disk label should now look something like this: 16 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60074784 0 unused 0 0 0 # (Cyl. 0 - 59597) e: 60074784 0 4.2BSD 0 0 0 # (Cyl. 0 - 59597) ##### 12.4.1.1.3 Building the File System The device node for `ccd0c` may not exist yet, so to create it, perform the following commands: cd /dev sh MAKEDEV ccd0 Now that you have all of the disks labeled, you must build the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4). To do that, use [ccdconfig(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccdconfig§ion=8), with options similar to the following: ccdconfig ccd0./imagelib/callouts/1.png 32./imagelib/callouts/2.png 0./imagelib/callouts/3.png /dev/ad1e./imagelib/callouts/4.png /dev/ad2e /dev/ad3e The use and meaning of each option is shown below: [ ./imagelib/callouts/1.png](raid.html#CO-CCD-DEV):: The first argument is the device to configure, in this case, `/dev/ccd0c`. The `/dev/` portion is optional.[ ./imagelib/callouts/2.png](raid.html#CO-CCD-INTERLEAVE):: The interleave for the file system. The interleave defines the size of a stripe in disk blocks, each normally 512 bytes. So, an interleave of 32 would be 16,384 bytes.[ ./imagelib/callouts/3.png](raid.html#CO-CCD-FLAGS):: Flags for [ccdconfig(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccdconfig§ion8). If you want to enable drive mirroring, you can specify a flag here. This configuration does not provide mirroring for [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4), so it is set at 0 (zero).[ ./imagelib/callouts/4.png](raid.html#CO-CCD-DEVS):: The final arguments to [ccdconfig(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccdconfig§ion=8) are the devices to place into the array. Use the complete pathname for each device. After running [ccdconfig(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccdconfig§ion8) the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4) is configured. A file system can be installed. Refer to [newfs(8)](http://leaf.dragonflybsd.org/cgi/web-man?command=newfs§ion=8) for options, or simply run: newfs /dev/ccd0c ##### 12.4.1.1.4 Making it All Automatic Generally, you will want to mount the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4) upon each reboot. To do this, you must configure it first. Write out your current configuration to `/etc/ccd.conf` using the following command: ccdconfig -g > /etc/ccd.conf During reboot, the script `/etc/rc` runs `ccdconfig -C` if `/etc/ccd.conf` exists. This automatically configures the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4) so it can be mounted. **Note:** If you are booting into single user mode, before you can [mount(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#mount§ion8) the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4), you need to issue the following command to configure the array: ccdconfig -C To automatically mount the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command#ccd§ion4), place an entry for the [ccd(4)](http://leaf.dragonflybsd.org/cgi/web-man?command=ccd§ion=4) in `/etc/fstab` so it will be mounted at boot time: /dev/ccd0c /media ufs rw 2 2 #### 12.4.1.2 The Vinum Volume Manager The Vinum Volume Manager is a block device driver which implements virtual disk drives. It isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. [vinum(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#vinum§ion8) implements the RAID-0, RAID-1 and RAID-5 models, both individually and in combination. See [vinum-vinum.html Chapter 13] for more information about [vinum(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#vinum§ion8). ### 12.4.2 Hardware RAID DragonFly also supports a variety of hardware RAID controllers. These devices control a RAID subsystem without the need for DragonFly specific software to manage the array. Using an on-card BIOS, the card controls most of the disk operations itself. The following is a brief setup description using a Promise IDE RAID controller. When this card is installed and the system is started up, it displays a prompt requesting information. Follow the instructions to enter the card's setup screen. From here, you have the ability to combine all the attached drives. After doing so, the disk(s) will look like a single drive to DragonFly. Other RAID levels can be set up accordingly. ### 12.4.3 Rebuilding ATA RAID1 Arrays DragonFly allows you to hot-replace a failed disk in an array. This requires that you catch it before you reboot. You will probably see something like the following in `/var/log/messages` or in the [dmesg(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#dmesg§ion8) output: ad6 on monster1 suffered a hard error. ad6: READ command timeout tag#0 serv0 - resetting ad6: trying fallback to PIO mode ata3: resetting devices .. done ad6: hard error reading fsbn 1116119 of 0-7 (ad6 bn 1116119; cn 1107 tn 4 sn 11) status#59 error40 ar0: WARNING - mirror lost Using [atacontrol(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#atacontrol§ion8), check for further information: # atacontrol list ATA channel 0: Master: no device present Slave: acd0 <HL-DT-ST CD-ROM GCR-8520B/1.00> ATA/ATAPI rev 0 ATA channel 1: Master: no device present Slave: no device present ATA channel 2: Master: ad4 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5 Slave: no device present ATA channel 3: Master: ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5 Slave: no device present # atacontrol status ar0 ar0: ATA RAID1 subdisks: ad4 ad6 status: DEGRADED 1. You will first need to detach the disk from the array so that you can safely remove it: # atacontrol detach 3 1. Replace the disk. 1. Reattach the disk as a spare: # atacontrol attach 3 Master: ad6 <MAXTOR 6L080J4/A93.0500> ATA/ATAPI rev 5 Slave: no device present 1. Rebuild the array: # atacontrol rebuild ar0 1. The rebuild command hangs until complete. However, it is possible to open another terminal (using **Alt** +***'F`***n***`***') and check on the progress by issuing the following command: # dmesg | tail -10 [output removed] ad6: removed from configuration ad6: deleted from ar0 disk1 ad6: inserted into ar0 disk1 as spare # atacontrol status ar0 ar0: ATA RAID1 subdisks: ad4 ad6 status: REBUILDING 0% completed 1. Wait until this operation completes. CategoryHandbook CategoryHandbook-storage