You are not logged in.

#1 2013-04-21 17:13:36

mraineri
Member
From: Massachusetts
Registered: 2013-04-20
Posts: 3

A Beginner's Guide to Installing on a RAID

I recently had to re-install everything on my system after I screwed up an upgrade, and I found myself struggling to piece everything together that I did last summer. Looking back on what's needed to be done, it really wasn't that much; it was just a matter of sifting through long guides online and picking out the right information. I decided to compile a quick guide for getting a fresh installation on top of a RAID as a reference for myself and everyone who may have been as lost as I was in the process. I'll include notes about why I chose to do things along the way, as well as spots where I may have done things not necessary, but still work for me. Comments and corrections are certainly appreciated for my own reference as well as others.

In the end, this is what I was trying to accomplish for my base RAID partitions:

/dev/md1: /boot (~100MB), RAID1 of /dev/sda1 and /dev/sdb1
/dev/md2: swap (~2GB), RAID0 of /dev/sda2 and /dev/sdb2
/dev/md3: root (~975GB), RAID1 of /dev/sda3 and /dev/sdb3

Since this is primarily a personal file server, I really just wanted redundancy in the event a disk fails. It didn't make sense to me to mirror the swap partition, so I figured I might as well double my swap size by making it RAID0.

These steps were done using the ArchBang 2013.03 x86_64 ISO. Also to forewarn people, since I was trying to preserve data from my first installation last summer, the first step was not done on this installation, so I tried putting together that step from memory.

Let's dive into the steps I took from booting off the Live CD.

Step 1: Configure the RAID Partitions

I created the base disk partitions on /dev/sda and /dev/sdb using cfdisk.

cfdisk /dev/sda
cfdisk /dev/sdb

Set up the base partitions on each disk. In my case, both disks will have the same partition layout. It starts with a boot partition at sda1 and sdb1, followed by the swap partition at sda2 and sdb2, then finally the root partition at sda3 and sdb3. In the end, both disks looked like this for me.

                              Disk Drive: /dev/sda
                      Size: 1000204886016 bytes, 1000.2 GB
             Heads: 255   Sectors per Track: 63   Cylinders: 121601

    Name        Flags      Part Type  FS Type          [Label]        Size (MB)
 ------------------------------------------------------------------------------
    sda1        Boot        Primary   ext4                                98.71
    sda2                    Primary   linux_raid_m                      1019.94
    sda3                    Primary   linux_raid_m                    999086.25*

Once the disks are prepared, create the RAID volumes with mdadm.
Note: When I originally did this part, I remember reading that if you plan on using Grub, you must set the meta data version to 0.90. I ended up not using Grub in the end, so setting the meta data version is probably not needed.

mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sd[ab]1
mdadm --create /dev/md2 --level=0 --raid-devices=2 --metadata=0.90 /dev/sd[ab]2
mdadm --create /dev/md3 --level=1 --raid-devices=2 --metadata=0.90 /dev/sd[ab]3

Once everything is created, wait until the volumes are synchronized by watching /proc/mdstat.

cat /proc/mdstat
Step 2: ArchBang Installation

Start the normal installation process outlined in the install guide.

abinstall

Set the system clock and time zone as you normally would.

Start the disk preparation step. When it asks you to partition disks, choose "DONE" since we already did this in step 1. Select the partition configuration you'd like to use. For me, /dev/md2 was used for the swap partition, /dev/md3 was used for the root partition, and /dev/md1 was used for the /boot partition. I also set up md1 and md3 to use ext4.

Run the installation process normally.

Set up the system configuration. The only special thing to do in here is to modify mkinitpcio.conf. Add "ext4" and "raid456" to the MODULES line, and add "mdadm" to the HOOKS line right after udev.
Note: raid456 may not be needed since I'm only using RAID0 and RAID1. You may also be able to get away with not specifying the module for your filesystem. I did these just to be on the safe side.

...
MODULES="ext4 raid456"
...
HOOKS="base udev mdadm autodetect modconf block filesystems keyboard"
...

Now install the bootloader. I chose syslinux since Grub was giving me a headache. When asked to review the syslinux.cfg file, choose "yes", and modify the APPEND lines to include your RAID volumes as kernel parameters. Do this for both the "arch" and "archfallback" labels

LABEL arch
        MENU LABEL Arch Linux
        LINUX ../vmlinuz-linux
        APPEND root=/dev/md3 ro md=1,/dev/sda1,/dev/sdb1 md=2,/dev/sda2,/dev/sdb2 md=3,/dev/sda3,/dev/sdb3
        INITRD ../initramfs-linux.img

LABEL archfallback
        MENU LABEL Arch Linux Fallback
        LINUX ../vmlinuz-linux
        APPEND root=/dev/md3 ro md=1,/dev/sda1,/dev/sdb1 md=2,/dev/sda2,/dev/sdb2 md=3,/dev/sda3,/dev/sdb3
        INITRD ../initramfs-linux-fallback.img

Exit the installer, but don't reboot just yet.

Step 3: Post Installation Configuration

Change the current root directory to be the installation's root directory.

mount -o bind /proc /mnt/install/proc
mount -o bind /dev /mnt/install/dev
chroot /mnt/install

Create the mdadm.conf file for your installation.

mdadm --examine --scan > /etc/mdadm.conf

Write out syslinux's boot code to the MBR on each disk.
Note: This may not be necessary, but I could not tell if the normal bootloader installation process sets up the MBR on both disks. One of my goals was to be able to boot from either disk in the event one explodes.

dd bs=440 count=1 conv=notrunc if=/usr/lib/syslinux/mbr.bin of=/dev/sda
dd bs=440 count=1 conv=notrunc if=/usr/lib/syslinux/mbr.bin of=/dev/sdb

Now you can reboot and start using your new system.

Fault Testing and Recovery

Once I started running off my new install, I wanted to check that the system remains operational in the event of a disk being removed. What I did was I unplugged sda (while the system was off), and turned it back on. It booted normally, and I saw /proc/mdstat report the missing disk. I put the disk back, recovered the RAID, and tried the same experiment with the other disk removed. This was also successful. These are some steps to rebuild a disk if one needs to be replaced. Please note that since I didn't have a fresh disk for testing, the recovery may have been simpler in my case since my "replacement disk" was really one of the original disks and is partitioned correctly. I didn't run all of the same steps below because I did not want to destroy my partitions.

Partition the new disk just like in step 1 and write out the bootloader code to the MBR.

cfdisk /dev/sdX
dd bs=440 count=1 conv=notrunc if=/usr/lib/syslinux/mbr.bin of=/dev/sdX

Add the disk to the RAID volumes.
Note: In my case, /dev/md2 was recovered automatically on the next reboot since it's my swap partition. I'm not 100% sure if anything is really needed in the event a truly new disk is added.

mdadm --add /dev/md1 /dev/sdX1
mdadm --add /dev/md3 /dev/sdX3

Wait until the volumes are synchronized by watching /proc/mdstat.

cat /proc/mdstat

And that's all there is to it. I hope this helps beginners such as myself!

Last edited by mraineri (2013-04-21 17:14:52)

Offline

#2 2013-04-21 18:55:12

oliver
Administrator
Registered: 2010-11-04
Posts: 2,209

Re: A Beginner's Guide to Installing on a RAID

nice guide... hope you don't mind, but I wifi-fied it at http://wiki.archbang.org/index.php?titl … _on_a_RAID

Totally agree on lack of need to mirror swap BTW

Offline

#3 2013-04-21 19:23:37

mraineri
Member
From: Massachusetts
Registered: 2013-04-20
Posts: 3

Re: A Beginner's Guide to Installing on a RAID

Go for it! smile

Offline

#4 2013-04-21 20:03:15

mike4ca
Happy (Arch|Manjaro)[B,b]ang User
From: Iowa, USA
Registered: 2011-10-17
Posts: 435

Re: A Beginner's Guide to Installing on a RAID

oliver wrote:

nice guide... hope you don't mind, but I wifi-fied it at http://wiki.archbang.org/index.php?titl … _on_a_RAID

Did you link this wiki to another article or add it to the TOC?  If not, I can take care of that.  Also, Stan had a long standing tradition of putting a link at the bottom of each article to get back to the TOC.  You can copy from another article or I can get it taken care of.


Free Software Foundation member 10865

Offline

#5 2013-04-22 00:09:20

mastis
Member
From: Finland
Registered: 2011-10-02
Posts: 136

Re: A Beginner's Guide to Installing on a RAID

Thank you for this, very much appreciated!

Offline

#6 2013-04-22 00:27:29

Mr Green
Administrator
Registered: 2010-11-07
Posts: 6,920

Re: A Beginner's Guide to Installing on a RAID

@mraineri Thank you for taking the time to post your guide.


Comments, suggestions please feel free to contact me mrgreen(at)archbang(dot)org

Offline

#7 2013-04-22 06:46:25

oliver
Administrator
Registered: 2010-11-04
Posts: 2,209

Re: A Beginner's Guide to Installing on a RAID

mike4ca wrote:
oliver wrote:

nice guide... hope you don't mind, but I wifi-fied it at http://wiki.archbang.org/index.php?titl … _on_a_RAID

Did you link this wiki to another article or add it to the TOC?  If not, I can take care of that.  Also, Stan had a long standing tradition of putting a link at the bottom of each article to get back to the TOC.  You can copy from another article or I can get it taken care of.

I just created it... no categories, no links etc.  I figured I'd come back to it later - just wanted to get it copied last night.  I'll see if I can get it all sorted and let you know if I run into issues

Offline

#8 2013-04-22 13:57:45

oliver
Administrator
Registered: 2010-11-04
Posts: 2,209

Re: A Beginner's Guide to Installing on a RAID

added TOC link to article and modified the TOC

Offline

Board footer

Powered by FluxBB