Create a Software RAID1 (gmirror)

From PFSenseDocs

This article is part of the HOWTO series.

This article is part of the DEVELOPMENT series. Any features, steps, or general information contained in this article is 100% unsupported and may break your system. Use at your own risk!

Creating a RAID1 Array with pfSense

WARNING

WARNING! THIS PAGE IS NOT COMPLETE YET. WIP

NOTE: requires snapshot RELENG_07_07_06 or better. Updated for RELEASE.

NOTE: Support for gmirror is partially missing from 1.2.1 and 1.2.2. Should be fixed by the release of 1.2.3, as the libraries are back in as of snapshots 2009/02/19.

This may break your car, eat your cat, submerge your basement in water, etc. Don't blame us for damage, it's unsupported!

These instructions assume that you are using ad0 (first IDE drive and ad1 which is the second IDE drive). SCSI drives will be da0 and da1. YMMV.

• Install pfSense as you normally would (option 99 from console menu). Let it reboot. Leave LiveCD in.
• If you have already done the above, for the love of god, backup your config from Diagnostics -> Backup/Restore
• Boot into the LiveCD - we need exclusive access to the volumes
• Invoke the shell (option 8)
• Run these commands to update fstab - if your first disk is not 0 doublecheck fstab to be sure gm0 (or whatever you named your raid) is used :

 mkdir /tmp/gtmp; mount /dev/ad0s1a /tmp/gtmp				# Create our mount point and mount volume
 sed -i".bak" "s%ad%mirror/gm%" /tmp/gtmp/etc/fstab 			# Change ad to gm/mirror
 echo geom_mirror_load="YES" >> /tmp/gtmp/boot/loader.conf		# Enable gmirror on boot
 umount /tmp/gtmp							# We are done with volume, dismount

NOTE: You can also use these types instead of round-robin, if you so wish:

• load : Read from the component with the lowest load.
• prefer : Read from the component with the biggest priority.
• round-robin : Use round-robin algorithm when choosing component to read.
• split : Split read requests, which are bigger than or equal to slice size on N pieces, where N is the number of active components.

• Run the following commands to create the mirror :

sysctl kern.geom.debugflags=16			# Enable footshooting
gmirror label -v -b round-robin gm0 /dev/ad0	# Tell gmirror to create raid0 array /dev/ad0 - should be
	 	 	 		 	#   the disk you have the pfsense install on already.
gmirror configure -a gm0			# Tell geom to automatically rebuild array
gmirror insert gm0 /dev/ad1			# Add the second disk to the mirror 
gmirror status					# Wait until the mirror builds
exit						# Exit when it's 100%

If it all went right, you should see a good mirror:

# gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ad0
                      ad1

• Reboot pfSense, remove the LiveCD, grab your feet and hope for the best.

Should you lose a drive, you can replace it physically and then run the following if it doesn't automatically try to rebuild:

gmirror insert gm0 /dev/ad1

If you still have trouble, you may need to make the mirror forget about the drive first:

gmirror forget gm0 /dev/ad1
gmirror insert gm0 /dev/ad1

Oh, and check out this article which goes into a little more detail.

Well, that didn't work!

WARNING

Here there be dragons

If you want to make a mirror on a version of pfSense where the mirroring is missing (1.2.1 or 1.2.2), you will find that it fails on the "gmirror label" step with an error not unlike:

gmirror: Unknown command: label.

Fear not, all is not lost!

• First, get thee a FreeBSD 7.0 LiveFS CD
• Boot from that CD.
  • To get to a prompt, first choose your region (e.g. USA), choose "Fixit", then "CDROM"
• Edit the fstab file to point to the mirror. If you already updated fstab/loader.conf, skip those particular lines, but you still need to copy geom_mirror.so to the hdd.

 mount /dev/ad0s1a /mnt				# Create our mount point and mount volume
 sed -i".bak" "s%ad%mirror/gm%" /mnt/etc/fstab 	# Change ad to gm/mirror
 echo geom_mirror_load="YES" >> /mnt/boot/loader.conf	# Enable gmirror on boot
 cp /dist/lib/geom/geom_mirror.so /mnt/lib/geom/	# Copy the geom_mirror.so library to the hdd
 umount /mnt						# We are done with volume, dismount

• Now, to start creating the array:

kldload /dist/boot/kernel/geom_mirror.ko	# Load GEOM_MIRROR support
sysctl kern.geom.debugflags=16			# Enable footshooting
gmirror label -v -b round-robin gm0 /dev/ad0	# Tell gmirror to create raid0 array /dev/ad0 - should be
	 	 	 		 	#   the disk you have the pfsense install on already.
gmirror configure -a gm0			# Tell geom to automatically rebuild array
gmirror insert gm0 /dev/ad1			# Add the second disk to the mirror 
gmirror status					# Wait until the mirror builds
exit						# Exit when it's 100%

• Done!

Alternate Method / Future Ideas

Ideas for future documentation:

• With working gmirror libs on the LiveCD, it should be possible to create the array before installing pfSense to the hdd.
• If there is a gmirror present before option 99 is chosen, it is offered as an installation target.
• When installing to gmirror, do NOT try to install bootblocks directly to the mirror, it just doesn't work. Install to the HDDs only.