Introduction to FlexRAID 2.0

Courtesy of: Mike Kirnak (Nak)

Introduction

Welcome to FlexRAID 2.0! This guide is for the beginner who has basic computer skills, but no expertise in RAID or data back up. In short, this is FlexRAID 2.0 for Dummies. This guide will help you decide if FlexRAID is right for you, and walk you through the most basic FlexRAID setup. FlexRAID is extremely flexible and is capable of a lot more than I’ll cover here. I’m just going to walk you through the most basic setup and get you up and running. I’ll ignore many of the possible configurations; I’ll just give you one that will work.

Check the date up by the title, there may be a later version of this guide. Always use the latest version and match the version number to the version of FlexRAID you’re using.

What is FlexRAID?

Flex is a software based system for protecting your data from data corruption and failed hard drives. FlexRAID is more flexible than hardware Raid in many ways. There’s no limit in the type of storage that FlexRAID can protect; whatever storage your Operating System can access can be protected by FlexRAID. This includes removable hard drives, optical disks, network shares, even flash memory cards! All of your drives remain accessible and can be read by any device that could read them before. In the event that more drives fail than allowed by your configuration, all data on the discs that did not fail remains intact and accessible. Only the data on the failed discs is lost, and only then if more drives fail than the tolerance of your chosen configuration. The main limitation to FlexRAID is that it doesn’t excel at protecting the disc your Operating System resides on. You should use hardware RAID or backup software designed for backing up your boot disk.

What can FlexRAID do?

FlexRAID has 3 main functions: Snapshot RAID, Live RAID, and storage pooling. This guide will discuss Snapshot RAID  only.

Snapshot RAID takes a picture of your data at one point in time, the point you last performed an update. Any changes since the last update are not protected. Snapshot RAID is best used on data that does not change too often. It’s perfect for large collections of multimedia content. Automatic updates can be scheduled to run every night, or whatever frequency you desire.

Overview of Operation

Let’s say you have four drives you need to protect. Or twenty—or more—it doesn’t really matter. You’ll need at least one extra drive to store the parity data. Drives you’re protecting are called “DRU”s—Data Risk Units–while drives you’re using to protect your data are “PPU”s—Parity Protection Units. You can have more than one PPU; you can have as many as you want. If you have one PPU, you can recover from one hard drive failure. If you have two PPUs, you can recover from two simultaneous hard drive failures and so on. Personally, I think two PPUs is a good number if you’re protecting more than three hard drives. For three or fewer DRUs, One PPU should do it. I might add more PPUs if I were protecting more than 15 DRUs. Each PPU MUST be as large as your largest DRU. If your biggest DRU is 2TB, each and every PPU MUST be at least 2TB in size. Your “array” is all of your PPUs and DRUs. The word “array” and “RAID” are used interchangeably.

You set your array up, now your data is protected. The next day you add a few movies to your collection; now you want to update your array so those new movies are protected too. You open FlexRAID, go to the update dialog, do a quick validate to make sure all of your data is still good then hit update. FlexRAID chugs away and after a bit your new movies are protected.

If you have more disc failures than you have PPUs, the data on the failed discs is lost, but all of your other data remains intact. Today that doesn’t happen though. You have two PPUs and 10 DRUs. Darn! Two of your drives fail, it could be any combination of DRUs and PPUs. You replace the two hard drives, open FlexRAID, open the restore dialog, select the two DRUs/PPUs that failed, hit restore and you’re back in business in a short while! Just make sure you replace the failed hard drives at your earliest opportunity. You don’t want to wait until you get another hard drive failure and end up with more failed hard drives than PPUs. Once you’ve restored your array, you can again have as many disc failures as you have PPUs.

Installing FlexRAID

OK, this is easy. The instructions are for Windows XP, other versions will be slightly different. Download the latest version of FlexRAID 2.x from Download / Try / Buy. While you’re there look at http://bug.flexraid.com/ look at any bug reports that are still open. That’ll tell you of any bugs you might need to work around. Run the installation program and accept all of the defaults. When it asks if you want to start as a service, the answer is “Yes”. Now open your anti-virus/firewall programs and make exceptions for FlexRAID.  If your AV program scans every file FlexRAID opens, your array creation and updates could take a very long time.

Open Windows Explorer and navigate to

“C:\Program Files\OpenEgg.org\FlexRAID 2.0 ” Right click on “log.log” Under “Send To” click on “Desktop (Create Shortcut)”. Make a copy of log4j.properties in the same directory. Name the copy “log4j.propertiesBAK”. Open “log4j.properties” with a text editor like Notepad. Replace every “TRACE” and “DEBUG” with “INFO”. (Skip the quotation marks, of course.) Save the file.

There should now be a “FlexRAID.html” icon on your desktop. When you double click this icon, FlexRAID will open in your default web browser. If you have Google Chrome installed, and it’s not your default browser, you can right click on the icon, open properties and select Chrome under “Opens With”. FlexRAID is very snappy under Chrome. You don’t have to do this, it runs fine with Firefox or Internet Explorer.

Success! FlexRAID is now installed on your PC.

Configuring Your Array

Double click on the “FlexRAID.html” icon. Your web browser will open a window with three icons. “Add A New Configuration”, “Scheduler”, and “About”. Click on “Add A New Configuration”.

The “Add A New Configuration” dialog box opens. Type a name in the “Name” field, it doesn’t matter what. “My FlexRAID Array” works fine.

Make sure the “Snapshot” radio button is selected. The “Initial Configuration” box should be checked. Select how many DRUs you might conceivably have total in the foreseeable future. Add a couple extra just in case. Select how many PPUs you are going to use. You MUST select the exact number you’re going to use. No adding any extras for future use with the PPUs. In the drop down dialog for “RAID Engine” select “Tx [RAID oo]”. Leave all other items unchecked and empty. Click “Create”.

There is now a new Icon in the web browser window, “My FlexRAID Array”, or whatever you named your array. Before the next step you have to make sure all of the discs (network shares, optical discs, flash memory, hard drives, etc.) that will be in the array are online. Both DRUs and PPUs.

Click on “My FlexRAID Array”. The “Data Configuration” window should be open. Your available drives will be on the left. A numbered list of DRUs and PPUs will be on the right. DRU1 should have an empty field underneath it. Drag the first drive you want to protect over into the field near the top. Click on DRU2, a field opens under it. Drag the second drive you’re protecting into that field, near the top, and so on. You do not have to fill each DRU with a drive. Do the same with the PPUs; you must fill each PPU with a drive.

Click on the “+” to the right of “Run-Time Properties”. Click on “Advanced to open the advanced properties. In the “Full PreValidate on Restore” dialog box, change “false” to “true”. Do not use the quotation marks… Do not change any other value. Scroll down and click “save”.

Click on the “+” to the right of “Command Execution Center” Click “Create”. You’ll initially see “The [<[create]>] task has successfully initiated… Process number: 2” There will be a little spinning gizmo indicating FlexRAID is working on the task. When it’s done the spinning gizmo will be replaced with “100% Completed”. This could take a few minutes— not too much data, local drives—to several days—many terabytes of data, slow 100mbs network drives.

Once it’s done, click “Quick Validate”. You’ll initially see “The [<[create]>] task has successfully initiated… Process number: 3” There will be a little spinning gizmo indicating FlexRAID is working on the task. When it’s done the spinning gizmo will be replaced with “100% Completed”. You should see “Success! The RAID is healthy and up to date!”. Woo Hoo! You’re done!

Maintaining Your Array

Anytime your data changes significantly, you’ll want to update your array. In the “Command Execution Center” click on “Validate”.

If you’ve only added data, you should get “Success! The RAID is healthy and up to date!”.

Click “Update”, it shouldn’t take too long. An hour or so if you’ve added 30 or 40 GB of data to a slow 100mbs network drive, less for local data.

If you’ve deleted data, or there is a problem with the array, you’ll only get the “100% complete”. Double click on the “log.log” shortcut we created on your desktop. It should tell you the problem. Hopefully there will just be some warnings that some files were deleted, and you meant to delete them. Today, that’s the case.

Click “Update”, it shouldn’t take too long. An hour or so if you’ve added 30 or 40 GB of data to a slow 100mbs network drive, less for local data.

If other problems are indicated in the log, DO NOT click update! You’ll lose whatever data is damaged. Instead, restore your array.

Even if you don’t change your data, it’s a good idea to validate at least weekly. Run a Verify for good measure once a month. A verify is just like a validate except it’s a little bit more thorough and takes a lot longer to run.

Restoring Your Array After Data Loss

If a “Validate” or “Quick Validate” indicates lost or corrupted data, you have a decision to make. You can simply restore the data. Or you can replace the hard drive and then restore the data. Determine if the hard drive is failing, or if this is a one time deal. If you’ve lost data on this drive before, or if there’s a lot of data lost on the drive, replace the drive. Try and use HDD diagnostics to see if the drive is failing.

Regardless of whether or not you replace the hard drive the recovery process is the same. In the “Command Execution Center” click on the arrow to the left of “Restore” to display the drop down dialog for the restore process. You’ll have one or more drop down dialog windows; you’ll have as many as you have PPUs. In one or more drop down dialogs, select the drives that need to be restored. If only one drive is impacted, you only need to pick that one. Click on the “Restore” button. If you didn’t replace a hard drive, and are just restoring a few files, only the lost/corrupted files will be restored. If you’ve replaced the hard drive, obviously the entire drive will be written. Time to completion will again depend on your setup.

After the restore is complete, run a quick validate. If all is good, run “Update” if needed to add any new files to the protected pool. (The Restore process won’t delete new files, it only restores lost/corrupted files.)

In Closing

Remember, you don’t have to sit there while long processes are running. It’ll wait till tomorrow for you. Also, don’t forget that WinXP can only access a 2TB drive or smaller. FlexRAID won’t restore the boot drive on your PC. What if you want to increase the number of PPUs or DRUs that you originally configured your RAID for? No big deal, just configure a new array with your current drives. All your data will remain intact; no data is harmed in the creation of an array. (Except the parity data on the PPUs; you’ll have to delete that prior to configuring a new array.) So yes, if you want to create an array with disks that already contain data, you can. (Try that with hardware RAID!) You may have heard that some disks have something called TLER disabled and can’t be used for RAID. No worries, not true with FlexRAID. ANY disc your OS can read can be used with FlexRAID.

The Scheduler can be used to automate routine tasks like Validate and Update.

Preview and Beta versions of FlexRAID expire after a few months. Just check the website and download the latest version before that happens. Final releases don’t expire.

That’s it in a nutshell. FlexRAID will do a lot more than what’s been outlined here, feel free to study it more on http://www.flexraid.com/ But if you just want to protect a large collection of multimedia files, or other files that don’t change too often, you’re already done!

Be Sociable, Share!

Revisions

Tags: ,

6 Responses to “Introduction to FlexRAID 2.0”

  1. Joe July 8, 2011 at 11:53 PM #

    Great guide, thanks!

  2. Phil July 23, 2011 at 10:39 AM #

    I have installed Flexraid on WHS 2011. I click on “Add new configuration”. A small windows opens I type a name in the box then click “Create” but nothing happens.

  3. mihawk90 August 12, 2011 at 10:53 PM #

    Hi, nice guide, explains some things quite nice, but there are (in my eyes as a totally beginner with flex raid) huge difficulties to overcome.

    The first thing, after installing and setting the log, there is no suddenly appearing FlexRAID.html Icon on desktop, so obviously you can’t just double-click it.

    So after figuring out (oh how difficult…) there is no html, I just found the webUI somewhere to download, I did this and found myself starting the webserver included, after that I got on my localhost via webbrowser and voilá there was a web interface suddenly, though I don’t know if it’s the same :)

    So, after that I guessed I needed to add a new host, and I did, no problems with that, but I can’t get the configuration to work.
    I clicked on new configuration (the database-looking icon with a plus on it I guess), and a small form appears where I can enter a name, or import an existing configuration.
    So I gave it a name, and clicked on “Add”, now there’s this form where I can add DRUs and PPUs. I guess its selfexplaining but, no, it’s not.
    I add some DRUs (some more then I actually have, cause you advised this in the guide), and some PPUs.
    Now here’s my problem, there is no Create Button? (or at least I’m too dumb to find it I guess) And there are no “other items” to check or uncheck, either. What to do? Whatever else I click it says “At least one PPU path must be provided!” And when I add one more it says Hey this Config supports just one PPU.

    So what do I do, I just click some random things in the window. Suddenly one of my drives appears on the configuration box for drag&drop to the DRUs.
    So I decide to rework the whole thing. I delete my volumes and re mount them in a folder (not as drive letter, no, an NTFS Folder; And yes, I’m working on a virtual machine I dont want to blast my real drives ;) ).
    Again in the FlexRAID Webconfig I try to get these folders to the configuration – what takes like hours cause double clicking doesn’t always work and drag&drop doesn’t work quite well…
    I finally get the idea (you need to D&D them a little, then one click on an item and it appears on the configuration box… who the hell developed this? ;P )
    Additionally, just to mention, you need to drag them one by one and it works.

    So now, we got the mountpoints ready and I now get the Idea of the “create Button”. It’s not a button like a…. button like you know it from windows or something, it’s the little arrow-thingy Icon on the top, click and yay we got it, click create in the popping window and you’re done.

    And by the way, just to mention, I personally don’t get the idea of multiple DRUs… I mean, you can add multiple volumes to one DRU, so what exactly is the point of making multiple DRUs?

    I mean these are just the things that took me hours to get working with, and I don’t think everyone has that much endurance to work like this for it, I think this guide might need to be updated because its for older versions of the software or something :S

    Best regards
    ~Mihawk~

  4. Mihawk90 August 12, 2011 at 11:04 PM #

    Ah and one (yeah I know real dumb) question I didn’t mention…
    How… do I get the configuration I just setup as a drive into Windows? Is this only with this.. FlexRAID View or how it is called? Or did I miss something in the guide?
    Is FlexRAID then “juts” for “protecting” my data from corruption?

    Also, what exactly is the difference between Validate, Quick Validate and Verify? I don’t see it in this guide though it’s something that might be important for totally beginners :)

    Regards
    ~Mihawk~

  5. Dan January 31, 2013 at 5:06 AM #

    Hi Mike, I’ve just given a try to snapshot, having 1 DRU and 1 PPU in configuration. I understood that any combination of DRU/PPU failure, not exceeding the number of PPUs in configuration will ensure data protection. In my particular test (1 DRU+1 PPU) I simulated a failure of the PPU which made the array impossible to restore because the metadata seemed to be lost. I agree that in this case the DRU data remains intact. However remains the question, how could be the PPU restored in such a case? Does this happen in all configurations with only one PPU?

    Regards,
    Dan

  6. NewYears1978 October 15, 2014 at 2:20 AM #

    Is there no way to set this up with 1 parity and 1 data drive only?

    I was trying to set it up and it wouldn’t let me..which means I have to buy another drive.

Leave a Reply

eleven + 4 =