You are viewing an old revision of this post, from March 26, 2011 @ 16:48:02. See below for differences between this version and the current revision.

Multi-Step Parity

Multi-step Parity is a unique feature in FlexRAID that enables you to RAID data that you cannot have online all at once.

Multi-step parity is supported only by the commmand-line client and is not supported yet by the WebUI. This current documentation requires FlexRAID 1.4 build 7 or higher. Multi-step parity supports only the T1 engine. The idea behind this feature is that, sometimes, you want to create parity for data you cannot have online all at once. A good example would be DVD media discs. If you only have a single DVD reader but want to create parity for say 10 discs, well you are stuck. With the new "Multi-step parity" feature, you can create parity for the 10 discs by inserting them one at a time. This feature is not limited to optical media discs. If you have more hard drives than you can have online, this feature will also fulfill your needs for parity protection. Further, you are not limited to inserting one disc at a time. You can insert as many discs as you have readers or drive mount points. In my particular case, I have 3 DVD readers. So, I can update the parity data (or restore data) using 3 discs at a time. FlexRAID™ captures a fingerprint for each disc and uses that fingerprint to differentiate between the discs (as opposed to the mount point). Multi-step parity is supported only by the commmand-line client and is not supported yet by the WebUI. This current documentation requires FlexRAID 1.4 build 7 or higher. Multi-step parity supports only the T1 engine. Multi-step parity currently supports two tasks (to create and restore): multi-step-create and multi-step-restore
  • multi-step-create: this task supports the same properties as the regular "create" task. This task knows which mounted data is which as it computes a fingerprint for each. It generates a metadata file (whose full name is the fingerprint) for each included data in the format flxr_xxx_xxx_xxx_xxx.meta. The multi-step-create task will also output the list of files for each included data in a file named flxr_xxx_xxx_xxx_xxx.meta.txt. You should look at the content of the flxr_xxx_xxx_xxx_xxx.meta.txt files to match the data to the fingerprints. The flxr_xxx_xxx_xxx_xxx.meta (fingerprint) and flxr_xxx_xxx_xxx_xxx.meta.txt (files list) files can be found in the first parity path.
    • multi-step-create supports one additional property: multi-step-remove If this property is set to "true", the task will switch into "remove" mode and remove the mounted DRUs from the RAID set. This feature is useful for when you need to retire a drive from the RAID set. Additionally, since Multi-step parity does not support rsynch, this feature can be used to remove a drive from the RAID set, update its content, and then add it back to the RAID set.
  • multi-step-restore: this task supports the same properties as the regular "restore" task with the following exceptions:
    • The "restorePath" property follows this pattern: <Paths to restore to>|<restore DRU's fingerprint> Ex: restorePath=D:\Data\RestorePath1;D:\Data\RestorePath2|flxr_xxx_xxx_xxx_xxx.meta When restoring, the failed data must be mounted to DRU1 for the very first pass. Note: The numbering of the DRUs is based on the number of mount points you have available. If you only have one data mount point, you will always refer to DRU1 and mount different data paths to it. Similarly, if you have two data mount points, you will be refering to DRU1 and DRU2. If there is any data mounted in DRU1, only the missing and corrupted files will be restored. If there is no data mounted, all data will be restored. I.e., if you have a scratched disc but only want to restore the unreadable files, you can insert the scratched disc and FlexRAID™ will restore only the unreadable files. If the disc is missing, leave the DRU1 mount point empty and FlexRAID™ will restore the whole disc. After the first pass (to indicate the failed data), you can mount as many data paths as you have mount points to resume and complete the restore process.
    • The multi-step-restore supports two additional properties:
      1. restoreInvalid: whether to restore existing but invalid files - DEFAULT: true
      2. restoreValid: whether to restore existing and valid files by just copying them (no parity computation) - DEFAULT: false This is useful when restoring a disc and you want FlexRAID™ to copy off all valid files before attempting to restore the invalid ones through parity.
    • After the restore is complete, a file named flxr-restore-completed.txt will be created in the first restore directory. You MUST delete this file after the restore process and not include it in the RAID.
 

Revisions

Revision Differences

March 26, 2011 @ 16:48:02Current Revision
Title
Multi-Step Parity  Understanding Multi-Step Parity
Content
<h4>Multi-step Parity is a unique feature in FlexRAID that enables you to RAID data that you cannot have online all at once.</h4> <h4>Multi-step Parity is a unique feature in FlexRAID that enables you to RAID data that you cannot have online all at once.</h4>
Multi-step parity is supported only by the commmand-line client and is not supported yet by the WebUI. Multi-step parity is supported only by the commmand-line client and is not supported yet by the WebUI.
This current documentation requires FlexRAID 1.4 build 7 or higher. This current documentation requires FlexRAID 1.4 build 7 or higher.
Multi-step parity supports only the T1 engine. Multi-step parity supports only the T1 engine.
The idea behind this feature is that, sometimes, you want to create parity for data you cannot have online all at once. The idea behind this feature is that, sometimes, you want to create parity for data you cannot have online all at once.
A good example would be DVD media discs. A good example would be DVD media discs.
If you only have a single DVD reader but want to create parity for say 10 discs, well you are stuck. If you only have a single DVD reader but want to create parity for say 10 discs, well you are stuck.
With the new "Multi-step parity" feature, you can create parity for the 10 discs by inserting them one at a time. With the new "Multi-step parity" feature, you can create parity for the 10 discs by inserting them one at a time.
This feature is not limited to optical media discs. This feature is not limited to optical media discs.
If you have more hard drives than you can have online, this feature will also fulfill your needs for parity protection. If you have more hard drives than you can have online, this feature will also fulfill your needs for parity protection.
Further, you are not limited to inserting one disc at a time. Further, you are not limited to inserting one disc at a time.
You can insert as many discs as you have readers or drive mount points. You can insert as many discs as you have readers or drive mount points.
In my particular case, I have 3 DVD readers. In my particular case, I have 3 DVD readers.
So, I can update the parity data (or restore data) using 3 discs at a time. So, I can update the parity data (or restore data) using 3 discs at a time.
FlexRAID™ captures a fingerprint for each disc and uses that fingerprint to differentiate between the discs (as opposed to the mount point). FlexRAID™ captures a fingerprint for each disc and uses that fingerprint to differentiate between the discs (as opposed to the mount point).
  Multi-step parity is supported only by the commmand-line client and is not supported yet by the WebUI.
  This current documentation requires FlexRAID 1.4 build 7 or higher.
  Multi-step parity supports only the T1 engine.
  <!--more-->
  Multi-step parity currently supports two tasks (to create and restore): <strong>multi- step-create</strong> and <strong>multi- step-restore</strong>
  <ul>
   <li><strong>multi- step-create</strong>: this task supports the same properties as the regular "create" task.
  This task knows which mounted data is which as it computes a fingerprint for each.
  It generates a metadata file (whose full name is the fingerprint) for each included data in the format flxr_xxx_xxx_ xxx_xxx.meta.
  The multi-step-create task will also output the list of files for each included data in a file named flxr_xxx_xxx_ xxx_xxx.meta.txt.
  You should look at the content of the flxr_xxx_xxx_ xxx_xxx.meta.txt files to match the data to the fingerprints.
  The flxr_xxx_xxx_ xxx_xxx.meta (fingerprint) and flxr_xxx_xxx_ xxx_xxx.meta.txt (files list) files can be found in the first parity path.
  <ul>
   <li>multi-step-create supports one additional property: <strong>multi- step-remove</strong>
  If this property is set to "true", the task will switch into <strong>"remove" mode</strong> and remove the mounted DRUs from the RAID set.
  This feature is useful for when you need to retire a drive from the RAID set.
  Additionally, since Multi-step parity <span style="text-decoration: underline;">does not</span> support <strong>rsynch</strong>, this feature can be used to remove a drive from the RAID set, update its content, and then add it back to the RAID set.</li>
  </ul>
  </li>
   <li><strong>multi- step-restore</strong>: this task supports the same properties as the regular "restore" task with the following exceptions:
  <ul>
   <li>The "restorePath" property follows this pattern: &lt;Paths to restore to&gt;|&lt;restore DRU's fingerprint&gt;
  Ex: restorePath=D: \Data\RestorePath1; D:\Data\RestorePath2|flxr_ xxx_xxx_xxx_xxx.meta
  When restoring, the failed data <strong>must</strong> be mounted to DRU1 for the very first pass.
  Note: The numbering of the DRUs is based on the number of mount points you have available.
  If you only have one data mount point, you will always refer to DRU1 and mount different data paths to it.
  Similarly, if you have two data mount points, you will be refering to DRU1 and DRU2.
  If there is any data mounted in DRU1, only the missing and corrupted files will be restored.
  If there is no data mounted, all data will be restored.
  I.e., if you have a scratched disc but only want to restore the unreadable files, you can insert the scratched disc and Flex<em>RAID</em>™ will restore only the unreadable files.
  If the disc is missing, leave the DRU1 mount point empty and Flex<em>RAID</em>™ will restore the whole disc.
  After the first pass (to indicate the failed data), you can mount as many data paths as you have mount points to resume and complete the restore process.</li>
   <li>The multi-step-restore supports two additional properties:
  <ol>
   <li><em>restoreInvalid</em>: whether to restore existing but invalid files - DEFAULT: true</li>
   <li><em>restoreValid</em>: whether to restore existing and valid files by just copying them (no parity computation) - DEFAULT: false
  This is useful when restoring a disc and you want Flex<em>RAID</em>™ to copy off all valid files before attempting to restore the invalid ones through parity.</li>
  </ol>
  </li>
   <li>After the restore is complete, a file named <strong>flxr- restore-completed.txt</strong> will be created in the first restore directory.
  You MUST delete this file after the restore process and not include it in the RAID.</li>
  </ul>
  </li>
  </ul>
  &nbsp;

Note: Spaces may be added to comparison text to allow better line wrapping.

Tags:

No comments yet.

Leave a Reply