Storage Acceleration

Storage Accelerators: Landing Disk & SSD Caching

Overview

Storage accelerators are the final pieces to the performance options under Transparent RAID.
These performance boosting features were the most obvious features to implement. However, we chose to implement them last and for good reason.

It was really important to us to design the core Transparent RAID engine to be as optimal as possible without resolving to external enhancements. No matter what type of storage acceleration one puts in front of a RAID array, the data needs to eventually tickle down to the array. Essentially, a good cache layer is nothing if the medium it has to flush to is too slow. So, rather than masking the challenges Transparent RAID faced, we were pretty open about them, which gave us the opportunity to address those challenges rather than sweeping them under the rug.

Additionally, we want and expect storage acceleration to be optional. Transparent RAID should be fully usable without any external enhancement for the typical user.

Our achievement

We have succeeded in our design goals. :)
Check out the charts posted in the Performance mode thread.

The average system in Energy mode using S.W.O performs far better than all other comparable NAS solutions on the market.
However, this was not enough for us. We boosted performance further with TCQ and OS Caching. OS Caching improves the user’s experience in normal usage.
Finally, for those users with high latency disks, we introduced a Performance mode that minimizes latency and boosts performance to a height theoretically not possible. In theory, Transparent RAID should not be able to achieve more than 1/3rd of the speed of a single disk. In practice, properly equipped systems are hitting close to 80% of the speed of a single disk.

It is important to understand that Transparent RAID does not stripe data and should not be compared to striped RAID implementations.
This stems from a critical design choice, which provides certain benefits but at the cost speed. Even then, Transparent RAID is actually faster than some striped RAID implementations. :P

External enhancements

Although the average user will be plenty satisfied with the performance out of Transparent RAID as designed, there is always a group of select few that always demand more, and they cannot be ignored.
We have pushed the design of the core Transparent RAID engine as far as we could from a performance perspective. Any significant performance boost would have to be external.

External accelerators are not a bad thing. Accelerators are great for transactional operations while large cheap storage arrays are great for warehousing.
If one’s daily activities can execute on a fast/hot storage while the inactive data remains on the slow/cold storage, we can achieve an ideal from many fronts.

Accelerators:

Landing Disk

The first solution we have developed is the concept of a Landing Disk.
A landing disk purely accelerates write operations for new data. Eventually, that data will get moved to the main array through a background process.
The landing disk can be a mirror set so that it is protected or it can be a single unprotected disk. Using an unprotected disk as landing disk should not be a big of a deal for deployments where it is merely viewed as a buffer. Those that require protection of that buffer can mirror the landing disk so that it is protected.

In Transparent RAID, a landing disk can be a folder on your OS disk or any other disk in your system that is not part of a Transparent RAID array. This means that, for most users, there will not be an additional hardware purchase. Of course, you can fully take advantage of the landing disk feature by using a dedicated super fast disk such as a fast SSD for such purpose.

If you have multiple Transparent RAID arrays, they can all share the same disk as landing disk.
Also included is a configuration for including and excluding what goes into the landing disk. We can foresee scenarios where a user might want certain data to just go to the array skipping the landing disk.

Configuring a Landing Disk

1. Register a volume on a disk not part of the Transparent RAID array. This is the volume on the disk where data will land.
Unless your OS disk is also the landing disk or unless you have other purpose for the landing disk, it is recommended that you remove all drive letters and mount points off the landing disk. Transparent RAID does not need any mount point or drive letter to make use of the landing disk.
Add or keep the drive letter on the landing disk only if you need access to the disk for other purposes.
1. Register Volume

2. Give the registered volume a name.
2. Registered Volume Name

3. Right-click on the registered volume and choose to register a folder. Unless the folder was previously created by Transparent RAID, please make sure that a folder with the name you will be choosing does not already exist on the selected volume.
Registered folders are protected hidden folders only accessible by the NZFS broker service by default, and Transparent RAID needs to create them as such.
There are current complications with changing existing folders so that they are restricted to external elements.
To allow multiple RAID configurations to use the same disk as landing disk, folders are used to partition the disk. Also, the landing disk can be used for other purposes. For instance, the landing disk can be your OS disk or any other disk not part of a Transparent RAID array. In such cases, folders are used to separate landing data from other data on that disk.
3. Register Folder

4. Give the registered folder a name.
4. Registered Folder Name

5. After at least one folder has been registered and that your RAID has been initialized, you can launch the Storage Acceleration panel from the Advanced Operations menu for your particular RAID configuration.
5. Storage Acceleration

6. Select a configured registered folder and add it to your RAID configuration to enable the feature.
6. Select Landing Folder

7. Once enabled, you will be presented with a number of options.
Restrictions: you can use regular expressions to restrict what data is allowed to land on the landing disk. By default, restrictions are disabled. When enabled, you can select whether the matches are inclusion matches or exclusion matches.
Priority: The priority level determines how and when the background mover process moves data off the landing disk onto the protective array. The default is to move the data during idle system times (when the system is not under heavy use). Essentially, you don’t want the mover to be taxing the landing disk while new data is being written to it as that would be counter productive.
Delay: The delay affects both the frequency of scan for data to move off the landing disk and the minimum age of the data on the landing disk before it qualifies to be moved to the protective array. The minimum age is used to avoid moving data that is currently being read or written to.
Please remember to click on the save button to commit your selections.
7. Landing Disk Options

8. If the storage pool was running, you will need to stop and restart it for the changes to take effect. Otherwise, simply start the storage pool and enjoy. :)

SSD Caching

(Coming soon)

Be Sociable, Share!

Revisions

2 Responses to “Storage Acceleration”

  1. Methanoid June 6, 2016 at 6:43 AM #

    SSD Caching “Coming Soon”… it’s been 7 months. Is it “soon” yet? :(

    • admin January 6, 2017 at 8:16 PM #

      Was on a long hiatus. Development has resumed. So, it shouldn’t take too long. No ETA, but is a planned major feature. In fact, development had started on this… just not yet completed.

Leave a Reply

nineteen + 16 =