Creating a NAS With ESXi and unRAID 3


Virtual Machine IconI’ve been using virtualization for quite some time now, with overwhelmingly positive results. In this series of blog posts, I’ll take you on a virutalization journey from start to finish. I’ll try to make the process as simple as possible, as fun as possible and as cheap as possible. Buckle your seat-belts, the journey starts now…

Disclaimer: This blog post is really not meant as an impartial critique of various solutions but rather I choice I made given my requirements, features I needed and resources I had.

Boring Terminology

For the uninitiated, virtualization is a techie term referring to the isolation of an OS (operating system) and its applications into what is called a VM or Virutal Machine. And Because these virtual machines are separate and independent, many can be run on a single computer at the same time, maximizing the use of your hardware. The benefit is that you could have a mixed environment of machines (Windows, Linux, FreeBSD, etc…) doing different tasks – something that you could otherwise not do unless running multiple computers. A dedicated operating system is necessary (technically, it’s called a hypervisor and provides a virtualization host for operating systems). The hypervisor is designated as the Host OS, and virtual machines that run on the host are called Guests.

Why Virtualizaton?

While targeted at enterprises as a way to reduce IT expenses, homes users can benefit from virtualization as well.  Let’s highlight a scenario where we can see the merits of investing the time (and money) to use virtualization software.

Digital files are ubiquitous. Storing data in a digital world becomes a problem because of data loss. Sure, I can use DVD/Blu-ray as a backup medium but that’s time consuming and unmanageable in the long run. Not to mention, no one really has a meaningful sense of the longevity of optical media in general. Hard drives are our next best option. But scattered, independent hard drives are just as bad – without warning, one may die on you and bam! You’ve lost everything! What I really needed was a NAS (Network-attached storage). I also needed something that’s scalable, sharable and reliable. Furthermore, I wanted to do a little more with my hardware; like run a database for my media and router software for the Internet (with advanced features like web filtering, and VPN access – way to connect to my home network from anywhere).

In terms of data storage software, I wanted the following:

  • Data equally distributed across all disks
  • Ability to expand the array
  • Basic data protection
  • Drive pooling or virtual shares
  • Strong  support community

And in terms of hardware, I wanted the ability to use the same hardware for other tasks.

NAS Software?

So I briefly mentioned what a NAS was and if I didn’t, it’s basically a computer with access to a network (hence, the network in the title) that houses any number of disks – it’s really both a hardware/software solution. Keep in mind, you could shortcut this whole blog post by going to your local tech store and buying a two or four disk NAS but chances are, you’d do better going the DIY route in terms of performance, expandability and flexibility.

Back when I was researching this (which is many moons ago now), I chose unRAID for quite simply, it’s ease of use and ability to be virtualizated (as a guest).  Of course,  if I had to do it all over again, I’d be confused as to the sheer amount of options out there: unRAID, FreeNAS,  FlexRAID, SnapRAID, NAS4Free, napp-it,  Open Media Vault, Windows Server and probably a ton more!

Ultimately, I chose to go with unRAID, but I’ll discuss another option, FreeNAS, just for sake of comparison. Be forewarned, this does get a tad bit technically.

Really it should be noted that it’s the ZFS filesystem that’s the true differentiating factor here and in reality you can install any Linux distro and load ZFS and not be stuck with FreeNAS.

unRAID Pros & Cons

The GoodThe Bad
If one disk dies, rebuild posible because of Single Parity - one disk sacrificed for this safetyCommercial and OpenSource GPL2 solution - so not free
If two disks die, you can take a working data disk and read it in another machine (every data disk has its own file system)Moderate amount of Linux expertise needed
Easy to administer for the novice (geared toward the home user not so much enterprise)No integrity checksumming, automatic data recovery, double-parity RAID-Z, file system snapshots
Power efficiency (you can have only one disk spinning and all the other disks nicely spun down while you're watching a movie for example)Not blazing fast (but quicker than low cost hardware NAS solutions)
Doesn't need much memory (RAM) to function properlyObscure Linux distro
Plugin model (user contributed but not complete)Data not striped across disks

FreeNAS (ZFS) Pros & Cons

The GoodThe Bad
FreeHarder to administer (geared toward the enterprise)
Speed wise, ZFS is much quicker than unRAIDAll disks will be spinning
ZFS - supports single, double or triple parityMemory hungry
ZFS - compression, encryption, deduplication, snapshots are supportedIf two disks die, you lose everything
Hot Spare (it would automatically start a rebuilding process in the event of a failed disk)

Ultimately, FreeNAS (ZFS) provides some mission critical features that I’d say are simply overkill for a home environment. Again, the choice is yours.

To Virtualize or Not?

Okay then, let’s complicate the matter. The aforementioned solutions are meant to be installed on bare metal – meaning installed directly on hardware rather than within a host (virtualized) operating system. In fact, that’s how I started with unRAID. But because of unRAID’s somewhat obscure Linux distro there was a push to get unRAID virtualized thus allowing the migration of plugins from unRAID to isolated VMs and accruing the  following benefits:

  • Improved stability of unRAID
  • Application isolation

However, there was always a case to be made that virtualizing unRAID may introduce stability and data integrity issues and as such there was a movement to develop unRAID 6 (the bare metal version is 5). I’d like to point out that I’ve been running version 5 (virtualized) quite successfully for many years. Version 6 is a pure bare metal solution but introduces virtualizaton as part of its base install (using Xen & KVM), as well as supporting the now popular Docker containers. Going this route though,  means you limit yourself to Xen or KVM as virutaulization platforms. I’ve been experimenting with version 6 $and Xen but currently it’s still in beta.

This particular blog post will not cover version 6 of unRAID and not delve into  Xen or KVM, future blog posts might. As such, this series will cover virtualizing unRAID 5.

Virtualizaton Software?

I hope I sold you on the merits of virtualization and you probably noticed I threw some more terms at you, sorry, didn’t mean to.  Xen, KVM, Hyper-V and ESXi  are all notable contenders in the arena of virtualization platforms. The choice to use ESXi came down to ease of use and brand reputation (I hope to have a subsequent blog post comparing these different virtualization platforms in the future).

Enterprises tend to gravitate toward the paid ESXi option for performance/features – but I’m sure some may argue that point. However, you can get a free version of ESXi by registering an account at vmware’s website. To be clear, there are some hardware limitations, with the free version of ESXi, although I believe the 32 gigabytes of ram limit has been lifted on ESXi version 5.5. The free license for version 5.1 (the one I’m currently using) only supports a single physical cpu and is limited to a maximum of 32 gigabytes of ram.  Totally sufficient for my use.

Conclusion

Wow, so that was long-winded and I hope, somewhat enlightening. Ultimately the choice was made to virtualize unRAID 5 on ESXi due in large part to stability and ease of use. Thanks for stopping by!

Update: 05/01/16

As of May 2016, the great folks over at Lime technology are on the verge of releasing unRAID 6.2. UnRaid 6 brings a whole slew of neat new features to an already stable NAS platform. New features include Docker support (with a plethora of available apps), an updated plugin and yes, it’s own brand of Virtualization thanks to KVM. That’s right, unRAID need no longer be virtualized in ESXi. Instead unRAID can be run bare metal and support virtualization out of the box, great features indeed for an already great product. I highly recommend checking it out!


Leave a comment

Your email address will not be published. Required fields are marked *

3 thoughts on “Creating a NAS With ESXi and unRAID