Monday, February 4, 2013

Part 2: FreeNAS Configuration, Time Machine Backups, etc.

It's been about a month since I devised my backup strategy for my mixed network of Windows, Mac's, and other machines. You can read more about that in Part 1. In this second part, I wanted to provide an update on some of my experiences with building a FreeNAS server, how I generally configured it, and some of the issues that I ran into along the way. I will also take a few detours (as usual) along the way.

First things first: building the FreeNAS server.  There is a lot of good information "out there" and I read the documentation. This was a great starting point. In fact, just about everything I ever needed to get up and running was posted there. Once I was a little closer to my design, I posted on the forums and asked for feedback.

This was my general process for coming up with the design:

1) I created a "map" of my current devices and data storage needs, as well as any potential new needs over the next few years. I made a diagram in Microsoft Visio that helped me. I was actually a little bit surprised by how many devices I had on my network.

2) Armed with a good diagram, I was able to visualize and estimate my storage requirements and planned growth. I've accumulated almost 1 TB of personal video and photo footage over the past 5 years on my current "live" storage, which doesn't include tons of video projects that I've archived (especially raw footage). This is the "important" stuff.  With my newer camera equipment (which uses larger formats), I suspect this will exponentially grow over the next 2-3 years. So I estimated about 6 TB of working space should be the minimum.  

I also want room to expand. With FreeNAS, I can either add an additional drive array, or upgrade individual drives themselves one by one. (FreeNAS upgrades work something like this: power off, upgrade a drive, reboot; allow new drive to sync; rinse and repeat until all drives are upgraded process and then expand the volume).

3) Last but not least, I had to consider my network... will I have enough bandwidth?  I have a 1 gigabit network. However, there are possible scenarios (eg, if I am editing on my Macbook Pro while streaming Media Center to 2-3 computers) that might put stress on it. Fortunately, this wasn't an immediate concern but something I may have to look at in the future due to another project.

After doing some reading and determining my "desired" set up (Raid Z1 as I mentioned in my last update), I then started making a list of components that I would need. This was probably my favorite part of the entire project!

System Component Requirements: 

1. Memory- The machine should have enough horsepower and memory for a 6 TB setup.  Based on research, I opted to go with 16 GB of RAM, although 8 would have been sufficient. With the price of memory quite low, it was an easy decision to go for the extra. Also,FreeNAS is more dependent on RAM than anything else. I went with Corsair PC3 12800 memory (2x8gb). 

2. CPU - From what I read, FreeNAS doesn't really need much CPU. Many people get by using a Celeron or even a Pentium 4 chip. I opted for an Intel i3 2105 which had a good price/performance point at the time of purchase. I didn't want to scrimp too much on CPU in case I wanted to re-purpose the machine at some point.

3.Case & Cooling - I wasn't entirely sure where I would put my FreeNAS box. I initially thought I might want to have it out in the open near my desk, so aesthetics and quiet operation were a priority. I opted to go with a Fractal Design R4 case which was a little pricier (a little over $100) but met my criteria. It had the right balance of drive space, configurability, and most importantly cooling.  I also added an additional Zalman 120mm fan so all the hard drives would get good airflow.

4. Motherboard - FreeNAS recommends an Intel chipset. I've had pretty good luck with MSI over the years, so I decided to go with an LGA 1155 model: Z77A-G41. It is also "military grade reliability". In the forums, a lot of people also tend to go with ASUS or Gigabyte. 

5. Power Supply - After all my years of building PC's, I've learned to never scrimp on the PSU. I am also a big fan of having good access to cables. Hence, I went with a modular Corsair 750W CX750M.  Yes, quite a bit bigger than what I probably needed, but I'd rather have the room for expansion.  It came with all the SATA power connectors needed.

6. USB stick-  FreeNAS is installed on a USB stick. It doesn't run from the HDD directly. I picked up a "slim" SanDisk Cruzer. You can barely even tell it's plugged in.  Some people opt to get internal USB headers, but I didn't want to spend the extra money, and figured I may need to swap it out in case the USB fails.

7. HDD's - I went with 5 Western Digital RED 2 TB Hard Drives. The "RED" brand are a bit pricier but are designed to be on 24x7, and used specifically for NAS. 

The overall price of the system was around $1100.  By comparison a Synology DS412+ would have been at least $600 just for the cabinet, and another $500 for just 4 hard drives- as that is the max. I would have probably saved a lot of time and research going with the Synology, but I would have had more limitations and less disk space available. 

Building the system took me about several hours on a Friday night.  Murphy was "in full effect" and just about everything that could go wrong did, including the PSU not being connected to the motherboard and me pulling out my hair wondering why it wouldn't post. I also had to make a late night last minute run to a local "screwdriver shop" (Computer Discount) to pick up some SATA cables and an extra fan, as the MSI motherboard only included 2 cables and I needed 5.

Making a bootable USB and copying over the FreeNAS disk image was also a little tricky. I wasn't able to get the USB to boot when using my MacBook Pro, so I had to use the PC (and diskpart to the rescue- the sucker didn't want to format properly until after I fully wiped it).

Other than those minor issues, it was fairly easy to get up and running.  Below are a few screenshots from my configuration, taken from the FreeNAS web GUI along with some brief explanations. The general process goes something like this: create your ZFS datasets, then your ZFS volumes, and then set up your sharing (windows, AFS, etc.)  You also need to add users / groups to access.  The rest is pretty much taken care of for you- it's really quite simple to get up and running.

General Setup with FreeNAS

After booting up FreeNAS, I was immediately able to see my drives using the web interface.

Next, you need to create your ZFS data set. I created one primary data set that used up the entire drive space. It is called "BigDisk" (really creative, eh?).  With 5 2TB drives I get a little over 6 TB of usable space. That is because there is some redundancy built in when you use the ZFS Raid Z1 file system, which allows for any one disk to fail at a time (which you can then recover by replacing).  

I created several volumes under the main ZFS dataset as follows:

Media contains all of my video and photography, TMBackup is where I host my Mac "time machines", and WIP is where I keep a scratchpad for my video editing projects. 

FreeNAS also gives you the ability to set up quotas for each of your volumes, and you can easily see a snapshot of how much space is being used:

After you have your shares set up, you can easily add windows (CIFS) or Apple (AFS) shares, to name a few:

You can also set up iSCSI, FTP, and many other services. Some are easier to set up than others, but I didn't encounter any problems in my setup. One thing is for sure: there is quite a bit of capability that FreeNAS offers. Hopefully the above gives you an idea of how easy it is to get up and running!

A note on Performance: I was also hoping to get 100 MB/sec (yes, that is MegaBYTES with a capital B) over the wired network. I'm happy to say that I'm easily getting that and it's more than fast enough for me to do video editing from my Macbook Pro, straight from Final Cut Pro. The possibilities are pretty much endless. The reliability has been rock solid so far, with 0 errors and it's pretty much been up since I've turned it on with an occasional power down when I was putting in the APS.

Current State of Backup Strategy... 

Fortunately, I have been able to get just about everything working that I posted about in my last entry:  FreeNAS, "dueling" Time Machine backups (one to my Time Capsule Router, one to the FreeNAS), etc.  I was also able to get in on a New Years deal for CrashPlan, which gives me 4 years of unlimited cloud backup storage.  I've been able to back up about 600GB so far and have a ways to go, but most of my important files are now securely in the cloud.

Now onto my next project... which is kind of TBD for the time being. I've got a few things in the hopper and will probably know more in a few weeks. Check back soon!


  1. Nice write up!

    I realise that you are using Z1 (equivalent to RAID-5 I suppose?) but why is the size your bigdisk/media larger than available space but not the case for bigdisk? Also, how did you test to see that you are actually getting ~100MB/sec out of your NAS?


    1. Thanks for the comment Louis!

      Yes, the configuration that I am using is most similar to Raid-5, though there are some differences. There is a decent writeup from someone in the forums here:

      To test the performance, I used a tool on my Macbook Pro called "Blackmagic Disk Speed Test". I believe it writes out a 1GB temp file and then reads it back to do the speed check.

      Good question on the bigdisk/media share being larger. I am not really sure why that is the case, and I also wondered when I took the screen shot. One would think that the top most volume would be the full size... I'll have to look into that.