Part 2: FreeNAS Configuration, Time Machine Backups, etc.

It's been about a month since I devised my backup strategy for my mixed network of Windows, Mac's, and other machines. You can read more about that in Part 1. In this second part, I wanted to provide an update on some of my experiences with building a FreeNAS server, how I generally configured it, and some of the issues that I ran into along the way. I will also take a few detours (as usual) along the way.

First things first: building the FreeNAS server.  There is a lot of good information "out there" and I read the freenas.org documentation. This was a great starting point. In fact, just about everything I ever needed to get up and running was posted there. Once I was a little closer to my design, I posted on the forums and asked for feedback.

This was my general process for coming up with the design:

1) I created a "map" of my current devices and data storage needs, as well as any potential new needs over the next few years. I made a diagram in Microsoft Visio that helped me. I was actually a little bit surprised by how many devices I had on my network.

2) Armed with a good diagram, I was able to visualize and estimate my storage requirements and planned growth. I've accumulated almost 1 TB of personal video and photo footage over the past 5 years on my current "live" storage, which doesn't include tons of video projects that I've archived (especially raw footage). This is the "important" stuff.  With my newer camera equipment (which uses larger formats), I suspect this will exponentially grow over the next 2-3 years. So I estimated about 6 TB of working space should be the minimum.  

I also want room to expand. With FreeNAS, I can either add an additional drive array, or upgrade individual drives themselves one by one. (FreeNAS upgrades work something like this: power off, upgrade a drive, reboot; allow new drive to sync; rinse and repeat until all drives are upgraded process and then expand the volume).

3) Last but not least, I had to consider my network... will I have enough bandwidth?  I have a 1 gigabit network. However, there are possible scenarios (eg, if I am editing on my Macbook Pro while streaming Media Center to 2-3 computers) that might put stress on it. Fortunately, this wasn't an immediate concern but something I may have to look at in the future due to another project.

After doing some reading and determining my "desired" set up (Raid Z1 as I mentioned in my last update), I then started making a list of components that I would need. This was probably my favorite part of the entire project!

System Component Requirements: 

1. Memory- The machine should have enough horsepower and memory for a 6 TB setup.  Based on research, I opted to go with 16 GB of RAM, although 8 would have been sufficient. With the price of memory quite low, it was an easy decision to go for the extra. Also,FreeNAS is more dependent on RAM than anything else. I went with Corsair PC3 12800 memory (2x8gb). 

2. CPU - From what I read, FreeNAS doesn't really need much CPU. Many people get by using a Celeron or even a Pentium 4 chip. I opted for an Intel i3 2105 which had a good price/performance point at the time of purchase. I didn't want to scrimp too much on CPU in case I wanted to re-purpose the machine at some point.

3.Case & Cooling - I wasn't entirely sure where I would put my FreeNAS box. I initially thought I might want to have it out in the open near my desk, so aesthetics and quiet operation were a priority. I opted to go with a Fractal Design R4 case which was a little pricier (a little over $100) but met my criteria. It had the right balance of drive space, configurability, and most importantly cooling.  I also added an additional Zalman 120mm fan so all the hard drives would get good airflow.

4. Motherboard - FreeNAS recommends an Intel chipset. I've had pretty good luck with MSI over the years, so I decided to go with an LGA 1155 model: Z77A-G41. It is also "military grade reliability". In the forums, a lot of people also tend to go with ASUS or Gigabyte. 

5. Power Supply - After all my years of building PC's, I've learned to never scrimp on the PSU. I am also a big fan of having good access to cables. Hence, I went with a modular Corsair 750W CX750M.  Yes, quite a bit bigger than what I probably needed, but I'd rather have the room for expansion.  It came with all the SATA power connectors needed.

6. USB stick-  FreeNAS is installed on a USB stick. It doesn't run from the HDD directly. I picked up a "slim" SanDisk Cruzer. You can barely even tell it's plugged in.  Some people opt to get internal USB headers, but I didn't want to spend the extra money, and figured I may need to swap it out in case the USB fails.

7. HDD's - I went with 5 Western Digital RED 2 TB Hard Drives. The "RED" brand are a bit pricier but are designed to be on 24x7, and used specifically for NAS. 

The overall price of the system was around $1100.  By comparison a Synology DS412+ would have been at least $600 just for the cabinet, and another $500 for just 4 hard drives- as that is the max. I would have probably saved a lot of time and research going with the Synology, but I would have had more limitations and less disk space available. 

Building the system took me about several hours on a Friday night.  Murphy was "in full effect" and just about everything that could go wrong did, including the PSU not being connected to the motherboard and me pulling out my hair wondering why it wouldn't post. I also had to make a late night last minute run to a local "screwdriver shop" (Computer Discount) to pick up some SATA cables and an extra fan, as the MSI motherboard only included 2 cables and I needed 5.

Making a bootable USB and copying over the FreeNAS disk image was also a little tricky. I wasn't able to get the USB to boot when using my MacBook Pro, so I had to use the PC (and diskpart to the rescue- the sucker didn't want to format properly until after I fully wiped it).

Other than those minor issues, it was fairly easy to get up and running.  Below are a few screenshots from my configuration, taken from the FreeNAS web GUI along with some brief explanations. The general process goes something like this: create your ZFS datasets, then your ZFS volumes, and then set up your sharing (windows, AFS, etc.)  You also need to add users / groups to access.  The rest is pretty much taken care of for you- it's really quite simple to get up and running.

General Setup with FreeNAS

After booting up FreeNAS, I was immediately able to see my drives using the web interface.

Next, you need to create your ZFS data set. I created one primary data set that used up the entire drive space. It is called "BigDisk" (really creative, eh?).  With 5 2TB drives I get a little over 6 TB of usable space. That is because there is some redundancy built in when you use the ZFS Raid Z1 file system, which allows for any one disk to fail at a time (which you can then recover by replacing).  

I created several volumes under the main ZFS dataset as follows:

Media contains all of my video and photography, TMBackup is where I host my Mac "time machines", and WIP is where I keep a scratchpad for my video editing projects. 

FreeNAS also gives you the ability to set up quotas for each of your volumes, and you can easily see a snapshot of how much space is being used:

After you have your shares set up, you can easily add windows (CIFS) or Apple (AFS) shares, to name a few:

You can also set up iSCSI, FTP, and many other services. Some are easier to set up than others, but I didn't encounter any problems in my setup. One thing is for sure: there is quite a bit of capability that FreeNAS offers. Hopefully the above gives you an idea of how easy it is to get up and running!

A note on Performance: I was also hoping to get 100 MB/sec (yes, that is MegaBYTES with a capital B) over the wired network. I'm happy to say that I'm easily getting that and it's more than fast enough for me to do video editing from my Macbook Pro, straight from Final Cut Pro. The possibilities are pretty much endless. The reliability has been rock solid so far, with 0 errors and it's pretty much been up since I've turned it on with an occasional power down when I was putting in the APS.

Current State of Backup Strategy... 

Fortunately, I have been able to get just about everything working that I posted about in my last entry:  FreeNAS, "dueling" Time Machine backups (one to my Time Capsule Router, one to the FreeNAS), etc.  I was also able to get in on a New Years deal for CrashPlan, which gives me 4 years of unlimited cloud backup storage.  I've been able to back up about 600GB so far and have a ways to go, but most of my important files are now securely in the cloud.

Now onto my next project... which is kind of TBD for the time being. I've got a few things in the hopper and will probably know more in a few weeks. Check back soon!

Part 1: NAS and Backup Strategy

"Never let a good crisis go to waste."

People I work with have heard me utter that phrase many times. For instance, when that production server decided to come down, it's especially frustrating when you know there was some action you could have implemented to avoid the crisis all together: unless we do X, Y might happen.  X is usually a minor inconvenience/expense up front, almost always less than the potential Y, but X is never mandatory. Businesses go through these quite frequently, and it's no different in our personal lives.

The reality is, that in many cases, a crisis is the only way to change behavior and improve things for the better. It's funny how life works like that. 

Consider the following:

• A burglar breaks into your house and steals your valuables.
• Your basement gets flooded, your house burns down, etc. 
• Your hard drive crashes and you didn't have a recent backup.

All of the above are unpleasant situations (of varying degrees, certainly) that usually result in not-so-fun consequences. 

• "I should have installed that alarm system" ($40 / month vs. priceless things gone forever)
• "Why did I wait to replace that sump pump?"  ($500 vs. $1000's in clean up costs)
• "I wish I would have been more faithful about backing up my system" ($xx vs. priceless things gone forever)

Sadly, I speak from personal experience with the above items (with the exception of being burglarized). By nature, I am a planner as well. I like to do as much as I can to avoid a crisis. "Be Prepared" as the old Boy Scout motto says.

I've decided that 2013 needs to be different, so this year I'm trying to address a few things on the consideration list. The focus of this point forward is a follow up to a recent post: my backup strategy for 2013. And yes, I am somewhat reacting to a crisis, but fortunately that crisis was averted.

Backup Strategy

My backup strategy is fairly simple. In concept, I am going with the "defense in depth" approach. Wikipedia does a good job explaining the concept: the purpose is to "... defend a system against any particular attack using several, varying methods. It is a layering tactic, conceived by the National Security Agency (NSA) as a comprehensive approach to information and electronic security"

In the context of data, the "attacker" is not necessarily a person, but the construct still applies. My assumption is not a matter of "if" a hard drive will fail, but "when" it will fail. Since I've been using PC's with  hard drives since the late 80's, I've witnessed more than 5 hard drives physically bite it. This has been for a variety of reasons: lightning strikes, power outage/brownouts, old drives eventually giving out (ie, "click of death"), and within the last few years, a few drives to plain old crappy manufacturing / poor quality.

My strategy is thus quite simple: create many backups (>3) of stuff that is important, and have at least one of those backups off site. 

On a related note- if you're looking for some good write ups on backup strategies I recommend reading Scott Hanselman. Scott always has a lot of good tips and I've been following him for years. His articles are straight forward, to the point and make a lot of sense. Some of my tactics are derived from Scott's tips.

Tactical Approach

The strategy is pretty simple, but the tactics are where the rubber meets the road. There are SO MANY OPTIONS to choose from. Before I get into my specific plan, I'd like to give a little background into my thought process which will be in the form of recommendations.

First, think about your overall current state context. 

• Type of devices you want to back up, thinking by OS if a computer is usually helpful
• PC's
• Mac's
• Linux
• Phones
• Tablets
• Use of Data
• Small sized but critical data that you  frequently work on day-to-day and typically resides on your computer: source code (if you're a programmer), documents, spreadsheets, etc.
• Large sized local critical data (if you're a creative type)- HD video WIP projects, photoshop files, etc. 
• Stuff you want to share, such as videos, photos, etc with other devices eg Apple TV, Media Center, other PC's, etc.
• Less important but sentimental things that just need to be archived somewhere (old emails, old work files)
• Stuff that comes and goes (eg. DVR recordings from Media Center)
• Priority of Data / Data Equality
• High Priority: Stuff that is super important that you can't replace: family photos, videos, etc. 
• Medium Priority: Stuff that you could rebuild if you had to, but would prefer not to (your OS install, program installs, rips of DVD's or Blu-Rays you own, iTunes/mp3 purchases)
• Low Priority: Stuff that would be a minor headache if you lost: ISO disc images of programs, your downloads folder, DVR recording
• Current Data
• How much data have you amassed?
• What extent of this is "high priority" data that needs a defense in depth approach to backup?

A good tactical plan should cover growth for at least the next 2-3 years. You may not have all the answers to the above, but it helps thinking through it. I spent a few hours sketching out the different types of data that  I have. I spent a week pondering my options, and several days researching once I had narrowed it down. Don't make a hasty decision- your data is at stake!

Now, on to the implementation.

As you might expect, one could spend a small fortune on implementing a backup strategy. Many corporations do this! However, most of us do not have the deep pockets of a corporation. I'm glad to say that there are still some good options out there for just about any budget.

I've already invested a lot into professional camera equipment. It is a passion, hobby, and much needed creative outlet- I film weddings, church events, kids sporting events, dances, and home videos to name a few. If I had set aside even half the money I've spent on my camera equipment I'd be able to rival the backup plans of most small businesses. However, I didn't do that- refer back to my opening remarks. :) Fortunately, my implementation will allow me to grow and adjust over time.

My overall implementation will look like this:

• NAS (NOT a backup, but has redundant storage)
• Choice of Nas: FreeNAS  
• an Open Source software built on FreeBSD, for my SAN.
• FreeNAS will allow me to use ZFS, which is a software based RAID. 
• I will be using a RaidZ1 setup with 5 2 TB drives. RaidZ1 is very similar to Raid 5, meaning I will have good performance and good reliability. If one drive goes bad, I won't lose my data. (RaidZ2 is even better but higher cost / lower performance- two drives can go bad and you won't suffer any data loss)
• The NAS will:
• Store backups of all priority data across my network (with the source machines having my primary copy)
• Store copies of backups for my Mac's 
• Will serve as a work in process location for my video editing
• Will be the primary storage used for video sharing
• Should have steady performance of around 100 MB/sec (megabytes) read and write, with expansion up to 200-250 MB/sec using Link Aggregation
• Side note: I need high performance because of my WIP video editing needs.
• Mirrored 1 TB "poor man's backups" 
• I will continue to copy the high priority stuff including photos, raw videos, finished videos to this drive.
• Goal: discipline myself to copy my final videos to my dual backup until I automate my workflow to backup the high priority stuff from NAS.
• In 3-6 months I will upgrade my mirrored 1 TB drives, send one of them to my parents (for off site backup) and upgrade to dual 3 TB drives.
• Time Machine on Time Capsule
• I will continue to use my Time Machine for backups of the Mac's we have
• It is a 1.5 TB drive. It keeps incremental history and I can go back quite a ways
• Since there is no disk redundancy, I want to back this up to the NAS as per above in case I lose it.
• Blu-Ray
• For archiving the high priority stuff
• CrashPlan Cloud Storage
• Because fires and floods happen, off site backup is required.
• For about $120 or so a year, I will backup my high priority files to the cloud (active backup)
• ALL data is encrypted
• There is an option to "seed' the backup with 1TB of data for additional cost, otherwise it may take several weeks or even months to backup all my data.
• If I want to recover, I can pay a fee to get 1TB of data sent to me.
• If my family / friends sign up, I can store backups on their machines (based on what space they make available) if I so desire. I believe this is also encrypted. Another cool option for redundancy. I would gladly open up some space on my SAN for other friends / family willing to trade space.
• SmugMug (also Cloud storage and paid site for photos/final videos)
• I've been using this site for a little over a year. It's great for photo sharing and even videos, though there is a 20 minute / 8 GB limit on videos.
• I don't put everything on this site, mostly stuff I want to share with others.  
• Allows me to download all my photos in one zip if I ever need them.
• Unlimited video / photo storage.
• It's about $150 a year but price seems to go up frequently. 
• A "Restore" Roadmap
• On a related note,  my wife and I are going to set up a Will / Trust this year. It's not fun to think about, but you need to consider that someone else many need to recover the data. 
• With all the trouble I'm going through to set this up, it would be a tragic waste if I didn't consider this.
• APS UPS Power Supplies
• In the event of a power outage:
• I want to make sure my networking components, including Time Capsule is available.
• I will use a APC 450 VA for this. I should have about 30 minutes of availability, which should allow enough time for things to shut down properly.
•  My NAS needs to be on a UPS.
• I will use an APC 750 for this.  Since the NAS will be headless, I anticipate about 30 minutes backup. FreeNAS can be configured to auto-shut down if the UPS signals that the power is out after X amount if time. I will be using this feature.

Why FreeNAS? Many reasons, but to summarize:

• I looked at Synology, QNap, Buffalo and many other vendors. 
• While the idea of just buying it and getting done with it appealed to me, the value proposition wasn't quite there. FreeNAS offers more features than most out-of-the-box. 
• In my research, I also came across many forum posts (and even Amazon reviews) who had gone with a vendor device and ultimately decided to go FreeNAS because they had issues.
• Performance also seemed limited (less than 100 MB/ sec) due to the crappy system specs in these units (ATOM processors, 512 GB Memory). Some people upgraded them and saw improvements, but lost the warranty.
• Software Raid (ZFS) was designed with data integrity from the ground up. In most cases it's just plain better than hardware RAID. Much is written on the subject. 
• RaidZ1 allows me to avoid the "Raid 5 write hole" while having performance and redundancy of traditional hardware Raid 5. (This basically means I should avoid corruption due to a power loss)
• Overall price vs. other vendors seems to be slightly better especially when considering future growth.
• There will be some learning curve, but the folks on the FreeNAS forums seem very helpful so far.
• I really enjoy tweaking and computing is still a hobby for me. While some may shy away, this is an exciting challenge for me and I look forward to tackling it.

In Part 2 I will cover the specific hardware and installation of the FreeNAS. Stay tuned!

Adventures with SSD's, Hard Drives and Backups (and Happy New Year!)

The holidays were quite busy in 2012. I took a bit of a break from my blog accordingly. Here we are in 2013 now, it's hard to believe! Happy New Year!

I last blogged about some key upgrades that I made to my 2011 17" MacBook Pro- including a 512MB OCZ Vertex 4 SSD and 16 GB of Ram. While my system drastically improved in terms of performance, the reliability factor seemingly dropped. Prior to the upgrade I can remember running "uptime" (terminal command) and seeing a time of 70 days- meaning it had been that long since my system had been rebooted or shut down. That is some rock solid reliability! Post upgrade, I was getting occasional lockups and system freezes primarily when browsing.  I was lucky to get 3-4 days of up time without a freeze.  Not so cool. Disclaimer: I upgraded the components myself and did not take them to the "Mac Cafe" (local Apple screwdriver shop that is an authorized Apple dealer). I do have Apple Care but from what I understand, you don't void your warranty if you do these simple upgrades yourself (so long as you can put the original stuff back in if you have any issues).

Anyway... the week before Christmas, everything came to a head. My SSD decided to bite the dust after apparently one too many crash.  Timing couldn't have been much worse, as I had a wedding that I was going to be shooting the Saturday before Christmas (Dec 22).

Other than the occasional freezes, the symptoms of my machine leading up to the crash came on suddenly and without any real warning. I restarted my MBP and I couldn't get past the grey screen with a progress bar. In fact, the MBP would shut down after about 30 seconds. I quickly went to Apple's support site and found out that this issue indicated a bad hard drive. A lot was going through my mind at that point:

• What all am I going to lose?  
• Will my backup will really work? 
• What about all my videos and photos that only live on my MBP that I hadn't yet backed up...

Then I quickly got a little frustrated:

• This drive is only 3 months old!
• I'm never buying another OCZ product again...
• WHY ME?? WHY NOW??

My mind was racing. I was subconsciously calculating the gigabytes of space of HD videos and Photos that I had done since the switch over from my previous Seagate Momentus XT drive. In my panic, I had forgotten that the majority of my raw video footage (at least) was stored the other HDD that I put in my Superdrive bay.

Then I remembered back to similar crash a few years ago. My 500 GB Seagate Baracudda drive "gave up the ghost"- which also happened to be the exact day of our wedding anniversary (Sept. 4th). The drive itself was less than 6 months old. Apparently the batch of HDD's that mine came from had an issue with Teflon coating coming off the heads of the drive, which eventually rendered the drive useless. I had several hundred GB of home video footage on that drive. I lost footage from some really cool video short movies that I made for a church youth group lock-in. Fortunately I still had all my mini-dv cassette tapes, so I was able to re-import those (which took about 3 full days!).  I could have used a hdd recovery service but they wanted almost $1500... just not quite worth it for what I lost.

Fortunately, I (somewhat) learned from that mistake and purchased 2 1 TB external Seagate drives and started a poor-man's backup with some custom written batch files. I started backing up all my photos and videos to this drive and still am using this solution today. But that solution was mainly intended for my PC, and not my MBP...

Fast forward back to my current dilemma.  I decided to immediately pull the SSD from the MBP and test it externally to see if it was the issue, or if my MBP had another problem. Fortunately I still had my previous HDD laying around in an enclosure. I popped the old drive out, set it aside, and then placed the SSD drive into said enclosure. I connected the drive from my Windows 7 machine and no dice. I also tried connecting the SSD with a USB to Sata cable... nothing. The SSD was definitely kaput.

The truth settled in. My SSD was *toast*. I might have lost my videos and/or photos. I might have lost programs and applications that I had installed.Final Cut projects of some WIP videos.  Ugh... I didn't know for sure that I was going to be able to get my data back. HOW COULD I LET THIS HAPPEN AGAIN? Well, maybe it wouldn't be so bad- after all I was using backups, but wasn't sure how reliable they would be. No time like the present to find out!

I quickly googled to see how to restore a system from my Time Machine backup, which was my best shot.  My "plan B" would be to utilize my HDD from prior to upgrading. My "plan C" would be to utilize my backup prior to installing that drive, which would have been basically how I ordered the machine. I found an article on Apple that was very straight forward. However, there was a slim possibility that I wouldn't be able to restore my system if I had chosen to exclude system files.  I didn't remember what I had excluded at that time.

I popped in my previous Seagate Momentus XT drive in the main HDD slot. Viola- success! No more grey screen with progress bar. That confirmed that the SSD was the issue, and deal with that later. Of primary importance was getting my system back.

Per the Apple instructions, I held down Command-R during boot. It found my Time Machine backup, which was stored on my Time Capsule. I quickly came to another dilemma because I only had this drive on hand  (which also contained my system prior to upgrading to the SSD). I would need to "have faith" in the Time Machine backup or settle for my system state as it was when I upgraded several months prior.  In retrospect, I made a pretty hasty decision. I probably should have went to Best Buy and purchased another Seagate Momentus XT ($120 or so). Nevertheless, I decided to "have faith" and wiped my backup drive in order to restore.

The time machine initially indicated it would take about 6 hours to restore over the network.  Luckily, I have a 1 gigabit network. Overall there was 400+ GB that needed restored.  I was a Nervous Nelly for the next few hours as the process went on. After about 2 hrs, the time machine restore was complete. I was immediately concerned, thinking that perhaps not all my data was there. From 6 hrs down to 2? Why? The skeptic in me immediately thought not all my data was going to be there.

After the Time Machine completed the restore, the MBP booted up.  The first thing I noticed was the spinning wheel- which I hadn't seen since my SSD upgrade. I did a few checks and immediately saw an application that I had recently installed on my task bar, so I knew that my backup was current. I then went to check my video folder.  Everything was there! That's when I remembered that my videos were on a separate disk, so there was really nothing much to worry about.  Next was checking iPhoto.  When I launched the application I was a bit concerned. Apparently, the thumbnails do not get backed up, because it had to generate thumbnails for all my "events".  I cancelled out of that and went to browse my events... Fortunately, everything was there as well.  The "last backup" time on my MBP was visible in the top menu bar, and it read about 1 hr before it crashed.  CRISIS AVERTED!

As you can see, there was a lot of drama in my SSD fiasco. I'm not a big fan of drama, especially when it comes to my own data. While I am very thankful for my Time Machine backup, I still feel a little exposed. I have countless hours of video footage and photographs that I do not want to lose.  After tweeting about some of my issues, I began exploring better backup options. @kpett pointed me to a nice article from @shanselman. I like his philosophy: make 3 copies of anything you care about, use 2 different formats, and have an offsite backup.  I am going to adopt a very similar strategy.

An old but wise SysAdmin once told me- "Never let a good Crisis go to waste". I don't enjoy invoking this mantra in most situations, as I'd rather avoid the crisis all together if possible. But there is some truth to it: a crisis is often the catalyst that can lead to a change in behavior.

Consequently, I'd like to end on this note: I have decided to make a New Years Resolution. I'm going to have a solid backup strategy for 2013. In my next entry, I'll share my massive overhaul plan (currently in process) for my new backup strategy.

A few other comments to tie up some lose ends: I still experienced system freezes after replacing the SSD with the old HDD.  I decided to run some memory tests. Everything checked out. I came across a few articles that suggested  swapping the memory slots. Since I've done this, I'm going on 5 days with no freeze ups. That doesn't really make a whole lot of sense, but it seems to have corrected the issue- pretty much a new record in uptime since my upgrades.

As far as the SSD goes, I shipped the drive out on Dec 19th. I'm still waiting for the RMA. My machine is painfully slow again, but I'm still able to be productive. Cash flow is a little tight right now, but I may purchase another 512 GB SSD to have on hand (probably from another manufacturer) in case I have another situation.  The SSD has a 5 year warranty on it.

Adventures in Upgrades

On Apple Upgrades & Refreshes...

I've learned a few things about owning Apple products.  The first and foremost is that you will likely be sinking some cash into them, especially if you want to stay "current". I'm stating the obvious and it's no surprise to anyone, and is an often cited reason as to why one should stick with what the other guys offer. Yes, I am learning by experience. Jason Follas (and most of my Microsoft friends and colleagues) warned me ahead of time, and even made fun of me for going with an iPhone and told me I'd regret it! But I am still glad I took the plunge, so Nyah! But.. while I may be out of some cash, I am also a bit wiser, and certainly not a rampant fanboi. And the reality is that the "other guys" aren't much better off, and are starting to borrow the same tricks from Apple anyway.

Even so, when I contemplate any future Apple purchase, I think about a few things:

1) Is it really worth the extra $$ to stay with Apple (better design, features, etc)?

2) Is Apple still the best option for what I am doing (will a hammer get the job done)?

3) Am I going to get really irritated when a new Apple product comes out and feel like I have to abandon what I have to get the latest thing? Especially in light of compatibility with other devices in the eco-system (eg Apple TV)?

When it comes to the more mainstream consumer devices like iPads and iPhones, I now get a little bothered. When I purchased an "original" iPad, I was quickly told by Apple that I needed to get the latest and greatest one. That message is now being reinforced by the things that I can't do with it (such as share it on my Apple TV).  Apple has stopped rolling out IOS updates for it. My iPhone 4 doesn't get Siri (not that I'd use it anyway, but it's the thought that counts), and after a year of heavy use at work, it requires an external battery case because I can't replace the internal battery like I have with my Android personal phone. And don't even get me started on the new connector for the iPhone 5...  I'm quickly concluding that the sweet spot for hanging on to Apple gadget technology is maybe 1 year, or whenever the "next one" comes out.  This is particularly true on the gadget end of the spectrum (phones, tablets, etc).  I guess it wouldn't be a stretch to say I find the rapid refresh cycle a bit irritating.

However, my rant not withstanding, one should also consider the alternatives. Google gives no incentive for devices to be upgraded.  Need a new OS? Wait for your hardware provider (and possibly carrier) to make it available. That's assuming it will even be made available... most likely not.  Blackberry is no longer relevant. What is left? Microsoft- and I think the market is big enough for them to finally pay attention and get a good product out. Windows Phone 7/7.5 was a flop (less than 1% market share?), but I think Windows Phone 8 may stand a chance (the old saying goes- never buy the first version of a Microsoft product).

Let's get back to the upgrade topic.  Fortunately, the refresh fares a little better for laptops. I know several people that are using machines 3-4 years old and still quite happy with them. However, similar to the gadget end of the market, Apple TV integration is not available on any Macbooks older than the 2011 models. There really isn't any technical reason for this that I'm aware of. Apple takes every chance it can get to "encourage" one to upgrade.

Despite this planned obsolescence, I keep pumping cash into my Macbook Pro because I think it is worth paying a premium to have a good experience for video, photo, etc and a machine that is very solid. Yes, that is worth the price of two mediocre PC laptops. I've had absolutely no problems with my hardware to date, while my wife's old ASUS with Windows 7 has been relegated to my daughter's gaming rig because of malware, virii, and blue screens that lead me to to reformat twice (and, it won't upgrade to Windows 8- it locks up during the install, consistently).  My MBP has never experienced a lock up or "gray screen" (similar to blue screen on PC), especially while I'm in the middle of editing video footage.  In other words, the answers to my first 2 questions above continue to be "Yes" in the context of my laptop. The answer to the third question is a tad bit trickier.  When it comes to the MBP, I would still say that I am happy with what I've got even with the latest models- and a big reason for that is because Apple chose not to release a 17" model.  The retina screens on the latest MBP models are really slick, but I don't feel that it's worth trading the 17" screen for. It also comforts me knowing that if and when I do decide to go to the newest, I should be able to get some nice residual value for my current machine. A colleague has a 3 year old 13" machine and is considering selling it- appears to be going for ~$900 on the used market (I think he paid $1500 for it).

MBP Upgrade

I recently decided to go the SSD route. A big reason for the upgrade was because I happened to win a sizable gift card through Ben's Bargains, which is a deals web site that I frequent. The reasoning for the SSD was mainly due to my video hobby. Video files can get extremely large (and thus time consuming when transferring around), so upgrading to an SSD should have some great impact in terms of getting a faster read/write.  How much faster?  For me, my Seagate Momentus SSD hard drive was getting about 100 MB/s.  (On a side note, the Seagate was the first "upgrade" that I did, as the original drive included with the Apple was around 50 MB/s.  Disk is BY FAR the slowest part of the system, and anything you can do to increase it will not go unnoticed.) The new SSD that I purchased is the 512MB OCZ Vertex 4.  It has a 5 year warranty and claims to have one of the fastest IOPS rates of drives in the market today. It even came with a sticker that said "My ssd is faster than your hdd" so it must be great! :)  In all seriousness, it blew my socks off- I am now getting over 400 MB (yes, that's MB) / second.  That's a 4x increase. Final Cut now opens in less than 5 seconds, whereas before I could grab a cup of coffee, come back, and still see the plugins loading up. Word, Excel, Powerpoint etc all load instantly-- I can click each icon on the doc, and they load as fast as I click.

In addition to the SSD, I put an order in for 16 GB of RAM this past weekend. It wasn't so much because I needed it, it was more since my wife's MBP was choking on 4GB.  Instead of buying 8GB for her, I thought I should spend the additional $30 ($80 total...) to upgrade mine to 16GB, and move my 8GB to her machine. The only reason her machine started choking on the 4GB was because I decided to install Sophos on it.  October is cyber-security month and our IT department recommends anti-virus software even for the Mac's, and the only free one I could find worth it's salt was Sophos.She went from 1.5 GB free to less than 50MB free after installing- what a hog!  To date we've not had any issues with virii, but Mac's are becoming more of a target- I think 20% of PC's sold today are Mac's now. Better safe than sorry.

Notes on the Upgrade Cycles

In conclusion, I am a little wiser than I was a few years ago. I started my adventure blog about a year ago and I am still happy about making the switch, at least for multimedia use. In my mind, the upgrade cycle it's a lot like flying first class:

iPad, iPhone = domestic first class. Nice, but not going to do it every flight (upgrade).
Macbook Pro = international first / business class.  Very nice, very expensive, fortunately I don't have to fly International all that often.

I'll end this Mac Adventure on a final note that might make the late, great Steve Jobs a bit unhappy:  I'm willing to give Microsoft another try, at least for my personal phone. And even if Windows 8 is dead before it arrives, I'll at least be able to play my Zune Pass music on something other than my Zune HD :)

The Canon XF100 Plunge

A while back I posted an entry about some upcoming multi-media projects. I wanted to provide a follow up. I shot my last wedding on my JVC GZ-HD7 this past June.  After a few weeks of editing in post (using Final Cut Pro 7) I was pretty happy with the final product and handed it over to the bride, groom and their respsective family. There were some things that I wish could have been better- namely the lighting and audio levels of the actual wedding ceremony. The JVC camera does not have an audio level setting, and the low light has always been an issue.  As I previously mentioned, the camera also decided to blow out a couple of pixels- which is also a known JVC issue. After a little magic in FCP things got cleaned up as good as I could get them.  The important thing is that the bride and groom were happy- and that's what ultimately matters!

A lot of professionals will tell you that it's not the tools that get in your way- and that you can do a lot with them before you buy new equipment.  While that is certainly true (and disclaimer: I am not a professional, but I play one on TV) my experience dictates that gear can help make a decent photographer / videographer a much better one. For the past few years, I was convinced that I just needed to learn how to handle my camera better and that I'd get better results. There was a lot that I had to ramp up on, especially around manual settings, composition, lighting, etc.  But once you are armed with the basics and have a decent base of experience to pull from, a great tool will help you get much better results.  In my experience, both my Digital Rebel and JVC camera were limiting factors. I needed a higher ISO to get usable pictures for indoor photography (which the 7D greatly helped with- 3200 iso with no apparent noise on the 7D, vs. 800 iso on the Rebel with plenty of noise).  Once I did this, my pictures had the professional look I was trying to get indoors. Similarly, the JVC had the aforementioned problems and caused a lot of extra work.

So in early August, I finally decided to pull the trigger and order the Canon XF100.  This is the closest thing to a "professional" level video camera that I've ever owned. From a professional perspective it is considered lower end, but for someone like me who does this semi-pro/ part time it is a huge improvement and offers great functionality that you just can't get in a consumer grade camera. The first day that I got it home and played with it I was able to shoot video like I never had before.  The focus ring was very intuitive and I was able to easily change focus close up / background.  The JVC had this capability but it was difficult to use effectively. Conversely, the lcd screen on the Canon XF100 was good enough to see the focus and manipulate it.  You can view my first shots on my smugmug site.  Perhaps the best thing about this camera is the 4:2:2 color space / chroma sub sampling.  This is a huge pro feature-- and as far as I know, isn't available on consumer cameras. What this means is that the original uncompressed video signal is marginally reduced when it is stored- meaning there is little to no visual difference.  For a good technical explanation, check wikipedia.  Many consumer grade cameras (and even many semi-pro)  are using 4:2:0 which is 1/2 the horizontal resolution, 1/2 the vertical resolution of the original signal. It also tends to result in more artifacts.  4:4:4 is the "ultimate" subsampling, because it's the original signal.  I believe most of the 4:4:4 cameras start around $10,000 and are typically much more (such as the RED Scarlet X). Technical features aside- after using this camera for a few months I feel that it's been worth the investment and have been much happier with the end result.

In the spirit of Mac Adventures, I'd also like to talk about my experience with it on the Mac so far. It has been a good camera and works very well with Final Cut Pro 7.0.3. I've managed a few HD projects so far- including one that was about 25 minutes of footage (our pastor's 25th anniversary). Things went really well and, apart from a few odd things (such as an mp3 having some audio clipping in one spot that I just couldn't get rid of) I was happy.  I blame the issues on my upgrade to OS X Mountain Lion mid-way through the project.  On a related note, I learned that Canon doesn't officially support the XF Utility on Mountain Lion- which is the tool that imports the video from the CF card to your computer. They have been slow in rolling out an updated version, which is due in October. Fortunately, since I did an upgrade and already had the CF utility installed, things still work fine for me.  However, if I were to try and do a fresh install I'd be out of luck, since the software installer doesn't let you continue if you are running Mountain Lion.

I'm looking forward to getting a lot of mileage out of the XF 100. I've got a lot of soccer footage that I'm pulling together of my oldest daughter and her team- plan on surprising everyone at the end of the season with a video. This camera really pushes the limits on my MBP- so I've got some "future upgrades" planned, which I'll be blogging about very soon.

OSX Mountain Lion Upgrade Adventure

If you are closely following Apple, you've certainly heard about the latest OS update released this past week on July 25th - Mountain Lion.  The upgrade was only $20 and there are a lot of new features. I upgraded last evening and wanted to talk a little bit about my experience...

First, I'll give my perspective on the new stuff and reasoning for the upgrade. One of my favorite features is the ability to mirror your Mac to an Apple TV device.  The apple TV's are around $100 or so at most stores. The idea behind Apple TV is that you can easily share videos and multi-media, rent videos from the Apple Store (for a pretty reasonable price), stream Netflix, etc. 

I've been using an Xbox 360 to do this currently and it has done a great job.  However, every time I turn on my Xbox 360 I fear the "red lights of death". I've already had that happen to me with 2 other Xbox consoles (one of which was still under warranty).  Another issue with the Xbox is that you have to pay for your membership just to get on line and use apps like Netflix, which also have an additional cost.  The benefit is that I can use my Zune Pass service for all the music, which gives me basically any song I care to listen to for $15 / month (as well as 10 to keep "permanently").

Now, one of the cool features about Mountain Lion is that with an Apple TV (and a Mac 2011 or newer) you can mirror anything on your screen to your Laptop. This eliminates a lot of the fus with trying to locate media content on a separate device as in the case of the Xbox (it's a real annoyance to try and find what I'm looking for, especially with the amount of content I have).  On the Mac, I can use the familiar navigation of the file system and easily get to the content I want.  I know of other people that put a Windows Media PC in their living room to do something similar on the Microsoft platform, so this isn't necessarily ground breaking... But the nice thing with the Mac is that you can do it without all the clutter and wires that the PC would require.  Nay sayers might argue that you could have a similar experience on Windows using the Media Center, but I find that interface a little clunky / irritating (see point above about file system navigation).

Another great point is that you could use subscription services like Hulu for free, and play that content on your TV.  In the case of Xbox, iPad, or other devices, Hulu charges a monthly fee to use them.  This makes it even more tempting to "unplug" from traditional cable TV and get only the content you want. I'm not quite there yet, but I can see that happening in the next 3-5 years based on all the new content services out there.

There are many other mountain lion features that I think are beneficial, especially since I have an iPad and iPhone.  There is tighter integration and Apple touts "you can go from one device to the other".  For example, if you are browsing on your iPhone, the "Cloud" will allow you to resume on your Mac.  It also claims to make sharing content/links/etc a bit easier with social media sites.  These features are nice but I don't see much value personally.

In the end, I decided to upgrade because I want to be up to date on the latest version.  It was only $20 and that is a one time charge. I can also put it on my wife's Mac for no additional cost.  Given the "200"+ features (many of which I probably won't use), being current, Air Play, and the low cost... it was an easy decision.  Especially when you compare to Microsoft, who charges an arm and a leg for an upgrade (and per PC at that), this was a no brainer. Hopefully Microsoft will take a page out of Apple's book and maybe even turn themselves back to profitability in the near future.  But I am guessing Windows 8 will go for a pretty penny by comparison (which I will probably also do on my PC).

Now for the adventure...  I was harkened back to my PC days a little bit on this particular upgrade.  First I purchased and downloaded it through the App Store.  That was no problem.  Once it downloaded, it asked me which drive to install it on. I was a little puzzled because the last time I did an upgrade, it just went on my main drive.  I mistakingly picked the wrong hard drive because I couldn't remember what the name of my boot drive was.  After the upgrad started processing I realized "woops, that's my video drive, I don't want to install it there" and turned off the Mac to my better judgement. It rebooted and said it could recover the install... but I didn't want to do that. I did have a moment of panic but was ability to use disk utility to set my original main drive as the active boot. I breathed a sigh of relief and cleaned off the install files from my video drive, and then tried again. (Had to re-download, which is slightly over 4gb for the install).

This time I paid closer attention and tried to tell the installer to upgrade the main drive that had Lion installed on it. I got an error something along the lines of "This is a Time Machine Backup drive".  I was a little confused... After about 20 minutes of googling the error, I came across an article that indicated that the installer will not let you upgrade the drive if there is an active time machine backup going on.  I then explored the backup folder and did indeed find that there was a time machine backup that I had started back in September of last year (not long after I got my Mac). I had actually aborted that back up but the files were still hanging around.  The "fix" was to delete the backup.backups folder where the in-process backup was.  I shelled out to terminal and tried doing an rm -rf (a unix command to delete the files) but it told me the files were locked by the system and I couldn't delete them.  I did some more googling and found that I needed to use the tmutil to delete them.  Success!  This was not a very pleasant experience, and it hearkened me back to the PC days. However, in the grand scheme of things it was pretty easy to clean up.  I really don't know why Apple would prevent you from installing if a time machine backup was still in process. I guess they figure better safe than sorry.

After that, things were quite simple. The installer defaulted to my boot drive and I was able to finish up.  It took roughly 30 minutes.  The "last minute" was the longest, taking probably at least 5. After that it rebooted and my wall paper changed to the new and updated Milky Way galaxy image. Had it not been for that, I wouldn't have even noticed that I upgraded.  I later found the notification icon up in the top menu.  Everything else seems to be about the same, so far.

So I'm pretty excited about OSX Mountain Lion, but it doesn't seem to be a major change so far. I would encourage anyone wanting to upgrade to do it just so you are up to date, and especially if you are considering an Apple TV.  I haven't yet purchased the Apple TV but I plan on doing so once my budget recoups from another recent project which I plan on blogging about next.

As always, thanks for reading!

Adventures over the Spring and Summer

Time for another Mac Adventure! It's been a few months and sorry I've slipped on my updates lately. I've had a few things going on that have been higher on the priority scale... Spring soccer with the oldest, busy work schedule, and vacation!  Those things are out of the way now and the summer is on its way in. Granted, June is always a busy month- two birthdays to celebrate, father's day, etc.

Now the Mac Adventure! Forewarning: this is a pretty long post. The content of this section is video / media related where I will be talking about video cameras, dslr's, and software.  Skip down several paragraphs to read about general mac stuff and my opinion on some of the WWDC 2012 news from Apple.

Recently, I filmed a wedding of some friends at our Church. I am now in the process of doing some video editing with Final Cut Pro.  I've known the couple for quite some time (since they were as tall as my waist anyway...) so I wanted to do something "extra special" for them.  I took a little more video footage than normal, including the bridal shower (which normally I do not go to!) as well as the rehearsal.  Got some great footage and also used two different cameras to collect it.

So far there have been a few challenges. Perhaps the biggest one has been with the low light capability on my cameras. I have been a little spoiled lately with my Canon 7D (though for pictures, not using it for run-and-gun video situations).  It has extraordinary capability and can take virtually noiseless pictures at 3200 ISO.  This means that low light, especially when shooting with an f 2.8 or wider aperture is not a problem (I got some great shots from a dance recital end of April).  At any rate, video cameras are not always so great. I usually shoot with my JVC GZ-HD7 which is now a pretty dated camera- but it was one of the first to do "Full" tapeless High Definition video. It has a 3 CCD sensor so the image quality is very good.. but not so in low light.  The auto-gain (in layman's terms, gain is similar to ISO on a camcorder) is very bad and typically saturates the picture with a reddish hue. I tend to shoot video without the gain on because it just doesn't look right, and it's very difficult to correct in post production.

So... I decided to shoot some footage with my "b-cam" (if you can even call it that), which happens to be a Kodak Playsport. The image quality is surprisingly good, and acceptable for something informal like a rehearsal dinner.  Unfortunately, the "scene" during dinner was extremely low light and I ended up with a lot of noise coming out of the Kodak.  The noise was so bad I thought I was going to have unusable footage, but I found a product called "NeatVideo" which has a great noise reduction filter and plugs in to different video camera software including FCP 7.  The filter did a great job and at some point I'll post a before/after (but you can see good examples of this on the site as well).  Problem #1 was solved, and now I'm currently on my way to dealing with Problem #2.

Problem #2 is with my JVC GZ-HD7 video camera. Unfortunately, this problem is due to a design flaw, and exacerbated by age. While my camera was cutting edge when I purchased it 5 years ago, it's getting older and the problem is becoming more pronounced. After the camera is going for a while, the CCD gets pretty hot and eventually, the "dreaded white dot" appears on the picture.  This wasn't so bad a few years ago when it was just one dot and came up after 2-3 hours of use.  However, after reviewing my latest footage, there is more than a few white dots, and one of them happens to be right smack in the middle of the picture!   Fortunately, there is a video filter out there that allows you to replace bad pixels by interpolating the ones bordering them.  Needless to say I am experimenting with this filter.  I found a great article and a sample here, using FXScript.

Given my issues with my video cameras, I think it is finally time for me to replace my trusty JVC. I've been researching the Canon XF100 since October of last year and I think I will be going with it. Apparently, the BBC has it on it's list as an approved camera for doing broadcast quality video. Slashcam (where I frequent) rates it very high, it seems to do very well in low lighting... etc. I think it will be the perfect camera for me.  There is also a JVC successor (the HM 150), but it appears to have the same "dreaded white dot" issue.  I frequently visit and post on dvinfo.net which is a GREAT resource for anyone that works with video a lot- coupled with slashcam, it's kind of like the video camera version of dpreview.

So... a lot of multi media work going on right now.  I haven't even touched on some of the other activities I'm planning, but I'll go ahead and list them here:

- Hungary Missions Trip - video and pictures, coming up in July
- 3 on 3 Soccer travel team
- Someone on the soccer team requested me to take pictures of a few softball teams they coach
- Cheerleading Camp
- Tumbling classes and associated performances
- Misc. stuff this summer (birthday parties, pool parties, etc) 
- Midwest Geobash (assuming we make it)

Shaping up to be a lot of work!! But looking forward to it.

WWDC 2012 News & Update.
Unless you've been living under a rock lately, you probably heard some of the recent Mac announcements.  Here are some of the ones that jumped out at me:

• Retina displays for the new Macbook Pro's! Usb 3.0!   Glad to see this new direction, but I'm certainly not going out and buying a new one any time soon. Especially since the new 15" models are almost the price of the former 17" models. Not enough value there yet.
• No more 17" MBP's (though I'm not sure if that will stay the case) - Makes me happy that I picked the last model year up.
• Apparently, no update this year for the Mac Pro or desktop stuff in general, but supposedly that will come next year - no big surprise to me, as this stuff is more targeted for niche professionals and Apple is probably not making a huge amount on this market segment.
• Mountain Lion will be $19.99 and released in July. It includes new cloud features, etc and AirPlay mirroring and may finally get me to purchase an Apple TV.  That means I'll be able to watch full screen Netflix videos without getting Mac-burn on my knees, and/or not having to power up the Xbox 360!

In general I think the news has been good. It looks like CEO Tim Cook is doing a fair job so far.  I think time will tell if Apple will keep the "cutting edge" reputation. I'm sure there are quite a few articles out there that are analyzing his every step and playing the "WWSJD" angle (what would Steve Jobs Do).

With all that news, I think I am planning for a few new Mac Adventures this year- some are still on the plate from last when I first got my Mac.

• Eventually get a thunderbolt SSD drive. I'd like something for when I'm working on videos. This will help immensely when doing the rendering in FCP.
• Replace my seagate hybrid drive with a full SSD internal drive. Prices are really starting to come down now. Once I can get a 512GB SSD for around $300 I may do so. As I write this, they are still around $400 but dropping fast.
• Reconsider "Zune" and embrace iTunes.  Zune just doesn't play well on the Mac platform, and I've noticed the selection on the Zune Pass is getting worse and worse. I may dump it for iTunes.  On the other hand, I like where Windows Phone is headed (I might consider a Win Phone 8 when they come out).

Well that is about it for this post. Hope you've enjoyed it, and I plan on being a little more regular with my postings this summer. I think I'll have a lot to blog about.