Login  |  Register
Videoguys News Blog Twitter Videoguys News Blog Facebook Videoguys News Blog Yaoo Videoguys News Blog RSS Home Research Support About Us

aja_234x60.jpg sony-320x60.jpg ADVCG.jpg

The Videoguys' Blog is our way of instantly communicating with you. We'll post articles and reviews that we find interesting from all over the web as well as new product information and promotions.

All posts are syndicated via RSS so you can set-up the Videoguys' Blog in your favorite RSS reader and/or e-mail program. You can also follow our blog by becoming a fan of Videoguys on Facebook or following Videoguys on Twitter!

Sept
13
Videoguys NLE Video Storage FAQ
by: 
09/13/2011 09:39 AM

One of the most confusing things about digital video editing is the storage requirement. In this article the Videoguys Techs will help you better understand just what kind of storage you'll need for your video productions. Mac or PC based video editors face the same storage challenges and questions, and we've got the straight forward answers you need!

Top 5 Storage Rules
You have a lot of choices for drives to store your digital video files. A stand-alone SATA hard drive is fast enough for editing Standard Definition (SD) footage using DV or MPEG2 compression. But, if you're working with uncompressed SD footage or HD footage, we recommend more robust storage solutions. Before we get into our FAQ, we'd like to review Videoguys' Five Rules of Video Storage:

2007best-1.gifYou can never have too much storage.

DV compressed video requires 13GB per hour of footage, ProRes 422 requires 66GB per hour. While this may not seem like a big deal to you today, it sure was not so many years ago. Back in 1998 a 9GB SCSI drive would cost you over $1500!! And if you wanted to create great looking video, you had no choice but to invest that kind of money. Today's SATA drives are faster, more reliable and most importantly offer far greater capacities at a fraction of the cost. You can find 1TB drives (1,000 GB) for under $100 online.

2007best-2.gifIt's the throughput baby!

Seek times and peak transfer rates mean nothing for video production. All we care about is sustained throughput. We don't care about the highest specs of the drive. We only care about the minimum. If the sustained data rate of the drive dips below the required transfer rate for our video, the result is jerky playback, messed up audio and dropped frames. Given today's technology, there is no excuse for this. When in doubt, get better storage than you think you will need.

RPMs are a good indicator of a drives over-all performance. For video work we recommend drives rated 7200 RPM or faster. We have found that many 5400 RPM drives do not have the sustained throughput required for NLE work.

2007best-3.gifA single drive will get slower as it fills with data.

A hard drive is a spinning disk. Back when we all had turntables and records, this was very easy to explain. If you placed a penny on the outer edge of the record, it would travel a much greater distance in a single rotation than a penny placed near the label on the inside of the LP. More distance over the same period of time equals greater speed. Using this analogy today just gets me a strange look by most people. But the reality is still the same: A single drive will get slower as it fills with data. Even with todays SATA technology we have seen that once a drive reaches 75% of capacity, the sustained data rate starts to drop off considerably.

Drives are so big, affordable & fast today that this rule at times does not even apply. You can buy a 1TB drive for under $100, that's big enough to store over 15 hours of ProRes 422 or 75 hours of DV/HDV video. Even though we don't want you to fill the drive beyond 75% capacity - that still leaves you with enough room for several hours of video!

2007best-4.gifUse a dedicated drive for your video projects and media.

With today's powerful processors, lightening fast memory, super 3D graphics cards and huge SATA hard drives; you can capture, edit & playback single stream DV video with your system drive. That said, we still urge you to get a dedicated hard drive (7200 RPM or faster) for all your video clips. This will produce the best results - especially for video projects over 1 hour in length. If you are going to be editing HD footage we still recommend a dedicated RAID for all your video and media. That will give you the best results.

2007best-5.gifgraid3-backsmall.jpgRAIDs are GREAT for video!!

RAID stands for Redundant Array of Independent Disks. That means that 2 or more drives are grouped and formatted together in order to provide greater storage and performance. There are several different types of RAID, which we outline in greater detail in the FAQ section of this article. Setting up a RAID is very simple and easy to do. If you are putting together a new machine for video editing we recommend selecting a motherboard with a built-in SATA RAID controller. Not only will this work great, but we've found they often include very easy to use RAID configuration software. If you have a laptop or you do not want to open up your computer you can ad an external RAID solution like our G-Tech G-RAIDs.


Now that you know our Five Rules of Video Storage, let's dig in and answer some very important video storage questions!

How should I configure the storage for my video editing workstation?

Videoguys recommends the following:

C: Boot drive. 500GB or bigger. A 7200 RPM drive will work just fine. If you want faster bootup and program opens, right now a 10K RPM drive is a better value than SSD, but not as fast.

D: RAID for Video. Can be internal or external. For External we recommend G-Tech and Glyph.

  • Option 1:A pair of drives  RAID 0 gives you performance and value. 2 x 2TB= 4TB of usable space, but no redundancy if a drive fails.
  • Option 2: 4 drives set up as a RAID 5. This gives you performance plus redundancy. 4 x 2TB = 6TB of usable space

E: Export drive. Single 7200RPM drive. Your exports will go smoother and faster if you output the files to a dedicated physical drive, rather than the D: RAID. Another benefit of having a dedicated E: drive is that you can use it for back-ups, digital photo library, MP3 library, etc.
(Note: You can also set up a partition or folder on your C: drive for the exports. Not as good as a dedicated drive, but better than exporting to the same drives as your project resides on.)

Why can't I just set up my computer with one big RAID-0 for all my storage?
We do NOT recommend setting up your computer with a single RAID 0 array as your boot drive and video storage. This will actually give you poor performance for video editing. The constant reading and writing of small bits of data to the boot drive works against the need to read & write large continuous video files. Add in the overhead of the RAID controller and you get potential bottlenecks. For video editing you always want to have a separate drive or RAID array dedicated to your video files.

What about SCSI?
Years ago we only recommended SCSI hard drives for video storage but now, with SATA drive technology, that is no longer the case. While SCSI drives are faster, more reliable and better than SATA drives, they are also much more expensive and, for video work, they are not worth the premium. You get much better price / performance with a SATA RAID than a SCSI drive.

What about SSD?
We are very big fans of SSD, but unfortunately we think for now they are too expensive and small.
Because of their high cost, SSD drives are not really practical for your video storage, but we are getting there. When we have 1TB SSD drives for around $500, I think you will see a shift from RAID storage for video editing towards SSD, and perhaps even external SSD based RAID-5 configurations. We are still a year or two away from that happening.

We do have customers using SSD drives for their boot drive, and the results are amazing. Their system boots alomost immediately and programs instantly open.  The downside is they are using a 256GB SSD or smaller, and they fill up very quickly, especially if you are loading up video editing suites. That is why we are waiting for 512GB SSD drives to drop in price to $400 or less before we recommend them for your boot drive. My guess is that this could happen by early 2012, but that will depend on the economy coming back and production capacities ramping much higher.

How much storage do I need for making Blu-ray discs?
Lets break it down:

  • For this exercise, lets say you are editing ProRes 422 footage, so you need 66GB per hour of video. (see chart below)
  • This means a 2 hour Blu-ray is 2 x 66 = 132GB of footage.
  • Add in the extra unused footage and it's more like 3 x 66 = 198GB.
  • Figure that you'll be adding some extra graphics and audio tracks. Let's say another 25GB for these.
  • Now you need room for the MPEG2 or H.264 footage for the Blu-Ray, that's 25GB more.

Add it all up and it comes to almost 250GB of high speed video storage to make a 2 hour Blu-ray!!

How much storage do I need for HD?
That depends on the compression you are using. HDV footage takes up about the same amount of space as DV, 13GB per hour, while ProRes 422, DNxHD or DVC Pro HD takes up about 4 times as much space,  over 60GB per hour. Uncompressed HD footage requires around 500GB per hour - almost 50x the storage capacity of compressed HD footage. Plus, uncompressed HD also requires substantially faster sustained throughput. A single stream of uncompressed HD footage requires over 100 MB/sec!! At first glance, uncompressed HD video capture cards are inexpensive, but once you add the cost of the necessary storage you'll realize that editing uncompressed HD is actually very expensive.

Below is a simple chart to help make this easier to understand:

Format

GB/Hour

Hours/TB

DV

13

75

HDV 720p

11

90

HDV 1080i

13

75

DVCPro HD

60

17

XDCAM EX

19

52

ProRes 422 1920x1080

66

15

ProRes 422 HQ 1920x1080

99

10

Cineform Low

36

28

Cineform High

58

17

Avid DNxHD 145 8-bit

61

16

Avid DNxHD 220 10-bit

100

10

Uncompressed SD 8-bit

80

12

Uncompressed SD 10-bit

97

10

Uncompressed HD 720p 8-bit

396

2.5

Uncompressed  HD 720p 10-bit

497

2

Uncompressed  HD 1080i 8-bit

432

2.25

Uncompressed  HD 1080i 10-bit

558

1.8

What are the different RAID configurations and how are they used for video storage?

  • str0.jpgRAID 0
    RAID 0 is two or more drives striped together for maximum throughput. The computer sees theses drives as one single, large drive. With RAID-0 you get 2x the drive capacity of the smaller drive. So two 250GB drives give you 500GB of storage. (Note: we strongly recommend using identical drives – same model and capacity – for any RAID).

    • Advantage: Maximum throughput
    • Disadvantage: If one of the drives in a RAID-0 should fail, you would lose all your data.

  • mir1.jpgRAID 1
    RAID 1 is often called mirroring because both drives have the same exact data on them. This is done for protection. If one drive fails, you still have all your data on the other. This means the total volume of storage in the RAID-1 is only as big as a single drive. (2 x 250GB drives in RAID-1 gives you 250GB of usable storage).

    • Advantage: Data protection.
    • Disadvantage: We don't recommend RAID-1 for video. You end up with less capacity and when you add on the overhead of the RAID, your throughput speed is actually less than a single drive alone.

  • 001.jpgRAID 0+1
    RAID 0+1 is one way to get all the throughput benefits of RAID-0 and the data protection of RAID-1. You would need 4 drives for this, with two pairs of drives striped together as RAID-0, than the two RAID-0s mirrored.

    • Advantage: Combination of maximum throughput and data protection.
    • Disadvantage: Cost of four drives for configuration with overall storage capacity equal to just two of the drives.


  • par5.jpgRAID 5
    RAID5 is our top recommendation for video editing! A RAID 5 set-up allows you to use parity bits that spread the risk of drive failure among the other drives in the RAID. Keeping with my previous illustrations 4 x 250GB gives you 750GB of usable storage with even faster throughput than RAID-0 and data protection.

    • Advantage: Combination of maximum throughput and data protection without sacrificing half of the overall storage capacity.
    • Disadvantage: Cost of four drives and please keep in mind that 4 or more disks create lots of heat, so you really need to take a solution like this outside of the computer. External housings and connections will add to the cost.

Which storage system should I use?
As of January, 2011 the Videoguys' Techs have agreed on these storage recommendations based on the availability and prices of current products:

  • If you want the lowest possible cost, get a pair of identical 7200RPM SATA drives (1TB or larger) and stripe them together yourself. This increases the throughput of the drives and lets you use the entire capacity. This is called a RAID-0 stripe. Striping together a pair of 1TB drives will give you 2TB of screaming fast storage. You can stripe them together using your operating systems Disk Management utility. For even more performance get a PCIe SATA RAID controller card with 3rd party RAID software utility and set up a RAID5 with four 1TB drives to give you 3TB of lightning fast, redundant storage.
  • In the long run an external hard drive solution is the better value for most digital video editors. While you will pay a little more for an external storage solution you will find it worth every penny.
  • No hassle opening up your computer to install them.
  • No excessive fan noise from the extra cooling required by multiple hard drives in your case
  • No worries about the heat they generate deteriorating other components in your machine.
  • You can "sneaker net" your data between multiple computers – which means physically walking it from one machine to the next.
  • They work with laptops and workstations.
  • You can easily migrate your media to a new video-editing computer when the time comes to upgrade to a new one.
  • We also strongly recommend external storage in any environment with a multi-purpose PC. You need to protect your precious video memories and all the time you spent working on your video. If you have to use the family computer for video editing wouldn't you feel better knowing that you could simply turn off the external drive or safely remove it while the kids used the computer?

If you want to add an external storage solution to your PC or Mac; workstation or laptop; then we recommend the following solutions:

  • If you are using standard-definition DV footage and making simple videos less than an hour long you can use a single external drive solution. In order of preference we recommend connecting that drive to eSata, FireWire 800, FireWire 400 or USB 2.0.
    • If you are working with videos longer than an hour, if you have over 3 layers of video and graphics, or if you have some very creative and complex ideas with even more layers and filters and effects, then you'll want an external RAID solution like the G-Tech G-RAIDs.
  • If you are editing HD footage (AVCHD, HDV, XDCAM, DVCPro) or working with ProRes or DNxHD CODECs then you want to get at the minimum an external RAID solution like the G-Tech G-Raids
    • If you are working with uncompressed HD or very long or complex HD timelines, or multiple layers of ProRes or DNxHD video, then you should look into a more robust external solution that supports RAID5, such as the G-Tech G-Speed eS or eS Pro.
  • We're not big fans of USB3. With the introduction by Apple & Intel of Thunderbolt, we don't see it really having a fit with our workflows. While USB3 external drives and RAIDs are now shipping, our best advice is to skip USB3, and get the new Intel Thunderbolt based drives when they become available in late 2011.

G-RAIDs come pre-configured in a RAID-0 stripe for maximum throughput and they're specifically designed for video editing with excellent cooling and air flow. Heat is the enemy of any hard drive and constant video storage will surely create a lot of heat. G-Tech has taken this into account on all the G-RAID solutions including the newest G-RAID 4. G-RAID 4 allows you to connect to your computer via eSATA (in addition to USB2, FW400 & FW800), which is much faster than FireWire and even more reliable.

  • If you are doing complex HD productions and you want to have protection against drive failures than we recommend the G-Tech G-SPEED eS. The G-SPEED eS connects to your PC or Mac via the G-Tech eSATA PCIe RAID card and allows you to use the Videoguys top recommended RAID-5 the next level of storage. Many times that will include rack-mounted solutions. configuration
  • If you are doing broadcast level uncompressed HD work then you need to step up to the higher performing G-Speed eS Pro.
  • If you need to support multiple levels of uncompressed HD you are getting into more expensive storage, requiring expensive controller cards like NAS or FibreChannel. These cards can cost up to and over $1,000. If you need allow access to your files by more than one editor, you to add special networking hardware, software and expertise.

We hope that this article gives you a better understanding of the various types of video storage and what you need for your own projects

Videoguys recommends G-Tech & Glyph as our preferred storage partners
Heat is the enemy of all drives and Lacie and many other external drive vendors do not properly ventilate or cool their units. In addition the quality control on the drives these other vendors use in their drives are simply not as tight or rigorous as those used by G-Tech & Glyph.

G-Tech drives are designed for good air flow and heat dissipation. Their Raids are also fan cooled. Their quality control is excellent. We've been selling GTech storage for over 5 years and the long-term reliability has been superb. We've been banging on a G-Speed eS unit with our DIY6 rig for almost 6 months now. It simply works and works. In a RAID5 you get throughput speed and redundancy.

G-Tech just introduced GRAID ver 4 with quad interface. Now you get USB, FireWire 400 & 800 and eSata. If your Mobo has an eSata port, this is a really nice solution. G-RAIDs are RAID0, so you don't get any redundancy.


techstorage gtech graid gspeed gspeedq lacie caldigit seagate glyph forte GT062 gsesn gsespron hitachi g-tech g-raid

CATEGORIES: 

Email | Permalink | Comments (0)


Order online anytime! The Videoguys are also available to answer your calls:
Mon - Fri from 9AM - 5PM EST. Local Phone: 516-759-1611 Fax: 516-671-3092
©2012 Videoguys.com