Data Protection

[Kris]

I recently lost a bunch of data due a drive failure on a RAID 0 array. It was ok, as it was backed up elsewhere. So when rebuilding the array I striped it in RAID 5 format to protect the data from future mishaps. As it turns out, the day before delivery to Channel 10 another drive failed... at 1am no less. I was able to deliver the show because of the RAID 5 protection but as I sit here archiving the project I realise how lucky I was as I can see the current file count is at 20,000+. So very glad I had the saftey of RAID 5.

Something to consider for all us ITB type people, especially for time sensitive projects. Remember, data doesn't exist unless it exists in two places.

[TimS]

Something to consider for all us ITB type people, especially for time sensitive projects. Remember, data doesn't exist unless it exists in two places.

That is SO true!!

[heathen]

Something to consider for all us ITB type people, especially for time sensitive projects. Remember, data doesn't exist unless it exists in two places.

And some dual layer dvds also just in case.

[anj]

Always have always will,
never had to use a backup yet,
but will continue backing up just in case

[Thirteen]

I have had one almost new SATA drive fail this year, and one external case. I use Time Machine on my OS drive, and I have a couple of Audio drives in the machine, I record to one, drag copy to another. Final backups ate made to DVD and 2 drives.

[Kris]

We have an 8tb shared storage system here for video which is also set up in Raid 5 (mirrored) protection mode. Whenever we digitise anything it automatically writes it to 2 locations. It's such a relief. I am about to swap out the faulty drive in my Mac Raid. All I have to do is pop in the new drive and it will rebuild the data. Drive was only 9 months old so still under warranty. Good times.

[heathen]

A mate of mine had 3 drives in a 5 drive raid array die almost simultaneously once, almost unheard of but go figure, boy was he upset too.

Thats why dvd's are good.

Can't wait till solid state drives are cheaper also, they should be a lot more robust.

[heathen]

The PC I'm on now is 6 years old, died last week several times, then I layed the boot into it literally and its been fine since. :) The hd is still fine has outlasted the power supply too, found a new 1 last week and put it in, like a brand new pc after a re-booot with my Colorados and a new power supply.

Anyone need a computer technician? I'm available, heh.

[TimS]

Is his name The Fonz?? ;-)

[Kris]

gives a whole new spin to "booting" your pc.

[Linear]

We have an 8tb shared storage system here for video which is also set up in Raid 5 (mirrored) protection mode. Whenever we digitise anything it automatically writes it to 2 locations. It's such a relief. I am about to swap out the faulty drive in my Mac Raid. All I have to do is pop in the new drive and it will rebuild the data. Drive was only 9 months old so still under warranty. Good times.

I'm really interested in RAID solutions for recording situations. I have a 1TB drive (which is actually 2 x 500GB drives) set up as RAID1, meaning that everything is written to both disks simultaneously. So I don't back up projects that I'm working on, although sometimes I think I should be.

Performance-wise, it feels even quicker than a single disk. I think it comes down to how smart the software is, I know that it can actually read better if it shares the 'reading' load from both disks.

It's important to note that RAID doesn't always give you backup protection, sometimes it's just a way of splitting data over multiple drives and/or making a set of disks much faster than a single disk. My first Raid stripe was done like this where I thought I was mirroring but in actual fact it was just striped.

Do you think RAID1 is good enough to stop doing backups? The thing that worries me is environmental - both disks are subject to the same environment, running off the same power supply and motherboard, however I guess the chances of BOTH drives failing at the same time would be pretty low.

Chris

[Martin]

Synk is a good utility for around $25 to incremental backups (basically mirror a drive and only copy changes that you make, including deletions)

has worked well here for a year or so, only needed to use them when things have accidently been changed etc

do it ones every day or two

[smash]

Hey Guys,

Been meaning to chip in here, but been flat out all day.
So firstly...

Do you think RAID1 is good enough to stop doing backups? The thing that worries me is environmental - both disks are subject to the same environment, running off the same power supply and motherboard, however I guess the chances of BOTH drives failing at the same time would be pretty low.

Basically, the answer is No. Although it is very unlikely that both drives will fail, it is feasibly possible; you'd be amazed at what I've seen, including but not limited to; a fault y drive interface toasting multiple drives, i've seen drives run over by industrial equipment, and you would be amazed at the damage that a bottle of coke can do.

However the main issue with a single Raid 1 disk is the "Human factor". Although the advantage of Raid 1 is that Data is written to both places simultaneously, the disadvantage is that data is erased/re-written in both places simultaneously too !!! I myself have written over backups, confident that I have the correct file / version.
Oops.

While we're on the issue of RAID, On a project that I was tech-consulting on earlier this year, We had 3 drives fail (of a 12 drive / 12TB array) in the space of six hours. Turned out the entire shipment had been dropped !!! Thankfully, we had a backup.

DVD's are a good option, however remember that DVD's can be quite fragile. A single scratch in the lead in, can mean lots of expensive Data recovery. Oh, and there is a virus that adapted to attack the organic layer that data is stored on. Pretty rare, but I have seen a few examples.

One of my clients has a strategy that works very well, for each new project they purchase 3 new hard drives, they store multiple versions / snapshots using Time Machine, so basically they can revert to any version of a file at any time. This is done and copied to all three drives. When the client leaves, (and has paid), they take 1 drive. When the client reports that they have stored the drive safely, the second drive ships, if there is some issue with the first drive, another copy is made, before sending it. Finally the studio holds the final drive for at least 12 months, and a final layer of safety.

Using the above method, there are always at least 2 copies of the Data accounted for at all times, with 1 in transit.

If only 2 copies existed, then if someone loses a drive, crashes the car etc, and the other drive fails, you have a problem.

I had a client, who was a photographer, he dropped the only copy of a hugely expensive photo shoot, involving a internationally known model, and a privately charted yacht, into a creek. The data was recovered @ phenomenal cost, but still less than re-doing the shoot, and the loss of his reputation.

As an aside, I also use an Application called Super Duper to make a bootable, working copy of my system disk. That way, if I lose a boot drive, I can still continue working, without having to spend the 2 or so days it usually takes me to rebuild and reauthorise my system.

Two final thoughts I will leave you with:

1) Remember that drive manufactures EXPECT their drives to fail, they have an "acceptable" failure rate. Please also remember, that what may be an acceptable failure rate for 1000 drives, becomes less acceptable if you are one of the inevitable statistics.

2) Think of the real cost of losing data, if you are prepared, it may be nothing, if you are unprepared but lucky, it may be the lost of some money and/or time, if you are unprepared and unlucky, not only could you get sued, but your entire reputation could be destroyed, potentially taking your business with it.

Media is cheap, your career isn't.

[TimS]

Hey Guys,

Been meaning to chip in here, but been flat out all day.
So firstly...

Do you think RAID1 is good enough to stop doing backups? The thing that worries me is environmental - both disks are subject to the same environment, running off the same power supply and motherboard, however I guess the chances of BOTH drives failing at the same time would be pretty low.

Basically, the answer is No. Although it is very unlikely that both drives will fail, it is feasibly possible; you'd be amazed at what I've seen, including but not limited to; a fault y drive interface toasting multiple drives, i've seen drives run over by industrial equipment, and you would be amazed at the damage that a bottle of coke can do.

However the main issue with a single Raid 1 disk is the "Human factor". Although the advantage of Raid 1 is that Data is written to both places simultaneously, the disadvantage is that data is erased/re-written in both places simultaneously too !!! I myself have written over backups, confident that I have the correct file / version.
Oops.

While we're on the issue of RAID, On a project that I was tech-consulting on earlier this year, We had 3 drives fail (of a 12 drive / 12TB array) in the space of six hours. Turned out the entire shipment had been dropped !!! Thankfully, we had a backup.

DVD's are a good option, however remember that DVD's can be quite fragile. A single scratch in the lead in, can mean lots of expensive Data recovery. Oh, and there is a virus that adapted to attack the organic layer that data is stored on. Pretty rare, but I have seen a few examples.

One of my clients has a strategy that works very well, for each new project they purchase 3 new hard drives, they store multiple versions / snapshots using Time Machine, so basically they can revert to any version of a file at any time. This is done and copied to all three drives. When the client leaves, (and has paid), they take 1 drive. When the client reports that they have stored the drive safely, the second drive ships, if there is some issue with the first drive, another copy is made, before sending it. Finally the studio holds the final drive for at least 12 months, and a final layer of safety.

Using the above method, there are always at least 2 copies of the Data accounted for at all times, with 1 in transit.

If only 2 copies existed, then if someone loses a drive, crashes the car etc, and the other drive fails, you have a problem.

I had a client, who was a photographer, he dropped the only copy of a hugely expensive photo shoot, involving a internationally known model, and a privately charted yacht, into a creek. The data was recovered @ phenomenal cost, but still less than re-doing the shoot, and the loss of his reputation.

There was also, allegedly, a famous TV commercial involving a children's choir singing in various locations around a particular country...
Great ad, looked even better the second time they shot it after losing the first batch of media. Allegedly.

As an aside, I also use an Application called Super Duper to make a bootable, working copy of my system disk. That way, if I lose a boot drive, I can still continue working, without having to spend the 2 or so days it usually takes me to rebuild and reauthorise my system.

Two final thoughts I will leave you with:

1) Remember that drive manufactures EXPECT their drives to fail, they have an "acceptable" failure rate. Please also remember, that what may be an acceptable failure rate for 1000 drives, becomes less acceptable if you are one of the inevitable statistics.

2) Think of the real cost of losing data, if you are prepared, it may be nothing, if you are unprepared but lucky, it may be the lost of some money and/or time, if you are unprepared and unlucky, not only could you get sued, but your entire reputation could be destroyed, potentially taking your business with it.

Media is cheap, your career isn't.

That is awesomely said.
Great points and drives it home (excuse the pun!!)

[Kris]

"Great ad, looked even better the second time they shot it after losing the first batch of media. Allegedly. "

Not true. A good story but not true. I worked on the back end of that job. It was shot on 35mm film. Even if the digitised media failed at some point there would still be the magnetic tape media and if that had been erased for whatever reason it would have been a simple case of re-transferring the film to tape. Worst dollar case in that instance would be $10k - 20k, max.

The scope of this job and others like it mean you don't leave location until you have, quite literally, "got it in the can".

But back to the data issue... there was an article in AT mag ages ago (I think) about how DVD's are an organic medium and are susceptible to things in the atmosphere that can eat away at the disc itself. Anyone know about this? Is it true?

[mfdu]

dvd's are definately suseptable. i have just gone through my optical media archive and i'm sad to say that a couple of the oldest cd's and dvd's are already showing sgns of deterioration.

the foil layer of a couple of the cd's is lifting at the edges, and one of my first dvd archives is unreadable.

thankfully, everthing is backed up on a hdd as well, but they're a few old maxtor dives, and sounding a bit noisy.
(also the issue that both formats are currently stored in the same room of my home - if one is too hot or cold, so is the other!)

time for me to transfer the full archive over to new hdd's, and keep one set off-site. especially now i see 1TB external drives for $300. my first hdd (40MB - back in the days of the original Leisure Suit Larry) cost double that!!!!!

c.mfdu

[Kris]

Leisure Suit Larry. Man, I remember that, and Space Quest. Go Sierra games!

[Martin]

police quest!!!
getting through the intro questions on LSL1 at age 9 was interesting...

[Thirteen]

I wonder though if there is likely to be problems using Time Machine on an audio drive if it starts automatically backing up while you are doing a recording at the same time?

[smash]

I wonder though if there is likely to be problems using Time Machine on an audio drive if it starts automatically backing up while you are doing a recording at the same time?

I leave Time Machine off while actually working on the session, and enable it briefly whenever there is a session break, roughly every 4 hours or so, and then once the backup is done, turn it off again.

Do the same thing at the end of the day also.

[smash]

Not true. A good story but not true.

Thanks for the clarification. I had heard this from multiple sources over the years, but will now file it in the Myth basket.

I had been told it was the actual 35mm film that was lost, hence not being able to rescan the film.

[Linear]

Hi,

Yes good points. However after doing some more research (ie reading Wikipedia) that RAID1 will probably suffice.

I think the biggest risk to a hard drive is mechanical failure, or IOW it turning into a coffee grinder. Most other things I can deal with as there's not drinks/industrial equipment that could damage anything.

Human Factor - well that would include the human running an incorrect backup in the first place. At least with RAID1, you're more-or-less doing backups without even thinking about it.

Remember I'm not talking about long-term backups, that's one for the client. I'm talking about minute-to-minute backups, as daily/weekly backups would still happen - not even the most conscientious person would do hourly backups of work during a session, it's just not feasible. So if a hard drive fails at 4pm, you don't lose a whole days work by having to go back to last night's backup. This has happened before, which is the main reason I wanted a RAID system (the lost day was my time).

So for mission-critical systems you probably need a regime of backups to go with a RAID system, but for all intents and purposes I think it's adequate.

Chris

[smash]

not even the most conscientious person would do hourly backups of work during a session, it's just not feasible.

Depends....

When you're tracking a string section @ $X,000 per hour, I would do an backup every hour, possibly after every take !!!

[Kris]

I finally got around to replacing the faulty drive last night. It was a sweet thing to pop it in, assign it as "the spare" and then watch the Raid utility automatically start rebuilding the array. Happy days.