Setting up a multi-recording system - RAID?

vbimport

#1

I’m trying to set up a system that can handle 4+ simultanious recordings, as one-to-one’s.
I’ve got three external recorders and one internal. Now, my RAID-5 array has an average reading rate of ~160MB/s - and I know those numbers differs when it comes to sequential/multi-threaded operations.

I can burn two discs at full speed simultaniously, which takes about 5.5mins - but, when I add another, it takes about 10 minutes for them to finish. I’ve been doing this for a while now and burnt a few hundred discs this way. It works like a charm, only downside is that it takes more time and I can only burn four discs.

I know using one HDD per recorder would work, but I’d rather have a RAID-array. The only RAID-solution that I imagine can do the job is a striped one, but I really have no clue.
Im very interested in viewpoints, especially from ppl who has any XP on this.

The ultimate goal here is burning 4 or 5 discs with the time it takes to burn one. The ultimate quest for lowering the time it takes to store all the shit that pops up in the daily life :bigsmile:


#2

Your time and money will be better spent in using [I]different[/I] physical HDDs for each burn. Multiple burns from a single drive/array are made possible more by the access times of the drive/array than by the sustained transfer rate. In that department, only Raptor drives can boast adequate access times for 2 or 3 high-speed burns.

Also keep in mind that only chipset based SATA controllers have enough bandwidth for more than 3 independent burns. Your external drives are all running from the PCI bus, as well as most every other peripheral device, so they are all sharing the PCI bus bandwidth as well as the USB or Firewire controller’s maximum bandwidth. Usually, 3-4 high-speed burns will max out this available bandwidth.

I would use multiple HDD’s, all SATA and all on a chipset controller. When you see increased burn times, it’s due to buffer under-run. It is not a good idea to burn discs this way, as quality can suffer a lot. If your read-buffer is fluctuating a lot, you have a problem. Lowering burn speed to 8x can solve under-run problems and even make for a faster burn time than 16x.


#3

[QUOTE=CDan;2033407]Your time and money will be better spent in using [I]different[/I] physical HDDs for each burn. Multiple burns from a single drive/array are made possible more by the access times of the drive/array than by the sustained transfer rate. In that department, only Raptor drives can boast adequate access times for 2 or 3 high-speed burns.

Also keep in mind that only chipset based SATA controllers have enough bandwidth for more than 3 independent burns. Your external drives are all running from the PCI bus, as well as most every other peripheral device, so they are all sharing the PCI bus bandwidth as well as the USB or Firewire controller’s maximum bandwidth. Usually, 3-4 high-speed burns will max out this available bandwidth.

I would use multiple HDD’s, all SATA and all on a chipset controller. When you see increased burn times, it’s due to buffer under-run. It is not a good idea to burn discs this way, as quality can suffer a lot. If your read-buffer is fluctuating a lot, you have a problem. Lowering burn speed to 8x can solve under-run problems and even make for a faster burn time than 16x.[/QUOTE]

I agree your asking for alot of trouble that you won’t to have to diagnosis. Doing what CDan is a better solution and what your asking to do would require a SUPER COMPUTER OR CRAY as the process and memory needed to do such a mutitasking burns usage would be very high. RAID array is usually reserved for server applications in backing up data in case the main drive fails. That is what RAID array is good at doing. Also some RAID array controller allows it to be used as IDE controller allowing 2 IDE on one RAID controller connector. I am basing this on IDE raid that I have on my motherboard as I have on one RAID two IDE drives and another one IDE.


#4

[QUOTE=CDan;2033407]Your time and money will be better spent in using [I]different[/I] physical HDDs for each burn. Multiple burns from a single drive/array are made possible more by the access times of the drive/array than by the sustained transfer rate. In that department, only Raptor drives can boast adequate access times for 2 or 3 high-speed burns.

Also keep in mind that only chipset based SATA controllers have enough bandwidth for more than 3 independent burns. Your external drives are all running from the PCI bus, as well as most every other peripheral device, so they are all sharing the PCI bus bandwidth as well as the USB or Firewire controller’s maximum bandwidth. Usually, 3-4 high-speed burns will max out this available bandwidth.

I would use multiple HDD’s, all SATA and all on a chipset controller. When you see increased burn times, it’s due to buffer under-run. It is not a good idea to burn discs this way, as quality can suffer a lot. If your read-buffer is fluctuating a lot, you have a problem. Lowering burn speed to 8x can solve under-run problems and even make for a faster burn time than 16x.[/QUOTE]
I agree that basing my setup on external drives is not the best idea I’ve had, but it works, altough slow. I’m planning on upgrading my system quite a bit and get a bigger chassis in the process, so I can put in some internal SATA recorders.
I’ve been doing some tests with different burn speed, so it is really no point in burning at 16x with my current setup, It’s just so annoying having to change the burn speed manually for each burn.
BTW - is there a way to change the default burning speed for my drives?

The main buffer seems to be quite stable during recording sessions. Only the Ultra Buffer moves up and down, but it usually changes direction when it reaches 5% or so. I haven’t had any problems thus far - all the discs i’ve tested works perfectly fine. I always test out some randoms from the batches I burn.

[QUOTE=coolcolors;2033471]I agree your asking for alot of trouble that you won’t to have to diagnosis. Doing what CDan is a better solution and what your asking to do would require a SUPER COMPUTER OR CRAY as the process and memory needed to do such a mutitasking burns usage would be very high. RAID array is usually reserved for server applications in backing up data in case the main drive fails. That is what RAID array is good at doing. Also some RAID array controller allows it to be used as IDE controller allowing 2 IDE on one RAID controller connector. I am basing this on IDE raid that I have on my motherboard as I have on one RAID two IDE drives and another one IDE.[/QUOTE]

Although this is a demanding operation, I sincerely doubt the need for supercomputers to handle the workload.
I’m going to test different RAID settings with my current RAID controller - but first I need to burn out a few hundrer gigs and migrate data. I’ll buy a new set of recorders and try out different setups, both RAIDed and without.

I’ve been using optical media for years and plan on doing it for quite some time. I suspect it will take quite some time before other viable solutions can be evaluated.
Dual-layer media hasn’t evolved into what I hoped it would, and with only 4.7GB per disc, it means we still need lots of discs to meet the ever increasing flow of data in all the shapes forms that is present. Also, more discs means that we have to interact more with the whole process - by swapping discs, starting new sessions and writing labels. The only way to really counter this issue is to burn more discs at the same time! :smiley:
The goal is four, which is very close to a single layer Blu-Ray disc and a good solution in-between, before BD - or any new medium for that matter - takes over.

Thanks for replying. I’ll post some updates when I’ve done some tests.
In the meantime, please reply if you got something on your mind! :iagree:


#5

There’s absolutely no way you are going to get more than 2 smooth simultaneous burns from a single array or drive, period. Unless it’s a Raptor, even 2 is doubtful at more than 4x. This is not something you can get around. A Raptor will accomplish 2 at high speed, but no more.

The fluctuating buffer you describe is the burn being paused while the buffer fills again, and this is what you don’t want. Every time the burn is paused, you have a “re-link” that can introduce errors. The drive will also automatically down-shift the burn speed in response to this activity. What speed is set by default is determined by the drive and the media in use.

The burners being external is not an issue in and of itself, unless you use multiple hard drives and max out the bandwidth of the USB controller or PCI bus. But until you are set up with multiple hard drives, you won’t know what that limit is. My hunch is that 3-4 will be the limit. USB controllers are not known for handing that kind of heavy use very well.

You need to manually set each burn at 4x with your current setup and hope for the best. You may coax 3 simultaneous burns out of it, or may not. Using high speed media and burning at 4x is not a great idea, especially if there’s a lot of buffer under-runs.

Your current RAID-5 setup is likely slower than a single drive would be, due to the overhead. As mentioned, the sustained transfer rates as measured by benchmarks are almost meaningless in this scenario.