CDFreaks Presents: Home PI/PIF scanning article

And I'm very glad for that fact, too!

I think this is a terrific article, and a lot of hard work has obviously gone into it

Well laid out, and easy to understand for even me! Kudos, Dee

It is a good article! Testing all those burners and showing all those scans took a lot of time, and that article is the first time all those explanations about ECMA, 'rough' interpreting of PIE/PIF results and is certainly the first time all these popular brands/burners have been combined in one place, as before someone would have had to go to the respective reviews to read the results (~ 10 pages each), but now they are all together, with the unusual 'twist' of 'scanning' for each other. Some of the issues of some burners not liking other burners burns are sometimes evident in these comparison, whereas before one had to read posts in the forum, scattered over many pages and sometimes threads to get this idea. Nice compilation, Dee!

Had I realized how long it would take me just to compare 4 of 8 pages, I would have done all the work on MS Word, then cut/paste here. I will however, finish it up and post the rest when done. I was kind of baffled by the CMC MAG AE1 results, as I had seen AE1 burns much better than that, so either the burns I saw were from much earlier versions of AE1 that were better, the AE1 quality is now majorly variable, or something like that.

I will say (without any technical explanations of it, since Dragemester did that already) that I never found "Good Accuracy" for scanning on my 716a to be signficantly different from "High Accuracy." You basically saved the time, and I always found PIE total was +/- 100 and PIF total was +/- 10 for each scan when compared, so I never had any problem using "Good Accuracy" once I tabulated my scan results.

I also found from my testing that for example 4x CLV for the LiteOn is far better and more consistent then using 8x, as I saw scanning anomalies, especially towards the end of the scan, where 8x would report things 4x never did. I have read 2 other posters that seemed to be meticulous in scanning their burns noticed that even 8x on the BenQ was slighly problematic, and that results were more consistent at 4x on it as well. However, I have not tested 8x vs. 4x on the BenQ, so I cannot comment intelligently on that. It is reasonable to say that CLV scanning should be more consistent, and that is true with 4x CLV with a LiteOn.

The scanning speed has no bearing on the physical burn characteristics of the disc, that's true, but we don't actually know those characteristics before we attempt to test them and that's one of the purposes of performing a Disc Quality scan - to get an idea of how good or bad the burn is.

The "goodness"/"badness" of a disc isn't an all-or-nothing property, shades of grey exist, and by scanning at different speeds it's possible to get a better idea of where the disc is on the good...bad scale.

Discs/burns that are marginal may show significantly worse scans at high speeds, whereas really good discs/burns may show little difference.

Yes this is a bug. PlexTools Professional shows the drive's reading speed *before* the scan was started instead of showing the actual scanning speed. The only selectable scanning speed in PlexTools Professional is 2x CLV and you have to use a trick to scan at any other speed, so you can ignore the reading speed in those screenshots.

The trick is to change the read speed after the scan has started.
I'm not sure how PlexTools Professional XL behaves, because I don't have this program myself.

Some of you guys take this stuff MEGA seriously...

I thought everybody did.

(I couldn't resist)

Saves me a ton of time & money to get right discs for right job and testing discs helps me a lot to get there

Thanks Dee-27 for taking the time to make the tests here. I never had a Plextor DVDR but this test is pretty similar to my impression of my NEC/Benq/LiteOns

There's serious and there's obsessive...

then there's seriously obsessive...hence, the term FREAKS.....

Fair enough, moto!

Great article Dee-27. FYI for those asking about Plextor's high accuracy tests. Here is one disc with all tests done at high accuracy. Just the testing/scanning part took 102 minutes and 19 seconds. http://club.cdfreaks.com/showpost.ph...postcount=2125. Needless to say I agree with Dee-27 and DrageMester that the Plextor takes way too long @high accuracy

It is better not to take the conclusion seriously, which may not be definitive, as Pchilson has pointed out that "there are different views, opinions and results if the methodology is changed". I shall give some more reasons when I post my full comments later.

thanks Dee for the article. i'm sure a tremenduous amount of work. it's very much appreciated.

possibly a noob question, but:why divide by 4? shouldn't it be by 8 ?

another point: i think it would've been interesting had the Lite-On been tested using CDSpeed, not KProbe. at least, this would've better enabled a comparison of the BenQ, NEC and Lite-On since one variable (software) wouldn've been eliminated.

as to the 8x CAV/?x CLV scanning speed for the BenQ, well i guess it depends what exactly you want to test/compare.
but out of curiousity of scoobiedoobie's comment, i just scanned a 4.30GB data disc with my BenQ DW1620, 8x CAV took 9:36, 6x CLV took 8:59 ! faster indeed.

---

Last edited by dodecahedron; 25-09-2006 at 00:00.

After all, I'm glad that I never bought a Plextor DVD writer of 12x to 18x speeds. It not just saved me a lot of money, but also a lot of scanning time! For which I have to feel grateful regarding the Plextor distributor in Seoul that still sells a PX-716A for 3x + price of a GSA-H10A. Even a Conroe E6300 CPU costs just 40% more.

Thanks, Dee, for another great article.

@Wesociety

Nice to see you are still alive.

@DrageMester

Great posts!

Interesting...

To pick up on a few other arguments:

1. Neutrality - some scanning drives seem to judge recalibration points extremely harshly, while others are unforgiving of other deviations from optimum.

2. Speed - in some cases, since reading faster is more demanding, the fastest speed can be best for a simple "is this burn any good" determination.

Would have been interesting to see how the same set of drives responded to a bad burn on poor quality media - what shows up first, nasty TRT curves or PI/PIF effects - seen a lot of ratty TRT's in CDRINFO's tests, but their scanning is PI/PO not PI/PIF

Not at all a noob question!

8 ECC scanners such as the BenQ drives report the sum of PIF per 8 ECC blocks (Sum8) instead of per 1 ECC block (Sum1) as required by ECMA standards.

Calculating the Sum8 value from 8 separate Sum1 values is trivial, the reverse however is impossible, because there are many ways the reported number of PIF could have been distributed among the 8 ECC blocks.

Examples:
7 = 1 + 0 + 1 + 2 + 0 + 1 + 0 + 2 (Maximum is 2)
7 = 0 + 0 + 1 + 3 + 0 + 1 + 1 + 1 (Maximum is 3)
7 = 0 + 0 + 1 + 6 + 0 + 0 + 0 + 0 (Maximum is 6)

So how does one go about coming up with a reasonable calculated value when it is in fact impossible to calculate it correctly?

There are basically four ways of doing it:

1. Worst case calculation
All the PIF reported in the 8 ECC block sample could be in the same ECC block (up to a maximum of 208), so we have to use the reported number directly:

Worst-case Max PIF (Sum1) = 1 * Reported PIF (Sum8)

If the worst-case calculated Max PIF (Sum1) is within the ECMA limit (4 or less), there's 100% confidence that the real PIF (Sum1) seen but not reported by the drive is also within the ECMA limit.

2. Best case calculation
At best the PIF are completely evenly distributed between the 8 ECC blocks, so we can divide the reported number by 8:

Best-case Max PIF (Sum1) = 1/8 * Reported PIF (Sum8)

If the best-case calculated Max PIF (Sum1) is at the ECMA limit of 4 PIF, there's very little confidence that the real PIF (Sum1) seen but not reported by the drive is also within the ECMA limit.

If the calculated Max PIF (Sum1) is above the ECMA limit of 4 PIF, there's 100% confidence that the real PIF (Sum1) seen but not reported by the drive is also above the ECMA limit.

3. Statistical calculation
If we know or make reasonable assumptions about the distribution of PIF it would be possible to calculate with a certain probability what the largest PIF per 1 ECC block is (Sum1), given the total over 8 ECC blocks (Sum8).
An attempt at such a calculation can be seen in this thread:

Probability Analysis of 8 ECC block versus 1 ECC blk scans

According to that analysis, there's a 70% probability of the highest PIF (Sum1) being 4 or less if the PIF (Sum8) is 16, and there's a 99% probability of the highest PIF (Sum1) being 4 or less if the PIF (Sum8) is 8. Depending on which probability is acceptable, a divisor can be selected. Here are the two examples:

Probable Max PIF (Sum1) <= 1/4 * Reported PIF (Sum8) at 70% confidence
Probable Max PIF (Sum1) <= 1/2 * Reported PIF (Sum8) at 99% confidence

4. Empirical value
Instead of calculating the value theoretically you can analyze a number of Sum8 and Sum1 scans on the same drives, which is e.g. possible using KProbe2 on LiteOn drives, and then arrive at the experimental value for Sum8 that is the closest equivalent to the 4 PIF limit for Sum1 scans.

I don't think this has been attempted in a scientific way, but Erik Deppe (the author of Nero CD-DVD Speed) has come up with the "max 16 PIF per 8 ECC blocks" rule for CDSpeed Disc Quality Scans, and many people on the forums have found that this is a reasonable approximation.

CDSpeed Max PIF (Sum1) = 1/4 * Reported PIF (Sum8)

However, as I said at the beginning, it is theoretically impossible to perform the reverse calculation from Sum8 to Sum1 accurately.

---

Last edited by DrageMester; 25-09-2006 at 01:36. Reason: Added notes about confidence levels, corrected typos, added examples

Thanks again everyone for your comments.
It's getting late here so i'll briefly attempt to answer some questions raised.

@Quema34
The batch of CMC MAG AE1 i have is ok but not fantastic. I have a few left from an older batch which are much better, but there was not enough of them to conduct the tests.

@Matth
Regarding drives being harsh with the calibration points. Yes indeed, there are many theories why this happens, but i would not like to speculate on this.

@dodecahedron
Regarding CD-Speed Vs KProbe. KProbe is designed by Lite-On engineers for Lite-On drives and as i stated in the article, we used in-house software where available. In my experience, CD-Speed and KProbe show very comparable results in any case.

Your interpretation is that if 8X scans show these "anomalies" and 4X doesn't, 8X is less reliable. My interpretation is the opposite: my take on this is that 4X is unable to catch problematic discs that have reading issues at higher speeds...

Everyone has to remember that discs with somewhat marginal overall quality or burning quality are less consistent in scans than better discs (overall better, or better burnt). There is no reason not to think that the behaviour of 8X scanning (including the Benq scans that I know rather well) is actually to blame, in many instances, on the media/burn rather than on the drive.

This seems to be confirmed by many tests I've conducted at even higher speeds scanning (12X and 16X) in my Benq drives, with discs showing very good scans @8X but quite troubled transfer rate tests in at least three drives (NEC 3540, Benq 1640 and BEnq 1650). These discs look bad in 12X and 16X scans despite great scans @8X. The higher scanning speeds show figures that do have a correlation with the real-world reading curve, while the standard speed doesn't show anything special. Discs with flawless TRTs, on the other hand, always pass 16X scanning without any problem.

I'm currently more and more rejecting the dogmas of "standard" speeds for scanning. While the standard speeds are OK to compare different burning methods, in my view it fails to determine if the result is actually a good result in terms of real-world reading behaviour of a given disc. Proof of that being that TRTs can be poor despite good-looking scans (at standard speeds) and some discs that look so-so in scans (still at standard speeds) can have flawless actual, real-world operation.
This wouldn't be an issue if people used scanning only to choose better burning methods, and would systematically use TRT to get an idea of the actual performance of a disc. Sadly, on this board I see mainly users blindly trusting PIE/PIF scans (at standard speeds) as the one-fits all solution to test media. This is fondamentally flawed and relies more on wishful thinking than anything else. I'm very glad that the article discussed here clearly states that scanning doesn't tell you everything about a burnt disc. This should be evident to everyone, but for some reason it's not , so each time a CD Freaks guru (and I consider Dee-27 as one ) clearly states this, I'm very glad.

I know that Dee-27 always asks, in the NEC forum, to produce transfer rate tests in addition to scans. This is very wise in my opinion. I think that the fact so few members of this board actually perform these TRT tests, but rely 100% on PIE/PIF scanning, is the reason why these issues I mention are scarcely discovered and mentioned, and hardly seriously discussed because they're disregarded with some disdain (I hope this is going to change soon). PIE/PIF scanning has become so prominent on this board that many people don't even try anymore to find a real correlation between PIE/PIF figures and real-world behaviour. When was the last time you saw an exhaustive, sound comparison between PIE/PIF figures in scans and actual reading performance?

These were the two cents (and a half.. ) from a heretic.

Franck, I completely agree with your analysis. My findings are very similar to yours

Keep in mind also, though, that a majority (over 50%, well over 50% in fact) of DVDs burned are DVD Video DVDs and they will be read at very slow speeds. While standalone players are pickier readers than DVD drives, a disc that tests out well at 4x or 8x on a DVD drive will usually exhibit no problems in their intended use, very slow speed playback in a standalone player. This is not to say that high speed testing isn't a bad idea as it DOES single out a disc's burn quality or lack of it even more so than 4-8x testing. But a less than ideal test result at a drive's maximum speed does not necessarily indicate that that disc will have any performance problems if it's intended use is slow speed playback. Also, some drives simply do not perform as well as others at reading at their maximum speed so that's something else to be aware of before you get too excited if you get poor tests at maximum speed, sometimes even slowing down slightly such as from 16x to 12x will show a significant improvement in test results.

Testing at a drive's top speed is good at separating truly great burn performance from media that may only perform well at slower read speeds. But as I said, depending on your intended use of the disc a good result at 4-8x speeds may be more than adequate. If you are planning on ripping/reading a disc's contents at maximum speed, then obviously full speed testing will be useful.

To avoid highjacking this thread with high-speed scanning discussions , I opened a dedicated thread with my reply to scoobiedoobie.

xcellent work,Dee

Just how accurate is K-Probe, PXScan/PXView, CDSpeed or PTP XL? What speed is best? Is 'Good' testing adequate or should I take the extra time and do 'High' quality testing? I only need to do it to check the quality of a new spindle or if I have questions on the quality of a burn, not for every burn. The disparate though similar error results in the test only show how each drive is testing that particular disc at that particular moment. I would have liked to see one or more of the burned discs tested on a CATS scanner first and the error results used as a 'standard' for comparing the discs with the testing programs and drives. Just to know that a drive is neutral for all media or consistently reads a disc and displays the same error curve for each read doesn't tell me if the results are truly accurate when compared to an industry standard. If one drive or one app was consistent with the CATS tester, I'd be more confident in using that as a test fixture.

I agree with bob11879
The present article is very interesting
but the part "read accuracy" should be completed by comparison tests between each drive as scanner and the CATS

There has beeen an article about this published on cdfreaks in 21 May 2005:
« CATS vs. Home-made scans »
http://www.cdfreaks.com/article/202
and a thread of discussion:
« cdfreaks presents: CATS vs. Home-made scans »
http://club.cdfreaks.com/showthread.php?t=137566

What we need is to put both together

People don't like to understand.
Read what I posted earlier on accuracy.

Also some newer issues have showed up with calibration points and so causing Benq/NEC drives to react on some burns by some drives.
Is this issue ? only one way to tell from low measurement data from CATS or DATARIUS drives !