How do I get ECC Blocks from DVD Media Data Area?

vbimport

#1

Hi!
I’m a newbie to your forum, I DID search for the answer to my question but was unsuccessful in finding it.

I have playing with CD Media Data Area analysis using the ATAPI command set and getting all the Header, Sub-Channel, C1 and C2 information from CDs using the ATAPI "READCD" command with all it's flags and options. I have been trying to port this over to DVD Data Area analysis but cannot find a command that returns anything more than the 2048 bytes of user data. I need to get the rest of it, including the PI/PO ECC data!!

I have been through the MMC-5 and SPC-3 documents about 50 times and have tried almost everything! I have either missed it and am in for a BIG DUUUH; or it is not in there.

Does anybody know what (or if) ATAPI Commands exist to return the ECC data from the DVD Media Data Area Sectors? 

Thanks,
Sean


#2

[B]Welcome[/B] to cdfreaks. :slight_smile:

I don’t have the technical knowledge to give you a straight answer to your question.

Consulting ECMA-267 and reading this thread might give you some ideas how to procede further.

I suggest you to get in touch with our moderator Womble and ask him if your question can be moved to our Optical Storage Technical Discussions forum section.


#3

pinto2, thank you. I have sent a private message to Womble to ask if moving the thread to Optical Storage Technical Questions would be better.

The thread that you mentiond covers the math related to Parity Encoding/Decoding and I KNOW that will be helpful when the time comes! :smiley: Initially, though, I need to be able to retrieve the raw data that is actually recorded on the DVD. arrrrgg. :confused:

For example, from the ECMA-267 reference:

[B]16 Data Frames[/B]
A Data Frame shall consist of 2 064 bytes arranged in an array of 12 rows each containing 172 bytes (figure 16). The first row shall start with three fields, called Identification Data (ID), the check bytes of the ID Error Detection Code (IED), and Copyright Management Information (CPR_MAI), followed by 160 Main Data bytes. The next 10 rows shall each contain 172 Main Data bytes, and the last row shall contain 168 Main Data bytes followed by four bytes for recording the check bits of an Error Detection Code (EDC). The 2 048 Main Data bytes are identified as D0 to D2047.

This 2064 byte Data Frame is one of the things I need. So far I have only been able to find ATAPI commands to return the 2048 bytes of Main Data Bytes.

[B]18 ECC Blocks[/B]
An ECC Block is formed by arranging 16 consecutive Scrambled Frames in an array of 192 rows of 172 bytes each
(figure 20). To each of the 172 columns, 16 bytes of Parity of Outer Code are added, then, to each of the resulting 208
rows, 10 byte of Parity of Inner Code are added. Thus a complete ECC Block comprises 208 rows of 182 bytes each.

[B]19 Recording Frames[/B]
Sixteen Recording Frames shall be obtained by interleaving one of the 16 PO rows at a time after every 12 rows of an ECC Block …
Thus the 37 856 bytes of an ECC Block are re-arranged into 16 Recording Frames of 2 366 bytes. Each Recording Frame consists of an array of 13 rows of 182 bytes.

Or the series of 16 frames of 2366 bytes each to give me the ECC Block is another needed block of data. Again, I have not been able to find an ATAPI command to retun this informaion.

The above is a clearer statement of what I am trying to get from the DVD Media via the ATAPI command interface. Anyone run into this problem? Help would be much appreciated if you have.

Thanks,
Sean


#4

“I need to be able to retrieve the raw data that is actually recorded on the DVD.”

Sean,

I read raw DVD date with the READ10 or12 command. Put 16 as your LBA parameter
as nothing exists before it. I just like to see 1 block (ECC) at a time so I use 1 for the transfer length (sector count).

Disable Page Out isn’t used & you don’t need to Force Unit Access. Read it into a
buffer & display it with a “hex to string” routine.

You’ll see everything that’s in the sector. Is this what you mean ?

Several formats of the Read Disc (DVD) Structure command will give you info for
Physical format, BCA info and Disc Control Block info.

BillM


#5

Thanks Bill,

That's close, and I have tried both of those ATAPI commands. But it only returns the 2048 bytes of User Data, and I need the ECC data also, not just the 2048 user data. And I'm really only interested in the Data Area, not the Lead-in or Lead-out areas so much. It's for data integrity analysis, so getting the User Data only without the ECC info is not enough.

 I have also tried the READSTREAM (0xBE) command which reads 16 blocks of 2048 bytes each and follows that with 2 bytes of EDC info; 1 bit for each of the 16 blocks. Each bit tells if there were errors reading the respective block. No matter how I configure the Error Recovery Mode Page, I never get any errors back from this command. Even on areas of the disc that I know are damaged!

What commands does Nero CD-Speed or Kprobe use to get the PI/PIF errors from the media? That'd do the trick! Any idea?

Sean


#6

Sean,

I’m no expert in this area. I do have the Read/Write Error Recovery
Parameters Mode Page setup & have Set Configuration implemented to
return the Enhanced Defect Reporting Feature (29h) and I have a way to
get sense errors & report them during/after a command is executed …
in this case a read command.

I haven’t used it fully. I’ll do some experimenting maybe over the weekend.

I don’t at this point know of any commands that will directly read
the non-user or main data area of a DVD block and dump
the bytes out so you can read them. What I think is, and I could be
wrong, the unit bios detects the EDC or whatever error related bytes
for that DVD type are and reports it using various error reporting
methods.

The way I understand it at this point from reading the lit is we use the
Error Recovery Parameters Mode Page and Enhanced Defect
Reporting feature to check the settings & capabilities of the DVD unit 1st.
Is it currently set to report errors ? Is it capable of EMCDR ? etc.

Depending on the capabilities & current settings, we use Mode Select
on the error recovery mode page to set the error reporting parameters in
byte 2 and particularly the PER bit if we want EMCDR.(byte 7).

Then when we do a Read (or Write) on a DVD, we should get error reporting.
The only thing I’m not clear about is how they are reported ?
Do we just interpret sense returns depending on the settings ?

Anyway if you are getting error reporting back from your ReadCD command on CDs
but not from a read10/12 or write10/12 on DVDs then maybe your unit error reporting
settings need to be changed ?

That’s my best guess for now. Maybe next week I’ll understand more and if
you learn more or figure it all out please post your results here.

BillM


#7

Don’t know why I didn’t think of this before. I’ve used his program for a couple of
years now. Goto www.dvdisaster.com and download Carsten Gnörlich’s “dvdisaster”.
Run it on that DVD that has the errors. If it detects them then download the source
and you have yourself code examples of how to detect errors, PI/PO scans, et. al.

Bill


#8

Bill,

You may not consider yourself an expert, but you certianly know a lot about talking to drives with ATAPI! At least IMHO. Thanks for taking the time to respond and muse on this problem with me.

I have also checked into the Enhanced Defect Reporting Feature (0x0029) and found that none of the 3 drives I have support it. Not to mention that I have no idea, either, how to get the Defect Information out of the drive even if they did! Possibly what you said, tagged on as extra Sense Area data? Don't know.

From MMC-5 GET CONFIGURATION COMMAND (0xAC):

6.8.2.5 Type
The Type field specifies the type of data requested.
Table 288 shows the valid values for Type.
Table 288 – Type Field Definitions
Type Field Description
00h Performance data
01h Unusable Area data
02h Defect Status data
03h Write Speed Descriptor
04h DBI
05h DBI Cache Zone
06h – FFh Reserved

If the logical unit does not support the specified value of Type field for the mounted medium, the logical unit shall terminate this command with CHECK CONDITION status and set SK/ASC/ASCQ values to ILLEGAL REQUEST/INVALID FIELD IN CDB.
I get data back OK from Data Type 0x00, 0x03 but nothing for the rest except REQUEST/INVALID FIELD IN CDB error. And I cannot set the EMCDR bit in the Error Recovery Mode Page without getting the same error, though I can set and reset the other bitfields in that Mode page fine.

dvdisaster:
Unfortunately, this program does not support DVD-ROM media (which I need to be able to analyse). I did look at the code; it uses the READ10 command and only deals with sectors that connot be read and return with a MEDIUMERROR in the ASCQ. There is code there for generating the PI/PO data needed for re-recording damaged sectors, which will be helpful, but no code (that I could see) for retrieving them off the media! This is not useful for me right now, since I need more detailed error information about sectors that are still readable but that had some recoverable/PI/PO errors encountered during the read. AND I need to get those errors back from the drive!

This is the thing I was wondering about also. That’s the next area of expirimentation for sure. I’m gonna follow up the READ12 or READ10 commands with the REQUEST SENSE (0x03); trying different Error Recovery Mode Page settings and see what I get back.

Nero’s CD-Speed supports a lot of DVD models and gets the PI/PIF errors out of them. It works fine on my BenQ DW1640! So the error data’s got to be available from somewhere on that thing! It can’t be THAT proprietary! Maybe the fellow who wrote KProbe can shed some light on it? I’ll post up in that thread and see if he can drop a hint or something.

Thanks again,
Sean


#9

No standard command will let you get PI/PO bytes. As for reporting PI/PO
errors, this is done via proprietary commands which you can easily find
by yourself with an ASPI/SPTI spy (bustrace, tracespti, …).


#10

Sean,

your quotes:
“DVD-ROM media (which I need to be able to analyse).”

Ask Carsten Gnörlich why he doesn’t support DVD-ROM. The answer might
help explain your difficulties. I’d be interested to know.
Did you read the part on his site about PI/PO errors ?

“… And I cannot set the EMCDR bit in
the Error Recovery Mode Page without getting the same error, though I can
set and reset the other bitfields in that Mode page fine.”

On EMCDR forget it. Probably your drive was made before the manufacturer implemented it in bios? In that case they would have treated it as a “Reserved” field and all reserved fields must be set to zero or you’ll get that CDB error almost every time. Check all your routines for that. Reserved fields = zero or errors result.

“I need more detailed error information about sectors that are still readable but that had some recoverable PI/PO errors encountered during the read. AND I need to get those errors back from the drive!”

The DVD data loss prevention methods seem pretty elaborate compared with CDs. So far in my reading/research on EDC, I get the impression the drive bios does all the work then outputs the results according to the "Software Defect Management” & “Enhanced Defect Reporting” schemes. What we as programmers can do is set the parameters for them then interpret the sense returns from read/writes based on those settings i. e. Type 1, Type 2 & Type 3. It’s block level detection & reporting. Supposedly this enables us to notify the user about the condition of the disc even in the early stages of deterioration well before actual data loss occurs.

I haven’t implemented all the pieces needed to do this yet so I have no practical testing results to share. But I’m glad you posted because it’s got me going on an area that I haven’t paid enough attention to.

From a standard DVD burner app point of view, these data loss prevention schemes which are already implemented & accessible to programmers are all that’s needed. It wouldn’t be enough for a more technical app designed to report errors in greater detail at the byte & sector level – the same level the bios does it at already.

“That’s the next area of expirimentation for sure. I’m gonna follow up the READ12 or READ10 commands with the REQUEST SENSE (0x03); trying different Error Recovery Mode Page settings and see what I get back.”

Exactly. That’s what I’m going to do too. I think I can confirm that sense returns is how the error reporting is done. Be sure your sense routine includes recovered errors. I’d do the read in blocks (not sectors) (Transfer Length = 16). Start at LBA 0 … or if you know the LBA where those errors are start just before them. I’m interested to see what you get.

I think (?) what spath is saying is that the commands to read bytes from the non-user data areas are proprietary and will be unique for each manufacturer.

BillM


#11

Thanks spath. I was worried that was the case. Arrg :doh:

Well. since CD_Speed doesn’t care if you have ROM or RW media, I didn’t think following that down would be fruitful. Also, after looking at the code, I could only find a standard READ(10) command with no Sense data analysis besides the Status.

Exactly. That’s what I’m going to do too. I think I can confirm that sense returns is how the error reporting is done. Be sure your sense routine includes recovered errors. I’d do the read in blocks (not sectors) (Transfer Length = 16). Start at LBA 0 … or if you know the LBA where those errors are start just before them. I’m interested to see what you get…[/QUOTE]

I followed spath’s advice over the weekend and discovered that he was spot on (thanks dude :cool: ); there are proprietary and undocumented commands; 0xF8, 0xF9, 0xFC, 0xFD and others that are passing to the drive. I’m not sure what they do, but I’m sure THEY are the commands that are returning the PI/PIF data.

The next step is to find out how to use these commands and what they are doing.

Anyone have the scoop on these 0xF? ATAPI/SCSI commands. Some Moderator can put a sock in my mouth if I’m treading on fobidden ground here. :o


#12

Sounds like fun but I don’t need to go there …

Just need to notify the user after a read/write operation if the disc is getting close to a data loss condition. DVD drives will report this info via sense returns. I’m way too lazy to make a Kprobe-like inspection of the disc when it’s not necessary in my case.

enjoy,

Bill


#13

Cool Bill. I’m glad you got to a point whre you have enough information to move forward! :smiley: I’m very close myself now.

Was fun to hack out those proprietary commands; basically hooked the DLL and logged each CDB command call to a file. But still hornery to decode what they are doing. I can see the CDB and all it’s bytes, but I don’t know what the significance of each byte is, except that the first one is the CDB Opcode. But each CDB has its own structure! Some send data only, some receive data only, etc.

So the question is; can I get my hands on some documentation of these 0xF? ATAPI/SCSI commands. Anyone point me in the right direction??


#14

Here’s the command sequence for PI/PIF scanning 4MB of the DVD-ROM disc in my BenQ DW1640:

START STOP UNIT – 0x1B 0x00 0x00 0x00 0x00 0x00
READ CAPACITY ---- 0x25 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
SET STREAMING ---- 0xB6 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x1C 0x00
MODE SENSE(10) — 0x5A 0x00 0x2A 0x00 0x00 0x00 0x00 0x00 0x28 0x00
SET MEDIUM SPEED - 0xBB 0x00 0xFF 0xFF 0x21 0x13 0x00 0x00 0x00 0x00 0x00 0x00
Unrecognized CDB - 0xF3 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Unrecognized CDB - 0x78 0x00 0x00 0x00 0x00 0x00 0x00 0x40 0x22 0x00 0x10 0x00
Unrecognized CDB - 0xD7 0x00 0xC0 0x03 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00
Set Limits ------- 0x33 0x6C 0x80 0x01 0x43 0x59 0x42 0x00 0x04 0x00
Set Limits ------- 0x33 0x6C 0xA9 0x80 0x00 0x00 0x00 0x00 0x00 0x00
Unrecognized CDB - 0xFD 0xF1 0x42 0x45 0x4E 0x51 0x00 0x00 0x00 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xFD 0xF2 0x42 0x45 0x4E 0x51 0x00 0x00 0x00 0x00 0x00 0x00
MODE SELECT (10) - 0x55 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x14 0x00
READ (10) -------- 0x28 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x00
Unrecognized CDB - 0xFD 0xF1 0x42 0x45 0x4E 0x51 0x00 0x00 0x00 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x0A 0x00 0x00 0x00
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x06 0x00 0x00 0x00
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
// these 2 repeat
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
// end of repeating - total 230 times to scan 4MB of disc
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x0A 0x00 0x00 0x00
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xF9 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x06 0x00 0x00 0x00
Unrecognized CDB - 0xF8 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x02 0x00 0x00 0x00
Unrecognized CDB - 0xFD 0xF2 0x42 0x45 0x4E 0x51 0x00 0x00 0x00 0x00 0x00 0x00
MODE SELECT (10) - 0x55 0x10 0x00 0x00 0x00 0x00 0x00 0x00 0x14 0x00
START STOP UNIT – 0x1B 0x00 0x00 0x00 0x00 0x00
Anyone know where I can get the usage details for the following CDB commands:
0x78, 0xD7, 0xF3, 0xF8, 0xF9 and 0xFD?


#15

Do you only have the CDBs or did you also trace the data that was transferred? You might want to have a look at [thread=108669]TraceSPTI[/thread].
It looks like you’ll have to find out on your own what these commands to. I guess only BenQ has a documentation.


#16

So far I’ve only logged the CDB going out to the drive (pre-execution). The next step is to log what comes back. But before that, I have to note which ones are sending data to the drive and which ones are receiving data from the drive and then figure out the amounts of data to log pre/post command execution.

I downloaded it and will check it out. Thank you very much! :slight_smile:


#17

Did you have any success decoding the commands?


#18

Still trying to get the PI and/or PO data from DVD media using ATAPI commands.

Anyone know how?

Is the Source for KProbe still available?


#19

Still looking…
Anyone know where I can get the usage details for the following ATAPI CDB commands:
0x78, 0xD7, 0xF3, 0xF8, 0xF9 and 0xFD?


#20

bump! :smiley: