Optical Storage Technical Informations

//edit// Moved this thread to a more appropriate forum //end of edit//

Source : Audio Compact Disc : Reading and Writing the data

For the complete paper start here. This is a small part of it :

Prologue :

Part I – A Brief Overview

Data storage in CD format is not a simple thing. Typically, a user pictures the “1s” and “0s” in the memory of the computer as being directly transferred to “pits” and “bumps” on the CD disk. Unfortunately, it is far from that easy.

To begin with the incoming data is subjected to a series of coding operations. These coding operations add a number of additional parity bits to the data for error detection and correction purposes. The data is also subject to an interleaving process (which means that adjacent data on the disk is not adjacent data from the incoming file).

Additionally, the physical form of the data is changed (EFM coding) to eliminate the possibility of adjacent “1s”. (This is done because it is the edges of the pit – not the pit itself – that represent l’s in the data stream.)

A. Simple error detection and correction codes

Error detection and correction codes are fundamental to the operation of any digital storage system. There are literally thousands of such codes. These codes typically rely on using additional bits (usually called parity bits) to carry the error detection and correction information.

In a simple binary parity check, a parity bit is a single bit that represents whether the total number of “1s” in a particular data stream is even (1) or odd (0). (Modulo two addition).

For example, assume that you are setting a parity bit over all the digits of the following word.

1101 0000

The total number of “1s” is odd, so the parity bit would be 1. The word might then be written as

1101 0000 1

where the last digit is the parity bit.

Even simple binary parity checks can become quite complex if more than one parity bit is used. For example, you may elect to have two parity bits – one on the first four bits of the word and one on the last four.

1 1 0 1 0 0 0 0 P1 P2
x x x x 1
x x x x 0

If enough parity bits are used, then error can not only be detected – they can also be corrected. For example, consider what happens if you use four parity bits. The first one is on the first four bits, the second one is on the second four bits, the third one is on the 1,2,5,6 bits and the fourth one is on the 2,3,6,7 bits.

1 1 0 1 0 0 0 0 P1 P2 P3 P4
x x x x 1
x x x x 0
x x x x 0
x x x x 1

Now, assume that there was an error in the final bit.

1 2 3 4 5 6 7 8 P1 P2 P3 P4
1 1 0 1 0 0 0 1 1 0 0 1
x x x x 1
x x x x 1
x x x x 0
x x x x 1

Parity bit P1 would agree with the parity bit in the transmitted word, P2 would NOT agree, P3 and P4 would agree. Since P2 is the only parity bit not agreeing with the transmitted word – then the error must be in the 8th bit.

Unfortunately, the majority of error-detection and correction algorithms used in CD players are not as simple as the binary check codes discussed above. Although an overview of these codes will be presented, in-depth analysis of the codes is beyond the scope of this course. (Interested students should consult more advanced references, such a W. Peterson, Error-Correcting Codes, MIT Press)

B. Simple interleaving

Interleaving is a very simple and powerful idea. To illustrate interleaving, assume that you have a frame consisting of several characters of information,

U N I V E R S I T Y O F W A S H I N G T O N

Assume that you spit on the disk and destroy several of the characters.

R S I T Y O F W A S H I N G T O N

The first word is then very hard to reconstruct! However, you can take the original frame and scramble it as,

U N I V E R S I T Y O F W A S H I N G T O N

O N S T H U G R F S I I O T W N N V E I Y A

Then you can damage it,

U G R F S I I O T W N N V E I Y A

Then you can unscramble it,

U N I V E R I Y O F W A S I G T N

It is much easier to “interpolate” or “guess” the missing letters. (A bit like the later stages of “hangman”!)

C. Concealment

In practice some errors are so large that they cannot be corrected by the error-detection and correction algorithms. Unless these errors are handled by some other means, they can result in audible clicks in the audio output. In order to avoid these clicks, several methods are used to conceal uncorrectable errors:

Interpolation: In this technique, some average is constructed using the valid data around an error. This average is then substituted in for the erroneous data. Since most music (with the possible exception of heavy metal!) is continuous – this method works well for concealing relatively short errors.

Muting: Muting is a last ditch technique – as it effectively creates a brief period of silence in the audio train. However, it is not effective to simply set all the binary digits to zero --as this produces exactly the click that we are trying to avoid! Instead, the volume is faded out and then back in again to conceal the error.


EFM means Eight to Fourteen Modulation and is an incredibly clever way of reducing errors. The idea is to minimize the number of 0 to 1 and 1-0 transitions – thus avoiding small pits. In EFM only those combinations of bits are used in which more than two but less than 10 zeros appear continuously.

For example, a digital 10 given as a binary 0000 1010 is an EFM 1001 0001 0000 00

EFM Table 1

EFM Table 2

The use of EFM coding means that pits come in discrete lengths ranging from 3 bits long (often written 3T) to 11 bits long (11T).

As the laser beam scans across these pits, a very distinct RF signal is formed. The shortest wavelength in this signal (highest frequency) is produced by the 3T pits. The longest wavelength in the signal (lowest frequency) is produced by the 11T pits. The zero crossings of the RF signal represent the edges of the pits – and thus the binary “1s” in the data stream[2]. (Notice that the longer the wavelength, the larger the amplitude of the signal.)

It is common to display the photodetector output on a scope with a conventional trigger. This results in a display where the nine possible frequencies (3T to 11T) all add up on top of each other. This type of display is termed an “eye” pattern and provides valuable information about the various alignment parameters of the CD player. Notice that the relationship between size and wavelength is very distinct in the eye pattern

The RF output is converted to a square wave, and then phase locks a clock with the period T. The CD player then begins to hunt for the characteristic start of frame symbol, which is three transitions separated by 11T. (100000000001000000000010 + 3 merge bits) Then, the player isolates the 33 17T symbols, and then kicks off the 3T merge bits – leaving the 33 14T active symbols.

//edit// Moved this thread to a more appropriate forum //end of edit//

Introduction (Market Success of CD-RW)

The market for CD-ReWritable drives has been growing at an unmatched speed for the past quarters. In 1999 16 million drives were sold and the volumes are expected to surpass 30 million units in 2000. PC-OEM’s are rapidly adopting CD-RW as a standard option on their PC’s. On the media side, 3 billion CD-R/RW discs are expected to be sold this year, compared to 1.8 billion floppy discs (Source: Japan Recording-Media Industries Association). In short, CD-RW has become a mainstream product in PC’s.

The challenge to the industry is now to secure that the product continues to be useful and enjoyable for the users, and that it is easy to support by the supplying companies.

Key Success Factors

The main factors that have been driving this success are:

· Compatibility within the data world and with the world of consumer electronics. The devices that can read a CD include data drives (e.g. CD-ROM, DVD-ROM, CD-R/RW) and audio/video players (e.g. CD-Audio and DVD-Video). The installed base of these taken together is approaching 1 billion devices.

·Ease of use. This is the primary request of the users. It has come a long way, but must still improve.

·Performance keeps improving as new drives and new media continue to be launched.

·Application support. CD-RW drives support a large scope of applications, ranging from audio and video creation to back-up, archiving, and general data storage.

·Durability of media. Optical media is more durable than magnetic media, which makes it a safer choice for archiving and backup.

·Low cost of media. The cost of CD-RW has dropped fast in the last years and is now below 3 USD per disc, or less than half a cent per MB of data.

·High capacity. The capacity of a CD-RW is still sufficient for most applications, but for others the upgrade path towards DVD is secured.

Main Applications

The most widely used applications, so far, are audio creation, back-up, archiving, and general data for temporary storage and interchange. Audio creation builds on the compatibility of recorded CD’s with the hundreds of millions of audio players in homes, cars, and elsewhere. The main advantage is that compatibility is secured and that the application is easy to use. Back-up and archiving require high performance, more than compatibility. Compatibility is, however, also important, to enable other users to read the archived material, and to enable the same user to read it on later generations of equipment.

General data storage, or “Drag and Drop Data on CD-RW”, is the focus of this paper. The success in serving that application has in the past been hampered by limited file system compatibility, lagging performance, and the need for special software to support the drive letter access functionality. However, CD-RW technology has the potential to overcome these issues and thus become the only removable storage drive most users need for their PC’s. That single drive would be able to replace the CD-ROM drive, the floppy, and any other high capacity drive the end-user will need. This would reduce the overall system cost of the PC and make it easier to use. In the following we will first look at the requirements towards this application from the end-user point of view and see what gaps must still be closed to fulfill these.

Drag and Drop Data on CD-RW Requirements

The overall requirement for “Drag and Drop Data on CD-RW” is that it just works like any other storage. The user should have drive letter access support in the operating system, without any need for additional application software. This means being able to drag and drop files to and from a drive within Explorer, save files directly to the drive from any application (word processing, presentations, video creation), or download files directly from the Internet. Media formatting should be completely transparent, not causing any delays for the user. The media should be read and write compatible with other CD-RW drives, and readable in ROM drives.

Current Shortcomings

Writing data (drag and drop) on CD’s is currently not supported by any operating system. The functionality is so far handled by special applications, that need to be installed on the PC running the drive. The discs, that are written this way, are not automatically readable by other CD-RW drives or CD-ROM drives, but require a proprietary read driver. New CD-RW discs need to be formatted before data files can be written to them. This still takes too long and is difficult for the end-user, keeping in mind the format choices required.

The Fix: The Mount Rainier Solution

The objective of the Mount Rainier solution is to overcome the mentioned shortcomings, by enabling native OS support of the drives and allowing CD-RW to be used as a drag and drop device of choice.

Below a high-level description is given of the key technical elements of Mount Rainier.

The 5 key elements this paper will focus on are:

  1. Physical defect management by the drive

  2. Logical write-addressing at 2k

  3. Background formatting

  4. Command set implementation

  5. Compatibility and standards-compliance

Defect Management in the Drive

In the last few years, applications have been developed allowing drag and drop on CD-R and CDR-W media. The leading solution for this use is currently based on UDF1.5, in which defect management handling in the UDF file-system is specified.

However, a solution allowing more commonality with standard data-drives (MO, DVD…) and file-systems is strongly preferred.
This requires adding physical defect-management into the drive. As a result, the responsibility for handling physical defects is where the defects can be handled optimally:
inside the drive. Otherwise, file-systems (or/and applications) need to have knowledge of drive/media defect characterization & capabilities, which is more complex. Furthermore, although CD-ROM drives have no knowledge how to handle defect-sparing, care has been taken by the Mount Rainier group, that CD-ROM’s will be able to read Mount-Rainier compliant discs though a generic read-driver, allowing read-back in legacy systems (see also further).

2 K Addressing

The logical addressing capability is another important factor. Where currently for CD-RW a block-size of 64 kB is required, most data systems are based on 2k or 4k addressing capabilities. The Mount-Rainier functionality has defined 2k logical addressing as a mandatory support requirement for drives. This will allow file-system and caching capabilities of the host to be transparent for the media related block-size.

Background Formatting

Time to use" is one of the key improvement areas of Mount Rainier, adding high value to the end-user.
With current CD-RW, the user must wait for the formatting process to complete before writing data to a new disc.

FS data lead-in lead-out FS data data.

The Mount Rainier specification will allow a user to drag and drop data within seconds after insertion & spin-up of the disc. Invisible to the host, a background formatting sequence will be performed by the drive. The use-experience is optimized by ensuring that drag and drop data-storage tasks will get priority above progress of “de-icing” and “finalization” in the background by the drive. Furthermore, measures have been taken to allow fast eject, keeping full compatibility with CD-ROM readers.

Command Set

Although some MMC-1 and MMC-2 adherence is present within current CD-RW drive solutions, there is currently a strong need for supporting vendor-unique feature implementations (commands and mode-pages) by ISV’s. For the Mount-Rainier compliant drive, implementing a mandatory command-set for
data-use will be required (fitting MMC-2 structure).

Compatibility and Standardization

The Mount Rainier group is focussing strongly to assure compatibility of the solution with future and existing hardware-platforms. Below is demonstrated how the CD-ROM read compatibility will be assured for systems with native-OS support and for legacy systems. For the latter case, a generic read-driver will be made available. Another key focus of the Mount Rainier group is driving standardization, and reducing the need for vendor-related legacy support. This will be handled through 3 paths :

  1. Driving drive/media level standardization by a specification from the Mount Rainier group.
  2. Defining interface level standardization through MMC.
  3. Proposing to the drive- and PC-industry the function-implementation (PC-200X requirements) and checking the drive-level support through WHQL.

Benefits

Once the Mount Rainier solution is fully functional in the market, the end-user can have a single drive in his PC, running all the applications he may now need three or four devices for. This will be easy to use, since the functionality will be fully transparent and the discs will be compatible with other reading devices. To the industry, this opens an opportunity for having added functionality with fewer devices. This will reduce the overall system cost and decrease the need for legacy support. Taken together, this initiative has the potential to grow the market by fulfilling customer needs.

The Mount Rainer Group

Compaq Computer Corporation, Microsoft Corporation, Royal Philips Electronics, and Sony Corporation have formed the Mount Rainier group to define and implement the proposals presented in this paper. Additionally, a group of companies has endorsed the initiative as early adopters.

Call to Action

CD-RW drive makers are encouraged to incorporate the Mount Rainier functionality in their drives by first half of 2001. By that time, the PC-OEM’s should request that functionality from their drive suppliers. ISV’s are encouraged to leverage the benefits to the end-user, by adapting their applications to the basic functionality supported by the Mount Rainier implementation.
More information is presented on the web page of Mount Rainer (www.mt-rainier.org).

Acronyms

cmd: Command
FS: File System
ISV: Independent Software Vendor
MMC: Multi Media Command set
OS: Operating System

Right now (Philips changes their web page structures so frequently), this Philips link gives a good starting point for newbie questions about standards, SuperAudio, Mount Rainier, DVD-RW/+RW, etc. This link could be added as a sticky thread somewhere, but I’m not real certain where to post this message and can’t make a thread become sticky anyway.
Feel free to add it wherever you wish, but I think even stealing that link and making a master list of FAQs which includes that link would be a good idea.

I won’t make a FAQ for this forum, since there are no questions here which are asked frequently.

But collecting a list with useful links is not a bad idea.

I don’t think people here will be interested in
non-free standards, but links to technical
informations on optical storage technologies
is a good idea. I’ll start, feel free to post your own :

http://www.ee.washington.edu/conselec/CE/kuhn/cdaudio2/95x7.htm
ftp://ftp.ecma.ch/ecma-st/Ecma-130.pdf
http://www.chipchapin.com/CDMedia/
http://narya.phy.qub.ac.uk/~mjl/phy803/
http://audi.nchu.edu.tw/~imtech/course/odsguideline.htm

ftp://ftp.ecma.ch/ecma-st/Ecma-267.pdf
ftp://ftp.t10.org/t10/drafts/

I’ve written a bunch of articles about copy protections, which should hopefully be easy to understand.

http://sirdavidguy.coolfreepages.com

I’ll also host the CD-related site of anyone who wants it (Withing reason).

I just added a Securom, CD-Cops, etc. article.

More an overview than precise technical informations :

http://home.mira.net/~gnb/mac-cdis/index.html

Very good explanation of CD-R structure including ATIP modulation.

http://www.interstage.dk/Sider/Produkter/Presse/Emner/CDR/CDR_PRE.shtml

I can’t access the ECMA docs from the links above. I don’t know if it is related to ftp (I often have problems with ftp), or if ECMA has moved. Anyway, the webpage has moved :

http://www.ecma-international.org/publications/standards/Standard.htm

very usefull page, thx

nice dvd page
http://www.pioneer.co.jp/crdl/tech/index-e.html

http://www.exp-math.uni-essen.de/~immink/pdf/cdstory.pdf
http://en.wikipedia.org/wiki/Compact_disc
http://www.breckrowell.com/cd101/

http://www.mcgoodwin.net/digitalaudio/digitalaudio.html
http://www.mbiweblink.com/BookInfo.html

Thanks so much, needed to read some of those.

ftp://ftp.t10.org/t10/drafts/mmc5
Does anyone have these pdf files and is willing to send them to me?

Pity that they’ve decided to restrict them. But for now (until they restrict that also) you can still get them from the mirror at t11:

http://www.t11.org/t10/drafts.htm

thanks!