PDA

View Full Version : Sub Channel Data



MonkeyMonkey
26.10.2005, 09:41
Greetings All,
I've been scouring the net and forums and have gathered what information I can on Sub Channel Data.

I've used many copy tools (blindwrite, clonecd and a few GPL tools like readcd etc...)

I have been analysing three sets of files with a hex editor:
- Blindwrite (B5T and B5I)
- CloneCD (SUB and IMG)
- ReadCD or cdrdao (Cooked -2048, RAW -2352, RAW+96 -2448)

I can identify some elements (sector ends and Sub Channel data) across these files, but have come across some stumbling blocks.

Goal
I don't like proprietry file formats, for my own reasons. I want to make 1:1 coppies of my CD's, and want to do it in a more standard and open manner. For me, this is recording RAW data and RAW Sub Channel data.

Blindwrite
I haven't reverse Engineered B5T and B5I, it seems quite proprietry from what I've seen internally. I'm aware there's a project out there to convert these files to ISO so I could get some information there. I do like the data collected in these files and the way they go about it, but I have to pay money to record my CD's that way and the money is better off going towards storage for the cds'.

CloneCD
I also like how CloneCD seperates the Sub Channel data into a seperate SUB file and then kept a RAW (2352 bytes/sector) in an IMG file. This is the concept I want to emulate, though maybe add some overhead which I've noticed in the Blindwrite files.

ReadCD
This is the Tool I want to use, it's open source and available on multiple platforms. I've used it to extract Raw R-W Sub Channel data with the RAW (2352) into a single image. If it got the data I want it too, I would simply write a quick tool to extract the sub channel data into a seperate file and hopefully end up with the CloneCD output.

What I don't understand
ReadCD will extract RAW R-W data, but I've read somewhere about RAW P-Q or something of the sorts... what's the difference? I can not find a great deal of info about the construction of sub channels. I would REALLY appreciate some links / info on the topic
I've looked at the subchannel data in a hex editor and it is does not even come close to resembling the sub channel output of clonecd. (I'm assuming the .SUB file lists each 96byte block of Sub Channel data consecutively for all sectors on the CD, that's how it appears).

Snippets of other Information
I do know that the CloneCD Sub channel file is not consistent in generation and that some byte patterns will vary each record. I believe that the CD-Burner does not perform error-checking and recovery on Sub Channel data yet it's reliable enough for copy protection, CD text and other data?
I stipulate that CloneCD extracts additional information from the CD-Recorder, which I'm not doing with ReadCD, and uses this data to decrypt the Sub Channel data.

If You've read this far, I appreciate your efforts already.

Edit: Updates
Well, who would have thought that when trying to learn what "CD+G" means, I've found heaps of info on Sub Channels data... not enough yet though. Keep the help comming :-)

Edit: Links, For those interested
http://en.wikipedia.org/wiki/CD-G

Underheaven
26.10.2005, 10:01
h**p://forums.afterdawn.com/thread_view.cfm/22344

Nikos
27.10.2005, 02:18
Veeeeeeeeeeeeery useful:

http://www.cdrfaq.org

If you don't read this, you'll remain in the dark forever. :)

MonkeyMonkey
31.10.2005, 10:07
Thanks for the responses. It's odd that after I wrote the post I started to find more and more.

I'm checking out cdrfaq... At the moment I've found someone's code that will deinterlace the sub channel data for me, but alas it was written for the microsoft compilers and I don't have these (nor do I wish to buy them). I'm trying to port the code over to gcc compatable.

Meanwhile, I'll check to see if there's enough info on cdrfaq so that maybe I can rewrite the algorythm myself.

Thanks.

johngalt
31.10.2005, 23:17
Keep us posted - if you write a good enough script to use readcd / re-write readcd / develop your own CD data extractor, and GPL it, I'll definitely take a looksie at it. I am all about OSS and support the OSI.

Nikos
31.10.2005, 23:25
I would even write an Alcohol/CloneCD-like GUI app for creating images. The only reason I didn't do so yet, is because I'm not really in the mood in learning about CDs and low-level SCSI/ATAPI access. But if some good code would exist, I would attempt it for sure.

johngalt
31.10.2005, 23:31
Perhaps you oughta look at readcd since it *is* GPLd

MonkeyMonkey
02.11.2005, 08:56
Thanks everyone for the support,

I too am interested in Open / Free software, since I use it then my development will fall under that domain.
I want to use ReadCD so currently I'm thinking of a script which does the following:

-Extract full data using readCD
-Develop a program to seperate the sub-channel / image data
-De-Interlace the Image data (software)

It's becomming obvious that I desire to mimic similiar output to CloneCD. This way DT doesn't need adjusting.

Where I'm up to:
I've found some code on the internet to deinterlace the sub-channel data, however, I had to port it to AT&T Assembler .... Anyway, that's done.
I've *de-interlaced* (using borrowed code) one sub-channel frame and the result is not what I was hoping. I hoped I would see something similiar to CloneCD's .SUB data output. Perhaps error correction is performed, I'm not sure. I'll have a look at the ReadCD parameters and see what I've got. I'm also considering viewing the red book (I think that's the right one) to verify my understanding of sub-channel data and modify my code appropriately.

Side Note:
As mentioned, I do appreciate people's support. I hope to invest whatever time I can into this project. It'll take me a while to complete things, however, as I usually only can spare a few hours a week. Though, this weekend is mostly free so I will devote my time to getting this done. Thanks again.

Underheaven
02.11.2005, 09:48
You cannot view the red book http://www2.daemon-tools.cc/dtcc/images/icons/icon9.gif .

The standard is not freely available and must be licensed from Philips. At the time of writing, the cost as per the relevant Philips order form (document no. 28/10/04-3122 783 0027 2) is US$5000

MonkeyMonkey
02.11.2005, 12:08
That does make more sense though, not being able to freely view the redbook. I was expecting this but I've read about a ton of people who have viewed it (???).

Thanks for the heads up.

Reverse Engineering it is then.

MonkeyMonkey
05.11.2005, 07:54
Well,
Know I'm having a look at Alcohol 120% files to see how they store the information.

I'm starting to realise a bit more complexity in this, looking at the alcohol settings. My plan is to still use Read CD, so I'll need to check that it supports the same reading options as Alcohol.

Learning a bit more about copy protections, my first thought that a 1:1 image of a CD would resolve copy protections seems to be faulty, nonetheless, that's why we have hackers. As long as I can capture enough information...

MonkeyMonkey
05.11.2005, 08:10
Does anybody know if ReadCD can read in channels P and Q? This may be part of what's causing me a problem.

I'm assuming ReadCD uses MMC, so I am considering checking the MMC specs and perhaps modifying ReadCD.

Xristaki
05.11.2005, 08:19
you completely lost me at "channels" :)

Nikos
05.11.2005, 09:02
you completely lost me at "channels" :)I posted a link to the CD-R FAQ in this thread. It's worth a read.

MonkeyMonkey
16.11.2005, 08:16
Thanks, I've read a bit on cdrfaq, particularly the subchannel or subcode information.

I don't want this project of mine to die off, though I haven't been online for a little over a week now, so it's slow sailing. People have expressed interest in helping me, and I appreciate it. When I get some momentum going on this project, I will contact you.

So where am I up to?

Well, I'm keeping to my goal of a GPL copy program similar to blindwrite / Alcohol etc... However, the hardcore copy protection stuff I'm going to leave to other people. If anyone has reverse engineered Alcohol or blindwrite files (mainly the header files), I'd appreciate picking your brains.

More Notes
I have looked inside the Alcohol 120% files and see something similar to the contents of CloneCD, however, they have embedded subchannel data in the single image file.

What I'm not sure about is that the subchannel data is not the same as my RAW copy of the data or the CloneCD sub channel data, there are similarities though.

Questions for me or others
Does CloneCD / Alcohol 120% Encrypt their Subchannel rips at all?
I'm sure that the read-soloman error correction should not be applied to this data and that I am applying an de-interleave algorythm to the data.

I'm still looking for a good source for Data De-Interleaving. From what I've read the data is stored as:
- 8 Channels stored in 8 consecutive bytes.
- Each bit of each byte corresponds to a channel
- To "De-Interleave" the data, one Channel Byte (say P) can be extracted by collecting the first bit of each byte for eight bytes.
However, this doesn't necessay correspond with the algorythms I've seen other people write. If anyone can confirm or correct this, I would appreciate it.

Thanks.

Nikos
16.11.2005, 22:32
Each clone program usually has its own drivers. The API changes, so I guess the format they use for the files is mandated by the API.

Also, there might (note, might, not sure) be differences in byte-ordering. Although I don't see a reason why a PC app would use anything else than little-endian, it might be an issue.

EDIT:
I don't think the apps encrypt the data. There's nothing to hide, after all.

MonkeyMonkey
17.11.2005, 01:01
Makes sense to me that they wouldn't encrypt the data.

Up untill now, I've been assuming that ReadCD packs all the subchannel data at the end of each sector (in the image). This is how it appears in a Hex editor, however, there may be more information in the actual sector which I'm currently investigating.

Do any Copy Protections store error checking information in their subchannel data?

MonkeyMonkey
17.11.2005, 05:26
Updates

Well, after a lot of tinkering around, and learning more about how the subchannel data is stored I have had SOME success.

I wrote my own code to DeInterleave the subchannel data, which seems to be working succesfully. The resulting data matches the data in the .SUB files of Clone CD.

I'm writing a small, inflexible program to confirm this. It will take a RAW input (R-W Raw read from ReadCD) and will spit out a .SUB file.

Will post if I have success.

This still leaves me to ponder how the data is organised in the Alcohol files, my next challenge is to investigate that.

MonkeyMonkey
17.11.2005, 08:12
Current Findings
Alcohol 120% MDF files contain all CD data including Sub channel data. The Sub channel data is coppied at the end of the 2352 byte sector (2352 + 96). This data is a RAW copy and has not been de-interleaved. The sub channel data in these files sometimes contains all 0's in the P channel OR all 1's in the P channel (or mostly). I believe this corresponds to CD encoding.

CloneCD seperates the Sub channel data into it's SUB file. CloneCD also performs a software deinterleave on the data prior to placing it into the SUB file. There are mostly 0's in the P Channels. The 1's found in the Alcohol 120% P-Channels do NOT correlate in the CloneCD sub channels.

Blindwrite does it's own thing, I haven't looked into those files. There are people out there reverse engineering Blindwrite files, I'm aware of a project on sourceforge that converts these to ISO's.

I've written a software De-interleave program (extremely roughly, it doesn't take command arguments nor performs any input / error checking). The above findings assume that this program of mine works.

ReadCD is capable of Raw reading, however, I don't know if it's as thorough as the first two programs.

I've compared Sub Channel data from all three programs (CloneCD, Alcohol and ReadCD) using the DeInterleave program, and there are many similarities between the data but they're not identical. I remember reading that consecutive reads can yield diffrent data for Sub Channels, but there are some strong differences as well. As mentioned, each software uses it's own driver to read the data, but would you expect large changes in the data?

I'll start testing consecutive reads using the one program to see if there are strong differences as well.

Underheaven
17.11.2005, 08:18
Very neat stuff. You could also try consecutive reads on different hardware with a known program like Alcohol or CloneCD.

Nikos
17.11.2005, 20:49
When reading in raw mode, you can't be sure that any data you get is actually correct. Alcohol and CloneCD performs a software check. However, the actual algorithm for performing data integrity checks on raw sectors+subs is beyong my knowledge.

MonkeyMonkey
03.12.2005, 00:25
Sounds like this could be that "Reed-Soloman" algorythm I've heard of. I guess that's not an option then.

I've just re-read the post about each company using their own driver, this is a good point and could yield differing results. Part of me still wants to hope that the data output should still be the same.

Well, this is getting tricky now. I guess this is probably why an open source version isnt' really available, no one's willing to donate a red-book to these folk?

I'm not sure what I do from here on, I have a rough roadmap:

First I want to yield similar results to Alcohol 120% using readCD. At least the ISO data should be the same. I then want to investigate the various options in Alcohol and see what compatability readCD has (as well as review the mmc standard, with some assistance). There may be some commands not being utilised by readCD.

If I can find a way of replicating the Alcohol 120% settings (as best as possible) and obtaining similar results, then that should be successful enough. Though I may note be able to record all protection and subchannel data succesfully, there are those people at DT and other places who can emulate the security and hopefully we'll do without.

Next step is to find out what the recommended Alcohol settings are for various copy protections, and code that into an app. I want the application to first Identify the copy protection and then copy the cd as appropriate. I want the output files to closely mimic alcohol files, so it can be used with DT.

Timeline? Well, I've got some time off over christmas, I hate letting these things drag out so long. I also realise that my first ambition of an open 1:1 copy program is not as realistic as I hoped it would, I still want something that does a bit more than ISO copying which is meaningful. I might check out some open CD+G rippers as well.

Anyway, I'm still all talk at the moment. I do look forward to spending some more time on this again shortly.

phoenix
10.12.2005, 11:39
If anyone has reverse engineered Alcohol or blindwrite files (mainly the header files), I'd appreciate picking your brains.

Since Alcohols MDS\MDF format was a joint development project with DTools, asking a question like this on this forum is a little stupid. Reverse engineering is not legal.

MonkeyMonkey
03.01.2006, 01:29
Thanks for your information, I was not aware that reverse engineering files could be illegal. Despite my personal opinions, law is law; I will research this further.

If you could please direct me to an appropriate location (either government legislation or even a copy of Alcohol 120% license agreement) where it states that reverse engineering file formats is illegal, I would appreciate it as it would expedite my research.

It's been a while since I've spent time on this project, I want to spend some time reading the MMC specs (since I believe this is used with read-cd).

I will admit that I'm loosing momentum on this, as I'm discovering more and more which is making this rather cumbersome.

I will shortly post whatever code snippets I've collected which may help someone else more experienced or anyone else who wishes to continue this. I'll also contact the developers of read-cd and see if they're interested in this.

Jito463
09.01.2006, 06:39
Straight from the Alcohol Software EULA. Relevant portions are highlighted in bold.


2. PROTECTION OF SOFTWARE.
You agree to protect the Software and the Documentation from unauthorized copying or use. You acknowledge that the source code for the Software and other trade secrets embodied in the Software have not been, and are not going to be, disclosed to you. Modifications of, additions to, or deletions from the Software (including any deletion or addition of code) are strictly prohibited. Except as specifically permitted in this Agreement, you agree not to, directly or indirectly, (1) use any Confidential Information to create any software or documentation that is similar to any of the Software or Documentation; (2) reverse engineer, disassemble or decompile the Software; (3) encumber, transfer, sublicense, rent, lease, time-share or use the Software in any service bureau arrangement; or (4) copy (except as provided herein), distribute, manufacture, adapt, create derivative works of, translate, localize, or otherwise modify Software or permit any third party to engage in any of the acts proscribed in clauses (1) through (4). You agree not to remove or alter any printed or on-screen copyright, trade secret or other legal notices contained on or in the Software or the Documentation.

MonkeyMonkey
21.01.2006, 04:05
Thank you, I appreciate the effort to find the reference. I may have misunderstood these statements to refer to execution binaries and libraries. Not generated files.

This leaves me in an awkward position then. I will check the Clone CD EULA for a similiar statement, though I suspect that it's a standard agreement.

My next path is to then develop something different then. My aim would be to split raw data from subchannel data into seperate files, but emulation is still a problem. Got to run, but i will put more thought to this later.

I also now understand how the B52 to ISO project is probably legal, they are not recreating, just reading and then outputting to a new format.

G

Jito463
21.01.2006, 04:42
That's part of the reason why you don't always see every company support all the different formats out there. For example, Alcohol still does not support Blindwrite 5 images (it can technically be possible to mount them in the virtual drive, but burning them is another matter). It all depends on the original author of the format and how much information they're willing to release about it.