I've been asked to look at PC-Engine and new methods of dumping CD-ROM based games for this system. Turns out there are some issues with current dumps, and most importantly some games are still unplayable. We're not sure if it's due to bugs in emulators or not but surely having a correct dump would help a lot.
The big word here is "correct". Dumping unprotected CDs compatible with Yellow Book is easy, however, when you get data sectors inside an audio track it's another story. There's been a lot of deliberation on the subject of CD-ROM dumps, especially when it comes to audio tracks and so-called "audio offset" - something that depends on particular drive/firmware and causes the actual data to be slightly shifted. This is usually way less than 1/75th of a second but many dumpers find it unacceptable that the same disc produces two different results, which cannot be directly compared.
The way I see it audio offset can't be fully accounted for when you dump the tracks on a PC drive. Sure, you can stick to one particular drive model to be able to compare the dumps but this cannot be really called "correct approach". Another problem is inability to directly read and dump special areas on the CD, like lead-in, lead-out, and some track pregaps (which might contain data - as part of a protection scheme or just due to shoddy mastering). I know that some people insist on dumping subchannel data for analysis and then proceed to read the disc according to that - but such methods are flawed as well. First of all, the subs are not protected by correction codes and contain bit glitches. Many are easy to spot and correct, either "by hand" or with a partial redump, but one can never be sure that the Q subchannel was read and reported as-is, or was it perhaps recreated by drive firmware when CRC checksum mismatched. Second, Q subs are not required for each and every section of the track. The drive must be able to deal with missing or damaged subs and this is also why data sectors carry their own address fields. Audio can only be addressed with Q sub, hence the seek function is only required to get as close as possible to requested audio chunk but it might not be a perfect hit. And this is why early drives had so many problems with stuttering audio rips - something that is mostly fixed on current models. Mostly
Lastly though, many people don't realize that Q sub packets are stored for CIRC-encoded data, that in turn is a shuffled version of the actual raw data. So, in other words, when you remove CIRC and fix any C1/C2 errors, the order of data changes back and it's hard to tell what Q address it really belongs to. Again, not a problem for data sectors that have their own header and address field but it makes audio positioning rather inaccurate - hence the offset.One method of dumping CDs that is not plagued with these issues is to try and grab raw frames, after EFM processing and before these enter CIRC decoder. We can have 1:1 mapping of data and subchannels, except we now have to process everything software-side in the emulator. Not only that, we need to actually get dumps like that first, and that is not possible with unmodified consumer equipment. Even when we do, we might end up with bit glitches (perfectly fixable with CIRC but still present in the dump) that will require a redump of perhaps several discs to get rid of. There are also weak EFM patterns in some protection schemes but these we can't do anything about, since even capturing analog stream coming directly from the optical pickup would not let us read those properly. So, no point in bothering. This new idea certainly has it's own problems but these can be overcome, so it's at least worth proper research.
So, what's this connection I mentioned? Well, I got a link to a document from KEEP. It stands for "KEEPING EMULATION ENVIRONMENTS PORTABLE" and it's EU-funded project aimed at preserving old games (well, multimedia in general). In theory it's great news that there are organizations like that and at least some politicians and/or civil servant could be convinced to fund them. In reality though the situation looks pretty grimm still.
Here's a link: http://www.keep-project.eu/ezpub2/index ... wsletter-4
I wrote about this several times already but I feel like I need to keep reminding you, and myself, every now and then - so that you don't take things for granted. I urge you people to read this document, it's not all that long. After I did, I came to realise three things:
1) These guys have money.
Apparently got more than 3 million euro so far. You can tour the world with it and have open, free meetings for people. Make no mistake, I'm not saying it's bad, far from it - we need people to understand what's at stake and what the problems are. I just hope this is not the only thing the money were used for.
2) All they did is code UI in JAVA.
Yes, the greatest achievement so far is some silly-looking UI for a couple of open-source emulators. I mean, seriously, most of the document praises this... thing... as if it was the ultimate solution to game preservation. I've seen high-school projects more ambitious than this. Are the authors of WinUAE or VICE aware that KEEP is using their work? I'm not sure so I might be somewhat out of line here, but it really angers me when open-source projects are used like this, just because these are free. Free as in free beer in this case.
3) They are fully aware it's illegal.
At least they worked that out. Even went as far as to specifically point out legal status of bypassing copy protection in Germany and France, and the fact that there are no exceptions, whatsoever, for individuals doing research. I'm willing to bet there were games for Amiga and C64 that were copy-protected, and that authors of WinUAE and VICE had to study those in order to properly emulate. Now, if you can play a game in emulator that was stored on a floppy or a tape in a way that prevented copying, then surely this system has now been bypassed. So, it's illegal. And yet... no-one cares if it's KEEP? I mean, if I was to throw a conference like this and show Makaron working, then I'd most likely have lawyers (and possibly police) knocking on my door the next morning.
Please spread this little rant of mine. I'd like to have these KEEP people made aware of my own existence, and all the other people that were doing emulation and preservation long before they even had the idea. And we are all individuals working for free in our spare time, to beat the clock. Because some of the systems are becoming dust as we speak and soon there won't be anything left to preserve. Whether it's dumping or emulation, we often have to bypass original protection systems to get results, which is illegal. As is storing software you're not licensed to have, even if it's the last copy on Earth. We have the skills and knowledge to do these things and must keep a low-profile, whereas organizations like KEEP do very little to actually change the law but have no problems boasting success when using existing emulators.
If you think I'm overreacting then just answer yourself this little question: What would KEEP be able to show during these meetings if they did not use free emulators? That's right, nothing. Not even that UI since there would be no point to it without actual emulation. And systems and games that were researched and dumped long before they started. I really love the idea of raising awareness on horrible laws we have and all, but to me it feels like KEEP is trying to grab the money while reinventing the wheel. Rather than try and help/fund/coordinate existing projects, they take what is free and claim to have done all the dirty work. As if.
Source : http://dknute.livejournal.com/39685.html
